J. Cent. South Univ. Technol. (2009) 16: 0478-0481
DOI: 10.1007/s11771-009-0080-9
Support vector machine forecasting method improved by
chaotic particle swarm optimization and its application
LI Yan-bin(李彦斌)1, 2, ZHANG Ning(张 宁)1, LI Cun-bin(李存斌)2
(1. School of Economics and Management, Beijing University of Aeronautics and Astronautics, Beijing 100191, China;
2. School of Business Administration, North China Electric Power University, Beijing 102206, China)
Abstract: By adopting the chaotic searching to improve the global searching performance of the particle swarm optimization (PSO), and using the improved PSO to optimize the key parameters of the support vector machine (SVM) forecasting model, an improved SVM model named CPSO-SVM model was proposed. The new model was applied to predicting the short term load, and the improved effect of the new model was proved. The simulation results of the South China Power Market’s actual data show that the new method can effectively improve the forecast accuracy by 2.23% and 3.87%, respectively, compared with the PSO-SVM and SVM methods. Compared with that of the PSO-SVM and SVM methods, the time cost of the new model is only increased by 3.15 and 4.61 s, respectively, which indicates that the CPSO-SVM model gains significant improved effects.
Key words: chaotic searching; particle swarm optimization (PSO); support vector machine (SVM); short term load forecast
1 Introduction
At present, many researchers have focused on the short term load forecast [1-4]. From the current research achievements of the short term load forecast, it is not difficult to find that there are mainly two kinds of forecasting methods on short term load: time series method [1-2] and artificial intelligent forecasting method based on neural networks [3-4].
Time series method establishes a time series model according to the historical data of short term load, which is used to predict the short term load. The key assumption is to make sure that the model’s historical power load is similar to the future power load forecast to some extent. Time series forecast model is easy to understand and simple to operate, but the fluctuation of the short term load is affected by many factors, and the time series method cannot deal with the problems such as multivariable and heteroscedasticity.
Recently, there have been some new studies on using the regression support vector machine (SVM) to forecast the short term load [5-6], which have achieved satisfactory results. Compared with the traditional neural network method, this method has advantages such as fast learning speed, global optimum and strong generalization ability, and its results are apparently better than those of pattern recognitions and regression forecast methods. However, there is a prominent problem in the specific application of SVM, that is, how to set some key parameters affecting the algorithm, such as the balance parameter c, insensitive parameter ε and kernel function parameter σ. Generally, it is time-consuming and blind by using the cross-validation method to do the trial calculation [7].
Taking the above aspects into consideration, an improved SVM forecast method (i.e. the CPSO (chaotic- particle swarm optimization)-SVM forecast method) was proposed. Firstly, the global search capability of the PSO was improved with the aid of the chaotic searching. And then some key parameters of SVM were optimized by using the CPSO. Lastly, the new SVM was applied to predicting the short term load of the South China Power Market. For the same sample data, the PSO-SVM and SVM methods were used as comparison.
2 Simple particle swarm optimization
In PSO [8], each solution to the optimized aspect can be treated as a bird, called a particle. Every particle has its fitness value determined by the optimized function and its velocity that will determine its flight course and distance. Particles will follow the optimized one and search among the random particles initialized by PSO in solutions space. And the best solution can be achieved by iteration. The particle updates itself by the following two extreme values in each iteration. One extreme value named PBest is the best solution obtained
by the particle itself. The other extreme value named gBest is the best solution found by the whole swarm at the present. Alternatively, the whole swarm can be substituted by some particles around the best one and the extreme value of the neighboring particles is called partial value [7]. Before finding these two optimized values, a particle updates its velocity and location according to the following formula:
Xi(k+1)=Xi(k)+vi(k+1)Δt (1)
(2)
where Xi is the position of particle i; vi represents the velocity of particle i, vi∈(vmin, vmax), vmin and vmax are constants in the range of [0, 1]; Pi is the best previous position of particle i; Pg is the best position among all particles in the population X=[X1, X2, …, X N]; N is the size of population; k is the current step number; w is the internal weight coefficient (i.e. the impact of the previous velocity of particle on its current one); c1 and c2 are acceleration constants; r1 and r2 are random real numbers in the range of [0, 1]. When the given maximum iteration frequency or fitness value meets the requirement to realize the given value, the calculation will be terminated [8].
3 Chaotic searching
The famous logistic equation is applied [10-11], which is defined as follows:
Xn+1=μxn(1-xn), 0≤x0≤1 (3)
where μ is the control parameter; x is a variable; n(=0, 1, 2, …) is the iteration number of the variable. Although Eqn.(3) is deterministic, it exhibits chaotic dynamics when μ=4 and x0
{0.25, 0.50, 0.75, 1.0}. In other words, it exhibits the sensitive dependence under initial conditions, which is the basic characteristic of chaos. A greater difference in the initial value of the chaotic variable will result in a considerable difference in its long time behavior. The track of chaotic variable can travel ergodically over the whole search space. In general, the above chaotic variable has special characteristics, i.e. ergodicity, pseudo-randomness and irregularity [9-10]. The process of the chaotic local searching can be referred to Ref.[10].
4 Chaotic-particle swarm optimization (CPSO) hybrid algorithm
The proposed algorithm is based on the excellence of both the chaotic searching and the particle swarm optimization. The new hybrid algorithm can be defined in two stages. In the first stage, it is mainly based on the sacrifice and memory property of the particle swarm optimization to make global exploring. In the second stage, the chaotic searching is applied to finding better particle around the global best particle for its efficiency in scanning.
The structure of the proposed CPSO hybrid algorithm is shown in Fig.1.
As shown in Fig.1, the circulation of the proposed algorithm will be terminated, while the critical condition of the setting maximal iteration times achieves. In this example, the stopping condition is that the iteration times equals 2 000.
Fig.1 Structure of proposed chaotic-PSO (CPSO) hybrid algorithm
5 SVM regression analysis
The forecast principle of the support vector machine regression model was investigated in Refs.[11-14]. The basic idea of using SVM to estimate the regression function is to map the data in the input space to another highly-dimensional space with a nonlinear mapping, and then to make a linear regression in this space. The more detail about the SVM regression can be referred to Ref.[13].
KUGIUMTZIS et al [11] found that the key parameters of SVM, i.e. σ, c and ε, have a great impact on the accuracy of SVM estimating regression. Among the three parameters, c determines the complexity of the model and the punishment level of the fitting deviation, and it is greater than ε on the basis of the sample data’s characteristic; σ precisely defines the structure of the highly-dimensional space, so it controls the complexity of the ultimate solution; ε expresses the system’s expectation on the estimating functions’ error of the sample data, and the larger the ε, the less the support vector number and the more sparse the solution expression. But large ε can also reduce the accuracy of regression estimation.
6 CPSO-SVM forecast model
The CPSO proposed in this work was used to optimize the three key parameters in the SVM forecast model, namely σ, c and ε. The operating steps are described as follows.
Step 1 Set the termination conditions of the CPSO.
Step 2 Initialize the CPSO. Each particle corresponds to different values of the SVM’s three key parameters: σ, c and ε.
Step 3 Set the fitness function of the CPSO as follows.
(4)
where ηMAPE is the mean absolute percentage error, xi is the actual hourly data, and 
is the predicted value.
Step 4 Start the CPSO to generate a corresponding composition of the SVM’s three key parameters until the stopping condition is met.
The computing process of CPSO-SVM is shown in Fig.2.
Fig.2 Computing process of CPSO-SVM
7 Application of proposed method
7.1 Evaluating index
To describe the performance of the model, the root mean square error (rRMSE) calculated from 24 h forecast data, was used. rRMSE is given as follows:
(5)
7.2 Application environment setting
The simple data of the real short term load were chosen from South China Power Market in date range of August 2, 2007 to August 26, 2007. The short term load from August 2, 2007 to August 25, 2007 was used as training data. The aim is to predict the short term load of on August 26, 2007 in the same market.
Based on the same computing environment (CPU is Intel(R) Core(TM)2 T5600 1.8 GHz, the operation system is Windows XP2 professional, and the computing software is Matlab 7.0), the proposed method was applied to predicting the real short term load, while the SVM and PSO-SVM algorithms were also applied for comparison.
7.3 Forecast results
The training process curves of different three methods are depicted in Fig.3. From Fig.3, it can be seen that the curve of the CPSO-SVM is smoother than the other two curves and the root mean square error of the CPSO-SVM is lower than that of the other two methods, which indicates that, by introducing the CPSO, the improved algorithm has better learning capacity in the PSO-SVM training process. The final training results show that the improved CPSO-SVM model gets a better forecasting effect.
The predicted results of different three methods are shown in Fig.4, in which the forecast short term load is compared with the real short term load. From Fig.4, we can see that the curve of the CPSO-SVM is closer to the real short term load curve than the other two forecast short term load curves. This means that the performance of SVM was markedly enhanced by introducing the CPSO.
Fig.3 Training process curves of different three methods
Fig.4 Comparison between forecast short term load and real one of South China Power Market 24 h-time-point with different models on August 26, 2007
The more detailed results comparison can be seen from Table 1. The forecast accuracy of CPSO-SVM method is improved by 2.23% and 3.87% compared with that of PSO-SVM and SVM methods, respectively. The time cost of the CPSO-SVM method is a little more than that of the PSO-SVM and SVM methods. The time cost of CPSO-SVM is increased by 3.15 and 4.61 s compared with that of the PSO-SVM and SVM methods, respectively. It can be inferred that the CPSO-SVM is more adaptable to the short term load prediction.
Table 1 Comparison of forecast results obtained by different methods
8 Conclusions
(1) By introducing the chaotic searching into the PSO and using the improved PSO to optimize the key parameters of the SVM regression forecast model, a new short term load forecast model named CPSO-SVM is proposed. The comparison of forecast short term load and real one shows that this model has higher forecast accuracy and efficiency than the previous models, proving that the proposed model has a bright application prospect in forecasting the short term load.
(2) To get higher forecast accuracy, much more attention should be paid to the following two aspects. The first is to optimize the structure of forecasting method, and the second is to further research the short term load fluctuating mechanism.
References
[1] BING D, CHENG C, LEE S E. Applying support vector machines to predict building energy consumption in tropical region [J]. Energy and Buildings, 2005, 37(5): 545-553.
[2] NIU Dong-xiao, CAO Shu-hua, ZHAO Yue. Technology and application of power load forecasting [M]. Beijing: China Power Press, 1998. (in Chinese)
[3] PAPPAS S S, EKONOMOU L, MOUSSAS V C, KARAMPELAS P, KATSIKAS S K. Adaptive load forecasting of the Hellenic electric grid [J]. Journal of Zhejiang University—Science A, 2008, 9(12): 1724-1730.
[4] CARLOS A M, MATEUS M G. Application of switched adaptive system to load forecasting [J]. Electric Power Systems Research, 2008, 78(4): 721-727.
[5] PING F P, WEI C H. Forecasting regional electricity load based on recurrent support vector machines with genetic algorithms [J]. Electric Power Systems Research, 2005, 74(3): 417-425.
[6] LI Yuan-cheng, FANG Ting-jian, YU Er-keng. Study of support vector machines for short-term power load forecasting [J]. Proceeding of the CSEE, 2003, 23(6): 55-59. (in Chinese)
[7] LI Xiang, YANG Shang-dong, QI Jian-xun. A new support vector machine optimized by improved particle swarm optimization and its application [J]. Journal of Central South University of Technology, 2006, 13(5): 568-572.
[8] SENJYU T, MANDAL P, UEZATO K, FUNABASHI T. Next day load curve forecasting using hybrid correction method [J]. IEEE Transactions on Power Systems, 2005, 20(1): 102-109.
[9] SENTHIL ARUMUGAM M, RAO M V C, CHANDRAMOHAN A. A new and improved version of particle swarm optimization algorithm with global-local best parameters [J]. Knowledge and Information Systems, 2008, 16(3): 331-357.
[10] LIAO G C, TSAO T P. Application of a fuzzy neural network combined with a chaos genetic algorithm and simulated annealing to short-term load forecasting [J]. Evolutionary Computation, 2006, 10(3): 330-340.
[11] KUGIUMTZIS D, LINGIARDE O C, CHRISTOPH-ERSEN N. Regularized local linear prediction of chaotic time series [J]. Physica D, 1998, 112(3): 344-360.
[12] HA C H, KUO W. Reliability redundancy allocation: An improved realization for nonconvex nonlinear programming problems [J]. European Journal of Operational Research, 2006, 171(1): 24-38.
[13] ZHAO C, LIU K, LI D S. Research on application of the support vector machine in freight volume forecast [J]. Journal of the China Railway Society, 2004, 28(4): 10-14 (in Chinese)
[14] HE Q, WANG L. An effective co-evolutionary particle swarm optimization for constrained engineering design problems [J]. Engineering Applications of Artificial Intelligence, 2007, 20(1): 89-99.
(Edited by CHEN Wei-ping)
Foundation item: Project(70572090) supported by the National Natural Science Foundation of China
Received date: 2008-08-28; Accepted date: 2008-10-23
Corresponding author: LI Yan-bin, Professor; Tel: +86-10-80798623; E-mail: liyb@ncepu.edu.cn
