APPLICATION OF SUPPORT VECTOR MACHINES TO THE MODELLING OF INFLATION APPLICATION OF SUPPORT VECTOR MACHINES THE MODELLING OF INFLATION

In Support Vector Machines (SVM’s), a non-linear model is estimated based on solving a Quadratic Programming (QP) problem. Based on work [1] we investigate the quantifying of econometric structural model parameters of inflation in Slovak economics. The theory of classical Phillips curve [8] is used to specify a structural model of inflation. We provide the fit of the models based on econometric approach for the inflation over the period 1993-2003 in the Slovak Republic, and use them as a tool to compare their approximation ability with those obtained using SVM’s method. Some methodological contributions are made for SVM implementations to the causal economic modelling.


Introduction
Model specification and estimation are two major components in econometric modelling. They are often treated as two separate but closely related steps in econometric model building. In modelling economic quantities, probably the most important step consists of identifying the relevant influential factors.
This contribution considers the econometric modelling of inflation in the Slovak Republic. The main tools, techniques and concepts involved in econometric modelling of inflation are based on the Phillips concept [8]. According to the Phillips inflation theory the variable inflation is generated on a set of underlying assumptions. In any case, the analysed inflation rates are explained by the behaviour of another variable or a set of variables, in our case by the wages and the unemployment (independent variables).
In this paper the resulting SVM's are applied using an ε-insensitive loss function developed by V. Vapnik. We motivate the approach by seeking a function which approximates mapping from an input domain to the real numbers based on a small subset of training points. The paper is organised as follows. The next section will provide a quick overview of the concept of SVM's theory. Section 3 analyses the data, discuses the Engle-Granger estimator and SVM estimator, and presents the fitted inflation rate values by the classical regression methods and SVM's models in Slovak economics. A section of conclusions will close the paper.

Support Vector for Functional Approximation
This section presents quickly a relatively new type of learning machine -the SVM applied in the regression (functional approximation) problems. For details we refer to [11], [12]. The general regression learning task is set as follows. The learning machine is given n training data, from which it attempts to learn the input- . The x i denotes the ith input and y i is the ith output. The SVM considers the regression functions of two forms [11]. The first one is where ␣ i , ␣ * i are positive real constants (Lagrange multipliers), b is a real constant, (./.) is the kernel function. Admissible kernels have the following forms: 2 ) (radial basis SVM), where is a positive real constant and other (spline, b-spline, exponential RBF, etc.).
The second regression function is of the form where (.) is a non-linear function (kernel) which maps the input space into a high dimensional feature space. In contrast to Eq. (1), the approximation function f(x, w) is explicitly written as a function of the weights w that are subject of learning.

APPLICATION OF SUPPORT VECTOR MACHINES
The Support Vector regression approach is based on defining a loss function that ignores errors that are within a certain distance of the true value. This type of function is referred to as an ε-insensitive loss function. Fig. 1 shows an example of one dimensional function with an ε -insensitive band. The variables , * measure the cost of the errors on the training points. These are zero for all points inside the band, and only the points outside the ε -tube are penalised by the so called Vapnik's ε -insensitive loss function.
In regression, typically some error of approximation is used. They are different error (loss) functions in use and that each one results from a different final model. Fig. 2 shows the typical shapes of three loss functions [2]. Left: quadratic 2-norm. Middle: absolute error 1-norm. Right: Vapnik's ε -insensitive loss function.
Formally, this results from solving the following Quadratic Programming problem subject to Ά To solve (3), (4) one constructs the Lagrangian subject to constrains which leads to the solution of the QP problem: After computing Lagrange multipliers ␣ i and ␣ i * , one obtains the form of (1), i.e.

Causal Models, Experimenting with Non-linear SV Regression
We demonstrate here the use of SV regression framework for dynamic modelling of economic time series where the time series or variable, say inflation, to be modelled can be explained by the behaviour of another variable or a set of variables. First, we present an econometric approach for modelling and investigating the relationship between the dependent variable of inflation measured by CPI (Consumption Price Index) and the two independent variables are the unemployment rate (U), and aggregate wages (W) in the Slovak Republic. Then, the SV regression is applied. Finally, the results are compared between a dynamic model based on econometric modelling and an SVR model.
To study the modelling problem of inflation quantitatively the quarterly data from 1993Q1 to 2003Q4 was collected concerning the consumption price index CPI, aggregate wages W and unemployment U. These variables are measured in logarithm, among others for the reason that the original data exhibit considerable inequalities of the variance over time, and the log transformation stabilises this behaviour. Fig. 3 illustrates the time plot of the CPI time series. This time series shows a slight decreasing trend without apparent periodic structure. The model specification of (11) is the lagged dependent variable model in which the dependent variables, lagged one period, appear as independent explanatory variables. This model is based on a distributed lag model [9]. One of the primary reasons for using this functional form is to determine the long-run response of dependent variable to change in each of the independent variables. The d statistic for model (11) is 1.49, falling in the inconclusive region, making the decision concerning first-order autoregression indeterminate. In this particular case, a decision has to be made whether or not to correct the autocorrelation. At this point, we will use this model and not correct for autocorrelation. In model (11), the regression coefficients ␤ 1 , ␤ 2 have not the appropriate magnitude. In addition, they are statistically insignificant at five percent level. In all probability the lagged dependent variable CPI tϪ1 substitutes for the inclusion of other lagged independent variables (W tϪ1 , U tϪ1 ). The inclusion of CPI tϪ1 and W tϪ1 , U tϪ1 is redundant and overspecified the model. A graph of the historical and the fitted values for inflation is presented in Fig. 4. The model follows the pattern of the current inflation very closely.
Finally another attempt was made supposing a more sophisticated dependence of current inflation on the previous observation performed with the help of SV regression. As stated in literature, we can not have a model in which the coefficients are statistically insignificant. We made an arbitrary decision which deleted "insignificant" explanatory variables. Then the equation (11) becomes the first-order autoregressive process (Markov process) because the observation of CPI at time t depends only on the observation of CPI at time t Ϫ 1, i. e.
where ␤ 0 , ␤ 1 are unknown parameters. If CPI t exhibits a curvilinear trend, one important approach to generating an appropriate model is to regress the CPI t against time. In Tab. 1 the SVR results of inflation were also calculated using an alternative time series model expressed by the following SVR form which is a time series model where x t ϭ (1, 2, …, 43) is the vector of time sequence (regressor variable).We report the results in Fig. 5.
Since this pattern of change is a common practice, we desire that our machine identify permanent changes and adjust the parameters to track the new process.
One crucial design choice is to decide on a kernel. Creating good kernels often requires lateral thinking: many measures of similarity between inputs have been developed in different contexts, and understanding which of them can provide good kernels depends on insight into the application domain. The Fig. 5 shows SVM learning by using various kernels. In Fig. 5a we have a piecewise-linear approximating function, while in Fig. 5b and Fig. 5c we have a more complicated approximating function. Both functions agree with the training points, but they differ on the three y values, they assign to other x inputs. The functions in Fig. 5d and Fig. 5e apparently ignore some of the example points but are good for extrapolation. The true f(x) is unknown, and without further knowledge, we have no way to prefer one of them, and so to resolve the design problem of choosing an appropriate kernel in our application. For example, the objective in pattern classification from sample data is to classify and predict successfully new data, while the objective in control applications is to approximate non-linear functions, or to make unknown systems follow the desired response.
Tab. 1 presents the results for finding the proper model by using the quantity R 2 (the coefficient of determination) on our application of the best approximation of the inflation rate. Since the estimation method for the SVR models is without any sound of statistical theory behind it, the values of the quantity R 2 are basically intuitive, there are no "statistically correct" coefficients of determination for SV regression models. As the calculating of R 2 uses the residuals of the goodness of fit of the least squares line to the data, we will use the R 2 in the case of the SV regression line as well. As shown in Tab. 1 the "best" is 0.9999 for the time series models with the RBF kernel and quadratic loss functions. In the cases of causal models the best R 2 is 0.9711 with the exponential RBF kernel and ε-insensitive loss function (standard deviation ϭ 0.52). The choice of was made in response to the data. In our case, the CPI, CPI Ϫ1 time series have ϭ 0.52. The radial basis function defines a spherical receptive field in and the variance 2 localises it.
The results shown in Tab.1 were obtained using ε-insensitive loss function (ε ϭ 0.2), with different kernels and degrees of capacity C ϭ 10 5 . We used partly modified software developed by Steve. R. Gunn [4] to train the SV regression models. The use of SV regression is a powerful tool to the solution of many economic problems. It can provide extremely accurate approximation of time series, the solution to the problem is global and unique. However, these approaches have several limitations. In general, as can by seen from equations (7), (8), the size of the matrix involved in the quadratic programming problem is directly proportional to the number of training data. For this reason there are many computing problems in which general quadratic programs become intractable in their memory and time requirements. To solve these problems many modified versions of SVM's were introduced. For example the generalized version of the decomposition strategy is proposed by Osuna et al. [7], the so-called SVM light proposed by Joachims, Thorsten [5] is an implementation of an SVM learner which addresses the problem of a large task, and finally, in [10] Tab.

Conclusion
In this paper, we have examined the SVM approach to study linear and non-linear models on a time series of inflation in the Slovak Republic. For the sake of approximation abilities we evaluated eight models. Two models are based on causal multiple regression in time series analysis, and six models are based on the Support Vector Machines methodology. Using the disposable data a very appropriate econometric model is the regression (11) in which the lagged dependent variable CPI tϪ1 can substitute for the inclusion of other lagged independent variables (W tϪ1 , U tϪ1 ). The benchmarking was performed between traditional statistical approaches and SVMs in regression approximation tasks. The SVM approach was illustrated on the regression function of (12), which was developed by statistical tools. This problem was readily solved by a SV regression with excellent approximation performance as it is visually clear from Fig. 5. Finally, the paper made some methodological contribution for SVM implementations to the causal economic modelling.