The process demonstrates how to use an SVM for solving a regression
problem. In this experiment RBF kernel SVMs are trained on the
Concrete Compressive Strength data set while
the value of the parameter
gamma of the RBF
kernel is changed. To obtain comparable results the value of the parameter
C is fixed to 10. The average RMS error from
10-fold cross-validation is determined for each SVM. As a result, the
gamma value yielding the best average RMS error
will be returned. Using this value for the parameter
gamma an RBF kernel SVM is trained on the entire
data set that is referred to as the “optimal RBF kernel SVM”
Figure 8.27. The average RMS error of the RBF kernel SVM obtained from
10-fold cross-validation against the value of the parameter
gamma, where the horizontal axis is
Figure 8.29. Predictions provided by the optimal RBF kernel SVM against the values of the observed values of the dependent variable.
The first figure shows that the best average RMS error is achieved
when the value of the parameter
2^-2 = 0.25.
The third figure shows that the average RMS error decreases with
the increasing value of the parameter gamma until it reaches its
minimum. However, further increase of the value of the parameter
gamma results in the degradation of the
performance, i.e., model overfitting occurs.