Using an SVM for solving a regression problem

Description

The process demonstrates how to use an SVM for solving a regression problem. In this experiment RBF kernel SVMs are trained on the Concrete Compressive Strength data set while the value of the parameter gamma of the RBF kernel is changed. To obtain comparable results the value of the parameter C is fixed to 10. The average RMS error from 10-fold cross-validation is determined for each SVM. As a result, the gamma value yielding the best average RMS error will be returned. Using this value for the parameter gamma an RBF kernel SVM is trained on the entire data set that is referred to as the optimal RBF kernel SVM below.

Input

Concrete Compressive Strength [UCI MLR] [Concrete]

Output

Figure 8.26. The optimal value of the gamma parameter for the RBF kernel SVM.

The optimal value of the gamma parameter for the RBF kernel SVM.

Figure 8.27. The average RMS error of the RBF kernel SVM obtained from 10-fold cross-validation against the value of the parameter gamma, where the horizontal axis is scaled logarithmically.

The average RMS error of the RBF kernel SVM obtained from 10-fold cross-validation against the value of the parameter gamma, where the horizontal axis is scaled logarithmically.

Figure 8.28. The kernel model of the optimal RBF kernel SVM.

The kernel model of the optimal RBF kernel SVM.

Figure 8.29. Predictions provided by the optimal RBF kernel SVM against the values of the observed values of the dependent variable.

Predictions provided by the optimal RBF kernel SVM against the values of the observed values of the dependent variable.

Interpretation of the results

The first figure shows that the best average RMS error is achieved when the value of the parameter gamma is 2^-2 = 0.25.

The third figure shows that the average RMS error decreases with the increasing value of the parameter gamma until it reaches its minimum. However, further increase of the value of the parameter gamma results in the degradation of the performance, i.e., model overfitting occurs.

Video

Workflow

svm_regr_exp1.rmp

Keywords

SVM
supervised learning
RMS error
regression
cross-validation
parameter optimization

Operators

Apply Model
Log
Multiply
Normalize
Optimize Parameters (Grid)
Performance (Regression)
Read Excel
Set Parameters
Support Vector Machine (LibSVM)
X-Validation