Using artificial neural networks (ANN)

Description

In this experiment, a number of algorithms that can be used for tranining artificial neural networks are compared for binary classification task. In the experiment the Spambase dataset is used. The classification accuracy of the resulting classifier is determined and the interpretation of related graphs is reviewed.

Input

Spambase [UCI MLR]

Before fitting the models the dataset is partitionated by the Data Partition operator according to the rates 60/20/20 among the training, validatation and test datasets.

Output

Firstly, a standard artificial neural network is fitted by the NeuralNetwork operator where the network topology of a multilayer perceptron is defined as 3 hidden neuron in one hidden layer. The goodness-of-fit of the resulting model can be verified using standard statistics (misclassification rate, the number of incorrectly classified cases) and graphics (response and lift curve).

Figure 19.9. Fitting statistics of the multilayer perceptron

Fitting statistics of the multilayer perceptron

Figure 19.10. The classification matrix of the multilayer perceptron

The classification matrix of the multilayer perceptron

Figure 19.11. The cumulative lift curve of the multilayer perceptron

The cumulative lift curve of the multilayer perceptron

In addition to the standard goodness-of-fit tests we get results which have meaning for artificial neural networks only. These results involve the graph of the weights of the neurons, the graph of the history of the teaching where the misclassification rate can be seen as the function of the iteration for training and validation datasets.

Figure 19.12. Weights of the multilayer perceptron

Weights of the multilayer perceptron

Figure 19.13. Training curve of the multilayer perceptron

Training curve of the multilayer perceptron

Similar graphs were obtained for the other two neural network fitting operator, namely for the DMNeural operator and for the AutoNeural operator. At the first, the exception is the following stepwise optimization statistics.

Figure 19.14. Stepwise optimization statistics for DMNeural operator

Stepwise optimization statistics for DMNeural operator

Figure 19.15. Weights of neural networks AutoNeural operátorral kapott háló neuronjainak súlyai

Weights of neural networks AutoNeural operátorral kapott háló neuronjainak súlyai

Finally, the three models can be compared by the Model Comparison operator. As a result, we obtain the following statistics and graphs.

Figure 19.16. Fitting statistics of neural networks

Fitting statistics of neural networks

Figure 19.17. Classification charts of neural networks

Classification charts of neural networks

Figure 19.18. Cumulative lift curves of neural networks

Cumulative lift curves of neural networks

Figure 19.19. ROC curves of neural networks

ROC curves of neural networks

Interpretation of the results

The above statistics and figures clearly show that the best model is the first artificial neural network with multi-layer perceptron architecture, where there is one hidden layer with 3 neurons.

Video

Workflow

sas_ann_svm_exp2.xml

Keywords

artificial neural network
supervised learning
classification

Operators

Data Source
Model Comparison
Neural Network
Data Partition