Effect of changing the number of neurons

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% Code create a neural network- MLP

%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

tic

[P,T]=simplefit_dataset;

plot(P,T,'x')

grid; xlabel('time (s)'); ylabel('output'); title('simplefit dataset');

net=newff([0 10], [5,1], {'tansig','purelin'},'traingd');

net1 = train(net, P, T);

%(defualts parameters) 1000 iterations, gradient 1 e-5 

N= sim(net1,P);

plot( P,T, P,N, P,N-T); grid;

legend('Original function', 'Simulated function','Error');

title('Neural Network Simulated fuction with # neurons in the hidden layer');

xlabel('time (s)'); ylabel('output');

toc

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

Effect of changing the number of neurons;

Rsults with five neurons:

10 neurons

Elapsed time is 10.174713 seconds.

15 neurons

Elapsed time is 10.453801 seconds.

20

Elapsed time is 11.307679 seconds.

25

Elapsed time is 12.205114 seconds.

30

Elapsed time is 12.442988 seconds.

difference between 25 and 30 nuerons is the same, increasing neurons will give no significant improvement on the gradient

Time increases as program has do do more calculations for extra neurpon.

Increasing the number of iterations improves performance, however

 the time taken is longer

For 30 neurons with 5000 iterations, elapsed time is 45.844242 seconds.

The training algorithm can be altered to improve learnin, ie replace traingd with trainlm, this is the Levenberg-Marquardt algorithm. It can give an output with very low error in fewer interationa and with fewer neurons in the hidden layer. Elapsed time is 15.018103 seconds. The time taken for 15 neurons at 500 iterations is longer than it would be for the traingd algorithm as the calculations made to adjust the weights are more complex, but it is still quicker using this algorithm to get an accurate representation of the original function.