Sigmoidal function pes can thus evolve step function pes

378 Chapter Nine—Applications

This method can be used to evolve the network structure indirectly. If the evolved slope is sufficiently small (the exact amount depends on the application), then the output is essentially constant regardless of the in-put. (In the case of the sigmoidal transfer function, the output would be 0.5, or very nearly so.) If the PE is in a hidden layer, it can therefore be re-moved. Its effect can be replicated by increasing the weights from the bias PE in that hidden layer to each of the PEs in the next layer by one-half the value of each weight from the PE being removed to the next-layer PEs. The method therefore can be used to prune PEs from the net-work, reducing network complexity.

Additionally, if the slope is sufficiently large (the exact amount de-pends on the application) then the sigmoid transfer function can be re-placed by a step transfer function. A sigmoid with a large positive slope is thus replaced by a step transfer function that has an output of 0 for in-puts less than or equal to 0 and 1 for positive inputs. A sigmoid with a large negative slope is replaced by a step function with an output of 1 for inputs ≤ 0 and 0 for positive inputs. Sigmoidal function PEs can thus evolve to be step function PEs, reducing the computational complexity of the network significantly.

Another feature of this methodology is the continuous nature of the PSO algorithm. Transfer function slopes are evolved in a continuous way; that is, slopes can vary continuously from large negative to large positive values. This results in an evolution of network structures that is also essentially continuous in nature. For example, as a hidden PE’s transfer function slope approaches zero, it is replaced with revised con-nection weights from the bias PE; as the slope becomes very large, the sigmoidal PE is replaced by a threshold PE. No sudden significant discon-tinuities exist in the evolutionary process such as those that plague other approaches to evolving network structures.

An Example Application