Introduction to Neuroph

1. Introduction

This article takes a look at Neuroph – an open-source library for creating neural networks and utilizing machine learning.

In the article, we have a look at the core concepts and several examples on how to put it all together.

2. Neuroph

We can interact with Neuroph using:

• a GUI-based tool
• a Java library

Both approaches rely on an underlying class hierarchy which builds artificial neural networks out of layers of neurons.

We’ll focus on the programmatic side but will refer to several shared classes from Neuroph’s GUI-based approach to help clarify what we’re doing.

For more on the GUI-based approach, take a look at the Neuroph documentation.

2.1. Dependencies

If order to use Neuroph, we need to add the following Maven entry:

<dependency>
    <groupId>org.beykery</groupId>
    <artifactId>neuroph</artifactId>
    <version>2.92</version>
</dependency>

The most recent version can be found on Maven Central.

3. Key Classes and Concepts

All of the basic conceptual building blocks used have corresponding Java classes.

Neurons are connected to Layers which are then grouped into NeuralNetworks. NeuralNetworks are subsequently trained using LearningRules and DataSets.

3.1. Neuron

The Neuron class has four primary attributes:

1. inputConnection: weighted connections between Neurons
2. inputFunction: specifies weights and vector sums applied to incoming connection data
3. transferFunction: specifies weights and vector sums applied to outgoing data
   
4. output: the output value resulting from the application of transferFunctions and inputFunctions to an inputConnection

Together those four primary attributes establish the behavior:

output = transferFunction(inputFunction(inputConnections));

3.2. Layer

Layers are essentially groupings of Neurons such that each Neuron in the Layer is (usually) only connected with Neurons in the preceding and subsequent Layers.

Layers, therefore, pass information between them through the weighted functions that exist on their Neurons.

Neurons can be added to layers:

Layer layer = new Layer(); 
layer.addNeuron(n);

3.3. NeuralNetwork

The top-level superclass NeuralNetwork is subclassed into several familiar kinds of artificial neural networks including convolutional neural networks (subclass ConvolutionalNetwork), Hopfield neural networks (subclass Hopfield), and multilayer perceptron neural networks (subclass MultilayerPerceptron).

All NeuralNetworks are composed of Layers which are usually organized into a trichotomy:

1. input layers
2. hidden layers
3. output layers

If we are using the constructor of a subclass of NeuralNetwork (such as Perceptron), we can pass the Layers, the number of Neurons for each Layer, and their index using this simple method:

NeuralNetwork ann = new Perceptron(2, 4, 1);

Sometimes we’ll want to do this manually (and it’s good to see what’s going on underneath the hood). The basic operation to add a Layer to a NeuralNetwork is accomplished like this:

NeuralNetwork ann = new NeuralNetwork();   
Layer layer = new Layer();
ann.addLayer(0, layer);
ann.setInputNeurons(layer.getNeurons());

The first argument specifies the index of the Layer in the NeuralNetwork; the second argument specifies the Layer itself. Layers added manually should be connected using the ConnectionFactory class:

ann.addLayer(0, inputLayer);    
ann.addLayer(1, hiddenLayerOne); 
ConnectionFactory.fullConnect(ann.getLayerAt(0), ann.getLayerAt(1));

The first and last Layer should also be connected:

ConnectionFactory.fullConnect(ann.getLayerAt(0), 
  ann.getLayerAt(ann.getLayersCount() - 1), false);
ann.setOutputNeurons(ann.getLayerAt(
  ann.getLayersCount() - 1).getNeurons());

Remember that the strength and power of a NeuralNetwork are largely dependent on:

1. the number of Layers in the NeuralNetwork
2. the number of Neurons in each Layer (and the weighted functions between them), and
3. the effectiveness of the training algorithms/accuracy of the DataSet

3.4. Training Our NeuralNetwork

NeuralNetworks are trained using the DataSet and LearningRule classes.

DataSet is used for representing and supplying the information to be learned or used to train the NeuralNetwork. DataSets are characterized by their input size, outputsize, and rows (DataSetRow).

int inputSize = 2; 
int outputSize = 1; 
DataSet ds = new DataSet(inputSize, outputSize);

DataSetRow rOne 
  = new DataSetRow(new double[] {0, 0}, new double[] {0});
ds.addRow(rOne);
DataSetRow rTwo 
  = new DataSetRow(new double[] {1, 1}, new double[] {0});
ds.addRow(rTwo);

LearningRule specifies the way the DataSet is taught or trained by the NeuralNetwork. Subclasses of LearningRule include BackPropagation and SupervisedLearning.

NeuralNetwork ann = new NeuralNetwork();
//...
BackPropagation backPropagation = new BackPropagation();
backPropagation.setMaxIterations(1000);
ann.learn(ds, backPropagation);

4. Putting It All Together

Now let’s put those building blocks together into a real example. We’re going to start by combining several layers together into the familiar input layer, hidden layer, and output layer pattern exemplified by most neural network architectures.

4.1. Layers

We’ll assemble our NeuralNetwork by combining four layers. Our goal is to build a (2, 4, 4, 1) NeuralNetwork.

Let’s first define our input layer:

Layer inputLayer = new Layer();
inputLayer.addNeuron(new Neuron());
inputLayer.addNeuron(new Neuron());

Next, we implement hidden layer one:

Layer hiddenLayerOne = new Layer();
hiddenLayerOne.addNeuron(new Neuron());
hiddenLayerOne.addNeuron(new Neuron());
hiddenLayerOne.addNeuron(new Neuron());
hiddenLayerOne.addNeuron(new Neuron());

And hidden layer two:

Layer hiddenLayerTwo = new Layer(); 
hiddenLayerTwo.addNeuron(new Neuron()); 
hiddenLayerTwo.addNeuron(new Neuron()); 
hiddenLayerTwo.addNeuron(new Neuron()); 
hiddenLayerTwo.addNeuron(new Neuron());

Finally, we define our output layer:

Layer outputLayer = new Layer();
outputLayer.addNeuron(new Neuron()); 

4.2. NeuralNetwork

Next, we can put them together into a NeuralNetwork:

NeuralNetwork ann = new NeuralNetwork();
ann.addLayer(0, inputLayer);
ann.addLayer(1, hiddenLayerOne);
ConnectionFactory.fullConnect(ann.getLayerAt(0), ann.getLayerAt(1));
ann.addLayer(2, hiddenLayerTwo);
ConnectionFactory.fullConnect(ann.getLayerAt(1), ann.getLayerAt(2));
ann.addLayer(3, outputLayer);
ConnectionFactory.fullConnect(ann.getLayerAt(2), ann.getLayerAt(3));
ConnectionFactory.fullConnect(ann.getLayerAt(0), 
  ann.getLayerAt(ann.getLayersCount()-1), false);
ann.setInputNeurons(inputLayer.getNeurons());
ann.setOutputNeurons(outputLayer.getNeurons());

4.3. Training

For training purposes, let’s put together a DataSet by specifying the size of both the input and resulting output vector:

int inputSize = 2;
int outputSize = 1;
DataSet ds = new DataSet(inputSize, outputSize);

We add an elementary row to our DataSet adhering to the input and output constraints defined above – our goal in this example is to teach our network to do basic XOR (exclusive or) operations:

DataSetRow rOne
  = new DataSetRow(new double[] {0, 1}, new double[] {1});
ds.addRow(rOne);
DataSetRow rTwo
  = new DataSetRow(new double[] {1, 1}, new double[] {0});
ds.addRow(rTwo);
DataSetRow rThree 
  = new DataSetRow(new double[] {0, 0}, new double[] {0});
ds.addRow(rThree);
DataSetRow rFour
  = new DataSetRow(new double[] {1, 0}, new double[] {1});
ds.addRow(rFour);

Next, let’s train our NeuralNetwork with the built in BackPropogation LearningRule:

BackPropagation backPropagation = new BackPropagation();
backPropagation.setMaxIterations(1000);
ann.learn(ds, backPropagation);

4.4. Testing

Now that our NeuralNetwork is trained up let’s test it out. For each pair of logical values passed into our DataSet as a DataSetRow, we run the following kind of test:

ann.setInput(0, 1);
ann.calculate();
double[] networkOutputOne = ann.getOutput();

An important thing to remember is that NeuralNetworks only output a value on the inclusive interval of 0 and 1. To output some other value, we must normalize and denormalize our data.

In this case, for logical operations, 0 and 1 are perfect for the job. The output will be:

Testing: 1, 0 Expected: 1.0 Result: 1.0
Testing: 0, 1 Expected: 1.0 Result: 1.0
Testing: 1, 1 Expected: 0.0 Result: 0.0
Testing: 0, 0 Expected: 0.0 Result: 0.0

We see that our NeuralNetwork successfully predicts the right answer!

5. Conclusion

We’ve just reviewed the basic concepts and classes used by Neuroph.