Step 8. Neural network for XOR

Video

World

XOR multi-layer ...
Neural network to do XOR.

Notes

• A multi-layer neural network can implement XOR.
• A single-layer neural network cannot.
• This multi-layer network starts random and learns XOR from exemplars.

Credits

• This is a modified port of a neural network to do XOR by the Coding Train.
• Code from here.
• Uses two libraries from here.
• See Coding Train live demo.
• See video.
• See long playlist of Coding Train videos: "Neural Networks" explaining the entire program and supporting libraries.

Look at the code

• See the "tweaker's box" at the top of the code. These are the things you can easily change.

• Initialise weights strategy:
  • Every time you run it, it starts with random weights.
  • In nn.js see the calls to Matrix.randomize().
  • Matrix.randomize() in matrix.js is edited so that it calls a function randomWeight() that we define in the tweaker's box.
  • Hence we can experiment with different weight initialisations.

Exercise

1. Change things in the "tweaker's box".

2. Switch between displaying all squares or just the 4 main squares.

3. See what happens when you:
   • Change the learning rate to 0.
   • Change the learning rate to 0.01.
   • Change the learning rate to 10.

4. See what happens when you:
   • Reduce hidden nodes to 1.
   • Increase hidden nodes to 50.

5. See what happens when you:
   • Change randomWeight to return zero.
   • Change randomWeight to return constant.

6. Change the exemplars to represent a different function (across 0,1). Tweak it to successfully learn that function from exemplars.