Neural Networks §

• Subset of Machine Learning
• Mimicks human brain

Theory §

3-Layer NN the scalar looks like this: $y=f_3(f_2(f_1(x)))$ where $f_1$ and $f_2$ are in the form of $f_1(z)=g_i(W_{iZ}+b_i)$

• $g1$ can be a sigmoid, tanh, relu function

• Hidden Nodes (hidden layer): perform computations and transfer input nodes to output nodes
• Output Nodes (output layer): transferring information from the network to the outside world.
• Connections and weights: each connection transferring the output of a neuron to the input of a neuron. Each connection is assigned a weight.
• Activation function: non-linear function that defines the output of that node given an input or set of inputs

Types of Neural Networks §

%% %%