🔗 GitHub

Documentation

Structures

---

DataSet

Structure to hold user training data and keep track of dimensions

 typedef struct DataSet_ { size_t rows; size_t cols; float** data; } DataSet; 

• rows
  

  The number of rows in the dataset

• cols
  

  The number of columns in the dataset

• data
  

  The data held in the dataset, of the aforementioned dimensions

Matrix

Structure to hold internal network data and keep track of dimensions

 typedef struct Matrix_ { size_t rows; size_t cols; float* data; } Matrix; 

• rows
  

  The number of rows in the matrix

• cols
  

  The number of columns in the matrix

• data
  

  The data held in the matrix, stored one-dimensionally in row-major order

Layer

Structure to represent a layer of a network and its data

 typedef enum LAYER_TYPE_ { INPUT, HIDDEN, OUTPUT } LAYER_TYPE; 

 typedef struct Layer_ { LAYER_TYPE type; size_t size; Activation activation; Matrix* input; } Layer; 

• type
  

  The type of layer (INPUT, OUTPUT, or HIDDEN)

• size
  

  The number of neurons in the layer

• activation
  

  The activation function for the layer

• input
  

  The data currently stored in the layer in the form of a row vector with one entry per neuron in the layer

Connection

Structure to represent a link between two adjacent layers, as well the respective weights and bias

 typedef struct Connection_ { Layer* from; Layer* to; Matrix* weights; Matrix* bias; } Connection; 

• from
  

  The layer on the input side of the connection

• to
  

  The layer on the output side of the connection

• weights
  

  The weights matrix between the "from" and "to" layers, of dimensions "from"->size by "to"->size

• bias
  

  The bias vector of the "to" layer, as a row vector of "to"->size

Network

Structure to represent a neural network, with its size, layers, and connections

 typedef struct Network_ { size_t numLayers; Layer** layers; size_t numConnections; Connection** connections; } Network; 

• numLayers
  

  The total number of layers in the network

• layers
  

  The network's layers, in order

• numConnections
  

  The total number of connections in the network

• connections
  

  The network's connections, in order

Training Data Functions

---

createDataSet

Returns a pointer to a dataset of the given size and with the given data using the provided pointer

 DataSet* createDataSet(size_t rows, size_t cols, float** data); 

• rows
  

  The number of rows in the dataset

• cols
  

  The number of columns in the dataset

• data
  

  The data, of the aforementioned dimensions, to be put into the dataset

createBatches

Returns a dataset split into a given number of batches, individually represented as dataset pointers

 DataSet** createBatches(DataSet* allData, int numBatches); 

• allData
  

  The overall dataset to be split

• numBatches
  

  The number of batches to split the dataset into

destroyDataSet

Frees a dataset and its data

 void destroyDataSet(DataSet* dataset); 

• dataset
  

  The dataset to be freed

Network Functions

---

Activation

Functions that are applied to the input of each neuron within a layer

 typedef void (*Activation)(Matrix*); 

• sigmoid
  

  Applies sigmoid function to layer (for hidden layers)

• relu
  

  Applies ReLU function to layer (for hidden layers)

• tanH
  

  Applies tanh function to layer (for hidden layers)

• softmax
  

  Applies softmax function to layer (for output layer with CROSS_ENTROPY_LOSS)

• linear
  

  Applies linear function to layer (for output layer with MEAN_SQUARED_ERROR)

createMatrix

Returns a pointer to a matrix of the given size and with the given data using the provided pointer

 Matrix* createMatrix(size_t rows, size_t cols, float* data); 

• rows
  

  The number of rows in the matrix

• cols
  

  The number of columns in the matrix

• data
  

  The data, of the aforementioned dimensions, to be put into the matrix

destroyMatrix

Frees a matrix and its data

 void destroyMatrix(Matrix* matrix); 

• matrix
  

  The matrix to free

createNetwork

Returns a pointer to a network with the given dimensions and functions

 Network* createNetwork(size_t numFeatures, size_t numHiddenLayers, size_t* hiddenSizes, Activation* hiddenActivations, size_t numOutputs, Activation outputActivation); 

• numFeatures
  

  The number of neurons in the input layer

• numHiddenLayers
  

  The number of hidden layers in the network

• hiddenSizes
  

  The sizes of the hidden layers, provided as a list of sizes in order

• hiddenActivations
  

  The activation functions of the hidden layers, provided as a list of functions, in order

• numOutputs
  

  The number of neurons in the output layer

• outputActivation
  

  The activation function of the output layer (should be softmax for classification, and linear for regression)

destroyNetwork

Frees a network, its connections, and its layers

 void destroyNetwork(Network* network); 

• network
  

  The network to free

optimize

Trains a neural network given training parameters

 typedef enum LOSS_FUNCTION_ { CROSS_ENTROPY_LOSS, MEAN_SQUARED_ERROR } LOSS_FUNCTION; 

 typedef struct ParameterSet_ { Network* network; DataSet* data; DataSet* classes; LOSS_FUNCTION lossFunction; size_t batchSize; float learningRate; float searchTime; float regularizationStrength; float momentumFactor; int maxIters; int shuffle; int verbose; } ParameterSet; 

• network
  

  The network to optimize

• data
  

  The training data to train the network with

• classes
  

  The target values for the training data

• lossFunction
  

  The loss function to use when training, should be CROSS_ENTROPY_LOSS for classification and MEAN_SQUARED_ERROR for regression

• batchSize
  

  The size of each batch (provide 1 for SGD, and data->rows for Batch)

• learningRate
  

  The constant factor for initial learning rate

• searchTime
  

  The parameter for "search-then-converge" learning rate annealing that describes when the learning rate will begin slowing down, provide 0 to not use annealing

• regularizationStrength
  

  The constant factor for regularization

• momentumFactor
  

  The constant factor for simple momentum (adds a fraction of the previous weight update)

• maxIters
  

  The number of epochs to run the optimization algorithm for

• shuffle
  

  If non-zero, will shuffle the data in between epochs (recommended)

• verbose
  

  If non-zero, will print loss every 100 epochs

 void optimize(ParameterSet params); 

• params
  

  The set of parameters to use when optimizing

forwardPass

Passes matrix input through a network, storing output in the network's last layer

 void forwardPass(Network* network, Matrix* input); 

• network
  

  The network to use

• input
  

  The data to pass through the network

forwardPassDataSet

Passes dataset input through a network, storing output in the network's last layer

 void forwardPassDataSet(Network* network, DataSet* input); 

• network
  

  The network to use

• input
  

  The data to pass through the network

getOutput

Returns the output in the last layer of a network (after using forwardPass)

 Matrix* getOuput(Network* network); 

• network
  

  The network to use

predict

Returns an int* containing the indices of the predicted classes in the order they were given

 int* predict(Network* network); 

• network
  

  The network to use

accuracy

Returns a float representing the network's accuracy on the provided data's predictions

 float accuracy(Network* network, DataSet* data, DataSet* classes) 

• network
  

  The network to use

• data
  

  The data to test on

• classes
  

  The target data to test accuracy against

Serialization Functions

---

saveNetwork

Saves a network and all of its configuration details to a file

 void saveNetwork(Network* network, char* path); 

• network
  

  The network to save

• path
  

  The file path to where the network is to be saved

readNetwork

Returns a network created from the serialization of a previous network

 Network* readNetwork(char* path); 

• path
  

  The file path to where the network had been saved