A simple implementation of a vanilla autoencoder. More...

#include <encoder.hpp>

Inheritance diagram for gpmp::ml::AutoEncoder:

Public Member Functions
	AutoEncoder (int input_size, int hidden_size, int output_size, double learning_rate)
	Constructor for the AutoEncoder class. More...

std::vector< double >	sigmoid (const std::vector< double > &x)
	Sigmoid activation function. More...

std::vector< double >	forward (const std::vector< double > &input)
	Forward pass through the autoencoder. More...

virtual void	train (const std::vector< std::vector< double >> &training_data, int epochs)
	Train the autoencoder on a dataset. More...

void	lrate_set (double initial_rate)
	Set the initial learning rate. More...

virtual void	lrate_update (int epoch)
	Update the learning rate based on a schedule. More...

void	display ()
	Print the weights of the autoencoder. More...

virtual void	save (const std::string &filename) const
	Save the model weights to a file. More...

virtual void	load (const std::string &filename)
	Load model weights from a file. More...

Public Attributes
int	input_size
	Size of the input layer. More...

int	hidden_size
	Size of the hidden layer. More...

int	output_size
	Size of the output layer. More...

double	learning_rate
	Learning rate for training the autoencoder. More...

std::vector< std::vector< double > >	weights_input_hidden
	Weights from the input layer to the hidden layer. More...

std::vector< std::vector< double > >	weights_hidden_output
	Weights from the hidden layer to the output layer. More...

Detailed Description

A simple implementation of a vanilla autoencoder.

This class represents a basic autoencoder with one hidden layer. It can be used for dimensionality reduction and feature learning

Definition at line 59 of file encoder.hpp.

Constructor & Destructor Documentation

◆ AutoEncoder()

gpmp::ml::AutoEncoder::AutoEncoder	(	int	input_size,
		int	hidden_size,
		int	output_size,
		double	learning_rate
	)

Constructor for the AutoEncoder class.

Parameters

input_size	Size of the input layer
hidden_size	Size of the hidden layer
output_size	Size of the output layer
learning_rate	Learning rate for training the autoencoder

Definition at line 92 of file encoder.cpp.

     : input_size(in_size), hidden_size(h_size), output_size(out_size),
       learning_rate(l_rate) {
  
     // initialize weights randomly
     weights_input_hidden.resize(input_size, std::vector<double>(hidden_size));
     weights_hidden_output.resize(hidden_size, std::vector<double>(output_size));
  
     for (int i = 0; i < input_size; ++i) {
         for (int j = 0; j < hidden_size; ++j) {
             // random values between 0 and 1
             weights_input_hidden[i][j] = (rand() % 1000) / 1000.0;
         }
     }
     for (int i = 0; i < hidden_size; ++i) {
         for (int j = 0; j < output_size; ++j) {
             weights_hidden_output[i][j] = (rand() % 1000) / 1000.0;
         }
     }
 }

References hidden_size, input_size, output_size, weights_hidden_output, and weights_input_hidden.

Member Function Documentation

◆ display()

void gpmp::ml::AutoEncoder::display ( )

Print the weights of the autoencoder.

Prints the weights from the input layer to the hidden layer and from the hidden layer to the output layer

Definition at line 203 of file encoder.cpp.

                                 {
     std::cout << "Input to Hidden Weights:\n";
     for (int i = 0; i < input_size; ++i) {
         for (int j = 0; j < hidden_size; ++j) {
             std::cout << weights_input_hidden[i][j] << " ";
         }
         std::cout << "\n";
     }
  
     std::cout << "\nHidden to Output Weights:\n";
     for (int i = 0; i < hidden_size; ++i) {
         for (int j = 0; j < output_size; ++j) {
             std::cout << weights_hidden_output[i][j] << " ";
         }
         std::cout << "\n";
     }
 }

Referenced by main().

◆ forward()

std::vector< double > gpmp::ml::AutoEncoder::forward ( const std::vector< double > & input )

Forward pass through the autoencoder.

Computes the output of the autoencoder given an input vector

Parameters

input Input vector

Returns: Output vector of the autoencoder

Definition at line 126 of file encoder.cpp.

                                                          {
     // forward passes
     std::vector<double> hidden(hidden_size);
     std::vector<double> output(output_size);
  
     // calculate hidden layer values
     for (int i = 0; i < hidden_size; ++i) {
         hidden[i] = 0;
         for (int j = 0; j < input_size; ++j) {
             hidden[i] += input[j] * weights_input_hidden[j][i];
         }
         hidden[i] = sigmoid({hidden[i]})[0];
     }
  
     // calculate output layer values
     for (int i = 0; i < output_size; ++i) {
         output[i] = 0;
         for (int j = 0; j < hidden_size; ++j) {
             output[i] += hidden[j] * weights_hidden_output[j][i];
         }
         output[i] = sigmoid({output[i]})[0];
     }
  
     return output;
 }

Referenced by main().

◆ load()

void gpmp::ml::AutoEncoder::load ( const std::string & filename )

virtual

Load model weights from a file.

Parameters

filename The name of the file to load the weights from

Definition at line 59 of file encoder.cpp.

                                                       {
     std::ifstream file(filename, std::ios::in | std::ios::binary);
  
     if (file.is_open()) {
         file.read(reinterpret_cast<char *>(&weights_input_hidden[0][0]),
                   weights_input_hidden.size() * weights_input_hidden[0].size() *
                       sizeof(double));
         file.read(reinterpret_cast<char *>(&weights_hidden_output[0][0]),
                   weights_hidden_output.size() *
                       weights_hidden_output[0].size() * sizeof(double));
  
         file.close();
         std::cout << "Model loaded successfully." << std::endl;
     } else {
         std::cerr << "Unable to open the file for loading." << std::endl;
     }
 }

◆ lrate_set()

void gpmp::ml::AutoEncoder::lrate_set ( double initial_rate )

Set the initial learning rate.

Parameters

initial_rate The initial learning rate

Definition at line 77 of file encoder.cpp.

                                                      {
     learning_rate = initial_rate;
 }

◆ lrate_update()

void gpmp::ml::AutoEncoder::lrate_update ( int epoch )

virtual

Update the learning rate based on a schedule.

Parameters

epoch The current training epoch

Definition at line 81 of file encoder.cpp.

                                               {
     // reduce the learning rate by half every N epochs
     const int decay_interval = 10;
     if (epoch % decay_interval == 0) {
         learning_rate /= 2.0;
         std::cout << "Learning rate updated to: " << learning_rate
                   << " at epoch " << epoch << std::endl;
     }
     // TODO?
 }

◆ save()

void gpmp::ml::AutoEncoder::save ( const std::string & filename ) const

virtual

Save the model weights to a file.

Parameters

filename The name of the file to save the weights to

Definition at line 41 of file encoder.cpp.

                                                             {
     std::ofstream file(filename, std::ios::out | std::ios::binary);
  
     if (file.is_open()) {
         file.write(reinterpret_cast<const char *>(&weights_input_hidden[0][0]),
                    weights_input_hidden.size() *
                        weights_input_hidden[0].size() * sizeof(double));
         file.write(reinterpret_cast<const char *>(&weights_hidden_output[0][0]),
                    weights_hidden_output.size() *
                        weights_hidden_output[0].size() * sizeof(double));
  
         file.close();
         std::cout << "Model saved successfully." << std::endl;
     } else {
         std::cerr << "Unable to open the file for saving." << std::endl;
     }
 }

References weights_hidden_output, and weights_input_hidden.

◆ sigmoid()

std::vector< double > gpmp::ml::AutoEncoder::sigmoid ( const std::vector< double > & x )

Sigmoid activation function.

Applies the sigmoid activation function element-wise to the input vector

Parameters

x	Input vector

Returns: Result of applying the sigmoid function to each element of the input vector

Definition at line 117 of file encoder.cpp.

                                                      {
     std::vector<double> result;
     for (double val : x) {
         result.push_back(1.0 / (1.0 + exp(-val)));
     }
     return result;
 }

◆ train()

void gpmp::ml::AutoEncoder::train	(	const std::vector< std::vector< double >> &	training_data,
		int	epochs
	)

virtual

Train the autoencoder on a dataset.

Uses gradient descent to train the autoencoder on the provided training data

Parameters

training_data	Matrix containing training data
epochs	Number of training epochs

Reimplemented in gpmp::ml::FullAutoEncoder, gpmp::ml::RecurrentAutoEncoder, gpmp::ml::VariationalAutoEncoder, gpmp::ml::ConcreteAutoEncoder, gpmp::ml::MDLAutoEncoder, gpmp::ml::ContractiveAutoEncoder, gpmp::ml::DenoisingAutoEncoder, and gpmp::ml::SparseAutoEncoder.

Definition at line 152 of file encoder.cpp.

                 {
     for (int epoch = 0; epoch < epochs; ++epoch) {
         for (const auto &input : training_data) {
             // forward pass
             std::vector<double> hidden(hidden_size);
             std::vector<double> output(output_size);
  
             // calculate hidden layer values
             for (int i = 0; i < hidden_size; ++i) {
                 hidden[i] = 0;
                 for (int j = 0; j < input_size; ++j) {
                     hidden[i] += input[j] * weights_input_hidden[j][i];
                 }
                 hidden[i] = sigmoid({hidden[i]})[0];
             }
  
             // calculate output layer values
             for (int i = 0; i < output_size; ++i) {
                 output[i] = 0;
                 for (int j = 0; j < hidden_size; ++j) {
                     output[i] += hidden[j] * weights_hidden_output[j][i];
                 }
                 output[i] = sigmoid({output[i]})[0];
             }
  
             // backward pass (gradient descent)
             for (int i = 0; i < output_size; ++i) {
                 for (int j = 0; j < hidden_size; ++j) {
                     weights_hidden_output[j][i] -=
                         learning_rate * (output[i] - input[i]) * hidden[j];
                 }
             }
  
             for (int i = 0; i < hidden_size; ++i) {
                 for (int j = 0; j < input_size; ++j) {
                     double error = 0;
                     for (int k = 0; k < output_size; ++k) {
                         error += (output[k] - input[k]) *
                                  weights_hidden_output[i][k];
                     }
                     weights_input_hidden[j][i] -= learning_rate * error *
                                                   input[j] * (1 - hidden[i]) *
                                                   hidden[i];
                 }
             }
         }
     }
 }

Referenced by main().

Member Data Documentation

◆ hidden_size

int gpmp::ml::AutoEncoder::hidden_size

Size of the hidden layer.

The number of nodes in the hidden layer, which forms the compressed representation

Definition at line 74 of file encoder.hpp.

Referenced by AutoEncoder().

◆ input_size

int gpmp::ml::AutoEncoder::input_size

Size of the input layer.

The number of input nodes in the autoencoder

Definition at line 66 of file encoder.hpp.

Referenced by AutoEncoder().

◆ learning_rate

double gpmp::ml::AutoEncoder::learning_rate

Learning rate for training the autoencoder.

The rate at which the weights are updated during the training process

Definition at line 89 of file encoder.hpp.

◆ output_size

int gpmp::ml::AutoEncoder::output_size

Size of the output layer.

The number of output nodes in the autoencoder, which should match the input size

Definition at line 82 of file encoder.hpp.

Referenced by AutoEncoder().

◆ weights_hidden_output

std::vector<std::vector<double> > gpmp::ml::AutoEncoder::weights_hidden_output

Weights from the hidden layer to the output layer.

A matrix representing the weights connecting the hidden layer to the output layer

Definition at line 105 of file encoder.hpp.

Referenced by AutoEncoder(), and save().

◆ weights_input_hidden

std::vector<std::vector<double> > gpmp::ml::AutoEncoder::weights_input_hidden

Weights from the input layer to the hidden layer.

A matrix representing the weights connecting the input layer to the hidden layer

Definition at line 97 of file encoder.hpp.

Referenced by AutoEncoder(), and save().

The documentation for this class was generated from the following files:

include/openGPMP/ml/encoder.hpp
modules/ml/encoder.cpp

Public Member Functions

Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ AutoEncoder()

Member Function Documentation

◆ display()

◆ forward()

◆ load()

◆ lrate_set()

◆ lrate_update()

◆ save()

◆ sigmoid()

◆ train()

Member Data Documentation

◆ hidden_size

◆ input_size

◆ learning_rate

◆ output_size

◆ weights_hidden_output

◆ weights_input_hidden