SparseAutoEncoder class, a derived class from AutoEncoder. More...

#include <encoder.hpp>

Inheritance diagram for gpmp::ml::SparseAutoEncoder:

Public Member Functions
	SparseAutoEncoder (int input_size, int hidden_size, int output_size, double learning_rate, double sparsity_weight, double sparsity_target)
	Constructor for the SparseAutoEncoder class. More...

void	train (const std::vector< std::vector< double >> &training_data, int epochs) override
	Trains the sparse autoencoder on the given training data. More...

Public Member Functions inherited from gpmp::ml::AutoEncoder
	AutoEncoder (int input_size, int hidden_size, int output_size, double learning_rate)
	Constructor for the AutoEncoder class. More...

std::vector< double >	sigmoid (const std::vector< double > &x)
	Sigmoid activation function. More...

std::vector< double >	forward (const std::vector< double > &input)
	Forward pass through the autoencoder. More...

void	lrate_set (double initial_rate)
	Set the initial learning rate. More...

virtual void	lrate_update (int epoch)
	Update the learning rate based on a schedule. More...

void	display ()
	Print the weights of the autoencoder. More...

virtual void	save (const std::string &filename) const
	Save the model weights to a file. More...

virtual void	load (const std::string &filename)
	Load model weights from a file. More...

Public Attributes
double	sparsity_weight
	Weight for the sparsity penalty term. More...

double	sparsity_target
	Target sparsity level. More...

Public Attributes inherited from gpmp::ml::AutoEncoder
int	input_size
	Size of the input layer. More...

int	hidden_size
	Size of the hidden layer. More...

int	output_size
	Size of the output layer. More...

double	learning_rate
	Learning rate for training the autoencoder. More...

std::vector< std::vector< double > >	weights_input_hidden
	Weights from the input layer to the hidden layer. More...

std::vector< std::vector< double > >	weights_hidden_output
	Weights from the hidden layer to the output layer. More...

Detailed Description

SparseAutoEncoder class, a derived class from AutoEncoder.

Definition at line 193 of file encoder.hpp.

Constructor & Destructor Documentation

◆ SparseAutoEncoder()

gpmp::ml::SparseAutoEncoder::SparseAutoEncoder	(	int	input_size,
		int	hidden_size,
		int	output_size,
		double	learning_rate,
		double	sparsity_weight,
		double	sparsity_target
	)

Constructor for the SparseAutoEncoder class.

Parameters

input_size	The size of the input layer
hidden_size	The size of the hidden layer
output_size	The size of the output layer
learning_rate	The learning rate for training
sparsity_weight	The weight for the sparsity penalty term
sparsity_target	The target sparsity level

Definition at line 221 of file encoder.cpp.

     : AutoEncoder(in_size, h_size, out_size, l_rate), sparsity_weight(s_weight),
       sparsity_target(s_target) {
 }

Member Function Documentation

◆ train()

void gpmp::ml::SparseAutoEncoder::train	(	const std::vector< std::vector< double >> &	training_data,
		int	epochs
	)

overridevirtual

Trains the sparse autoencoder on the given training data.

Overrides the train method in the base class with sparsity considerations

Parameters

training_data	The training data
epochs	The number of training epochs

Reimplemented from gpmp::ml::AutoEncoder.

Definition at line 231 of file encoder.cpp.

                 {
  
     const double SPARSITY_TARGET_DECAY = 0.1;
  
     for (int epoch = 0; epoch < epochs; ++epoch) {
         for (const auto &current_input : training_data) {
             // forward pass
             std::vector<double> hidden = forward(current_input);
  
             // backward pass (gradient descent)
             for (int i = 0; i < output_size; ++i) {
                 for (int j = 0; j < hidden_size; ++j) {
                     weights_hidden_output[j][i] -=
                         learning_rate * (hidden[i] - current_input[i]) *
                         hidden[j];
                 }
             }
  
             for (int i = 0; i < hidden_size; ++i) {
                 for (int j = 0; j < input_size; ++j) {
                     double error = 0;
                     for (int k = 0; k < output_size; ++k) {
                         error += (hidden[k] - current_input[k]) *
                                  weights_hidden_output[i][k];
                     }
  
                     double sparsity_term =
                         sparsity_weight * (sparsity_target - hidden[i]);
  
                     weights_input_hidden[j][i] -=
                         learning_rate * (error + sparsity_term) *
                         current_input[j] * (1 - hidden[i]) * hidden[i];
                 }
             }
  
             for (int i = 0; i < hidden_size; ++i) {
                 double average_activation = 0.0;
                 for (const auto &input : training_data) {
                     std::vector<double> current_hidden = forward(input);
                     average_activation += current_hidden[i];
                 }
                 average_activation /= training_data.size();
                 sparsity_target =
                     (1.0 - SPARSITY_TARGET_DECAY) * sparsity_target +
                     SPARSITY_TARGET_DECAY * average_activation;
             }
         }
     }
 }

Referenced by main().

Member Data Documentation

◆ sparsity_target

double gpmp::ml::SparseAutoEncoder::sparsity_target

Target sparsity level.

Definition at line 203 of file encoder.hpp.

◆ sparsity_weight

double gpmp::ml::SparseAutoEncoder::sparsity_weight

Weight for the sparsity penalty term.

Definition at line 198 of file encoder.hpp.

The documentation for this class was generated from the following files:

include/openGPMP/ml/encoder.hpp
modules/ml/encoder.cpp

Public Member Functions

Public Attributes