BNN Class Reference

Bayesian Neural Network class. More...

#include <bayes_net.hpp>

Public Member Functions
	BNN (int input_size, int hidden_size, int output_size, double prior_variance=1.0)
	Constructor for the BNN class. More...
void	fit (const std::vector< std::vector< double >> &X_train, const std::vector< std::vector< double >> &y_train, int epochs=1000)
	Train the Bayesian Neural Network. More...
std::vector< double >	predict (const std::vector< double > &input_vector)
	Predict the output for a given input. More...
double	activation_function (double x)
	Activation function for the hidden layer. More...
double	prior_log_likelihood ()
	Compute the log-likelihood of the prior distribution. More...
double	log_likelihood (const std::vector< std::vector< double >> &X, const std::vector< std::vector< double >> &y)
	Compute the log-likelihood of the data. More...
double	compute_loss (const std::vector< std::vector< double >> &X, const std::vector< std::vector< double >> &y)
	Compute the negative log posterior (loss function) More...
void	update_weights (const std::vector< std::vector< double >> &X, const std::vector< std::vector< double >> &y, double learning_rate)
	Update weights using stochastic gradient descent. More...

Public Attributes
int	input_size
	Number of input features. More...
int	hidden_size
	Number of hidden units in the network. More...
int	output_size
	Number of output units in the network. More...
double	prior_variance
	Variance for the prior distribution. More...
std::vector< std::vector< double > >	input_to_hidden_weights
	Weights from input to hidden layer. More...
std::vector< double >	hidden_biases
	Biases for the hidden layer. More...
std::vector< std::vector< double > >	hidden_to_output_weights
	Weights from hidden to output layer. More...
std::vector< double >	output_biases
	Biases for the output layer. More...
std::mt19937	rng
	Mersenne Twister random number generator. More...

Detailed Description

Bayesian Neural Network class.

Definition at line 45 of file bayes_net.hpp.

Constructor & Destructor Documentation

BNN()

BNN::BNN	(	int	input_size,
		int	hidden_size,
		int	output_size,
		double	prior_variance = 1.0
	)

Constructor for the BNN class.

Parameters

input_size	Number of input features
hidden_size	Number of hidden units in the network
output_size	Number of output units in the network
prior_variance	Variance for the prior distribution (default is 1.0)

Definition at line 36 of file bayes_net.cpp.

    : input_size(in_size), hidden_size(h_size), output_size(out_size),
      prior_variance(p_variance), rng(std::random_device{}()) {
    // initialize weights and biases with random values
    std::normal_distribution<double> normal_distribution(0.0, 1.0);
 
    input_to_hidden_weights.resize(hidden_size,
                                   std::vector<double>(input_size));
    for (int i = 0; i < hidden_size; ++i) {
        for (int j = 0; j < input_size; ++j) {
            input_to_hidden_weights[i][j] = normal_distribution(rng);
        }
    }
 
    hidden_biases.resize(hidden_size);
    for (int i = 0; i < hidden_size; ++i) {
        hidden_biases[i] = normal_distribution(rng);
    }
 
    hidden_to_output_weights.resize(output_size,
                                    std::vector<double>(hidden_size));
    for (int i = 0; i < output_size; ++i) {
        for (int j = 0; j < hidden_size; ++j) {
            hidden_to_output_weights[i][j] = normal_distribution(rng);
        }
    }
 
    output_biases.resize(output_size);
    for (int i = 0; i < output_size; ++i) {
        output_biases[i] = normal_distribution(rng);
    }
}

References hidden_biases, hidden_size, hidden_to_output_weights, input_size, input_to_hidden_weights, output_biases, output_size, and rng.

Member Function Documentation

activation_function()

double BNN::activation_function

(

double

x

)

Activation function for the hidden layer.

Parameters

x	Input value

Returns

Activated output

Definition at line 69 of file bayes_net.cpp.

                                        {
    // Using sigmoid activation function for simplicity
    return 1.0 / (1.0 + std::exp(-x));
}

Referenced by log_likelihood(), predict(), and update_weights().

compute_loss()

double BNN::compute_loss	(	const std::vector< std::vector< double >> &	X,
		const std::vector< std::vector< double >> &	y
	)

Compute the negative log posterior (loss function)

Parameters

X	Input features
y	True labels

Returns

Negative log posterior

Definition at line 194 of file bayes_net.cpp.

                                                                {
    // Negative log posterior (loss function)
    return -log_likelihood(X, y) - prior_log_likelihood();
}

References log_likelihood(), and prior_log_likelihood().

Referenced by fit().

fit()

void BNN::fit	(	const std::vector< std::vector< double >> &	X_train,
		const std::vector< std::vector< double >> &	y_train,
		int	epochs = 1000
	)

Train the Bayesian Neural Network.

Parameters

X_train	Input training data
y_train	Target training data
epochs	Number of training epochs (default is 1000)

Definition at line 74 of file bayes_net.cpp.

                          {
    const double learning_rate = 0.01;
 
    for (int epoch = 0; epoch < epochs; ++epoch) {
        update_weights(X_train, y_train, learning_rate);
 
        // Print loss for monitoring training progress
        if (epoch % 100 == 0) {
            double loss = compute_loss(X_train, y_train);
            std::cout << "Epoch: " << epoch << ", Loss: " << loss << std::endl;
        }
    }
}

References compute_loss(), and update_weights().

log_likelihood()

double BNN::log_likelihood	(	const std::vector< std::vector< double >> &	X,
		const std::vector< std::vector< double >> &	y
	)

Compute the log-likelihood of the data.

Parameters

X	Input features
y	True labels

Returns

Log-likelihood of the data

Definition at line 156 of file bayes_net.cpp.

                                                                  {
    // Compute log-likelihood of the data
    double log_likelihood = 0.0;
 
    for (size_t i = 0; i < X.size(); ++i) {
        // Forward pass
        std::vector<double> hidden_output(hidden_size);
        for (int j = 0; j < hidden_size; ++j) {
            hidden_output[j] = activation_function(
                std::inner_product(X[i].begin(),
                                   X[i].end(),
                                   input_to_hidden_weights[j].begin(),
                                   0.0) +
                hidden_biases[j]);
        }
 
        std::vector<double> output(output_size);
        for (int j = 0; j < output_size; ++j) {
            output[j] = activation_function(
                std::inner_product(hidden_output.begin(),
                                   hidden_output.end(),
                                   hidden_to_output_weights[j].begin(),
                                   0.0) +
                output_biases[j]);
        }
 
        // Log-likelihood for Gaussian noise
        for (int j = 0; j < output_size; ++j) {
            log_likelihood +=
                -0.5 * std::log(2 * M_PI) - 0.5 * std::log(prior_variance) -
                0.5 * std::pow(y[i][j] - output[j], 2) / prior_variance;
        }
    }
 
    return log_likelihood;
}

References activation_function(), hidden_biases, hidden_size, hidden_to_output_weights, input_to_hidden_weights, output_biases, output_size, and prior_variance.

Referenced by compute_loss(), and prior_log_likelihood().

predict()

std::vector< double > BNN::predict

(

const std::vector< double > &

input_vector

)

Predict the output for a given input.

Parameters

input_vector

Input data for prediction

Returns

Predicted output vector

Definition at line 90 of file bayes_net.cpp.

                                                                    {
    // Forward pass
    std::vector<double> hidden_output(hidden_size);
    for (int i = 0; i < hidden_size; ++i) {
        hidden_output[i] = activation_function(
            std::inner_product(input_vector.begin(),
                               input_vector.end(),
                               input_to_hidden_weights[i].begin(),
                               0.0) +
            hidden_biases[i]);
    }
 
    std::vector<double> output(output_size);
    for (int i = 0; i < output_size; ++i) {
        output[i] = activation_function(
            std::inner_product(hidden_output.begin(),
                               hidden_output.end(),
                               hidden_to_output_weights[i].begin(),
                               0.0) +
            output_biases[i]);
    }
 
    return output;
}

References activation_function(), hidden_biases, hidden_size, hidden_to_output_weights, input_to_hidden_weights, output_biases, and output_size.

prior_log_likelihood()

double BNN::prior_log_likelihood

(

)

Compute the log-likelihood of the prior distribution.

Returns

Log-likelihood of the prior

Definition at line 115 of file bayes_net.cpp.

                                 {
    // Compute log-likelihood of the prior
    double log_likelihood = 0.0;
 
    // Prior for input-to-hidden weights
    for (int i = 0; i < hidden_size; ++i) {
        for (int j = 0; j < input_size; ++j) {
            log_likelihood +=
                -0.5 *
                (std::pow(input_to_hidden_weights[i][j], 2) / prior_variance +
                 std::log(2 * M_PI * prior_variance));
        }
    }
 
    // Prior for hidden biases
    for (int i = 0; i < hidden_size; ++i) {
        log_likelihood +=
            -0.5 * (std::pow(hidden_biases[i], 2) / prior_variance +
                    std::log(2 * M_PI * prior_variance));
    }
 
    // Prior for hidden-to-output weights
    for (int i = 0; i < output_size; ++i) {
        for (int j = 0; j < hidden_size; ++j) {
            log_likelihood +=
                -0.5 *
                (std::pow(hidden_to_output_weights[i][j], 2) / prior_variance +
                 std::log(2 * M_PI * prior_variance));
        }
    }
 
    // Prior for output biases
    for (int i = 0; i < output_size; ++i) {
        log_likelihood +=
            -0.5 * (std::pow(output_biases[i], 2) / prior_variance +
                    std::log(2 * M_PI * prior_variance));
    }
 
    return log_likelihood;
}

References hidden_biases, hidden_size, hidden_to_output_weights, input_size, input_to_hidden_weights, log_likelihood(), output_biases, output_size, and prior_variance.

Referenced by compute_loss().

update_weights()

void BNN::update_weights	(	const std::vector< std::vector< double >> &	X,
		const std::vector< std::vector< double >> &	y,
		double	learning_rate
	)

Update weights using stochastic gradient descent.

Parameters

X	Input features
y	True labels
learning_rate	Learning rate for the update

Definition at line 200 of file bayes_net.cpp.

                                               {
    // Update weights using stochastic gradient descent
    for (size_t i = 0; i < X.size(); ++i) {
        // Forward pass
        std::vector<double> hidden_output(hidden_size);
        for (int j = 0; j < hidden_size; ++j) {
            hidden_output[j] = activation_function(
                std::inner_product(X[i].begin(),
                                   X[i].end(),
                                   input_to_hidden_weights[j].begin(),
                                   0.0) +
                hidden_biases[j]);
        }
 
        std::vector<double> output(output_size);
        for (int j = 0; j < output_size; ++j) {
            output[j] = activation_function(
                std::inner_product(hidden_output.begin(),
                                   hidden_output.end(),
                                   hidden_to_output_weights[j].begin(),
                                   0.0) +
                output_biases[j]);
        }
 
        // Backward pass (stochastic gradient descent)
        for (int j = 0; j < output_size; ++j) {
            // Gradient for output layer
            double output_gradient =
                (y[i][j] - output[j]) * output[j] * (1.0 - output[j]);
 
            // Update hidden-to-output weights
            for (int k = 0; k < hidden_size; ++k) {
                hidden_to_output_weights[j][k] +=
                    learning_rate * output_gradient * hidden_output[k];
            }
 
            // Update output biases
            output_biases[j] += learning_rate * output_gradient;
        }
 
        // Backward pass for hidden layer
        for (int j = 0; j < hidden_size; ++j) {
            // Gradient for hidden layer
            double hidden_gradient = 0.0;
            for (int k = 0; k < output_size; ++k) {
                hidden_gradient += input_to_hidden_weights[k][j] *
                                   (y[i][k] - output[k]) * output[k] *
                                   (1.0 - output[k]);
            }
            hidden_gradient *= hidden_output[j] * (1.0 - hidden_output[j]);
 
            // Update input-to-hidden weights
            for (int k = 0; k < input_size; ++k) {
                input_to_hidden_weights[j][k] +=
                    learning_rate * hidden_gradient * X[i][k];
            }
 
            // Update hidden biases
            hidden_biases[j] += learning_rate * hidden_gradient;
        }
    }
}

References activation_function(), hidden_biases, hidden_size, hidden_to_output_weights, input_size, input_to_hidden_weights, output_biases, and output_size.

Referenced by fit().

Member Data Documentation

hidden_biases

std::vector<double> BNN::hidden_biases

Biases for the hidden layer.

Definition at line 105 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

hidden_size

int BNN::hidden_size

Number of hidden units in the network.

Definition at line 85 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

hidden_to_output_weights

std::vector<std::vector<double> > BNN::hidden_to_output_weights

Weights from hidden to output layer.

Definition at line 110 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

input_size

int BNN::input_size

Number of input features.

Definition at line 80 of file bayes_net.hpp.

Referenced by BNN(), prior_log_likelihood(), and update_weights().

input_to_hidden_weights

std::vector<std::vector<double> > BNN::input_to_hidden_weights

Weights from input to hidden layer.

Definition at line 100 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

output_biases

std::vector<double> BNN::output_biases

Biases for the output layer.

Definition at line 115 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

output_size

int BNN::output_size

Number of output units in the network.

Definition at line 90 of file bayes_net.hpp.

Referenced by BNN(), log_likelihood(), predict(), prior_log_likelihood(), and update_weights().

prior_variance

double BNN::prior_variance

Variance for the prior distribution.

Definition at line 95 of file bayes_net.hpp.

Referenced by log_likelihood(), and prior_log_likelihood().

rng

std::mt19937 BNN::rng

Mersenne Twister random number generator.

Definition at line 120 of file bayes_net.hpp.

Referenced by BNN().

---

The documentation for this class was generated from the following files:

• include/openGPMP/ml/bayes_net.hpp
• modules/ml/bayes_net.cpp