A sparse auto-encoder tries to reconstruct the inputs from a compressed representation. More...

#include <SparseAutoEncoder.h>

Inheritance diagram for OpenANN::SparseAutoEncoder:

Public Member Functions
	SparseAutoEncoder (int D, int H, double beta, double rho, double lambda, ActivationFunction act)
	Sparse auto-encoder. More...

virtual Eigen::VectorXd	operator() (const Eigen::VectorXd &x)
	Make a prediction. More...

virtual Eigen::MatrixXd	operator() (const Eigen::MatrixXd &X)
	Make predictions. More...

virtual bool	providesInitialization ()
	Check if the object knows how to initialize its parameters. More...

virtual void	initialize ()
	Initialize the optimizable parameters. More...

virtual unsigned int	dimension ()
	Request the number of optimizable parameters. More...

virtual void	setParameters (const Eigen::VectorXd &parameters)
	Set new parameters. More...

virtual const Eigen::VectorXd &	currentParameters ()
	Request the current parameters. More...

virtual double	error ()
	Compute error on training set. More...

virtual bool	providesGradient ()
	Check if the object provides a gradient of the error function with respect to its parameters. More...

virtual Eigen::VectorXd	gradient ()
	Compute gradient of the error function with respect to the parameters. More...

virtual void	errorGradient (double &value, Eigen::VectorXd &grad)
	Calculates the function value and gradient of all training examples. More...

virtual Learner &	trainingSet (DataSet &trainingSet)
	Set training set. More...

virtual void	forwardPropagate (Eigen::MatrixXd x, Eigen::MatrixXd &y, bool dropout, double *error=0)
	Forward propagation in this layer. More...

virtual void	backpropagate (Eigen::MatrixXd ein, Eigen::MatrixXd &eout, bool backpropToPrevious)
	Backpropagation in this layer. More...

virtual Eigen::MatrixXd &	getOutput ()
	Output after last forward propagation. More...

virtual Eigen::VectorXd	getParameters ()
	Get the current values of parameters (weights, biases, ...). More...

virtual OutputInfo	initialize (std::vector< double * > &parameterPointers, std::vector< double * > &parameterDerivativePointers)
	Fill in the parameter pointers and parameter derivative pointers. More...

virtual void	initializeParameters ()
	Initialize the parameters. More...

virtual void	updatedParameters ()
	Generate internal parameters from externally visible parameters. More...

Eigen::MatrixXd	getInputWeights ()

Eigen::MatrixXd	getOutputWeights ()

Eigen::VectorXd	reconstruct (const Eigen::VectorXd &x)

Public Member Functions inherited from OpenANN::Learner
	Learner ()

virtual	~Learner ()

virtual Learner &	trainingSet (Eigen::MatrixXd &input, Eigen::MatrixXd &output)
	Set training set. More...

virtual Learner &	removeTrainingSet ()
	Remove the training set from the learner. More...

virtual Learner &	validationSet (Eigen::MatrixXd &input, Eigen::MatrixXd &output)
	Set validation set. More...

virtual Learner &	validationSet (DataSet &validationSet)
	Set validation set. More...

virtual Learner &	removeValidationSet ()
	Remove the validation set from the learner. More...

Public Member Functions inherited from OpenANN::Optimizable
virtual	~Optimizable ()

virtual void	finishedIteration ()
	This callback is called after each optimization algorithm iteration. More...

virtual unsigned	examples ()
	Request number of training examples. More...

virtual double	error (unsigned n)
	Compute error of a given training example. More...

virtual Eigen::VectorXd	gradient (unsigned n)
	Compute gradient of a given training example. More...

virtual void	errorGradient (int n, double &value, Eigen::VectorXd &grad)
	Calculates the function value and gradient of a training example. More...

virtual Eigen::VectorXd	error (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN)
	Calculates the errors of given training examples. More...

virtual Eigen::VectorXd	gradient (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN)
	Calculates the accumulated gradient of given training examples. More...

virtual void	errorGradient (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN, double &value, Eigen::VectorXd &grad)
	Calculates the accumulated gradient and error of given training examples. More...

Public Member Functions inherited from OpenANN::Layer
virtual	~Layer ()

Additional Inherited Members
Protected Attributes inherited from OpenANN::Learner
DataSet *	trainSet

DataSet *	validSet

bool	deleteTrainSet

bool	deleteValidSet

int	N

Detailed Description

A sparse auto-encoder tries to reconstruct the inputs from a compressed representation.

Its objective function includes a penalty term for the distance to the desired mean activation of the hidden nodes as well as the reconstruction error. Sparse auto-encoders (SAEs) can be used to train multiple layers of feature detectors unsupervised.

Constructor & Destructor Documentation

OpenANN::SparseAutoEncoder::SparseAutoEncoder	(	int	D,
		int	H,
		double	beta,
		double	rho,
		double	lambda,
		ActivationFunction	act
	)

Sparse auto-encoder.

Parameters

D	number of inputs
H	number of outputs
beta	weight of sparsity
rho	desired mean activation of hidden neurons
lambda	L2 norm penalty
act	activation function of the hidden layer

Member Function Documentation

virtual void OpenANN::SparseAutoEncoder::backpropagate	(	Eigen::MatrixXd *	ein,
		Eigen::MatrixXd *&	eout,
		bool	backpropToPrevious
	)

virtual

Backpropagation in this layer.

Parameters

ein	pointer to error signal of the higher layer
eout	returns a pointer to error signal of the layer (derivative of the error with respect to the input)
backpropToPrevious	backpropagate errors to previous layers

Implements OpenANN::Layer.

virtual const Eigen::VectorXd& OpenANN::SparseAutoEncoder::currentParameters ( )

virtual

Request the current parameters.

Returns: current parameters

Implements OpenANN::Optimizable.

virtual unsigned int OpenANN::SparseAutoEncoder::dimension ( )

virtual

Request the number of optimizable parameters.

Returns: number of optimizable parameters

Implements OpenANN::Optimizable.

virtual double OpenANN::SparseAutoEncoder::error ( )

virtual

Compute error on training set.

Returns: current error on training set or objective function value

Implements OpenANN::Optimizable.

virtual void OpenANN::SparseAutoEncoder::errorGradient	(	double &	value,
		Eigen::VectorXd &	grad
	)

virtual

Calculates the function value and gradient of all training examples.

Parameters

value	function value
grad	gradient of the function, lenght must be dimension()

Reimplemented from OpenANN::Optimizable.

virtual void OpenANN::SparseAutoEncoder::forwardPropagate	(	Eigen::MatrixXd *	x,
		Eigen::MatrixXd *&	y,
		bool	dropout,
		double *	error = `0`
	)

virtual

Forward propagation in this layer.

Parameters

x	pointer to input of the layer (with bias)
y	returns a pointer to output of the layer
dropout	enable dropout for regularization
error	error value, will be updated with regularization terms

Implements OpenANN::Layer.

Eigen::MatrixXd OpenANN::SparseAutoEncoder::getInputWeights ( )

virtual Eigen::MatrixXd& OpenANN::SparseAutoEncoder::getOutput ( )

virtual

Output after last forward propagation.

Returns: output

Implements OpenANN::Layer.

Eigen::MatrixXd OpenANN::SparseAutoEncoder::getOutputWeights ( )

virtual Eigen::VectorXd OpenANN::SparseAutoEncoder::getParameters ( )

virtual

Get the current values of parameters (weights, biases, ...).

Returns: parameters

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::SparseAutoEncoder::gradient ( )

virtual

Compute gradient of the error function with respect to the parameters.

Returns: gradient

Implements OpenANN::Optimizable.

virtual void OpenANN::SparseAutoEncoder::initialize ( )

virtual

Initialize the optimizable parameters.

Implements OpenANN::Optimizable.

virtual OutputInfo OpenANN::SparseAutoEncoder::initialize	(	std::vector< double * > &	parameterPointers,
		std::vector< double * > &	parameterDerivativePointers
	)