Fully connected layer with compressed weights. More...

#include <Compressed.h>

Inheritance diagram for OpenANN::Compressed:

Public Member Functions
	Compressed (OutputInfo info, int J, int M, bool bias, ActivationFunction act, const std::string &compression, double stdDev, Regularization regularization)

virtual OutputInfo	initialize (std::vector< double * > &parameterPointers, std::vector< double * > &parameterDerivativePointers)
	Fill in the parameter pointers and parameter derivative pointers. More...

virtual void	initializeParameters ()
	Initialize the parameters. More...

virtual void	updatedParameters ()
	Generate internal parameters from externally visible parameters. More...

virtual void	forwardPropagate (Eigen::MatrixXd x, Eigen::MatrixXd &y, bool dropout, double *error=0)
	Forward propagation in this layer. More...

virtual void	backpropagate (Eigen::MatrixXd ein, Eigen::MatrixXd &eout, bool backpropToPrevious)
	Backpropagation in this layer. More...

virtual Eigen::MatrixXd &	getOutput ()
	Output after last forward propagation. More...

virtual Eigen::VectorXd	getParameters ()
	Get the current values of parameters (weights, biases, ...). More...

Public Member Functions inherited from OpenANN::Layer
virtual	~Layer ()

Detailed Description

Fully connected layer with compressed weights.

The number of optimizable parameters is reduced due to compressed indirect weight representation. The weight matrix is generated by $W = \alpha \Phi$ , where $\alpha$ is an optimizable parameter matrix with less components than $ W $ and $\Phi$ is a fixed matrix that could e. g. be generated randomly. Note that the compressed representation of weights is equivalent to compressing the input of the layer with $\Phi x$ and regarding $\alpha$ as the weight matrix.

Supports the following regularization types:

L1 penalty
L2 penalty

[1] A. Fabisch, Y. Kassahun, H. Wöhrle and F. Kirchner: Learning in compressed space, Neural Networks 42, pp. 83-93, ISSN 0893-6080, 2013.

Constructor & Destructor Documentation

OpenANN::Compressed::Compressed	(	OutputInfo	info,
		int	J,
		int	M,
		bool	bias,
		ActivationFunction	act,
		const std::string &	compression,
		double	stdDev,
		Regularization	regularization
	)

Member Function Documentation

virtual void OpenANN::Compressed::backpropagate	(	Eigen::MatrixXd *	ein,
		Eigen::MatrixXd *&	eout,
		bool	backpropToPrevious
	)

virtual

Backpropagation in this layer.

Parameters

ein	pointer to error signal of the higher layer
eout	returns a pointer to error signal of the layer (derivative of the error with respect to the input)
backpropToPrevious	backpropagate errors to previous layers

Implements OpenANN::Layer.

virtual void OpenANN::Compressed::forwardPropagate	(	Eigen::MatrixXd *	x,
		Eigen::MatrixXd *&	y,
		bool	dropout,
		double *	error = `0`
	)

virtual

Forward propagation in this layer.

Parameters

x	pointer to input of the layer (with bias)
y	returns a pointer to output of the layer
dropout	enable dropout for regularization
error	error value, will be updated with regularization terms

Implements OpenANN::Layer.

virtual Eigen::MatrixXd& OpenANN::Compressed::getOutput ( )

virtual

Output after last forward propagation.

Returns: output

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::Compressed::getParameters ( )

virtual

Get the current values of parameters (weights, biases, ...).

Returns: parameters

Implements OpenANN::Layer.

virtual OutputInfo OpenANN::Compressed::initialize	(	std::vector< double * > &	parameterPointers,
		std::vector< double * > &	parameterDerivativePointers
	)