Fully connected higher-order layer. More...

#include <SigmaPi.h>

Inheritance diagram for OpenANN::SigmaPi:

Classes
struct	Constraint
	A helper class for specifying weight constrains in a higher-order neural network Derive a new class from this interface and simple reimplement the function call operator for the corresponding higher-order term. More...

struct	HigherOrderUnit

Public Member Functions
	SigmaPi (OutputInfo info, bool bias, ActivationFunction act, double stdDev)
	Construct a SigmaPi layer that can be extended with different higher-order nodes. More...

virtual OutputInfo	initialize (std::vector< double * > &parameterPointers, std::vector< double * > &parameterDerivativePointers)
	See OpenANN::Layer::initialize(std::vector<double>& pParameter, std::vector<double>& pDerivative) More...

virtual SigmaPi &	secondOrderNodes (int numbers)
	Add a specific number of second-order node to this layer. More...

virtual SigmaPi &	thirdOrderNodes (int numbers)
	Add a specific number of third-order node to this layer. More...

virtual SigmaPi &	fourthOrderNodes (int numbers)
	Add a specific number of fourth-order node to this layer. More...

virtual SigmaPi &	secondOrderNodes (int numbers, const Constraint &constrain)
	Add a specific number of second-order nodes that uses the same weight sharing topology. More...

virtual SigmaPi &	thirdOrderNodes (int numbers, const Constraint &constrain)
	Add a specific number of third-order nodes that uses the same weight sharing topology. More...

virtual SigmaPi &	fourthOrderNodes (int numbers, const Constraint &constrain)
	Add a specific number of fourth-order nodes that uses the same weight sharing topology. More...

virtual size_t	nodenumber () const

virtual size_t	parameter () const

virtual void	initializeParameters ()
	Initialize the parameters. More...

virtual void	updatedParameters ()
	Generate internal parameters from externally visible parameters. More...

virtual void	forwardPropagate (Eigen::MatrixXd x, Eigen::MatrixXd &y, bool dropout=false, double *error=0)
	Forward propagation in this layer. More...

virtual void	backpropagate (Eigen::MatrixXd ein, Eigen::MatrixXd &eout, bool backpropToPrevious)
	Backpropagation in this layer. More...

virtual Eigen::MatrixXd &	getOutput ()
	Output after last forward propagation. More...

virtual Eigen::VectorXd	getParameters ()
	Get the current values of parameters (weights, biases, ...). More...

Public Member Functions inherited from OpenANN::Layer
virtual	~Layer ()

Protected Types
typedef std::vector < HigherOrderUnit >	HigherOrderNeuron

Protected Attributes
OutputInfo	info

bool	bias

ActivationFunction	act

double	stdDev

Eigen::MatrixXd	x

Eigen::MatrixXd	a

Eigen::MatrixXd	y

Eigen::MatrixXd	yd

Eigen::MatrixXd	deltas

Eigen::MatrixXd	e

std::vector< double >	w

std::vector< double >	wd

std::vector< HigherOrderNeuron >	nodes

Detailed Description

Fully connected higher-order layer.

For encoding invariances into the topology of the neural network you can specify a weight constraint for a given higher-order node.

[1] Max B. Reid, Lilly Spirkovska and Ellen Ochoa Rapid training of higher-order neural network for invariant pattern recognition Proc. IJCNN Int. Conf. Neural Networks, Vol. 1, pp. 689-692, 1989

[2] C. L. Gilles and T. Maxwell Learning, invariance, and generalization in high-order neural networks Appl. Opt, Vol. 26, pp. 4972-4978, 1987

Member Typedef Documentation

typedef std::vector<HigherOrderUnit> OpenANN::SigmaPi::HigherOrderNeuron

protected

Constructor & Destructor Documentation

OpenANN::SigmaPi::SigmaPi	(	OutputInfo	info,
		bool	bias,
		ActivationFunction	act,
		double	stdDev
	)

Construct a SigmaPi layer that can be extended with different higher-order nodes.

Parameters

info	OutputInfo of previous, connected layer
bias	flag if this layer supports a bias term for the next, connected layers
act	specifies using activation function for all higher-order nodes
stdDev	defines the standard deviation for the random weight initialization

Member Function Documentation

virtual void OpenANN::SigmaPi::backpropagate	(	Eigen::MatrixXd *	ein,
		Eigen::MatrixXd *&	eout,
		bool	backpropToPrevious
	)

virtual

Backpropagation in this layer.

Parameters

ein	pointer to error signal of the higher layer
eout	returns a pointer to error signal of the layer (derivative of the error with respect to the input)
backpropToPrevious	backpropagate errors to previous layers

Implements OpenANN::Layer.

virtual void OpenANN::SigmaPi::forwardPropagate	(	Eigen::MatrixXd *	x,
		Eigen::MatrixXd *&	y,
		bool	dropout = `false`,
		double *	error = `0`
	)

virtual

Forward propagation in this layer.

Parameters

x	pointer to input of the layer (with bias)
y	returns a pointer to output of the layer
dropout	enable dropout for regularization
error	error value, will be updated with regularization terms

Implements OpenANN::Layer.

virtual SigmaPi& OpenANN::SigmaPi::fourthOrderNodes ( int numbers)

virtual

Add a specific number of fourth-order node to this layer.

Parameters

numbers number of nodes to add

Returns: return a reference of this instance for convient layer construction

virtual SigmaPi& OpenANN::SigmaPi::fourthOrderNodes	(	int	numbers,
		const Constraint &	constrain
	)

virtual

Add a specific number of fourth-order nodes that uses the same weight sharing topology.

Parameters

numbers	number of nodes to add
constrain	specifies shared weight groups for signal korrelations from higher-order terms

Returns: return a reference of this instance for convient layer construction

virtual Eigen::MatrixXd& OpenANN::SigmaPi::getOutput ( )

virtual

Output after last forward propagation.

Returns: output

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::SigmaPi::getParameters ( )

virtual

Get the current values of parameters (weights, biases, ...).

Returns: parameters

Implements OpenANN::Layer.

virtual OutputInfo OpenANN::SigmaPi::initialize	(	std::vector< double * > &	parameterPointers,
		std::vector< double * > &	parameterDerivativePointers
	)

virtual

See OpenANN::Layer::initialize(std::vector<double*>& pParameter, std::vector<double*>& pDerivative)

Implements OpenANN::Layer.

virtual void OpenANN::SigmaPi::initializeParameters ( )

virtual

Initialize the parameters.

This is usually called before each optimization.

Implements OpenANN::Layer.

virtual size_t OpenANN::SigmaPi::nodenumber ( ) const

inlinevirtual

virtual size_t OpenANN::SigmaPi::parameter ( ) const

inlinevirtual

virtual SigmaPi& OpenANN::SigmaPi::secondOrderNodes ( int numbers)

virtual

Add a specific number of second-order node to this layer.

Parameters

numbers number of nodes to add

Returns: return a reference of this instance for convient layer construction

virtual SigmaPi& OpenANN::SigmaPi::secondOrderNodes	(	int	numbers,
		const Constraint &	constrain
	)

virtual

Add a specific number of second-order nodes that uses the same weight sharing topology.

Parameters

numbers	number of nodes to add
constrain	specifies shared weight groups for signal korrelations from higher-order terms

Returns: return a reference of this instance for convient layer construction

virtual SigmaPi& OpenANN::SigmaPi::thirdOrderNodes ( int numbers)

virtual

Add a specific number of third-order node to this layer.

Parameters

numbers number of nodes to add

Returns: return a reference of this instance for convient layer construction

virtual SigmaPi& OpenANN::SigmaPi::thirdOrderNodes	(	int	numbers,
		const Constraint &	constrain
	)

virtual

Add a specific number of third-order nodes that uses the same weight sharing topology.

Parameters

numbers	number of nodes to add
constrain	specifies shared weight groups for signal korrelations from higher-order terms

Returns: return a reference of this instance for convient layer construction

virtual void OpenANN::SigmaPi::updatedParameters ( )

virtual

Generate internal parameters from externally visible parameters.

This is usually called after each parameter update.

Implements OpenANN::Layer.

Member Data Documentation

Eigen::MatrixXd OpenANN::SigmaPi::a

protected

ActivationFunction OpenANN::SigmaPi::act

protected

bool OpenANN::SigmaPi::bias

protected

Eigen::MatrixXd OpenANN::SigmaPi::deltas

protected

Eigen::MatrixXd OpenANN::SigmaPi::e

protected

OutputInfo OpenANN::SigmaPi::info

protected

std::vector<HigherOrderNeuron> OpenANN::SigmaPi::nodes

protected

double OpenANN::SigmaPi::stdDev

protected

std::vector<double> OpenANN::SigmaPi::w

protected

std::vector<double> OpenANN::SigmaPi::wd

protected

Eigen::MatrixXd OpenANN::SigmaPi::x

protected

Eigen::MatrixXd OpenANN::SigmaPi::y

protected

Eigen::MatrixXd OpenANN::SigmaPi::yd

protected

The documentation for this class was generated from the following file:

OpenANN/layers/SigmaPi.h

Classes

Public Member Functions

Protected Types

Protected Attributes

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation