OpenANN  1.1.0
An open source library for artificial neural networks.
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
List of all members | Public Member Functions
OpenANN::IntrinsicPlasticity Class Reference

Learns the parameters of a logistic sigmoid activation function. More...

#include <IntrinsicPlasticity.h>

+ Inheritance diagram for OpenANN::IntrinsicPlasticity:

Public Member Functions

 IntrinsicPlasticity (int nodes, double mu, double stdDev=1.0)
 
virtual unsigned int examples ()
 Request number of training examples. More...
 
virtual unsigned int dimension ()
 Request the number of optimizable parameters. More...
 
virtual bool providesInitialization ()
 Check if the object knows how to initialize its parameters. More...
 
virtual void initialize ()
 Initialize the optimizable parameters. More...
 
virtual double error ()
 Compute error on training set. More...
 
virtual double error (unsigned int n)
 
virtual const Eigen::VectorXd & currentParameters ()
 Request the current parameters. More...
 
virtual void setParameters (const Eigen::VectorXd &parameters)
 Set new parameters. More...
 
virtual bool providesGradient ()
 Check if the object provides a gradient of the error function with respect to its parameters. More...
 
virtual Eigen::VectorXd gradient ()
 Compute gradient of the error function with respect to the parameters. More...
 
virtual Eigen::VectorXd gradient (unsigned int n)
 
virtual Eigen::VectorXd operator() (const Eigen::VectorXd &a)
 Make a prediction. More...
 
virtual Eigen::MatrixXd operator() (const Eigen::MatrixXd &A)
 Make predictions. More...
 
virtual OutputInfo initialize (std::vector< double * > &parameterPointers, std::vector< double * > &parameterDerivativePointers)
 Fill in the parameter pointers and parameter derivative pointers. More...
 
virtual void initializeParameters ()
 Initialize the parameters. More...
 
virtual void updatedParameters ()
 Generate internal parameters from externally visible parameters. More...
 
virtual void forwardPropagate (Eigen::MatrixXd *x, Eigen::MatrixXd *&y, bool dropout, double *error=0)
 Forward propagation in this layer. More...
 
virtual void backpropagate (Eigen::MatrixXd *ein, Eigen::MatrixXd *&eout, bool backpropToPrevious)
 Backpropagation in this layer. More...
 
virtual Eigen::MatrixXd & getOutput ()
 Output after last forward propagation. More...
 
virtual Eigen::VectorXd getParameters ()
 Get the current values of parameters (weights, biases, ...). More...
 
- Public Member Functions inherited from OpenANN::Learner
 Learner ()
 
virtual ~Learner ()
 
virtual LearnertrainingSet (Eigen::MatrixXd &input, Eigen::MatrixXd &output)
 Set training set. More...
 
virtual LearnertrainingSet (DataSet &trainingSet)
 Set training set. More...
 
virtual LearnerremoveTrainingSet ()
 Remove the training set from the learner. More...
 
virtual LearnervalidationSet (Eigen::MatrixXd &input, Eigen::MatrixXd &output)
 Set validation set. More...
 
virtual LearnervalidationSet (DataSet &validationSet)
 Set validation set. More...
 
virtual LearnerremoveValidationSet ()
 Remove the validation set from the learner. More...
 
- Public Member Functions inherited from OpenANN::Optimizable
virtual ~Optimizable ()
 
virtual void finishedIteration ()
 This callback is called after each optimization algorithm iteration. More...
 
virtual double error (unsigned n)
 Compute error of a given training example. More...
 
virtual Eigen::VectorXd gradient (unsigned n)
 Compute gradient of a given training example. More...
 
virtual void errorGradient (int n, double &value, Eigen::VectorXd &grad)
 Calculates the function value and gradient of a training example. More...
 
virtual void errorGradient (double &value, Eigen::VectorXd &grad)
 Calculates the function value and gradient of all training examples. More...
 
virtual Eigen::VectorXd error (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN)
 Calculates the errors of given training examples. More...
 
virtual Eigen::VectorXd gradient (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN)
 Calculates the accumulated gradient of given training examples. More...
 
virtual void errorGradient (std::vector< int >::const_iterator startN, std::vector< int >::const_iterator endN, double &value, Eigen::VectorXd &grad)
 Calculates the accumulated gradient and error of given training examples. More...
 
- Public Member Functions inherited from OpenANN::Layer
virtual ~Layer ()
 

Additional Inherited Members

- Protected Attributes inherited from OpenANN::Learner
DataSettrainSet
 
DataSetvalidSet
 
bool deleteTrainSet
 
bool deleteValidSet
 
int N
 

Detailed Description

Learns the parameters of a logistic sigmoid activation function.

Activation functions of the form $ y = 1 / (1 + \exp (-s a - b)) $ with slopes $ s $ and biases $ b $ are adapted such that the output distribution is approximately exponential with mean $ \mu $ and with respect to a input distribution given by a training set. This procedure prevents saturation. Note that changing the incoming weights might require readjustment.

[1] Jochen Triesch: A Gradient Rule for the Plasticity of a Neuron’s Intrinsic Excitability, Proceedings of the International Conference on Artificial Neural Networks, pp. 1–7, 2005.

[2] Jochen Triesch: Synergies between intrinsic and synaptic plasticity mechanisms, Neural Computation 19, pp. 885-909, 2007.

Constructor & Destructor Documentation

OpenANN::IntrinsicPlasticity::IntrinsicPlasticity ( int  nodes,
double  mu,
double  stdDev = 1.0 
)

Member Function Documentation

virtual void OpenANN::IntrinsicPlasticity::backpropagate ( Eigen::MatrixXd *  ein,
Eigen::MatrixXd *&  eout,
bool  backpropToPrevious 
)
virtual

Backpropagation in this layer.

Parameters
einpointer to error signal of the higher layer
eoutreturns a pointer to error signal of the layer (derivative of the error with respect to the input)
backpropToPreviousbackpropagate errors to previous layers

Implements OpenANN::Layer.

virtual const Eigen::VectorXd& OpenANN::IntrinsicPlasticity::currentParameters ( )
virtual

Request the current parameters.

Returns
current parameters

Implements OpenANN::Optimizable.

virtual unsigned int OpenANN::IntrinsicPlasticity::dimension ( )
virtual

Request the number of optimizable parameters.

Returns
number of optimizable parameters

Implements OpenANN::Optimizable.

virtual double OpenANN::IntrinsicPlasticity::error ( )
virtual

Compute error on training set.

Returns
current error on training set or objective function value

Implements OpenANN::Optimizable.

virtual double OpenANN::IntrinsicPlasticity::error ( unsigned int  n)
virtual
virtual unsigned int OpenANN::IntrinsicPlasticity::examples ( )
virtual

Request number of training examples.

Returns
number of training examples

Reimplemented from OpenANN::Optimizable.

virtual void OpenANN::IntrinsicPlasticity::forwardPropagate ( Eigen::MatrixXd *  x,
Eigen::MatrixXd *&  y,
bool  dropout,
double *  error = 0 
)
virtual

Forward propagation in this layer.

Parameters
xpointer to input of the layer (with bias)
yreturns a pointer to output of the layer
dropoutenable dropout for regularization
errorerror value, will be updated with regularization terms

Implements OpenANN::Layer.

virtual Eigen::MatrixXd& OpenANN::IntrinsicPlasticity::getOutput ( )
virtual

Output after last forward propagation.

Returns
output

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::IntrinsicPlasticity::getParameters ( )
virtual

Get the current values of parameters (weights, biases, ...).

Returns
parameters

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::IntrinsicPlasticity::gradient ( )
virtual

Compute gradient of the error function with respect to the parameters.

Returns
gradient

Implements OpenANN::Optimizable.

virtual Eigen::VectorXd OpenANN::IntrinsicPlasticity::gradient ( unsigned int  n)
virtual
virtual void OpenANN::IntrinsicPlasticity::initialize ( )
virtual

Initialize the optimizable parameters.

Implements OpenANN::Optimizable.

virtual OutputInfo OpenANN::IntrinsicPlasticity::initialize ( std::vector< double * > &  parameterPointers,
std::vector< double * > &  parameterDerivativePointers 
)
virtual

Fill in the parameter pointers and parameter derivative pointers.

Parameters
parameterPointerspointers to parameters
parameterDerivativePointerspointers to derivatives of parameters
Returns
information about the output of the layer

Implements OpenANN::Layer.

virtual void OpenANN::IntrinsicPlasticity::initializeParameters ( )
inlinevirtual

Initialize the parameters.

This is usually called before each optimization.

Implements OpenANN::Layer.

virtual Eigen::VectorXd OpenANN::IntrinsicPlasticity::operator() ( const Eigen::VectorXd &  x)
virtual

Make a prediction.

Parameters
xInput vector.
Returns
Prediction.

Implements OpenANN::Learner.

virtual Eigen::MatrixXd OpenANN::IntrinsicPlasticity::operator() ( const Eigen::MatrixXd &  X)
virtual

Make predictions.

Parameters
XEach row represents an input vector.
Returns
Each row represents a prediction.

Implements OpenANN::Learner.

virtual bool OpenANN::IntrinsicPlasticity::providesGradient ( )
virtual

Check if the object provides a gradient of the error function with respect to its parameters.

Returns
does the optimizable provide a gradient?

Implements OpenANN::Optimizable.

virtual bool OpenANN::IntrinsicPlasticity::providesInitialization ( )
virtual

Check if the object knows how to initialize its parameters.

Returns
does the optimizable object provide a parameter initialization?

Implements OpenANN::Optimizable.

virtual void OpenANN::IntrinsicPlasticity::setParameters ( const Eigen::VectorXd &  parameters)
virtual

Set new parameters.

Parameters
parametersnew parameters

Implements OpenANN::Optimizable.

virtual void OpenANN::IntrinsicPlasticity::updatedParameters ( )
inlinevirtual

Generate internal parameters from externally visible parameters.

This is usually called after each parameter update.

Implements OpenANN::Layer.


The documentation for this class was generated from the following file: