Simple ANN (MLP) with C++. Synapses
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
1
down vote
favorite
Project
I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.
The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.
For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.
Implementation
Communication between neurons. Description.
This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon
and Dendrite
classes, so neurons do not know about each other explicitely. Axon
and Dendrite
classes are responsible for connecting of neurons. Also Axon
class performs output operations for a neuron, Dendrite
performs input operations for a neuron.
Connection is carried out by the following steps:
Neuron neuronSender
creates anAxon
instanceaxon
axon
requests connection withNeuron neuronReceiver
- If
neuronReceiver
agrees to connect then it asks itsdendtite
to registerneuronSender
as a sender, after thataxon
registersneuronReceiver
as a receiver, assuming thatneuronReceiver
hasDendrite
instancedendrite
Once connection is set communication (or signal propagation) is as follows
Neuron neuronSender
emits signal. That means thatneuronSender
tells to itsAxon axon
to transmit signal to all receivers that are connected to itaxon
in turn tries to updateNeuron neuronReceiver
with signal value fromneuronSender
neuronReceiver
in turn asks itsDendrite dendrite
to transmit weighted signal fromneuronSender
.dendrite
checks ifneuronSender
in the list of senders and if so returns weighted signal and sets status ofneuronSender
as being received- Then
neuronReceiver
asks itsdendrite
if it is complete which means that signals from all senders have been received and if so emits itself
So signal propagation is kind of recursive process.
Communication between neurons. Code.
All stuff is placed in namespace sann
.
Auxiliary objects
sann.h
#ifndef SANN_H
#define SANN_H
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>
namespace sann
enum NeuronType INPUT, HIDDEN, OUTPUT ;
inline void sannwrn( std::string message )
std::cout << "WARNING: " << message << "n";
#endif
Neuron
Neuron.h
#ifndef NEURON_H
#define NEURON_H
#include "sann.h"
namespace sann
class Axon;
class Dendrite;
class Neuron
private :
Axon* _axon;//stores pointers to receivers
Dendrite* _dendrite;//stores pointers to senders
protected :
std::string _name;
NeuronType _type;//input, hidden or output
int _value;
public :
Neuron( std::string name, NeuronType type, int value = 0 ) :
_axon( nullptr ),
_dendrite( nullptr ),
_name( name ),
_type( type ),
_value( value )
/*Ask _dendrite to update current value with a weighted
*signal from a sender (fromWhich)*/
void Update( int signal, const Neuron* fromWhich );
Axon* CreateAxon();
/*Decide to accept or not a synapse with a sender*/
bool AcceptSynapse( Neuron* sender );
/*Ask _axon to send signal to all receivers*/
void Emit() const;
inline std::string GetName() const return _name;
inline NeuronType GetType() const return _type;
inline int GetValue() const return _value;
size_t GetDendriteSize() const;
size_t GetAxonSize() const;
virtual int Activation() const = 0;//returns signal Act(x)
virtual void Print() const = 0;
;
#endif
Neuron.cpp
#include "../inc/Neuron.h"
#include "../inc/Axon.h"
#include "../inc/Dendrite.h"
namespace sann
void Neuron::Update( int signal, const Neuron* fromWhich )
switch( _type )
case( INPUT ):
std::cout << "Update for INPUT neuron " << _name << "n";
return;
case( HIDDEN ):
std::cout << "Update for HIDDEN neuron " << _name << "n";
break;
case( OUTPUT ):
std::cout << "Update for OUTPUT neuron " << _name << "n";
break;
_value += _dendrite->Transmit( fromWhich, signal );
if( _dendrite->IsComplete() )
_dendrite->Reset();
Emit();
Axon* Neuron::CreateAxon()
if( _axon == nullptr )
_axon = new Axon( this );
return _axon;
bool Neuron::AcceptSynapse( Neuron* sender )
if( this->_type == INPUT )
sannwrn( "Input neuron cannot have a dendrite." );
return false;
else if( sender == this )
sannwrn( "Neuron cannot be connected with itself." );
return false;
else if( sender->GetType() == OUTPUT )
sannwrn( "Output neuron cannot have an axon" );
return false;
else
if( _dendrite == nullptr )
_dendrite = new Dendrite();
_dendrite->CreateSynapse( sender );
return true;
void Neuron::Emit() const
if( _axon == nullptr ) return;
_axon->Transmit( this->Activation() );
size_t Neuron::GetDendriteSize() const
if( _dendrite == nullptr ) return 0;
else
return _dendrite->GetSize();
size_t Neuron::GetAxonSize() const
if( _axon == nullptr ) return 0;
else
return _axon->GetSize();
Axon
Axon.h
#ifndef AXON_H
#define AXON_H
#include "Neuron.h"
namespace sann
class Axon
protected :
Neuron* _root;//master of this axon
std::vector<Neuron* > _tree;//pointers to receivers
void Transmit( int signal ) const;//send signal to all receivers
/*Only Neuron can create an instance of this class*/
Axon( Neuron* root ) : _root( root )
Axon( const Axon& ) = delete;
Axon& operator=( const Axon& ) = delete;
public :
void CreateSynapse( Neuron* receiver );//connect a receiver
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Axon.cpp
#include "../inc/Axon.h"
namespace sann
void Axon::Transmit( int signal ) const
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]->Update( signal, _root );
void Axon::CreateSynapse( Neuron* receiver )
if( receiver->AcceptSynapse( _root ) )
_tree.push_back( receiver );
Dendrite
Dendrite.h
#ifndef DENDRITE_H
#define DENDRITE_H
#include "Neuron.h"
namespace sann
class Dendrite
protected :
/*Not sure about this.*/
struct Synapse
Synapse( const Neuron* neuron, bool state, int weight ) :
_neuron( neuron ),
_weight( weight ),
_state( state )
const Neuron* _neuron;
int _weight;
bool _state;
;
/*stores pointer to senders with
*their weights and statuses: either neuron
*has received signal from a sender or not.*/
std::vector<Synapse> _tree;
/*Returns weighted signal from sender (fromWhich)
*if sender is in the sender's list*
*and zero otherwise*/
int Transmit( const Neuron* fromWhich, int signal );
bool IsComplete() const;//check if neuron have received signals from all senders
void Reset();//reset statuses of all senders
/*Only Neuron can create an instance of this class*/
Dendrite()
Dendrite( const Dendrite& ) = delete;
Dendrite& operator=( const Dendrite& ) = delete;
public :
void CreateSynapse( const Neuron* sender );//connect a sender
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Dendrite.cpp
#include "../inc/Dendrite.h"
namespace sann
int Dendrite::Transmit( const Neuron* fromWhich, int signal )
for( size_t i = 0; i < _tree.size(); i++ )
if( fromWhich == _tree[i]._neuron )
_tree[i]._state = true;
return signal * _tree[i]._weight;
return 0;
bool Dendrite::IsComplete() const
for( size_t i = 0; i < _tree.size(); i++ )
if( !_tree[i]._state ) return false;
return true;
void Dendrite::Reset()
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]._state = false;
void Dendrite::CreateSynapse( const Neuron* sender )
_tree.push_back( Synapse( sender, false, 1 ) );
Discussion
Neuron class
Emit function
The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?
Creation of Dendrite and Axon
I have Axon
and Dendrite
as a pointers. So they are initialized as nullptr
in the constructor. When they are needed they are created via new
operator (note that Neuron
instance can have only one instance of Axon
and Dendrite
). I suspect that it smells like potential memory leak. Would destructor resolve this problem?
Dendrite class
Struct Synapse
I am not sure about declaring struct Synapse
inside a Dendrite
class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron
's together with its status and weight.
IsComplete function
I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit
that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?
Mediator class
As you could see Axon
and Dendrite
has a lot in common. I am thinking about create a MediatorComponent
class to place all of similarities inside it and then inherit from this class. What do you think?
So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.
c++ neural-network
add a comment |Â
up vote
1
down vote
favorite
Project
I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.
The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.
For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.
Implementation
Communication between neurons. Description.
This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon
and Dendrite
classes, so neurons do not know about each other explicitely. Axon
and Dendrite
classes are responsible for connecting of neurons. Also Axon
class performs output operations for a neuron, Dendrite
performs input operations for a neuron.
Connection is carried out by the following steps:
Neuron neuronSender
creates anAxon
instanceaxon
axon
requests connection withNeuron neuronReceiver
- If
neuronReceiver
agrees to connect then it asks itsdendtite
to registerneuronSender
as a sender, after thataxon
registersneuronReceiver
as a receiver, assuming thatneuronReceiver
hasDendrite
instancedendrite
Once connection is set communication (or signal propagation) is as follows
Neuron neuronSender
emits signal. That means thatneuronSender
tells to itsAxon axon
to transmit signal to all receivers that are connected to itaxon
in turn tries to updateNeuron neuronReceiver
with signal value fromneuronSender
neuronReceiver
in turn asks itsDendrite dendrite
to transmit weighted signal fromneuronSender
.dendrite
checks ifneuronSender
in the list of senders and if so returns weighted signal and sets status ofneuronSender
as being received- Then
neuronReceiver
asks itsdendrite
if it is complete which means that signals from all senders have been received and if so emits itself
So signal propagation is kind of recursive process.
Communication between neurons. Code.
All stuff is placed in namespace sann
.
Auxiliary objects
sann.h
#ifndef SANN_H
#define SANN_H
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>
namespace sann
enum NeuronType INPUT, HIDDEN, OUTPUT ;
inline void sannwrn( std::string message )
std::cout << "WARNING: " << message << "n";
#endif
Neuron
Neuron.h
#ifndef NEURON_H
#define NEURON_H
#include "sann.h"
namespace sann
class Axon;
class Dendrite;
class Neuron
private :
Axon* _axon;//stores pointers to receivers
Dendrite* _dendrite;//stores pointers to senders
protected :
std::string _name;
NeuronType _type;//input, hidden or output
int _value;
public :
Neuron( std::string name, NeuronType type, int value = 0 ) :
_axon( nullptr ),
_dendrite( nullptr ),
_name( name ),
_type( type ),
_value( value )
/*Ask _dendrite to update current value with a weighted
*signal from a sender (fromWhich)*/
void Update( int signal, const Neuron* fromWhich );
Axon* CreateAxon();
/*Decide to accept or not a synapse with a sender*/
bool AcceptSynapse( Neuron* sender );
/*Ask _axon to send signal to all receivers*/
void Emit() const;
inline std::string GetName() const return _name;
inline NeuronType GetType() const return _type;
inline int GetValue() const return _value;
size_t GetDendriteSize() const;
size_t GetAxonSize() const;
virtual int Activation() const = 0;//returns signal Act(x)
virtual void Print() const = 0;
;
#endif
Neuron.cpp
#include "../inc/Neuron.h"
#include "../inc/Axon.h"
#include "../inc/Dendrite.h"
namespace sann
void Neuron::Update( int signal, const Neuron* fromWhich )
switch( _type )
case( INPUT ):
std::cout << "Update for INPUT neuron " << _name << "n";
return;
case( HIDDEN ):
std::cout << "Update for HIDDEN neuron " << _name << "n";
break;
case( OUTPUT ):
std::cout << "Update for OUTPUT neuron " << _name << "n";
break;
_value += _dendrite->Transmit( fromWhich, signal );
if( _dendrite->IsComplete() )
_dendrite->Reset();
Emit();
Axon* Neuron::CreateAxon()
if( _axon == nullptr )
_axon = new Axon( this );
return _axon;
bool Neuron::AcceptSynapse( Neuron* sender )
if( this->_type == INPUT )
sannwrn( "Input neuron cannot have a dendrite." );
return false;
else if( sender == this )
sannwrn( "Neuron cannot be connected with itself." );
return false;
else if( sender->GetType() == OUTPUT )
sannwrn( "Output neuron cannot have an axon" );
return false;
else
if( _dendrite == nullptr )
_dendrite = new Dendrite();
_dendrite->CreateSynapse( sender );
return true;
void Neuron::Emit() const
if( _axon == nullptr ) return;
_axon->Transmit( this->Activation() );
size_t Neuron::GetDendriteSize() const
if( _dendrite == nullptr ) return 0;
else
return _dendrite->GetSize();
size_t Neuron::GetAxonSize() const
if( _axon == nullptr ) return 0;
else
return _axon->GetSize();
Axon
Axon.h
#ifndef AXON_H
#define AXON_H
#include "Neuron.h"
namespace sann
class Axon
protected :
Neuron* _root;//master of this axon
std::vector<Neuron* > _tree;//pointers to receivers
void Transmit( int signal ) const;//send signal to all receivers
/*Only Neuron can create an instance of this class*/
Axon( Neuron* root ) : _root( root )
Axon( const Axon& ) = delete;
Axon& operator=( const Axon& ) = delete;
public :
void CreateSynapse( Neuron* receiver );//connect a receiver
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Axon.cpp
#include "../inc/Axon.h"
namespace sann
void Axon::Transmit( int signal ) const
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]->Update( signal, _root );
void Axon::CreateSynapse( Neuron* receiver )
if( receiver->AcceptSynapse( _root ) )
_tree.push_back( receiver );
Dendrite
Dendrite.h
#ifndef DENDRITE_H
#define DENDRITE_H
#include "Neuron.h"
namespace sann
class Dendrite
protected :
/*Not sure about this.*/
struct Synapse
Synapse( const Neuron* neuron, bool state, int weight ) :
_neuron( neuron ),
_weight( weight ),
_state( state )
const Neuron* _neuron;
int _weight;
bool _state;
;
/*stores pointer to senders with
*their weights and statuses: either neuron
*has received signal from a sender or not.*/
std::vector<Synapse> _tree;
/*Returns weighted signal from sender (fromWhich)
*if sender is in the sender's list*
*and zero otherwise*/
int Transmit( const Neuron* fromWhich, int signal );
bool IsComplete() const;//check if neuron have received signals from all senders
void Reset();//reset statuses of all senders
/*Only Neuron can create an instance of this class*/
Dendrite()
Dendrite( const Dendrite& ) = delete;
Dendrite& operator=( const Dendrite& ) = delete;
public :
void CreateSynapse( const Neuron* sender );//connect a sender
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Dendrite.cpp
#include "../inc/Dendrite.h"
namespace sann
int Dendrite::Transmit( const Neuron* fromWhich, int signal )
for( size_t i = 0; i < _tree.size(); i++ )
if( fromWhich == _tree[i]._neuron )
_tree[i]._state = true;
return signal * _tree[i]._weight;
return 0;
bool Dendrite::IsComplete() const
for( size_t i = 0; i < _tree.size(); i++ )
if( !_tree[i]._state ) return false;
return true;
void Dendrite::Reset()
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]._state = false;
void Dendrite::CreateSynapse( const Neuron* sender )
_tree.push_back( Synapse( sender, false, 1 ) );
Discussion
Neuron class
Emit function
The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?
Creation of Dendrite and Axon
I have Axon
and Dendrite
as a pointers. So they are initialized as nullptr
in the constructor. When they are needed they are created via new
operator (note that Neuron
instance can have only one instance of Axon
and Dendrite
). I suspect that it smells like potential memory leak. Would destructor resolve this problem?
Dendrite class
Struct Synapse
I am not sure about declaring struct Synapse
inside a Dendrite
class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron
's together with its status and weight.
IsComplete function
I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit
that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?
Mediator class
As you could see Axon
and Dendrite
has a lot in common. I am thinking about create a MediatorComponent
class to place all of similarities inside it and then inherit from this class. What do you think?
So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.
c++ neural-network
add a comment |Â
up vote
1
down vote
favorite
up vote
1
down vote
favorite
Project
I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.
The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.
For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.
Implementation
Communication between neurons. Description.
This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon
and Dendrite
classes, so neurons do not know about each other explicitely. Axon
and Dendrite
classes are responsible for connecting of neurons. Also Axon
class performs output operations for a neuron, Dendrite
performs input operations for a neuron.
Connection is carried out by the following steps:
Neuron neuronSender
creates anAxon
instanceaxon
axon
requests connection withNeuron neuronReceiver
- If
neuronReceiver
agrees to connect then it asks itsdendtite
to registerneuronSender
as a sender, after thataxon
registersneuronReceiver
as a receiver, assuming thatneuronReceiver
hasDendrite
instancedendrite
Once connection is set communication (or signal propagation) is as follows
Neuron neuronSender
emits signal. That means thatneuronSender
tells to itsAxon axon
to transmit signal to all receivers that are connected to itaxon
in turn tries to updateNeuron neuronReceiver
with signal value fromneuronSender
neuronReceiver
in turn asks itsDendrite dendrite
to transmit weighted signal fromneuronSender
.dendrite
checks ifneuronSender
in the list of senders and if so returns weighted signal and sets status ofneuronSender
as being received- Then
neuronReceiver
asks itsdendrite
if it is complete which means that signals from all senders have been received and if so emits itself
So signal propagation is kind of recursive process.
Communication between neurons. Code.
All stuff is placed in namespace sann
.
Auxiliary objects
sann.h
#ifndef SANN_H
#define SANN_H
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>
namespace sann
enum NeuronType INPUT, HIDDEN, OUTPUT ;
inline void sannwrn( std::string message )
std::cout << "WARNING: " << message << "n";
#endif
Neuron
Neuron.h
#ifndef NEURON_H
#define NEURON_H
#include "sann.h"
namespace sann
class Axon;
class Dendrite;
class Neuron
private :
Axon* _axon;//stores pointers to receivers
Dendrite* _dendrite;//stores pointers to senders
protected :
std::string _name;
NeuronType _type;//input, hidden or output
int _value;
public :
Neuron( std::string name, NeuronType type, int value = 0 ) :
_axon( nullptr ),
_dendrite( nullptr ),
_name( name ),
_type( type ),
_value( value )
/*Ask _dendrite to update current value with a weighted
*signal from a sender (fromWhich)*/
void Update( int signal, const Neuron* fromWhich );
Axon* CreateAxon();
/*Decide to accept or not a synapse with a sender*/
bool AcceptSynapse( Neuron* sender );
/*Ask _axon to send signal to all receivers*/
void Emit() const;
inline std::string GetName() const return _name;
inline NeuronType GetType() const return _type;
inline int GetValue() const return _value;
size_t GetDendriteSize() const;
size_t GetAxonSize() const;
virtual int Activation() const = 0;//returns signal Act(x)
virtual void Print() const = 0;
;
#endif
Neuron.cpp
#include "../inc/Neuron.h"
#include "../inc/Axon.h"
#include "../inc/Dendrite.h"
namespace sann
void Neuron::Update( int signal, const Neuron* fromWhich )
switch( _type )
case( INPUT ):
std::cout << "Update for INPUT neuron " << _name << "n";
return;
case( HIDDEN ):
std::cout << "Update for HIDDEN neuron " << _name << "n";
break;
case( OUTPUT ):
std::cout << "Update for OUTPUT neuron " << _name << "n";
break;
_value += _dendrite->Transmit( fromWhich, signal );
if( _dendrite->IsComplete() )
_dendrite->Reset();
Emit();
Axon* Neuron::CreateAxon()
if( _axon == nullptr )
_axon = new Axon( this );
return _axon;
bool Neuron::AcceptSynapse( Neuron* sender )
if( this->_type == INPUT )
sannwrn( "Input neuron cannot have a dendrite." );
return false;
else if( sender == this )
sannwrn( "Neuron cannot be connected with itself." );
return false;
else if( sender->GetType() == OUTPUT )
sannwrn( "Output neuron cannot have an axon" );
return false;
else
if( _dendrite == nullptr )
_dendrite = new Dendrite();
_dendrite->CreateSynapse( sender );
return true;
void Neuron::Emit() const
if( _axon == nullptr ) return;
_axon->Transmit( this->Activation() );
size_t Neuron::GetDendriteSize() const
if( _dendrite == nullptr ) return 0;
else
return _dendrite->GetSize();
size_t Neuron::GetAxonSize() const
if( _axon == nullptr ) return 0;
else
return _axon->GetSize();
Axon
Axon.h
#ifndef AXON_H
#define AXON_H
#include "Neuron.h"
namespace sann
class Axon
protected :
Neuron* _root;//master of this axon
std::vector<Neuron* > _tree;//pointers to receivers
void Transmit( int signal ) const;//send signal to all receivers
/*Only Neuron can create an instance of this class*/
Axon( Neuron* root ) : _root( root )
Axon( const Axon& ) = delete;
Axon& operator=( const Axon& ) = delete;
public :
void CreateSynapse( Neuron* receiver );//connect a receiver
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Axon.cpp
#include "../inc/Axon.h"
namespace sann
void Axon::Transmit( int signal ) const
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]->Update( signal, _root );
void Axon::CreateSynapse( Neuron* receiver )
if( receiver->AcceptSynapse( _root ) )
_tree.push_back( receiver );
Dendrite
Dendrite.h
#ifndef DENDRITE_H
#define DENDRITE_H
#include "Neuron.h"
namespace sann
class Dendrite
protected :
/*Not sure about this.*/
struct Synapse
Synapse( const Neuron* neuron, bool state, int weight ) :
_neuron( neuron ),
_weight( weight ),
_state( state )
const Neuron* _neuron;
int _weight;
bool _state;
;
/*stores pointer to senders with
*their weights and statuses: either neuron
*has received signal from a sender or not.*/
std::vector<Synapse> _tree;
/*Returns weighted signal from sender (fromWhich)
*if sender is in the sender's list*
*and zero otherwise*/
int Transmit( const Neuron* fromWhich, int signal );
bool IsComplete() const;//check if neuron have received signals from all senders
void Reset();//reset statuses of all senders
/*Only Neuron can create an instance of this class*/
Dendrite()
Dendrite( const Dendrite& ) = delete;
Dendrite& operator=( const Dendrite& ) = delete;
public :
void CreateSynapse( const Neuron* sender );//connect a sender
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Dendrite.cpp
#include "../inc/Dendrite.h"
namespace sann
int Dendrite::Transmit( const Neuron* fromWhich, int signal )
for( size_t i = 0; i < _tree.size(); i++ )
if( fromWhich == _tree[i]._neuron )
_tree[i]._state = true;
return signal * _tree[i]._weight;
return 0;
bool Dendrite::IsComplete() const
for( size_t i = 0; i < _tree.size(); i++ )
if( !_tree[i]._state ) return false;
return true;
void Dendrite::Reset()
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]._state = false;
void Dendrite::CreateSynapse( const Neuron* sender )
_tree.push_back( Synapse( sender, false, 1 ) );
Discussion
Neuron class
Emit function
The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?
Creation of Dendrite and Axon
I have Axon
and Dendrite
as a pointers. So they are initialized as nullptr
in the constructor. When they are needed they are created via new
operator (note that Neuron
instance can have only one instance of Axon
and Dendrite
). I suspect that it smells like potential memory leak. Would destructor resolve this problem?
Dendrite class
Struct Synapse
I am not sure about declaring struct Synapse
inside a Dendrite
class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron
's together with its status and weight.
IsComplete function
I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit
that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?
Mediator class
As you could see Axon
and Dendrite
has a lot in common. I am thinking about create a MediatorComponent
class to place all of similarities inside it and then inherit from this class. What do you think?
So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.
c++ neural-network
Project
I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.
The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.
For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.
Implementation
Communication between neurons. Description.
This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon
and Dendrite
classes, so neurons do not know about each other explicitely. Axon
and Dendrite
classes are responsible for connecting of neurons. Also Axon
class performs output operations for a neuron, Dendrite
performs input operations for a neuron.
Connection is carried out by the following steps:
Neuron neuronSender
creates anAxon
instanceaxon
axon
requests connection withNeuron neuronReceiver
- If
neuronReceiver
agrees to connect then it asks itsdendtite
to registerneuronSender
as a sender, after thataxon
registersneuronReceiver
as a receiver, assuming thatneuronReceiver
hasDendrite
instancedendrite
Once connection is set communication (or signal propagation) is as follows
Neuron neuronSender
emits signal. That means thatneuronSender
tells to itsAxon axon
to transmit signal to all receivers that are connected to itaxon
in turn tries to updateNeuron neuronReceiver
with signal value fromneuronSender
neuronReceiver
in turn asks itsDendrite dendrite
to transmit weighted signal fromneuronSender
.dendrite
checks ifneuronSender
in the list of senders and if so returns weighted signal and sets status ofneuronSender
as being received- Then
neuronReceiver
asks itsdendrite
if it is complete which means that signals from all senders have been received and if so emits itself
So signal propagation is kind of recursive process.
Communication between neurons. Code.
All stuff is placed in namespace sann
.
Auxiliary objects
sann.h
#ifndef SANN_H
#define SANN_H
#include <iostream>
#include <iomanip>
#include <string>
#include <vector>
namespace sann
enum NeuronType INPUT, HIDDEN, OUTPUT ;
inline void sannwrn( std::string message )
std::cout << "WARNING: " << message << "n";
#endif
Neuron
Neuron.h
#ifndef NEURON_H
#define NEURON_H
#include "sann.h"
namespace sann
class Axon;
class Dendrite;
class Neuron
private :
Axon* _axon;//stores pointers to receivers
Dendrite* _dendrite;//stores pointers to senders
protected :
std::string _name;
NeuronType _type;//input, hidden or output
int _value;
public :
Neuron( std::string name, NeuronType type, int value = 0 ) :
_axon( nullptr ),
_dendrite( nullptr ),
_name( name ),
_type( type ),
_value( value )
/*Ask _dendrite to update current value with a weighted
*signal from a sender (fromWhich)*/
void Update( int signal, const Neuron* fromWhich );
Axon* CreateAxon();
/*Decide to accept or not a synapse with a sender*/
bool AcceptSynapse( Neuron* sender );
/*Ask _axon to send signal to all receivers*/
void Emit() const;
inline std::string GetName() const return _name;
inline NeuronType GetType() const return _type;
inline int GetValue() const return _value;
size_t GetDendriteSize() const;
size_t GetAxonSize() const;
virtual int Activation() const = 0;//returns signal Act(x)
virtual void Print() const = 0;
;
#endif
Neuron.cpp
#include "../inc/Neuron.h"
#include "../inc/Axon.h"
#include "../inc/Dendrite.h"
namespace sann
void Neuron::Update( int signal, const Neuron* fromWhich )
switch( _type )
case( INPUT ):
std::cout << "Update for INPUT neuron " << _name << "n";
return;
case( HIDDEN ):
std::cout << "Update for HIDDEN neuron " << _name << "n";
break;
case( OUTPUT ):
std::cout << "Update for OUTPUT neuron " << _name << "n";
break;
_value += _dendrite->Transmit( fromWhich, signal );
if( _dendrite->IsComplete() )
_dendrite->Reset();
Emit();
Axon* Neuron::CreateAxon()
if( _axon == nullptr )
_axon = new Axon( this );
return _axon;
bool Neuron::AcceptSynapse( Neuron* sender )
if( this->_type == INPUT )
sannwrn( "Input neuron cannot have a dendrite." );
return false;
else if( sender == this )
sannwrn( "Neuron cannot be connected with itself." );
return false;
else if( sender->GetType() == OUTPUT )
sannwrn( "Output neuron cannot have an axon" );
return false;
else
if( _dendrite == nullptr )
_dendrite = new Dendrite();
_dendrite->CreateSynapse( sender );
return true;
void Neuron::Emit() const
if( _axon == nullptr ) return;
_axon->Transmit( this->Activation() );
size_t Neuron::GetDendriteSize() const
if( _dendrite == nullptr ) return 0;
else
return _dendrite->GetSize();
size_t Neuron::GetAxonSize() const
if( _axon == nullptr ) return 0;
else
return _axon->GetSize();
Axon
Axon.h
#ifndef AXON_H
#define AXON_H
#include "Neuron.h"
namespace sann
class Axon
protected :
Neuron* _root;//master of this axon
std::vector<Neuron* > _tree;//pointers to receivers
void Transmit( int signal ) const;//send signal to all receivers
/*Only Neuron can create an instance of this class*/
Axon( Neuron* root ) : _root( root )
Axon( const Axon& ) = delete;
Axon& operator=( const Axon& ) = delete;
public :
void CreateSynapse( Neuron* receiver );//connect a receiver
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Axon.cpp
#include "../inc/Axon.h"
namespace sann
void Axon::Transmit( int signal ) const
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]->Update( signal, _root );
void Axon::CreateSynapse( Neuron* receiver )
if( receiver->AcceptSynapse( _root ) )
_tree.push_back( receiver );
Dendrite
Dendrite.h
#ifndef DENDRITE_H
#define DENDRITE_H
#include "Neuron.h"
namespace sann
class Dendrite
protected :
/*Not sure about this.*/
struct Synapse
Synapse( const Neuron* neuron, bool state, int weight ) :
_neuron( neuron ),
_weight( weight ),
_state( state )
const Neuron* _neuron;
int _weight;
bool _state;
;
/*stores pointer to senders with
*their weights and statuses: either neuron
*has received signal from a sender or not.*/
std::vector<Synapse> _tree;
/*Returns weighted signal from sender (fromWhich)
*if sender is in the sender's list*
*and zero otherwise*/
int Transmit( const Neuron* fromWhich, int signal );
bool IsComplete() const;//check if neuron have received signals from all senders
void Reset();//reset statuses of all senders
/*Only Neuron can create an instance of this class*/
Dendrite()
Dendrite( const Dendrite& ) = delete;
Dendrite& operator=( const Dendrite& ) = delete;
public :
void CreateSynapse( const Neuron* sender );//connect a sender
inline size_t GetSize() const return _tree.size();
/*To have an access to ctor from Neuron*/
friend class Neuron;
;
#endif
Dendrite.cpp
#include "../inc/Dendrite.h"
namespace sann
int Dendrite::Transmit( const Neuron* fromWhich, int signal )
for( size_t i = 0; i < _tree.size(); i++ )
if( fromWhich == _tree[i]._neuron )
_tree[i]._state = true;
return signal * _tree[i]._weight;
return 0;
bool Dendrite::IsComplete() const
for( size_t i = 0; i < _tree.size(); i++ )
if( !_tree[i]._state ) return false;
return true;
void Dendrite::Reset()
for( size_t i = 0; i < _tree.size(); i++ )
_tree[i]._state = false;
void Dendrite::CreateSynapse( const Neuron* sender )
_tree.push_back( Synapse( sender, false, 1 ) );
Discussion
Neuron class
Emit function
The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?
Creation of Dendrite and Axon
I have Axon
and Dendrite
as a pointers. So they are initialized as nullptr
in the constructor. When they are needed they are created via new
operator (note that Neuron
instance can have only one instance of Axon
and Dendrite
). I suspect that it smells like potential memory leak. Would destructor resolve this problem?
Dendrite class
Struct Synapse
I am not sure about declaring struct Synapse
inside a Dendrite
class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron
's together with its status and weight.
IsComplete function
I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit
that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?
Mediator class
As you could see Axon
and Dendrite
has a lot in common. I am thinking about create a MediatorComponent
class to place all of similarities inside it and then inherit from this class. What do you think?
So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.
c++ neural-network
edited Apr 30 at 11:28
asked Apr 30 at 11:16
LRDPRDX
2366
2366
add a comment |Â
add a comment |Â
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f193254%2fsimple-ann-mlp-with-c-synapses%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password