Simple ANN (MLP) with C++. Synapses

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
1
down vote

favorite












Project



I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.



The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.



For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.



Implementation



Communication between neurons. Description.



This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon and Dendrite classes, so neurons do not know about each other explicitely. Axon and Dendrite classes are responsible for connecting of neurons. Also Axon class performs output operations for a neuron, Dendrite performs input operations for a neuron.



Connection is carried out by the following steps:




  1. Neuron neuronSender creates an Axon instance axon


  2. axon requests connection with Neuron neuronReceiver

  3. If neuronReceiver agrees to connect then it asks its dendtite to register neuronSender as a sender, after that axon registers neuronReceiver as a receiver, assuming that neuronReceiver has Dendrite instance dendrite

Once connection is set communication (or signal propagation) is as follows




  1. Neuron neuronSender emits signal. That means that neuronSender tells to its Axon axon to transmit signal to all receivers that are connected to it


  2. axon in turn tries to update Neuron neuronReceiver with signal value from neuronSender


  3. neuronReceiver in turn asks its Dendrite dendrite to transmit weighted signal from neuronSender. dendrite checks if neuronSender in the list of senders and if so returns weighted signal and sets status of neuronSender as being received

  4. Then neuronReceiver asks its dendrite if it is complete which means that signals from all senders have been received and if so emits itself

So signal propagation is kind of recursive process.



Communication between neurons. Code.



All stuff is placed in namespace sann.



Auxiliary objects



sann.h



#ifndef SANN_H
#define SANN_H

#include <iostream>
#include <iomanip>
#include <string>
#include <vector>

namespace sann

enum NeuronType INPUT, HIDDEN, OUTPUT ;

inline void sannwrn( std::string message )

std::cout << "WARNING: " << message << "n";


#endif


Neuron



Neuron.h



#ifndef NEURON_H
#define NEURON_H

#include "sann.h"

namespace sann

class Axon;
class Dendrite;

class Neuron

private :
Axon* _axon;//stores pointers to receivers
Dendrite* _dendrite;//stores pointers to senders

protected :
std::string _name;
NeuronType _type;//input, hidden or output

int _value;

public :
Neuron( std::string name, NeuronType type, int value = 0 ) :
_axon( nullptr ),
_dendrite( nullptr ),
_name( name ),
_type( type ),
_value( value )



/*Ask _dendrite to update current value with a weighted
*signal from a sender (fromWhich)*/
void Update( int signal, const Neuron* fromWhich );
Axon* CreateAxon();
/*Decide to accept or not a synapse with a sender*/
bool AcceptSynapse( Neuron* sender );
/*Ask _axon to send signal to all receivers*/
void Emit() const;

inline std::string GetName() const return _name;
inline NeuronType GetType() const return _type;
inline int GetValue() const return _value;
size_t GetDendriteSize() const;
size_t GetAxonSize() const;

virtual int Activation() const = 0;//returns signal Act(x)
virtual void Print() const = 0;
;


#endif


Neuron.cpp



#include "../inc/Neuron.h"
#include "../inc/Axon.h"
#include "../inc/Dendrite.h"

namespace sann

void Neuron::Update( int signal, const Neuron* fromWhich )

switch( _type )

case( INPUT ):
std::cout << "Update for INPUT neuron " << _name << "n";
return;
case( HIDDEN ):
std::cout << "Update for HIDDEN neuron " << _name << "n";
break;
case( OUTPUT ):
std::cout << "Update for OUTPUT neuron " << _name << "n";
break;

_value += _dendrite->Transmit( fromWhich, signal );

if( _dendrite->IsComplete() )

_dendrite->Reset();
Emit();




Axon* Neuron::CreateAxon()

if( _axon == nullptr )

_axon = new Axon( this );

return _axon;



bool Neuron::AcceptSynapse( Neuron* sender )

if( this->_type == INPUT )

sannwrn( "Input neuron cannot have a dendrite." );
return false;

else if( sender == this )

sannwrn( "Neuron cannot be connected with itself." );
return false;

else if( sender->GetType() == OUTPUT )

sannwrn( "Output neuron cannot have an axon" );
return false;

else

if( _dendrite == nullptr )

_dendrite = new Dendrite();

_dendrite->CreateSynapse( sender );
return true;




void Neuron::Emit() const

if( _axon == nullptr ) return;
_axon->Transmit( this->Activation() );



size_t Neuron::GetDendriteSize() const

if( _dendrite == nullptr ) return 0;
else

return _dendrite->GetSize();




size_t Neuron::GetAxonSize() const

if( _axon == nullptr ) return 0;
else

return _axon->GetSize();





Axon



Axon.h



#ifndef AXON_H
#define AXON_H

#include "Neuron.h"

namespace sann

class Axon

protected :
Neuron* _root;//master of this axon
std::vector<Neuron* > _tree;//pointers to receivers

void Transmit( int signal ) const;//send signal to all receivers

/*Only Neuron can create an instance of this class*/
Axon( Neuron* root ) : _root( root )
Axon( const Axon& ) = delete;
Axon& operator=( const Axon& ) = delete;

public :
void CreateSynapse( Neuron* receiver );//connect a receiver

inline size_t GetSize() const return _tree.size();

/*To have an access to ctor from Neuron*/
friend class Neuron;
;


#endif


Axon.cpp



#include "../inc/Axon.h"

namespace sann

void Axon::Transmit( int signal ) const

for( size_t i = 0; i < _tree.size(); i++ )

_tree[i]->Update( signal, _root );




void Axon::CreateSynapse( Neuron* receiver )

if( receiver->AcceptSynapse( _root ) )

_tree.push_back( receiver );





Dendrite



Dendrite.h



#ifndef DENDRITE_H
#define DENDRITE_H

#include "Neuron.h"

namespace sann

class Dendrite

protected :
/*Not sure about this.*/
struct Synapse

Synapse( const Neuron* neuron, bool state, int weight ) :
_neuron( neuron ),
_weight( weight ),
_state( state )

const Neuron* _neuron;
int _weight;
bool _state;
;
/*stores pointer to senders with
*their weights and statuses: either neuron
*has received signal from a sender or not.*/
std::vector<Synapse> _tree;

/*Returns weighted signal from sender (fromWhich)
*if sender is in the sender's list*
*and zero otherwise*/
int Transmit( const Neuron* fromWhich, int signal );
bool IsComplete() const;//check if neuron have received signals from all senders
void Reset();//reset statuses of all senders

/*Only Neuron can create an instance of this class*/
Dendrite()
Dendrite( const Dendrite& ) = delete;
Dendrite& operator=( const Dendrite& ) = delete;

public :
void CreateSynapse( const Neuron* sender );//connect a sender

inline size_t GetSize() const return _tree.size();

/*To have an access to ctor from Neuron*/
friend class Neuron;
;

#endif


Dendrite.cpp



#include "../inc/Dendrite.h"

namespace sann

int Dendrite::Transmit( const Neuron* fromWhich, int signal )

for( size_t i = 0; i < _tree.size(); i++ )

if( fromWhich == _tree[i]._neuron )

_tree[i]._state = true;
return signal * _tree[i]._weight;


return 0;



bool Dendrite::IsComplete() const

for( size_t i = 0; i < _tree.size(); i++ )

if( !_tree[i]._state ) return false;

return true;



void Dendrite::Reset()

for( size_t i = 0; i < _tree.size(); i++ )

_tree[i]._state = false;




void Dendrite::CreateSynapse( const Neuron* sender )

_tree.push_back( Synapse( sender, false, 1 ) );




Discussion



Neuron class



Emit function



The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?



Creation of Dendrite and Axon



I have Axon and Dendrite as a pointers. So they are initialized as nullptr in the constructor. When they are needed they are created via new operator (note that Neuron instance can have only one instance of Axon and Dendrite). I suspect that it smells like potential memory leak. Would destructor resolve this problem?



Dendrite class



Struct Synapse



I am not sure about declaring struct Synapse inside a Dendrite class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron's together with its status and weight.



IsComplete function



I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?



Mediator class



As you could see Axon and Dendrite has a lot in common. I am thinking about create a MediatorComponent class to place all of similarities inside it and then inherit from this class. What do you think?



So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.







share|improve this question



























    up vote
    1
    down vote

    favorite












    Project



    I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.



    The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.



    For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.



    Implementation



    Communication between neurons. Description.



    This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon and Dendrite classes, so neurons do not know about each other explicitely. Axon and Dendrite classes are responsible for connecting of neurons. Also Axon class performs output operations for a neuron, Dendrite performs input operations for a neuron.



    Connection is carried out by the following steps:




    1. Neuron neuronSender creates an Axon instance axon


    2. axon requests connection with Neuron neuronReceiver

    3. If neuronReceiver agrees to connect then it asks its dendtite to register neuronSender as a sender, after that axon registers neuronReceiver as a receiver, assuming that neuronReceiver has Dendrite instance dendrite

    Once connection is set communication (or signal propagation) is as follows




    1. Neuron neuronSender emits signal. That means that neuronSender tells to its Axon axon to transmit signal to all receivers that are connected to it


    2. axon in turn tries to update Neuron neuronReceiver with signal value from neuronSender


    3. neuronReceiver in turn asks its Dendrite dendrite to transmit weighted signal from neuronSender. dendrite checks if neuronSender in the list of senders and if so returns weighted signal and sets status of neuronSender as being received

    4. Then neuronReceiver asks its dendrite if it is complete which means that signals from all senders have been received and if so emits itself

    So signal propagation is kind of recursive process.



    Communication between neurons. Code.



    All stuff is placed in namespace sann.



    Auxiliary objects



    sann.h



    #ifndef SANN_H
    #define SANN_H

    #include <iostream>
    #include <iomanip>
    #include <string>
    #include <vector>

    namespace sann

    enum NeuronType INPUT, HIDDEN, OUTPUT ;

    inline void sannwrn( std::string message )

    std::cout << "WARNING: " << message << "n";


    #endif


    Neuron



    Neuron.h



    #ifndef NEURON_H
    #define NEURON_H

    #include "sann.h"

    namespace sann

    class Axon;
    class Dendrite;

    class Neuron

    private :
    Axon* _axon;//stores pointers to receivers
    Dendrite* _dendrite;//stores pointers to senders

    protected :
    std::string _name;
    NeuronType _type;//input, hidden or output

    int _value;

    public :
    Neuron( std::string name, NeuronType type, int value = 0 ) :
    _axon( nullptr ),
    _dendrite( nullptr ),
    _name( name ),
    _type( type ),
    _value( value )



    /*Ask _dendrite to update current value with a weighted
    *signal from a sender (fromWhich)*/
    void Update( int signal, const Neuron* fromWhich );
    Axon* CreateAxon();
    /*Decide to accept or not a synapse with a sender*/
    bool AcceptSynapse( Neuron* sender );
    /*Ask _axon to send signal to all receivers*/
    void Emit() const;

    inline std::string GetName() const return _name;
    inline NeuronType GetType() const return _type;
    inline int GetValue() const return _value;
    size_t GetDendriteSize() const;
    size_t GetAxonSize() const;

    virtual int Activation() const = 0;//returns signal Act(x)
    virtual void Print() const = 0;
    ;


    #endif


    Neuron.cpp



    #include "../inc/Neuron.h"
    #include "../inc/Axon.h"
    #include "../inc/Dendrite.h"

    namespace sann

    void Neuron::Update( int signal, const Neuron* fromWhich )

    switch( _type )

    case( INPUT ):
    std::cout << "Update for INPUT neuron " << _name << "n";
    return;
    case( HIDDEN ):
    std::cout << "Update for HIDDEN neuron " << _name << "n";
    break;
    case( OUTPUT ):
    std::cout << "Update for OUTPUT neuron " << _name << "n";
    break;

    _value += _dendrite->Transmit( fromWhich, signal );

    if( _dendrite->IsComplete() )

    _dendrite->Reset();
    Emit();




    Axon* Neuron::CreateAxon()

    if( _axon == nullptr )

    _axon = new Axon( this );

    return _axon;



    bool Neuron::AcceptSynapse( Neuron* sender )

    if( this->_type == INPUT )

    sannwrn( "Input neuron cannot have a dendrite." );
    return false;

    else if( sender == this )

    sannwrn( "Neuron cannot be connected with itself." );
    return false;

    else if( sender->GetType() == OUTPUT )

    sannwrn( "Output neuron cannot have an axon" );
    return false;

    else

    if( _dendrite == nullptr )

    _dendrite = new Dendrite();

    _dendrite->CreateSynapse( sender );
    return true;




    void Neuron::Emit() const

    if( _axon == nullptr ) return;
    _axon->Transmit( this->Activation() );



    size_t Neuron::GetDendriteSize() const

    if( _dendrite == nullptr ) return 0;
    else

    return _dendrite->GetSize();




    size_t Neuron::GetAxonSize() const

    if( _axon == nullptr ) return 0;
    else

    return _axon->GetSize();





    Axon



    Axon.h



    #ifndef AXON_H
    #define AXON_H

    #include "Neuron.h"

    namespace sann

    class Axon

    protected :
    Neuron* _root;//master of this axon
    std::vector<Neuron* > _tree;//pointers to receivers

    void Transmit( int signal ) const;//send signal to all receivers

    /*Only Neuron can create an instance of this class*/
    Axon( Neuron* root ) : _root( root )
    Axon( const Axon& ) = delete;
    Axon& operator=( const Axon& ) = delete;

    public :
    void CreateSynapse( Neuron* receiver );//connect a receiver

    inline size_t GetSize() const return _tree.size();

    /*To have an access to ctor from Neuron*/
    friend class Neuron;
    ;


    #endif


    Axon.cpp



    #include "../inc/Axon.h"

    namespace sann

    void Axon::Transmit( int signal ) const

    for( size_t i = 0; i < _tree.size(); i++ )

    _tree[i]->Update( signal, _root );




    void Axon::CreateSynapse( Neuron* receiver )

    if( receiver->AcceptSynapse( _root ) )

    _tree.push_back( receiver );





    Dendrite



    Dendrite.h



    #ifndef DENDRITE_H
    #define DENDRITE_H

    #include "Neuron.h"

    namespace sann

    class Dendrite

    protected :
    /*Not sure about this.*/
    struct Synapse

    Synapse( const Neuron* neuron, bool state, int weight ) :
    _neuron( neuron ),
    _weight( weight ),
    _state( state )

    const Neuron* _neuron;
    int _weight;
    bool _state;
    ;
    /*stores pointer to senders with
    *their weights and statuses: either neuron
    *has received signal from a sender or not.*/
    std::vector<Synapse> _tree;

    /*Returns weighted signal from sender (fromWhich)
    *if sender is in the sender's list*
    *and zero otherwise*/
    int Transmit( const Neuron* fromWhich, int signal );
    bool IsComplete() const;//check if neuron have received signals from all senders
    void Reset();//reset statuses of all senders

    /*Only Neuron can create an instance of this class*/
    Dendrite()
    Dendrite( const Dendrite& ) = delete;
    Dendrite& operator=( const Dendrite& ) = delete;

    public :
    void CreateSynapse( const Neuron* sender );//connect a sender

    inline size_t GetSize() const return _tree.size();

    /*To have an access to ctor from Neuron*/
    friend class Neuron;
    ;

    #endif


    Dendrite.cpp



    #include "../inc/Dendrite.h"

    namespace sann

    int Dendrite::Transmit( const Neuron* fromWhich, int signal )

    for( size_t i = 0; i < _tree.size(); i++ )

    if( fromWhich == _tree[i]._neuron )

    _tree[i]._state = true;
    return signal * _tree[i]._weight;


    return 0;



    bool Dendrite::IsComplete() const

    for( size_t i = 0; i < _tree.size(); i++ )

    if( !_tree[i]._state ) return false;

    return true;



    void Dendrite::Reset()

    for( size_t i = 0; i < _tree.size(); i++ )

    _tree[i]._state = false;




    void Dendrite::CreateSynapse( const Neuron* sender )

    _tree.push_back( Synapse( sender, false, 1 ) );




    Discussion



    Neuron class



    Emit function



    The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?



    Creation of Dendrite and Axon



    I have Axon and Dendrite as a pointers. So they are initialized as nullptr in the constructor. When they are needed they are created via new operator (note that Neuron instance can have only one instance of Axon and Dendrite). I suspect that it smells like potential memory leak. Would destructor resolve this problem?



    Dendrite class



    Struct Synapse



    I am not sure about declaring struct Synapse inside a Dendrite class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron's together with its status and weight.



    IsComplete function



    I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?



    Mediator class



    As you could see Axon and Dendrite has a lot in common. I am thinking about create a MediatorComponent class to place all of similarities inside it and then inherit from this class. What do you think?



    So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.







    share|improve this question























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      Project



      I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.



      The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.



      For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.



      Implementation



      Communication between neurons. Description.



      This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon and Dendrite classes, so neurons do not know about each other explicitely. Axon and Dendrite classes are responsible for connecting of neurons. Also Axon class performs output operations for a neuron, Dendrite performs input operations for a neuron.



      Connection is carried out by the following steps:




      1. Neuron neuronSender creates an Axon instance axon


      2. axon requests connection with Neuron neuronReceiver

      3. If neuronReceiver agrees to connect then it asks its dendtite to register neuronSender as a sender, after that axon registers neuronReceiver as a receiver, assuming that neuronReceiver has Dendrite instance dendrite

      Once connection is set communication (or signal propagation) is as follows




      1. Neuron neuronSender emits signal. That means that neuronSender tells to its Axon axon to transmit signal to all receivers that are connected to it


      2. axon in turn tries to update Neuron neuronReceiver with signal value from neuronSender


      3. neuronReceiver in turn asks its Dendrite dendrite to transmit weighted signal from neuronSender. dendrite checks if neuronSender in the list of senders and if so returns weighted signal and sets status of neuronSender as being received

      4. Then neuronReceiver asks its dendrite if it is complete which means that signals from all senders have been received and if so emits itself

      So signal propagation is kind of recursive process.



      Communication between neurons. Code.



      All stuff is placed in namespace sann.



      Auxiliary objects



      sann.h



      #ifndef SANN_H
      #define SANN_H

      #include <iostream>
      #include <iomanip>
      #include <string>
      #include <vector>

      namespace sann

      enum NeuronType INPUT, HIDDEN, OUTPUT ;

      inline void sannwrn( std::string message )

      std::cout << "WARNING: " << message << "n";


      #endif


      Neuron



      Neuron.h



      #ifndef NEURON_H
      #define NEURON_H

      #include "sann.h"

      namespace sann

      class Axon;
      class Dendrite;

      class Neuron

      private :
      Axon* _axon;//stores pointers to receivers
      Dendrite* _dendrite;//stores pointers to senders

      protected :
      std::string _name;
      NeuronType _type;//input, hidden or output

      int _value;

      public :
      Neuron( std::string name, NeuronType type, int value = 0 ) :
      _axon( nullptr ),
      _dendrite( nullptr ),
      _name( name ),
      _type( type ),
      _value( value )



      /*Ask _dendrite to update current value with a weighted
      *signal from a sender (fromWhich)*/
      void Update( int signal, const Neuron* fromWhich );
      Axon* CreateAxon();
      /*Decide to accept or not a synapse with a sender*/
      bool AcceptSynapse( Neuron* sender );
      /*Ask _axon to send signal to all receivers*/
      void Emit() const;

      inline std::string GetName() const return _name;
      inline NeuronType GetType() const return _type;
      inline int GetValue() const return _value;
      size_t GetDendriteSize() const;
      size_t GetAxonSize() const;

      virtual int Activation() const = 0;//returns signal Act(x)
      virtual void Print() const = 0;
      ;


      #endif


      Neuron.cpp



      #include "../inc/Neuron.h"
      #include "../inc/Axon.h"
      #include "../inc/Dendrite.h"

      namespace sann

      void Neuron::Update( int signal, const Neuron* fromWhich )

      switch( _type )

      case( INPUT ):
      std::cout << "Update for INPUT neuron " << _name << "n";
      return;
      case( HIDDEN ):
      std::cout << "Update for HIDDEN neuron " << _name << "n";
      break;
      case( OUTPUT ):
      std::cout << "Update for OUTPUT neuron " << _name << "n";
      break;

      _value += _dendrite->Transmit( fromWhich, signal );

      if( _dendrite->IsComplete() )

      _dendrite->Reset();
      Emit();




      Axon* Neuron::CreateAxon()

      if( _axon == nullptr )

      _axon = new Axon( this );

      return _axon;



      bool Neuron::AcceptSynapse( Neuron* sender )

      if( this->_type == INPUT )

      sannwrn( "Input neuron cannot have a dendrite." );
      return false;

      else if( sender == this )

      sannwrn( "Neuron cannot be connected with itself." );
      return false;

      else if( sender->GetType() == OUTPUT )

      sannwrn( "Output neuron cannot have an axon" );
      return false;

      else

      if( _dendrite == nullptr )

      _dendrite = new Dendrite();

      _dendrite->CreateSynapse( sender );
      return true;




      void Neuron::Emit() const

      if( _axon == nullptr ) return;
      _axon->Transmit( this->Activation() );



      size_t Neuron::GetDendriteSize() const

      if( _dendrite == nullptr ) return 0;
      else

      return _dendrite->GetSize();




      size_t Neuron::GetAxonSize() const

      if( _axon == nullptr ) return 0;
      else

      return _axon->GetSize();





      Axon



      Axon.h



      #ifndef AXON_H
      #define AXON_H

      #include "Neuron.h"

      namespace sann

      class Axon

      protected :
      Neuron* _root;//master of this axon
      std::vector<Neuron* > _tree;//pointers to receivers

      void Transmit( int signal ) const;//send signal to all receivers

      /*Only Neuron can create an instance of this class*/
      Axon( Neuron* root ) : _root( root )
      Axon( const Axon& ) = delete;
      Axon& operator=( const Axon& ) = delete;

      public :
      void CreateSynapse( Neuron* receiver );//connect a receiver

      inline size_t GetSize() const return _tree.size();

      /*To have an access to ctor from Neuron*/
      friend class Neuron;
      ;


      #endif


      Axon.cpp



      #include "../inc/Axon.h"

      namespace sann

      void Axon::Transmit( int signal ) const

      for( size_t i = 0; i < _tree.size(); i++ )

      _tree[i]->Update( signal, _root );




      void Axon::CreateSynapse( Neuron* receiver )

      if( receiver->AcceptSynapse( _root ) )

      _tree.push_back( receiver );





      Dendrite



      Dendrite.h



      #ifndef DENDRITE_H
      #define DENDRITE_H

      #include "Neuron.h"

      namespace sann

      class Dendrite

      protected :
      /*Not sure about this.*/
      struct Synapse

      Synapse( const Neuron* neuron, bool state, int weight ) :
      _neuron( neuron ),
      _weight( weight ),
      _state( state )

      const Neuron* _neuron;
      int _weight;
      bool _state;
      ;
      /*stores pointer to senders with
      *their weights and statuses: either neuron
      *has received signal from a sender or not.*/
      std::vector<Synapse> _tree;

      /*Returns weighted signal from sender (fromWhich)
      *if sender is in the sender's list*
      *and zero otherwise*/
      int Transmit( const Neuron* fromWhich, int signal );
      bool IsComplete() const;//check if neuron have received signals from all senders
      void Reset();//reset statuses of all senders

      /*Only Neuron can create an instance of this class*/
      Dendrite()
      Dendrite( const Dendrite& ) = delete;
      Dendrite& operator=( const Dendrite& ) = delete;

      public :
      void CreateSynapse( const Neuron* sender );//connect a sender

      inline size_t GetSize() const return _tree.size();

      /*To have an access to ctor from Neuron*/
      friend class Neuron;
      ;

      #endif


      Dendrite.cpp



      #include "../inc/Dendrite.h"

      namespace sann

      int Dendrite::Transmit( const Neuron* fromWhich, int signal )

      for( size_t i = 0; i < _tree.size(); i++ )

      if( fromWhich == _tree[i]._neuron )

      _tree[i]._state = true;
      return signal * _tree[i]._weight;


      return 0;



      bool Dendrite::IsComplete() const

      for( size_t i = 0; i < _tree.size(); i++ )

      if( !_tree[i]._state ) return false;

      return true;



      void Dendrite::Reset()

      for( size_t i = 0; i < _tree.size(); i++ )

      _tree[i]._state = false;




      void Dendrite::CreateSynapse( const Neuron* sender )

      _tree.push_back( Synapse( sender, false, 1 ) );




      Discussion



      Neuron class



      Emit function



      The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?



      Creation of Dendrite and Axon



      I have Axon and Dendrite as a pointers. So they are initialized as nullptr in the constructor. When they are needed they are created via new operator (note that Neuron instance can have only one instance of Axon and Dendrite). I suspect that it smells like potential memory leak. Would destructor resolve this problem?



      Dendrite class



      Struct Synapse



      I am not sure about declaring struct Synapse inside a Dendrite class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron's together with its status and weight.



      IsComplete function



      I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?



      Mediator class



      As you could see Axon and Dendrite has a lot in common. I am thinking about create a MediatorComponent class to place all of similarities inside it and then inherit from this class. What do you think?



      So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.







      share|improve this question













      Project



      I want to write a library that implements simple artificial neural network in C++. In this post I would like to discuss only a part of my project, namely connection between neurons: synapses.



      The idea is to create an interface to implement ANN's. I decided to make neurons - elementary units of ANN, to be a substantive blocks from which one builds ANN. I think this approach provides big flexibility, though something becomes more complex or cumbersome.



      For more information please visit this git repo. There you will find the list of definitions of the concepts I use below and test example that (I hope) will help understand what I have reached so far and what I want it to be.



      Implementation



      Communication between neurons. Description.



      This section contains a description of how neurons communicate with each other. The idea is to have a mediator that performs such kind of communication which are Axon and Dendrite classes, so neurons do not know about each other explicitely. Axon and Dendrite classes are responsible for connecting of neurons. Also Axon class performs output operations for a neuron, Dendrite performs input operations for a neuron.



      Connection is carried out by the following steps:




      1. Neuron neuronSender creates an Axon instance axon


      2. axon requests connection with Neuron neuronReceiver

      3. If neuronReceiver agrees to connect then it asks its dendtite to register neuronSender as a sender, after that axon registers neuronReceiver as a receiver, assuming that neuronReceiver has Dendrite instance dendrite

      Once connection is set communication (or signal propagation) is as follows




      1. Neuron neuronSender emits signal. That means that neuronSender tells to its Axon axon to transmit signal to all receivers that are connected to it


      2. axon in turn tries to update Neuron neuronReceiver with signal value from neuronSender


      3. neuronReceiver in turn asks its Dendrite dendrite to transmit weighted signal from neuronSender. dendrite checks if neuronSender in the list of senders and if so returns weighted signal and sets status of neuronSender as being received

      4. Then neuronReceiver asks its dendrite if it is complete which means that signals from all senders have been received and if so emits itself

      So signal propagation is kind of recursive process.



      Communication between neurons. Code.



      All stuff is placed in namespace sann.



      Auxiliary objects



      sann.h



      #ifndef SANN_H
      #define SANN_H

      #include <iostream>
      #include <iomanip>
      #include <string>
      #include <vector>

      namespace sann

      enum NeuronType INPUT, HIDDEN, OUTPUT ;

      inline void sannwrn( std::string message )

      std::cout << "WARNING: " << message << "n";


      #endif


      Neuron



      Neuron.h



      #ifndef NEURON_H
      #define NEURON_H

      #include "sann.h"

      namespace sann

      class Axon;
      class Dendrite;

      class Neuron

      private :
      Axon* _axon;//stores pointers to receivers
      Dendrite* _dendrite;//stores pointers to senders

      protected :
      std::string _name;
      NeuronType _type;//input, hidden or output

      int _value;

      public :
      Neuron( std::string name, NeuronType type, int value = 0 ) :
      _axon( nullptr ),
      _dendrite( nullptr ),
      _name( name ),
      _type( type ),
      _value( value )



      /*Ask _dendrite to update current value with a weighted
      *signal from a sender (fromWhich)*/
      void Update( int signal, const Neuron* fromWhich );
      Axon* CreateAxon();
      /*Decide to accept or not a synapse with a sender*/
      bool AcceptSynapse( Neuron* sender );
      /*Ask _axon to send signal to all receivers*/
      void Emit() const;

      inline std::string GetName() const return _name;
      inline NeuronType GetType() const return _type;
      inline int GetValue() const return _value;
      size_t GetDendriteSize() const;
      size_t GetAxonSize() const;

      virtual int Activation() const = 0;//returns signal Act(x)
      virtual void Print() const = 0;
      ;


      #endif


      Neuron.cpp



      #include "../inc/Neuron.h"
      #include "../inc/Axon.h"
      #include "../inc/Dendrite.h"

      namespace sann

      void Neuron::Update( int signal, const Neuron* fromWhich )

      switch( _type )

      case( INPUT ):
      std::cout << "Update for INPUT neuron " << _name << "n";
      return;
      case( HIDDEN ):
      std::cout << "Update for HIDDEN neuron " << _name << "n";
      break;
      case( OUTPUT ):
      std::cout << "Update for OUTPUT neuron " << _name << "n";
      break;

      _value += _dendrite->Transmit( fromWhich, signal );

      if( _dendrite->IsComplete() )

      _dendrite->Reset();
      Emit();




      Axon* Neuron::CreateAxon()

      if( _axon == nullptr )

      _axon = new Axon( this );

      return _axon;



      bool Neuron::AcceptSynapse( Neuron* sender )

      if( this->_type == INPUT )

      sannwrn( "Input neuron cannot have a dendrite." );
      return false;

      else if( sender == this )

      sannwrn( "Neuron cannot be connected with itself." );
      return false;

      else if( sender->GetType() == OUTPUT )

      sannwrn( "Output neuron cannot have an axon" );
      return false;

      else

      if( _dendrite == nullptr )

      _dendrite = new Dendrite();

      _dendrite->CreateSynapse( sender );
      return true;




      void Neuron::Emit() const

      if( _axon == nullptr ) return;
      _axon->Transmit( this->Activation() );



      size_t Neuron::GetDendriteSize() const

      if( _dendrite == nullptr ) return 0;
      else

      return _dendrite->GetSize();




      size_t Neuron::GetAxonSize() const

      if( _axon == nullptr ) return 0;
      else

      return _axon->GetSize();





      Axon



      Axon.h



      #ifndef AXON_H
      #define AXON_H

      #include "Neuron.h"

      namespace sann

      class Axon

      protected :
      Neuron* _root;//master of this axon
      std::vector<Neuron* > _tree;//pointers to receivers

      void Transmit( int signal ) const;//send signal to all receivers

      /*Only Neuron can create an instance of this class*/
      Axon( Neuron* root ) : _root( root )
      Axon( const Axon& ) = delete;
      Axon& operator=( const Axon& ) = delete;

      public :
      void CreateSynapse( Neuron* receiver );//connect a receiver

      inline size_t GetSize() const return _tree.size();

      /*To have an access to ctor from Neuron*/
      friend class Neuron;
      ;


      #endif


      Axon.cpp



      #include "../inc/Axon.h"

      namespace sann

      void Axon::Transmit( int signal ) const

      for( size_t i = 0; i < _tree.size(); i++ )

      _tree[i]->Update( signal, _root );




      void Axon::CreateSynapse( Neuron* receiver )

      if( receiver->AcceptSynapse( _root ) )

      _tree.push_back( receiver );





      Dendrite



      Dendrite.h



      #ifndef DENDRITE_H
      #define DENDRITE_H

      #include "Neuron.h"

      namespace sann

      class Dendrite

      protected :
      /*Not sure about this.*/
      struct Synapse

      Synapse( const Neuron* neuron, bool state, int weight ) :
      _neuron( neuron ),
      _weight( weight ),
      _state( state )

      const Neuron* _neuron;
      int _weight;
      bool _state;
      ;
      /*stores pointer to senders with
      *their weights and statuses: either neuron
      *has received signal from a sender or not.*/
      std::vector<Synapse> _tree;

      /*Returns weighted signal from sender (fromWhich)
      *if sender is in the sender's list*
      *and zero otherwise*/
      int Transmit( const Neuron* fromWhich, int signal );
      bool IsComplete() const;//check if neuron have received signals from all senders
      void Reset();//reset statuses of all senders

      /*Only Neuron can create an instance of this class*/
      Dendrite()
      Dendrite( const Dendrite& ) = delete;
      Dendrite& operator=( const Dendrite& ) = delete;

      public :
      void CreateSynapse( const Neuron* sender );//connect a sender

      inline size_t GetSize() const return _tree.size();

      /*To have an access to ctor from Neuron*/
      friend class Neuron;
      ;

      #endif


      Dendrite.cpp



      #include "../inc/Dendrite.h"

      namespace sann

      int Dendrite::Transmit( const Neuron* fromWhich, int signal )

      for( size_t i = 0; i < _tree.size(); i++ )

      if( fromWhich == _tree[i]._neuron )

      _tree[i]._state = true;
      return signal * _tree[i]._weight;


      return 0;



      bool Dendrite::IsComplete() const

      for( size_t i = 0; i < _tree.size(); i++ )

      if( !_tree[i]._state ) return false;

      return true;



      void Dendrite::Reset()

      for( size_t i = 0; i < _tree.size(); i++ )

      _tree[i]._state = false;




      void Dendrite::CreateSynapse( const Neuron* sender )

      _tree.push_back( Synapse( sender, false, 1 ) );




      Discussion



      Neuron class



      Emit function



      The main problem I see is that this function might be waiting for a long time before return. This is because of recursive process of signal propagation. The worst case is when all the receivers has almost full dendrite and get the last signal from a sender and emit theirselves and this process repeats until the last neuron in the propagation chain. is it a problem or I am missing something?



      Creation of Dendrite and Axon



      I have Axon and Dendrite as a pointers. So they are initialized as nullptr in the constructor. When they are needed they are created via new operator (note that Neuron instance can have only one instance of Axon and Dendrite). I suspect that it smells like potential memory leak. Would destructor resolve this problem?



      Dendrite class



      Struct Synapse



      I am not sure about declaring struct Synapse inside a Dendrite class. Is it bad practice? The motivation is I do not need to use that struct outside of this class. And also I need to store Neuron's together with its status and weight.



      IsComplete function



      I see that I check if signals from all senders have been received by iterating over entire dendrite tree while having function Transmit that already responsible for setting statuses of senders. Is it actually a problem or kind of a design flaw?



      Mediator class



      As you could see Axon and Dendrite has a lot in common. I am thinking about create a MediatorComponent class to place all of similarities inside it and then inherit from this class. What do you think?



      So let me know what do you think about it. As always any suggestion, critic or help would be appreciated. And sorry for a lot of material.









      share|improve this question












      share|improve this question




      share|improve this question








      edited Apr 30 at 11:28
























      asked Apr 30 at 11:16









      LRDPRDX

      2366




      2366

























          active

          oldest

          votes











          Your Answer




          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "196"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );








           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f193254%2fsimple-ann-mlp-with-c-synapses%23new-answer', 'question_page');

          );

          Post as a guest



































          active

          oldest

          votes













          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes










           

          draft saved


          draft discarded


























           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f193254%2fsimple-ann-mlp-with-c-synapses%23new-answer', 'question_page');

          );

          Post as a guest













































































          Popular posts from this blog

          Chat program with C++ and SFML

          Function to Return a JSON Like Objects Using VBA Collections and Arrays

          Will my employers contract hold up in court?