Synaptic Current Model

Synaptic currents are generated by synaptic currents triggered by arrival of presynaptic spikes $S_{j}^{(l)} (t)$ . Spike trains $S_{j}^{(l)} (t)$ are denoted as a sum of Dirac delta functions $S_{j}^{(l)} (t) = \sum_{s \in C_{j}^{(l)}} δ (t - s)$ , where $s$ runs over the firing times $C_{j}^{(l)}$ of neuron $j$ in layer $l$ .

A good first-order approximation of the synaptic current is one of exponential decay. Synaptic currents are also assumed to sum linearly.

$\frac{d I_{i}^{(l)}}{d t} = - \underset{\exp . decay}{\underset{⏟}{\frac{I_{i}^{(l)} (t)}{τ_{syn}}}} + \underset{feed-forward}{\underset{⏟}{\sum_{j} W_{i j}^{(l)} S_{j}^{(l - 1)} (t)}} + \underset{recurrent}{\underset{⏟}{\sum_{j} V_{i j}^{(l)} S_{j}^{(l)} (t)}}$

A single LIF neuron can be simulated with 2 linear differential equations whose initial conditions change instantaneously when a spike occurs. Combining the reset term with the equation for the Leaky Integrate-And-Fire model, we get:

$\frac{d U_{i}^{(l)}}{d t} = - \frac{1}{τ_{mem}} ((U_{i}^{(l)} - U_{rest}) + R I_{i}^{(l)}) + S_{i}^{(l)} (t) (U_{rest} - ϑ)$

The solutions to Equations eq:scm and eq:lif_with_reset are approximated numerically by discretizing time, and expressing the output spike-train $S_{i}^{(l)} (n)$ of neuron $i$ in layer $l$ at time-step $n$ as a non-linear function of the membrane voltage $S_{i}^{(l)} (n) \equiv Θ (U_{i}^{(l) (n)} - θ)$ where $θ$ is the Heaviside step function, and $θ$ is the firing threshold.

Setting $U_{rest} = 0$ , $R = 1$ , $θ = 1$ , and using some small simulation time step $δ t > 0$ , we get:

$I_{i}^{(l)} [n + 1] = α I_{i}^{(l)} [n] + \sum_{j} W_{i j}^{(l)} S_{j}^{(l)} [n] + \sum_{j} V_{i j}^{(l)} S_{j}^{(l)} [n]$

with decay strength $α \equiv \exp (- \frac{Δ_{t}}{τ_{syn}})$ . Equation eq:lif_with_reset can then be expressed as:

$U_{i}^{(l)} [n + 1] = β U_{i}^{(l)} [n] + I_{i}^{(l)} [n] - S_{i}^{(l)} [n]$

with $β \equiv \exp (- \frac{Δ_{t}}{τ_{mem}})$ . These two equations characterise the dynamics of a RNN. Specifically, the state of neuron $i$ is given by the instantaneous synaptic currents $I_{i}$ and the membrane voltage $U_{i}$ .

References

(Neftci, Mostafa, and Zenke, n.d.)

Bibliography

Neftci, Emre O., Hesham Mostafa, and Friedemann Zenke. n.d. “Surrogate Gradient Learning in Spiking Neural Networks.” http://arxiv.org/abs/1901.09948v2.