LIMODENet as a spiking network

A replay of what the trained spiking LIMODENet-skip reconstructor does, timestep by timestep, while it restores real 1 dB DVB-S2X EuroSAT test images. Every GELU of the network became a leaky integrate-and-fire (LIF) neuron; convolutions, normalisation, residual and skip additions stay graded. It converts with zero blocked operations.

Everything shown was recorded from the released checkpoint (limodenet-spiking-skip-recon-t8.pth, T = 8, seed 0) by demo/tools/record_spiking_dynamics.py. Nothing is simulated in this page. The run is a GPU simulation of the spiking network, not a measurement on neuromorphic hardware.

t = 1/8

1 · The restoration is built up over time

The same degraded image is fed at every timestep (direct coding). The output is the running mean of the per-step images (a rate readout). One timestep alone is worse than the input: the deep layers have not started firing yet. Integrating over eight timesteps recovers most of the non-spiking network's quality.

Degraded input
Degraded input (1 dB) ·
Spiking output at current timestep
Spiking output, mean of steps 1..t ·
Non-spiking LIMODENet-skip output
Non-spiking LIMODENet-skip ·
Clean reference
Clean reference

PSNR of this one image against its clean reference. Paper, full test set, 3 seeds: spiking 35.95 ± 0.05 dB, non-spiking 38.07 ± 0.26 dB.

2 · Activity reaches the deep layers over the first few steps

Each LIF population is coloured by the fraction of its neurons that spike at the current timestep. Solid arrows carry spikes. Dashed blue paths are graded: residual adds and the encoder→decoder skip projections, which stay synaptic and need no spiking substitute. Click any population to inspect it below.

spike rate0%

3 · Where the selected population fires

Fraction of channels spiking per pixel at . Click a marker to choose a probe.
Degraded input for spatial reference
Input, for spatial reference
U[t] = β·U[t−1] + I[t] − θ·S[t−1]
S[t] = 1 if U[t] > θ, else 0

4 · Individual neurons at the probe pixel

Spike raster of the first channels at the selected probe pixel (rows are neurons, columns are timesteps). Click a row to see that neuron's membrane potential: it integrates input, crosses the threshold θ, fires, and is reset by subtraction.

Data: . Membrane potentials are stored quantised to 1/32 and spike maps to 16 levels; spikes and output images are exact (lossless PNG).