This page was generated from /builds/synsense/rockpool/docs/tutorials/quick-xylo/deploy_to_xylo.ipynb. Interactive online version: Binder badge

⚑️🐝 Quick-start with Xylo

This notebook gives you an ultra-quick overview of taking a network from high-level Python simulation through to deployment on the Xylo SNN HDK.

The notebook shows you
  • how to build a network with Rockpool, using standard network modules and combinators

  • how to extract the computational graph for that network, containing all parameters needed to specify the network on the Xylo HDK

  • how to map the computational graph to the Xylo HW, assigning hardware resources for the network

  • how to quantize the network specification to match the integer SNN core on Xylo, and obtain a hardware configuration

  • how to connect to and configure the Xylo HDK hardware, to deploy the network

  • how to evaluate the network on the Xylo HDK

  • how to simulate the configuration in a bit-precise simulator of the Xylo HDK.

[1]:
# - Numpy
import numpy as np

# - Matplotlib
import matplotlib.pyplot as plt
%matplotlib inline
plt.rcParams['figure.figsize'] = [12, 4]
plt.rcParams['figure.dpi'] = 300

# - Rockpool time-series handling
from rockpool import TSEvent, TSContinuous

# - Pretty printing
try:
    from rich import print
except:
    pass

# - Display images
from IPython.display import Image

# - Disable warnings
import warnings
warnings.filterwarnings('ignore')

Xylo in numbers

These are the most important numbers to keep in mind when building networks for Xylo. See also Overview of the Xylo development kit. Note that the network size constraints apply only to the Xylo hardware via XyloSamna, and you can simulate larger networks using XyloSim.

Description

Number

Max. input channels

16

Max. input spikes per time step

15

Max. hidden neurons

1000

Max. hidden neuron spikes per time step

31

Max. input synapses per hidden neuron

2

Max. alias targets (hidden neurons only)

1

Max. output neurons

8

Max. output neuron spikes per time step

1

Max. input synapses per output neuron

1

Weight bit-depth

8

Synaptic state bit-depth

16

Membrane state bit-depth

16

Threshold bit-depth

16

Bit-shift decay parameter bit-depth

4

Max. bit-shift decay value

15

Longest effective time-constant

\(32768 \cdot \textrm{d}t\) (but subject to linear decay)

Step 1: Build a network in Rockpool

We will demonstrate with a straightforward network architecture, which shows off the support of Xylo. The network architecture is shown below. It consists of linear weights (β€œW”), spiking neuron layers (β€œLIF”), and a residual block containing a recurrent LIF layer (recurrent weights \(W_{rec}\)).

[2]:
Image(filename='network-architecture.png')
[2]:
../../_images/tutorials_quick-xylo_deploy_to_xylo_6_0.png
[3]:
# - Import the computational modules and combinators required for the networl
from rockpool.nn.modules import LIFTorch, LinearTorch
from rockpool.nn.combinators import Sequential, Residual
[4]:
# - Define the size of the network layers
Nin = 2
Nhidden = 4
Nout = 2
dt = 1e-3
[5]:
# - Define the network architecture using combinators and modules
net = Sequential(
    LinearTorch((Nin, Nhidden), has_bias = False),
    LIFTorch(Nhidden, dt = dt),

    Residual(
        LinearTorch((Nhidden, Nhidden), has_bias = False),
        LIFTorch(Nhidden, has_rec = True, threshold = 10., dt = dt),
    ),

    LinearTorch((Nhidden, Nout), has_bias = False),
    LIFTorch(Nout, dt = dt),
)
print(net)
TorchSequential  with shape (2, 2) {
    LinearTorch '0_LinearTorch' with shape (2, 4)
    LIFTorch '1_LIFTorch' with shape (4, 4)
    TorchResidual '2_TorchResidual' with shape (4, 4) {
        LinearTorch '0_LinearTorch' with shape (4, 4)
        LIFTorch '1_LIFTorch' with shape (4, 4)
    }
    LinearTorch '3_LinearTorch' with shape (4, 2)
    LIFTorch '4_LIFTorch' with shape (2, 2)
}
[6]:
# - Scale down recurrent weights for stability
net[2][1].w_rec.data = net[2][1].w_rec / 10.

Step 2: Extract the computational graph for the network

To obtain a graph describing the entire network, which contains the computational flow of information through the network as well as all parameters, we simply use the as_graph() method.

For more information, see Computational graphs in Rockpool.

[7]:
print(net.as_graph())
GraphHolder "TorchSequential__140559315283664" with 2 input nodes -> 2 output nodes

Step 3: Map the network to a hardware specification

We now need to check that the network can be supported by the Xylo hardware, and assign hardware resources to the various aspects of the network architecture. For example, each neuron in the network must be assigned to a hardware neuron. Each non-input weight element in the network must be assigned to a global hidden weight matrix for Xylo. Output neurons in the final layer must be assigned to output channels.

[8]:
# - Import the Xylo HDK support packages for Rockpool
from rockpool.devices import xylo as x
from rockpool.devices.xylo import xylo_devkit_utils as xu
from rockpool.transform import quantize_methods as q

To convert the computational graph to a Xylo specfication, we use the xylo.mapper() function. In order to retain floating-point representations for parameters, we can specifiy weight_dtype = float. See the documentation for mapper() for further details.

[9]:
# - Call the Xylo mapper on the extracted computational graph
spec = x.mapper(net.as_graph(), weight_dtype = 'float')
[10]:
print(spec)
{
    'mapped_graph': GraphHolder "TorchSequential__140197188005328" with 2 input nodes -> 2
output nodes,
    'weights_in': array([[-0.99185878, -1.60321856,  1.41740334, -0.1252259 ,  0.        ,
         0.        ,  0.        ,  0.        ],
       [-0.55567956, -1.02018309, -0.04639089,  1.21827328,  0.        ,
         0.        ,  0.        ,  0.        ]]),
    'weights_out': array([[ 0.        ,  0.        ],
       [ 0.        ,  0.        ],
       [ 0.        ,  0.        ],
       [ 0.        ,  0.        ],
       [-0.51009029, -0.58346516],
       [-0.21264744, -0.4107787 ],
       [ 0.75371635,  1.20019042],
       [-1.04836726,  0.02810061]]),
    'weights_rec': array([[ 0.        ,  0.        ,  0.        ,  0.        ,  0.60704756,
        -0.75569618,  0.05267775, -0.85779089],
       [ 0.        ,  0.        ,  0.        ,  0.        ,  0.3372246 ,
         1.09767163,  0.8726486 ,  0.66629589],
       [ 0.        ,  0.        ,  0.        ,  0.        , -0.31703937,
        -0.26017576,  0.5676502 ,  1.02769387],
       [ 0.        ,  0.        ,  0.        ,  0.        ,  0.96591651,
         0.36234963,  0.66385448,  0.12330735],
       [ 0.        ,  0.        ,  0.        ,  0.        ,  0.03064639,
         0.09468726, -0.08120839,  0.04835235],
       [ 0.        ,  0.        ,  0.        ,  0.        , -0.03147189,
        -0.0057995 ,  0.10644694,  0.10671083],
       [ 0.        ,  0.        ,  0.        ,  0.        , -0.07627202,
         0.01339039,  0.0097085 ,  0.07502105],
       [ 0.        ,  0.        ,  0.        ,  0.        , -0.08816451,
        -0.00698344,  0.07585951,  0.04850018]]),
    'dash_mem': array([4, 4, 4, 4, 4, 4, 4, 4], dtype=uint8),
    'dash_mem_out': array([4, 4], dtype=uint8),
    'dash_syn': array([4, 4, 4, 4, 4, 4, 4, 4], dtype=uint8),
    'dash_syn_2': array([0, 0, 0, 0, 0, 0, 0, 0], dtype=uint8),
    'dash_syn_out': array([4, 4], dtype=uint8),
    'threshold': array([ 1,  1,  1,  1, 10, 10, 10, 10], dtype=int16),
    'threshold_out': array([1, 1], dtype=int16),
    'weight_shift_in': 0,
    'weight_shift_rec': 0,
    'weight_shift_out': 0,
    'aliases': [[4], [5], [6], [7], [], [], [], []],
    'dt': 0.001
}

Step 4: Quantize the specfication for the Xylo integer logic

Rockpool provides a number of functions for quantizing specifications for Xylo, in the package rockpool.transform.quantize_methods.

Here we will use global_quantize() to automatically find a good shared representation of the network parameters, that is compatible with the integer logic on Xylo.

[11]:
# - Quantize the specification
spec.update(q.global_quantize(**spec))
print(spec)
{
    'mapped_graph': GraphHolder "TorchSequential__140197188005328" with 2 input nodes -> 2
output nodes,
    'weights_in': array([[ -79, -127,  112,  -10,    0,    0,    0,    0],
       [ -44,  -81,   -4,   97,    0,    0,    0,    0]]),
    'weights_out': array([[   0,    0],
       [   0,    0],
       [   0,    0],
       [   0,    0],
       [ -54,  -62],
       [ -23,  -43],
       [  80,  127],
       [-111,    3]]),
    'weights_rec': array([[  0,   0,   0,   0,  48, -60,   4, -68],
       [  0,   0,   0,   0,  27,  87,  69,  53],
       [  0,   0,   0,   0, -25, -21,  45,  81],
       [  0,   0,   0,   0,  77,  29,  53,  10],
       [  0,   0,   0,   0,   2,   8,  -6,   4],
       [  0,   0,   0,   0,  -2,   0,   8,   8],
       [  0,   0,   0,   0,  -6,   1,   1,   6],
       [  0,   0,   0,   0,  -7,  -1,   6,   4]]),
    'dash_mem': array([4, 4, 4, 4, 4, 4, 4, 4], dtype=uint8),
    'dash_mem_out': array([4, 4], dtype=uint8),
    'dash_syn': array([4, 4, 4, 4, 4, 4, 4, 4], dtype=uint8),
    'dash_syn_2': array([0, 0, 0, 0, 0, 0, 0, 0], dtype=uint8),
    'dash_syn_out': array([4, 4], dtype=uint8),
    'threshold': array([ 79,  79,  79,  79, 792, 792, 792, 792]),
    'threshold_out': array([106, 106]),
    'weight_shift_in': 0,
    'weight_shift_rec': 0,
    'weight_shift_out': 0,
    'aliases': [[4], [5], [6], [7], [], [], [], []],
    'dt': 0.001
}

Step 5: Convert the specification to a hardware configuration

We now convert the network specification to a hardware configuration object for the Xylo HDK.

[12]:
# - Use rockpool.devices.xylo.config_from_specification
config, is_valid, msg = x.config_from_specification(**spec)
if not is_valid:
    print(msg)

Step 6: Connect to the Xylo HDK and deploy the configuration

[13]:
# - USe rockpool.devices.xylo.xylo_devkit_utils.find_xylo_boards
xylo_hdk_nodes = xu.find_xylo_boards()
print(xylo_hdk_nodes)
[<samna.xyloTestBoard.XyloTestBoard object at 0x7f82343ed2b0>]
[14]:
found_xylo = len(xylo_hdk_nodes) > 0

if not found_xylo:
    print('This tutorial requires a connected Xylo HDK to demonstrate.')
else:
    db = xylo_hdk_nodes[0]
[15]:
# - Use rockpool.devices.xylo.XyloSamna to deploy to the HDK
if found_xylo:
    modSamna = x.XyloSamna(db, config, dt = dt)
    print(modSamna)
XyloSamna  with shape (2, 8, 2)

Step 7: Evolve the network on the Xylo HDK

Now we will generate a random Poisson input, and evolve the network on the Xylo HDK using this input. Using Rockpool we can record all internal states on the Xylo HDK during evolution. Note that recording internal state slows evolution below real-time.

[16]:
# - Generate some Poisson input
T = 100
f = 0.1
input_spikes = np.random.rand(T, Nin) < f
TSEvent.from_raster(input_spikes, dt, name = 'Poisson input events').plot();
../../_images/tutorials_quick-xylo_deploy_to_xylo_29_0.png
[17]:
# - Evolve the network on the Xylo HDK
if found_xylo:
    out, _, r_d = modSamna(input_spikes, record = True)

    # - Show the internal state variables recorded
    print(r_d.keys())
dict_keys(['Vmem', 'Isyn', 'Isyn2', 'Spikes', 'Vmem_out', 'Isyn_out', 'times'])
[18]:
# - Plot some internal state variables
if found_xylo:
    plt.figure()
    plt.imshow(r_d['Spikes'].T, aspect = 'auto', origin = 'lower')
    plt.title('Hidden spikes')
    plt.ylabel('Channel')

    plt.figure()
    TSContinuous(r_d['times'], r_d['Vmem'], name = 'Hidden membrane potentials').plot(stagger = 127)

    plt.figure()
    TSContinuous(r_d['times'], r_d['Isyn'], name = 'Hidden synaptic currents').plot(stagger = 127)
../../_images/tutorials_quick-xylo_deploy_to_xylo_31_0.png
../../_images/tutorials_quick-xylo_deploy_to_xylo_31_1.png
../../_images/tutorials_quick-xylo_deploy_to_xylo_31_2.png

Step 8: Simulate the HDK using a bit-precise simulator

Rockpool comes bundled with XyloSim, a bit-precise simulator of the Xylo SNN core. The interface to evolve and record from a XyloSim object is identical to other Rockpool modules.

We create a XyloSim object using the same hardware configuration that we used to deploy the network to the Xylo HDK.

[19]:
# - Use rockpool.devices.xylo.XyloSim
modSim = x.XyloSim.from_config(config)
print(modSim)
XyloSim  with shape (16, 1000, 8)
[20]:
# - Evolve the input over the network, in simulation
out, _, r_d = modSim(input_spikes, record = True)

# - Show the internal state variables recorded
print(r_d.keys())
dict_keys(['Vmem', 'Isyn', 'Isyn2', 'Spikes', 'Vmem_out', 'Isyn_out'])
[21]:
# - Plot some internal state variables
plt.figure()
plt.imshow(r_d['Spikes'].T, aspect = 'auto', origin = 'lower')
plt.title('Hidden spikes')
plt.ylabel('Channel')

plt.figure()
TSContinuous.from_clocked(r_d['Vmem'], dt, name = 'Hidden membrane potentials').plot(stagger = 127)

plt.figure()
TSContinuous.from_clocked(r_d['Isyn'], dt, name = 'Hidden synaptic currents').plot(stagger = 127);
../../_images/tutorials_quick-xylo_deploy_to_xylo_36_0.png
../../_images/tutorials_quick-xylo_deploy_to_xylo_36_1.png
../../_images/tutorials_quick-xylo_deploy_to_xylo_36_2.png

If you compare the traces here with those recorded from the Xylo HDK above, you will see they are identical. XyloSim provides a way to quickly verify a network configuration without requiring the Xylo HDK hardware.

Summary

The flow-chart below summarises the steps in taking a Rockpool network from a high-level definition to deployment on the Xylo HDK.

[22]:
Image(filename='xylo-pipeline.png')
[22]:
../../_images/tutorials_quick-xylo_deploy_to_xylo_39_0.png