Harvesting the computation of Elementary Cellular Automata With Reservoir Computing

Abstract

This PhD thesis focuses on Reservoir Computing (RC) and Cellular Automata (CA), with an emphasis on the simplest form of CA, namely Elementary CA (ECA) and the combination of RC with CA (ReCA). Biologic and modern Artificial Intelligence (AI) have stark differences regarding specific capacities and energy intensity. Therefore, it is clear that computer science and AI still have much to learn from biology. RC and CA both have roots and are inspired by biology. It is theorized that RC is used in certain parts of the brain, and CA is used as a simple model for biological processes. More importantly, both rely on a locality feature; RC operates its training only on the output layer, and CA relies on local interactions, meaning neither requires the same level of global control as other algorithms. This lack of global control can be translated into energy efficiency at the cost of some performance. For RC, the reservoir is only dynamically set up but not tuned, which means that the output layer is the only trained component; this leads to energy-efficient training. For CA, the simple and local interactions mean that a translation into hardware is simple, affording energy-efficient inference. Therefore, ReCA is a viable option for energy-efficient AI such as EdgeAI. ReCA still needs to be well established, and this thesis aims to explore the behaviour and characteristics of ReCA. It aims to focus on the hyper-parameter space and the individual ECA rules. It compares it to similar substrates and identifies implementation challenges and best practices. The main findings are that the hyper-parameter space is highly dynamic and requires careful balance between them. When RC with ECA is compared to similar substrates, both the critical range and the number of behaviours are larger in ECA. We also show evidence that ReCA works better on more locally dependent benchmarks, such as MNIST, in contrast to the UCR archive. The findings highlight specific practical engineering challenges with deploying ReCA in its current stage. However, the overall results are promising and show much potential for progress.