Lateral Interactions Spiking Actor Network for Reinforcement Learning

EasyChair Preprint 11103

Abstract

Spiking neural network (SNN) has been shown to be a biologically plausible and energy efficient alternative to Deep Neural Network (DNN) in Reinforcement Learning (RL). In the prevailing SNN models for RL, fully-connected architectures with inter-layer connections are commonly employed, while the incorporation of intra-layer connections is neglected, thereby impeding the feature representation and information processing capacities of SNN in the context of reinforcement learning. To address these limitations, we propose a high-performance Lateral Interactions Spiking Actor Network (LISAN) to improve decision-making in reinforcement learning tasks. Our LISAN integrates lateral interactions between neighboring neurons into the spiking neuron membrane potential equation. Moreover, recognizing the significance of residual potentials in preserving valuable information within biological neurons, we incorporate soft reset mechanism to enhance model's functionality. To verify the effectiveness of our proposed framework, LISAN is evaluated using four continuous control tasks from OpenAI gym as well as different encoding methods. The results show that LISAN substantially achieves better performance compared to state-of-the-art models. We hope that our work will contribute to a deeper understanding of the mechanisms involved in information capturing and processing within the brain.

Keyphrases: Reinforcement Learning, Spiking Neural Networks, lateral interactions

@booklet{EasyChair:11103,
  author    = {Xiangyu Chen and Rong Xiao and Qirui Yang and Jiancheng Lv},
  title     = {Lateral Interactions Spiking Actor Network for Reinforcement Learning},
  howpublished = {EasyChair Preprint 11103},
  year      = {EasyChair, 2023}}