Hierarchical Control and Learning of a Foraging CyberOctopus

Chia Hsien Shih, Noel Naughton, Udit Halder, Heng Sheng Chang, Seung Hyun Kim, Rhanor Gillette, Prashant G. Mehta, Mattia Gazzola

Research output: Contribution to journalArticlepeer-review


Inspired by the unique neurophysiology of the octopus, a hierarchical framework is proposed that simplifies the coordination of multiple soft arms by decomposing control into high-level decision-making, low-level motor activation, and local reflexive behaviors via sensory feedback. When evaluated in the illustrative problem of a model octopus foraging for food, this hierarchical decomposition results in significant improvements relative to end-to-end methods. Performance is achieved through a mixed-modes approach, whereby qualitatively different tasks are addressed via complementary control schemes. Herein, model-free reinforcement learning is employed for high-level decision-making, while model-based energy shaping takes care of arm-level motor execution. To render the pairing computationally tenable, a novel neural network energy shaping (NN-ES) controller is developed, achieving accurate motions with time-to-solutions 200 times faster than previous attempts. The hierarchical framework is then successfully deployed in increasingly challenging foraging scenarios, including an arena littered with obstacles in 3D space, demonstrating the viability of the approach.

Original languageEnglish (US)
Article number2300088
JournalAdvanced Intelligent Systems
Issue number9
StatePublished - Sep 2023


  • bioinspiration
  • hierarchical control
  • soft robotics

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Mechanical Engineering
  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Materials Science (miscellaneous)


Dive into the research topics of 'Hierarchical Control and Learning of a Foraging CyberOctopus'. Together they form a unique fingerprint.

Cite this