Hermes Gadget Released: Open SDK for Handheld AI Agent Hardware with ESP32 Firmware and Desktop Simulator

An open-source hardware SDK bridging physical handheld devices directly to Nous Research's autonomous Hermes agent has been released. Featuring push-to-talk voi

tau · October 5, 2026

#NousResearch #HermesAgent #ESP32 #OpenSDK #HardwareAI

Hermes Gadget Released: Open SDK for Handheld AI Agent Hardware with ESP32 Firmware and Desktop Simulator

A new open-source hardware development kit, 'Hermes Gadget', has been unveiled, bridging physical electronic devices directly to autonomous AI agent workflows. Developed by builder Adolan (@adolandev), the SDK allows developers to assemble compact handheld devices that interact with Nous Research's autonomous Hermes Agent via voice and visual displays, complete with a desktop simulator that eliminates the need for physical microcontroller hardware during initial development.

Hermes Gadget hardware prototype featuring a compact ESP32 embedded display and push-to-talk voice button

Image source: @adolandev / X

In contrast to proprietary commercial AI wearables and dedicated consumer gadgets that impose rigid recurring cloud subscriptions and closed vendor ecosystems, Hermes Gadget adopts a purely open, self-hosted architecture. By directly interfacing with a user-governed Hermes instance, the project has quickly gained traction among embedded hardware makers, hackers, and AI engineers.

Push-to-Talk Voice Interaction and Real-Time On-Screen Status

Hermes Gadget provides a tactile, voice-first physical interface designed for frictionless interaction with an autonomous agent:

  • Push-to-Talk (PTT) Audio Input: Users press and hold a single physical tactile button to ask a question or issue a directive. Releasing the button transmits the recorded audio payload directly to the Hermes backend.
  • Voice Response and Dynamic Status Display: Hermes processes the query, engages its reasoning loop, and answers aloud via synthesized speech. Simultaneously, the device's compact color screen displays what the agent is doing in real-time, detailing active tool invocations and execution steps.
  • On-Device Permission Prompts (Human-in-the-Loop): When an autonomous agent attempts sensitive operations—such as executing shell commands, modifying local files, or calling external APIs that require explicit user approval—the confirmation request appears directly on the device's screen. Users can grant or reject permissions physically without opening a smartphone application or switching terminal windows.

Unified Architecture: ESP32 Firmware and Desktop Simulator

A major practical strength of Hermes Gadget is its low barrier to entry for developers who do not yet own target hardware.

The repository includes a standalone desktop simulator that runs the exact same state machine and application logic as the target ESP32 firmware. Developers can test speech interactions, inspect display layouts, and verify agent communication loops within a virtual desktop environment, ensuring that code tested on a laptop will flash and execute identically onto physical microcontrollers.

Following its initial release, community members have reported building and deploying working physical prototypes in under an hour, porting the codebase to alternative development platforms such as the AIPI Lite, and adapting the architecture into browser-based web applications for immediate testing.

System Architecture and Implementation Considerations

Teams looking to build or integrate Hermes Gadget into their setups should keep several key architectural characteristics in mind:

  1. Networked Relay Rather Than On-Device Model Inference: The lightweight ESP32 microcontroller does not run large language models locally. Instead, the hardware acts as an interactive bridge—handling audio sampling, audio playback, display rendering, and network communications—while streaming commands over the network to a self-hosted Hermes Agent instance running on a local workstation, a $5 VPS, a GPU server, or via Nous Portal.
  2. Prerequisite Hermes Environment: Connecting a device requires an operational Hermes Agent deployment, typically configured in OpenAI-compatible API server mode (http://localhost:8642/v1) or connected through the Nous Portal gateway. Text-to-speech (TTS) engines and volume settings can be tailored directly within the agent configuration.
  3. Screen Size Adaptability in Active Development: While the initial reference build targets 2.0 to 2.1-inch embedded display modules, support for additional screen resolutions and hardware dimensions is actively being expanded based on community feedback.

Hermes Gadget represents a practical open-source blueprint for bringing autonomous, tool-using AI agents out of software terminals and into ambient physical devices.

Sources