Enrico Santagati

Closing the loop between software, hardware, and AI.

Let the AI Out

AI agents live in text windows. Everything they know comes from what someone typed, pasted, or piped in. To be useful outside the window, they need direct access to the physical world: interfaces they can control, and streams they can perceive.

This series is about giving them that access, one MCP server at a time.

Two threads so far:

Control interfaces. Started with BLE — general device control, any consumer or embedded target the agent needs to poke. Progressed to serial (console + logs) and debug probes (execution state, memory, breakpoints). Along the way this thread grew into a hardware developer toolkit: closing the code → test → deploy → observe loop that breaks when firmware is on a device the agent can’t see.

Perception interfaces. Starts with a microphone: continuous audio in, event-gated recognition out, memory that grows through use. Naturally extendable to other perception protocols (I2S, sensor arrays, vision) as the pattern generalizes.

The goal across both is the same: narrow the gap between “AI that can reason about the physical world” and “AI that can actually interact with it.”

The series

  1. What Is MCP? — The protocol that makes all of this work. What MCP is, where it came from, and why it matters for hardware.

  2. BLE MCP Server — Giving the agent access to Bluetooth Low Energy. Scan, connect, read characteristics, subscribe to notifications. The poke→spec→plugin development arc.

  3. Serial MCP Server — Giving the agent a seat at the serial console. Boot banners, CLI commands, debug logs, PTY mirroring, control lines.

  4. BLE + Serial Demo — Both interfaces at once against the same device. The agent correlates the product surface (BLE) with the debug surface (serial) to find and recover from a firmware bug.

  5. Debug Probe MCP Server — The deepest level of access. Halt the CPU, set breakpoints, inspect registers and memory, and flash firmware — giving the agent direct access to execution state, not just logs or external behavior.

  6. Same Hardware Tools, Every AI Agent — The same MCP servers working across Claude Code, VS Code + Copilot, and Cursor. Write the server once, any agent can use it.

  7. Edge AI on a Microcontroller — Deploying a TFLite Micro keyword spotting model on an nRF52840 in 90 minutes from a single terminal session. Two custom debug probe plugins turn raw memory access into an edge-AI development environment.

  8. Agents as a Control Layer — From presentation layer to control layer. AI agents that don’t just answer questions about hardware but watch it, reason about it, and act autonomously. Two approaches tested: a hand-architected agent system and a multi-agent platform that hired its own team from a single prompt.

  9. Audient: An Ambient Audio Perception Layer — First perception post in the series. Continuous audio in, event-gated recognition out, a concept memory that grows through use. Where the earlier posts gave the agent hands, this one gives it ears.

Repos

Taken together, these interfaces form an AI-native hardware control plane — a standardized surface between reasoning and physical execution. audient extends the same pattern into perception: the agent doesn’t just act on the world, it can listen to it.