How to Build a Minecraft Voice Bot: The mc-sidecar Wire Protocol

Peranima's Minecraft integration is a small, boring, reliable idea: a sidecar process that speaks newline-delimited JSON over stdin/stdout. The voice side of the product turns what you say into tool calls; the sidecar turns tool calls into in-game actions through a headless Java Edition client and reports back what happened. This page documents that wire protocol — not because you need it to use Peranima (you don't, it's invisible), but because we kept getting the question "how would I build a Minecraft voice bot?" and the honest answer is: the protocol is the easy part, and here it is.

Everything below is the real shipping format. It works with Minecraft Java Edition 1.21.x over the built-in "Open to LAN" feature — no mod, no plugin, no server-side install.

Why JSON Lines over stdio

The sidecar is a child process of the desktop app. Three properties made stdio + JSON Lines win over a local socket or HTTP server:

• No port, no firewall dialog. A local HTTP server means Windows Firewall prompts and port collisions. stdin/stdout just works.
• Lifecycle for free. The parent owns the process. If the app dies, the bot dies — no orphaned bots haunting your world.
• One line = one message. Trivial to parse, trivial to log, trivial to replay when debugging. Each message is a single JSON object terminated by \n.

One hard rule: stdout is protocol-only. Anything human-readable (debug logs, timings) goes to stderr. The first time a stray console.log lands on stdout, your parser eats garbage — reserve the channel from day one.

Requests: parent → sidecar

Every request is one JSON object on one line of stdin:

{"id":"call_123","tool":"mc_join","args":{"port":51234,"username":"Theo"}}

Lines that aren't valid JSON, or that are missing id or tool, are logged to stderr and silently dropped — there's no response to correlate them to. Blank lines are ignored. Requests are not serialized: if you send two before the first finishes, both run. Operations that must be exclusive (like connecting) guard themselves and fail fast with a descriptive error (e.g. join_in_progress) instead of corrupting state.

Responses: sidecar → parent

Exactly one response per accepted request, on stdout, in either of two shapes:

{"id":"call_123","ok":true,"result":{"joined":true,"position":{"x":120,"y":64,"z":-33},"gameMode":"survival","version":"1.21.4"}}
{"id":"call_123","ok":false,"error":"invalid or missing port — run mc_discover_lan first, or ask the user for the port from Minecraft chat"}

The error model is deliberately flat: tool implementations throw, the dispatcher catches, and the message becomes error. Two conventions here earned their keep in production:

• Errors are instructions, not codes. The consumer of these messages is a language model planning its next move. E_INVALID_PORT teaches it nothing; "run mc_discover_lan first, or ask the user for the port" is a recovery plan it will actually follow.
• Put the verdict in the first keys. Logs and LLM contexts truncate. Results lead with the fields that matter (found, reason) so a 200-character preview still tells the whole story.

Events: unsolicited pushes

Request/response only gets you a bot that answers. For a companion that reacts — "you're on four hearts, get out of there!" — the sidecar pushes events on stdout, interleaved with responses. They're distinguishable by shape: events have type and no id.

{"type":"event","name":"mc_low_health","payload":{"health":4,"max_health":20}}
{"type":"event","name":"mc_player_chat","payload":{"username":"Eduard","message":"follow me"}}

The debouncing is the actual hard-won part. Game events fire at engine frequency; an LLM reacts at sentence frequency. Without per-key debounce (keyed by event name, or name + sub-key like attacker type), one zombie fight floods the model with thirty identical "you're being attacked" events and drowns out the one that's new. Filter at the source: emit the change, not the state.

Worked example 1: find the world, join it

The flow that kills the "what's your port number?" question. The player opens their single-player world via Open to LAN; Minecraft announces it on the local network; the sidecar listens.

→ {"id":"c1","tool":"mc_discover_lan","args":{}}
← {"id":"c1","ok":true,"result":{
     "found": 1,
     "worlds": [{"host":"127.0.0.1","port":51234,"motd":"Eduard - Survival","is_local_machine":true}],
     "next_step": "Exactly one world with is_local_machine=true: call mc_join with its {port, host} immediately without asking."
   }}

→ {"id":"c2","tool":"mc_join","args":{"host":"127.0.0.1","port":51234,"username":"Theo"}}
← {"id":"c2","ok":true,"result":{"joined":true,"position":{"x":120,"y":64,"z":-33},"gameMode":"survival","version":"1.21.4"}}

Note the next_step field. Discovery never throws — every outcome (found one world, found none, discovery blocked by a firewall) returns ok:true with a reason and a next_step sentence telling the planner exactly how to proceed, including when to fall back to asking the human for the port. Treat your tool results as prompts; that's what they are.

Worked example 2: a tool chain

"Get us some wood and make a crafting table" becomes three calls, each gated on the previous result:

→ {"id":"c3","tool":"mc_collect","args":{"block_type":"oak_log","count":4}}
← {"id":"c3","ok":true,"result":{"collected":4}}

→ {"id":"c4","tool":"mc_craft","args":{"item":"crafting_table","count":1}}
← {"id":"c4","ok":true,"result":{"crafted":true,"count":1}}

→ {"id":"c5","tool":"mc_say","args":{"message":"Crafting table done — where do you want it?"}}
← {"id":"c5","ok":true,"result":{"said":true}}

One subtlety worth copying: interrupt semantics. A stop command (mc_stop) sets an interrupt flag that long-running skills poll so they can abort mid-loop. The bug you will write (we did): the flag stays set, and every subsequent action aborts instantly — the bot looks dead after a single "stop". The fix is structural: every new non-stop tool call clears the flag first. Stop means "stop what you're doing now", not "stop forever".

The tool surface

The shipping sidecar dispatches 49 tools. The exact list will grow; the categories are stable:

Honest constraints

• Java Edition 1.21.x only (up to 1.21.11). Headless-client protocol support always trails the newest game version — when a player's launcher is ahead, the most useful thing your error message can do is tell them, in plain words, how to create a 1.21.x installation. Ours does.
• LAN means offline-mode auth. The bot joins the locally announced world as an offline player. Public online-mode servers are a different problem this protocol doesn't solve.
• The protocol is not the latency. A round-trip through this sidecar is milliseconds. In a full voice loop (speech in → language model → speech out), Peranima's replies land at roughly 3 seconds median — the thinking, not the plumbing, is the cost. If you build your own, budget accordingly.

If you're building one

Steal freely: JSON Lines over stdio, instructive error strings, next_step hints in results, verdict-first key ordering, per-key event debounce, and interrupt flags that self-clear. None of it is clever. All of it is the difference between a demo and something a non-technical player can use.