AWS Builder Center
Build an MCP server in Rust with rmcp: a walk-through ๐Ÿฆ€

Build an MCP server in Rust with rmcp: a walk-through ๐Ÿฆ€

Step-by-step: scaffolding a real MCP server in Rust with the official rmcp SDK โ€” tools, JSON schemas, AWS calls, stdio transport, testing the protocol by hand, and wiring it into Claude Code

Making the Impossible - Possible!

๏ปฟ

๏ปฟ

This tutorial walks through building an MCP server in Rust with ย ย rmcpย ย , the official Model Context Protocol Rust SDK.
The example is a real one: a devops agent that manages AWS EC2 G5g instances โ€” Graviton2 boxes with NVIDIA T4G GPUs โ€” serving Gemma 4 under vLLM. It launches instances, drives them over SSM, and health-checks the model. There's an existing Python version, so at the end we can put the two side by side.
Follow along and you'll have a working, registerable MCP server. ๐Ÿฆ€

๏ปฟ

Why Rust for this?

Worth answering properly, because the weak version of the argument is easy to make and easy to demolish โ€” and the real one is better anyway.
Start with what it isn't: these tools are I/O bound. Every one is an AWS API call โ€” ย describe_instancesย , ย send_commandย , polling SSM โ€” so 100โ€“500 ms of network per call. The caller's language contributes nothing measurable there. Anyone selling you a Rust rewrite on raw speed for this workload is selling something.
Three claims that don't hold, so nobody has to make them in the comments:
Claim
Why it fails
"462 ms startup is slow"
stdio servers spawn once per session, not per call
"Rust is faster"
the work is network round-trips to AWS
"smaller supply chain"
241 crates vs 34 Python packages โ€” it's worse
What actually justifies it, for this codebase:
1. It's a fleet, not a server. This monorepo has 16 rigs, each with its own MCP server. That changes the units:
All loaded together
๐Ÿ Python
๐Ÿฆ€ Rust
Resident memory
16 ร— 83 MB โ‰ˆ 1.33 GB
16 ร— 12 MB โ‰ˆ 192 MB
Session startup
16 ร— 462 ms โ‰ˆ 7.4 s
16 ร— 2.5 ms โ‰ˆ 40 ms
A gigabyte of resident Python to expose sixteen tool lists is a real cost.
2. No shared interpreter. These rigs install system-wide โ€” no virtualenvs, by policy โ€” so all sixteen share one Python. Sixteen servers with independently drifting ย boto3ย  and ย mcpย  pins in one interpreter is a standing conflict risk. A static binary has no such coupling; each rig pins whatever it likes in its own ย Cargo.lockย .
3. The schema can't drift from the code. More on this at Step 3, but it's the one that survives longest: ย schemarsย  generates the tool schema from the same struct the handler destructures.
So: distribution and correctness, not speed. โœ… If you have one MCP server and it works, this is not a reason to rewrite it.

๏ปฟ

How does this all fit together?

Two halves. The agent and the MCP server run on your machine; the GPU box is remote, and it has no inbound SSH โ€” everything goes through the AWS APIs.
YOUR MACHINE AWS us-east-1
โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
โ”‚ โ”‚ โ”‚ โ”‚
โ”‚ Claude Code / IDE โ”‚ โ”‚ โ”Œโ”€ EC2 g5g.4xlarge โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
โ”‚ | โ”‚ โ”‚ โ”‚ Graviton2 (aarch64) โ”‚ โ”‚
โ”‚ | MCP ยท JSON-RPC 2.0 โ”‚ โ”‚ โ”‚ + NVIDIA T4G (SM 7.5) โ”‚ โ”‚
โ”‚ | over stdio โ”‚ โ”‚ โ”‚ โ”‚ โ”‚
โ”‚ v โ”‚ EC2 โ”‚ โ”‚ [PY] vLLM + [RUST] vllm-rs โ”‚ โ”‚
โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ API โ”‚ โ”‚ listening on :8000 โ”‚ โ”‚
โ”‚ โ”‚ [RUST] โ”‚โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€>โ”‚ โ”‚ โ”‚ โ”‚
โ”‚ โ”‚ gpu-vllm-g5g-2b โ”‚ โ”‚ โ”‚ โ”‚ Gemma 4 E2B โ”‚ โ”‚
โ”‚ โ”‚ โ”‚ โ”‚ SSM โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
โ”‚ โ”‚ rmcp 3.1.2 โ”‚โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€>โ”‚ ^ โ”‚
โ”‚ โ”‚ tokio ยท schemars โ”‚ โ”‚ Run โ”‚ | no inbound SSH, โ”‚
โ”‚ โ”‚ aws-sdk-ec2 / -ssm โ”‚ โ”‚ Cmd โ”‚ | no key pair, โ”‚
โ”‚ โ”‚ 1 binary ยท 2.5 ms โ”‚ โ”‚ โ”‚ | no port 22 rule โ”‚
โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ โ”‚ โ”‚
โ”‚ 9 tools โ”‚ โ”‚ IAM instance profile carries โ”‚
โ”‚ list / start / stop / โ”‚ โ”‚ AmazonSSMManagedInstanceCore โ”‚
โ”‚ terminate / endpoint / โ”‚ โ”‚ โ”‚
โ”‚ run_remote / health ... โ”‚ โ”‚ โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
The agent never talks to the GPU box directly. It calls a tool; the tool calls EC2 to manage the instance's lifecycle, or SSM Run Command to execute something on it. That's what lets the box run with no inbound rules at all โ€” which is the main reason this is worth building as a server rather than a pile of shell scripts.
The ย [RUST]ย  on the right-hand side is vLLM's own Rust frontend โ€” the other article in this series. This one is the ย [RUST]ย  on the left: the Rust that drives the box.

๏ปฟ

What is MCP, in one paragraph?

Model Context Protocol is how an AI agent discovers and calls your tools. Your server advertises a list of tools with JSON Schemas; the client (Claude Code, an IDE, whatever) calls them over JSON-RPC 2.0. Transport is usually stdio โ€” the client spawns your binary and talks over stdin/stdout.
That last detail matters for the Rust pitch: if the client spawns your process on every session, process startup is a user-visible cost.

๏ปฟ

Step 1 โ€” Scaffold

cargo new --bin rust-mcp --name gpu-vllm-g5g-2b-mcp
cd rust-mcp
Now the dependencies. Feature flags are the thing to get right here โ€” ย cargo add rmcpย  on its own compiles fine and gives you almost nothing:
cargo add rmcp --features server,macros,transport-io
Feature
What it brings
ย serverย 
the ย ServerHandlerย  trait and router types
ย macrosย 
ย #[tool]ย , ย #[tool_router]ย , ย #[tool_handler]ย 
ย transport-ioย 
stdio transport
The crate also ships ย clientย , ย authย , ย elicitationย , ย transport-streamable-http-serverย  and more, all off by default. Add them when you need them.
Then the rest:
cargo add tokio --features rt-multi-thread,macros,process,time
cargo add serde serde_json anyhow schemars
cargo add aws-config aws-sdk-ec2 aws-sdk-ssm aws-sdk-secretsmanager
Resulting ย Cargo.tomlย :
[package]
name = "gpu-vllm-g5g-2b-mcp"
version = "0.1.0"
edition = "2024"
๏ปฟ
[dependencies]
rmcp = { version = "3.1.2", features = ["server", "macros", "transport-io"] }
tokio = { version = "1.53.1", features = ["rt-multi-thread", "macros", "process", "time"] }
aws-config = "1.10.1"
aws-sdk-ec2 = "1.246.0"
aws-sdk-ssm = "1.118.0"
serde = "1.0.229"
serde_json = "1.0.151"
schemars = "1.2.2"
anyhow = "1.0.104"

๏ปฟ

๐Ÿ”Ž Tip: where the canonical examples live

ย rmcpย  moves fast, and rendered docs lag. The vendored tests on your own disk are compiled against the exact version you resolved:
ls ~/.cargo/registry/src/*/rmcp-3.1.2/tests/
ย tests/test_tool_macros.rsย  is a complete, working server in about 60 lines. When an API question comes up, that file answers it faster and more reliably than anything else. โšก

๏ปฟ

Step 2 โ€” The server struct

An rmcp server is a struct that owns a ย ToolRouterย :
use rmcp::{
ErrorData, ServerHandler, ServiceExt,
handler::server::{router::tool::ToolRouter, wrapper::Parameters},
model::{CallToolResult, ContentBlock, Implementation, ServerCapabilities, ServerInfo},
tool, tool_handler, tool_router,
transport::stdio,
};
use schemars::JsonSchema;
use serde::{Deserialize, Serialize};
๏ปฟ
#[derive(Clone)]
struct G5gServer {
tool_router: ToolRouter<Self>,
}

๏ปฟ

Step 3 โ€” Describe your inputs as types

This is the part that sold me on the whole exercise. Your tool's input is a plain struct, and ย schemarsย  turns it into the JSON Schema the agent sees โ€” doc comments and all:
#[derive(Debug, Serialize, Deserialize, JsonSchema)]
struct InstanceId {
/// EC2 instance id, e.g. `i-0123456789abcdef0`.
instance_id: String,
}
That doc comment becomes the field's ย descriptionย  in the tool schema. Rename the field and the schema follows. The compiler checks the type your handler destructures. There is no second artifact to keep in sync. โœ…

๏ปฟ

Step 4 โ€” Write the tools

ย #[tool_router]ย  on the impl block, ย #[tool]ย  on each method:
#[tool_router(router = tool_router)]
impl G5gServer {
fn new() -> Self {
Self { tool_router: Self::tool_router() }
}
๏ปฟ
#[tool(description = "List EC2 instances tagged ManagedBy=gpu-vllm-g5g-2b.")]
async fn list_g5g_instances(&self) -> Result<CallToolResult, ErrorData> {
let conf = aws_config::defaults(aws_config::BehaviorVersion::latest())
.region(aws_config::Region::new("us-east-1"))
.load()
.await;
let ec2 = aws_sdk_ec2::Client::new(&conf);
๏ปฟ
let resp = match ec2.describe_instances()
.filters(Filter::builder()
.name("tag:ManagedBy").values("gpu-vllm-g5g-2b").build())
.send().await
{
Ok(r) => r,
Err(e) => return ok(format!("โŒ describe_instances failed: {e}")),
};
๏ปฟ
let mut rows = Vec::new();
for res in resp.reservations() {
for inst in res.instances() {
rows.push(format!("| `{}` | {} | {} |",
inst.instance_id().unwrap_or("?"),
inst.instance_type().map(|t| t.as_str()).unwrap_or("?"),
inst.state().and_then(|s| s.name())
.map(|n| n.as_str()).unwrap_or("unknown"),
));
}
}
ok(format!("๐Ÿ“ก Instances\n\n| id | type | state |\n|---|---|---|\n{}",
rows.join("\n")))
}
}
Tools that take arguments wrap them in ย Parameters<T>ย :
#[tool(description = "Terminate a G5g instance. Permanent โ€” destroys the root volume.")]
async fn terminate_g5g_instance(
&self,
Parameters(args): Parameters<InstanceId>,
) -> Result<CallToolResult, ErrorData> {
// โ€ฆ
}
And a small helper, since every tool returns the same shape:
fn ok(text: String) -> Result<CallToolResult, ErrorData> {
Ok(CallToolResult::success(vec![ContentBlock::text(text)]))
}

๏ปฟ

Step 5 โ€” Implement ServerHandler

ย #[tool_handler]ย  wires the router in, so you never write a dispatch ย matchย :
#[tool_handler(router = self.tool_router)]
impl ServerHandler for G5gServer {
fn get_info(&self) -> ServerInfo {
let mut info = ServerInfo::new(
ServerCapabilities::builder().enable_tools().build()
);
info.server_info = Implementation::new(
"gpu-vllm-g5g-2b", env!("CARGO_PKG_VERSION")
);
info.instructions = Some(
"Devops agent for AWS EC2 G5g (Graviton2 + NVIDIA T4G) serving Gemma 4 \
under vLLM. Remote administration goes through SSM; there is no inbound SSH."
.to_string(),
);
info
}
}
๐Ÿ’ก These model structs are ย #[non_exhaustive]ย , so use the constructors (ย ServerInfo::newย , ย Implementation::newย ) and then assign fields โ€” a struct literal won't compile, even with ย ..Default::default()ย .

๏ปฟ

Step 6 โ€” main

Four lines:
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let service = G5gServer::new().serve(stdio()).await?;
service.waiting().await?;
Ok(())
}
cargo build --release

๏ปฟ

Step 7 โ€” Test the protocol by hand

An MCP server is a protocol implementation, so test it with a protocol transcript. Three JSON-RPC lines on stdin โ€” no client required:
printf '%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2026-07-28","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' \
'{"jsonrpc":"2.0","method":"notifications/initialized"}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
'{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"get_help","arguments":{}}}' \
| ./target/release/gpu-vllm-g5g-2b-mcp
initialize OK: gpu-vllm-g5g-2b 0.1.0 proto 2026-07-28
tools/list OK: 9 tools -> get_deployment_config, get_endpoint, get_help,
list_g5g_instances, run_remote, start_g5g_instance, stop_g5g_instance,
terminate_g5g_instance, verify_model_health
tools/call get_help OK
๐ŸŸข rmcp 3.1.2 negotiates the 2026-07-28 spec version by default.
Keep this snippet. It's the fastest way to tell "my server is broken" from "my client config is broken."

๏ปฟ

Step 8 โ€” Register it

Point your MCP client at the binary. For Claude Code, ย .mcp.jsonย :
{
"mcpServers": {
"gpu-vllm-g5g-2b": {
"command": "/abs/path/to/rust-mcp/target/release/gpu-vllm-g5g-2b-mcp",
"env": { "AWS_REGION": "us-east-1" }
}
}
}
The server name prefixes every tool โ€” ย mcp__gpu-vllm-g5g-2b__list_g5g_instancesย  โ€” so name it after the thing it manages, especially if you run several.
Credentials come from the standard AWS provider chain, so whatever ย aws sts get-caller-identityย  resolves is what the server gets. Set ย AWS_PROFILEย  to pick one.

๏ปฟ

Tests worth writing

The interesting assertions aren't about the code, they're about the machine. Turing has no bf16 datapath and no fp8, so the serving flags must differ from every L4-class box โ€” exactly the sort of thing that silently reverts when someone copies a flag set from a neighbour:
#[test]
fn serve_flags_are_turing_shaped() {
let f = serve_flags("google/gemma-4-E2B-it", "g5g.2xlarge");
assert!(f.contains("--dtype float16"), "Turing has no bf16 datapath");
assert!(f.contains("--kv-cache-dtype auto"), "Turing has no fp8 datapath");
assert!(!f.contains("attention-backend")); // not a real vLLM v0.27 variable
}
๏ปฟ
#[test]
fn unknown_types_are_rejected_and_never_need_swap() {
assert!(validate_instance_type("t4g.2xlarge").is_err()); // burstable CPU box, no GPU
assert!(!needs_swap("t4g.2xlarge")); // 0 GiB must not read as "tiny"
}
That second one earns its keep: ย host_memory_gbย  returns ย 0ย  for an unknown instance type, and a naive ย ram < 16ย  would decide an unrecognised machine needs a swapfile.
running 5 tests
test result: ok. 5 passed; 0 failed; finished in 0.00s

๏ปฟ

๐Ÿ vs ๐Ÿฆ€ โ€” the scoreboard

Cold start measured the way a client experiences it: spawn the process, send ย initializeย  + ย initializedย  + ย tools/listย , stop the clock when the tool list comes back. Seven runs, median.
๏ปฟ
๐Ÿ Python (FastMCP)
๐Ÿฆ€ Rust (rmcp)
Cold start to ย tools/listย 
462.0 ms (437โ€“530)
2.5 ms (1.8โ€“2.9)
Peak RSS
83 MB
12 MB
Artifact
Python runtime + 34 packages
one binary, 39.4 MB (19.9 MB stripped)
Direct dependencies
3
14
Total resolved packages
34
241
Tools implemented
15
9
Source
759 lines
560 + 52 of tests
Clean release build
n/a
5 m 28 s
185x on cold start โ€” but per the section up top, resist quoting that on its own. One server spawned once a session, 460 ms, nobody notices. It only becomes a number worth having when you multiply it by sixteen rigs, and even then it's the 12 MB vs 83 MB row that does the heavier lifting.
Read the rest honestly too. 241 resolved packages against 34 means the static binary is not a smaller supply chain, just the same one audited in ย Cargo.lockย . The port covers 9 tools to Python's 15 โ€” the provisioning path (cloud-init rendering, AMI resolution, spot options) is the fiddly half and isn't ported. And 5 m 28 s of clean build against an interpreter that starts instantly is a real cost while you're iterating. ๐Ÿ“Š

๏ปฟ

So, worth it?

For a single MCP server that already works: no. Don't rewrite it.
For sixteen of them sharing one system Python, shipped to machines that shouldn't need a Python environment at all: yes โ€” and note that neither half of that sentence is about speed. It's a packaging answer.
The part that'll still be true next year is ย schemarsย . The tool schema the agent sees is generated from the same struct the handler destructures, checked by the compiler, documented by the doc comments on its fields. In the Python version the schema, the runtime types and the docs are three artifacts that agree by convention โ€” and go quiet when they stop agreeing.
The 462 ms is a bonus. The schema not being able to lie about the code is the reason. โœ…

๏ปฟ

Cheat sheet

# scaffold
cargo new --bin my-mcp && cd my-mcp
cargo add rmcp --features server,macros,transport-io
cargo add tokio --features rt-multi-thread,macros
cargo add serde serde_json schemars anyhow
๏ปฟ
# canonical examples for YOUR resolved version
ls ~/.cargo/registry/src/*/rmcp-*/tests/test_tool_macros.rs
๏ปฟ
# build + smoke test
cargo build --release
printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2026-07-28","capabilities":{},"clientInfo":{"name":"p","version":"0"}}}' \
'{"jsonrpc":"2.0","method":"notifications/initialized"}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
| ./target/release/my-mcp
Four macros to remember: ย #[tool_router]ย  on the impl, ย #[tool]ย  on each method, ย #[tool_handler]ย  on the ย ServerHandlerย  impl, ย Parameters<T>ย  around your input struct.

๏ปฟ
Rust 1.97.1, rmcp 3.1.2, aws-sdk-ec2 1.246.0, edition 2024. Startup measured on the dev host โ€” it's a comparison of two MCP servers, not a hardware result. Single machine, seven runs per side, median reported.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article