AWS Builder Center
Which LLM API is the Fastest? iamspeed.dev Now Has Race Mode, 10+ Providers, and a Community Leaderboard

Which LLM API is the Fastest? iamspeed.dev Now Has Race Mode, 10+ Providers, and a Community Leaderboard

iamspeed.dev has been updated with Race Mode, support for more than ten LLM API providers, and a community leaderboard. Race Mode allows users to run parallel benchmark tests across two or three providers simultaneously, comparing results by total generation time and tokens per second.

Full Stack Developer @Salesforce @MuleSoft | Building JMeter.AI | IAmSpeed.dev | ai.dosa.dev
A few months ago I shipped iamspeed.dev as a fast.com for LLM APIs. Open the browser, configure your API key, hit Run, get a real tokens-per-second number. No backend, no data collection, no noise.
That version did one thing well: it told you how fast a single provider was on a single run.
But developers kept asking the same follow-up question: “Okay, but how does it compare to the others?”
That question pushed me to build the next version. iamspeed.dev now has Race Mode, 10+ provider integrations including xAI and z.ai, and a community leaderboard backed by Amazon DynamoDB. Here is what changed and why.

What is New in iamspeed.dev

Race Mode: Watch LLMs Compete in Real Time

Race Mode is the feature I am most excited about. You pick two or three providers, give them the same prompt, and watch them race side by side on a live racetrack.
Each provider gets its own lane. As tokens stream in, the cars move. The race ends when the last provider finishes generating. Winner is determined by TTLT, which is the total time to generate the complete response, with tokens per second as the tiebreaker. A separate “Fastest Start” award goes to whoever had the lowest TTFT.
Under the hood, Race Mode fires 2 to 3 parallel streaming requests simultaneously, one per provider, each using your own API key. Nothing is proxied through a server. The browser handles all three streams directly.
This is not a toy. If you are trying to choose between Groq, Cerebras, and OpenAI for a production use case, you can race them on your actual prompt and watch the answer in about 15 seconds.

10+ Providers Now Supported

iamspeed.dev now supports:
  • OpenAI (GPT-4o, GPT-4.1, o-series)
  • Anthropic (Claude Sonnet, Claude Haiku, Claude Opus)
  • Groq (Llama 3, Mixtral, Gemma)
  • Cerebras (one of the fastest inference endpoints available today)
  • Fireworks AI
  • Mistral
  • OpenRouter (access to dozens of models through one key)
  • Google Gemini
  • xAI (Grok)
  • z.ai
  • Local models via Ollama and LM Studio
Adding a new provider is still just three files: a small adapter in  src/lib/providers/ , a registration entry in  src/lib/providers/index.ts , and metadata in  src/lib/config.ts . Most OpenAI-compatible endpoints reuse a single generic adapter and take about 10 lines of config to wire up.
One thing worth calling out: Cerebras numbers are genuinely surprising. If you have not benchmarked it yet, run it against GPT-4o in Race Mode. The tokens-per-second gap is large enough to change your architectural decisions.

Full Feature List

  • Live streaming output with real-time metric updates
  • TTFT, TPS, and TTLT measured precisely during the stream
  • AES-GCM encrypted API key storage, local only, never transmitted
  • Race Mode with animated Cars-themed racetrack, up to 3 providers in parallel
  • Speed Grade verdict on every result (Kachow!, Ludicrous Speed, Warp Drive, Cruising, Rush Hour)
  • Community Leaderboard backed by Amazon DynamoDB with provider filtering and sorting
  • Leaderboard stats bar showing total submissions, average TPS, and fastest record
  • Shareable image cards generated client-side via Canvas, with native share on mobile
  • Social sharing to X, LinkedIn, Reddit, and Facebook
  • Run history drawer with sparkline TPS trend chart
  • Local model support via Ollama and LM Studio

Speed Grade: Your Result Gets a Verdict

Every benchmark result now gets a Speed Grade based on tokens per second. The tiers are calibrated to real-world provider performance:
Grade
TPS
Emoji
Kachow!
500+
??
Ludicrous Speed
200-499
?
Warp Drive
100-199
?
Cruising
50-99
?
Rush Hour
under 50
?
The grade shows up under your hero TPS number in Simple Mode and on the Race Mode podium. It also appears in the leaderboard stats bar next to the fastest recorded submission. Cerebras and Groq hit Kachow! regularly. Most frontier models land in Warp Drive or Ludicrous Speed.

Shareable Image Cards

After a benchmark completes, you can generate a branded PNG image card directly in the browser using the Canvas API. The card includes your provider, model, TPS, TTFT, and a generated handle derived from your device fingerprint.
On mobile, clicking the image button triggers  navigator.share({ files })  which opens the native share sheet so you can drop it straight into a message or post. On desktop it downloads as a PNG.
No server involved. The card is generated entirely client-side and never touches our infrastructure.

Run History with Sparkline

The history panel tracks your recent benchmarks in a slide-out drawer. It shows a table of model, TPS, TTFT, and total time for each run, and if you have two or more runs, it renders a sparkline chart of your TPS trend over time. Useful for spotting variance across runs or comparing models back to back without resetting.

Community Leaderboard Powered by Amazon DynamoDB

This is the biggest structural change. iamspeed.dev now has a backend.
After you run a benchmark, you can submit your result to the community leaderboard with a nickname. The leaderboard shows the top results globally, filterable by provider, and ranks by tokens per second. You can also see your own past submissions.
The leaderboard is backed by Amazon DynamoDB. Here is why that was the right database choice for this use case.
DynamoDB is HTTP-based, which means Vercel serverless functions can talk to it directly with no connection pool, no VPC, and no cold-start penalty from a connection being unavailable. Every relational database I considered would have introduced friction here. DynamoDB just works.
The schema uses three Global Secondary Indexes:
  •  by-tps : partition key is the string  "SIMPLE" , sort key is TPS descending. This is how the global top-N leaderboard query works. It hits one GSI partition and returns in under 5ms regardless of how many total rows exist.
  •  by-provider : partition key is the provider name, sort key is TPS descending. This powers the provider filter on the leaderboard page.
  •  by-device : partition key is the device fingerprint hash, sort key is the submission timestamp. This powers the “my submissions” view.
The table uses on-demand capacity mode. There is no provisioned throughput to manage. If the site gets picked up by a newsletter or hits the top of Hacker News, DynamoDB absorbs the spike automatically. I do not get paged, nothing falls over.

Leaderboard Stats Bar

At the top of the leaderboard page there is a live stats bar that shows total community submissions, average TPS across all results, and the current fastest recorded submission with its Speed Grade emoji. It also shows a breakdown of the top providers by submission count. These numbers update in real time from the  /api/stats  endpoint.

On the trust layer

Because benchmarks run entirely in the user’s browser with their own API key, I cannot re-run submissions server-side to verify the numbers. That is an honest limitation and I am upfront about it on the leaderboard page with a “self-reported” label.
What I did build is a pragmatic defence layer:
  • Cloudflare Turnstile stops automated bot submissions
  • HMAC-signed single-use tokens stop replay attacks (the client gets a signed nonce, uses it once, the server rejects any reuse)
  • Device fingerprinting and per-IP rate limiting slow down anyone trying to flood the leaderboard
  • Server-side sanity validation rejects numbers that fall outside plausible TPS and TTFT ranges
It is not tamper-proof. But it raises the bar enough to make gaming the leaderboard more effort than it is worth, which is good enough for a community tool.



Key Metrics: A Quick Refresher

If you are new to iamspeed.dev, here is what the three numbers mean.

TTFT: Time to First Token

The time between sending the request and receiving the first token back from the model. This is what users perceive as the “delay” before the response starts appearing. Reasoning models like o3 and Claude with extended thinking can have TTFTs of 10 to 30 seconds because of the additional compute happening before any visible output arrives.

TPS: Tokens Per Second

The rate at which the model streams tokens after the first one arrives. This determines how fluid and fast the response feels as it generates. Cerebras and Groq regularly exceed 1,000 tokens per second on smaller models. GPT-4o is typically in the 100 to 200 range. The difference is visible.

TTLT: Time to Last Token

The total wall-clock time from request to the final token. This is the actual end-to-end latency for a complete response and what Race Mode uses to declare a winner.



Why This Matters for Performance Engineers

I want to speak to the performance engineers reading this for a moment.
We are used to measuring HTTP APIs with JMeter, k6, and Gatling. We understand throughput, latency percentiles, and concurrency. LLM APIs add a new dimension to all of this.
When you are integrating an LLM into a production system, you are not just measuring response time anymore. You are dealing with:
  • TTFT as a user-perceived latency metric, equivalent to time-to-interactive in web performance
  • Streaming throughput as a sustained delivery rate, not a one-shot measurement
  • Provider variability, where the same model can perform very differently across regions and time of day
  • Cost as a multiplier on performance decisions
The community leaderboard gives you something new: a baseline from real users running real requests. Before you design a full load test for your LLM-powered feature, check the leaderboard to see where your provider lands relative to others. It is not a substitute for your own profiling, but it is a fast sanity check.
I have written about LLM performance metrics on QAInsights before, and I built the  jmeter-llm-sampler  plugin for measuring TTFT and TTLT inside JMeter test plans. iamspeed.dev is the browser-friendly companion: fast to open, zero setup, real numbers.



Local Models: Ollama and LM Studio

If you are running models locally, iamspeed.dev supports that too.
Select “Local” in the settings panel. The default base URL is  http://localhost:11434/v1  for Ollama. LM Studio uses  http://localhost:1234/v1 . Click “Discover models from endpoint” and iamspeed.dev fetches the available models directly from your local server.
One caveat: if you are accessing iamspeed.dev over HTTPS, browsers will block HTTP requests to a local endpoint due to mixed-content restrictions. The cleanest workaround is a Cloudflare tunnel:
npx cloudflared tunnel --url http://localhost:11434 --http-host-header localhost

This gives you a secure HTTPS endpoint that proxies to your local Ollama instance.



How to Try It

Go to iamspeed.dev
For a single benchmark: click the gear icon, add your API key for any supported provider, choose a model, and hit Run. You will have your TTFT and tokens-per-second numbers in about 10 seconds.
For Race Mode: switch to Race Mode at the top, pick two or three providers, configure each API key, and hit Run. Watch them race.
For the leaderboard: run a benchmark in Simple mode, click “Submit to Leaderboard”, add a nickname, and your result goes live. Head to iamspeed.dev to see how you stack up.
The source code is MIT licensed and available on  GitHub .

What is Next

A few things on the roadmap:
  • Race Mode leaderboard: the leaderboard submission infrastructure already exists for Simple Mode. Extending it to Race Mode is the next step, storing head-to-head results and showing which provider consistently wins across the community.
  • Multi-run statistical mode: single runs are noisy. A Run x5 or Run x10 option reporting median, mean, and p95 for TPS and TTFT would produce numbers you can actually cite in a report or design doc.
  • Cost metrics: surface dollars per request and dollars per million tokens alongside TPS. A lot of the time the question is not which is fastest but which gives the best speed per dollar. Adding a Best Value award to Race Mode is the plan.
  • Tournament and bracket mode: seed 4 to 8 models into a knockout bracket, run head-to-head rounds, crown a champion.
If any of these sound useful, star the  GitHub repo  and let me know which one to ship first.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article