Rebuilding a brain-to-text decoder with Amazon Bedrock
A speech BCI decoder trained from public neural data on one CPU instance for about $5, with Amazon Bedrock picking sentences. Word errors: 50.0% to 23.5%.
Series: Toward a world without monitors and keyboards (3 articles)
- 2Rebuilding a brain-to-text decoder with Amazon Bedrock This article
A brain-to-text decoder reads the neural activity of someone trying to speak and turns it into words. I rebuilt one from the public dataset of Willett et al. (Nature 2023), trained it on a single CPU instance in Amazon EC2, and used Amazon Bedrock to choose the final sentence.
This post covers the AWS side: why I trained on CPUs, how the training stack cleans up after itself, and how I called Bedrock without letting the LLM make things up. The results come at the end. It is an offline reanalysis of recorded data. Nothing ran in real time and nothing touched a person.
The pipeline
- Neural features: 256 features per 20 ms bin from area 6v (threshold crossings and spike-band power on 128 electrodes)
- Decoder: a bidirectional GRU trained with CTC that outputs phoneme probabilities, shaped like the PyTorch baseline (cffan/neural_seq_decoder)
- Word search: a pronunciation lexicon and a 3-gram language model turn the phoneme probabilities into the 10 best word sequences
- Selection: Claude Sonnet 4.6 on Amazon Bedrock picks one of the 10 by number
The data is 8,800 training sentences and 880 test sentences recorded over 24 days, released on Dryad under CC0.
Why CPUs
The On-Demand quota for G and VT instances was 0 in this account, and the standard-instance vCPU limit was 32, with 4 already in use. A c7i.4xlarge (16 vCPUs) was the largest instance I could launch.
Before the real run, I trained for 20 batches and timed them:
| Setup | Per batch | 10,000 batches | Estimated cost |
|---|---|---|---|
| Baseline size (5 layers, 1,024 units), random batches | 11.7 s | about 33 h | about $29 |
| Baseline size, length-bucketed batches | 4.9 s | about 14 h | about $12 |
| Reduced (3 layers, 512 units), length-bucketed batches | 1.5 s | about 4.3 h | about $3.8 |
Length bucketing groups sentences of similar length so less compute goes to padding. It is not in the baseline, so it is one of the differences to keep in mind when comparing numbers.
I trained the reduced model. The run took 5 hours 12 minutes and cost about $4.7 at the On-Demand price, plus about $0.1 for the benchmark.
A training stack that deletes itself
One AWS CloudFormation stack holds the instance, its launch template and an IAM role:
- At boot, the instance pulls the code and the preprocessed data from Amazon S3, and syncs checkpoints and logs back every two minutes
- When training finishes, the instance shuts itself down. The shutdown behavior is set to terminate, so the instance is deleted
- At boot it also schedules a hard shutdown after a time limit, in case training hangs
- The IAM role can read only the code and data prefixes and write only its own run prefix
- IMDSv2 is required, and the EBS volume is encrypted
1
2
3
scripts/aws_train.sh half1 half ondemand c7i.4xlarge # reduced model on c7i.4xlarge
scripts/aws_status.sh half1 # status written to S3 by the instance
scripts/aws_fetch_and_clean.sh half1 # download results, delete the stackThe first deployment failed with On-Demand. The launch template only set market options, and only for Spot, so with On-Demand it had no settings at all and CloudFormation rejected it. Putting the metadata options in the template on every path fixed it.
Calling Bedrock without letting it write
My first approach asked Sonnet 4.6 to write an English sentence from the top phoneme candidates. Word errors fell from 50.0% to 39.2%, but the LLM invented text when the phonemes were poor. For "Quite a few movies are based off of that.", the nearest words were almost right ("quite a you movie are base off of that"). The LLM returned "why do you think they are based off of that".
So I stopped letting it write. The decoder now produces 10 complete word sequences, and the LLM only replies with the number of the most likely one. A wrong pick can only be another decoder candidate, never a new sentence.
Three things I had to handle on the Bedrock side:
- Throttling. Cross-region inference for Sonnet 4.6 allowed 50 requests per minute in this account, and my first run hit it. I capped my client at 40 per minute and retried only the failures
- Content filtering. 153 of 1,760 calls in the first approach came back empty with content_filtered, probably because broken phoneme strings formed unfortunate words, though I did not confirm that. I re-sent only those calls to Claude Haiku 4.5, which answered all of them
- Explanations instead of numbers. When every candidate looked wrong, the model sometimes explained instead of answering (120 of 880). A stricter retry ("reply with the closest one, never explain") fixed all but one, which went to Haiku 4.5
Every response went into a local cache, so reruns and new variants did not pay for the same calls twice.
Results
Word error rate on the 880 test sentences:
| Step | Word error rate |
|---|---|
| Nearest training-vocabulary word per phoneme group, no LLM | 50.0% |
| LLM writes sentences, switched off when its word count drifts | 34.6% |
| Word-level decoding with a 3-gram from the training sentences | 29.7% |
| Same, LLM picks from the top 10 | 26.0% |
| 3-gram built from Tatoeba English (about 2.04 million sentences) plus training text, LLM picks | 23.5% |
The LLM fixed small errors such as "they close sometime after eat" to "eight" and "solving the variables in the occasion" to "equation". It replaced a correct first candidate with a wrong one in only 3 of 880 sentences.
A stronger model did not help. I split the test set into a 440-sentence development half and a 440-sentence evaluation half. I compared methods on the first, then ran the Sonnet baseline and Claude Opus 4.6, the best on development, once each on the second. Sonnet 4.6 scored 23.0% and Opus 23.1%, a difference whose bootstrap 95% interval includes zero, so 23.0% is my held-out number. Twenty candidates instead of ten did not help either. The ceiling was set by whether the right sentence was among the candidates, which depends on the language model and the decoder, not on the LLM.
The paper reports 23.8%, but its model size, language model, vocabulary and evaluation differ from mine, so the numbers are not directly comparable.
Cleanup
Word-level decoding and evaluation ran on my laptop, using the trained model downloaded from S3. When I finished, I deleted the stack and the bucket, and checked that no instance, stack or bucket remained.
Tatoeba data is licensed CC BY 2.0 FR (https://tatoeba.org). The neural dataset is at https://datadryad.org/dataset/doi:10.5061/dryad.x69p8czpq.
Series: Toward a world without monitors and keyboards (3 articles)
- 2Rebuilding a brain-to-text decoder with Amazon Bedrock This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article