Toward a world without monitors and keyboards
Retinal projection, EMG and brain decoding could remove the screen and the keyboard. What works today, and which parts an AWS prototype can cover.
Series: Toward a world without monitors and keyboards (3 articles)
- 1Toward a world without monitors and keyboards This article
I want to see what an AI generates with my own eyes, not on a monitor, and to give instructions without touching a keyboard. So I asked one narrow question: can AI send signals to the brain so that an image appears on a person's retina?
The short answer is that the retina part is backwards, but the goal itself is reachable in stages. I have not built anything yet. This post is a survey plus a design for an AWS prototype. Numbers come from papers or official announcements, and I mark the ones I could only find in press coverage. The timelines are my own read.
The retina sits on the wrong side
The retina is the input end of vision. It turns light into electrical signals and sends them to the brain. I found no path that sends an image back the other way.
If you stimulate the brain directly, the image is not projected onto the retina. It is a percept that forms in the visual cortex. That splits the question in two:
- Output (replacing the monitor): how does an AI-generated image become part of my vision?
- Input (replacing the keyboard): how does my intent reach the AI?
Two ways to remove the monitor
Retinal projection needs no surgery
Light is focused to a narrow spot at the center of the pupil and painted directly onto the retina. Because the light skips the eye's focusing, the image stays sharp for people with near- or farsightedness.
Products exist. QD Laser sells a retinal scanning display that uses RGB lasers and a MEMS mirror, mainly for people with low vision. Meta showed Orion in 2024, a prototype pair of AR glasses with a 70 degree diagonal view (as reported by the press), and released the Ray-Ban Display in 2025, which shows an image to the right eye only (reported as 600 by 600 pixels). Both Meta devices use waveguides rather than retinal scanning. Mojo Vision paused its contact lens display in 2023 and moved to microLED displays.
The open problems are optical:
- Field of view and the tolerance to eye movement (the eyebox) trade off against each other
- The exit pupil has to follow the gaze, so eye tracking must be fast
- Filling a 100 by 100 degree view at 20/20 sharpness takes about 36 million pixels per eye (my own arithmetic at 60 pixels per degree)
Foveated rendering, which draws only the point you look at in full detail, cuts that number sharply. AI fits in three places: gaze tracking, computing holograms (neural holography, from Stanford's Peng and colleagues at SIGGRAPH Asia 2020, with public code), and streaming generated images that follow the gaze.
Direct cortical stimulation still produces dots
Electrical stimulation of the visual cortex makes a person see small points of light called phosphenes. An image is built from many of them.
- A 96-electrode Utah array in a fully blind 57-year-old participant, implanted for six months, let them identify some letters and object outlines (Fernández et al., 2021)
- Stimulating electrodes in sequence to trace letter shapes let participants, including two blind ones, recognize letters (Beauchamp et al., Cell 2020)
- In monkeys, a 1,024-channel implant produced perceived shapes (Chen et al., Science 2020)
- Neuralink received an FDA Breakthrough Device designation for Blindsight in September 2024. I could not confirm a human implant
Ninety-six points are far from the number a useful image needs. My understanding is hundreds to thousands, though I did not check a specific paper for that figure. What I did confirm is the bandwidth gap: the retina's output is estimated at about 10 Mbps, extrapolated from guinea pig recordings (Koch et al., 2006). Even counting one bit per pulse, a 1,000-electrode implant firing 10 times a second carries about 10 kbps, three orders of magnitude below that (my own estimate).
Two ways to remove the keyboard
EMG and voice work now
Meta's Neural Band reads muscle signals at the wrist. A Nature paper in 2025 reported a model that works without per-person calibration and reached 20.9 words per minute for handwriting in the air. The band launched on September 30, 2025 together with the Ray-Ban Display.
Brain to text starts in medicine
Electrodes read activity from motor and speech areas, and AI turns it into phonemes and text.
| Study | Setup | Result |
|---|---|---|
| Willett et al., Nature 2023 | 256 intracortical electrodes, one person with ALS | 62 words per minute, 23.8% word error rate at 125,000 words |
| Metzger et al., Nature 2023 | 253-channel surface electrodes, one person after a brainstem stroke | 78 words per minute, 25.5% error rate on general sentences from a 1,024-word vocabulary |
| Card et al., NEJM 2024 | 256 intracortical electrodes, one person with ALS | 97.5% accuracy held for 8.4 months, about 32 words per minute |
These systems read attempted speech, not arbitrary thoughts. A 2025 Cell paper from Kunz and colleagues decoded some inner speech in four participants and added a safeguard: the decoder only activates when the person imagines a specific keyword. Non-invasive methods are far behind. Meta's Brain2Qwerty preprint reports character error rates of 32% with MEG and 67% with EEG.
Comparing the four routes
| Route | Goal | Surgery | My outlook |
|---|---|---|---|
| Retinal projection | See with my own eyes | No | Years |
| EMG and voice | No keyboard | No | Available now |
| Brain to text | Thought input | Yes | Medical use first |
| Cortical stimulation | No eyes at all | Yes | Natural vision is decades away |
Retinal projection plus EMG and voice gets close to a world without monitors or keyboards, with no surgery. Brain interfaces matter for people who cannot use their eyes or hands, and for replacing a sense outright.
What an AWS prototype would look like
The software side can run on AWS. Electrodes, optics, lasers and MEMS mirrors cannot. The design assumes ap-northeast-1 and has four parts:
- Input: AWS IoT Core and Amazon Kinesis Data Streams take the sensor stream, and replayed public data goes through the same path
- Decode: an Amazon EC2 GPU instance runs a GRU decoder trained with CTC loss, and Amazon Bedrock adds language model rescoring
- Render: another EC2 GPU instance runs a phosphene simulator or hologram computation, and Amazon DCV streams the result to a browser
- Store: Amazon S3 holds datasets and models
Public data is available for each part. Willett's 2023 dataset is on Dryad, and Kaggle ran Brain-to-Text '25 on the data from Card et al. Meta released emg2qwerty for EMG (108 people, 346 hours, CC BY-NC-SA 4.0, so no commercial use). Dynaphos (eLife 2024) is a differentiable simulator that predicts what a given stimulation would look like.
Four constraints shape the design:
- The loop from gaze tracking to exit pupil steering has to live on the device, because a round trip to AWS is too slow
- When I checked in September 2026, my account's Amazon SageMaker AI quota for GPU training jobs was 0 in ap-northeast-1, so training would go to EC2 GPU instances, or CPU for a small GRU
- Public data keeps neural data handling simple, but recording my own EEG or EMG would need encryption and a retention rule
- Instance prices vary, so I would check them before starting
What I would build first
The first prototype would be a simulated cortical vision demo: a webcam frame becomes the phosphene image that 1,000 electrodes could produce, shown in a browser. It needs no data collection, it is easy to show, and it runs entirely on AWS. A replay of a public brain-to-text dataset through the decoder is second.
Back to the question
Sending signals to the brain to show an image is already possible in the form of dots, though "onto the retina" is the wrong description. Natural images are decades away. If the goal is only to see AI output with my own eyes, projecting onto the retina is the shortest path, and EMG and voice cover input first. Most of the monitor and keyboard can go without waiting for surgery.
I have not implemented any of this yet. When I do, I will start with the simulated cortical vision demo.
References
- Willett et al., Nature 2023: https://pmc.ncbi.nlm.nih.gov/articles/PMC10468393/
- Metzger et al., Nature 2023: https://www.nature.com/articles/s41586-023-06443-4
- Card et al., NEJM 2024: https://doi.org/10.1056/nejmoa2314132
- Kunz et al., Cell 2025: https://pubmed.ncbi.nlm.nih.gov/40816265/
- Fernández et al., J Clin Invest 2021: https://www.jci.org/articles/view/151331
- Beauchamp et al., Cell 2020: https://www.cell.com/cell/fulltext/S0092-8674(20)30496-7
- Chen et al., Science 2020: https://www.science.org/doi/10.1126/science.abd7435
- Koch et al., Current Biology 2006: https://www.cell.com/current-biology/fulltext/S0960-9822(06)01639-3
- Meta sEMG, Nature 2025: https://www.nature.com/articles/s41586-025-09255-w
- Neuralink Blindsight (FDA designation): https://neuralink.com/updates/neuralink-receives-breakthrough-device-designation-for-blindsight/
- Meta Orion: https://www.uploadvr.com/meta-connect-2024-orion-prototype-ar-glasses/
- Brain2Qwerty (arXiv): https://arxiv.org/abs/2502.17480
- Neural holography code: https://github.com/computational-imaging/neural-holography
- Dynaphos (eLife 2024): https://elifesciences.org/articles/85812
- Willett 2023 dataset (Dryad): https://datadryad.org/dataset/doi:10.5061/dryad.x69p8czpq
- Brain-to-Text '25 (Kaggle): https://www.kaggle.com/competitions/brain-to-text-25
- emg2qwerty: https://github.com/facebookresearch/emg2qwerty
Series: Toward a world without monitors and keyboards (3 articles)
- 1Toward a world without monitors and keyboards This article
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article