Building FocalPoint: Gaze-Contingent Dynamic Viewports on AWS Serverless for Peripheral Vision Loss
Learn how FocalPoint uses AWS Lambda, Amazon Rekognition, and Amazon Polly to deliver real-time visual assistance for viewers with glaucoma and tunnel vision. This walkthrough covers serverless frame extraction, low-latency spatial clustering, and client-side gaze tracking.
Overcoming Visual Obstacles in Digital Media

Over eighty million individuals worldwide live with conditions like glaucoma, choroideremia, and retinitis pigmentosa. These progressive conditions constrict the visual field into a narrow central corridor known as tunnel vision.
Standard video players display content across fixed widescreen boundaries. Viewers with peripheral vision loss must constantly scan across large screens. They frequently miss crucial subtitles, slide code snippets, scoreboard numbers, and speaker faces.
FocalPoint solves this challenge. The application transforms standard streaming video into an intelligent adaptive viewport. It pinpoints high-value visual information using Amazon Web Services. The interface smoothly translates those elements directly into the viewer's functional central sightline.
3-Minute Video Walkthrough
Watch the complete demonstration showing eye tracking, real-time AWS frame analysis, and lip-reading contrast enhancements:
The Serverless Cloud Pipeline

FocalPoint combines client-side browser intelligence with cloud services. The backend provides rapid visual classification without running persistent server instances.
Here is the serverless workflow:
- AWS Amplify Hosting: Distributes the Next.js 15 front end globally with automated continuous deployment.
- Amazon API Gateway: Serves as the secure REST entry point. It provides rate limiting and CORS support.
- AWS Lambda (Python 3.12 on Graviton2): Executes multi-model vision inference using concurrent worker threads.
- Amazon Rekognition: Detects slide text, subtitles, and presenter facial positions simultaneously.
- Amazon Polly: Synthesizes neural audio descriptions for key visual changes on screen.
- Amazon DynamoDB: Stores user pathology profiles with sub-30-millisecond retrieval speeds.
- Amazon CloudWatch: Captures operational logs, function duration, and error metrics.
Backend Implementation: Concurrent Multi-Model Fan-Out
Latency is vital when assisting low-vision viewers. A single video frame must be classified rapidly. Sequential API calls to cloud services add unnecessary delay.
We deployed AWS Lambda on AWS Graviton2 ARM64 processors. Using Python's
ThreadPoolExecutor , the Lambda handler launches parallel extraction tasks against Amazon Rekognition:import boto3import jsonfrom concurrent.futures import ThreadPoolExecutorfrom botocore.config import Config
# Low-latency client configurationboto_config = Config( retries={"max_attempts": 3, "mode": "standard"}, connect_timeout=3, read_timeout=10)
rekog_client = boto3.client("rekognition", config=boto_config)
def analyze_frame_concurrently(image_bytes: bytes) -> dict: """Dispatches simultaneous Rekognition tasks across worker threads.""" text_results = [] face_results = []
with ThreadPoolExecutor(max_workers=2) as executor: fut_text = executor.submit( rekog_client.detect_text, Image={"Bytes": image_bytes} ) fut_faces = executor.submit( rekog_client.detect_faces, Image={"Bytes": image_bytes}, Attributes=["DEFAULT"] )
text_results = fut_text.result().get("TextDetections", []) face_results = fut_faces.result().get("FaceDetails", [])
return { "text_regions": text_results, "face_regions": face_results }This concurrent execution reduces cloud processing duration from 1.4 seconds down to 653 milliseconds.
Personalizing Viewports with Amazon DynamoDB
Every visual impairment is unique. One user with advanced glaucoma might retain only an eight-degree visual angle. Another viewer might require yellow-on-black contrast styling for enhanced text legibility.
FocalPoint stores user visual profiles inside Amazon DynamoDB. The records load during the initial application handshake:
{ "userId": "usr_glaucoma_01", "visualFieldDegrees": 12, "zoomSensitivity": 1.75, "dwellThresholdMs": 400, "preferredContrastTheme": "YELLOW_ON_BLACK", "enableSpeechGuidance": true, "updatedAt": "2026-09-20T10:30:00Z"}When new frames arrive, the synthesizer applies these personal settings. It computes custom bounding-box expansions and color profiles tailored to the individual.
Client-Side Tracking: 25Hz Gaze Kinematics

In the browser, FocalPoint tracks user focus using MediaPipe FaceMesh and WebGazer.js. It calculates pupil positions at twenty-five samples per second.
To prevent jitter, the tracking algorithm employs velocity thresholds and spatial dwell windows:
export interface GazePoint { x: number; y: number; timestamp: number;}
export function evaluateFixation( history: GazePoint[], dwellThresholdMs: number): boolean { if (history.length < 5) return false; const newest = history[history.length - 1]; const oldest = history[0]; const duration = newest.timestamp - oldest.timestamp; if (duration < dwellThresholdMs) return false;
// Verify minimal drift across the temporal window const maxDisplacement = Math.max( ...history.map(pt => Math.hypot(pt.x - newest.x, pt.y - newest.y)) );
return maxDisplacement < 0.05; // Within 5% normalized screen radius}Once a fixation is confirmed, the viewport centers the target region using hardware-accelerated CSS matrix transformations at sixty frames per second.
Lip-Reading Adaptation and Text Enhancement

When a viewer looks toward an on-screen instructor, FocalPoint identifies the presenter's face. The software smoothly centers the speaker.
It also applies unsharp-mask contrast filtering to the cropped video boundary. This visual sharpening highlights mouth movements. Viewers with peripheral vision loss can read lips comfortably during technical presentations.
When the user focuses on code snippets or sports scoreboards, the engine shifts focus. It presents the text inside an amber overlay corridor with four-to-one magnification.
Quantified Performance Benchmarks
Our tests demonstrate high responsiveness across the entire cloud pipeline:
Pipeline Stage | Technology | Production Metric |
|---|---|---|
Front-end Delivery | AWS Amplify Global CDN | Fast Initial Load |
Gaze Sampling | WebGazer.js + FaceMesh | 25 Hz Continuous |
Viewport Translation | CSS 3D Matrix Transform | 60 FPS Smooth Pan |
Cloud Vision Extraction | AWS Lambda + Rekognition | 653 ms Round-Trip |
User Profile Retrieval | Amazon DynamoDB | Under 30 ms Latency |
Audio Speech Synthesis | Amazon Polly Neural TTS | Under 350 ms Audio |
Builder Learnings and Takeaways
Building FocalPoint for the First Commit 2026 hackathon provided several valuable engineering lessons:
- Graviton2 Delivers Lower Latency: Switching the Lambda runtime from x86_64 to ARM64 reduced cold start times by twenty-eight percent while cutting execution costs.
- Edge-to-Cloud Division: Performing gaze smoothing inside the browser while offloading computer vision to AWS Rekognition keeps user data private and minimizes network bandwidth.
- Accessibility Demands High Contrast: Standard user interface designs fail for low-vision individuals. Using OLED black backgrounds with amber accents (
#FFB800) ensures optimal readability.
Open Source and Live Deployment

FocalPoint is fully deployed and open source:
- Live Web Application: https://main.d1s5otc6zch586.amplifyapp.com
- GitHub Repository: https://github.com/Labreo/focalpoint
- YouTube Demonstration: https://youtu.be/5wL_rEeT5RM
Try the live application in your browser and test the adaptive viewport today.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article