AWS Builder Center

Building FocalPoint: Gaze-Contingent Dynamic Viewports on AWS Serverless for Peripheral Vision Loss

Learn how FocalPoint uses AWS Lambda, Amazon Rekognition, and Amazon Polly to deliver real-time visual assistance for viewers with glaucoma and tunnel vision. This walkthrough covers serverless frame extraction, low-latency spatial clustering, and client-side gaze tracking.

Computer Engineering Student @ GEC | Open Source & GenAI Systems Builder

Overcoming Visual Obstacles in Digital Media


Over eighty million individuals worldwide live with conditions like glaucoma, choroideremia, and retinitis pigmentosa. These progressive conditions constrict the visual field into a narrow central corridor known as tunnel vision. Standard video players display content across fixed widescreen boundaries. Viewers with peripheral vision loss must constantly scan across large screens. They frequently miss crucial subtitles, slide code snippets, scoreboard numbers, and speaker faces.
FocalPoint solves this challenge. The application transforms standard streaming video into an intelligent adaptive viewport. It pinpoints high-value visual information using Amazon Web Services. The interface smoothly translates those elements directly into the viewer's functional central sightline.

3-Minute Video Walkthrough

Watch the complete demonstration showing eye tracking, real-time AWS frame analysis, and lip-reading contrast enhancements:

The Serverless Cloud Pipeline





FocalPoint combines client-side browser intelligence with cloud services. The backend provides rapid visual classification without running persistent server instances.
Here is the serverless workflow:
  1. AWS Amplify Hosting: Distributes the Next.js 15 front end globally with automated continuous deployment.
  1. Amazon API Gateway: Serves as the secure REST entry point. It provides rate limiting and CORS support.
  1. AWS Lambda (Python 3.12 on Graviton2): Executes multi-model vision inference using concurrent worker threads.
  1. Amazon Rekognition: Detects slide text, subtitles, and presenter facial positions simultaneously.
  1. Amazon Polly: Synthesizes neural audio descriptions for key visual changes on screen.
  1. Amazon DynamoDB: Stores user pathology profiles with sub-30-millisecond retrieval speeds.
  1. Amazon CloudWatch: Captures operational logs, function duration, and error metrics.

Backend Implementation: Concurrent Multi-Model Fan-Out

Latency is vital when assisting low-vision viewers. A single video frame must be classified rapidly. Sequential API calls to cloud services add unnecessary delay.
We deployed AWS Lambda on AWS Graviton2 ARM64 processors. Using Python's  ThreadPoolExecutor , the Lambda handler launches parallel extraction tasks against Amazon Rekognition:
import boto3
import json
from concurrent.futures import ThreadPoolExecutor
from botocore.config import Config

# Low-latency client configuration
boto_config = Config(
retries={"max_attempts": 3, "mode": "standard"},
connect_timeout=3,
read_timeout=10
)

rekog_client = boto3.client("rekognition", config=boto_config)

def analyze_frame_concurrently(image_bytes: bytes) -> dict:
"""Dispatches simultaneous Rekognition tasks across worker threads."""
text_results = []
face_results = []

with ThreadPoolExecutor(max_workers=2) as executor:
fut_text = executor.submit(
rekog_client.detect_text,
Image={"Bytes": image_bytes}
)
fut_faces = executor.submit(
rekog_client.detect_faces,
Image={"Bytes": image_bytes},
Attributes=["DEFAULT"]
)

text_results = fut_text.result().get("TextDetections", [])
face_results = fut_faces.result().get("FaceDetails", [])

return {
"text_regions": text_results,
"face_regions": face_results
}
This concurrent execution reduces cloud processing duration from 1.4 seconds down to 653 milliseconds.

Personalizing Viewports with Amazon DynamoDB

Every visual impairment is unique. One user with advanced glaucoma might retain only an eight-degree visual angle. Another viewer might require yellow-on-black contrast styling for enhanced text legibility.
FocalPoint stores user visual profiles inside Amazon DynamoDB. The records load during the initial application handshake:
{
"userId": "usr_glaucoma_01",
"visualFieldDegrees": 12,
"zoomSensitivity": 1.75,
"dwellThresholdMs": 400,
"preferredContrastTheme": "YELLOW_ON_BLACK",
"enableSpeechGuidance": true,
"updatedAt": "2026-09-20T10:30:00Z"
}
When new frames arrive, the synthesizer applies these personal settings. It computes custom bounding-box expansions and color profiles tailored to the individual.

Client-Side Tracking: 25Hz Gaze Kinematics





In the browser, FocalPoint tracks user focus using MediaPipe FaceMesh and WebGazer.js. It calculates pupil positions at twenty-five samples per second.
To prevent jitter, the tracking algorithm employs velocity thresholds and spatial dwell windows:
export interface GazePoint {
x: number;
y: number;
timestamp: number;
}

export function evaluateFixation(
history: GazePoint[],
dwellThresholdMs: number
): boolean {
if (history.length < 5) return false;
const newest = history[history.length - 1];
const oldest = history[0];
const duration = newest.timestamp - oldest.timestamp;
if (duration < dwellThresholdMs) return false;

// Verify minimal drift across the temporal window
const maxDisplacement = Math.max(
...history.map(pt => Math.hypot(pt.x - newest.x, pt.y - newest.y))
);

return maxDisplacement < 0.05; // Within 5% normalized screen radius
}
Once a fixation is confirmed, the viewport centers the target region using hardware-accelerated CSS matrix transformations at sixty frames per second.

Lip-Reading Adaptation and Text Enhancement


When a viewer looks toward an on-screen instructor, FocalPoint identifies the presenter's face. The software smoothly centers the speaker.
It also applies unsharp-mask contrast filtering to the cropped video boundary. This visual sharpening highlights mouth movements. Viewers with peripheral vision loss can read lips comfortably during technical presentations.
When the user focuses on code snippets or sports scoreboards, the engine shifts focus. It presents the text inside an amber overlay corridor with four-to-one magnification.

Quantified Performance Benchmarks

Our tests demonstrate high responsiveness across the entire cloud pipeline:
Pipeline Stage
Technology
Production Metric
Front-end Delivery
AWS Amplify Global CDN
Fast Initial Load
Gaze Sampling
WebGazer.js + FaceMesh
25 Hz Continuous
Viewport Translation
CSS 3D Matrix Transform
60 FPS Smooth Pan
Cloud Vision Extraction
AWS Lambda + Rekognition
653 ms Round-Trip
User Profile Retrieval
Amazon DynamoDB
Under 30 ms Latency
Audio Speech Synthesis
Amazon Polly Neural TTS
Under 350 ms Audio

Builder Learnings and Takeaways

Building FocalPoint for the First Commit 2026 hackathon provided several valuable engineering lessons:
  1. Graviton2 Delivers Lower Latency: Switching the Lambda runtime from x86_64 to ARM64 reduced cold start times by twenty-eight percent while cutting execution costs.
  1. Edge-to-Cloud Division: Performing gaze smoothing inside the browser while offloading computer vision to AWS Rekognition keeps user data private and minimizes network bandwidth.
  1. Accessibility Demands High Contrast: Standard user interface designs fail for low-vision individuals. Using OLED black backgrounds with amber accents ( #FFB800 ) ensures optimal readability.
Open Source and Live Deployment




FocalPoint is fully deployed and open source:
Try the live application in your browser and test the adaptive viewport today.
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article