To our colleagues, clients, regulators, and long-term owners:

Financial markets have always rewarded speed, but speed is not the same thing as haste. A trading desk must absorb incomplete information, challenge strong convictions, preserve capital, serve clients, satisfy regulatory obligations, and continue operating when conditions are least forgiving. The best desks do not eliminate pressure. They build mechanisms that prevent pressure from silently converting uncertainty into concealed risk.

Real-time voice and video agents create a new opportunity in this environment. They can make approved knowledge, escalation procedures, operational evidence, and expert assistance available during a live conversation. They can help a trader document a decision without leaving the workflow. They can help an operations specialist show an observable system condition while describing it. They can preserve continuity when a conversation is interrupted and resumed. They can reduce the distance between noticing a problem and obtaining the right support.

They also create a new class of danger. A system that listens continuously can become a system that monitors continuously. A system that can see a person can be misused to infer private characteristics. A memory designed to preserve useful context can become a permanent behavioral dossier. A model that summarizes speech can be mistaken for a clinician, a lie detector, or a reliable judge of emotional state. In a workplace where compensation, access, authority, and reputation are unequal, nominal consent may not be meaningful consent.

For that reason, this paper rejects the premise that an AI agent should diagnose a trader’s mental health or infer hidden psychological traits from voice, face, gaze, posture, or conversational style. Such an application is scientifically fragile, ethically corrosive, difficult to reconcile with psychological safety, and prohibited or heavily restricted in important jurisdictions. The European Union AI Act prohibits emotion inference in workplaces except for narrow medical or safety purposes. Hong Kong privacy guidance emphasizes lawful purpose, data minimization, governance, human oversight, transparency, security, retention control, and accountability when organizations use AI with employee data.

The responsible use case is different. The agent should help people report what they choose to report, retrieve approved procedures, capture observable operational facts, coordinate an escalation, support a voluntary pause, and preserve a controlled record of business decisions. It should not decide whether a person is mentally healthy. It should not turn tone of voice into a risk score. It should not rank employees by inferred resilience, anxiety, loyalty, confidence, honesty, or emotional stability. It should not send covert behavioral alerts to management.

This distinction is not a limitation placed around the product after it has been built. It is the product definition.

The long-term opportunity is to create a human risk conversation layer for the financial institution. This layer helps a person ask for help without stigma, helps a supervisor understand verified facts without pretending to know another person’s interior state, and helps the enterprise act before operational pressure becomes an unmanaged event. Amazon Bedrock AgentCore Runtime, WebRTC, WebSocket streaming, AgentCore Memory, AgentCore Identity, AgentCore Gateway, Amazon Kinesis Video Streams TURN relay, Amazon Bedrock foundation models, AWS Key Management Service, AWS CloudTrail, Amazon CloudWatch, and Amazon S3 can provide the technical foundation. The harder foundation is a culture that makes speaking up safe and makes surveillance unacceptable.

A trader can record trades on a personal device and still leave the institution without usable memory. The issue is not whether a note exists. The issue is whether the note is authorized, complete enough to support oversight, linked to the relevant decision and control state, protected under records policy, and available to the right person when the situation changes. A private note may help individual recall. It does not necessarily help the firm understand concentration, challenge a deteriorating thesis, prove timely escalation, or learn across desks.

The most dangerous information on a trading floor is often not secret. It is unspoken. A trader notices that liquidity is thinner than the model assumes. An operations analyst sees a recurring settlement exception. A junior colleague believes a senior person’s analogy is wrong. A desk head senses that the volume and tempo of decisions are becoming unsustainable. Nobody has enough certainty to make an accusation. Everyone has enough uncertainty to feel concern. In a weak system, the concern remains private until the market produces an objective loss.

The institution usually responds to this problem with more policy, more attestations, more dashboards, and more retrospective review. These controls are necessary, but their timing is often wrong. A person under pressure is unlikely to stop, search several portals, reconstruct an old incident, complete a long form, and initiate a formal escalation unless the situation already feels severe. The control mechanism asks for the most cognitive effort precisely when the user has the least cognitive capacity available.

A real-time conversational agent can lower that activation energy. The trader can say that a limit is approaching, that two data sources disagree, or that a client request has created uncertainty. The agent can retrieve the current procedure, confirm the applicable legal entity, identify the required approver, create a draft incident record, and offer a connection to a human supervisor. If video is genuinely necessary for an operational use case, the user can choose to show a screen, document, device, or physical indicator. The agent can identify observable content, subject to model capability and explicit policy, without interpreting facial expressions or private emotional state.

The business value comes from faster access to evidence, better continuity, lower documentation friction, and earlier voluntary escalation. It does not come from claiming to read minds.

Google’s Project Aristotle research identified psychological safety as a central dynamic of effective teams, together with dependability, structure and clarity, meaning, and impact. Psychological safety is frequently misunderstood as comfort or an absence of accountability. On a trading desk, it means that a person can reveal incomplete information, admit uncertainty, disagree with hierarchy, or request assistance without expecting humiliation or retaliation.

This is directly relevant to real-time agents. If employees believe every hesitation, pause, vocal variation, or facial movement is being analyzed for management, they will adapt their behavior. They will speak less candidly, avoid the channel, perform confidence, and move sensitive conversations elsewhere. The system designed to improve visibility will reduce visibility. Surveillance does not merely create a privacy problem. It damages the information quality on which risk management depends.

Amazon’s Leadership Principles provide the performance counterweight. Customer Obsession requires working backward from the person who must make a safe decision, not forward from the existence of a new media protocol. Ownership assigns one accountable owner for the service, one for each data domain, and one for every control exception. Invent and Simplify favors a conversational path that makes the correct action easier than the unsafe workaround. Are Right, A Lot demands diverse perspectives and active disconfirmation. Learn and Be Curious treats retrieval misses, interruptions, false transcriptions, and escalation failures as evidence for improvement. Insist on the Highest Standards makes consent, latency, availability, provenance, deletion, and human override product requirements. Dive Deep requires the team to distinguish network delay from model delay, user frustration from audio packet loss, and model inference from verified fact. Have Backbone; Disagree and Commit protects principled challenge while preserving accountable execution. Strive to be Earth’s Best Employer and Success and Scale Bring Broad Responsibility require the institution to consider the effect of its mechanisms on people, not only their technical efficiency.

Bridgewater’s emphasis on radical truth and radical transparency contributes a demanding lesson. Hidden disagreement is deferred risk. Yet radical transparency is not total transparency. Bridgewater’s published principles explicitly recognize legitimate confidentiality, including private health matters, security-sensitive information, intellectual property, and the timing of trades. In a regulated institution, transparency must be policy-bounded. Authorized people should be able to inspect the reasoning, evidence, dissent, decision, and outcome. They should not automatically gain access to private conversation, biometric data, health information, or unrelated personal context.

Alfred Adler’s emphasis on belonging, contribution, encouragement, and social interest helps explain the human side of this mechanism. People disclose uncertainty when membership in the group does not depend on appearing invulnerable. A system should reinforce the idea that requesting a second check is a contribution to the desk, not evidence of weakness. It should help the individual move from private pressure toward constructive participation. It should never convert an Adlerian language of belonging into a hidden personality taxonomy.

Nonviolent Communication offers a practical structure for capturing a difficult operational conversation. The agent can help separate observation from interpretation, identify the user’s stated concern or need, and formulate a concrete request. The system can record that a market-data feed diverged by a stated amount at a stated time, that the user is requesting a second review, and that the relevant need is decision clarity or capital protection. It should not transform that record into a conclusion that the user is anxious, unstable, dishonest, or unfit.

Chris Voss’s tactical empathy contributes a disciplined way to listen under pressure. Labels, mirrors, summaries, and calibrated language can make it easier for a person to explain constraints. Used responsibly, tactical empathy acknowledges the speaker’s perspective without declaring it objectively true. The agent can say that it sounds as though two obligations are in conflict, then restate the observable facts and offer approved actions. It must not use empathy as manipulation, simulate clinical authority, or pressure an employee into disclosing personal information.

These ideas form a coherent operating model. Psychological safety grants permission to speak. Radical truth creates an expectation that material risk is not concealed. Nonviolent Communication improves precision. Tactical empathy improves listening. Adlerian belonging makes contribution sustainable. Amazon’s Leadership Principles convert values into owners, mechanisms, measures, and correction loops. Technology can support this model only if it is prohibited from becoming an invisible judge of the people using it.

Amazon Bedrock AgentCore became generally available in October 2025 as a managed platform for deploying and operating agents with runtime, memory, identity, gateway, observability, and related enterprise capabilities. AgentCore Runtime provides a serverless, framework-agnostic execution environment designed for isolated agent sessions and production scaling.

AgentCore Runtime first supported bidirectional streaming through WebSocket. A persistent WebSocket connection carries data in both directions over TCP and supports streaming interactions in which the agent can receive input while generating output. This is useful for text, control messages, audio pipelines, tool events, and real-time agent applications where a full-duplex application channel is required.

On March 20, 2026, AWS announced WebRTC support for real-time bidirectional audio and video streaming between clients and agents on AgentCore Runtime. WebRTC is particularly useful for browser and mobile voice agents because browser APIs and mobile SDKs can capture and play media using a low-latency transport designed for interactive communication.

A critical correction is necessary. AgentCore Runtime does not expose a documented service called a “Bedrock WebRTC Media Gateway” that AWS automatically provisions as a private SFU or MCU cluster for each customer. The documented architecture places the agent application inside AgentCore Runtime. The application creates and manages the WebRTC peer connection, exchanges session descriptions and ICE candidates, obtains TURN configuration, and connects to the model or processing pipeline. AgentCore Runtime provides the secure hosting and invocation boundary. TURN relay can be supplied by Amazon Kinesis Video Streams, a third-party provider, or self-hosted infrastructure such as coturn.

A second correction is equally important. The current AgentCore Runtime WebRTC documentation requires VPC network mode and TURN relay for media traffic between the client and the agent. The VPC must permit outbound UDP traffic to the selected TURN endpoints. The AWS tutorial uses Amazon Kinesis Video Streams managed TURN servers and a Kinesis Video Streams signaling channel resource to obtain temporary TURN credentials. WebRTC signaling can be exchanged through the AgentCore Runtime invoke API or through a Kinesis Video Streams signaling channel.

This is therefore not a generic internet peer-to-peer architecture in which a browser is guaranteed to connect directly to an AWS model endpoint. In the documented AgentCore pattern, TURN relay is required for the Runtime deployment. That has consequences for cost, network design, encryption boundaries, packet paths, troubleshooting, and regional availability.

A third correction concerns latency. WebRTC can reduce media transport delay and improve conversational responsiveness, but AWS does not guarantee that end-to-end multimodal interaction will remain below 200 milliseconds. Total latency includes device capture, audio processing, browser buffering, network conditions, TURN relay, agent execution, model time to first output, tool calls, safety evaluations, synthesis, playback buffering, and interruption handling. Natural conversation is an engineering objective measured by percentile, not a universal product claim.

A fourth correction concerns native multimodal model processing. AgentCore Runtime can host an agent that forwards streamed audio or video to a capable foundation model or to a processing pipeline. This does not mean every Bedrock model accepts arbitrary native audio and video streams, nor that Runtime automatically decodes media into every selected model. Model modality, codec requirements, streaming API, quotas, regional support, and application adapters must be verified for the chosen design. The AWS WebRTC tutorial demonstrates bidirectional voice using a Bedrock speech-to-speech foundation model. Video processing requires a model and application path that explicitly support the intended frames, rates, formats, and inference pattern.

These corrections strengthen the architecture. Enterprise systems should depend on documented contracts rather than imagined internal components.

The architecture should separate five concerns: identity, signaling and control, media transport, agent execution, and durable memory.

Identity begins before media capture. An enterprise web or mobile client authenticates the user through the institution’s identity provider. AgentCore Identity can support inbound and outbound authentication patterns, while Amazon Cognito or workforce federation may provide application identity depending on the design. The back end maps the authenticated principal to a business role, legal entity, desk, jurisdiction, permitted purpose, and data policy. The client does not declare its own authority.

Signaling establishes the WebRTC relationship. The client creates an RTCPeerConnection, requests explicit microphone or camera permission, selects the approved input devices, and generates a Session Description Protocol offer. The offer describes supported codecs and transport parameters. The client transmits the offer through the selected signaling path, such as the AgentCore Runtime invoke API or a Kinesis Video Streams signaling channel. The agent application creates the corresponding peer connection, evaluates compatible codecs, and returns an SDP answer. Both sides exchange ICE candidates until connectivity checks identify the viable relay path.

Media transport carries time-sensitive audio and approved video. WebRTC uses ICE for candidate negotiation, DTLS to establish cryptographic parameters, and SRTP to protect media. In the documented AgentCore Runtime configuration, traffic is relayed through TURN. The client and agent do not need to expose arbitrary inbound ports to each other. The selected TURN service relays encrypted media packets between them, while the agent application terminates the WebRTC session and processes the media.

Control traffic should remain logically separate from media even if some application messages use a WebRTC data channel. Session lifecycle, consent state, tool approvals, transcript events, model instructions, escalation commands, and checkpoint writes are not interchangeable with audio frames. They have different reliability, ordering, retention, and authorization requirements. High-rate media can tolerate some packet loss. A tool approval cannot. A transcript correction must be ordered. A consent revocation must take effect immediately.

Agent execution occurs inside AgentCore Runtime. The agent can implement a direct speech-to-speech path, a cascaded speech-to-text, language-model, text-to-speech path, or a hybrid. Direct speech-to-speech can reduce transformation overhead and preserve conversational timing. A cascaded architecture can offer greater component control, explicit transcription, model substitution, and audit separation. The right choice depends on the use case, language, explainability, latency, records policy, and model availability.

Durable memory should use AgentCore Memory rather than an invented “managed session storage” abstraction. AgentCore Memory supports short-term and long-term memory. Short-term memory stores turn-by-turn events inside a session, including messages and tool-related events supplied by the application. Long-term memory applies configured strategies, such as semantic extraction, summarization, user preference, episodic, or custom strategies, and organizes records through namespaces. Memory is scoped using an actor identifier and session identifier, allowing the application to isolate people and conversations.

The system can resume useful context by retrieving the same actor and session history, but “seamless resume” is not magic. The application must authenticate the returning user, authorize access to the referenced session, retrieve permitted events or memory records, validate freshness, reconstruct application state, and determine whether an interrupted tool operation can safely continue. A session identifier is a locator, not authority. Possession of a session identifier must never grant access by itself.

The system should also avoid promising that it will restore an “emotional atmosphere.” That phrase invites unsupported inference and persistent profiling. What can be restored safely is explicit business context: the user was reviewing a settlement exception, the procedure was at step four, the user had approved a read-only account lookup, the last tool call returned a timeout, and a human supervisor had not yet joined. This is useful, auditable, and bounded.

A layered memory design remains valuable, but its layers must be real and controlled.

The immediate application state exists in process memory inside the isolated Runtime session. It includes the active peer connection, current turn, jitter state, current model stream, pending tool request, cancellation tokens, and the minimum context required for the next action. Model providers may use internal attention caches during inference, but AgentCore Memory should not be described as directly mapping business context into a foundation model’s KV cache. KV-cache implementation is model-specific and is not the enterprise memory contract.

Short-term memory stores selected interaction events associated with actorId and sessionId. The application decides what to write. For a regulated trading workflow, it may store an approved transcript, the user’s corrections, policy citations, tool calls, tool results, explicit confirmations, escalation events, and a structured checkpoint. Raw audio and video should not be stored by default. If recording is required for a defined legal or business purpose, it should use a separate, explicit recording path with clear notice, restricted access, retention limits, encryption, and legal approval.

Long-term memory stores durable, useful abstractions through configured strategies. Appropriate content includes the user’s stated interface preference, a summary of a resolved operational case, or a checkpoint needed to resume a business workflow. Inappropriate content includes inferred mental-health status, inferred personality, engagement scores, deception scores, emotional baselines, or speculative labels about resilience.

The architecture should distinguish personal memory from institutional memory. Personal memory is scoped to the actor and supports continuity for that user. Case memory is scoped to a business process and supports authorized participants. Institutional knowledge belongs in governed repositories such as Amazon S3, Amazon S3 Vectors, or another approved knowledge architecture. These scopes should not be merged simply because a model can retrieve across them.

Retention should be purpose-specific. Ephemeral media buffers may live for seconds. Temporary transcripts may live until user correction or case completion. Short-term events may follow operational retention. Long-term memory records may have a shorter or longer period depending on purpose. Formal records may be transferred to a records system and then removed from conversational memory. Deletion must include derived summaries and indexes, not only the raw event.

A robust design treats media and control as separate planes with explicit synchronization.

The media plane contains audio, approved video, Real-time Transport Control Protocol feedback, packet timing, codec state, and interruption signals. Its priority is bounded latency and graceful degradation. It should not block because an audit write is slow. It should adapt to loss, jitter, and bandwidth changes. Voice should remain usable when video is disabled or degraded.

The control plane contains authentication, authorization, consent, signaling, session state, memory writes, model configuration, tool requests, tool approvals, policy decisions, escalation, and observability correlation identifiers. Its priority is correctness, ordering, traceability, and durable acknowledgement.

A single conversation needs a correlation identifier shared by both planes, but that identifier should reveal no personal or business information. Each media event, transcript segment, tool call, and memory checkpoint can carry the correlation identifier and a monotonically increasing sequence number. The system then knows which transcript segment preceded a tool proposal, which explicit confirmation authorized an action, and which media interruption cancelled it.

The two planes must meet at controlled gates. A voice statement may produce a proposed tool call, but the proposal cannot execute merely because the speech recognizer considers the transcript final. The application checks identity, policy, required confirmation, tool schema, amount limits, market state, and idempotency. In financial services, the model proposes and deterministic policy decides. A human remains responsible where policy demands human authorization.

The most credible use case is a voluntary, real-time desk support and escalation assistant rather than a mental-health monitor.

A trader begins a private session and states that two liquidity indicators have diverged, the position is approaching an internal threshold, and the trader wants a second review. The agent reads the current timestamp and approved desk context, retrieves the active escalation procedure, and confirms the legal entity and book. It asks the trader to verify observable facts rather than infer intent. It can retrieve current limits through an approved read-only tool, summarize the discrepancy, and prepare a structured escalation note.

The agent may say that it has recorded a stated concern about decision clarity and capital preservation. It may offer approved choices such as contacting the desk head, opening a risk review, pausing a non-urgent workflow, or displaying the relevant procedure. It does not state that the trader is anxious. It does not score confidence from voice. It does not decide that the trader is impaired. It does not notify management about an inferred condition.

If the user voluntarily states that they feel overwhelmed or need a break, the system can acknowledge the statement and present organizationally approved support options. If there is an immediate safety concern, predetermined emergency procedures should direct the person to qualified human support. The agent must not diagnose, treat, or claim clinical expertise.

A team-level view may measure process indicators that do not require psychological inference. Examples include frequency of voluntary second checks, time to obtain a supervisor response, unresolved operational exceptions, concentration of after-hours incidents, repeated failed tool calls, workflow abandonment, and time spent waiting for approvals. These are observable system and process measures. Even these measures require governance because they can become proxies for individual performance or health if used carelessly.

The purpose is risk management, not judgment. The design does not label a person right or wrong, good or bad, weak or strong. It helps the institution keep business decisions, access, escalation, and operations within controlled boundaries.

The implementation begins with an explicit use-case charter approved by business, risk, compliance, privacy, employment counsel, information security, model risk, records management, and employee representatives where appropriate. The charter states what the system does, what it does not do, which data it processes, whether recording occurs, how long each class of data is retained, who can see it, and how a person can challenge or correct a record.

The first AWS environment should be a non-production account governed through AWS Organizations. Service control policies restrict Regions and prohibited services. AWS CloudTrail, AWS Config, Amazon GuardDuty, and AWS Security Hub establish the initial evidence and security baseline. AWS KMS customer-managed keys separate media-related records, memory, logs, and business artifacts where policy requires distinct cryptographic boundaries.

A VPC is required for AgentCore Runtime WebRTC. The architecture uses private subnets for Runtime network interfaces and controlled egress through a NAT gateway or another approved egress architecture. Network ACLs, security groups, route tables, and DNS controls allow only the endpoints needed for Bedrock, Kinesis Video Streams TURN, observability, approved tools, and organizational services. The design verifies that outbound UDP to TURN endpoints works from the Runtime environment. A network test belongs in deployment validation because a successful signaling exchange does not prove that media packets can flow.

For the AWS-managed TURN option, the team creates a Kinesis Video Streams signaling channel and grants the Runtime execution role the narrowly scoped permissions needed to call GetIceServerConfig and the selected signaling operations. TURN credentials are temporary and must not be embedded in browser code or durable logs. A third-party or self-hosted TURN design requires equivalent credential rotation, regional resilience, capacity testing, abuse controls, and packet-path review.

The client application authenticates through the enterprise identity provider. Amazon Cognito can broker identity for customer or workforce applications when appropriate, while AgentCore Identity can support authorized agent access to downstream services. The back end exchanges the human identity for short-lived, purpose-limited access. The browser does not receive broad AWS credentials.

The agent is packaged as a container compatible with AgentCore Runtime. It exposes the expected invocation path, creates the WebRTC peer connection, performs signaling, obtains TURN configuration, accepts only approved codecs, forwards media to the selected Bedrock model or pipeline, handles interruptions, and emits structured control events. For voice, the official AWS tutorial provides a reference architecture using a browser, AgentCore Runtime in a VPC, Kinesis Video Streams TURN relay, and a Bedrock speech-to-speech model.

Codec choices should follow tested model and client support rather than assumption. Opus is a strong voice codec because of its interactive audio characteristics, but forward-error-correction behavior depends on configuration and transport conditions. VP8 or H.264 may be appropriate for video depending on browser, licensing, hardware acceleration, model input, and regional service support. The system should negotiate the smallest approved codec set and reject unexpected media types.

The application creates an AgentCore Memory resource or selects managed memory through an AgentCore harness where that development model is used. Short-term memory captures selected interaction events. Long-term memory strategies are configured only for approved categories such as summaries or workflow checkpoints. Namespaces include actor and session scope to reduce collisions and prevent cross-user retrieval. IAM policies restrict creation, reading, writing, and deletion. The application never uses a client-supplied actor identifier without mapping it to the authenticated principal.

AgentCore Gateway exposes approved APIs, Lambda functions, or MCP-compatible tools to the agent. A read-only market-data lookup, policy retrieval, case creation, supervisor contact, and voluntary pause request may be appropriate initial tools. Trade execution, limit modification, customer communication, and sensitive personnel actions should remain outside the first release. Every tool has a schema, timeout, maximum result size, idempotency rule, policy check, and audit event.

The generation path uses a Bedrock model approved for the selected modality. Bedrock Guardrails can help enforce content boundaries, but guardrails do not replace application authorization. The system prompt defines the agent as a process assistant, not a clinician, therapist, investigator, or decision maker. It forbids inference of mental state from biometrics and forbids conclusions about fitness, honesty, intent, or personality.

Amazon CloudWatch captures technical telemetry such as connection success, TURN allocation failures, packet loss, jitter, audio underruns, model time to first response, tool latency, memory latency, interruption success, and end-to-end percentile latency. AgentCore observability can trace agent reasoning and tool calls according to the configured controls. Sensitive transcript text, raw media, tokens, and tool payloads should not be copied indiscriminately into logs. CloudTrail records relevant administrative and data operations. Audit evidence goes to a protected S3 bucket with versioning, retention, encryption, and access monitoring.

The web client displays unmistakable media state. A visible indicator shows when the microphone or camera is active. Video is off by default unless the use case requires it. The user can stop media immediately. Recording, if any, requires a separate notification and cannot be hidden inside general terms. The interface provides transcript correction, memory review where appropriate, and a clear way to request deletion subject to legal retention duties.

The first stage is identity validation and session authorization. The client authenticates to the enterprise application. The application obtains a short-lived authorization for a specific AgentCore Runtime invocation and business purpose. The server resolves the actor identifier, session identifier, allowed tools, data classification, and permitted memory scope. Authentication proves who the principal is. Authorization establishes what the principal may do in this session.

The second stage is signaling. The browser acquires media only after visible permission and creates the WebRTC offer. The offer and ICE information travel through the selected signaling mechanism. In the official Kinesis Video Streams TURN tutorial, the application can use the AgentCore Runtime invocation endpoint to obtain TURN credentials and exchange signaling data, while Kinesis Video Streams supplies the managed relay. The agent returns an SDP answer and candidates compatible with the approved media configuration.

The third stage is connectivity establishment. The client and agent perform ICE connectivity checks using the configured TURN servers. Because TURN relay is required for AgentCore Runtime WebRTC, the design should not wait indefinitely for a direct path that the deployed architecture does not support. The client gathers candidates, establishes the relay path, and reports selected candidate-pair telemetry without exposing credentials.

The fourth stage is secure media activation. DTLS authenticates the WebRTC endpoints and establishes keying material for SRTP. The client begins transmitting audio and, when explicitly enabled, video. The agent application terminates the WebRTC connection and forwards decoded or appropriately transformed media to the approved model pipeline. The returned media follows the reverse path. Control events continue independently so that cancellation, tool approval, consent withdrawal, and session termination remain reliable.

The system should define a latency budget for every stage. Capture and browser processing consume part of the budget. TURN relay and network distance consume another part. Voice activity detection, codec buffering, model ingress, time to first model output, tool calls, synthesis, and playback consume the rest. The team measures median, 95th percentile, and 99th percentile performance under realistic packet loss and concurrency.

A sub-200-millisecond end-to-end target may be unrealistic for many full model interactions. The more useful objective is conversational quality. Barge-in should stop agent playback quickly. Short acknowledgements should arrive sooner than complex answers. Tool calls should announce progress rather than create silence. Video may operate at a lower frame rate than human conferencing because the model needs selected evidence frames, not cinematic continuity. The application can dynamically degrade from video and audio to audio only, then to text, while preserving the control session.

The desk should never confuse network smoothness with business correctness. A fast wrong answer is worse than a slower abstention. The latency budget must reserve time for authorization, policy checks, source validation, and required human confirmation.

The institution must assume that voice and video contain personal data and may contain biometric data, confidential business information, client information, material non-public information, and incidental data about bystanders. Purpose limitation begins at capture. A camera pointed at a broken device is different from a camera pointed continuously at an employee’s face. The safest architecture minimizes the field of view and duration.

Emotion recognition and mental-health inference should be prohibited by policy and architecture. The system should not derive emotional labels from facial expression, voice prosody, gaze, posture, pulse, typing rhythm, or combined behavior. It should not infer depression, anxiety, mania, substance use, deception, aggression, burnout, suitability, or future conduct. These are not appropriate outputs for a trading-risk assistant.

Voluntary self-report is different from inference. If a user states that they need support, the agent can respond to the statement. It should collect no more than necessary, avoid diagnostic language, and offer approved human channels. Sensitive self-disclosures should not automatically enter long-term memory. A default rule should exclude health-related statements from durable personalization unless a lawful, explicit, and separately governed purpose requires retention.

Managers should not receive individual “wellness dashboards.” Team-level process analytics should use minimum group sizes, suppress small cohorts, avoid biometric inputs, and be reviewed for proxy discrimination. No metric should feed compensation, promotion, discipline, access removal, or termination without a separate lawful process and meaningful human review. In many cases, the prudent choice is never to use conversational telemetry for employment decisions.

Model risk management should maintain an inventory of models, versions, modalities, prompts, tools, memory strategies, and known limitations. Testing includes accents, languages, speech impairments, background noise, interruptions, market terminology, sarcasm, partial sentences, and code-switching. The team measures transcription error, inappropriate completions, false tool proposals, memory contamination, cross-session leakage, and differential performance. A red team attempts prompt injection through speech, visible text, documents shown to the camera, and tool output.