Jev + Lambda: System one gateway with typed decisions, PII redaction and confidence gates

lambda-decide is a typed client plus a one-file CDK gateway for Jev (or any /v1/systemone backend)

Saddam Azad

Building the open source PaaS - thunder.so

Published Oct 6, 2026

Last modified Oct 6, 2026

Most of the "AI" in backend code is a classifier in a chat costume. You have a support ticket, a webhook payload or a log line, and you want to know which queue it belongs in, how bad it is, or whether the customer is asking for their money back. You do not want a paragraph. You want a label and a number you can branch on.

That is the job of a System One decision model. Jev is the first hosted one I have used: you hand it some state and a set of typed questions, and it answers with probabilities instead of prose. There is no generation step, so there is nothing to parse, nothing to hallucinate into your schema, and a confidence you can threshold.

This post is about the plumbing around it. lambda-decide is two things:

A small typed client for Node. choice, score and noul questions in, narrowly typed results out.
A one-file CDK gateway that puts a Lambda Function URL in front of the model. It holds the API key, redacts PII, caches, writes an audit log and applies confidence gates.

By the end you will have deployed the gateway, called it with a signed request, and branched on confidence in code. Part 2 of this series benchmarks Jev alternatives hosted on Lambda. This part is the product walkthrough.

Why a gateway at all?

Jev's weights are closed. You cannot run it yourself, so every decision is an HTTPS call to someone else's server. That is fine, but if every service in your account calls it directly you get a few problems at once:

Every service holds the upstream API key.
Raw customer text, emails and card numbers leave your account unredacted.
Identical requests are paid for repeatedly, with no shared cache.
Nobody can answer "how many decisions did we make yesterday, and how confident were they?"

A gateway fixes all four in one place. The Lambda holds the key, scrubs the state, remembers recent answers, logs numbers (never content) and can apply your confidence policy server-side. Callers only need permission to invoke it.

One scope note before the code. Jev is the only backend this project is tested against. The unit of compatibility is the /v1/systemone protocol, not a model, so other servers that speak it can be pointed at with one config change. Hosting those on Lambda, and what that costs, is part 2.

What you are building

1
2
3
4
5
6
7
8
9
10
11
12
13
 your service                 your AWS account                                 upstream
┌────────────┐  SigV4   ┌────────────────────────────────────────────┐      ┌───────────────┐
│ lambda-    │ ───────► │ Lambda Function URL (AWS_IAM)              │      │ Jev           │
│ decide     │          │   └─ handler                               │      │ (TypeSafe /   │
│ client     │ ◄─────── │        1. validate body + questions        │ ───► │  OpenRouter / │
└────────────┘          │        2. redact state                     │      │  Vercel)      │
                        │        3. cache lookup (per environment)   │ ◄─── │               │
                        │        4. call upstream (key from SSM)     │      └───────────────┘
                        │        5. parse answers, apply gates       │
                        │        6. write audit line (no state)      │
                        └────────────────────────────────────────────┘
                                   │
                                   └── SSM SecureString: /lambda-decide/jev/api-key

Layer	Lives in	Responsibility
Typed client	your service (`lambda-decide`)	Question builders, typed results, retries, local `gate()`, abort signals
Gateway	your AWS account (`lambda-decide/cdk`)	Key custody, redaction, cache, audit log, server-side gates, spend ceiling
Upstream	TypeSafe, OpenRouter or Vercel	The model

The client in five minutes

Install it, with or without the CDK pieces:

1
2
bun add lambda-decide                                         # client only
bun add lambda-decide aws-cdk-lib constructs aws-cdk tsx      # client + gateway

It needs Node 22.18+ and is ESM only. The package ships TypeScript sources as they are, so consume it with a TypeScript-aware runtime (bun, tsx). aws-cdk-lib and constructs are optional peer dependencies, so the client-only install stays small.

Three kinds of question

1
2
3
4
5
6
7
8
9
import { decide, backends, choice, score, noul } from "lambda-decide";

const d = decide({ backend: backends.typesafe() }); // reads TYPESAFE_API_KEY

const r = await d.ask(ticketText, {
  team:     choice("Which team?", { billing: "charges, refunds", technical: "bugs, outages" }),
  severity: score("How severe?", ["cosmetic", "workaround exists", "blocking"]),
  refund:   noul("Is the customer asking for money back?"),
});

choice picks exactly one option. The option keys are returned verbatim and typed as a union.
score places the state on an ordered scale. List the levels from least to most. Levels come back as "0", "1", "2" and so on.
noul is the probability that a yes/no proposition holds.

One ask is one round trip for all questions, so asking three things costs one request, not three.

Answers are typed from your questions

You do not cast anything. The result type is derived from the question definitions you pass in:

1
2
3
4
5
6
7
8
9
const t: "billing" | "technical" = r.answers.team.choice;   // ok
// @ts-expect-error "sales" is not one of your option keys
const bad: "sales" = r.answers.team.choice;

r.answers.team.probabilities.billing;     // number
r.answers.severity.score;                 // probability-weighted level, e.g. 1.15
r.answers.severity.legend["2"];           // "blocking"
r.answers.refund.verdict;                 // boolean (probability \>= 0.5)
r.usage?.inputTokens;

Each result also carries the full probability vector and three comparable confidence numbers, which the next section explains. A choice result looks like this (values illustrative):

1
2
3
4
5
6
7
{
  type: "choice",
  choice: "billing",
  probabilities: { billing: 0.94, technical: 0.06 },
  top: 0.94, margin: 0.88, certainty: 0.67,
  vendorConfidence: 0.94,   // only if the backend sent one
}

Limits and untrusted input

Limit	Value
Options per `choice`	2 to 255
Levels per `score`	2 to 10
Questions per request	32
Question name length	64 chars
Instruction or description text	4,000 chars

LIMITS exports the numbers. parseQuestions validates untrusted JSON, such as a request body, into typed questions and throws a DecideError with code invalid_request and a message that names the offending question:

1
2
3
4
5
6
7
8
9
import { LIMITS, parseQuestions, DecideError } from "lambda-decide";

try {
  const questions = parseQuestions(JSON.parse(rawBody).questions);
} catch (e) {
  if (e instanceof DecideError && e.code === "invalid_request") {
    // 'question "team": choice needs 2-255 options, got 1'
  }
}

The gateway uses the same function on incoming requests, so a malformed question is rejected with a 400 before any paid upstream call is made.

You can also cancel a call:

1
const r = await d.ask(ticketText, questions, { signal: AbortSignal.timeout(5_000) });

Backends

Any server that implements POST /v1/systemone works. The presets cover the three places Jev is hosted:

Call	Base URL	Key from env	Model id
`backends.typesafe()`	`https://api.typesafe.ai`	`TYPESAFE_API_KEY`	`jev-1.13`
`backends.vercel()`	`https://ai-gateway.vercel.sh/typesafe`	`AI_GATEWAY_API_KEY`	`typesafe-ai/jev`
`backends.openrouter()`	`https://openrouter.ai/api`	`OPENROUTER_API_KEY`	`typesafe/jev-1.13`
`backends.gateway(url, { fetch })`	your deployed gateway	none	`"gateway"`

These are three hosting routes to the same Jev weights, not three models. The ids differ because each provider names versions its own way. Pinning another version is an override, not a fork:

1
backends.typesafe({ model: "jev-1.14" });

backends.gateway() is the one you will use against your own deployment, and we will get to it after the deploy.

Deploy the gateway

1. Create the key parameter

This is the only secret step in the whole flow:

1
2
aws ssm put-parameter --name /lambda-decide/jev/api-key --type SecureString \
  --value "$(cat ~/.jev-api-key)" --overwrite --profile prod --region us-east-1

The naming convention is /lambda-decide/\<backend\>/api-key. It only has to be unique. The key never enters your code, your .env or the Lambda's configuration. The function reads and decrypts it at runtime.

2. Point `.env` at it

1
cp .env.example .env

1
2
3
# .env (gitignored): a POINTER to the key, never the key
DECIDE_API_KEY_SSM=/lambda-decide/jev/api-key
DECIDE_BACKEND=typesafe          # typesafe | openrouter | vercel

There are three ways to choose the upstream:

1
2
3
4
5
6
7
8
9
10
# 1) A preset (openrouter is the default if you set nothing)
DECIDE_BACKEND=typesafe

# 2) A preset pinned to a version: how you upgrade Jev without editing code
DECIDE_BACKEND=typesafe
DECIDE_BACKEND_MODEL=jev-1.14

# 3) Any server speaking /v1/systemone. The model is required alongside the URL.
DECIDE_BACKEND_URL=http://localhost:8000
DECIDE_BACKEND_MODEL=von-1.0.0

DECIDE_BACKEND_URL without DECIDE_BACKEND_MODEL fails the deploy on purpose, because a URL alone does not say which model to ask for. An exported environment variable always beats the file. If your backend needs no key, leave DECIDE_API_KEY_SSM unset and the gateway runs keyless; the construct adds a synth warning so that choice is deliberate.

3. The whole CDK app is one file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
// handler.ts
import { decideApp, DecisionGateway } from "lambda-decide/cdk";
import { apiKeySsmFromEnv, backendFromEnv, loadDotEnv } from "lambda-decide/cdk";

// npx cdk deploy --app "npx tsx handler.ts" --profile prod
loadDotEnv();

const apiKeySsm = apiKeySsmFromEnv();

decideApp("DecisionGateway", (stack) =\> {
  new DecisionGateway(stack, "Decisions", {
    backend: backendFromEnv(),                                 // DECIDE_BACKEND, or URL + MODEL
    ...(apiKeySsm ? { apiKeySsmParameter: apiKeySsm } : {}),   // a pointer to the key, never the key
    redact: ["email", "card"],                                 // scrubbed before state leaves your account
    cache: { ttlSeconds: 300 },                                // identical requests skip the upstream call
    // reservedConcurrency: 20,                                // hard spend ceiling
  });
});

There is no build step. The CDK app is the TypeScript you wrote and tsx runs it. decideApp creates the App and Stack, takes account and region from your current CDK CLI credentials, and calls app.synth().

4. Bootstrap and deploy

1
2
npx cdk bootstrap # once per account/region
npx cdk deploy --app "npx tsx handler.ts" --profile prod

This will output the Lambda URL as stack output.

What just got created

Resource	Settings
Lambda function	`nodejs24.x`, arm64, 256 MB, 30 s timeout, handler `entry.handler`
Log group	one-month retention, destroyed with the stack
Function URL	`AWS_IAM` auth by default
IAM statement	SSM read access on the one parameter's ARN, nothing else
Stack output	`GatewayUrl`

Get the URL

1
2
aws cloudformation describe-stacks --stack-name DecisionGateway --profile prod \
  --query "Stacks[0].Outputs[?OutputKey=='GatewayUrl'].OutputValue" --output text

The output key is explicit (GatewayUrl) because CDK's default logical ids carry a hash suffix that you cannot grep for.

Every construct option

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import { Duration } from "aws-cdk-lib";
import { RetentionDays } from "aws-cdk-lib/aws-logs";

const gateway = new DecisionGateway(stack, "Decisions", {
  backend: { preset: "typesafe", model: "jev-1.13" },   // or { baseUrl, path?, model }
  apiKeySsmParameter: "/lambda-decide/jev/api-key",
  auth: "iam",                                          // "none" makes the URL public and warns at synth
  memorySize: 256,
  timeout: Duration.seconds(30),
  reservedConcurrency: 20,
  cache: { ttlSeconds: 300, maxEntries: 500 },
  redact: ["email", "phone", "card", "\\bACCT-\\d{6}\\b"],   // built-ins plus a custom regex source
  maxBodyBytes: 256 * 1024,
  logRetention: RetentionDays.ONE_MONTH,
});

Option	Default	Notes
`backend`	`{ preset: "openrouter" }`	Set it explicitly when you mean something else
`apiKeySsmParameter`	none	Omitting it warns at synth, and is correct for a keyless backend
`auth`	`"iam"`	`"none"` warns: anyone with the URL spends your upstream quota
`redact`	none	`"email"`, `"phone"`, `"card"`, or any case-insensitive regex source
`cache`	off	In memory, per execution environment
`reservedConcurrency`	none	A hard ceiling on spend; needs spare account concurrency
`maxBodyBytes`	256 KiB
`memorySize` / `timeout` / `logRetention`	256 MB / 30 s / one month	The gateway holds no model

Call the gateway

The Function URL uses IAM auth, so requests must be signed with SigV4. That is the right default: it means a leaked URL is not a leaked quota.

With curl

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# short-lived credentials from your profile
eval "$(aws configure export-credentials --profile prod --format env)"

cat \> request.json \<\<'JSON'
{
  "state": "Hi, I was charged twice for my March invoice. Please refund the duplicate.",
  "questions": {
    "team":   { "type": "choice", "instructions": "Which team?",
                "criteria": { "billing": "charges, refunds", "technical": "bugs, outages" } },
    "refund": { "type": "noul", "instructions": "Is the customer asking for money back?" }
  },
  "gate": { "team": { "act": 0.9, "review": 0.6 }, "refund": { "act": 0.9, "review": 0.6 } }
}
JSON

curl -sS "${GATEWAY_URL}v1/systemone" \
  --aws-sigv4 "aws:amz:us-east-1:lambda" \
  --user "$AWS_ACCESS_KEY_ID:$AWS_SECRET_ACCESS_KEY" \
  -H "x-amz-security-token: $AWS_SESSION_TOKEN" \
  -H "content-type: application/json" \
  -d @request.json

The handler ignores the path, so the bare Function URL and /v1/systemone both work. Only POST is accepted; anything else gets a 405.

The response is the upstream { answers, usage, model, id } plus gates and a gateway block (shape exact, values illustrative):

1
2
3
4
5
6
7
8
9
10
11
{
  "id": "…",
  "model": "jev-1.13",
  "answers": {
    "team":   { "choice": "billing", "probabilities": { "billing": 0.94, "technical": 0.06 }, "confidence": 0.94 },
    "refund": { "noul": 0.97 }
  },
  "usage": { "input_tokens": 212, "output_tokens": 0 },
  "gates": { "team": "act", "refund": "act" },
  "gateway": { "cached": false, "latency_ms": 143 }
}

Two details worth knowing. First, a model field in the request is ignored on purpose: the model is fixed by deployment config, so a caller cannot ask your gateway to spend your key on something else. Second, answers is the raw upstream shape. The client turns it into typed results.

From TypeScript

backends.gateway() targets a deployed gateway. You give it the bare URL from the stack output and a fetch that signs requests:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import { AwsClient } from "aws4fetch";
import { fromNodeProviderChain } from "@aws-sdk/credential-providers";
import { decide, backends, choice, noul } from "lambda-decide";

const creds = await fromNodeProviderChain()();
const aws = new AwsClient({
  accessKeyId: creds.accessKeyId,
  secretAccessKey: creds.secretAccessKey,
  ...(creds.sessionToken ? { sessionToken: creds.sessionToken } : {}),
  service: "lambda",
  region: process.env["AWS_REGION"] ?? "us-east-1",
  retries: 0,                       // lambda-decide already retries; do not stack another layer
});

const d = decide({
  backend: backends.gateway(process.env["GATEWAY_URL"]!, {
    fetch: (input, init) =\> aws.fetch(input, init),
    retries: 0,                     // see "Retry stacking" below
    timeoutMs: 20_000,
  }),
});

const r = await d.ask(ticketText, {
  team:   choice("Which team?", { billing: "charges, refunds", technical: "bugs, outages" }),
  refund: noul("Is the customer asking for money back?"),
});

Session credentials expire. In a long-lived process, build the signing client per call or refresh it from the provider.

Why `backends.gateway` has no `apiKey`

1
2
3
backends.gateway(url, { fetch: signedFetch });                  // ok
// @ts-expect-error apiKey is omitted from GatewayOptions
backends.gateway(url, { fetch: signedFetch, apiKey: "sk-…" });

A bearer token and a SigV4 signature both write the same authorization header. If backends.gateway() read an API key from your environment, a stray OPENROUTER_API_KEY would overwrite your signature and every call would fail with a puzzling 403. So apiKey is excluded from the options type and discarded at runtime for untyped callers. The gateway already holds the upstream key, so the caller sends no upstream credentials at all.

Letting another role call it

1
2
3
4
import { Role } from "aws-cdk-lib/aws-iam";

const consumer = Role.fromRoleArn(stack, "Consumer", "arn:aws:iam::123456789012:role/ticket-router");
gateway.grantInvoke(consumer);   // lambda:InvokeFunctionUrl on this function

What the gateway does

Redaction

Redaction runs on state before it leaves your account, recursively through strings, arrays and objects. Built-ins run in a fixed order, email then card then phone, because card digits would otherwise be eaten as a phone number. Any other string is compiled as a case-insensitive regex.

Spec	Matches	Replacement
`"email"`	addresses	`[REDACTED:email]`
`"card"`	13 to 19 digits, optional spaces or dashes	`[REDACTED:card]`
`"phone"`	international and local formats	`[REDACTED:phone]`
any other string	your regex	`[REDACTED:custom]`

1
2
3
state in    "Hi, I'm jane.doe@example.com. Card 4111 1111 1111 1111, call +1 (555) 010-9999, ref ACCT-483920."
upstream    "Hi, I'm [REDACTED:email]. Card [REDACTED:card], call [REDACTED:phone], ref [REDACTED:custom]."
            (redact: ["email", "card", "phone", "\\bACCT-\\d{6}\\b"])

Structured state is scrubbed field by field:

1
2
3
4
await d.ask(
  { customer: { email: "jane.doe@example.com" }, messages: ["call me on +1 (555) 010-9999"] },
  { urgent: noul("Does the customer need a same-day response?") },
);

Be clear-eyed about what this is. It is regex-based, so it reduces exposure and does not guarantee removal. It applies to state and not to your question instructions. Long digit runs such as order numbers can over-match as phone numbers. If you handle regulated data, treat this as a safety net on top of not sending it in the first place, and test your own patterns against real samples.

Cache

Identical requests skip the upstream call. The cache key is a SHA-256 over a canonical, sorted-key serialisation of the model, the redacted state and the questions. Gates are not part of the key, so changing your bands does not cause misses.

It is an in-memory LRU with a TTL, per execution environment. It is not shared between Lambda instances, so the hit rate depends on traffic concentrating on warm environments. Two identical requests that arrive before the first one finishes will both miss. That is a fine trade for a cache whose job is to absorb retries and repeated payloads, and a bad one if you expected a shared cache; for that, put something external in front.

The response tells you what happened:

1
2
curl … -d @request.json | jq .gateway   # { "cached": false, "latency_ms": 143 }
curl … -d @request.json | jq .gateway   # { "cached": true,  "latency_ms": 3 }

It also gives you stability. Jev's probabilities vary slightly between identical calls, so the cache makes repeated identical requests return the same answer, which is useful when a downstream system compares results.

Audit log

One JSON line per request: request id, backend and model, whether it was cached, latency, input tokens, and per question the type, top, margin and band. Never the state, never the instructions.

1
{"msg":"decision","requestId":"c3f1…","backend":"typesafe","model":"jev-1.13","cached":false,"latencyMs":143,"inputTokens":212,"questions":{"team":{"type":"choice","top":0.94,"margin":0.88,"band":"act"},"refund":{"type":"noul","top":0.97,"margin":0.94,"band":"act"}}}

Because it is structured, CloudWatch Logs Insights can answer the questions from earlier directly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# latency and cache hit rate
fields @timestamp, cached, latencyMs
| filter msg = "decision"
| stats count() as n, avg(latencyMs) as avg_ms, pct(latencyMs, 95) as p95_ms by cached

# how often does "team" escalate? (requires callers to send a gate)
fields questions.team.band as band
| filter msg = "decision"
| stats count() by band

# errors by code
fields code, status
| filter msg = "error"
| stats count() by code, status

Errors are logged too, with the code, status, message and underlying cause, while the caller only ever receives a generic message. Upstream response bodies are never leaked to callers.

Spend control

reservedConcurrency is a hard ceiling on concurrent executions and therefore on spend, as long as your account has spare concurrency to reserve. Requests above the ceiling are throttled, which the client sees as a 429 and treats as retryable.

auth: "none" makes the URL public, and the construct warns about it at synth: anyone with the URL can spend your upstream quota. Only use it behind your own auth, and set a ceiling.

Key rotation

1
2
aws ssm put-parameter --name /lambda-decide/jev/api-key --type SecureString \
  --value "$(cat ~/.jev-api-key.new)" --overwrite --profile prod --region us-east-1

The handler caches the key per execution environment for five minutes, so rotation needs no redeploy and takes effect within that window. The IAM grant covers that one parameter only.

Confidence: what to do with the number

Backends disagree about what "confidence" means, so comparing them by their own field is a trap. Every result therefore carries three numbers computed locally from the returned probabilities. The backend's own value, if it sent one, is kept untouched as vendorConfidence.

Metric	Meaning	choice / score	noul
`top`	probability of the winner	max p	max(p, 1 − p)
`margin`	winner minus runner-up	p₁ − p₂	\|2p − 1\|
`certainty`	1 − normalised entropy; 0 is uniform, 1 is fully concentrated	yes	yes

1
2
3
4
5
6
7
import { metricsOf } from "lambda-decide";

metricsOf([0.7, 0.2, 0.1]);
// { top: 0.7, margin: 0.5, certainty: ≈0.27 }

metricsOf([0.5, 0.5]);       // a noul sitting at p = 0.5
// { top: 0.5, margin: 0, certainty: 0 }

top is the simple one. margin punishes a close runner-up, which matters when two options are nearly tied. certainty looks at the whole distribution and is useful when you have many options and the probability is smeared across several.

Gating

gate() buckets an answer into one of three outcomes by comparing a metric to two thresholds:

1
2
3
4
r.gate("team", { act: 0.9, review: 0.6 });                        // metric defaults to "top"
r.gate("team", { act: 0.6, review: 0.3, metric: "margin" });
r.gate("team", { act: 0.5, review: 0.2, metric: "certainty" });
r.gate("team", { act: 0.9, review: 0.6, metric: "vendor" });      // falls back to top if none was sent

metric \>= act returns "act": run it automatically. metric \>= review returns "review": confirm it, or get a second opinion. Anything lower returns "escalate": send it to a human or a stronger model. Then the code is just a branch:

1
2
3
4
5
switch (r.gate("team", { act: 0.9, review: 0.6 })) {
  case "act":      return route(r.answers.team.choice);      // automatic
  case "review":   return queueForConfirmation(ticketId, r); // human or second opinion
  case "escalate": return sendToHumanTriage(ticketId);       // the model does not know
}

Bands are validated before any paid call, so a typo cannot cost money:

1
2
r.gate("team", { act: 0.5, review: 0.9 });
// DecideError(invalid_request): bands need 0 \<= review \<= act \<= 1

For a noul, treat a value near 0.5 as "don't know" and let the gate catch it:

1
2
3
4
5
const asking = r.answers.refund;
const band = r.gate("refund", { act: 0.9, review: 0.6 });

if (asking.verdict && band === "act") issueRefundDraft(ticketId);
else if (band === "escalate") flagAsUncertain(ticketId);

A caution that belongs here and not in a footnote: calibration holds on average, not per answer. A model that says 0.9 is right about nine times in ten across many answers, not necessarily on this one. Pick your bands from your own labelled examples, because a different model, or a different domain, will be calibrated differently. Choosing bands from data is a topic for its own post. Here I only want you to see the mechanism.

Errors, retries and what goes wrong

Every failure is a DecideError with a code:

`DecideError.code`	HTTP from gateway	Retried by the client?
`invalid_request`	400	no
`auth`	502	no, never
`rate_limited`	429	yes
`upstream`	502	yes (502, 503, 504, 529)
`timeout`	504	yes
`network`	502	yes
`malformed_response`	502	no

Unexpected exceptions return a 500 with a generic message. Handling them on the caller side looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import { DecideError } from "lambda-decide";

try {
  await d.ask(ticketText, questions);
} catch (e) {
  if (!(e instanceof DecideError)) throw e;
  switch (e.code) {
    case "invalid_request":    /* fix the caller */ break;
    case "rate_limited":
    case "timeout":
    case "upstream":
    case "network":            /* e.retryable === true: back off or queue */ break;
    case "auth":               /* the gateway cannot read or use its key: page the saddam-azad */ break;
    case "malformed_response": /* the upstream returned something unparseable */ break;
  }
}

Retry stacking

The client retries twice by default on retryable statuses and network errors, with jittered backoff, honouring retry-after. The gateway's own upstream call does the same. Put them together and one caller request can become up to 3 × 3 = 9 upstream attempts, each with its own timeout. A signing library with its own retry layer, such as aws4fetch's AwsClient, adds a third multiplier unless you turn it off.

Let one layer retry. The simplest policy is to keep the gateway's retries, which see the real upstream, and turn the client's off:

1
backends.gateway(url, { fetch: signedFetch, retries: 0 });

Timeout arithmetic

Each upstream attempt has a 15 second timeout by default, and the function itself has 30 seconds. A slow upstream can therefore exhaust the function timeout in the middle of a retry and surface to the caller as an opaque 502 rather than a clean 504. The rule to follow is that timeoutMs × (retries + 1) must stay below the function timeout, and the caller's own deadline should sit a little above the function timeout.

Troubleshooting

Symptom	Likely cause	What to check
403 from the Function URL	Unsigned request, wrong region or service in the signature, or a bearer token overwriting the signature	Sign with service `lambda`. Use `backends.gateway`, never `apiKey`
502 "gateway upstream authentication failed"	SSM parameter missing, not a SecureString, role lacks access	The gateway log names the SDK exception class
400 naming a question	Question validation failed	The message says which question and why
400 "body exceeds … bytes"	Request over `maxBodyBytes`	Raise the limit, or send less state
429	Upstream rate limit, or the reserved concurrency ceiling	Audit log `code` and `status`
502 after about 30 seconds	Upstream slower than the function timeout allows	See timeout arithmetic above

Testing without an upstream

createHandler takes its dependencies as arguments, so the whole gateway runs in-process with a fake backend, a fake SSM, a fake fetch, a clock and a logger. No AWS account, no network, no key:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import { test } from "node:test";
import assert from "node:assert/strict";
import type { APIGatewayProxyEventV2 } from "aws-lambda";
import type { Backend } from "lambda-decide";
import { createHandler } from "lambda-decide/handler";

const fake: Backend = {
  name: "fake",
  model: "fake-1",
  async evaluate() {
    return { answers: { refund: { noul: 0.93 } }, model: "fake-1" };
  },
};

test("redacts, calls upstream and gates", async () =\> {
  const handler = createHandler({
    config: { backend: { preset: "typesafe" }, redact: ["email"] },
    backend: fake,
    log: () =\> {},
  });

  const event = {
    isBase64Encoded: false,
    requestContext: { requestId: "t-1", http: { method: "POST" } },
    body: JSON.stringify({
      state: "mail me at jane.doe@example.com",
      questions: { refund: { type: "noul", instructions: "Is the customer asking for money back?" } },
      gate: { refund: { act: 0.9, review: 0.6 } },
    }),
  } as unknown as APIGatewayProxyEventV2;

  const res = await handler(event);
  assert.equal(res.statusCode, 200);
  assert.equal(JSON.parse(res.body ?? "{}").gates.refund, "act");
});

The other seams are ssmSend, which stands in for SSMClient.send(GetParameterCommand), fetch, which replaces the upstream transport, and now, which makes cache TTLs and the key refresh deterministic. The repo's own suite runs on node --test with native TypeScript, so there is no test build either.

Limits worth knowing before you adopt it

Jev is the only backend tested. Other /v1/systemone servers should work and are untested here.
Probabilities vary a little between identical calls. Gate on bands, not exact values.
The cache is per execution environment, not shared across instances.
none_of_the_above options are not modelled. If you need an abstain option, express it with a noul or with the escalate band.
A hand-written Backend object is client-only. The construct serialises its backend as { preset } or { baseUrl, model } and the Lambda rebuilds it from that, so only those two forms can be deployed through DecisionGateway.
A backend that returns no probabilities still parses, but margin and certainty flatten to 0 and every gate fires. A label with no probabilities is a typed answer with useless confidence.
Redaction is regex-based. It reduces exposure; it is not a PII guarantee.
Calibration is your job. The gateway applies the thresholds you choose. It cannot tell you whether they are right.

Tearing it down

1
2
npx cdk destroy --app "npx tsx handler.ts" --profile prod
aws ssm delete-parameter --name /lambda-decide/jev/api-key --profile prod --region us-east-1

The log group is destroyed with the stack. The SSM parameter is yours, so it is deleted separately.

What is next

Everything above assumed a hosted upstream. The interesting question for the next post is what happens when you swap it: the same gateway, pointed at an open-weight System One model that you host yourself on a container-image Lambda.

The swap itself is two lines of .env:

1
2
DECIDE_BACKEND_URL=https://models.internal.example.com
DECIDE_BACKEND_MODEL=von-1.0.0

The part that is not two lines is what the model costs you to run: cold starts, memory sizing, x86 versus arm64 for fp32 inference, quantisation pitfalls and cost per million decisions against hosted Jev. I will measure those instead of guessing, and publish the scripts.

The honest framing going in is that hosted Jev is likely to be cheaper per decision at short state lengths, and the case for self-hosting is data residency, open weights and control, not savings. Part 2 will show where that holds and where it does not.

The code is at github.com/saddam-azad/lambda-decide . If you deploy it, I would like to hear how your confidence bands turned out.

# aws-cdk # aws-lambda # ai # ai-agents # ai-engineering

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Jev + Lambda: System one gateway with typed decisions, PII redaction and confidence gates

Saddam Azad

Why a gateway at all?

What you are building

The client in five minutes

Three kinds of question

Answers are typed from your questions

Limits and untrusted input

Backends

Deploy the gateway

1. Create the key parameter

2. Point .env at it

3. The whole CDK app is one file

4. Bootstrap and deploy

What just got created

Get the URL

Every construct option

Call the gateway

With curl

From TypeScript

Why backends.gateway has no apiKey

Letting another role call it

What the gateway does

Redaction

Cache

Audit log

Spend control

Key rotation

Confidence: what to do with the number

Gating

Errors, retries and what goes wrong

Retry stacking

Timeout arithmetic

Troubleshooting

Testing without an upstream

Limits worth knowing before you adopt it

Tearing it down

What is next

2. Point `.env` at it

Why `backends.gateway` has no `apiKey`