Strands Decider 2B vs Jev: a two-round file-sorting match
I gave a local 2B decision model and the Jev API the same job: clean up a messy folder by actually moving files. Round 1 was a tie. Round 2 was not.
AWS released Strands Decider 2B through Strands Labs on October 1, 2026. It does not write text. You hand it a list of options, and it points at one of them and returns a probability for each.
Jev, announced by TypeSafe AI on September 15, 2026, is the same kind of model, offered as an API.
The Strands Decider server accepts requests in the same /v1/systemone format as Jev. The repository even notes that a benchmark built for Jev runs against it unchanged. So I could send the same question and the same input to both and watch them fight.
I didn't want a quiz. I wanted them to do a job where a wrong answer has consequences, so I made each model clean up a messy folder. Whatever folder a model picks, the file really moves there.
Two questions going in:
- Can a 2B model on my laptop keep up with Jev?
- If there is a gap, which files expose it?
The fighters
| Strands Decider 2B | Jev | |
|---|---|---|
| From | AWS (Strands Agents) | TypeSafe AI |
| Runs on | My machine (Apple M1 Mac, 16 GB) | TypeSafe AI's API |
| Inside | Qwen3.5-2B-Base with the text-generation head removed and a small pointer head added, about 1.9B parameters | I did not check |
| License / cost | Apache-2.0, no per-call cost locally | Paid API |
| Model used | strands-decider-2B-hobson-v19 | jev-latest |
Both return the same three things: the chosen option, a probability for every option, and a confidence score.
Rules of the match
The test data is in Japanese. Both models handled Japanese without any extra setup.
Each file had to go into one of six folders. Both models received the same question with the same description for every folder:
1
2
3
4
5
6
7
8
9
10
11
12
QUESTION = {
"type": "choice",
"instructions": "このファイル(filename と content_preview)を、家のパソコンのどのフォルダに片付けるのがよいか。ファイル名だけでなく中身も見て判断する。",
"criteria": {
"経理": "お金の出入りの証拠になる書類。請求書・領収書・レシート・見積書・経費精算・注文確認など。何の支払いかは問わない。",
"仕事": "勤め先の業務で作った・受け取った資料。企画書・議事録・打ち合わせメモ・営業資料・顧客リスト・社内発表資料など。",
"写真": "写真やスクリーンショット、アイコンなどの画像ファイル。",
"学習": "自分の勉強のための資料。資格試験の対策ノート・語学・講座のメモ・論文・読書メモ・演習問題など。",
"手続き": "役所・税務署・年金機構・保険会社などとの公的な手続きや契約の書類。届出・住民票・確定申告・保険の更新案内など。",
"ゴミ箱候補": "消してよい不要なファイル。中身が空のファイル、一時ファイル、ロックファイル(~$で始まる)、OSが自動で作る管理ファイルなど。",
},
}In English, the six folders are Accounting (receipts, invoices, quotes, order confirmations), Work, Photos, Study, Paperwork (government, tax, insurance), and Trash candidates (empty files, temp files, lock files, OS metadata). The Trash candidates description explicitly says "empty files". Keep that in mind for round 2.
For each file, the input was the file name plus the first 300 characters of its content. Empty files were sent with a note saying the file is empty (0 bytes), and binary files with a note saying the content is not readable, plus the byte count.
One rule is enforced by my script, not by the models:
1
2
dest = ans["choice"] if ans["confidence"] >= THRESHOLD else HOLD # THRESHOLD = 0.6
shutil.move(str(p), ws / dest / p.name)If confidence is below 0.6, the file goes to an On hold folder for a human to check. The correct folder for every file was fixed in advance. Each request was timed on the client side, so Jev's numbers include the network round trip.
Round 1: an ordinary messy folder
36 files, the kind of mess most of us have: invoices, a taxi receipt, meeting minutes, a sales deck, IMG_2041.jpg, exam notes, a residence certificate, .DS_Store, Thumbs.db, a Word lock file starting with ~$.
Eight of them needed the content to sort correctly. "メモ.txt" (memo.txt) is actually a drugstore receipt. "無題.docx" (untitled.docx) is meeting notes.
| 2B | Jev | |
|---|---|---|
| Moved correctly | 36/36 | 36/36 |
| Moved wrongly | 0 | 0 |
| On hold | 0 | 0 |
| Median confidence | 0.96 | 1.00 |
| Median response time | 1418 ms | 185 ms |
A draw. Both sorted every file correctly, and the resulting folders were identical.
The only differences were confidence and speed. Jev returned 1.00 for 24 of the 36 files. Its least confident calls were photos named only by number (IMG_2041.jpg at 0.62, IMG_2042.jpg at 0.64), where the extension was the only clue. 2B ranged from 0.66 to 0.99.
No winner yet, so I wrote a nastier round.
Round 2: names that lie
11 files where the name points one way and the content points another. The content decides the correct folder.
| File name (English) | Actual content | Correct folder |
|---|---|---|
| 請求書の書き方_簿記3級.md (how to write an invoice, bookkeeping exam) | Bookkeeping practice problems | Study |
| 写真.txt (photo.txt) | City tax assessment notice | Paperwork |
| IMG_3301.jpg | A Starbucks receipt, as text | Accounting |
| 旅行の思い出.docx (trip memories) | A business travel expense report | Accounting |
| ゴミ.txt (trash.txt) | Notes on a client's contract terms | Work |
| temp.xlsx | Department budget plan | Work |
| TOEIC_公式問題集.pdf (TOEIC practice book) | Empty, 0 bytes | Trash candidates |
| 確定申告.md (tax return) | Study notes about how tax returns work | Study |
| 保険.pdf (insurance) | An insurance premium payment receipt | Accounting |
| 住民票_写し(1).pdf (residence certificate copy (1)) | Empty, 0 bytes | Trash candidates |
| 手続き書類.pdf (paperwork) | Python decorator review notes | Study |
| 2B | Jev | |
|---|---|---|
| Top answer correct | 8/11 | 10/11 |
| Moved correctly | 7 | 10 |
| Moved wrongly | 2 | 0 |
| On hold | 2 | 1 |
| Median confidence | 0.88 | 0.96 |
| Median response time | 1414 ms | 183 ms |
Jev takes this round on points. Both models read through trash.txt, photo.txt and trip memories correctly. Four files made the difference:
| File | Correct | 2B | Jev |
|---|---|---|---|
| TOEIC practice book (empty) | Trash candidates | Study (0.78), moved | Trash candidates (0.96) |
| Residence certificate copy (1) (empty) | Trash candidates | Paperwork (0.42), on hold | Trash candidates (0.95) |
| temp.xlsx | Work | Work (0.51), on hold | Work (0.76) |
| tax return notes | Study | Paperwork (0.71), moved | Paperwork (0.44), on hold |
2B trusted the name of an empty file
The input for the TOEIC file said the file was empty (0 bytes). 2B still filed it under Study, the folder the name suggests, with 0.78 confidence. That is above my 0.6 threshold, so the safety net did not catch it.
The other empty file went the same direction: 2B's top answer was Paperwork, matching the name. This time confidence was 0.42, so the file stopped in On hold instead of being misfiled.
Jev put both empty files in Trash candidates at 0.95 or higher. It looks as if the "empty files" line in the folder description outweighed the file name.
Both missed the tax notes
The tax return file is a set of study notes, so its correct folder was set to Study. Both models chose Paperwork.
What separated them was confidence. 2B moved it at 0.71. Jev was at 0.44, so the file waited in On hold for a human.
To be fair, this file could reasonably go under Paperwork. One borderline file can flip a result, so I would not read much into this one.
What the two rounds say
Back to the two questions.
On ordinary files, 2B kept up. It went 36 for 36 in round 1, running on my laptop with no network and no API bill.
The gap appeared on files whose names lie, especially empty ones. 2B followed the file name, sometimes with high confidence. A confidence threshold only protects you when the model is unsure. When it is confidently wrong, the file moves anyway.
Jev was also faster in this setup: about 0.18 seconds per file including the network, against about 1.4 seconds for 2B on an M1.
If I were using 2B for this job, I would let it handle the bulk of a folder and catch empty files in code before asking the model. Checking for 0 bytes does not need a model at all.
Caveats
- The correct folders were fixed before the run, and the tax notes file is open to interpretation
- 36 and 11 files are small samples, and each file was judged once
- 2B ran on an Apple M1 Mac with 16 GB, using MPS. Another 2B server was loaded but idle on the same Mac during the run
- The official 2B figure is a 153 ms median on an M3 Pro for questions under 300 tokens. My inputs were longer (six folder descriptions plus file content) on an older chip, so the numbers are not directly comparable
References
- Strands Decider repository: https://github.com/strands-labs/strands-decider
- Strands Decider 2B model card: https://huggingface.co/StrandsAgents/strands-decider-2B-hobson-v19
- Introducing Strands Decider (AWS Strands Labs, October 1, 2026): https://strandsagents.com/blog/introducing-strands-decider/
- Jev explainer (DataCamp, announcement date and API format): https://www.datacamp.com/blog/system-one-models-jev
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article