AI-Powered UI Test Automation with Kiro CLI and Playwright MCP

Go from a URL to dozens of passing Playwright tests in three prompts. Kiro CLI explores your app, writes the scripts, runs them, and fixes its own locator errors.

Chellapparimanam, Sakthivel

Published Apr 13, 2026

Last modified Apr 13, 2026

Co-authored with

Karthik Thirugnanasambandam

Natasha

Writing UI tests is tedious. You open the app, click around, note what works, figure out the selectors, write the test code, debug the locators, fix the flaky bits, and repeat. For any reasonably sized app, that's easily a full day of work.

What if you could hand an AI assistant a URL and have it explore the app, generate test cases, write Playwright scripts, run them, and fix failures — all in a single conversation?

That's exactly what Kiro CLI with Playwright MCP enables. In this post, we walk through the complete pattern: from a URL to a passing Playwright regression suite — without writing a single test manually.

What is Kiro CLI?

Kiro CLI is an AI coding assistant that runs in your terminal. It can read and write files, execute commands, and — through Model Context Protocol (MCP) servers — interact with external tools like browsers, APIs, and databases.

What is Playwright MCP?

Playwright MCP is an MCP server that gives Kiro the ability to control a browser. It exposes browser actions — navigation, clicking, typing, taking screenshots, reading accessibility trees — as tools that Kiro can invoke during a conversation.

Together, Kiro CLI and Playwright MCP turn your terminal into an AI-powered testing workbench.

What You Need

Kiro CLI — installed and configured (setup guide )
Playwright MCP — added as an MCP server in your Kiro configuration
A URL — any web application you want to test

No source code access required. No manual test writing. Just a URL.

The Pattern: Three Phases

The workflow follows three phases, each mapping to a role a human tester would play:

App Exploration — AI as the Tester
Test Case Generation — AI as the Test Analyst
Script Creation & Execution — AI as the Automation Engineer

Phase 1: App Exploration

Kiro uses Playwright MCP's browser tools to interact with the live application — just like a human tester would during exploratory testing.

How It Works

Playwright MCP gives Kiro the ability to:

Navigate to URLs — open pages, follow links
Take accessibility snapshots — read the full element tree (headings, buttons, inputs, labels, roles)
Click elements — buttons, links, radio buttons, checkboxes
Type into fields — text inputs, search boxes, forms
Capture screenshots — visual verification of page state

This means Kiro doesn't guess what your app looks like — it actually uses it.

What Exploration Looks Like

When you give Kiro a URL, it systematically:

Navigates to the page and takes an accessibility snapshot to identify all visible elements — headings, buttons, inputs, links, labels
Discovers navigation paths — clicks through menus, tabs, and links to find all pages and views
Interacts with functional elements — fills forms, selects options, clicks buttons, observes responses
Notes dynamic behavior — toast notifications, modals, loading states, validation messages, page transitions
Tests boundaries — tries empty inputs, disabled elements, edge cases it encounters naturally

The accessibility snapshot is key. It gives Kiro a structured view of the page — not just pixels, but semantic elements with roles, names, and states. This is far more reliable than screenshot-based approaches for understanding what's interactive and what's not.

When Source Code Helps (But Isn't Required)

If source code is available, Kiro can read it to supplement browser exploration:

Confirm hardcoded data (default values, pre-set states)
Understand validation logic (which fields are required, what order checks run)
Discover CSS patterns that affect interactivity (hidden inputs, custom-styled controls)
Identify timing values (animation durations, auto-dismiss timeouts)

But source code is optional. The browser exploration alone is enough to generate meaningful test cases for any publicly accessible web app.

Phase 2: Test Case Generation

With the app explored, Kiro generates structured test cases covering positive flows, negative scenarios, and edge cases.

Output Format

Test cases are saved as a CSV:

1
Test Case ID, Test Scenario, Test Type, Test Steps, Expected Result

This format is tool-agnostic — you can import it into any test management system, or use it as a reference for manual review.

Coverage Strategy

Kiro organizes test cases into natural categories:

Positive Tests — verifying things work:

Page loads with expected content
Navigation between pages and views
Form inputs accept valid data
Interactive elements respond correctly
Successful form submissions

Negative Tests — verifying things fail gracefully:

Empty or missing required fields
Invalid input (special characters, extremely long text, script injection)
Invalid URL paths
Rapid repeated actions (double-click, spam submit)
Page refresh mid-flow (state reset verification)

What the AI Catches That Humans Often Miss

Because Kiro reads both the UI and (when available) the implementation, it tends to catch:

Validation order quirks — which error message appears first when multiple fields are invalid
Whitespace handling — whether spaces-only input passes validation
Hidden element interactions — CSS-hidden inputs that need label clicks instead
Dynamic content — elements populated by JavaScript after page load
Infrastructure behavior — CDN error codes (403 vs 404) for invalid paths

Phase 3: Script Creation & Execution

This is where the pattern becomes powerful. Kiro doesn't just generate test scripts — it runs them, reads the failures, and fixes them in a tight loop.

Setup

Kiro creates a minimal Playwright config:

1
2
3
4
5
6
7
8
9
10
11
12
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './test_scripts',
  timeout: 30000,
  use: {
    baseURL: 'https://your-app-url.com',
    headless: true,
    video: 'on',
  },
  projects: [{ name: 'chromium', use: { browserName: 'chromium' } }],
});

And organizes tests into spec files grouped by feature area. The exact grouping may vary between runs — you can control it by specifying your preferred structure in the prompt (e.g., "group by page" or "one spec per test case"). In one run, it might look like:

1
2
3
4
5
test_scripts/
├── page_structure.spec.js     # Page load, content verification
├── interactions.spec.js       # Clicks, selections, form inputs
├── submissions.spec.js        # Form submission flows
└── negative.spec.js           # Error handling, edge cases

The Iterative Fix Cycle

Here's the core of the pattern. Kiro follows a generate → run → fix loop:

Generate — Write test scripts based on the test cases and exploration data.
Run — Execute the full suite with npx playwright test.
Analyze — Read the error output. Playwright's error messages include the failing locator, what it was waiting for, and a call log showing exactly what happened.
Fix — Apply targeted fixes based on the root cause.
Re-run — Verify the fixes and check for new issues.

This cycle typically repeats 1–3 times before the full suite passes. Each iteration takes seconds because Kiro can read error messages, understand the root cause, and apply precise fixes.

Common Issues Kiro Fixes Automatically

Selector mismatches — The initial selectors are based on the accessibility snapshot, but sometimes the actual DOM uses different attributes. Kiro adjusts:

1
2
3
4
5
6
// Initial attempt based on visible text
await page.locator('input[type="radio"][value="Developer"]').check();

// Fixed after reading actual HTML structure
await page.locator('label[for="cat1_opt1"]').click();

Strict mode violations — When a text selector matches multiple elements, Playwright throws a strict mode error. Kiro resolves with { exact: true }, .nth(), or more specific selectors.

Dialog handling — JavaScript alert() dialogs block Playwright's click(). Kiro learns to register the handler before the triggering action:

1
2
3
4
5
6
// Register handler first, then click
page.on('dialog', async dialog => {
  alertMsg = dialog.message();
  await dialog.accept();
});
await page.locator('#submit').click();

Timing issues — Dynamic content that isn't immediately available. Kiro adds appropriate waits or switches to waitForSelector patterns.

Video Recording

With video: 'on' in the config, every test execution is recorded as a .webm file. This gives you:

Visual proof of test execution for stakeholders
Debugging aid for failures without re-running
Documentation of actual app behavior at test time

Other options: 'retain-on-failure' keeps videos only for failed tests (saves disk space), or 'off' to disable.

The Complete Workflow

The pattern works in three prompts — start, create one, scale:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Prompt 1 (Start):
You:    "Here's my app: https://my-app.example.com — explore it and
         generate test cases, save to /ui-tests/test_cases.csv"

Kiro:   [navigates, clicks, snapshots, explores all pages]
        [generates test_cases.csv with 29 positive + negative cases]

Prompt 2 (Create one):
You:    "Create a Playwright test script for TC_001 and run it"

Kiro:   [installs Playwright, creates config]
        [writes TC_001.spec.js, runs it — passes]
        [updates CSV with execution results]
        "TC_001 passed in 357ms"

Prompt 3 (Scale):
You:    "Create Playwright test scripts for all test cases in the CSV
         so I can use them as an automated regression suite"

Kiro:   [generates spec files for all 29 test cases]
        [runs suite — 4 failures due to locator issues]
        [analyzes errors, fixes selectors]
        [re-runs — all passing]
        "29/29 tests passing in 14.5s"

This incremental approach lets you validate the pattern on a single test before committing to the full suite. Each prompt builds on the previous one's output.

Things to Know

Dynamic content — Pages with timestamps, ads, or personalized content may cause flaky assertions. Use toContainText over toHaveText for content that changes between runs, or scope assertions to stable elements.
Authentication — For apps behind login, provide credentials or a pre-authenticated session cookie. Kiro can handle login flows, but it adds steps to every test run.
Single-page apps — SPAs with client-side routing work well. Kiro detects URL changes and waits for content to settle before asserting.
Rate limiting — Running 30+ tests in parallel against a live app may trigger rate limits. Use workers: 1 in the Playwright config if you hit this.
Test maintenance — Generated tests are standard Playwright scripts. You can extend, refactor, or integrate them into your existing CI pipeline like any hand-written test.

Getting Started

Prerequisites

Node.js (v16 or higher)
Kiro CLI installed (kiro.dev )

Setup

Install Kiro CLI:

1
npm install -g kiro-cli

Add Playwright MCP to your Kiro configuration (.kiro/settings/mcp.json):

1
2
3
4
5
6
7
8
9
10
{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": [
        "@playwright/mcp@latest"
      ]
    }
  }
}

Start a conversation:

1
kiro-cli chat

Give it a URL:

1
2
Explore https://your-app.example.com and generate test cases,
then create Playwright scripts and run them.

Kiro handles the rest — exploration, test case generation, script creation, execution, and debugging.

Conclusion

The pattern is simple: give Kiro a URL, let it explore with Playwright MCP, generate test cases, write scripts, and iterate to green. It works because Kiro can switch between roles — explorer, analyst, coder, debugger — in the same conversation. The tests it produces are standard Playwright tests you can run in CI, extend manually, or hand off to your team.

The difference isn't the quality of the tests. It's the time it takes to get there.

Kiro CLI is an AI coding assistant from AWS. Playwright MCP setup instructions are in the Kiro documentation .

Appendix: The Three Prompts

These are the exact prompts used in the workflow described above. Adapt the URL, output paths, and coverage requirements to your app.

Prompt 1 — Explore and Generate Test Cases

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
# Test Case Generation for Web Application

## Objective
Generate a comprehensive test case suite for the web application by
exploring and analyzing the application at <your-app-url>

## Task Requirements

### 1. Application Exploration
- Use Playwright MCP to navigate and interact with the app
- Identify all available features, pages, forms, buttons, and user workflows
- Document the application's functionality and user interactions
- Note any input fields, validation rules, and interactive elements

### 2. Test Case Coverage
Generate at least 20 test cases that include:
- Positive scenarios: Valid inputs and expected user flows (minimum 60%)
- Negative scenarios: Invalid inputs, error handling, and edge cases (minimum 40%)
- Coverage across ALL identified features and pages
- Boundary value testing where applicable
- Input validation testing (empty fields, special characters, max length, etc.)
- Navigation and workflow testing
- UI element interaction testing

### 3. Output Format
Create a CSV file with the following structure:

Column Headers:
Test Case ID,Test Scenario,Test Type,Test Steps,Expected Result

Column Specifications:
- Test Case ID: Format as TC_XXX (e.g., TC_001, TC_002)
- Test Scenario: Brief description of what is being tested
- Test Type: Either "Positive" or "Negative"
- Test Steps: Numbered steps separated by semicolons
- Expected Result: Clear, measurable outcome

### 4. Deliverable
- File name: test_cases.csv
- Location: /ui-tests/test_cases.csv
- Encoding: UTF-8
- Format: Properly escaped CSV

## Success Criteria
- Minimum 20 test cases generated
- All visible features and pages covered
- Balanced mix of positive (12+) and negative (8+) scenarios
- Test steps are clear, sequential, and reproducible
- Expected results are specific and measurable
- CSV file is properly formatted and saved to specified location

Prompt 2 — Create and Validate One Test

1
2
3
4
Can you help me create a Playwright test script for test case TC_001
from the CSV file and save it in the /test_scripts directory? After
running the test, please update the CSV file with the test execution
results (status, actual result, date time of the execution and any notes).

Prompt 3 — Scale to Full Suite

1
2
Can you help me create Playwright test scripts for all test cases in
the CSV file so I can use them as an automated regression suite?

# kiro # integration-testing # ai-assisted # ai-engineering

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

AI-Powered UI Test Automation with Kiro CLI and Playwright MCP

Chellapparimanam, Sakthivel

What is Kiro CLI?

What is Playwright MCP?

What You Need

The Pattern: Three Phases

Phase 1: App Exploration

How It Works

What Exploration Looks Like

When Source Code Helps (But Isn't Required)

Phase 2: Test Case Generation

Output Format

Coverage Strategy

What the AI Catches That Humans Often Miss

Phase 3: Script Creation & Execution

Setup

The Iterative Fix Cycle

Common Issues Kiro Fixes Automatically

Video Recording

The Complete Workflow

Things to Know

Getting Started

Prerequisites

Setup

Conclusion

Appendix: The Three Prompts

Prompt 1 — Explore and Generate Test Cases

Prompt 2 — Create and Validate One Test

Prompt 3 — Scale to Full Suite