AI can write a test before you finish your coffee or tea. It can inspect a user story, suggest scenarios, generate Playwright code, and explain why a test failed far quicker than humans can.
That’s impressive, especially if your morning routine already includes arguing with a flaky login test.
The uncomfortable part is that an AI-generated test can look perfectly reasonable while missing the behavior that matters most. The test may check the button, the page transition, and the success message while quietly ignoring what happens when a customer enters an invalid account number or when a downstream service responds slowly.
AI is already absorbing large portions of test authoring and maintenance. QA engineers and SDETs will ultimately remain responsible for deciding what deserves to be tested, whether the resulting coverage is meaningful, and whether a green pipeline gives the organization evidence it can trust.
The details of this shift become clearer when we compare the traditional workflow with the one emerging around AI-assisted testing, and when we examine what teams need to make that workflow dependable. Read on to see why AI will transform QA jobs but won’t entirely absorb them.
The Part Nobody Can Delegate
Writing test code has always been only one part of quality engineering. A lot of the harder work involves strategic reasoning: Understanding the business process behind a requirement and identifying where that process could fail. It also requires deciding which risks deserve coverage and recognizing when a green test has validated the wrong condition.
AI can accelerate several activities around that work. An agent can propose scenarios from a user story, generate setup code, convert recorded behavior into test steps, and help troubleshoot familiar implementation problems. It can also reduce the time engineers spend writing repetitive Playwright code, especially when the expected behavior is clear and the application is well understood.
The quality of the generated output still depends on human-in-the-loop workflows. For instance, an agent might:
- Misunderstand a requirement
- Omit an important scenario
- Choose an unreliable locator
- Produce a test that passes while checking the wrong thing
- Misinterpret failures in the test, application, or environment
- Use unreliable or environment-specific test data
- Miss synchronization issues, causing flaky tests
… And without human review, teams may not detect these errors before it’s too late. A large collection of plausible-looking tests can create the appearance of coverage without delivering much confidence.
QA engineers therefore assess the tests themselves, not merely their execution results. They determine whether a test reflects the requirement, whether its failure would reveal something meaningful, and whether the result could be explained to a developer, release manager, customer, or auditor.
A Day in the Life Before AI
The difference becomes clearer when we follow one QA engineer through the same piece of work before and after AI enters the workflow.
For the example, imagine a QA engineer working on an online lending application. A new feature allows customers to upload additional income documents during an application review.
Let’s start with the pre-AI workflow. Before AI-assisted testing, the QA engineer would clarify the requirement with the product manager and developer and identify scenarios for valid files, unsupported formats, oversized uploads, interrupted transfers, duplicate documents, and permissions. She would then create test data and write the Playwright scripts. The QA engineer would also add assertions, connect the tests to the pipeline, run them across supported browsers, and investigate failures.
When the interface changed, she would update selectors and test steps manually. If several tests used the same upload flow, she might repair the same logic in multiple places. She would also separate product defects from environment problems and summarize the evidence for the release decision.
AI’s Place in QA
Now, let’s look at what happens when the QA engineer uses an AI-assisted workflow for the same process.
First, it’s worth noting what AI cannot do. AI doesn’t possess the years of experience and accumulated knowledge one can only obtain by working as a QA engineer for years. After all, the results of a difficult QA decision may be present in an agent’s training data, but the multitude of human thoughts that led to that decision likely aren’t present.
The QA engineer’s ultimate job is to ensure quality outcomes, not only write tests. In this sense, QA engineers are validation architects: they employ discernment, judgment, scrutiny, and informed decision-making that only years of experience can inform. Because of that, QA engineers are also some of the only people in a business who can consistently ensure quality outcomes. Developers and AI can’t ensure quality outcomes as proficiently as QA engineers because they do not have the lived experience that QA engineers have.
With an AI-based workflow, AI takes care of a lot of the busy work associated with being a QA engineer, while the engineer gets to leverage their experience to take on more strategic, governance, and supervisory roles that AI couldn’t do on its own.
So, using AI, the QA engineer could use AI by giving an agent the requirement and asking it to propose coverage. The agent could draft scenarios, generate Playwright code, create fixtures, suggest assertions, and produce variations for different browsers or data conditions. If the QA engineer recorded the upload process, the agent could turn that behavior into an initial test. During maintenance, AI could identify likely locator changes, explain a failure, and propose a repair.
The QA engineer’s role would thus shift toward validating the work produced by the agent. She would confirm that the scenarios cover the actual risks, reject tests that confuse a successful upload with a valid underwriting decision, review the generated code, and check that each assertion proves the intended behavior. She would decide which tests belong in a pull request, nightly run, or release gate.
When a test failed, she would investigate what the failure meant rather than simply asking an agent to make the test green. She would approve a repair only when the new behavior matched the application and the requirement.
The typing may decrease; the responsibility does not.
In practice, that means QA engineers spend less time producing every individual test step and more time governing the quality, coverage, and meaning of the entire validation process.
From Agent Session to Test Asset
That distinction creates a practical requirement for teams adopting AI: fast generation needs structure around it. Without that structure, every useful first draft can become another one-off script that only its author understands.
Leapwork Play is designed for teams that want AI-assisted Playwright authoring alongside governance, reuse, and auditability. It imports and reuses existing Playwright tests, repositories, and AI models, allowing teams to preserve their investment without a rewrite.
Play provides reusable components, role-based access controls, approvals, and a full audit trail around Playwright automation. Visual debugging helps teams understand what happened during a run, while self-healing helps tests stay current as applications change. Reuse and self-healing can reduce maintenance effort by 50 to 70 percent compared with raw AI output.
Play also follows the Leapwork Continuous Validation Platform’s Deterministic by Design approach. AI contributes during authoring, where speed and adaptability are useful. Once a test enters execution, it runs as fixed, explicit logic without a model call improvising during the run. That supports repeatable execution, explainable outcomes, and evidence teams can review.
QA engineers still decide what quality means. Leapwork Play helps turn the agent’s first draft into a governed, reusable test asset the wider organization can own.
