The New Reality: AI in the Pull Request

If you write tests for a living, or you write code and then dread writing the tests…, the past 18 months have been weird. Your AI coding assistant ships features, but the test suite hasn’t kept up. The pull request that took 40 minutes to review last year now takes two hours. The CI run that used to be a coffee break is now a lunch break. And somewhere in the queue, a flaky Selenium test is mocking you.

This article is for developers, SDETs, and quality engineers who are tired of reading about “AI revolutionizing testing” without anyone explaining what that looks like inside a “.github/workflows” folder. We’ll be specific. We’ll talk about SQAI Suite, the platform we build, but we’ll also tell you where it doesn’t fit, what it costs, and how skeptical you should be of the marketing numbers (including ours).

What “AI in the pull request” means

For most of us, PR is the moment of truth. It’s where code stops being a draft and starts being an obligation. It’s also where quality gates live: the checks that must turn green before main accepts the merge.

“AI in the pull request” doesn’t mean a chatbot dropping comments on your diff. It means three concrete things:

1. AI generates the tests that prove the new code does what the ticket said it should do.
2. AI pushes those tests back into the repo as a Pull Request, on its own branch, reviewable line-by-line, version-controlled like any other change.
3. AI runs as a quality gate: a required check inside GitHub Actions or Azure Pipelines that must pass before merge.

That’s the workflow SQAI Suite is built for. Our Virtual Test Engineers (VTEs), basically quality-trained AI agents, read your Jira/Azure Boards ticket, launches queries on other information sources like Confluences, Azure Wikis, MS Teams folders…, looks at your existing repo to understand your framework conventions, generates test cases and the automation code to run them, and opens a PR. Your CI pipeline executes those tests. The result either gates the human PR or unblocks it.

Why the feedback loop matters more than the model

Developers usually have a 10-minute rule: if the pipeline takes longer than that, you context-switch. By the time the build returns, the cost of re-engaging is higher than the wait itself. Slow pipelines cause developers to batch commits, which makes reviews larger, which makes everything worse, a pattern documented across recent CI/CD literature by Bytemonk.

Traditional test automation makes this loop slower in two ways:

• You write the tests by hand. Even with Playwright’s auto-waiting and trace viewer, a non-trivial E2E test costs you 30 to 90 minutes of authoring time. Multiplied by every user story…
• The tests you wrote yesterday break today. Test maintenance is a known time sink, and only a minority of organizations reach the 80%+ coverage they aim for, per “Mabl’s 2025 Testing in DevOps Report”, only 14% achieve that threshold.

SQAI Suite collapses the authoring time. Our Virtual Test Engineers pull context from your connected sources (Confluence, Jira, Azure Boards, your repo…) and produces test cases plus the executable Playwright/Cypress/Selenium/Pytest code in a single click.

 “Requirements Analysis, Test Cases,
Automation & Test Data
in minutes, not weeks.”

On HeadIT’s deployment with BrowserStack and Playwright, they reported “60% faster testing” end-to-end (SQAI blog, Stop Scripting. Start Delivering.).

The honest version: the model itself is not the bottleneck anymore. Context is. The reason AI-generated tests have historically been flaky and embarrassing is that the LLM was guessing your conventions instead of reading them. As our docs put it, SQAI reads your existing codebase, framework conventions, and folder structure within the repository to generate high-quality, maintainable automation scripts that match your team’s standards.

Quality gates, defined for AI-generated code

A quality gate is a binary, auditable check that either blocks or unblocks a merge. The traditional gates (like lints, type-checks, coverage thresholds, SonarQube quality gates, security scans…) were designed for human-written code. They are necessary but no longer sufficient when AI is writing a meaningful share of the diff according to DEV Community

The Sonar 2026 State of Code Developer Survey (1,149 respondents, October 2025) reported that 38% of developers say reviewing AI-generated code requires more effort than reviewing code written by their human colleagues, and developer overhead sits at roughly 24% of the work week regardless of AI usage. The author of a vibe-coded PR often can’t fully defend the line they prompted into existence. The gate must catch what review now misses.

A modern PR quality gate for an AI-augmented team looks like this:

1. Lint + type check (unchanged).
2. Unit + integration tests (unchanged, but now usually AI-augmented).
3. Coverage on changed lines (not the whole repo — that’s the trick to making this realistic).
4. Generated end-to-end tests for the changed behavior. This is the new layer SQAI provides.
5. Security scan (unchanged, but you now have an AI for that, Aikido Security).

The fourth layer is where most teams currently have a blind spot. The PR introduces new behavior; no human wrote tests for it; the existing suite doesn’t cover it.

How SQAI plugs into GitHub Actions

SQAI Suite connects to your GitHub repository. Once configured, our platform reads your existing test automation code, your framework, your locators, your Page Object Model conventions… and writes generated tests to a branch that it opens as a Pull Request. The connection allows “SQAI to push generated automation code (e.g., Playwright or Selenium scripts) directly to a specified branch in your repository, ready for review and execution,” and “all generated code is immediately part of your version control history via a Pull Request (PR).”

In your workflow, you wire it like any other check. A representative pattern:

Mark the generated-tests job as a required status check on the protected branch. The merge button stays grey until SQAI’s tests pass. That is the gate.

How it plugs into Azure Pipelines

For Azure DevOps shops the story is the same and the integration is first-class. SQAI connects to Azure Repos via a PAT with Code: Full scope, and pushes scripts “directly to a uniquely created branch in your repository as a Pull Request, ready for review and execution… linking back to the original test case by ID” (SQAI Azure Repos integration docs). It also reads from Azure DevOps Boards (for ticket context), Azure DevOps Test Plans, Azure DevOps Wiki, and supports Entra ID SSO for enterprise authentication.

A minimal azure-pipelines.yml step that fails the build when the generated suite fails:

Combine it with a branch policy that requires the build to succeed, and SQAI is now a quality gate in Azure DevOps the same way it is in GitHub.

How is this different from GitHub Copilot?

GitHub Copilot is a general-purpose code generator that needs a developer to prompt it, review each suggestion, and stitch the output into the right files, while SQAI Suite is a specialized QA agent that autonomously connects your ticket system, test management tool, and repo — then delivers a complete, framework-conformant test PR end-to-end without anyone driving the prompt.

How does SQAI Suite add value to GitHub Copilot?

SQAI Suite is an AI Software Quality Orchestrator, not an AI Assistant or Automation Tool, it gives GitHub Copilot the QA-specific tools and connections it doesn’t have out of the box, so Copilot stops guessing at tests and starts generating repo-aware, traceable test PRs grounded in your actual quality workflow.

But AI-generated tests are flaky, Are they?

This is the most honest objection in QA right now, and we should answer it directly.

The 2025 Stack Overflow Developer Survey (49,000+ respondents) found that 46% of developers said they don’t trust the accuracy of the output from AI tools, a significant increase from 31% the prior year and only 33% actively trust AI output. The reason is real: generic prompt-driven test generation produces tests that look right and miss the point. Practitioners we talk to have lived through the same four phases; excitement, experimentation, reality check, selective use and end up with measured skepticism.

So, what’s different about a context-grounded VTE in a PR workflow versus pasting your ticket into ChatGPT?

The agent reads your repo. It matches your locator strategy, your test data factories, your fixtures. The generated test compiles against your real code, not a hallucinated one.

It runs a self-correcting build loop before it ever opens the PR. When configured, as soon as errors are detected, the VTE automatically attempts to fix the code, iteratively improving the output.” That way Tests that don’t compile or don’t run are less likely to reach you, and the full PR is reviewable. This is the part marketing decks tend to skip: a human still merges. The point isn’t autonomy, it’s eliminating the boring 80% of authoring so SDETs and quality engineers can focus the strategic 20%, more time for critical thinking.

Will some tests still be wrong? Yes. But Qodo’s State of AI Code Quality found that “teams integrating AI review into their workflow see quality improvements jump to 81% compared to just 55% for similar teams without review.” The lesson is that AI generation works when it’s paired with a verification layer, not when it’s left to run alone.