E2E QA Automation

QA Bottlenecked by Repetitive Basic Checks

When I joined, three POs, including me, shared feature verification. For each release version, QA alone usually took a week and sometimes two. Deciding what to check depended on individual memory and experience, while login, reading, creating, and updating data consumed hours of repeated manual work.

Even a change to a single feature could affect other features and workflows in different ways. Despite the time spent on QA, it was difficult to check every impact, and post-release hotfixes were frequent. Repeating the same basic checks left less time to explore exceptions and regressions in changed features. That was the workflow I wanted to change.

QA Tracking Connected from Google Sheets to Jira

I first connected verification records to manage QA work in one place. From Google Sheets, where I wrote and managed test cases, I used Google Apps Script to call the Jira API so that recording a bug would create a Jira issue.

Previously, every bug found during QA meant manually writing another Jira report: which feature was being checked, which screen it occurred on, the actual behavior (AS-IS), and the expected behavior (TO-BE). I changed this so that details entered in Google Sheets were sent to a Jira ticket, reducing repeated switching between the two tools and duplicate entry. I also polled the Jira API to synchronize ticket status in real time and made content summaries available in the sheet. This made it easier to track tickets being handled by developers from Google Sheets alone.

We could now manage verification scope and bug-fix status in one place, but the next release still meant clicking through the same screens again. Connecting the tools reduced duplicate work without removing the time spent on basic checks. As long as people kept repeating those checks, there was little time left to examine new policies and exceptions.

Building E2E Test Infrastructure from 0→1 for Repetitive QA Automation

To address that limit, I initiated the E2E infrastructure effort to turn recurring checks into executable code. I first scoped automation around login, saving and reading data, and essential contract, schedule, and collection workflows. People would continue to judge new policies and exceptions, while automation took on repeated checks of behavior that should already work.

There was no existing automation foundation to extend, so I designed the execution structure from scratch. Using Playwright and TypeScript, I built shared login, navigation, input, test-data creation, and cleanup helpers, then connected scenario ordering, failure handling, CI execution, and reporting. The work included both writing tests and building the infrastructure to run them again for the next verification cycle.

Integrating Cross-Platform User Workflows into a Single E2E Verification Flow

Admin creates contracts, schedules, and work requests; the Driver app performs assigned collections; Admin then checks the resulting history. Passing isolated screens is not enough when data must remain correct across an entire business process.

The test suite covers Admin, Customer, and Driver, with separate, reusable login states for each role. For workflows between Admin and Driver, Playwright project dependencies and teardown connect setup, execution, verification, and cleanup. When Driver verification became more urgent, I extended coverage to follow Admin-created data through field execution rather than checking Driver screens in isolation.

A Reproducible E2E Environment Under the Same Conditions, Every Run

Assuming the test server already contains the right contracts and schedules makes checks depend on mutable data. The pipeline creates its own business records and passes identifiers between stages. A mutex protects shared-file writes, while normal and fallback cleanup paths reduce leftovers between runs.

A success toast alone is insufficient, so saved states and amounts are also cross-checked through APIs. I select coverage from cases, historical Jira bugs, code changes, and connected workflows. More recently, I use the RAG platform I built to gather this context, review it myself, and define role, state, exception, and regression cases. Recurring checks are implemented as test code and run in CI; browser QA complements automation where policies and combinations need human judgment.

Expanding E2E Automation with Per-PR Isolated k3s Environments

I extended E2E verification so developers could check basic functionality in small, feature-level PRs before deploying to the test server. They could review the results on their PR and fix problems before deployment.

Checking PR code before deploying it to the test server required a separate environment to run the applications and DB. I deployed Frontend, Backend, and DB as containers on the in-house Xeon k3s cluster and used the existing Helm chart to create the verification environment.

A branch name alone can resolve to different code if new commits arrive during a build. I configured the pipeline to validate and pin the Front PR SHA, resolve Backend to an exact commit, and tag images and the DB snapshot per run. This made results traceable to the specific code-and-data combination tested.

I gave each run a separate namespace and DB, and configured Playwright to connect to the application for that PR rather than the test server. I passed runtime settings through Secrets and blocked fork requests and stale PR payloads before they reached self-hosted runners.

Test Reports and Logs That Preserve Failure Context

The container must remain available until test reports and logs have been copied. I added a step that saves the exit code and waits for report collection before removing the Pod. Pods with failed report copies and failed test environments remain available for diagnosis.

Existing CI merges results from each platform and shares them in Slack. The k3s pipeline posts HTML, JSON, and JUnit reports, test and Backend logs, Pod state, and events to the Front PR. It distinguishes test failures from environment-setup failures, removes successful environments, and retains failed ones for a limited investigation period.

I verified the complete run from image builds and isolated deployment through tests, report upload, and PR reporting. I aligned browser and Backend timezones for date-sensitive schedule tests and corrected state handling so report preparation would not be treated as a timeout.

+31% QA Throughput with One QA Owner, Post-deploy Bugs Down from 7.0% to 3.9%

When I joined, three POs spent one to two weeks on QA. I now own QA alone and complete pre-release verification in about three days on average.

E2E runs take roughly one hour, checking core contract, schedule, and Driver workflows first. With fewer repeated interactions to perform by hand, I can spend more time on changed policies, exceptions, and effects on connected workflows. Google Sheets and Jira keep discovered bugs and fix status linked through re-verification.

There were more features to verify. Comparing regular releases before E2E automation (2.13.0–2.14.2) and after (2.15.0–2.16.1), verified tickets rose from 712 to 992, up 39%. I handled most QA in the post-E2E period alone.

Average QA volume — tickets divided by actual QA days, excluding hotfixes — rose from 22.3 to 29.2 per day, about 31% more, meaning more features checked within the same QA time.

The post-release bug ratio also fell from 7.0% to 3.9%. I combined recurring E2E regression checks with hands-on verification of new features and exceptions, plus Jira follow-up. This reduced the need for other POs to join repetitive checks while allowing me to verify more features.

Baseline Checks in Code, QA Focused on New Features and Exceptions

I started with checks written in Excel, connected their records, automated core behaviors, and built the environments that execute them. Keeping repeatable checks in code freed time for new features and exceptions, helping me verify more features while owning QA alone. I also built PR-isolated k3s environments to check basic functionality before deployment to the test server, with results and failure details available to developers on the PR.