QA Handbook · Test Execution
Test Execution
Running the planned tests, recording what actually happened, reporting defects, confirming fixes and telling the team where quality stands.
Test Execution turns prepared test cases into evidence. Each result is tied to a build and environment, and every failure is investigated before anyone draws a conclusion.
Run a test cycle
Check readiness, run in risk order, record results, report defects, confirm fixes and share status. The example follows the logout runs in the STLC guide.
Check Readiness
Confirm that the build, environment, data and tools are ready, so failures point to the product rather than to the setup.
What to include
- Entry criteria from the Test Plan.
- The exact build and environment under test.
- A short smoke check of the most important paths.
QA Engineer’s tips
- Record the build number before the first test; results without it cannot be compared.
- If the smoke check fails, stop and report instead of running the full suite.
Example · Secure User Logout
Readiness · RUN-LOGOUT-001
- Build
- logout-demo.1 deployed to QA / Staging
- Environment
- ENV-LOGOUT-001 · Chromium · Authentication API · isolated test database
- Data
- Synthetic accounts with active sessions; fault injection switch available
- Smoke check
- Log in and open /account: passed, so the full run can start
Common mistake Starting execution on a half-deployed build and reporting environment problems as product defects.
Run Tests in Risk Order
Execute manual and automated tests in an order that finds the most serious problems as early as possible.
What to include
- Critical and high-priority cases first.
- Automated checks in CI alongside manual and exploratory testing.
- Notes on anything unexpected, even when the test passes.
QA Engineer’s tips
- Follow the steps as written; if you deviate, record what you did.
- Use short exploratory sessions around failures to learn more about the problem.
Example · Secure User Logout
Execution order · RUN-LOGOUT-001
- TC-LOGOUT-002 — session replay (Critical) runs first.
- TC-LOGOUT-001, 003, 004 and 005 (High) follow.
- An exploratory session checks Back, refresh and cache behaviour around the failure.
Common mistake Running cases in document order, so the most important risk is checked last when time is short.
Record Results & Evidence
Compare actual and expected results and record each outcome with a status, the build and evidence that someone else can review.
What to include
- Status per case: Pass, Fail, Blocked or Not Run.
- Actual result in factual words, with screenshots, logs or traces.
- Build, environment and date of the run.
QA Engineer’s tips
- Remove tokens, passwords and personal data from evidence before sharing it.
- Blocked and Not Run are not passes; report them separately.
Example · Secure User Logout
| Test case | Status | Evidence |
|---|---|---|
| TC-LOGOUT-001 | Pass | EV-LOGOUT-001 · expected behaviour observed |
| TC-LOGOUT-002 | Fail | EV-LOGOUT-002 · old session accepted by /api/profile |
| TC-LOGOUT-003 | Pass | EV-LOGOUT-003 · expected behaviour observed |
| TC-LOGOUT-004 | Pass | EV-LOGOUT-004 · expected behaviour observed |
| TC-LOGOUT-005 | Pass | EV-LOGOUT-005 · expected behaviour observed |
4 Pass + 1 Fail + 0 Blocked + 0 Not Run = 5 planned cases.
Common mistake Writing “works” as the actual result, which gives no evidence when the outcome is questioned later.
Report Defects
Investigate each failure, then report confirmed product defects with enough detail for a developer to reproduce and prioritise them.
What to include
- A title that states the wrong behaviour.
- Severity (impact) and priority (urgency).
- Environment, steps to reproduce, expected and actual results, evidence.
QA Engineer’s tips
- First rule out a test mistake or an environment problem.
- Describe what you observed; leave the diagnosis to the investigation.
Example · Secure User Logout
BUG-LOGOUT-001
- Title
- Protected profile API accepts the previous session token after successful logout
- Severity / priority
- Critical · P1, fix before release
- Environment
- ENV-LOGOUT-001 · logout-demo.1 · Chromium / Authentication API
- Steps
- 1. Log in. 2. Log out successfully. 3. Send GET /api/profile with the previous session token.
- Expected
- 401 Unauthorized without protected data (AC-AUTH-002).
- Actual
- 200 OK with synthetic profile data, although the UI shows sign-in.
- Evidence
- EV-LOGOUT-002, sanitised request and response; token redacted.
Common mistake Reporting “logout is broken” without steps, build or evidence, so the defect bounces back as “cannot reproduce”.
Confirmation & Regression Testing
Confirm that the fix works and check that the change has not broken related behaviour.
What to include
- Confirmation testing: rerun the failed case on the fixed build.
- Regression testing: rerun cases selected by change impact.
- Updated defect status with the confirming evidence.
QA Engineer’s tips
- Use the original steps and data so the confirmation is comparable.
- Ask the developer what changed; it guides which regression checks to run.
Example · Secure User Logout
RUN-LOGOUT-002 · logout-demo.2
- Confirmation
- TC-LOGOUT-002 passes: GET /api/profile returns 401 without data; /account requires sign-in.
- Regression
- TC-LOGOUT-001, 003, 004 and 005 selected by impact analysis; all pass.
- Defect
- BUG-LOGOUT-001 closed with evidence EV-LOGOUT-002-R2.
- Latest snapshot
- 5 / 5 unique cases passed on logout-demo.2.
Common mistake Closing a defect after the developer says it is fixed, without rerunning the test on the new build.
Report Progress & Status
Keep the team informed during the cycle and finish with a clear summary of results, open risks and a recommendation.
What to include
- Daily progress: executed, passed, failed, blocked and not run.
- Open defects and blockers, with owners.
- A completion summary with remaining risks and limitations.
QA Engineer’s tips
- Raise critical failures immediately, not at the next meeting.
- Give a recommendation based on evidence; the release decision belongs to the agreed owners.
Example · Secure User Logout
Status update · Daily Stand-up
Day 8: 5 / 5 cases executed. BUG-LOGOUT-001 (Critical) fixed on logout-demo.2 and confirmed; regression passed. No open Critical or High defects in scope. Not covered: other browsers and other device sessions, as agreed in the plan.
This summary feeds the Test Cycle Closure report described in the STLC guide.
Common mistake Reporting only a pass percentage, without saying what was not tested or which risks remain.
Execution statuses and the defect lifecycle
| Status | Meaning |
|---|---|
| Pass | Actual result matches the expected result on the recorded build. |
| Fail | Actual result differs from the expected result; investigate before reporting. |
| Blocked | The test could not run because a prerequisite failed, such as a broken environment. |
| Not Run | The test has not been attempted yet. |
Defect lifecycle · example workflow
- Open: QA reports BUG-LOGOUT-001 with evidence.
- Triaged: the team agrees severity and priority.
- In Progress: the developer fixes the session check.
- Ready for confirmation: logout-demo.2 is deployed.
- Closed: QA confirms the fix and regression passes.
Teams name these states differently; what matters is that a defect is closed only after confirmation.
Retesting vs. regression testing
| Retesting (confirmation) | Regression testing | |
|---|---|---|
| Goal | Show that a reported defect is fixed | Find unintended effects of a change |
| Scope | The failed test case | Related or affected test cases |
| Logout example | TC-LOGOUT-002 on logout-demo.2 | TC-LOGOUT-001, 003, 004 and 005 |