Lesson 14 / الدرس 14

Reading a red build / قراءة بناء أحمر

A failing automated test asks one question before any other: is the product wrong, or is the test wrong? Answering that quickly and honestly is what makes a tester trusted, and answering it lazily is how suites get abandoned.

الاختبار الآلي الراسب يطرح سؤالًا قبل كل شيء: أالمنتج خاطئ أم الاختبار؟ والإجابة عنه بسرعة وصدق هي ما يجعل المختبِر موثوقًا به، والإجابة عنه بكسل هي كيف تُهجَر المجموعات.

There are only four reasons a test is red, and telling them apart is most of the job. The temptation — especially under pressure before a release — is to assume the fourth, adjust the test until it passes, and move on. That is how a suite quietly stops testing anything, one small adjustment at a time.

The four reasons

Reason	How to tell	What to do
The product broke	Reproduce it by hand — it fails there too	Report it. This is why the suite exists
The product changed on purpose	The new behaviour is correct and intended	Update the test, and say so in the commit
The test was always wrong	It passed by luck, or asserted the wrong thing	Fix the assertion, and check what else it was hiding
The environment	Passes locally, fails only in the pipeline	Fix the setup, never the assertion

Row one is the only one where the answer is a bug report. Rows two and three are maintenance. Row four is the one people get wrong most often, by weakening an assertion to silence an environment problem.

Reproduce it by hand first

Before touching the test, do what it did, yourself, in a browser. It takes two minutes and it answers the question outright: if it fails for you too, the product is broken and you have a bug report — with steps, because the test just gave you them. If it works by hand, the test is wrong or the environment is, and now you know which half to look in.

// A failure worth reading, from Playwright:
//
//   1) sign-in.spec.js:12 > a wrong password is refused
//
//   Error: expect(locator).toHaveText(expected)
//     Locator:  getByTestId('form-error')
//     Expected: "Email or password is incorrect"
//     Received: "Invalid input"
//
//   attachments: trace.zip, screenshot.png, video.webm

// This is reason 1 or 2, and only a person can say which: the message CHANGED.
// If somebody rewrote the error text on purpose, update the test.
// If nobody did, the product regressed and this is a bug report, written for you.

A good failure names the locator, the expectation and what it actually got — and modern runners attach a screenshot, a video and a trace you can step through. Read those before forming a theory; most wrong diagnoses come from guessing at a failure whose evidence was sitting in the artefacts.

لا تُضعف توكيدًا قط لتجعل بناءً أخضر. فتغيير toHaveText إلى toBeVisible لأن الصياغة تتغير كثيرًا يحوّل فحصًا حقيقيًا إلى زينة، ويفعلها بلا أثر مرئي — فالاختبار ما زال موجودًا، وما زال ينجح، وصار لا يحرس شيئًا. فإن كانت الصياغة تتغير كثيرًا حقًا، فذلك حوار يُجرى لا توكيد يُليَّن.

Check yourself / اختبر نفسك

1. What is the first thing to do when an automated test goes red?

2. A test fails only in the pipeline and always passes locally. What should you NOT do?

3. Why record in the commit message that a test was updated for a legitimate change?