When AI verification gets it wrong, retract the conclusion and recheck the evidence

When a check reports that an article title is missing, the immediate response may be to edit the article. But if the title is already present in the HTML and the checker is comparing the wrong representation, that response changes content that was never broken.

This site encountered that kind of false failure during its notes-output checks in October 2026. A title contained <button>. The HTML held the escaped text, but the check searched directly for the original title. It treated a failure to find that character sequence as a failure to output the title.

When I work with AI, what matters to me is not whether it immediately says, “Sorry, I was wrong.” It is whether it stops relying on the mistaken conclusion, preserves the evidence as it stands, and corrects the cause in a way that can be checked.

Retract the claim that went beyond the evidence

A check failing to find something is not the same as a page failing to output it. The comparison method still needs to be examined.

In this case, the failed result initially supported only one statement: the current search rule did not match. Concluding that the page lacked a title required inspecting the actual HTML, including the search target, the text representation, and the page structure.

If the AI has already described the result as missing article content, it should retract that judgment first. It should not keep planning changes on the assumption that the article is wrong, then relegate the faulty check to a footnote at the end.

Retracting a conclusion does not mean immediately replacing it with another guess, such as blaming a cache or a package version. A more reliable approach is to narrow the claim to what is known: the output check failed, the cause still needs comparison against the evidence, and the source article stays unchanged for now.

Reduce the mismatch to a readable example

This case can be reproduced with a very small snippet. The example below keeps the existing test fixture’s original wording, including 使用 (“use”); it is not the title of another actual article:

const title = '使用 <button> & links'
const html = '<h3>使用 &lt;button&gt; &amp; links</h3>'

html.includes(title) // false

Readers can see title text containing <button> and &, while this HTML string represents them as <button> and &. Searching for the unescaped title as a substring therefore gives a different result.

The site later changed this comparison to use Astro’s escapeHTML, converting the source title to the same text representation as the output before searching. Replaying the snippet with the helper in use when this note was written gives false for the original comparison and true for the escaped comparison. That demonstrates the difference for these inputs, not that every page is correct.

This minimal example makes the direction of the fix clear: adjust the verifier’s expected text representation, rather than remove a valid technical identifier from the title. Whether the article should use a link or a button is a separate question of HTML semantics; a false failure in the output check should not derail that discussion.

Valid cases must pass, and invalid ones must still fail

When fixing a check, it is easy to loosen the rule until this particular case turns green.

For example, accepting any occurrence of button might sidestep the escaping difference, but it would abandon the requirement to match the complete title. Decoding every HTML entity before comparison could also conflate that case with a title intended to display the literal text <button>.

The existing regression tests therefore cover more than a single passing example:

Input	Distinction the check must preserve
A title contains <button> or &, and the HTML escapes it correctly	Recognize that the text is present
The title itself is the literal string &lt;button&gt;	Do not confuse it with text that displays <button>
The HTML contains an actual button tag	Do not treat the element as the same literal text
The output contains a different title	It must still be rejected

These cases protect the original requirement. A completed fix does not merely make an error message disappear. It lets cases that should not have failed pass, while preserving the ability to reject real errors.

The helper also has a limited scope: it looks for a corresponding substring in serialized HTML; it is not a DOM parser or a sanitizer. On its own, it cannot prove that the text is in the correct h1, that the page has only one h1, or that the entire HTML document is safe. Those require separate structural and output checks.

Passing the test must not quietly change the requirement

If we had simply removed <button> from the article title, readers would have lost a meaningful identifier and the verifier’s problem would have remained. The same false failure could recur in the next title containing & or other text that needs escaping.

When a check fails, I first separate the possibilities: the product or content is wrong, the check is wrong, or the environment makes this result inconclusive. This is not a fixed process of choosing exactly one of three causes; several can coexist. The point is not to assume in advance that the content under test must be at fault.

A minimal reproduction isolates the problem. Afterward, the affected checks still need to be rerun and the actual output inspected. Changing a test expectation without revisiting the page originally flagged as wrong does not reconnect the fix to the result the user cares about.

A correction must replace the old conclusion

The report for this case could be written as follows. This is a writing example, not a transcript of the original conversation:

I previously treated the failed title check as missing article content. That conclusion does not hold. The actual difference is between the source text and its HTML-escaped representation. The source article remains unchanged; the comparison rule is corrected, with checks that still distinguish the correct text, literal entity strings, and an incorrect title. This fix does not remove the need to check the structure of the whole page.

This explanation makes clear that the earlier judgment has been withdrawn, identifies the actual cause, and states what the fix has not proved. If the full checks have not yet been rerun, the report should say so rather than prematurely adding, “Everything is working now.”

Facts, inferences, and unknowns is about reading a conclusion. This case goes one step earlier: when the check that produced the conclusion is wrong, correct the check instead of asking valid content to accommodate it.

One correction cannot prove that AI will not make mistakes again. What can remain is a reproducible difference, regression cases that protect the requirement, and an updated explanation that no longer encourages reliance on the old judgment.