On 18 June 2026, an AI coding agent audited my Bengali status site and reported 140 duplicate lines spread across five pairs of pages. Five days later a short Python script found zero. Every one of those duplicates was a false positive, and one of them came close to getting a page merged out of existence.
The agent was Claude Code, working through the site against the content rules I keep in the repository. Its report also caught a real problem in the same pass, and that's why this is worth writing down.
What did the AI audit report?
The audit checked all 40 collection pages on the site, and its duplicate-content section was specific. It named five pairs of pages that it said shared status lines word for word, with a count for each pair:
| Page pair | Shared lines reported (18 Jun) | Shared lines found (23 Jun) |
|---|---|---|
| dosti ↔ friendship | 34 | 0 |
| sad ↔ emotional | 39 | 0 |
| romantic ↔ love status for girlfriend | 32 | 0 (1 near-duplicate) |
| romantic ↔ WhatsApp status | 25 | 0 |
| attitude status ↔ attitude captions | 10 | 0 |
| Total | 140 | 0 |
Each finding came with a task. Three were de-duplication jobs: strip the shared lines from one page in each pair and write new ones to replace them. The fourth went further. For the dosti and friendship pair, the recommended fix was a merge, with a 301 redirect sending the dosti page into the friendship page for good.
Why AI audit false positives are hard to spot
An AI audit's false positive is a finding stated with the same confidence, format and precision as a true one. Nothing in this report read as uncertain. There were exact counts, a verification command printed next to each one, and a fix with a reason attached.
The worry behind it was also real. Two pages on the same site competing for the same searcher is a genuine SEO problem, and I'd been watching for it. A finding that confirms what you already suspect is the one you're least likely to check.
How did the re-check find zero?
The re-check pulled the actual text of every status line on both pages of each pair and compared them in Python, first as exact strings and then again after normalising the text. Every pair came back at zero exact matches. The romantic and girlfriend pages shared one near-duplicate, a single line worth rewording, not a cluster.
Git history settled the rest. None of the flagged pages had had its list of statuses edited since the site was first built, so nothing had been quietly fixed in between. The duplicates hadn't gone away. They'd never been there.
What actually went wrong?
The audit counted shared lines with two standard command-line tools, sort and comm, and both compare text using the computer's locale settings. GNU's manual says comm only works if both input files are sorted using the collating sequence of the LC_COLLATE locale, and the sort documentation warns that a non-POSIX locale can produce an order you're not accustomed to, with LC_ALL=C as the way out. On these Bengali lines the ordering didn't hold, so comm reported lines as shared when they weren't.
If "collation" is a new word, the explain a concept prompt will walk you through it with an analogy from your own work. Collation is the rule a computer uses to decide which piece of text comes first, and the rule changes with the language settings.
So the model didn't make up a fact out of nothing. It ran a reasonable-looking command, trusted the output, and wrote the result up well. For me, reading the report, the difference didn't matter: I'd have acted on the number either way.
The half of the audit that was right
The same report found a second problem, and that one was real. Pages on the site promised more statuses than they held: 35 of the 40 collections over-declared their count, including a romantic page that said 120 and held 85, and a birthday page that said 120 and held 70. Every reader who scrolled to the end of one of those pages got less than the title promised.
It was fixed between 18 and 23 June. The count shown on each page is now computed from the items actually on it, and the inflated "120+" style claims came out of the intro text.
So I don't take "AI audits are useless" from this. One finding saved me from misleading readers on 35 pages. The other nearly had me rewriting pages that were fine and merging one of them away.
What I check now before acting on an AI audit
Three habits came out of that week.
- Spot-check one example by hand before acting on any finding. Open both pages, take one "shared" line, search for it. Here that would have taken two minutes and ended the whole question.
- Re-run the count with a different method. For anything in Bengali on that site, the rule is now Python, never
sort,commorgrep. It's written into the notes file the agent reads when each session opens, which works the same way a team keeps its working prompts in one shared prompt library: the lesson lives somewhere the next run will find it. - Slow down before anything that's hard to undo. Once search engines have followed a 301 redirect, reversing a merge is slow. The devil's advocate prompt asks you to say which step can't be taken back and then, under each objection, the cheapest proof either way. For this audit, that proof was a search for one line.
Counts are only one kind of finding. For research claims, quotes and citations, the verification pass in the research guide covers the checks worth running before you repeat anything an AI hands you. And if you're still at the stage of writing the request itself, the five variables guide covers the brief; this post is about what to do with the answer.
The dosti page that was going to be merged away is still live. That same week it got a sharper focus on its slang angle instead, and in October it was one of the pages chosen to link to the site's Kali Puja page. The audit was right that the page needed attention and wrong about the reason, and the difference came down to one line of Bengali nobody had searched for.
Sources
- GNU Coreutils manual: comm invocation
- GNU Coreutils manual: sort invocation
- The audit report, the four tasks it created and the re-check notes are dated 18 to 23 June 2026 in the site's own repository.
Re-checked 23 June 2026 with a Python script comparing exact and normalised status lines on every flagged pair.
Common questions
- Can you trust an AI audit of your website?
- Trust a finding after you've checked one example of it yourself, not before. In this audit one finding was right (35 of 40 pages advertised more items than they held) and the other was entirely wrong (140 duplicate lines that didn't exist), and both were written with the same confidence.
- Why did the duplicate check fail on Bengali text?
- The audit compared pages with the command-line tools sort and comm, which order text by the computer's locale settings. comm only gives correct results when both inputs are sorted the same way, and that ordering didn't hold for these Bengali lines, so it reported lines as shared that weren't. A Python comparison of the exact strings doesn't depend on sort order, and it found none.
- What's the fastest way to check an AI's finding?
- Pick one example from the finding and verify it by hand. For a duplicate-content claim, copy one of the lines it says is shared and search for it on both pages. If that first example fails, treat the whole count as suspect until you've re-run it another way.
