Duplicate Content Checker: How to Measure and Fix Text Similarity
Understand with AI
Discuss with your preferred AI assistant
Studies repeatedly find that a large share of websites carry meaningful duplicate-content issues across their URLs.
Phrase-level similarity above roughly 80% is treated by search engines as effectively duplicate.
This checker compares your text locally with deterministic math — nothing is uploaded or stored.
Duplicate content is text that appears, word-for-word or in substantially similar form, in more than one place. It shows up when you reuse boilerplate across pages, when an agency repurposes a draft, when two writers cover the same topic, or when content gets scraped and republished. A duplicate content checker measures exactly how similar two passages are so you can fix the overlap before it costs you traffic.
This guide explains what duplicate content is, why it matters for SEO, how a similarity checker actually calculates a percentage, and how to read your result so you know whether to rewrite, consolidate, or move on.
What Is Duplicate Content?
Search engines define duplicate content as substantive blocks of content within or across domains that either completely match or are appreciably similar. There are two flavors that matter in practice:
- Internal duplication — the same or near-identical text on multiple URLs of your own site (think printer-friendly pages, faceted URLs, or copy-pasted product descriptions).
- External duplication — your text matching a competitor's, a syndication partner's, or a scraper's. This is also how plagiarism shows up.
Not all duplication is malicious or even avoidable. The goal is rarely zero overlap — it is making sure each page earns its place with enough unique value that search engines and readers can tell pages apart.
Why Duplicate Content Hurts SEO
Google has stated there is no blanket "duplicate content penalty," but duplication still damages performance in three concrete ways:
- Diluted ranking signals. When two pages target the same query with the same words, links and engagement split between them instead of concentrating on one strong page.
- Wasted crawl budget. Crawlers spend time re-reading near-identical pages instead of discovering your fresh content.
- The wrong page ranks. Google picks one version to show and filters the rest, and its pick is not always the one you wanted.
For writers and editors, similarity checking is also a fast originality and plagiarism screen — catch lifted passages before they ship.
How a Similarity Percentage Is Calculated
A good checker does not just look for an exact match; it scores degree of overlap. SemlyPro's checker reports several complementary scores so a single edge case never misleads you.
Jaccard similarity on word shingles
The headline score breaks each text into overlapping word sequences called shingles (n-grams). With a shingle size of three, "the quick brown fox" yields "the quick brown" and "quick brown fox." The tool then computes the Jaccard index — the number of shingles both texts share divided by the total number of distinct shingles across both. This rewards genuinely shared phrasing, not just shared vocabulary.
Word-set and character similarity
A word-set Jaccard score measures how much vocabulary the two texts have in common regardless of order, while a Sørensen–Dice score over character trigrams catches fuzzy matches like minor edits, typos, and tense changes. Reading all three together tells you whether overlap is structural or incidental.
Containment and the longest verbatim run
Containment shows how much of text A appears inside text B (and vice versa) — invaluable when a short passage has been lifted into a longer one. The longest verbatim run surfaces the single longest stretch of identical wording, the clearest fingerprint of copy-paste.
How to Read Your Result
Use these thresholds as a practical guide rather than hard law:
| Similarity | Verdict | What to do |
|---|---|---|
| 80–100% | Near-duplicate | Rewrite, canonicalize, or merge the pages. |
| 50–79% | Substantial overlap | Differentiate the shared sections meaningfully. |
| 20–49% | Some overlap | Review the shared phrases; tighten where needed. |
| 5–19% | Mostly unique | Usually fine — incidental shared phrasing. |
| 0–4% | Unique | No action needed. |
How to Fix Duplicate Content
- Rewrite for a distinct angle. Same topic is fine; same sentences are not. Change structure, examples, and emphasis.
- Consolidate thin pages. If two URLs say the same thing, merge them into one stronger page and 301-redirect the loser.
- Use canonical tags. When duplication is unavoidable (e.g. syndication or parameters), point rel=canonical at the version you want indexed.
- Template carefully. Keep boilerplate short and make the unique body the bulk of every page.
Expert Tips
Read the shared phrases, not just the score
A high percentage built from common stock phrases is harmless, while a low score can still hide one lifted paragraph. Always scan the extracted shared phrases and the longest verbatim run.
Differentiate, then consolidate
When two pages overlap heavily, decide whether they should be one page. If yes, merge and 301-redirect; if no, rewrite the shared sections with distinct structure, examples, and angle.
Frequently Asked Questions
Is there a penalty for duplicate content?
There is no automatic penalty for ordinary duplication. Google simply filters near-identical pages and shows one version, which can suppress the others. Deliberately scraped or doorway content can trigger manual action, but everyday duplication is a ranking-dilution problem, not a penalty.
What similarity percentage is considered duplicate?
As a rule of thumb, anything above roughly 80% phrase-level similarity behaves like a duplicate, and 50–79% is enough overlap to warrant a rewrite. Below 20% is usually incidental. Always read the shared phrases too — a high score driven by common stock phrases matters less than a low score hiding one lifted paragraph.
Does this tool detect plagiarism?
It detects similarity between two texts you provide, which is the core of plagiarism checking. It does not crawl the web for matching sources, so pair it with a SERP search when you need to find where a passage originated.
Is my text uploaded anywhere?
No. The comparison runs entirely in your browser using deterministic math — nothing you paste is sent to a server, stored, or logged.