Duplicate URLs on your own site: types, how to find them and how to fix each

What counts as a duplicate URL

A duplicate URL exists when your site serves the same or almost the same content at more than one address. It is not about two separate articles on a similar topic. It is about one page that can be reached in several ways.

Most duplicates are created by accident. The server answers on both http and https, with and without www, with and without a trailing slash. Tracking parameters, sorting options and print versions add more addresses for the same content. To a search engine, every one of these is a separate URL.

What Google officially says

Google calls the process of choosing between duplicates canonicalization:

The most important point for this lesson is that duplicates are not a punishable offence:

There is no “duplicate content penalty” for ordinary duplicates on your own site. Google groups pages that look the same into a cluster and picks one of them, the one that “is objectively the most complete and useful for search users”. The others are normally not shown.

Google's documentation names the sources of duplicates: region variants, device variants, protocol variants, site-function variants and accidental variants. Language versions are a special case: “Different language versions of a single page are considered duplicates only if the primary content is in the same language”.

Google gives three reasons to tell it which URL you prefer:

• “To specify which URL that you want people to see in search results.”
• “To consolidate signals for similar or duplicate pages.”
• “To avoid spending crawling time on duplicate pages.”

It also describes how strong each signal is. A redirect is “A strong signal that the target of the redirect should become canonical.” A rel="canonical" element is “A strong signal”, and a sitemap entry is “A weak signal”. Google “prefers HTTPS pages over equivalent HTTP pages as canonical”, and asks you to “link to the canonical URL rather than a duplicate URL.”

Two methods are explicitly ruled out:

• “Don't use the robots.txt file for canonicalization purposes.”
• “We don't recommend using noindex to prevent selection of a canonical page within a single site.”

The five kinds of duplicates

Knowing which kind of duplicate you have tells you which fix to use. Five kinds are URL variants of one page. A sixth, content duplicates, are separate pages with the same text.

1. Protocol and host variants

The same page answers on several combinations of protocol and host:

• http://example.com/page
• https://example.com/page
• http://www.example.com/page
• https://www.example.com/page

These are four different URLs. Pick one protocol (HTTPS) and one host (with or without www), and redirect every other combination to it.

2. URL format variants

Small differences in how the path is written also create separate URLs:

• Trailing slash: /page and /page/
• Letter case: /Page and /page
• Index files: /page/ and /page/index.html
• Parameter order: ?color=red&size=9 and ?size=9&color=red

They usually come from server settings that accept every form, and from links written differently in different parts of the site.

3. Tracking and session parameters

Query strings added for analytics or to track a visitor's session do not change the content:

• https://example.com/page?utm_source=newsletter
• https://example.com/page?sessionid=123

Session IDs are the bigger problem: if each visit gets its own ID in the URL, one page can turn into an unlimited number of addresses. Tracking parameters are manageable as long as they stay in external campaign links and never appear in your own internal links.

4. Site-function variants

Features of the site can show the same content at another address:

• Sorting and filters: https://example.com/shoes?sort=price
• Print versions: https://example.com/page?print=1
• AMP versions: https://example.com/page/amp
• Feeds that repeat the content of your articles

These variants are useful to visitors, so you usually keep them. You still need to tell Google which URL is the main one.

5. Region and device variants

The same content may be served to different audiences, for example https://example.com/us/ and https://example.com/uk/ in the same language, or a separate mobile site such as m.example.com.

Region versions in the same language are duplicates in Google's sense, but they can still be legitimate. Connect them with hreflang and give each one a self-referencing canonical, so Google can show the right version in each country. Versions in different languages are not duplicates at all.

Plus: content duplicates

Here the URLs are genuinely different, but the pages are almost the same:

• Boilerplate pages with nearly identical text, such as empty tag pages or several thin pages generated from one template.
• Product variants, where each colour or size has its own URL and the same description.
• Location pages where only the city name changes and the rest of the text is copied.

A redirect rule cannot fix these. You decide page by page whether to merge them, point them to a main page or rewrite them with real, distinct content.

Why duplicates still cost you

No penalty does not mean no cost. Duplicates hurt in four practical ways.

1. Signals are split. If some sites link to http://example.com/page and others to https://www.example.com/page, the links point at different URLs. Consolidation only works when Google can see that they belong together and which one is the main URL.
2. The wrong URL can appear in search. If you do not state a preference, Google picks one itself. It may choose a URL with a parameter or the format you did not want.
3. Crawling is spent on copies. Every duplicate Googlebot fetches is a fetch that did not go to a new or updated page. On a small site this rarely matters; on a large shop with many filters and parameters it can slow down how quickly new pages are found.
4. Reports become fragmented. Traffic and conversions for one page are spread across several URLs in analytics and Search Console, which makes the page's real performance harder to read.

How to find duplicates

1. Check Search Console. In the Page indexing report, three statuses relate to duplicates:
   • Duplicate without user-selected canonical: Google sees this URL as a duplicate and you did not state a canonical for it, so Google chose one.
   • Duplicate, Google chose different canonical than user: you declared a canonical, but Google chose another URL.
   • Alternate page with proper canonical tag: the URL points to another page as canonical and Google agreed. This is usually fine.
   Open each status and look at the example URLs. They often reveal a pattern, such as a parameter or a missing slash, that affects a whole section of the site.
2. Crawl the site and group the results. Run a crawler over the whole site and group pages by identical title, H1 or content hash. Groups with more than one URL are duplicate candidates. This also catches content duplicates that look different in the URL.
3. Test host and protocol variants by hand. Open http://example.com/, http://www.example.com/ and https://www.example.com/. Each should redirect with a 301 to your main version in one step. If any of them shows the page without redirecting, you have a host or protocol duplicate. Do the same for a page with and without the trailing slash and in upper case.
4. Check internal links for parameters. In the crawl, look for internal links that contain utm_, session IDs or sort parameters. Each one tells Google that a duplicate URL matters.
5. Read your server logs. Logs show which URLs Googlebot actually requests. Repeated requests for the same path with different parameters or slash styles show which duplicates are costing you crawling and should be fixed first.

How to fix each type

The right fix depends on whether users need the duplicate URL. If they do not, redirect it. If they do, keep it and point it to the main URL with a canonical. In both cases, make your internal links use the main URL. The canonical tag itself is covered in detail in the Canonical tags lesson.

A few rows need more detail:

• Filters as landing pages. A filter you built on purpose, with its own title, text and search demand (for example red running shoes), should be its own canonical and be linked from your navigation. Sorting orders never qualify.
• Product variants with demand. If people search for a specific colour or model and the variant page has its own content, let it be its own canonical. Otherwise point it to the main product.
• Parameter handling. Search Console's old URL Parameters tool no longer exists. Parameters are handled on your own site: with canonicals, clean internal links and, for session IDs, cookies.

Redirect rules every site should have

These rules belong on the server, so they apply to every URL without anyone having to remember them:

1. HTTPS. Redirect all http requests to https.
2. One host. Choose www or non-www and redirect the other. Use the same host in canonicals, internal links and the sitemap.
3. One slash style. Choose URLs with or without a trailing slash and redirect the other style. The site root / is the exception: it always has a slash.
4. Lowercase. If all your URLs are lowercase, redirect upper- and mixed-case paths to the lowercase version. Leave parameter values alone if your application depends on their case.
5. Index files. Redirect /page/index.html (or index.php) to the clean folder URL in your chosen slash style.

Use 301 (permanent) redirects, keep the query string, and send each variant straight to the final URL in one hop. Here is an example for nginx, with HTTPS, no www and no trailing slash. Adapt it to your own host, slash style and certificate paths before using it:

# Example to adapt: https, one host (example.com), no trailing slash
server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

server {
    listen 443 ssl;
    server_name www.example.com;
    # ssl_certificate and ssl_certificate_key go here
    return 301 https://example.com$request_uri;
}

server {
    listen 443 ssl;
    server_name example.com;
    # ssl_certificate and ssl_certificate_key go here

    # /page/index.html -> /page (one hop)
    rewrite ^/index\.html$ / permanent;
    rewrite ^/(.+)/index\.html$ /$1 permanent;

    # /page/ -> /page (the root "/" is not affected)
    rewrite ^/(.+)/$ /$1 permanent;

    # ... the rest of your site configuration
}

Lowercasing is usually easier in your CMS or application than in nginx. After any change, test a handful of variants and check that each returns a single 301 to the final URL, not a chain.

Duplicates you should keep

Not every duplicate needs to disappear. Some exist for good reasons. Keep them, and make the relationship clear:

• AMP pagesThe AMP page has rel="canonical" to the main page, and the main page links to the AMP version with rel="amphtml".
• Print versions and feedsUseful to readers. Give print pages a canonical to the main article, and keep internal links pointing to the main URL.
• Filtered and sorted listsKeep them for visitors, with a canonical to the base listing, apart from filters you deliberately built as landing pages.
• Region versionsSame-language pages for different countries: hreflang between them and a self-referencing canonical on each.
• Separate mobile URLsThe mobile page has a canonical to the desktop page, and the desktop page has rel="alternate" to the mobile one.

Paginated pages (page 2, 3 and so on) are not duplicates, because each one lists different items. Give each page a self-referencing canonical and link the pages to each other with normal links. Google no longer uses rel="prev" and rel="next" as an indexing signal.

Common mistakes

• ✕Canonical without matching internal linksThe canonical says https://example.com/page while the menu links to https://example.com/page/. Update the links to match the canonical.
• ✕302 instead of 301 for permanent normalisationA 302 says the move is temporary, so the old URL may stay the canonical. Use 301 for protocol, host and format rules.
• ✕Canonical to a page with different contentCanonicals connect duplicates. Between pages that are not the same, Google is likely to ignore the hint.
• ✕Blocking duplicates in robots.txtGoogle cannot fetch a blocked URL, so it never sees the canonical on it and cannot consolidate it properly.
• ✕noindex on duplicates instead of consolidatingnoindex removes a URL from search but does not tell Google which URL is the main one. Use a redirect or a canonical.
• ✕Inconsistent trailing slashNavigation, sitemap and canonical each use a different slash style. All three must use the same format.
• ✕Redirect chainshttp://www.example.com/page/ → https://www.example.com/page/ → https://example.com/page/ → https://example.com/page. Send each variant to the final URL in one hop where you can.

FAQ

Sources

• Google Search Central — What is canonicalization
• Google Search Central — How to specify a canonical URL
• Google Search Console Help — Page indexing report

Quotes checked against the official pages on 29 October 2026.