Checking for duplicate content

by ian on July 20, 2010

Duplicate content won’t get you penalized any more. But it can still wreak havoc with your SEO campaign. That’s because duplicate content can split your link ‘votes’, just as I discussed in the module on canonicalization.

So, here’s two easy ways to see if your site has duplicate content:

If your site has less than 1000 pages

Note: This technique works if your site’s less than 1000 unique pages.

If you suspect your site has a canonicalization issue but you don’t have Google Webmaster Tools set up yet, and want to do a quick check:

First, in your web browser, type in this exact search, replacing [YOUR DOMAIN] with your web site domain name: http://www.google.com/search?q=site:[YOUR DOMAIN]&start=990.

That will show the last few results from your site. If you see this:

Uh-oh - we have duplicate content.

Uh-oh - we have duplicate content.

You’ve got duplicate content.

If your site has more than 1000 pages

If your site has more than 1000 pages, the start=990 trick may not work. In that case, you need something a little more robust. Portent has a fantastic duplicate content checker, but alas, it’s not quite ready for public use just yet. On the other hand, I’ve looked high and low for another duplicate content checker that’s as good, and have yet to find one.

However, because you’ve bitten the bullet and signed up for the Fat Free Guide long before anyone else, I’ll make you a deal: Submit a Q&A request that includes the domain you want checked, and I’ll have our crawler go through the site. I can then send you a list of potential duplicate pages.

So there you go: A little extra bonus for FFG readers.

Related/other modules in this section:

  1. Finding and fixing canonicalization problems
  2. Diagnosing and fixing SEO roadblocks
  3. Creating an XML sitemap
  4. Why copy matters more than anything else
  5. Increasing Traffic with Rich Snippets

Previous post:

Next post: