How I Avoid Google’s Wrongful Penalty For Duplicated Content

We’ve heard it all before – duplicated content is a no-no.

These include effects like duplicated pages not showing up in the search engine results pages (SERPs), and multiple copies of the same page maxing out your crawl budget. A helpful way to figure out how much similarity is too much is to look at how exactly Google detects copied content.

How does Google detect duplicate content?

Crawling entire webpages
This might sound obvious, but Google detects copied content by crawling the content on webpages. However, that’s not to say that every bit of duplicated content is treated similarly.

Google recognises that it’s natural for webpages to contain a certain level of similarity. For example, website footers and product descriptions are some pieces of content that are widely copied within and across websites. According to Mueller, the Google algorithms are robust enough to excuse these types of duplicated content.

However, when entire chunks or pages are copied in full, this can become a problem. This forces the search engine to have to choose between the duplicated pages to show the best possible one to the user.

That’s why SEO experts generally choose to err on the side of caution. If you need to take content from another source, it’s best to rephrase it into your own words, and supplement it with additional content that value-adds to the page. This way, you minimise the risk of your content becoming invisible to the search engines.

Predictive method based on website URL
Recently, Mueller also explained another way by which Google detects duplicated content – through a predictive method which looks at the website’s URL.

In principle, this method seeks to save resources by judging if a website is likely to be duplicated before even having to crawl the webpage itself. Google goes based on the website’s URL, and compares this to other duplicated pages that it has encountered before. If the website URL is in a similar format to those used by duplicated pages, Google makes a guess that this page is also copied.

However, Mueller admitted that this could give rise to websites being falsely written off by the search engine. Websites that are not duplicated could also have URLs that are similar to that of duplicated pages, causing the search engine to mistakenly label them as duplicates.

For an example of what might happen, imagine this: An events company has a page introducing its upcoming event in City X, with the city name in its URL. As the event is also relevant for people in City Y – just next to City X – there is a duplicate page with City Y in the URL. Multiply this by as many cities as this event is targeted to, and you get a whole lot of duplicate pages, which Google’s predictive algorithm will rightfully catch.

However, imagine that you have 10 other unique pages on the same website which include City X or City Y in their URLs. The predictive algorithm may flag them out due to their similarity to the previous duplicated pages. So, although their contents are in fact different, these pages will not be crawled by Google as they are thought to be duplicates.

This is an unfortunate fact of Google’s current algorithm, I know. But there are still things you can do to work around it.

How to avoid being mistakenly labelled as duplicated content

Google suggests making use of canonical tags or redirects. This way, Google gets the message that these pages are meant to be similar. This will also distinguish this set of duplicated pages from your other pages which are meant to be unique. With this, you can feel more assured that Google won’t mistake your unique pages as duplicate pages.

Conclusion

Being careful about duplicated content is something every SEO expert should do. And being careful about making sure your unique content isn’t mistaken as duplicate content – that’s just a sensible thing to do.

But when your website is mega-huge, or dealing with technical SEO just isn’t your cup of tea, it’s worth considering engaging the experts to give you a hand. For all your digital marketing needs in Singapore, our SEO agency is here for you!