What is Duplicate Content? Why is it bad? How do you avoid it? 

There’s no denying that there’s a lot of content on the web, and inevitably, at least some of it will be similar to content elsewhere. But there’s a difference between similar content and duplicate content, and one is inherently worse than the other. 

What is duplicate content?

Duplicate content is online content that is identical – or virtually identical – to content found elsewhere on the web. It may be copied intentionally to cut corners or an innocent mistake. Either way, though, it can substantially harm your website. To be clear, duplicate content can be:

• A piece of content which has been deliberately copied from another site
• Content taken from another one of your sites or elsewhere on the same site
• Generic manufacturer’s product descriptions, which have been copied and pasted
• Content which is just a mishmash of copied text from different sources

It’s important to realise that there is no distinction between these types of duplicate content as far as the search engines are concerned, and just because you didn’t mean to do it doesn’t mean you’ll be let off Scot-free.

Having duplicate content may not mean your site will be penalised, but it can impact your search rankings. This is because ‘appreciably similar’ content can confuse search engines when deciding which version is more relevant to a given search query. 

What’s the problem?

Google and other search engines always do their best to answer users’ queries with the most relevant and valuable results. They do this by scouring the internet for web pages that best fit the user’s search term.

If a page on your site is a duplicate of another elsewhere on the internet or on your own site, it is deemed to have far less value than the original content. In this way, a competitor with inferior products or services to yours could do far better than you in SERPs simply by having original content.

In practical terms, Google and most other search engines will not index duplicate content. That means that for search engine optimisation (SEO) purposes, these pages will be worthless—they will not help you rank more highly for users’ queries, and more importantly, they will simply not appear in searches.

To be more specific, this content presents issues for search engines and site owners. For search engines, duplicate content offers three main concerns:

1. They don’t know which version to include/exclude from their indices. If you have copied content from elsewhere on your site or from another website, Google may not index your website page. 

• They don’t know whether to direct the link metric to one page or separate it between multiple versions. If Google decides to spread its resources between multiple pages, then chances are you won’t get a fair share of traffic to your site. 

• They don’t know which version to rank on the SERPs. If Google identifies your site as unoriginal, chances are you’re getting left behind. 

Once the search engine has figured out what it wants to do with a batch of duplicate content, the site owner will suffer nine times out of ten. Your rankings and traffic will lose out.

Moz says, “To provide the best search experience, search engines rarely show multiple versions of the same content, and thus are forced to choose which version is most likely to be the best result. This dilutes the visibility of each of the duplicates”.

What does Google currently say about Duplicate content?

Okay, so Google finally put every web admin’s big worry to bed when they released a statement to say there is no duplicate content penalty – at least not in the way everyone assumes. Google said: 

“There are some penalties related to the idea of having the same content as another site—for example, if you’re scraping content from other sites and republishing it, or if you republish content without adding any additional value. These tactics are clearly outlined (and discouraged) in our Webmaster Guidelines:

• Don’t create multiple pages, subdomains, or domains with substantially duplicate content.
• Avoid… “cookie cutter” approaches such as affiliate programs with little or no original content.
• If your site participates in an affiliate program, make sure that your site adds value. Provide unique and relevant content that gives users a reason to visit your site first.”

So, in short, Google continues: “Duplicate content on a site is not grounds for action unless it appears that the content intends to be deceptive and manipulate search engine results.”

In a way, if you’re copying content from other pages or plagiarising the words on your website, then—in our eyes (and Google’s)—manipulating search results must be the only reason.

Bing’s guidelines are a little snappier than Google’s, but they reflect a very similar sentiment. Bing says, “Avoid duplicate content within your site; duplicating content across multiple URLs can lead to Bing losing trust in some of those URLs over time.”

The content you publish online, which aims to help users navigate the sales funnel, is crucial in the trust-building process. If the search engine isn’t likely to trust your site’s content, it won’t recommend it to its users. And let’s be real – when was the last time you took a trip to page six of the SERPs to find an answer, product or a little inspiration? 

Copyright and duplicate content

As with all forms of written and artistic content, the copyright of a piece of online content is automatically assigned to its author (or to the party to which the copyright has been handed over in the case of copywriting and ghostwriting). If you ‘borrow’ (read: ‘steal’) content from someone else without their consent, you are in breach of copyright laws and may be liable to a lawsuit. Though such cases are seldom enforced, it’s something to remember, particularly with the potential for one landmark lawsuit to open the floodgates to more.

Is it ever okay? 

There are, of course, a few scenarios where duplicate content may be seen as legitimate. These include: 

• Products listing on different selling platforms.
• Quotes or snippets of text referenced from other sources. 
• Printer-friendly or PDF versions of your website pages. 
• Archived content. 

If you know there are possible pieces of copied content on your site, properly using canonical tags can indicate to search engines which version of a duplicate page they should index. 

What causes duplicate content? 

29% of pages crawled on the internet have duplicate content, so it’s not uncommon for this problem to arise on your site. But if you know what causes it, it’s easier to fix. Here’s what you should keep an eye out for: 

1. Content Management Systems (CMS) defaults: Some CMS platforms automatically generate duplicate content, especially if not configured properly. For instance, default settings may create multiple URLs for the same content.

2. URL variations: Different URLs pointing to the same content can lead to duplication. This may happen due to URL parameters, tracking codes, or session IDs appended to URLs.

3. Repetitive content: Content spread across multiple pages, especially when similar, can result in duplicate content issues. Each page may have similar or identical content except for the page heading. 

4. HTTP and HTTPS versions: If both HTTP and HTTPS versions of a website are accessible, search engines might index both, resulting in duplicate content issues.

5. WWW and non-WWW versions: Similarly, having both www and non-www versions of a website can lead to duplication.

6. Syndicated content: Publishing content syndicated from other sources without proper attribution or canonicalisation can result in duplicate content.

7. Product variations: E-commerce sites may have multiple product pages for similar products with only slight variations, causing duplication.

8. Scraped content: If a website scrapes content from other sources without proper attribution or permission, it can lead to website penalties.

9. Internationalisation and Localisation: Offering the same content in multiple languages or for different regions without hreflang tags or proper canonicalisation can result in duplication.

Duplicate content and your brand identity

Aside from these pressing SEO matters, duplicate content is not good for your brand. If you’re copying content word for word from elsewhere on the web, it’s not creative – it’s plagiarism. And when something isn’t written by you, customers will catch on.

Rock Content sums this up well: “A brand’s ability to win an audience’s trust is a crucial part of its ultimate success. While it takes time and many instances of acting in good faith to win someone’s trust, it only takes one bad incident to destroy all that hard work.”

The most effective content is tailored specifically to your brand’s tone of voice and your customers. Your content should be highly original and distinctively recognisable as yours—it should have a vibrant personality all of its own. 

Duplicate content is not memorable and will not help you stand out. Instead, it will result in you being seen as just another generic carbon copy brand with nothing fresh to say or offer. Your customers will disappear, readers will get bored, and if you keep up the copying act, you could have a copyright case to deal with. 

How Google’s menagerie has impacted upon duplicate content

As we mentioned earlier, search engines have a vested interest in keeping duplicate content out of their results. To this end, they are constantly fine-tuning their search algorithms to detect and remove such content. Google is by far the most visible and vocal search engine in this regard and has released several animal-themed updates over the past few years to tackle the issue.

Google Panda was first unleashed upon the world in 2011. Its primary purpose was to weed out low-quality sites from the results, particularly ‘scraper sites’ which exclusively published copied content. There have since been numerous reissues of the update, the most recent being 4.1, which aimed to further penalise ‘thin’ content.

Google Penguin was released in April 2012, with the aim of penalising sites which use ‘black hat’ tactics in contravention of Google’s Webmaster Guidelines. Amongst these tactics is the use of duplicate content.

Google Hummingbird, released in 2013, may not specifically deal with duplicate content. However, it substantially changed how Google handles user queries, injecting a semantic element aimed at ‘understanding’ the query rather than just matching the words. You need original content tailored specifically to your site to do well in this new online world.

After the latter update, Google moved on to giving their algorithms more human names. The Google Fredupdate, which rolled out in March 2017, targeted websites with low-quality content and aggressive monetisation tactics, further emphasising the importance of original and valuable content.

Next, in late 2019, Google BERT was introduced. This made significant advancements in understanding the context of words in search queries, leading to more relevant search results. While not directly aimed at duplicate content, BERT’s focus on context underscores the importance of providing unique and valuable content to users.

Fast-forward to 2021. Google’s Page Experience Update finished rolling out to users in August of that year, prioritising user experience metrics such as page loading speed, mobile-friendliness, and interactivity. In March of this year, a similar update was released, improving spam policies and ranking quality by tackling low-quality, unoriginal content found on the web. 

How to fix duplicate content issues on your site

If you’re concerned about duplicate content on your website, there’s no need to fret, but you should take care of it as soon as possible. Failure to do so may significantly hamper your online marketing efforts and hold your business back from reaching its full potential.

Fortunately, dealing with duplicate content is not a complicated process, particularly if you enlist the help of an experienced copywriting agency to assist you. All you need to do is follow this simple two-step process:

• Identify – Work out what content you have by doing a content audit and figure out which items, if any, are a copy of content elsewhere. This may be content that you have used multiple times in good faith, or if you have used an unscrupulous copywriting service in the past, it may have been directly ‘ripped’ from other sites like yours. There are various tools out there that you can use to automatically detect plagiarised or cloned content, including Copyscape and Grammarly. 

• Replace—This bit is relatively simple but time-consuming, so many businesses outsource it to a professional web copywriter or copywriting agency. Instead of duplicate content, you should create exceptional content relevant to your offerings, specific to your brand, and aimed directly at your customers.

When it comes to fixing the technical issues duplicate content can cause, you should: 

1. Implement canonical tags: Use canonical tags to indicate to search engines the preferred version of duplicate pages. Ensure that all versions of duplicate content point to the canonical URL.
2. 301 redirects: Redirect duplicate URLs to the canonical URL using 301 redirects to consolidate link equity and avoid indexing of duplicate content.
3. Parameter handling: Use URL parameters handling tools in Google Search Console or rel=”canonical” tags to consolidate content caused by URL parameters.
4. Hreflang Tags: Use hreflang tags for internationalised content to specify language and regional variations, helping search engines understand the relationship between duplicate content.
5. Consolidate similar pages: Merge similar or redundant pages into one comprehensive page to eliminate technical SEO issues and improve user experience.
6. Unique content creation: Create unique and valuable content for each page to avoid duplication and provide users with distinct information and experiences.
7. Use noindex tags: For non-essential duplicate content, consider using the noindex meta tag to prevent search engines from indexing duplicate pages while keeping them accessible to users.
8. Regular monitoring: Continuously monitor your website for new instances of identical content and promptly address them using the above methods.

Check for plagiarism of your own content

As much as you can copy content from elsewhere on the web, your site—full to the brim with great, original content—can become a victim of plagiarism. This is unfair, of course, but in practical terms, it pays to be proactive in checking to see if anyone has helped themselves to your content.

You can do so relatively quickly and easily with tools like the ones mentioned above, as well as Writer.com’s Free Plagiarism Checker and Article Checker. Should you identify duplicates, consider contacting the offending party to request its removal, along with a firm reminder of copyright law and the potential consequences of unauthorised copying.

Minimising the risk

Publishing unique and original content on your site is one of the most important things you can do for content marketing. 

This means that your content should avoid using AI to generate copy, poor research, misleading information, and direct copying. Do any of the latter, and your content will likely be duplicates of content found elsewhere on the web, but more crucially, it won’t engage your customers. 

The thing with AI, and especially the Large Language models that hit the market after the release of ChatGPT, is that they gain the ability to produce content quickly by consuming articles, resources, and other material that already exists on the web. In short, AI is a big old algorithm. It’s fed data, learns from that data, and comes up with ideas, answers and generated content based on the things it learned. But the latter is where it gets tricky. 

When producing responses, large language models like ChatGPT regurgitate content, leading many to question how ethical these tools are. This has escalated quickly, and some online publications have even taken legal action against the organisations behind these AI tools. 

The New York Times has recently made headlines for its decision to sue OpenAI and Microsfot for copyright infringement over the unauthorised use of their published work to train AI technologies. The lawsuit demands that the defendants be held responsible for “billions of dollars in statutory and actual damages” related to the “unlawful copying and use of The Times’s uniquely valuable works.” It also calls for the companies to destroy any chatbot models and training data that use copyrighted material from The Times.

The bottom line here is that cases like these prove that AI content isn’t original; while it uses already published content to produce more similar content, it runs the risk of falling into the parameters of duplicate content and plagiarism. And neither of those are courts you want to play ball in. 

Creating original, high-quality content 

Content that is unique and original is more likely to grab users’ attention for longer and, in turn, prove to Google that your site is a trustworthy source. To be a good piece of original content, it needs to be: 

• A new, unique idea or angle on a topic that has never been published before
• Well-written with a consistent and considered tone of voice
• Delivering on the promise contained within the title
• Content that your user determines to be valuable 

It needs to be written by you, offering unique insight, and tailored perfectly to your target user. Other sites won’t be able to hit the nail on the head like you, so there is no point in plagiarising their content. 

If you invest time and resources into original content, it pays, especially when it comes to SEO, customer engagement and brand reputation. Here’s why: 

• Search engines favour unique content, which can help your site rank higher in search results and attract more organic traffic.
• Unique content increases the likelihood of search engines indexing your pages, improving your website’s visibility in search results.
• Search engine bots are more likely to crawl and index websites that consistently produce fresh, original content, leading to better crawl frequency and indexation.
• Original content allows for strategic targeting of relevant keywords, improving your site’s chances of ranking for specific search queries.
• Original content helps differentiate your brand by offering unique insights, perspectives, and information that cannot be found elsewhere.
• Providing valuable and original content establishes your brand as an authority in your industry, earning trust and credibility from your audience.
• Compelling and original content is more likely to be shared on social media and other platforms, increasing brand exposure and earning valuable backlinks.
• Unique content tailored to your target audience’s needs and interests enhances the overall user experience, leading to higher engagement and repeat visits.
• Original content can address different stages of the customer journey, from awareness to decision-making, guiding and nurturing prospects through the sales funnel.
• Original content sets your brand apart from competitors, helping you stand out in a crowded marketplace and build a distinctive brand identity.

Banish duplicate content for good 

Hopefully, we’ve helped you better understand why duplicate content really is bad news for your website, and you feel energised to do something about it. Doing so will help you build a healthy online presence and a memorable brand and can improve your search rankings.

If you need any help identifying technical SEO issues or creating lively new original content that sells, don’t hesitate to contact Big Star Copywriting.