The Real Duplicate Content Penalty – Google reveals the truth about Duplicate Content

Duplicate content issues are something we’ve been paying a lot of attention to recently with our clients’ sites, just as it has been flexing a great many other SEO’s minds. A number of SEO companies have been pedalling the Duplicate Content Penalty as the latest hot item to sell to clients and prospective clients, promising ingenious solutions to the issue that has been causing their websites so much grief and drop in their rankings.

Firstly, what is duplicate content? Google describes it simply as “substantive blocks of content within or across domains that either completely match other content or are appreciably similar”. In other words pages which are identical or, at least, very, very similar.

Secondly, why is it such an issue? Spammers (website owners/masters who try to artificially manipulate search results via ‘black hat’ methods) have used the copying of pages and sites as a quick way to boost their rankings, after all you can soon create a website if you copy all the content from someone else. Similarly, some people have tried to achieve manipulation of search results by presenting their site across a number of different domains so that they have more than one bite of the search cherry.  This is obviously bad, and has led to the rise of the “Google Duplicate Content Penalty” myth, where people have believed that Google penalises, even removes sites and pages from its index which ’suffer’ from Duplicate Content.

Google has stated through various sources and on a number of occasions, that no such penalty exists, it simply wishes to provide its users (the searcher) with the most relevant results and it does not wish to provide multiple links to what is, basically, the same content.  We have largely believed this, feeling that the main damage that duplicate content inflicts is through incorrect indexing, i.e. Google indexing your non-preferred URL’s and through page rank dilution, i.e. if you have internal and external links to multiple variants of the same pages, your page rank is potentially diluted across however many variants, which is what we have sought to address with our SEO Cients.

Now in a post on Google’s Official Webmaster Blog, Google has categorically denied that a Duplicate Content Penalty exists, stating that whereas they do penalise spammers and spammers who use Duplicate Content, “Spammers also use Bold tags” but they don’t penalise everyone who uses them.

Google realise that not all duplicate content is malicious. They point to real examples of multiple versions of a home page, e.g.

• zelst.co.uk/
• zelst.co.uk/?
• zelst.co.uk/index.html
• zelst.co.uk/home.php
• www.zelst.co.uk/
• www.zelst.co.uk/?
• www.zelst.co.uk/index.html
• www.zelst.co.uk/home.php

This is something that we have found on most of our client sites (and fixed), Google points out that it even occurs on the Queen’s website !

When you add in that some people also have .com and .net, etc., versions of their site, can you see the potential issues? In addition to which URL should Google, Yahoo, Bing, etc., select, which URL do other webmasters use when they link to your site? As well as the issue of different urls being selected by the search engine, if links are split potentially 8, 16, 24, etc., times the value of these links in your page rank is diluted accordingly.

There are also issues relating to the breadcrumb trail of your URL, different size, weight options of the same product, if you are selling online, different country versions of the same language site, e.g. Irish, UK, US and so on.

Added to that we can think of Affiliate versions of pages, inadvertent/innocent cloning of pages, non-approved copying of content and a number of other issues, which all can affect a site/pages particular ranking.

Google re-emphasises the use of permanent “301 redirects” and use of the “rel=canonical” tag as ways to address this, which Zelst has been successfully using for a number of months to deal with these issues.

If you would like to discuss your potential duplicate content issues or want help to address them contact Zelst today.

And Then There Were Two!

Microsoft + Yahoo = <Google?

Well, no one saw that one coming did they?

Of the hundreds of different internet search engines available, up until last Wednesday there were only really three meaningful ones. And now that number has shrunk by a third, with Microsoft’s announcement that it was joining forces with Yahoo. The essence is that Yahoo will replace its own search engine with Microsoft’s new search engine, Bing, for users of its popular pages (and split the resulting revenue) and Yahoo, errr…. Well, they’re going to get their salespeople to sell all the advertising, although they won’t be selling advertising to their self service customers, i.e. the vast majority of them, this is Bing’s job. Oh yes, and Yahoo has licensed its Search Technology to the new Microsoft Bing operation, although it hasn’t clarified the monetary aspects of this deal. Sounds like a good deal, especially if you’re a Yahoo shareholder….not?

Microsoft and Yahoo have been flirting and/or playing the mating game for over 18 months now and the recent intensified spats with Google have flagged up that the two were getting ever closer to consummation.

The launch of Bing, a much improved search engine compared to its predecessor, which took  big chunks out of Yahoo’s market share, was Microsoft’s trump card or the final nail in Yahoo’s coffin, depending on your perspective and choice of metaphor.  Not only did Yahoo see its users deserting it for Bing but the realisation of the scale of resources needed to keep its own search engine in the game not to mention the marketing costs to persuade people to use it left Yahoo feeling increasingly irrelevant.

So Yahoo is now positioning itself as a media company, has ditched its tech roots (and founder Jerry Yang in the process) and has launched a new home page! And the main thing that Yahoo was known for and the source of the bulk of its current revenue will be provided by Microsoft.  Yahoo Search Marketing, aka Overture, the inventor of the PPC search model that has powered its rival Google to world domination will, essentially, cease to exist.

What Does This All Mean to You?

In real terms to most people there was only really one search engine last week and there still is today. Google is so far ahead of the game that, to a lot of people, putting Bing and Yahoo together is largely irrelevant.

With most of our clients we are seeing shares of between 85 and 95% to Google, 5-10% for Yahoo and 1 to 2% for Bing, with around 1% for the rest. Yes, this is distorted because most of our clients are running Google AdWords campaigns but even stripping out the paid clicks doesn’t make the comparisons any less stark.  As an example for the keyword  ”cheap beds for sale” even with a Yahoo ranking of 6 compared to a Google ranking of 12 Google produced 3 times more traffic and of a significantly higher quality for our bed client. With the keyword  ”buy cheap beds online” despite a no 1 Bing Ranking compared to a no 8 Google ranking Google produced 20 times more traffic than its newest challenger. With Ottoman Beds despite ranking of #1 on Yahoo,#7 on Bing and #10 on Google the split was 4% Bing, 36% Yahoo and 60% Google.

Advertisers should welcome a stronger competitor to AdWords, but it will mean there will now only be one meaningful alternative rather than two. Bing’s new PPC offer is considerably improved but it still probably isn’t as good as Yahoo’s Overture PPC set-up and even if you put the absolute best bits of Microsoft’s paid search together with the absolute best bit’s of Yahoo’s, it still wouldn’t be as good as Google’s AdWords.

The main difference should be volume. To smaller PPC advertisers it simply isn’t viable to run ads on either Bing or Yahoo. Search volumes are so much smaller in comparison that the cost of setting up, monitoring and managing campaigns across Bing and Yahoo only makes sense for large advertisers. That, added to the fact that generally clicks are poorer quality from these two has left us recommending to most of our clients that they just stick with Google. Hopefully an improvement in the PPC search engine combined with the increase in search volumes of the two will change this. The danger is that in putting two smaller mediocre operations together the end result will be one slightly larger mediocre operation, which will play once more into Google’s hands.

The best case scenario is that with Bing’s much improved search engine combined with the more than doubling in volume will provide a worth competitor for Google and keep it on its toes.

There are also things like the venerable and still really useful Yahoo! Directory and Yahoo’s sponsored placement ‘Search Submit’, of which as yet announcements have still to be made. What will happen to these?

And What for the Immediate Future?

One of the big issues here is that the deal is subject to regulatory approval and could take up to two years to complete, which is a heck of a long time in search engine terms. The big worry is that Yahoo will limp along as a lame duck, lose even more of its best people and users in the uncertainty, Bing will continue to get a little better and raise its market share a little and Google will rub its hands and get even stronger. Stay tuned…….