Crawl of the week - Google vs Yahoo! vs Live Search - battle of the blogs

Every week or so I'll be posting "celebrity site crawls" and talk about the results.

So this week in celebrity site crawls it's the battle of the giants.

In the corner with the bean bags and M&Ms dispenser, hailing from Mountain View, doing no evil whatsoever but still weighing in with a market capitalization of $137b, is the King of spin, the geek with a mean streak, Gooooooogle.

In the red corner, the perennial under achiever, with a record of 2000 websites and 2 worth using, hailing from Sunnyvale and weighing in with a market cap of $44b is Yahoooooooo!

And finally, the rank outsider, hailing from Redmond, with a record of 200 fights, 199 victories (199 by way of buy-out), is Miiiiiicrosoft (aka Live Search).

Let's get ready to ruuuuuuuumble....

Ok, I think I've stretched the whole boxing analogy a bit too far now so let's get down to business.

History

The Yahoo! Search blog started way back in 2004, Google webmaster tools blog around 2006 and Live Search blog in 2007.

Index statistics

Other statistics

Not that it really matters but the pageranks are 5, 7 and 8 for Live Search, Yahoo! and Google respectively.

Google's blog is the only one that features Feedburner (with 25k subscribers). None of the blogs are over endowed with bookmarking options.

Crawl Score overview

Let's take a look at the overall search engine friendliness of the sites. This is made up of page size and duplicate page titles

Live

Yahoo!

Google

Googles pages are, on average, around 320kb. This is because of the number of comments on each post. It's a fairly arbitary figure but Crawl Score flags up any page that has more than 60kb of HTML as too big - with Google's being over 300kb I don't think there's much argument that even with broadband, etc that's too big.

Duplicate page titles

Crawl Score statistics - Yahoo! Search blog

As you can see from this cropped screen shot from the Crawl Score reporting centre, there are actually a total of 429 pages with 15 404 errors and 3 500 errors. That's interesting because the search engines have far more pages indexed so let's see what that's all about.

So Google has around 550 pages indexed for Yahoo! search blog but Craw Score is finding 429 - after investigation this is because of orphaned pages on the site that Google still has indexed but the site no longer links to....

Crawl Score statistics - Google Webmaster Central blog

Google fares better with their general site quality in terms of HTTP issues but what's with the page size?! Actually it's nothing terrible - it's just that there are so many comments on a lot of the blog posts that the pages get fairly big. I would suggest they implement paging for comments or perhaps accept fewer comments?

Crawl Score statistics - Live Search blog

Not too bad - only a few 404's and the 500's are probably caused by IIS ;-) 500 errors are generally difficult to tie down as they are usually caused by a temporary problem. It could suggest their application is strugging to cope with the traffic but let's face it, it's Live.com - they can't be getting that much traffic....

And the winner is....

And so... it's down to the judges... the winner... by unanimous decision is.... the best official search engine blog in the world (well, out of those three anyway) is.... Yahoo! Search blog.

If you need to see your site as the search engines do, visit Crawl Score.