Image CAPTCHA
Copy the characters (respecting upper/lower case) from the image.

There's some interesting sites out there

There's some interesting sites out thereThere's some interesting sites out there
Posted by Dan Frost on Wed, 25/07/2007 - 16:23

We've been watching sites go through the crawler and the first, most immediate issue is that our crawler isn't coping with malformed HTML as well as it might. There's a fix on the way and the crawler should be able to handle those issues shortly. We'll re-crawl and notify those users that had that problem.

Unclosed tags and so on are commonplace and we had a basic HTML validator on the "to do" list but I think we'll do it sooner rather than later.

If your site crawls but shows 0 or 1 page crawled it's either because you've got major issues with your HTML or none of the URLs on the site are the same as as the initial URL - eg: if you tell crawlscore that your URL is http://google.co.uk but your links are actually http://www.google.co.uk then crawlscore will treat them as external URLs.

There's an argument either way for following sub-domain URLs but for the moment, we're treating each site individually.

To crawl a subdomain simply add it as a site and it will be crawled independently.

As far as we know, most search engines treat subdomain differently anyway so we'll do the same :)

We're trying to communicate directly with users where possible to let them know of issues (either their end or ours) but we're getting a high number of sign-ups so please bear with us.