Are sitemaps worth the hassle? Posted by Dan Frost on Fri, 27/07/2007 - 23:23
It really depends on how much hassle is involved in creating them. Obviously if you're using crawlscore then it's no hassle but please don't think that sitemaps are a silver bullet to your indexing problems. Talking from personal experience, sitemaps for large sites do very little other than enable Google to check for dodgy content. A client left some commented code in their site (to help AdSense bring in relevant content) - within 24 hours the site had been de-indexed because of comment spam. In fairness, Google re-instated the site within 48 hours of the comments being removed but there was absolutely no difference in the indexing of the site. In reality, most people will upload and/or create a sitemap "just in case". That's fine and it's what we've done on large sites but it's probably wiser to spend time on looking at your internal linking structure and page titles. I really don't understand why any site would now have obviously dynamic URLs (IE "?" in the URL). Most CMS have options to rewrite URLs and there's thousands of guides out there on how to solve this problem. SessionIDs (or anything that looks like them) are a crawlers worse nightmare. All the major search engines will steer clear of anything resembling a session - I'll put a more detailed post up another time about exactly how we define a session based URL. We actually regard sitemaps as a by-product of the crawl that we do rather than it being the raison d etre. Many users may use crawlscore initially because of the free sitemaps but it could be that the crawl reports show many other issues that they weren't previously aware of. Some may read this post and think I'm "diss'ing" our own product - we're not, we're just saying that sitemaps aren't the be all and end all. |
Inter