Dynamic URLs Ensure search engines can crawl all of your content by analysing the use of dynamic URLs with Crawl Score. Search engines will often avoid dynamic URLs as they can use huge amounts of resources by getting the crawler stuck in a loop. Crawl Score will identify any page it identifies as a dynamic URL. Most search engines have a list of patterns that suggest a dynamic URL and Crawl Score operates in the same way. Any URL with a "?" in it is usually avoided (certainly on new sites) as are sites with variables such as "PHPSESSIONID". Session IDs are particularly bad because they are temporary URLs and a search engine crawler would be finding "new" pages all the time (because each session ID is unique and is generated when a session is opened). |