Forum Moderators: phranque
I have URLs with two underscores together like [widget.com...]
Will this affect search engine spiders crawling?
In my experience, no. It will not affect search engines from spidering your site.
I suggest you implement Google Sitemap at your site. I did that long time ago before Google came out with this solution and managed to have one of my sites indexed by Google. It has over 70,000 pages (dynamic plus HTML). I did that with text file (i.e. getting all the urls listed), and I was surprised to find out recently that Google came out with similar solution for sitemap i.e. sitemap in XML or text file format.
Having sitemap at your site, you need not worry much about not getting spidered by Google. Other search engines can make use of this sitemap too.
I don't know why you want to put two underscores. Whether you have underscore, -, or combined the words together, search engines today are smart enough to be able to differentiate words. From my experience, meaningful words are ranked better by Google PageRank.
