Forum Moderators: open
But I dont have this problem with other crawlers
Actual url:www.site.com/aa/bbb/widgets.html
It tries to crawl:www.site.com/aa/bbb/widg
66.196.90.49 - - [25/Feb/2004:23:41:35 -0700] "GET /site-map.html HTTP/1.0" 200 34860 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]
66.196.90.49 - - [25/Feb/2004:23:41:33 -0700] "GET /aa/bbb/widg HTTP/1.0" 302 292 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; [help.yahoo.com...]