Page is a not externally linkable
sdani - 1:15 am on Jul 4, 2004 (gmt 0)
Although this url (HTTP://WWW.ALLTHEWEB.COM/HELP/WEBMASTER/CRAWLER) does not exists, but this is what we are getting in crawler identification.
Yahoo crawler used to come to our site once in few days.. we recently started second round of link development and we got about 20-30 new links in past week.. we had about 50 links earlier.
Since yesterday yahoo crawler is visiting the home page every 3-4 hours.. does that mean we have crossed some link threshold which qualifies us as a better site (compared to before)... we are getting hits from this crawler:
YAHOO-VERTICALCRAWLER-FORMERWEBCRAWLER/3.9 CRAWLER AT TRD DOT OVERTURE DOT COM; [ALLTHEWEB.COM...]