I was dumb enough to block all robots but the biggest search engines. I saw Gigablast's robot and others were on a single-hit visit (most likely robots.txt), saw that they are not welcome and then went away. :(
Now I have changed my robots.txt into allowing all bots, instead blocking bad User-agents by force manually. Will Gigabot and others ever come back? If so, how long before they come back and see that they are now allowed in?
How long after the robots visit before the search engines include your site? I can see that several robots have visited in the past 24 hrs, but I am not listed on their search engines, I was visited by googlebot, yahoo slurp, msn media and a couple of unidentified ones....how long do I need to wait before resubmitting after changing?