Hand submit or wait for it?

[matuloo]

Hello everybody

I just registered so let me say hello to everybody :)

And right at the start I have some newbie questions - I am not a webmaster newbie thou, just new to the google game.

1. is it better to hand submit a site to google or to make some link trades with already listed sites and then wait for the spider to pick the site up?

2. I heard something about the robots.txt file, like - the spider searches for the file first and that it has to return a 404 error and not 403. Can anyone bring a little light into this for me?

Thanks for your answers.

[GoogleGuy]

1. Both. :) Submit it, but also get good links from sites like ODP, Yahoo, etc.

2. Good summary. The robots.txt proposal recommends (but does not require) that a 403 response to the robots.txt should mean "stay out of this site completely." We followed that behavior until the last couple months. However, we saw several users (and at least one ISP) return a 403 as a mistake so often that we changed our behavior for this. It's fine to have no robots.txt file.

Hope that helps, and welcome to WebmasterWorld!
GoogleGuy

[vitaplease]

>We followed that behavior until the last couple months. However, we saw several users (and at least one ISP) return a 403 as a mistake so often that we changed our behavior for this. It's fine to have no robots.txt file

Interesting Googleguy,

that behavioral change must have added quite some extra pages to your index ;)

[rfgdxm1]

>that behavioral change must have added quite some extra pages to your index

Not very likely. Only a clueless webmaster would configure things to return a 403 for robots.txt. Any host doing this by default is too incompetent to be in the business. I can't believe stupidity is all that widespread.

[thunderpaste]

Wouldn't it be faster to block Google with a robots file than setting up a 403? I don't really understand this either.

[matuloo]

Thanks for your answers guys.

Now one more question : when a site is submitted to google by hand, does a human visit it or is it just crawled by the spider?

[Macguru]

Welcome to WebmasterWorld [webmasterworld.com] matuloo.

It is usually crawled by a GoogleBot spider. When a human from Google visits your site, you could be in big trouble! :)

Directories such as ODP and Yahoo sent human visitors to sites, search engines such as Google and Fast send spiders.

[thunderpaste]

This is why it is important for you to get a good ODP listing. Google knows ODP and Yahoo sites are human reviewed so when Googlebot picks up a link in ODP it knows automaticly that your site has met the conditions required by ODP, hence a good link in ODP means more to your site.

[rogerd]

Welcome, Matuloo... If you haven't already found it, you might find the Google Knowledge Base [webmasterworld.com] a source for information about how Google works and things you can do to make your site more attractive to Google.