Forum Moderators: open
I just resubmitted the URL to Google and noted that there is a disclaimer according to which Google doesn't guarantee that all URLs will be added to the database.
How does this work? Who decides at Google which sites get added and which not? According to which criteria?
The ONLY way for a site to get in Google is for it to have links to it from other sites that ARE listed in Google. Google will then find your site automatically when it does it's deep crawl once a month, and you will be added in the next major update after that. Total time for this process is typically 4-8 weeks.
If Freshbot finds your site, it is possible to get listings in as little as 12-48hrs! These listings may only include a few of your pages, and can be unstable, and may bounce in and out of the index, this is all normal. You listings will stabilize once you are found through the normal update process described above. The best way to try to entice Freshbot to come visit your new site is to get links to you from sites that are regularly freshbotted.
But either way, getting good links from other Google-listed sites is the key.
Whether or not submission is of any use, there is no doubt that the thing that Google cares most about is links from sites it already knows.
The usual advice is to look at the directories for the first few links. Local directories can be more responsive than those that attempt to cover the whole web. In the not-for-profit world fellow webmasters can act more like colleagues than competitors, they may be happy to give a fellow site a link.
The ONLY way for a site to get in Google is for it to have links to it from other sites that ARE listed in Google
Not true in my experience. None of my sites have had incoming links prior to being listed. (The fact that Google subsequently and for no reason reduced these sites to 0's, 2's and 3's somewhat negates this benefit.)
My last two sites were included in the index within 24hrs but had no incoming links. However, these sites are still grey-barred after 5 weeks.
(I'm now seriously considering hiring an SEO expert to find out why my sites were penalised - £1,000 if he can tell me, £0 if he doesn't. Any takers? ;) )
My last two sites were included in the index within 24hrs but had no incoming links.
This is not possible without an inbound link from a Freshbotted page. Maybe you are thinking about another SE.
Per Google, even using the Add URL page does not add you into the index until after the next monthly crawl. It also doesn't get you listed any faster than normal. Inbound links are also necessary, even an existing site will be "dropped" from the index if it looses all of it's inbound links!
A few words from Google on this subject -
"Google finds sites through a process known as "crawling" the web. This involves robot software that follows hyperlinks from site to site."
"Google is a fully automated search engine, which employs robots known as 'spiders' to crawl the web on a monthly basis and find sites for inclusion in the Google index.""When a URL is submitted to Google, we look for it in our next crawl."
"Submission is not necessary...it's likely your pages will be found in an automatic crawl before they make it into our index through the URL submission form."
"If we have not picked up your site and it has been several months, then it is likely that our spiders are not able to find your site. If you increase the links pointing to the page, Google will likely find your site in the future."
"The best way to ensure Google finds your site is for your page to be linked from lots of pages on other sites. "
You should refer to Google's help section for more infomation on how this process actually works.
All I can think of is that they were included because I put both on Adwords as soon as they were live. Perhaps referencing a site through Adwords gets it listed straight away (with a grey bar).
The only other explanation is that a site unknown to me has a link to my sites, although I find this difficult to believe as nothing appears on linkback and the domain was not detagged.
And I know the Google "info" - what there is of it - almost off by heart. What disturbs me most about the above section is the wording:
The best way to ensure Google finds your site is for your page to be linked from lots of pages on other sites.
I agree this is the best way to get found. But it's also the best way to get a PR0 once you're in there. Perhaps it would be better worded as "...for your page to be non-reciprocally linked from selected pages on other sites."
Good old Google, never give 10 words when a couple will do.
Craig
Ours was spidered within about 6 hours - and it was not submitted, had no incoming links.
Just about 3 weeks ago we got a new domain name and uploaded a temporary website to the server. While on the site I clicked on the Google bar "vote" thingy and a short time later Google showed up.
Only spidered the index.htm and still not listed, but it was there.
By guess would be a unknown link from a Freshbotted page.
You might want to check your server logs, if you see something like -
64.68.82.*
around the 10th or 11th, then that was Freshbot making it's rounds.
