Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Why isn't googlebot crawling my website?

         

ashis06

5:15 am on Jul 21, 2008 (gmt 0)

10+ Year Member



Hi,

My website is been one month old. But still Google crawler not visited websites. Can anybody suggest ? In my experience it normally takes 10-15 days to be crawled by search engines

tedster

10:05 am on Jul 21, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What steps have you taken to invite googlebot to visit?

ashis06

6:50 am on Jul 22, 2008 (gmt 0)

10+ Year Member



I have not taken any steps yet. I am waiting for google's first visit. Is it required to follow all steps to invite googlebot after designing a new website?

My anticipation was; googlebot visits every new sites whether you have all necessary files or not . But it's already one month passed my site not yet crawled.

Should I upload nesecerry files like: robots.txt, sitemap.xml ?

Kindly suggest

tedster

7:14 am on Jul 22, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My point is that googlebot needs to find out about a new site, somehow, in order to make a first visit. That mechanism might be a link from another site, a directory listing, a sitemap, a ping from an RSS feed, opening a Webmaster Tools account - or even just a basic submission [google.com].

In the past you may have been lucky, but Google cannot know that a domain now has its first content unless that knowledge gets communicated somehow. You probably were not aware of how that happened for your past websites. Even uploading a sitemap.xml wouldn't do it unless you also submit that sitemap to Google.

Robert Charlton

6:26 am on Jul 23, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Get some inbound links from sites already indexed in Google. Ideally, these should be from sites you don't own.

ramsaywebs

5:07 pm on Jul 30, 2008 (gmt 0)

10+ Year Member



Submit your URL to Google: [google.com...]

venti

5:30 pm on Jul 30, 2008 (gmt 0)

10+ Year Member



I've had Googlebot visit a site without incoming links or submissions, from what I assumed was the discovery of the domain name registration a couple months later. Tedster's advice is spot on, I'd go with the incoming link myself as it always seems to work.

Robert Charlton

7:59 pm on Jul 30, 2008 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I've had Googlebot visit a site without incoming links or submissions, from what I assumed was the discovery of the domain name registration a couple months later.

More likely, it's from a publicly available stats page or server log. Not likely to give you a big boost in rankings, but often enough to get a page discovered (including pages your don't want discovered). See this thread about 'why is Google crawling my website?'...

Why is Google indexing my entire web server?
[webmasterworld.com...]

Good inbound links are the best way to go if you want your site both to be indexed and to rank.