homepage Welcome to WebmasterWorld Guest from 54.226.147.84
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Google not spidering
It has been 3 months now
johnnie

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27608 posted 12:02 pm on Jan 19, 2005 (gmt 0)

Hello,

Three months ago I submitted my site to google. Since then, I am only able to see my URL if I search for my domain. No decription, no cache, nothing. Google is aware of my existance, but the bot won't come by!

The occasional visits from mediapartners-google/2.1 seem to be related to adsense instead of G. I don't get it, my meta-stuff is optimized, my robots.txt is valid and my meta robots tag is set to allow, follow.

Why won't G spider my site?

Any help is appreciated, I will sticky my url if requested.

 

Import Export

10+ Year Member



 
Msg#: 27608 posted 12:05 pm on Jan 19, 2005 (gmt 0)


Perhaps this should have been a post in the Google forum?

The short answer to your question is incoming links.

f00kie

10+ Year Member



 
Msg#: 27608 posted 9:00 pm on Jan 20, 2005 (gmt 0)

I have the same problem, except in my case, it's been 7+ months since I primarily submitted my site. The web address doesn't even appear in Google at all (unlike yours, where it does). I've emailed Google and they told me they cannot check on my site but said that it was not blocked by their bot.

kaled

WebmasterWorld Senior Member kaled us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27608 posted 10:16 pm on Jan 20, 2005 (gmt 0)

A few thoughts :-

1) Whilst rare, hosts have banned Googlebot in the past.
2) Incoming links are essential.
3) Second-hand domain names may have been banned due to the previous owner's activities.
4) Invalid markup may cause robots to puke.
5) Overtly hidden text may cause sites to be banned temporarily.
6) Viewing pages with IE and the toolbar (with PR tool etc. enabled) DEFINITELY encourages Googlebot.

Kaled.

nileshkurhade

10+ Year Member



 
Msg#: 27608 posted 7:27 pm on Jan 22, 2005 (gmt 0)

Viewing pages with IE and the toolbar (with PR tool etc. enabled) DEFINITELY encourages Googlebot.

It never has it never will . . .

kaled

WebmasterWorld Senior Member kaled us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27608 posted 11:32 pm on Jan 22, 2005 (gmt 0)

Given that Google has indexed urls that I have typed in (to test script behaviour) and do not otherwise exist, I think I am pretty solid ground.

It's hard to prove a negative (as you seem to believe) but very easy sometimes to prove a positive and to my satisfaction, the case is proven definitively that Googlebot responds to toolbar data.

Kaled.

cadh

10+ Year Member



 
Msg#: 27608 posted 11:14 am on Jan 23, 2005 (gmt 0)

Hello,

I had the same problem last year.
Try to use many spider simulators available on the net.
See if you encounter '403 forbidden' error message while using the simulators

The problem that I had was with the host. No spider / crawler could index my site. So I had to change the host, and it fixed the problem

johnnie

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27608 posted 12:12 am on Jan 24, 2005 (gmt 0)

Yay! I got very lucky. I posted a legitimate link on a guestbook (not spam) and it actually had google crawling all the way to my site. I'm indexed now, on 2nd spot for one pop. keyowrd! :D

ncw164x

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27608 posted 12:18 am on Jan 24, 2005 (gmt 0)

It never has it never will . . .

Hmmm... you sound quite sure about that comment...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved