Forum Moderators: open

Message Too Old, No Replies

Googlebot not spidering my site at all

is visiting but not spidering

         

Typester

4:29 pm on Oct 9, 2002 (gmt 0)

10+ Year Member



Hi all,

I've noticed Googlebot has been to my site 51 times in the past week (6 times yesterday) and has spidered "0" pages.

Apparently I have a PR0 rating. Does this have something to do with the lack of spidering by Googlebot?

I'm frustrated by this whole rating thing because I don't know how to fix it. Help!

Typester

Grumpus

4:32 pm on Oct 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



What did it do on the 50 visits? Pull the robots.txt 51 times? I'm not clear on what you're saying it DID do, if it didn't spider you.

G.

Typester

5:05 pm on Oct 9, 2002 (gmt 0)

10+ Year Member



Thanks for responding, Grumpus.

I don't use a robots.txt file. I use a robots re-visit tag.

It apparently didn't do anything. On my reports it said Googlebot visited my site but did "0" (zero) pages. I don't know what this means.

Typester

Powdork

5:17 pm on Oct 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I VERY SERIOUSLY DOUBT this is your problem but...
you have a link to a file called blackjack with alt text download free fonts. Given Googles distaste for the online casino thing, perhaps you should change the filename.

Sasquatch

5:31 pm on Oct 9, 2002 (gmt 0)



It apparently didn't do anything. On my reports it said Googlebot visited my site but did "0" (zero) pages. I don't know what this means.

I believe this relates to the if-modified-since discussion.

It might mean that it has requested 51 files, but since you haven't changed them since the last time that google got them, your server did not have to send any.

If that is the case, and I do not *know* it to be true, then google will just use the copy it already has, saving both of you some bandwidth.

Typester

6:56 pm on Oct 9, 2002 (gmt 0)

10+ Year Member



I have modified my content in the past couple of days so I would think it would do a crawl for updated info.

Interesting take on the blackjack thing. I never thought of that. Since that free font is a popular download, I wouldn't want to change the name. Mind you, if it posed a problem with Google, I'd change it in a heartbeat. Interesting thought, though.

I don't know. I'm ready to give up. I have the feeling my site will appear sooner or later on the SE's but it's frustrating trying to figure this out.

Thanks for your help.

Powdork

7:21 pm on Oct 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



How long has your site been around?

taxpod

7:52 pm on Oct 9, 2002 (gmt 0)

10+ Year Member



Cool site dude or dudette. Very handy.

I'm thinking that maybe it came to your site as a result of inbound links? Looks like a pretty new site as Alexa doesn't have sufficient data to rank yet. I get no hits on Dogpile for you except those on Google where your domain name is included in the text on a couple sites. These don't count as inbound links though so they don't help.

My only suggestion is for you to get some inbounds. I would do a search of every site that indexes other sites with fonts and see if you can get a few thousand of these to link to you. If you've already done that, it's just a matter of time. Oh, and get yourself on dmoz and Yahoo! These are necessary. Dmoz will take you time but you can pay for Yahoo! - it's worth it.

Typester

8:37 pm on Oct 9, 2002 (gmt 0)

10+ Year Member



Powdork, my site has been around since May 2002. Pretty new.

Taxpod, thanks for the compliment on my site. I am listed in the DMOZ and Google directories, the first places I submitted in June 2002. My site seems to have found it's way into a zillion little directories. Don't know why I have a PR0.

As for paying for Yahoo...I'm Canadian so their price for submission goes up by at least 50 percent for me. Tough decision.

Thanks for the suggestions. Gotta get working on those inbound links...maybe that will help my lousy PR.

Typester

julinho

11:13 pm on Oct 10, 2002 (gmt 0)

10+ Year Member



Typester,

From down here, all your pages have a gray toolbar, not a PR0.
A gray bar means that either your pages were not indexed yet or they were banned.

I checked your robots.txt using this tool [searchengineworld.com...] and it returned several errors.
I would advise you to put a robots.txt. If you donīt have any protected subdirectories, the robots.txt will be very simple, and the same tool above will help you with that.

jady

11:43 pm on Oct 10, 2002 (gmt 0)

10+ Year Member



I can agree with the above - and I would get rid of the blackjack (in the coding has href=blackjack.html).

Also - Internal javascripting has caused us problems in the past. We now make all Java external in a seperate directory called /js... Then ban it from the robots in your robots.txt.