Forum Moderators: goodroi

Message Too Old, No Replies

Robots/Slurp, etc - what does this mean

I am a newbie - not sure what this means

         

ramnyc

12:58 pm on Jun 22, 2005 (gmt 0)

10+ Year Member



I look at my website stats everyday, and everyday I notice that:
Inktomi Slurp goes to my site
MSNBot goes every day
Googlebot goes everyday
LinkChecker goes almost everyday
Jeeves goes almost everyday.

Then there are numbers like:
Inktomi: 2262+215
MSNBot: 415+41
Googlebot: 358+397

What does it mean? Is this good? bad? what's the plus number?

What "robots" and all do I want at my site - like which should I be looking out for?

My most searched for keyword/term comes up super high on MSN & Yahoo but is nowhere to be found on google - is that related to all this?

Any info would be really appreciated.

Clint

2:21 pm on Jun 22, 2005 (gmt 0)



That's just the SE (search engine) spiders crawling your site. It's normal, and you want it for if they did not, you would not get indexed in the SE's. I don't know about those numbers and the + sign.

"Slurp" is Yahoo, the rest are self-explanatory.

See msg 2 here were I explain about the robots.txt file. (It turned out to be N/A for the thread, but the info is still valid).
[webmasterworld.com...]

You should have a robots.txt file in your root directory of your websites, that's where all that data goes. I don't know if not having one can affect you in Google. It could be that, or it could be due to the massive screw-ups that went on and are still going on for some with G. [webmasterworld.com...] . Or, your site could be 'sandboxed'.

ramnyc

2:39 pm on Jun 22, 2005 (gmt 0)

10+ Year Member



What does sandboxed mean? Is there a way to find out how Google is viewing your site, etc?

Clint

3:01 pm on Jun 22, 2005 (gmt 0)



In short, it's an "area in cyberspace" where G places new websites. You need to browse this thread. It's a TON of posts, I can't type all of it in one post, and I'm really busy now, but all the details are there.
[webmasterworld.com...]

Look through these topics for "Sandbox" and "sandboxed", just do a "find" on each of the pages for 'sand' and you'll probably find some. [webmasterworld.com...]

You can also find details on it by searching any SE for 'google sandbox OR sandboxed' (no quotes).