homepage Welcome to WebmasterWorld Guest from 54.237.99.131
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Google Spider Question
BMach

10+ Year Member



 
Msg#: 23566 posted 5:01 am on Apr 24, 2004 (gmt 0)

Google spiders my home page everyday but it doesn't seem to go any deeper than that. Do I need to wait for a deep crawl and if so how often do those occur?

 

wanderingmind

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 23566 posted 9:28 am on Apr 24, 2004 (gmt 0)

There are too many factors, including whether your site has a link from a high PR site or not.

However, if Googlehot has already visited your homepage, here's what has worked for me. Change the content/ make modifications on the homepage everyday. Even minor modifications will do. You will notice that the visits get speeded up, and then there will be a deep crawl. Without any modifications to content, too, you will get a deep crawl, but much later.

sem4u

WebmasterWorld Senior Member sem4u us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 23566 posted 9:33 am on Apr 24, 2004 (gmt 0)

You probably need some more incoming links and if you don't have a site map then build one. Also, check that all of your links work.

Hobbs

WebmasterWorld Senior Member hobbs us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 23566 posted 1:44 pm on Apr 24, 2004 (gmt 0)

Sorry if any of the this is obvious:

1) In your meta:
meta name="robots" content="index,follow"
2) Make Sure your Robots.txt does not ban any furthur crawling
3) www.google.com/webmasters to be memorized
Regards

GoogleGuy

WebmasterWorld Senior Member googleguy us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 23566 posted 4:03 pm on Apr 24, 2004 (gmt 0)

What everyone else said, plus check your links to make sure they're good, and see whether you can get a few more incoming links. If the site is new or has low toolbar PageRank listed, that could help.

BMach

10+ Year Member



 
Msg#: 23566 posted 7:47 pm on Apr 24, 2004 (gmt 0)

I will try using meta name="robots" content="index,follow" on my home page. I have never used that on pages in order for them to be indexed and followed. Other than that I seem to be doing everything else right.

BMach

10+ Year Member



 
Msg#: 23566 posted 3:36 pm on Apr 26, 2004 (gmt 0)

I still can't get google to crawl past my home page. Google hits my home page and updates the cache everyday for my home page but it just won't follow any of the links. What am I doing wrong? I can think of plenty of sites that get a complete crawl everyday. Here's some info about my site and what I am doing:

- PR of 5 and has been indexed for 1 year.
- 300 inbound links with 50 that show up in a G link check.
- meta name="robots" content="index,follow" on every page.
- The home page is the site map. All pages are one step away from the home page.
- The home page content is changed everyday and receives a hit from google everyday.
- All links are working.

What am I missing? How do sites get a deep crawl every day?

micsten

10+ Year Member



 
Msg#: 23566 posted 4:02 pm on Apr 26, 2004 (gmt 0)

"Google spiders my home page everyday but it doesn't seem to go any deeper than that."

I have exactly the same situation happening with the site I administer at the moment BMach. Although I recently updated the whole site. It is also PR5 and has been indexed for about 2 yrs.

I am just waiting for the next big deep crawl by Google (I guess that will be happening within the next week or two), before i start trying anything else. - not that I know what to try!

john_Max

10+ Year Member



 
Msg#: 23566 posted 12:14 am on Apr 27, 2004 (gmt 0)

I have Start one website 8th of April, and at 25th of April Google toolbar showing PR4. Is it normal?

Also Google is visiting my website home everyday, but not rest of that pages. Google has visited others pages 2 weeks agon.

Please let me know the reason.

john_Max

10+ Year Member



 
Msg#: 23566 posted 12:16 am on Apr 27, 2004 (gmt 0)

fyi, when the google deep crowl will start in this month?

lunarboy1

10+ Year Member



 
Msg#: 23566 posted 12:18 am on Apr 27, 2004 (gmt 0)

In my expierience with a new site recently, the PR will drop soon. I was high in the serps then fell into oblivion after like a 2 week "grace period" and i'm now crawling out of the "sandbox" i guess. Some support it, some don't. Other threads talk about this in greater detail, just search for sandbox or check out the "90 Days" thread.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved