Welcome to WebmasterWorld Guest from 54.197.94.30

Forum Moderators: open

Started Hosting 16th Feb. No Deep Crawling Yet

Started Hosting on the 16th of Feb. Still no Deep Crawling.

   
1:08 am on Mar 9, 2004 (gmt 0)

10+ Year Member



I Deleted the Old Website & made a completelty new one. and I launched it on the 16th of Feb .
My Index page got indexed & cashed on the 23rd Feb.
Starting from 1st of march I My index page gets cashed & indexed every single night for the past seven days , But till No I haven't been deep crawled yet?

How long should I wait , Did I Do something wrong?
I don't have a robots.txt & I have the meta (index,follow).
When do I get deep crawled .

4:49 am on Mar 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



bhefny,

Check out this thread, you are not alone.
[webmasterworld.com...]

M i n n a p p l e

8:26 am on Mar 9, 2004 (gmt 0)

10+ Year Member



my website started yesterday, and there's still no deep crawling yet :p ah, give good old google sometime, and it'll definately find you.
8:30 am on Mar 9, 2004 (gmt 0)

10+ Year Member



I'm ready to wait but for how long?
would someone give a rough estimate!
1 month , 6 weeks 3 month?
because if I did something wrong I want to fix it now
8:31 am on Mar 9, 2004 (gmt 0)

10+ Year Member



facing the same problem for my 4 websites for more than one month now.
the pages are not deep crawled, while google indexes my main page everyday

the websites are having PR 5 and 6
best regards,
dhaliwal

10:44 am on Mar 9, 2004 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



"would someone give a rough estimate"

A month.

10:54 am on Mar 9, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Since I've been seeing this Googlebot behavior for about a month already, I'll up that to two months.
9:41 am on Mar 10, 2004 (gmt 0)

10+ Year Member



we are monitoring the googlebot using the crawler alert tool on many sites , we do see a decrease in deep crawl but on high PR sites it still does the deep one like always
/BP
6:02 am on Mar 12, 2004 (gmt 0)

10+ Year Member



well what do you mean by high pr?
is 5 pr taken as good pr to be deep crawled?

i have 3 websites with pr 5, but still no deep crawl while it crawls my main page everyday.

i ma loosing lot of traffic just cause of this

dhaliwal

7:03 am on Mar 12, 2004 (gmt 0)

WebmasterWorld Senior Member bigdave is a WebmasterWorld Top Contributor of All Time 10+ Year Member



from [google.com...]

5. How long does the Google robot take to index a URL once it's been submitted?

Depending on the timing of the submission and of our crawl, the entire process can take between six and eight weeks.

This has been their response for years, and even with freshbot they haven't changed it.

What I have found is that without deep links, you can expect to get your index in the first month, a moderate crawl the second month, and a complete crawl the third month. That has been for sites of a few hundred to a few thousand pages.

If you have deep links, it seems to behave that way for each link, so you can get a lot more of your site crawled sooner by getting those links.

Of course, getting good quality high PR links will urge googlebot on even more.

7:25 am on Mar 12, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>If you have deep links

That's the trick.

Think of Googlebot as aimlessly following links around the web. It generally likes high PR pages better simply because they have more incoming links (that's how they got their high PR).

To get a good/fast deepcrawl you need a large number of good deep links. Then the law of averages (and PR) will come into play and Googlebot will get to your deep pages much quicker.

For a new site with only links to the index page I generally plan on 60 days for a complete crawl of 20K to 30K pages. Getting some strategic deep links can bring that time down to 20 to 30 days.

7:43 am on Mar 12, 2004 (gmt 0)

10+ Year Member



"would someone give a rough estimate"

Could be 1 day, or 1 month. It depends on how many links your site has, and your timing. Sites with good PR or a high number of links get crawled frequently. Lesser linked sites need to wait for the "crawl a huge chunk of web" googlebot cycle, which isn't on any known schedule.

8:07 am on Mar 12, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>>Started Hosting 16th Feb. No Deep Crawling Yet

Took over the marketing around the same time for two sites that had zero backlinks and were indexed as 'Supplemental Results'.

Re-did the titles, etc. and added a few relevant PR4 pointers, and the index pages are pretty much freshened daily, but nothing else is crawled & all other pages still show up as 'Supplemental Results' after a month.

Perhaps it's just the newbie test bots wreaking havoc: [webmasterworld.com...]

 

Featured Threads

Hot Threads This Week

Hot Threads This Month