Forum Moderators: open

Message Too Old, No Replies

Google didn't crawl my sites for 10 days now...

does anyone else have this problem?

         

chris007

10:08 pm on Feb 11, 2004 (gmt 0)



Since the beggining of this month (1-2 Feb) google's spider didn't visit my sites, although I put lots of new pages up and i updated other.

Is this happening to anybody else? I am beggining to worry...

Any imput is apprecited.

Bobby_Davro

1:06 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



Multiple domains pointing to one site are not to blame for this I am sure. The site that I am looking at has just one domain and no links pointing to the domain without the 'www'.

Googleguy, can you shed some light onto this matter please? It looks like a real enough phenomenon.

yves1

1:21 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



>>I have now successfully installed a 301 on my site. Does anyone have any information on how long it takes for Google to catch up with this?
For one of my PR6 sites, it took google one dance to list the new domain, and 3 dances to fully transfer PR (PR 0, 5 and finally 6)

>>is there anything else that can be done to speed the process?
Ask your backlinks (including dmoz) to switch to the new pages...

BeeDeeDubbleU

1:35 pm on Feb 17, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Yves,
This is not a new domain, it's just a redirect from domain.co.uk to www.domain.co.uk. DMOZ already points to my site with the www and it has so far retained its PR5. Is this different from your experience?

yves1

2:23 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



Yes it is different. I was changing my domain name. Sorry.

Jakpot

6:43 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



Now that 64.233.161.99 has rolled out maybe the crawl problem will be addressed

Roel

6:57 pm on Feb 17, 2004 (gmt 0)

10+ Year Member



After reading the updated thread, I too can confirm that:

>>The front page, PR6, isn't cached, but it does have recorded back links.

Yes, I have the same, no index page (although mine is PR4) listed, but it has backlinks

>>links to domain with www. and without www.

Yes, I have links pointing to both without and with www.

>>Similarity in pages

I use a template, but my main content (ie. the article is completely different each time) - The sites that have been having this issue, have they been using templates? This started happening when I changed to my new template I believe.

Checked my pages again, and I noticed that only a very limited number got updated to the new versions since I changed to the template...

Another really interesting thing is this:

Google total pages in October 2003: 3,307,998,701 web pages
(see the WayBackMachine) It has been like this for most of 2003.

Google total pages in February 2004: 4,285,199,774 web pages
That's almost a 30% increase! Surely even Google can have issues growing this much in a short timeframe.

GoogleGuy, can you comment on this, a lot of webmasters having serious issues with this. Was checking on some other forums and more and more serious, decent webmasters are having this issue...

nacho_man

1:31 am on Feb 18, 2004 (gmt 0)

10+ Year Member



Having spent the last 12 months using these forums and building sites which I would like Google to look at I have come to several conclusions.

1 Google is like a grandfather. A grandfather that is frustrating, cantankerous, impossible to understand and suffering from recurring Alzheimer’s, who doesn't visit for a month, then spends the following week camped out in your front room like you are the only friend he has in the world. However he is the only Grandfather I have, and he does send me gifts in the post which are always welcome. There are days I may feel like suffocating him and doing without a Grandfather but what choice do I really have?

2 No one knows what is going on. This is difficult to accept I know. Being a geek myself I like to think I can figure stuff out, but I am yet to find one definitive reply in the whole site. Look at the forum posts and this thread. I am reminded of saying " A SEO trying to find the best way to optimise a site for Google is like a blind man in a dark room looking for a black cat that isn't there." We don't know, we may never know.

3 I don't think Google really know. :0 My experience of distributed computing and complex algorithms is limited. However I think the system is so complex that one employee of Google will give you their best guess, no more, no less, no one person knows enough, and never could. Like Microsoft and windows.. to complex.. there is no longer such a thing as a simple tweak or data update.

4 Knowing these things I understand that my traffic while rise and fall, Google will index all 35,000 pages this week, then not come back for three months. I will be 1st for my phrase this month.. never to be seen again next month. Some sites it will give 0 PR after 8 months with decent links. It won't answer our questions with a simple answer because none exists, there is no transparency because no one person understands how it all works. So pop a bud, put your feet up and enjoy the ride :)

BallochBD

8:55 am on Feb 18, 2004 (gmt 0)

10+ Year Member



Your analogy worries me. Alzheimer's is a progressive disease that gets gradually worse and for which there is no known cure. Death is inevitable. Hmmm?

Jakpot

12:33 pm on Feb 18, 2004 (gmt 0)

10+ Year Member



I have a problem with 75% of my pages showing a url only. Googebot has not visited for 10 days.

GoogleGuy, can you comment on this, a lot of webmasters having serious issues with this. Was checking on some other forums and more and more serious, decent webmasters are having this issue...

Sent inquiry to webmaster @google.com subject brandyupdate but as yet no response.

Roel

6:45 pm on Feb 18, 2004 (gmt 0)

10+ Year Member



I was doing some more checking and found that:

www.mysite.com is not listed in google
mysite.com is

I recently updated all my internal links to www.mysite.com/page

Maybe mysite.com is dying (since I removed the links) and www.mysite.com is going through a "new site" cycle (taking couple of months)?

Couple of questions
1. Has anyone seen a decrease in visitors on their site because of this
2. What if Google (recently) started seeing www. and without www. as different pages (explaining the 30% increase)

although if you go to mysite.com the index page still will come up (obviously)

King

11:46 pm on Feb 18, 2004 (gmt 0)

10+ Year Member



Hi there...

this is my first post inhere... I am a webmaster (from Slovenia EU) for a long time now...

my new pages example.tld1 and example.tld2 are in the 1st places under keyword "example"... but PR is still white! Than you can look www.example.tld3 from 1998... which is my old page with PR4... also in the 1st page in G...

What can you learn from this? Patience my friends...

BIG G isn't some coffee making machine... just imagine how freaky thing that is... Terminator and Arnold S. cums in my mind ;) it is only a matter of time, when G will aware itself and say "MAMA" one sunny day... than what!?!

U know what matters now 4 me? example.tld1 in the 1st place on G... U know what don't matters 2 me right now... white PR ;) I still have phonecalls every day... nomatter of PR 0/10 ;)

I hope and believe, that I found a really good place on the net for SEO and SEM etc... and some really cool people inhere... I hope you will enjoy my bad English ;)

K!ng

[edited by: heini at 11:31 am (utc) on Feb. 23, 2004]
[edit reason] Yup, please don't use real urls and keywords, thank you! [/edit]

BeeDeeDubbleU

8:46 am on Feb 19, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



King, you are going to get a slap on the wrist for using real domain names :o)

King

10:57 pm on Feb 19, 2004 (gmt 0)

10+ Year Member



Hmmm... sorry!

I didn't know that I shouldn't do that... in mydomain.biz anyone may add their own real urls into my guest book or forum... in my opinion... how else could anyone know with who they deal with?

K!ng

Bobby_Davro

8:05 am on Feb 21, 2004 (gmt 0)

10+ Year Member



Googlebot returned to one of the sites that I referred to (the non-spammy one) and took a few pages (23) yesterday for the first time in 20 days. Nothing has been heard from Google wrt the site yet.

Still nothing for the other site.

phxmichael

2:16 am on Feb 22, 2004 (gmt 0)

10+ Year Member



We are experiencing the same problem -- haven't seen the googlebot in about 2 weeks. (Normally, it come by every few days)

leedslad73

3:08 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



Same problem here 7 - 20 February no bot, and no fresh cache. My website is PR5, and have been adding pages and links during this period.

Prior to this my site was regularly getting spidered and fresh pages indexed.

On the 22 Feb, I saw a fresh date of 20 Feb (why 2 days behind?) on the 20 the cache was the same but today on the 22 it is the new cache.

I don't think it's affected to PR because Ive heard it happen to other sites that are anything from PR2 to PR7!

Just out of interest are any of you in the UK who are having problems? I was beginning to think it was a server problem because I couldn't send emails to AOL members for 24 hours due to a problem.

BallochBD

3:18 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



I am in Scotland

Jakpot

4:19 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



Googlebot came by at 11:45 last night.
Had a small bite and left.
75% of my pages show only urls.
Wonder when this will be fixed.
Thanks

Huge1102

4:31 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



I have a new site up. I was indexed within 4 days of going "live". I was getting daily googlebot visits, then on the 18th of this month I got to googlebot visits within 3 hours, and then "POOF" the googlebot has disappeared. It hasn't been back since the 18th and my home page (or any other pages) are indexed.

I'm a bit of newbie, so I've been kind of hoping that this was a normal variance, and that the bot would be back to usual visits soon. Now you guys have me concerned.

leedslad73

5:05 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



Jakpot, how long is a peice of string?

King

8:33 pm on Feb 22, 2004 (gmt 0)

10+ Year Member



I believe, that U should go to the basics and check a bible again...

[google.com...]

I believe, that even god G itself don't know the answers to your questions...

K!ng

Bobby_Davro

11:09 am on Feb 23, 2004 (gmt 0)

10+ Year Member



King you are probably right - Google is such a complicated set of algorithms that one small change can have unforeseen repercussions. The absence of Googleguy from this thread suggests that this is not necesarily an effect that they were hoping for. Or, if it is, it has been pushed too hard. This is the way of many Google changes - they push very hard to see the effects and then reduce the impact over time.

I would hope to see the non-spidering thing fixed towards the end of March.

BallochBD

5:15 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



Bobby, my biggest fear is that you may be right!

I have pleaded with GoogleGuy, sent emails with codewords to Google as he requested, asked him how long I have to wait for them to index me again all (so far) with no useful response. I am afraid that the end of March may be too late for me. I suffered from Florida (seriously), Austin (drastically) and Brandy (catastrophically). My one man show may not be able to survive another month or more. I have told Google this but then - why should they care?

King

6:02 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



;)

Hey... bible G is one solution... but what bout other religions?;)

Internet is much more than Google...

K!ng

rudy0826

6:26 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



I have a script on my site that emails me everytime google comes. I used to get these every 3 days or so, and I havent received an email in over 2 weeks. Also from my logs it seems as though google is not crawling as often. On my other sites It seems to be updating currently indexed content but not adding new pages. Perhaps they have a new algo that tells the bot which sites to spider. all falls into my idea of how google is forcing people to buy adwords. you want your new products to show up? buy adwords... oh we are not indexing fast enough and you have a new offer out...buy adwords. any correlation btw florida update happening right around the holiday season which makes up about 40-60% of sites reveunes for the year? killing the little guy and making it easy for the big sites with money. ebay and amazon have thousands of ad words, just like everything else it has to end for the little guy, the guys with money have to be on top if you were lucky enough to build your biz for free in google over the last years then you should be doing well right now and have a nice budget for adwords. Everything gets harder for the little guy or the market becomes saturated with little guy crap. do a search on computers [google.com...] on google is it by chance that apple, dell, compaq are top three what about the little guy that sells computers oh there he is to the right buying adwords. It has to be hard for the little guy or eveyone will be able to make money real easy and when was easy money ever promoted?

BallochBD

7:20 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



Welcome to Webmaster World Rudy. There has been much speculation on this forum about Google's motives during the last three months. Maybe I am being naive but personally I don't think that Adwords are a factor. If you forget about your personal problems with Google and have a look at a few random search results you will see that there are still plenty of commercial sites up there at the top of the rankings. Good for them I say!

The big problem for all of us who are not getting crawled is that nothing seems to be getting done about it. Last week GoogleGuy suggested that those of us who were not getting spidered could have been affected by "canonicalization". This is the process where your site is seen as two sites, [yourdomain....] com and [yourdomain....] com. He offered us a possible fix if we emailed him. You can read his comments
in message 42 of this thread: [webmasterworld.com...] I understand that a few of us did this but as far as I know none of the sites has since been spidered. Please correct me if I am wrong on this.

You do however have a point with your search on "computers". What I did notice about this particular search was that the big companies at the top of the list don't use their Keyword at all on their home pages. In fact it looks like they are avoiding it altogether. Whoever heard of a company that sells computers having a text rich home page that does not mention their main KW anywhere?

Is this taking Latent Semantic Indexing to some kind of level that I don't understand? These sites have PR8, 9 and 10 but surely they should not be found when the KW does not appear on the page? As far as I can see they use only brand and model names which LSI should not recognise unless it is specifically programmed to do so.

I am totally confused about this and I smell something fishy so can someone who knows more about this than me please help me to understand how this can happen? Following Google's guidelines should mean that these sites should just NOT be ranked. Are they paying to be at the top of the FREE listings?

Snookered

8:04 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



Googlebot finally visited my site for the first time this month, 2 days ago. It was preceded by the string "MediaPartners". Does anyone know what this is? Is it connected to AdSense?

Imaster

8:12 pm on Feb 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yes, MediaPartners is the Adsense bot.

Imaster

8:14 pm on Feb 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



..
with your search on "computers".

- Anchor text in backlinks is the answer. See the cached version of the page which says "These terms only appear in links pointing to this page: computers"
http*//66.102.7.104/search?q=cache:5iRmnZTn43cJ:www.apple.com/+computers&hl=en&ie=UTF-8

BallochBD

8:27 pm on Feb 23, 2004 (gmt 0)

10+ Year Member



I appreciate that anchor text on backlinks is important but I still think that there is something fishy about the fact that these large corporations appear to be avoiding the use of the word computer. If you sell computers you would surely expect to see this somewhere on your home page (or even something like "PC").

If I have a site selling fish I would make sure that I mentioned fish somewhere, (or perhaps fishy!) ;o{

Think about it, you sell computers and by some trick of fate your website copywriters manage to create your home page without mentioning the word computer once - even by accident? I still think this is very, very strange. This is no accident. There must be some reason for it.

This 89 message thread spans 3 pages: 89