Forum Moderators: Robert Charlton & goodroi
I had 20,300 pages showing for a site:www.example.com search yesterday and for the past month. Today it dropped to 509 but my traffic is still pretty constant. I normally get around 4,500 - 5,000 to that site per day and today I've already got 4,000.So, either Google doesn't account for even a small percentage of my traffic (which I doubt) or the way Google stores information about my site has changed. i.e. the 20,300 pages are still there, Google will only tell me about 509 of them. As far as I can tell, I think the other pages have been supplemented.
That resonated with something that I was talking about with the crawl/index team. internetheaven, was that post about the site in your profile, or a different site? Your post aligns exactly with one thing I've seen in a couple ways. It would align even more if you were talking about a different site than the one in your profile. :) If you were talking about a different site, would mind sending the site name to bostonpubcon2006 [at] gmail.com with the subject line of "crawlpages" and the name of your site, plus the handle "internetheaven"? I'd like to check the theory.
Just to give folks an update, we've been going through the feedback and noticed one thing. We've been refreshing some (but not all) of the supplemental results. One part of the supplemental indexing system didn't return any results for [site:domain.com] (that is, a site: search with no additional terms). So that would match with fewer results being reported for site: queries but traffic not changing much. The pages are available for queries matching the supplemental results, but just adding a term or stopword to site: wouldn't automatically access those supplemental results.
I'm checking with the crawl/index folks if this might factor into what people are seeing, and I should hear back later today or tomorrow. In the mean time, interested folks might want to check if their search traffic has gone up/down by a major amount, and see if there are fewer/more supplemental results for a site: search for their domain. Since folks outside Google couldn't force the supplemental results to return site: results, it needed a crawl/index person to notice that fact based on the feedback that we've gotten.
Anyone that wants to send more info along those lines to bostonpubcon2006 [at] gmail.com with the subject line "crawlpages" is welcome to. So you might send something like "I originally wrote about domain.com. I looked at my logs and haven't seen a major decrease in traffic; my traffic is about the same. I used to have about X% supplemental results, and now I hardly see any supplemental results with a site:domain.com query."
I've still got someone reading the bostonpubcon email alias, and I've worked with the Sitemaps team to exclude that as a factor. The crawl/index folks are reading portions of the feedback too; if there's more that I notice, I'll stop by to let you know.
[edited by: Brett_Tabke at 8:07 pm (utc) on May 8, 2006]
Guess what? I've joined your club. Gone from 150 pages or more to roughly 50 pages.
Great stuff.
The site is quite new. Less than 1 year old, launched in August last year so close to 9 months old.
Some directory links of course but others as well have been obtained.
12mths
>>Have you used link directories as your main source of link development?
nope most one way links from friends
but a friend that is experiencing the same problems his site is 3 years old but I don't know about his links
Do you know if your friend has changed his registar info recently? If so the domain can then again be seen as newish.
Plus even if you dont get directories links, doesn't mean there isn't a web of directories with your friends sites and therefore the links from there sites are not as powerful anymore.
[edited by: Relevancy at 5:31 pm (utc) on May 9, 2006]
As of today G got rid of our supplemental results.
However, they are not indexing most of our pages.
Whichever way I do site: (w/ or w/o www) it would come up 24 pages. (as of today)
Should I expect now, the fact that G got rid of the supplementals they will start indexing more pages of our site?
nothing has changed
>The dropped pages were all added within about a three-month period starting in December
thats about the only thing I'm picking up from this that the dropped pages are pages that have been added over the last 4 months and that applies to both our sites. It's almost like anything newish has been dropped
I hear some sites are only now starting to loose pages, mine have been gone for almost 2 months! and have been up and down over the last month about 5% each way, currently on a high 432 last week it was 375 should be 990 ish.
Seems like a stage thing to me, our moans and groans have been the same, just at different times over the last 2 months or so.
Another site, 8 months old, pages gone from 60,000 down to 252. Pages are coming back at a rate of 10 a week...at this rate I might have to get a job!
As I don't want to get a job I have been doing some testing. I find that I can get small new sites, to rank well within a week and stay there. OK I would have to make a lot of new sites to get over this Google indexing problem but I've got to do something. How can new sites have pages indexed before established sites? It just doesn't make any sense.
Come on Google, sort it out.
If thats the case, hopefully they're doing quickly!
The "same snippet for every page" appears to be happening in most (if not all) datacentres.
