Forum Moderators: Robert Charlton & goodroi
I had 20,300 pages showing for a site:www.example.com search yesterday and for the past month. Today it dropped to 509 but my traffic is still pretty constant. I normally get around 4,500 - 5,000 to that site per day and today I've already got 4,000.So, either Google doesn't account for even a small percentage of my traffic (which I doubt) or the way Google stores information about my site has changed. i.e. the 20,300 pages are still there, Google will only tell me about 509 of them. As far as I can tell, I think the other pages have been supplemented.
That resonated with something that I was talking about with the crawl/index team. internetheaven, was that post about the site in your profile, or a different site? Your post aligns exactly with one thing I've seen in a couple ways. It would align even more if you were talking about a different site than the one in your profile. :) If you were talking about a different site, would mind sending the site name to bostonpubcon2006 [at] gmail.com with the subject line of "crawlpages" and the name of your site, plus the handle "internetheaven"? I'd like to check the theory.
Just to give folks an update, we've been going through the feedback and noticed one thing. We've been refreshing some (but not all) of the supplemental results. One part of the supplemental indexing system didn't return any results for [site:domain.com] (that is, a site: search with no additional terms). So that would match with fewer results being reported for site: queries but traffic not changing much. The pages are available for queries matching the supplemental results, but just adding a term or stopword to site: wouldn't automatically access those supplemental results.
I'm checking with the crawl/index folks if this might factor into what people are seeing, and I should hear back later today or tomorrow. In the mean time, interested folks might want to check if their search traffic has gone up/down by a major amount, and see if there are fewer/more supplemental results for a site: search for their domain. Since folks outside Google couldn't force the supplemental results to return site: results, it needed a crawl/index person to notice that fact based on the feedback that we've gotten.
Anyone that wants to send more info along those lines to bostonpubcon2006 [at] gmail.com with the subject line "crawlpages" is welcome to. So you might send something like "I originally wrote about domain.com. I looked at my logs and haven't seen a major decrease in traffic; my traffic is about the same. I used to have about X% supplemental results, and now I hardly see any supplemental results with a site:domain.com query."
I've still got someone reading the bostonpubcon email alias, and I've worked with the Sitemaps team to exclude that as a factor. The crawl/index folks are reading portions of the feedback too; if there's more that I notice, I'll stop by to let you know.
[edited by: Brett_Tabke at 8:07 pm (utc) on May 8, 2006]
Again thanks a lot.
rachel
[edited by: tedster at 4:54 pm (utc) on May 13, 2006]
My situation stands better now than it has since this sites issues began last sept.
Until the current situation of missing pages started my site was fully indexed but only a handful of pages were ranking.
1100 pages go missing and the remaining pages returned to their pre september rankings. A few days after emailing the bostonpubcon2006 at gmail.com addy my site's pages started returning.
According to todays reply I have "939 pages listed".
When i site search on google using the 4 variants suggested they all return 10,300 pages. Of course the site doesn't have that many pages, it has around 1300 and up to 999 there are no supps.
Other points to note from the email:
"This suggests to me that the
situation is currently self-correcting"
&
"I've verified
that your site has not been manually penalized"
I hope my mess and slow improvements gives others a little hope in these times of trouble.
Thanks again to whoever took the time to look at my site and reply.
Edit to add that most of the reindexed pages are back to pre sept positions.
4 out of the 10 pages are added a %22 to it for example:
www.widget.com/super-widgets.html%22 and that is the reason we get a 404 error.
What could be the problem?
Would anyone know if this has anything to do with the site not getting fully indexed?
They were all returned May 7th and 9th as 404 not found. I have no idea why or where google got these url's. They are not in our site map.
.com/eoifcbfxfc.html
.com/lxcunhvf.html
.com/qbstjxyvx.html
Hope Google Guy looks into this as it seems wide spread
2 days.
PS, if you did not use sitemap yet, I recommend you use it now.
The sooner webmasters take G's crap affiliate program (adsense) off their sites, the sooner this whole mess will be solved.
Don't believe me?
Get 10,000+ webmasters to remove their adsense codes today and I'll bet this "impossible" issue will be resolved by sometime next week.
As soon as webmasters look at the bigger picture, (honestly is a couple weeks of missed adsense income REALLY hurting you/us?), the sooner we get our voices heard.
