Welcome to WebmasterWorld Guest from 54.166.222.116

Message Too Old, No Replies

Pages Dropping Out of Big Daddy Index

Part 2

     
7:59 pm on May 8, 2006 (gmt 0)

WebmasterWorld Senior Member googleguy is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Continued from: [webmasterworld.com...]


internetheaven, you said:

I had 20,300 pages showing for a site:www.example.com search yesterday and for the past month. Today it dropped to 509 but my traffic is still pretty constant. I normally get around 4,500 - 5,000 to that site per day and today I've already got 4,000.

So, either Google doesn't account for even a small percentage of my traffic (which I doubt) or the way Google stores information about my site has changed. i.e. the 20,300 pages are still there, Google will only tell me about 509 of them. As far as I can tell, I think the other pages have been supplemented.

That resonated with something that I was talking about with the crawl/index team. internetheaven, was that post about the site in your profile, or a different site? Your post aligns exactly with one thing I've seen in a couple ways. It would align even more if you were talking about a different site than the one in your profile. :) If you were talking about a different site, would mind sending the site name to bostonpubcon2006 [at] gmail.com with the subject line of "crawlpages" and the name of your site, plus the handle "internetheaven"? I'd like to check the theory.

Just to give folks an update, we've been going through the feedback and noticed one thing. We've been refreshing some (but not all) of the supplemental results. One part of the supplemental indexing system didn't return any results for [site:domain.com] (that is, a site: search with no additional terms). So that would match with fewer results being reported for site: queries but traffic not changing much. The pages are available for queries matching the supplemental results, but just adding a term or stopword to site: wouldn't automatically access those supplemental results.

I'm checking with the crawl/index folks if this might factor into what people are seeing, and I should hear back later today or tomorrow. In the mean time, interested folks might want to check if their search traffic has gone up/down by a major amount, and see if there are fewer/more supplemental results for a site: search for their domain. Since folks outside Google couldn't force the supplemental results to return site: results, it needed a crawl/index person to notice that fact based on the feedback that we've gotten.

Anyone that wants to send more info along those lines to bostonpubcon2006 [at] gmail.com with the subject line "crawlpages" is welcome to. So you might send something like "I originally wrote about domain.com. I looked at my logs and haven't seen a major decrease in traffic; my traffic is about the same. I used to have about X% supplemental results, and now I hardly see any supplemental results with a site:domain.com query."

I've still got someone reading the bostonpubcon email alias, and I've worked with the Sitemaps team to exclude that as a factor. The crawl/index folks are reading portions of the feedback too; if there's more that I notice, I'll stop by to let you know.

[edited by: Brett_Tabke at 8:07 pm (utc) on May 8, 2006]

8:01 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi Folks

You might find Matt's today post of relevance to this thread:

Indexing timeline
May 16, 2006 @ 12:24 pm

[mattcutts.com...]

8:23 pm on May 16, 2006 (gmt 0)

10+ Year Member



f that's the case, then i'm not surprised your rank has dropped loads. because google rates all the links that point TO your site by seeing if they are relevant to the page content. if you've changed to a new subject then all of your old inbound links are going to be about completely different subjects.
so you are effectively starting from scratch again - you've got no decent inbound links.

sorry, let me add that my site lost ALL its rankings last sept. after much thought i decided that the mobile / cell phone related stuff had to go. the subject of each page didnt change, it was just they were focused at the phone market.

I redesigned the site to freshen it up, make it more user friendly and rid my on site association with anything to do with mobile phones. I still never had any rankings. My rankings returned when 1000+ pages disappeared. My remaining 200/300 regained there old positions.

Until recently a lot of the page elements were of the following type:
red widgets from my wonderful site
green widgets from my wonderful site
pink widgets from my wonderful site

I made an effort to rid many of these similarities. Be it in the title, description, keywords, h tags, body text.

Ive tried many many combinations to see what works best. All I know is that since Ive started changing these more of my pages started to return, now they all have been reindexed but some are ranking higher than others depending on what combinations Ive been messing with.

8:41 pm on May 16, 2006 (gmt 0)

10+ Year Member



I just finished reading the blog, and what scares me is the outbound links. How do you know if google feels they're relevant enough to link to? I'm not linking to any ringtone site from a real estate site, but it's possible some of my outbound links may go to questionable sites.

I recently saw somewhere a tool to do a 'banned' check on google, should I download that and make sure every site I link to isn't banned? If so, can anyone recommend a more legit tool for that? I was unsure of the site I went to, so wasn't about to just install some random .exe file. I'd like a trusted tool, or even a search command to check for banned domains. Thanks!

8:47 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You might find Matt's today post of relevance to this thread:

Wow, Iíll say; thatís a very comprehensive rundown on whatís happening. Certainly some very interesting comments he made about reciprocal links, link relevancy and affiliate sites.

You gotta give Google credit for communicating so openly, (wether you like the message or not). Certainly more information than you would ever get from other SEís.

Now off to figure out what it all means.

8:47 pm on May 16, 2006 (gmt 0)

5+ Year Member



ooh this is a good idea... if you get such a tool, please share it with us!
9:04 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member steveb is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Matt's post is interesting, both because it covers good territory, but also because it shows that folks at Google really don't have a good appreciation of the actual problems.

Matt thinks the crawl is better, while this bot is far weaker than the old one.

I sent an example of a site that has 300 inbound links to the main page, and 50+ to internal pages. Most inbound links are PR4 or higher, several PR5, a couple PR6. The links out are mostly to government sites. It has no affiliate links and makes not one nickel. Yet it dropped 2/3 of its 70 pages in the big drop last month. It now is fully indexed again, when I threw another PR6 link at it. The point is, the weakling bot is the culprit, not anything else. That's not to say what he mentioned doesn't cover MOST things, its just to point out that (as usual) something they think is working pretty good is actually not working nearly as well as they think.

The worst thing though, by far, is apparently they have done nothing about supplementals and the rotting effect they have on domains, other than removing the oldest half. To be blunt, what a dumb idea that is. Apparently a lot of us will have to wait another year and a half for Google to obey 301s that have been in place for eleven months.

9:10 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



randle

"You gotta give Google credit for communicating so openly, (wether you like the message or not). Certainly more information than you would ever get from other SEís."

Sure..credit to Google for communicating openly. Having said that, I wish to give Forum30, The Mother of All Forums, credit too for motivating Google to communicate openly, IMO ;-)

9:12 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Originally posted in wrong thread
I'm not sure how to respond Google employee 1's post.

Big Daddy is out and fully loaded.
Everything is fine and if your site is missing pages then it's because you have some type of penalty ie. too many recip links, no authority, poor linking out, etc.

And it seems Google is happy with their new index.

After doing some searches for personal matters like Joe Q. Searcher, I immediately switched to Yahoo and MSN.(where I found what I was looking for)

I suppose my searching skills are "too advanced" vs. Joe Q. Searcher for objective opinions. 4-5 word search stings, etc.
Joe Q. simply doesn't search for "widget reviews of unique article"?

They simple type in "widget" or "widget review" or "widget article"?

If that's the case, once again, why is Google considered the "best"?
Any basic SE can pull up the 10 most relevant sites for broad terms like that.
Google doesn't (and never) held any superiority for that.

Google USED to hold superiority when it came to "odd phrase in unique town" or "unique product for unique problem".

Seems that's no longer the case...

Fair enough, enjoy your new "algo" Google.
Rest on your laurels and Phds.
I don't support nor use inferior products simply because they have the biggest advertising budgets.
Unfortunately, most people do.
At least, that's what they are betting on....

9:16 pm on May 16, 2006 (gmt 0)

10+ Year Member



it's possible some of my outbound links may go to questionable sites

Drop them if you arent 100% certain of them. Why jepordise your site for something you arent so sure of or have doubts about. Unless they bring you a ton of traffic which then means you need to choose between traffic from the links and building a site to please only Google.

9:49 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Anyone know a good lawyer? Class action anyone :) ...?
10:03 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



kidder

"Anyone know a good lawyer? Class action anyone :) ...? "

Trust me. A good SEO will serve you better than a good lawyer :-)

10:11 pm on May 16, 2006 (gmt 0)

5+ Year Member



everyone seems to have comments...

does anyone have answers or suggestions how to get a site back to being indexed?

10:21 pm on May 16, 2006 (gmt 0)

5+ Year Member



"does anyone have answers or suggestions how to get a site back to being indexed? "
Clean your site and submit a reinclution request.
I did it and today my site is back after 17 months of heavy penalty.
10:25 pm on May 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



newyawkseahawk

In case you need more info on how to file a reinclusion request, Matt wrote very informative post here:

[mattcutts.com...]

Good luck.

10:30 pm on May 16, 2006 (gmt 0)

5+ Year Member



Based on Matt Cutts post..

We have a site that is half affiliate and half our own products with our own shopping cart...

Are we being penalized for having affiliate links?

If that's the case what should we do get rid of half of our income due to Google?

This 249 message thread spans 17 pages: 249
 

Featured Threads

Hot Threads This Week

Hot Threads This Month