Big Daddy Part 6

I think Google should pay you guys for monitoring their servers 24/7. Good Job!

Btw, today I saw a different Mozilla Googlebot IP addrss (66.249.65.1) very active crawling my site. From what I read in this forum, it takes 3 rounds (2 rounds by Mozilla and 1 round by normal Gbot) for the page to be in the serp. Is that true?

Google certainly has a big problem. But Yahoo isn't handling 301's real well either :

[webmasterworld.com ]

But at least the pages appear!

Makes me wonder if 301's are a wise move if you can find some other workaround.

Wonder if the 301 redirect is causing google fits?

Without doubt. But see above also.

yes i just noticed a google spider at my site getting confused by a 301 redirect. Most of my site is PHP also. All I'm doing is rewriting so if google trys to crawl widgets.php it redirects to widgets.html. My html pages are back but not my php pages.

Matts Blog:- "Um, now that Bigdaddy is almost done rolling out, I think you’re more likely to see a PR update soon."

[mattcutts.com ]

The latest comment from Matt to me on the homepage only + supplementals issue:
............................
Steve Said,
March 23, 2006 @ 9:20 am

@Matt

Thank you for your answer.

But my page has 50k Mozilla-Bot visits every day for weeks (!) - and still all is Supplemental except the main page!

I believe that this is not a crawling problem.

.......................................
Gary(Ellio)Said,
March 23, 2006 @ 10:23 am

Steve,

We have also been crawled every day for the last 2-3 weeks by Mozilla Googlebot but not a single page has returned yet.

Thats why its hard to believe its “just” a crawl problem.

Could there be a little more to it Matt?

........................................

Matt Cutts Said,
March 24, 2006 @ 2:33 pm

Gary, Steve: we can still fetch those pages, it’s just that they don’t always make it live. I believe someone checked in another threshold yesterday based on the meeting that I had some with crawl/index folks. That will still take a while to go fully into effect (1-2 weeks was the estimate that I heard), but should help the remaining people who still saw some of this issue left over.

......................................

Gary (Ellio) Said,
March 24, 2006 @ 4:05 pm
Matt,

Thanks for the continuing updates much appreciated.

Based on your reply I guess we need to sit tight for another 1-2 weeks to see if the latest “threshold tweak” has the desired result and allows the lost pages return with rank and position.

Please let me know if you hear more from the crawl/index folks in the meantime.

I am passing you comments on in the WPW and WebmasterWorld threads as there are still many effected parties eager for information.

Gary

Could you explain the "threshold"? I asked him to explain his post but it was deleted. I can't even ask for an explanation without that being deleted.

Gary / Elio

Thanks for keeping us in the loop. Also for keeping things focused & polite in these testing times.

Likewise Matt & his team - appreciate your genuine concerns to sort this out and keep us informed.

Does anyone seriously believe that a minor tweak to a "threshold" could possibly explain ALL of the missing pages?

99% of my site is missing from the BD indexes. What kind of threshold tweak could possibly put that right?

Looks like Google still aren't taking this seriously enough to look for the real cause of the problem.

fyi - No more Google monitoring via the index team at sesnyc

I was following up on our communication to Google's index team and received an email saying that the "stillsupplemental" mailbox is no longer monitored.

The reply asked to communicate via Google's user support forum and "contact us" links on the various Google services

No doubt the index team has to be left alone to do their job, but gee, I think webmasters are feeling utterly isolated out here, unless you're one of the lucky few [ amongst the 100's maybe ] that received an acknowledgment and have seen an improvement - I'm not blaming anyone - just conveying our sense and situation out here.

I don't know if this means anything but for the first time since Feb 2nd of 2005 we are number one for our unique site name. Home page is still the only page not supplemental but doing the site search it is now the first page listed. This all happened sometime today for us.

it's cool the google guys are working on it however why couldn't big daddy have been tested BEFORE implementing the changes in the real world?...you know the world where people depend on google search results for their living. Why does the public have to troubleshoot for these huge corporations...I don't mind doing it but we should get paid.

Hi,

my site is not supplemental, but every subpage is suddenly totally lost in the serps. My homepage is found quite normally. Is this the same problem as supplemental hell? Thanks.

I´m able to track my visitors steps on my page - i think a lot of you can do so. So I can say they came from xyz to me and visited pages abc...

Now there are three sences.

#1 I think my side is best of all but it isn´t and therefor google don´t rank me - lets not follow this further

#2 I get a lot of visitors a day from google. let say 1000. From this 100 visitors about 80% go on and deeper into my page. Lets say they visit even more then 10 pages beginning from the entrance page.
MOst of them will buy the thing they searched for and even more.
I would say they did a search at google and found what they searched for. - Good search results

#3 I get a lot of visitors a day from google. Lets say about 1000. Only 10% of them go deeper into my page. Not even 5 pages they do visit. Probably they will not buy anything.
I would say they did a search at google and didn´t find what they are looking for. - Maybe good search result but not for this keywords.

What has google to do in case #2:
Now this (my) page is in top position google has to determine if this side is spamy or keywordstuffed....
If its ok - the ranking is ok.

What has google to do in case #3:
google has to figure out why this side is probably ok but not for this keywords. Is it spamy or something else? The side has to be reranked.

Since Jagger I see this behaviour of google in a regular cycle to rank my side from case #2 over to case #3 and then it suddenly is completly gone. But only to come back for case #3 upgoing to case #2.
This cycle remains about 2-3 weeks. On Big Daddy the phenomenom is even strong as on non-BD.

If I´m able to see good results in case #2, why doesn´t google do? They have the ability to track the users behaviour by their toolbar. So they should see the same things?

If I´m able to see good results in case #2, why doesn´t google do? They have the ability to track the users behaviour by their toolbar. So they should see the same things?

well, that is what they want to do. analytics are just for that. they give cool numbers and nice graphs but most important feature is google know user's behaviour.
in fact, I find it cool.
dumb MFA sites shall disappear from the SERPs.

what about websites with pages being deindex little by little?. does that fall into the supplemental problem too? or is google just doesn't care about that issue?

>> important feature is google know user's behaviour. <<

What do you want to say with that? Google manipulates the serps in the manner inveigling the users clicking on the adword-ads?

Looks like BD is now completely rolled out! There is a God afterall...

Hi,
When we do a: allinurl:www.website.com search on any of the following ip's:
BD
216.239.51.104
66.249.93.104
64.233.179.104
66.102.11.99
64.233.187.104
64.233.171.105
64.233.161.99
216.239.37.99

Non-BD(?)
66.102.9.99
66.102.7.99
66.102.11.104
64.233.183.104
64.233.167.147

We get figures of 9,200+ to 9,500.
However when we do the same function on www.google.co.uk then we get 940 returned.
The last time we saw this was approx 2 years ago.
This seems a bit odd, especially as it was exactly 940 2 years ago.

--- Non-BD(?)
--- 66.102.9.99
--- 66.102.7.99
--- 66.102.11.104
--- 64.233.183.104
--- 64.233.167.147

All of the above are BD. The rollout is complete.

>>> All of the above are BD. The rollout is complete.

And now the real fun begins :-)

>>>>And now the real fun begins :-)

Hopefully.

"And now the real fun begins :-) "

Ok, any idea what happens next?

After seeing many pages turning supplemental and rankings drop considerably there's been very little fun to be had here!

"I think I'm turing supplemental, I think I'm turing supplemental, I really think so.."

Well I thought I couldn't get any more depressed but this morning I can see a friend's 5 sites out of supplemental h*ll and I am still neck deep - it's as if all the pages have been lost except the main index and of course 100's of ones showing that actually no longer exist on my site for months. I have been filing sitemaps with Google since they started that game, he never has, I used a redirect to isolate the canonical problem, he never has, I update frequently, he does not - I just don't get it, follow the rules and get slammed and slammed and slammed.

>>>Ok, any idea what happens next?

My thoughts are still that the PR/BL caculation etc as done by Mozilla Googlebot is applied to the serp ranking.

"100's of ones showing that actually no longer exist on my site for months. "

Snap, its like a way back when machine.

IF BD is complete. why is my non-www redirect still only visible on one DC (little daddy)?

Matt Cutts has already suggested there will soon be a PR update. I believe that will happen very soon.

Just because we can't find a non-BD DC doesn't mean they've taken some servers offline to make these changes. I'd have to believe Matt when he says another week or so.

I'm getting a lot of Mozilla Googlebot and Googlebot 2.1 activity starting last night.

I think all the DCs are still syncing up. I noticed at least two different sets of results and several different page counts. Perhaps another quick round of spidering to catch up a bit followed by a PR recalculation.

My guess is everything will be ready just in time for April Fools day.