The reason behind "Big Daddy"

Big Daddy = 64 bit Linux (?)

[arran]

Does anyone else think the "change in infrastructure" is google moving to 64 bit Linux?

arran.

[gmiller]

No, that wouldn't require any obvious changes. They're most likely changing the data structures used for their indices and such. A change to 64-bit IDs in the indices is reasonably likely to be part of this revamp, but that has nothing to do with 64-bit operating systems or 64-bit CPUs.

[BillyS]

>>Does anyone else think the "change in infrastructure" is google moving to 64 bit Linux?

That's what I think, a move to 64 bit CPUs. That could be the reason why it's taking so long - they are actually swapping out hardware.

[Jamie_uk]

i'm with gmiller on this i suspect if anything it will be to upgrade to 64-bit IDs in the indices as well as some other database architecture changes

[longen]

Its because they're planning to install MS Vista.

[Lord Majestic]

32-bit document ID limitation is nonsense - as a person who created sufficiently big index I can vouch that.

Big Daddy has got to be about hardware - native 64-bit operations will allow fast access to lots more memory than normal 2 GB - it is possible to access more than that while being in 32-bit mode, but memory speed won't be that fast. Dealing with data using 64-bit registers would also help performance. All in all pretty big changes which explains why it takes so long to release it.

[otech]

So why would a move to new hardware change the index, and why would'nt Matt Cutts mention that (its hardley sensitive info)?

I dont believe that moving to 64bit hardware is going to cause any improvement to the canonical issue, and that is something Matts told us to keep an eye on for improvements.

'Infrastructure' is so wishy-washy it could mean anything, but if it was just a hardware update I doubt we would even notice...

[kaled]

As a general rule, when migrating to new a new platform, you make only the absolute minimum of changes to your software (to make it work precisely as before). Only when you're satisfied that everything is working correctly do you proceed with algo changes, etc.

This being the case, whilst it is possible that Google is testing a 64bit platform, it is unlikely that they would conduct such tests using prototype algos. In other words, when testing a 64bit platform, they would compare results with those of a 32bit platform and expect them to be identical.

Kaled.

[longen]

The changes may not relate especially to the Index - they may be looking forward to a more competitive future, new free and paid services, more revenues.

[Lord Majestic]

So why would a move to new hardware change the index

Its not a secret that new index is much bigger than current one, seems to be twice as big - 16 bln pages. If it is indeed a 64-bit platform (necessary to break through 2 GB memory limit without memory bank switching penalties) with more memory and possibility to rewrite some key hotspots using 64-bit assembly, then it makes sense to increase data sizes so that this move is justified.

Why not port existing code and index to it first? Because it must have been done already and such a huge hardware change should ultimately come with real improvements visible to the end users.

[Oliver Henniges]

Funny; I had a similar idea just the past days

>A change to 64-bit IDs in the indices is reasonably likely to be part of this revamp, but that has nothing to do with 64-bit operating systems or 64-bit CPUs.

I don't think so. Taking into account the massive number of operations needed for PR-calculation it is quite clear that this can only be done with very concise programming on machine-language-level. Given this, it is a huge difference whether you can acces the whole index with one CPU-step or whether you need two or even more.

I also think that going beyond 4 billion pages has required a massive restructuring of the algos on just this machine-language-level. On the other hand this 32/64-bit disussion now lasts for more than two years and I doubt it takes so long for google to exchange the hardware. It seems likely that there is much more to big daddy than just this issue (eg IPV6?), but indeed it might play a role. However, it'll always remain speculation.

[Lord Majestic]

Taking into account the massive number of operations needed for PR-calculation

PR calculation is off-line.

[phish]

Personally I think the biggest part of this is the switch to Mozilla bot which uses the http 1.1 protocol. They said this was about better understanding of redirects, and fixing canonical issues. With the 1.1 protocol they can actually get the headers, where as 1.0 is less efficient at this. I do believe once Big Momma rolls out completely, the old Gbot along with the older protocol will be phased out. It will definately help them with some of the ongoing issues they have. Just my 2 cents