homepage Welcome to WebmasterWorld Guest from 54.205.105.23
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

This 75 message thread spans 3 pages: < < 75 ( 1 [2] 3 > >     
Is The "Sandbox" Ending?
Is this Doomsday, or is Something Wonderful about to happen?
blasphemer

10+ Year Member



 
Msg#: 27172 posted 6:59 pm on Dec 19, 2004 (gmt 0)

The sandbox is coming to an end soon. It will be rolled out gradually, perhaps as fast as google used to be able to incorporate new documents before they started having problems late last winter. Even though they could probably incorporate all the documents above 2^32 (2 to the 32nd power or 4.2 Billion) level of documents established last winder at once and begin doing their cycles of algorithmic calculations, I think they are just going to introduce documents into the expanded matrix at a gradual rate. That is if they introduced all the documents into the new, exapanded matrix, they could do the full set of various algorithmic calcuations and we would see tremendous upheaval in the SERPs, on a much larger scale than ever seen before, because their new matrix is simply capable of that, and to demonstrate such power would be an admission that there was a capacity problem. Rather I think they will roll all the documents sandboxed into the expanded index over the next 6 to 8 weeks and that the process has already started. It will look pretty much like the old rolling updates google used to, except it will be strong and nearly continuous, punctuated by periods of stabality though the majority of the matrix at any one time.

To admit there was a capacity problem after all this time might be taken by some to be an admission of culpable negligence in their failure to advise potential investors regarding serious technical issues during their IPO period. I don't think there will be any culpable negligence issues because Google will not fail. However, if it did fail, I think the fact that they have withheld such information would make them subject to suit, perhaps even criminally if some of the ones who profited on the IPO were the ones who concealed the capacity problem. They would only be guilty of negligent deception IF THEY FAILED.

It's kind of like you wake up in the middle of the night and there is someone in your bedroom and they make a frightenting sound and it's dark and you see something flash towards you and you are so scared you shoot into the darkness to later find it's a serial killer wanted in a nationwide manhunt and you are a freaking hero and on talk shows everywhere, or for a change of scene, it's the neighbor's senile grandfather and you are doing 10 to 20 in max lockdown with Bubba Joe who likes to scratch his ass and sniff his fingers when he's not telling you how pretty your eyes are.

I don't have anything that would serve as proof of what I'm saying, but it pretty much stands to reason that if Google has been perfectly mum about the sandbox to this point, that they are not going to so quickly incorporate new and faster expanded technology at such a rate that it requires public statement.

If the sandbox phenomenon is over and/or in the process of ending, what would it likely look like? Would it be rolled out all at once? by topological area? by chronological time in the sandbox? alphabetically? by pages or by domains?

What will the results look like to us as they change? There must be tens of thousands of sites that have been released since last Winter that are sandboxed, as they take their place in the higher SERPs, will there be a mad assualt? or will it be more like a gradual infiltration? should we expect to see gradual changes in every area over time, steady like an hour glass, or will we see a week of dramatic change to be followed every couple of more weeks with dramatic change for a couple of months? or will we just wake up one morning to find that hurricane google has re-written the face of the internet with major devastation in it's wake and young hopeful sites seeing sunlight for the first time?

With MSN's new engine expected to go online perhaps as early as February, and google's known fondness for upstaging MS, how much later can they wait before they release the sandbox? The SERPs are apparently beginning to change. I've already heard of several people who've claimed their many-month-long-sandboxed site is out of the sandbox. Could it be that this is really the beginning of the end?

 

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27172 posted 6:03 pm on Dec 29, 2004 (gmt 0)

google (and others) can continue to innovate

They bought Blogger, and an email client with lots of storage space isn't exactly a huge innovation (not to mention that it's just an online copy of Opera's M2).

hasm

10+ Year Member



 
Msg#: 27172 posted 7:01 pm on Dec 29, 2004 (gmt 0)

Discussion of the sandbox issue has been limited to the SEO community. I have yet to see any coverage in the mainstream or financial press.

- It would be great if someone in the financial press (cbs marketwatch, jim cramer, motley fool) picked up on this, and posed the same questions that blasphemer does. That could force a real response out of google.

- It would be great if one of the wall street analysts posed a direct question on blasphemer's concerns on the next conference call. It's one thing to side-step an issue in the SEC filings, and another thing to directly lie about it.

- If the real mainstream media picked up on this, the average surfer would start to realize that google results are losing relevance, and people would start to switch back to yahoo and then the new msn. The reverse of when we all switched from yahoo to google several years ago. The MSN public relations folks would be wise to highlight how they are superior to google in relevance, when they do their launch PR blitz.

Kirby

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 7:12 pm on Dec 29, 2004 (gmt 0)

>sandbox = problem with calculating page rank

I dont think this is the issue. Marcia posted this Block Rank: Exploiting the Block Structure of the Web for Computing PageRank [citeseer.ist.psu.edu] in another thread [webmasterworld.com] which addresses their ability to calculate page rank.

In stead, you should ask if this "feature / phenomenon" is an inherent part of the way the system is supposed to work, or not.

(a) If no, then this "thing" is a sideeffect, and my best bet on timing is "gradual", or perhaps in a few "jumps".

(b) If yes, then this "thing" is found in both architectures, and it will not "end" after the supposed switch.

claus, I'm going with (b).

I don't think the sandbox will ever end - it's part of their migration to the idea of Hilltop. It is the introduction of Hilltop ideas that make it more difficult for new websites to rank quickly. PageRank, LocalScore, SEO, the rules have changed slightly.

I also once posted my theory about two indexes. The SERPs I see and the experience I have with a new website can all be explained by Hilltop. It may be two physical indexes or it may be virtual, but when google does not find Hilltop results, it returns "old" results and that is why some new site rank on uncommon queries - not queries that have less than 500,000 results as some suggest.

In light of Google's use of multiple PageRank vectors as described in the BlockRank paper, this seems to go along with claus' option (b) as well.

xcomm

10+ Year Member



 
Msg#: 27172 posted 7:18 pm on Dec 29, 2004 (gmt 0)

Sachac,


As a competitor, if G had only Yahoo to contend with, it would have been like a walk in the park.

False, to me Y! is at this time near at Googles best days end of 2003. So Yahoo has learned the search engine stuff know. And Y! is really knowing the Internet - much in the sence of Apple knowing there field.


However, MSN search is another kettle of fish.

False, MSN beta is simply delivering crap this days. So I do not see how they will be able to change anything to the better here in two month regardless how much money they can drop in. Longhorn is far away and MS is already on the way downhill. They are knowing a few fields - but this really are not SE's.

So whats about Google? Googles problem is made on her own. They simply devestated their crown juwel - their SERP - overtweaking it since two years in all directions!

What will happen? Users slowly recognize this - althrough slowly at the beginning.

MSN will play no part here with there crappy engine in the near future.

But Yahoo! may be the real danger as they really learned the SERP issue from Google (are the smart engineers gone here already from Google to Y!?).

chopin2256

5+ Year Member



 
Msg#: 27172 posted 7:33 pm on Dec 29, 2004 (gmt 0)

After 6 months of optimization, building music links, and waiting, I now rank in the old MSN and Yahoo for my bidding music words. I am still not even in the top 1000 in Google. Whatever this sandbox is, it has to favor the bigger sites over the smaller. I will never rank in Google unless I hire people to help me build tons of links I think. I used to think Yahoo or MSN would never rank me, but I waited, and finally I am out of the MSN and Yahoo sandbox for my bidding terms. Google is definitely the most biased search engine, as the only words I can rank for are non relevant and unpopular.

BillyS

WebmasterWorld Senior Member billys us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27172 posted 7:52 pm on Dec 29, 2004 (gmt 0)

It would be great if someone in the financial press (cbs marketwatch, jim cramer, motley fool) picked up on this, and posed the same questions that blasphemer does. That could force a real response out of google.

Why would google be forced to comment on something that would be considered proprietary information?

If the real mainstream media picked up on this, the average surfer would start to realize that google results are losing relevance, and people would start to switch back to yahoo and then the new msn.

Average surfers could give a flying potato about a "sandbox." Do you think they would even read the article? It is only the webmasters that complain. The average surfer will just go to another engine if they are not satisfied with the results they are seeing.

It would be negligent for a company to write an article without knowing the facts or conducting a test of their own. Google still returns better results than MSN or Yahoo, that is all that counts.

randle

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 8:07 pm on Dec 29, 2004 (gmt 0)

Google still returns better results than MSN or Yahoo

That is so true, and maybe part of the overall answer. We have been as frustrated by the sandbox as anyone out there. However, it is beginning to dawn on us that just maybe it is an effort to push back the tidal wave of spam. We continue to be fascinated how you can be first page for allintitle, allintext, allinanchor and any other command you can think of and then not be able to crack the top 500 for your main keyword. Clearly there is an artificial means at work. Hilltop, age of authoritative links? Who knows.

However what is making us think more and more about the sandbox as spam fighter is Yahoo. The results there started out pretty good, but as time has gone on they have degraded to awful. Cloakers, redirectors and link farm specialists are running totally unchecked. Everyday it gets worse. We were excited about Yahoo launching their own results and initially were quite surprised at the quality. Clearly what they lacked was the ability to at least keep some sort of check on the black hat element.

MSN has a good looking beta, but the fact is they have not spent 1 single day in the search game live. Before we all crown them “kings of search”, lets see how they do live, and against veteran spammers.

lizardx

10+ Year Member



 
Msg#: 27172 posted 8:20 pm on Dec 29, 2004 (gmt 0)

Claus, good questions, but most of them really aren't questions, it's one of the main problems with these google threads, very few programmers, almost no hardware types, zero networking types:

# So, who produces these 64-bit thingies, anyway? Is this technology mature or is it still relatively new in terms of high scale production? Is production quality reliable or does it still fluctuate, ie. how many fabs can deliver to specs and what are the tolerances?

AMD, the technology is mature, all the big box makers sell opteron servers, and have for quite a while. HOWEVER, big point, last year it was much more expensive, and Linux 64 bit support was not as mature, now it is, so if you're wondering about why it's taking so long, look not much past this. How can you ask this question really. this is very old news, obviously you don't follow hardware or servers very closely, if at all.

# Who sells this stuff? Are they really standard components, available in large quantities, like, a hundred thousand or so? From a vendor where a sale of this magnitude wouldn't seem unnatural (ie spread rumors)? worldwide shipping? anytime?

Please, who sells big blocks of hardware? Dell, HP, IBM, system builders in taiwan, you name it, they have it. Yes they are standard components.

# What's the specs like? Do they consume more power, and if so, how much more? What's the error level of these thingies, how stable are they - ie. how many should be replaced each day if you run a hundred K of them?

Yes, they consume more power. Just like a 2 gigahertz processor consumes more than a 1 gigaherz. So I guess google, or any other large consumer of computing power should stick with 1 gigaherz? I don't think so. Same space as any other rackmount machine, 1 unit.

# What do you physically do when you upgrade? Do you insert a new motherboard and connect some wires, and that's it? Or do you need new powersupplies or something else also? Whole boxes? How long does this process take per machine?

New units, it's cheaper than trying to upgrade power supplies etc.

# How about existing hardware? Scrap or re-use? Giveaway/donate? Publicity? Costs?

all big companies have this issue, I've got a very nice 1 gigahertz machine I got for free from the last big organization I worked for, they'd upgraded and were giving away the old ones. Otherwise they have to pay to have them recycled. Computer disposal is the same for all organizations, they don't keep the stuff.

# How much software should be rewritten, and how much of a rewrite is required? How many man-years are we talking about here?

only google knows, my guess is substantial, which is why it's taking so long. That plus hardware availability, it's a huge update, sort of like Windows moving from ME to XP on the consumer front. To see how hard this type of thing is, check out MS's release schedule for longhorn, check out how many components were cut out to make the deadlines.

# Precisely which benefits will this shift give? Does it add something that can't be obtained in another way?

How much benefit from moving into the future, creating an algo that can handle all the webpages at one punch, no flakey hacks like secondary indexes? Why does everyone insist on treating google differently than you'd treat any other business. You invest in your future or you lose. Car companies invest in new engine technologies, etc. That's how business works, google is a business although you'd be hard pressed realizing that reading some of the posters here.

# What's the exit strategy like, if this for some reason turns out to be all wrong?

Exit strategy? It's an upgrade, what is MS's 'exit strategy' for Windows XP? Please, there is no exit strategy, it's business, you take chances, you invest, you improve, you work on your stuff to make it stronger and better than the competition. Or you try to create monopolies to take the pressure off, not an option for Google.

# Is Google in any way forced to make that shift? Will they be forced to for some reason at some point? If so, exactly when?

If the last year hasn't shown this I don't know what would. Are they forced to upgrade, update, improve...? If they want to stay in business, yes.

# Will it decrease costs, or increase revenue?

irrelevant, it is a cost, and it's required to maintain revenues long term, just like any other major infrastructure spending, and it's a very small amount of money, by the way, do the math.

# In terms of end-user satisfaction, how much quicker than 0.31 seconds for a search of "The world" would a search become?

It's not a question of quickness, it's a question of being able to handle a world wide web that is growing constantly, which I don't think google currently is able to do.

Keep in mind this: when google started, they didn't go with old 16 bit processors, they went with modern 32 bit units. That's because that was the best you could buy and still keep costs down. Now the best you can buy and still keep costs down are things like AMD opterons. Play a little game here: why would you use very old, restricted technology when you are a: a worldwide technology leader, b: a market leader, c: flush with cash from recent IPO. Just to make some WebmasterWorld members happy? Please.

lizardx

10+ Year Member



 
Msg#: 27172 posted 8:20 pm on Dec 29, 2004 (gmt 0)

Claus, good questions, but most of them really aren't questions, it's one of the main problems with these google threads, very few programmers, almost no hardware types, zero networking types:

# So, who produces these 64-bit thingies, anyway? Is this technology mature or is it still relatively new in terms of high scale production? Is production quality reliable or does it still fluctuate, ie. how many fabs can deliver to specs and what are the tolerances?

AMD, the technology is mature, all the big box makers sell opteron servers, and have for quite a while. HOWEVER, big point, last year it was much more expensive, and Linux 64 bit support was not as mature, now it is, so if you're wondering about why it's taking so long, look not much past this. How can you ask this question really. this is very old news, obviously you don't follow hardware or servers very closely, if at all.

# Who sells this stuff? Are they really standard components, available in large quantities, like, a hundred thousand or so? From a vendor where a sale of this magnitude wouldn't seem unnatural (ie spread rumors)? worldwide shipping? anytime?

Please, who sells big blocks of hardware? Dell, HP, IBM, system builders in taiwan, you name it, they have it. Yes they are standard components.

# What's the specs like? Do they consume more power, and if so, how much more? What's the error level of these thingies, how stable are they - ie. how many should be replaced each day if you run a hundred K of them?

Yes, they consume more power. Just like a 2 gigahertz processor consumes more than a 1 gigaherz. So I guess google, or any other large consumer of computing power should stick with 1 gigaherz? I don't think so. Same space as any other rackmount machine, 1 unit.

# What do you physically do when you upgrade? Do you insert a new motherboard and connect some wires, and that's it? Or do you need new powersupplies or something else also? Whole boxes? How long does this process take per machine?

New units, it's cheaper than trying to upgrade power supplies etc.

# How about existing hardware? Scrap or re-use? Giveaway/donate? Publicity? Costs?

all big companies have this issue, I've got a very nice 1 gigahertz machine I got for free from the last big organization I worked for, they'd upgraded and were giving away the old ones. Otherwise they have to pay to have them recycled. Computer disposal is the same for all organizations, they don't keep the stuff.

# How much software should be rewritten, and how much of a rewrite is required? How many man-years are we talking about here?

only google knows, my guess is substantial, which is why it's taking so long. That plus hardware availability, it's a huge update, sort of like Windows moving from ME to XP on the consumer front. To see how hard this type of thing is, check out MS's release schedule for longhorn, check out how many components were cut out to make the deadlines.

# Precisely which benefits will this shift give? Does it add something that can't be obtained in another way?

How much benefit from moving into the future, creating an algo that can handle all the webpages at one punch, no flakey hacks like secondary indexes? Why does everyone insist on treating google differently than you'd treat any other business. You invest in your future or you lose. Car companies invest in new engine technologies, etc. That's how business works, google is a business although you'd be hard pressed realizing that reading some of the posters here.

# What's the exit strategy like, if this for some reason turns out to be all wrong?

Exit strategy? It's an upgrade, what is MS's 'exit strategy' for Windows XP? Please, there is no exit strategy, it's business, you take chances, you invest, you improve, you work on your stuff to make it stronger and better than the competition. Or you try to create monopolies to take the pressure off, not an option for Google.

# Is Google in any way forced to make that shift? Will they be forced to for some reason at some point? If so, exactly when?

If the last year hasn't shown this I don't know what would. Are they forced to upgrade, update, improve...? If they want to stay in business, yes.

# Will it decrease costs, or increase revenue?

irrelevant, it is a cost, and it's required to maintain revenues long term, just like any other major infrastructure spending, and it's a very small amount of money, by the way, do the math.

# In terms of end-user satisfaction, how much quicker than 0.31 seconds for a search of "The world" would a search become?

It's not a question of quickness, it's a question of being able to handle a world wide web that is growing constantly, which I don't think google currently is able to do.

Keep in mind this: when google started, they didn't go with old 16 bit processors, they went with modern 32 bit units. That's because that was the best you could buy and still keep costs down. Now the best you can buy and still keep costs down are things like AMD opterons. Play a little game here: why would you use very old, restricted technology when you are a: a worldwide technology leader, b: a market leader, c: flush with cash from recent IPO.

Chndru

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 8:30 pm on Dec 29, 2004 (gmt 0)

My blind guess is, sandbox is just google's reply to yahoo's paid-inclusion. Google couldn't just stand to see it's competitor making $ of the "supposed" organic search. So Google put a filter on money terms and thereby inflating the demand for adwords.

A nice no-press-nor-seo-will-ever-get-a-clue strategy.

pure speculation here..

WebFusion

10+ Year Member



 
Msg#: 27172 posted 8:33 pm on Dec 29, 2004 (gmt 0)

and an email client with lots of storage space isn't exactly a huge innovation

Ah yes...but serving keyword targeted text-based ads is. Look how many of their competitors are still scrambling to catch up to that one.

As for blogger, that is a case of (once again) beating MS to the punch (or did you miss the recent MS blog service launch in an attempt to once again play catch up). Google was the first of the big 3 to recognize the power of blogging, and to utilize it as a powerful reenue stream (ala adsense - another innovation that MS missed the boat on).

Don't get me wrong here, I think Google's overall relevance has been in decline for over 18 months now. However, it amuses me that for all the people shouting how user's are "abandoning" google, our traffic patterns has seen very little change.

False, to me Y! is at this time near at Googles best days end of 2003.

I couldn't disagree more. Yahoo is more relevant than it used to be, but it is all too easy for spammers to abuse yahoo these days. I've seen cloaked pages spend MONTHS at #1 on Yahoo for VERY competitive phrases. Couple that with "nerchants" who use the shotgun approach to SEO (i.e. build dozens of sites selling the same products from the same database) and Yahoo is a joke commercially.

So whats about Google? Googles problem is made on her own. They simply devestated their crown juwel - their SERP - overtweaking it since two years in all directions!

I think that is being overly simplisitic. Google didn;t "create" their problem, their problem arose from attempting to deal with all the "webmasters" who think the height of a good site is 100k+ datafeed affiliate pages, couples with evey otherspam technique that sprang up in rhe last 24 months or so. While I agree they've handled the spam problem poorly, there are many a site owner that can consider themselves directly responsible for all teh algo tweaks that google has been trying to combat the problem. Every time we hear from a webmaster that put up a "new" site that has tens of thousands of pages at launch who can;t understand why he can;t get his datafeed-driven site to rank, we all know what the problem is - and it ain't google.

I am out of the MSN and Yahoo sandbox

No such thing. Yahoo simply has a slower/less frequent crawling schedule.

However what is making us think more and more about the sandbox as spam fighter is Yahoo. The results there started out pretty good, but as time has gone on they have degraded to awful. Cloakers, redirectors and link farm specialists are running totally unchecked. Everyday it gets worse. We were excited about Yahoo launching their own results and initially were quite surprised at the quality. Clearly what they lacked was the ability to at least keep some sort of check on the black hat element.

Couldn't agree more. Yahoo is drowning in spam as we speak.

hasm

10+ Year Member



 
Msg#: 27172 posted 8:53 pm on Dec 29, 2004 (gmt 0)

I agree that that the mainstream won't care about arcane "sandbox" theories. But if the financial or mainstream press started to raise questions about the google's computing capacity or quality of search results -- it would start to impact the stock price, especially with MSN looming. This would "force" (put a lot of pressure) on Google to do something...or at least comment on it.

Sure the google results that one gets are still good in in relevance. But the mainstream doesn't know that they are completely missing out on 9 months worth of new sites. I think many would care. People won't switch overnight. I still use google as the first stop for any search, but I'm using yahoo as a secondary a lot more than before.

dvduval

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 11:59 pm on Dec 29, 2004 (gmt 0)

There is a big difference between a site not ranking well because of the hilltop effect, and a site not even appearing in the top 1000.

Regardless of what Google does, my sites that were launched in Spring 2004 are starting to rank and prosper on Yahoo (and even better on MSN after release of Beta). I never realized how much traffic Yahoo could produce. I have a site that is #2 on Yahoo for an 8 million result keyword, #1 on MSN Beta, and not even in the top 1000 on Google. I actually have several sites like this.

I have turned into a Yahoo and MSN fan, and you can bet I will be recommending these engines, and showing my customers where they are getting traffic. Just this week, I showed them the #2 on Yahoo, and they showed everybody in the office, and promised me lunch at a fancy restaurant next week.

The real question to me is...
How long can Google lockout new sites on 2 million+ result keywords and continue to be respected?

I can assure you it can't go on like this forever. We are not far from the 1 year point for many. How do you think Google will fare after 2 years like this?

claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 12:40 am on Dec 30, 2004 (gmt 0)

Thanks for the answers lizardx - you're right, i certainly don't follow hardware markets very closely, and i'm not very much a hardware person but nevermind, i'm glad you took the time to give some answers. I must say, though, that i don't reach the same conclusions from the information you gave as you do yourself.

>> Play a little game here: why would you use very old, restricted technology when
>> you are a: a worldwide technology leader, b: a market leader, c: flush with cash

Exactly for reason a+b. As for C, some people believe that too much cash is never good, so if you follow that school it should be invested of course, but that can be done in a lot of ways, the most traditional being aquisitions (Keyhole comes to mind).

Anyway, the words are not "very old, restricted" in my book. In stead, it's "proven and accounted for" ie. "mature". In big business, the term "accountability" is very important - it beats "benefits" in most cases. Lots of big businesses have never upgraded to XP for the exact same reason, and in computing terms those PC's and business-critical networks are peanuts compared to the Google operations. Not only that, for other companies the network might be a very important part of the business, but in Googles case it is the whole business.

All i'm saying is that you need a reason that's a whole lot more important and tangible (as in dollars and cents) than "nice to" or "futureproof" before you start messing with the very backbone of the company. Doing the same things in another way (even a better way) just isn't always enough, especially if that other way is also more expensive and more insecure.

Oh, and you do need an exit strategy. You do need to be able to roll back to something. Especially if you're Google. Other firms are less manic about this, afaik.

That said, the people at Google might still have arrived at another conclusion and decided to use 64-bit tech. After all, it's their problem and not mine, so they know a lot of things that i simply don't. I might still be totally wrong.


Nevermind all this business talk. I maintain my own POW that the so-called sandbox could be implemented regardless of technology, ie. that it could be there even if Google was already running 64 bit equipment. Back to the boxing arena.
dvduval

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 1:18 am on Dec 30, 2004 (gmt 0)

Anyway, the words are not "very old, restricted" in my book. In stead, it's "proven and accounted for" ie. "mature".

That statement will only get you so far. As I questioned 2 posts above yours, it has been almost a year for some sites, and they are not even in the top 1000, yet in the top 3 on MSN beta and Yahoo.

It really matter to me much anymore about this Sandbox effect. I'm doing great with newer sites on Yahoo (and soon MSN). Google is becoming less important to me. Whether you like Yahoo SERPS or not, they represent a big share on the search market, and are definitely "proven and accounted for" AND showing new sites. ;)

claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 1:38 am on Dec 30, 2004 (gmt 0)

dvduval those words were about 32-bit technology, not websites ;)

As for the thing known as sandbox, i still find it very hard to believe that this should not be the effect of something that was intentional (ie. not that "the sandbox" as such is intentional, but that something else - leading to "the sandbox" - is). IOW it's not something that i reckon will be fixed, until something else takes over.

So, when i say that "there's no such thing as a sandbox" (and i still do say that, even though each day it gets easier to refer to it plainly as "the sandbox") i don't deny that sites are experiencing those sandbox symptoms, i only maintain that it's a symptom, and not the cause.

dvduval

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 1:54 am on Dec 30, 2004 (gmt 0)

i don't deny that sites are experiencing those sandbox symptoms, i only maintain that it's a symptom, and not the cause.

Do you deny that people who get listed in Yahoo and MSN first will more quickly become fans of Yahoo and MSN?
I haven't seen any signs of abatement since March 2004. How long can new sites have these "symptoms" before people simple stop worrying about Google anymore?

lizardx

10+ Year Member



 
Msg#: 27172 posted 4:37 am on Dec 30, 2004 (gmt 0)

claus, I think people are forgetting something: when google created their first systems standard white box 32 bit processors were about 2 years old, maybe 3. In other words, google took the best current technology available that was a: mature and b: cheap.

then they built their algo around that. A 32 bit algo, for 32 bit machines. Now it's almost 8 years later. MSN is coming, Windows 64 bit is late, as usual for MS. MSN beta is also late, oddly enough. By roughly exactly the same time Windows 64 bit was late.

Now you can assume that MS decided to go traditional for the sake of memories or happy times in 1996, but I'm assuming that they are taking an aggressive, uptodate approach. Throwing the best tech available at the problem.

You can also assume that Google is deciding to play it safe by building systems around rapidly aging, and soon to become obsolute, technology. However, given that they didn't do this when they first built their systems, when they were much poorer, and had many fewer technical resources, it would be silly to assume that they are going to do it now. If that's what you assume, you probably don't want to look for work as a technology officer for a major company.

The current technology is here, it's mature, linux supports it, and has for some time. And it's gotten fairly cheap. And it supports a possible capacity that the current one doesn't. Which means, a 21st century solution for the 21st century.

The rollback strategy on this type of thing is exactly the same as the rollback strategy for XP or Longhorn, it's called alpha and beta testing. You test the systems, then once they are stable, you roll it out. Compare Google to any large software company releasing a new piece of software, or an OS, that's a much more accurate analogy. Like with MS, google will release its new version when it's stable enough to handle the job it needs to do. This can take time, as you can see, it's already taking over a year. And in the meantime, they've been applying hacks to the old system since they know they are replacing it. This is just common sense, it's how any smart company would do a major upgrade, MS did the same thing with ME, it was a hack because they couldn't get XP out in time for consumer release in 2000. ME got the same kind of negative press that Google currently should have been getting over the last year were it not for the unending lovefest between the market and google. Talk about the importance of a name! Cute and cuddly versus corporately bland. Google vs Microsoft. But they are both just companies with bright aggressive people working for them, who would like to become very rich as a result of their work. I see them as virtual carbon copies of each other, except ms has been around a lot longer.

Google is a software producer, it's not a software consumer, it's the one creating the next generation, the example of large companies holding off on XP isn't right [by the way, they are holding off on XP generally because XP sucks, and it's almost the same OS as the 2000 they already have installed. They are waiting for the next real generation OS from MS, not some consumer junk]

<< MS is not, nor has it ever been an innovator. As long as google (and others) can continue to innovate, MS will be at a loss, as it is now an old, slowly turning juggernaught that has never had a single original idea of it's own. >>

ha ha, they did have one idea, wipe out almost every major OS and software company out there that interferes with their core business plan: that's Windows, Office, they thought they could take over the server market as easily but luckily there are enough competent network admins out there in positions to make decisions who rejected that idea, knowing that unix/linux is a superior os for many server applications. However, don't hold your breath on google, they only have so long before they get old too, and greedy, and arrogant. I'd give them a few years on the outside.

<< As long as google (and others) can continue to innovate, MS will be at a loss >>

this one is really funny, what was google's innovation? Make a good search engine, then hire really good people out of college? Hotbot used to be pretty decent, altavista too. Then hire more phds when you can't improve it anymore yourself? That's what MS does too. Make a webbased email app, with 1 gigabyte of storage instead of 2 mB? Gee, that's really original, amazing nobody thought of that before. All google is doing is better engineering, hiring good people. Where does this google love come from, is it because we want to believe in something as we stare at these screens? It's beyond me. Oh, I know what the real innovation was, it was to make the search portal very clean and simple. That was a good idea.

xcomm

10+ Year Member



 
Msg#: 27172 posted 12:40 pm on Dec 30, 2004 (gmt 0)

1) Google

Google didn't "create" their problem ... While I agree they've handled the spam problem poorly ...

Thanks WebFusion for bringing it at the point!

Thats what I meant to say with:

They simply devestated their crown juwel - their SERP - overtweaking it ...

(But BTW: Their foundation on links is selfcreated and there is the most black hat SEO today. This was Googles main advantage once turned against them now badly.)

2) Y!
Yahoo simply has a slower/less frequent crawling schedule.

This is not true anymore. Since about 4-5 month is the crawling dense and frequency of MSN and Yahoo significantly higher as GoogleBot.

The results there started out pretty good, but as time has gone on they have degraded to awful.

Y! major changed their SERP 15/16 Dec. again:
[webmasterworld.com...]

The results from this are quite impressive - simply try it out in FireFox using the tryangla at the Google icon in the search box to swap between :-).


I agree Yahoo still has a SPAM problem, but their results are quite imressive now again. Google in their fight against SPAM is going in the opposite direction. Their results begin to going bader instead of better now. Somehow as they lost their goal in some way or math?


claus

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 12:54 pm on Dec 30, 2004 (gmt 0)

Should have kept my big mouth shut... i'll never learn to keep out of these threads... anyway, a few answers are needed, so sorry about another long post...

>> Do you deny that people who get listed in Yahoo and MSN first will
>> more quickly become fans of Yahoo and MSN?

Nope. Not at all. I agree. Of course they will.

>> You can also assume that Google is deciding to play it safe

I would say this is a fair assumption. It's pretty well aligned with what i know about that company. Still, of course i don't know everything, and i might be wrong.

>> building systems around rapidly aging, and soon to become obsolute, technology

This, obviously, conflicts with the above. They will do what it takes to improve and protect their basic product of course. That product is still search, although AdSense is their money-maker.

>> you probably don't want to look for work as a technology officer for a major company.

Regardless of my assumptions, that is not really an ambition i have :)

>> The current technology is here, it's mature, linux supports it, and has for some time. And it's gotten fairly cheap.

I have to say, lizardx, that the things you say do make sense in general. You seem to believe firmly in the above statement, while i'm holding back slightly, especially on the maturity issue. It's all about business risks, and benefits vs. costs to me. That's the main point i think separates us here. I will not rule out that you (as well as OP, others) are right and i am wrong on this particular issue, though.

As for linux, i have pointed out already that the Google flavour of linux is not what you see on your average linux box. I'd say that it pretty much should support anything that the Google engineers want it to support (given sufficient development time and funding, of course)

>> Which means, a 21st century solution for the 21st century.

The above statement... well... i'll just pretend you didn't write that. You probably know something i don't know on some issues, and i'd like to let that count in your favour.

>> You test the systems, then once they are stable, you roll it out.
>> Like with MS, google will release its new version when it's stable enough to handle the job it needs to do

Now we're speaking the same language :) Also, that's what Google has always been doing, ie. with the updates (although that's a slightly different issue).

When i mentioned rollback and exit strategy i was considering the several years worth of old indexes that Google (supposedly) has stored. Although these would not be first priority, you would still need to make sure that whatever you use them for could still be done with them after an upgrade. But basically, all this means is extra development time and extra costs.

>> This can take time, as you can see, it's already taking over a year. And in the meantime,
>> they've been applying hacks to the old system since they know they are replacing it.

Again, this makes perfect sense given the assumption that they started an upgrade process a year ago. Even though (as with most things) it's probable, we still have no proof that this assumption is right, though. They could never have started, or they could have finished it already.

What i maintain is that this "sandbox thing" could very well be something that does not relate to the current choice of technology. I could imagine more than one thing that would lead to those symptoms, that is.

You, as well as the original poster and others, connect these two issues as if none of them could exist without the other. Although i can perfectly well follow the arguments on how these things could be related, i disagree on the particular point that they need to be related.


OT:
>> the example of large companies holding off on XP isn't right [by the way, they are holding off on XP generally

- just trolling a bit here by quoting the above - please ignore - one's gotta have a little fun sometimes ;)

>> However, don't hold your breath on google, they only have so long before they get old too, and greedy, and arrogant.

Although OT; again, you're making perfect sense here (i'd have chosen other words though). This is one of the generic challenges facing growing firms, ie. how to keep that "entrepreneurial spirit" as size, business systems, business risks, and complexness grows. Imho, this is the greatest challenge facing Google at this time.

>> All google is doing is better engineering, hiring good people..

These people, however bright they might be, are not typically entrepreneur types. So, that's making the challenge above a bit harder.

mfishy

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 27172 posted 1:42 pm on Dec 30, 2004 (gmt 0)

Does Hilltop explain how the same content on a new domain, retaining an identical set of backlinks, can have dramatically different results?

This should be required reading before anyone posts in forum3.

Yes it does, this has to do with content of the site itself and recognizing it as an authority site.

Have you read the hilltop papers? The links themselves are what determines authority!

steveb

WebmasterWorld Senior Member steveb us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27172 posted 7:11 pm on Dec 30, 2004 (gmt 0)

I wish this post would put an end to the posts saying Hilltop is the sandbox. You simply can't have any understanding of Hilltop to think that.

lizardx

10+ Year Member



 
Msg#: 27172 posted 10:17 pm on Dec 31, 2004 (gmt 0)

<< When i mentioned rollback and exit strategy i was considering the several years worth of old indexes that Google (supposedly) has stored. Although these would not be first priority, you would still need to make sure that whatever you use them for could still be done with them after an upgrade. But basically, all this means is extra development time and extra costs. >>

Good points. Which could well account for why this is taking so long, there's no need to fixate on any one component, hardware, OS, algo, it can be partly each.

Re the sandbox being part of the problem. The main problem I have with trying to believe that a system like the google filesystem/algo isn't a cohesive whole where problems in one area don't affect problems in other areas is very simple. For over a year, people here argued vehemently against the idea the algo was maxed out. All the while pages indexed count stayed at 2^32, right there on the home page for all to see. After a while this became a bit too obvious, and people here finally began to go, gee, maybe the index is maxed out. Dugh. And maybe there is a secondary index, another thing that people vehemently argued against, until one day google added the count of their 2^32 page secondary index to the primary index, giving a magically doubled 2*2^32 index size. Any theorizing over the last year + that failed to take what is now increasingly clearly a fact that was spun to be speculative lunacy is as far as I'm concerned totally worthless.

It's unfortunate some of the people who used to post here aren't around anymore, there used to be some pretty good db guys who actually understood real large scale programming issues, but they seem to have moved on.

Bentler

10+ Year Member



 
Msg#: 27172 posted 3:32 am on Jan 1, 2005 (gmt 0)

It's unfortunate some of the people who used to post here aren't around anymore, there used to be some pretty good db guys who actually understood real large scale programming issues, but they seem to have moved on.

That's why you're here, Lizard, to enlighten us with your understanding of db and large scale programming issues. I just wish you'd type a little slower and use smaller words-- it's hard to keep up sometimes. ;-)

On topic, it seems if the sandbox were going away (assuming there is one -- I didn't get stuck in it personally) it would be because Google is tapping into a new data source that exposes spam sites. Maybe it has something to do with Google's job announcement for several website quality evaluators (or some such title) who could work from home. Anyone remember that listing?

lizardx

10+ Year Member



 
Msg#: 27172 posted 4:08 am on Jan 1, 2005 (gmt 0)

<<< That's why you're here, Lizard, to enlighten us with your understanding of db and large scale programming issues. >>>

when I say I miss those guys, I mean I miss those guys, when they talked they knew what they were talking about, the fact that they aren't here is revealing to me, but I can't find them anywhere else either...

Bentler

10+ Year Member



 
Msg#: 27172 posted 4:25 am on Jan 1, 2005 (gmt 0)

the fact that they aren't here is revealing to me, but I can't find them anywhere else either...

NDA's have a way of quieting those who value their solvency.

Google would do well to combine with AOL to tap new spam data, IMO.

xcomm

10+ Year Member



 
Msg#: 27172 posted 4:31 am on Jan 1, 2005 (gmt 0)

Lizardx,

Great!

until one day google added the count of their 2^32 page secondary index to the primary index, giving a magically doubled 2*2^32 index size

Maybe they are still not on 64Bit AMD Opteron yet and simply built up a second 32Bit data center or use prior redundancy for creating the second 32Bit index?
(e.g. if they have trouble porting their software to 64Bit that fast - or trouble setup the 64Bit hardware in such a bunch of new 200000 blades/1U's - or they simply had not the money to do it before going to Wallstreet)

BTW: 've read the speculations above about they maybe moved to 64Bit ... as far as I estimate this kind of setup would usually include the setup of a fully new data center like a new fab from a chipmaker like AMD and the old one is still running. Someone in the US should have a clue if Google has build something this big up somewhere or?

[edited by: xcomm at 5:06 am (utc) on Jan. 1, 2005]

lizardx

10+ Year Member



 
Msg#: 27172 posted 4:32 am on Jan 1, 2005 (gmt 0)

<< NDA's have a way of quieting those who value their solvency >>

LOL, so that's why I'm always insolvent... they can tell, no secrets, more fun that way...

<<< as far as I estimate this kind of setup would usually include the setup of a fully new data center like a new fab from a chipmaker like AMD >>>

I think you estimate wrong. The infrastructure stays the same, power requirement goes up a little bit, same 1 U racks, same data lines, same networking, unless they move up to something faster too, why not? LAN stuff is getting stunningly fast, might as well upgrade it all. Only difference is how data is processed, IMO...

<<< Someone in the US should have a clue if Google has build something this big up somewhere or? >>>

As Bentler says, picture this, you are an AMD salesman. Google comes to you, condition is you do not report this to the media. You have to sign such a NDA. Your choices are this:
1. Sign it, make a monstrously huge sale.
2. Don't sign it, and post here on WebmasterWorld.
Presumably the choice is obvious. I don't expect to read about any Google hardware purchases any time in the near future, I never have before.

For anyone taking this too seriously, HAPPY NEW YEAR!

xcomm

10+ Year Member



 
Msg#: 27172 posted 4:56 am on Jan 1, 2005 (gmt 0)

Lizardx,

The infrastructure stays the same, power requirement goes up a little bit, same 1 U racks, same data lines, same networking, unless they move up to something faster too, why not?

I would not go with you where you say it is simply to do with a farm upgrade. There should be much more power and cooling requirements to move on, let's say from a fab built before some years for 200K of 1U/2U HP ProLiants (let's estimate Pentium III 833) to 200K+ of (?) with double Opterons and double RAM maybe.

Also when I look at their crawling frequency. They are for my site still far behind MSIE and Yahoo for the last 5-6 month now. This suggests either they don't love me anymore or they are simply still short in computing power for a half year?

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 27172 posted 5:30 am on Jan 1, 2005 (gmt 0)

Also when I look at their crawling frequency.

Now this is an interesting observation. I can't personally attest to any slowdown in crawling activity, but I've heard other reports from people that I trust as reliable.

Thanks for bringing this angle up. Anyone else care to comment? Seen any slowdowns in spider activity?

eyezshine

10+ Year Member



 
Msg#: 27172 posted 7:48 am on Jan 1, 2005 (gmt 0)

I just think google went to this double database until they convert to the new 64 bit system. Which right now the new 8 Billion page database is just a cover up until they finish the upgrade.

You got to remember that Google is cheap. They are the masters of making CHEAP look like a billion dollars.

With a few hundred thousand dollars we could create a search engine that could compete with google's search right now.

The only thing google has that we don't is massive amounts of traffic. That is their biggest problem.

Search isn't trivial. It's basically simple to figure out. If we can figure out how to rank high in the search engines, we can surely create one that works well. That's the easy part.

The hard part is getting the masses to like your engine. And google has that figured out. Because they are innovative and fresh (so to speak).

I think the 64 bit system is in place right now. They are just working on the software to run it right now. The proof is in the new Mozilla 5.0 crawler they have now.

I seen it crawling my pages but they are not indexed yet on a new site I have. Then the old crawler hit and those pages got indexed right away. So I think they are about to switch over to the new 64 bit system soon.

This 75 message thread spans 3 pages: < < 75 ( 1 [2] 3 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved