|Inside the IT Strategy at Google|
An absolutly fascinating article referenced on /. talks indepth about Google IT strategy:
|Google's great IT advantage is its ability to build high-performance systems that are cost efficient (we didn't say cheap) and that scale to massive workloads. Because of that, IT consultant Stephen Arnold argues, Google enjoys huge cost advantages over competitors such as Amazon, eBay, Microsoft, and Yahoo. Google's programmers are 50% to 100% more productive than their peers at other Web companies, a result of the custom libraries Google developed to support programming of massively parallel systems, Arnold says. He estimates the company's competitors have to spend four times as much to keep up. |
|He estimates the company's competitors have to spend four times as much to keep up. |
that might be the case now, but technology is getting faster, newer, etc. all day.
when i started my US platform 2 years ago, i still went NFS for distributed content (over 10 Terabytes now) and it is horrible to manage today. Last week we went online with the german platform with around 4 Terabytes of data and all experience went into the new "load-balanced" setup.
Advanced technologies helped a lot, like the Red Hat GFS or very very cheap forking routers that could handle Gigabits of Traffic per second to a beyond web server farm.
Google might be very proud of their setup, but everything has a downside, even if they do not see it now. I would agree, that they might be state-of-the-art right now, but handling massive traffic loads gets easier every day with Open Source products.
Further: To compare eBay and Google is just plain wrong, because ebay does something differently to search. What they have learned about people who want to sell used coffee cups and used female underwear is far beyond everything Google could catch up to within short time either.
Handling traffic and handling business needs to be looked at seperately, IMHO...
Great article, thanks.
Not sure why Microsoft would make this list. Don't they make software too?
And I can't help thinking of that GooglePlex video, most of those "programmers are 50% to 100% more productive than their peers" seemed to spend a lot of time working with toys and walking around.
Don't get me wrong, I really do like Google. It's just that everyone seems to GoogleGush when they talk about them.
Fascinating .. thanks for pointing this out...
And now to poke a fork at my favourite target.
Since this is all based on open source software, what has been contributed back to the community.
Before someone points it out, it is perfectly *legal* under GPL 2.0+ to modify GPL licensed software for use internally without revealing the modifications.
But, it contravenes the *spirit* of open source.
By tinkering with the low-level behavior of Linux, Google engineers have solved data corruption and bottleneck problems, while increasing overall system reliability. The kernel alterations also made Google's computer clusters faster by making them communicate more efficiently.
Betcha Torvalds would love to get his hands on this.
[edited by: plumsauce at 3:36 am (utc) on Sep. 1, 2006]
> what has google contributed?
They just helped bring what was the top OCR package on the planet out of the dark bowels of forgotten code lockers at HP and turn it into quality open source code.
Where it goes from there is up to coders - not google.
Details here: [code.google.com...]
>> seemed to spend a lot of time working with toys and walking around.
it's like saying that hiring Jack Welch as an adviser for, say, $20 million a year would be way out of line because can only "work" 2 hours a week. Well, what value does he bring to the company and how much have could the company profit from his 2hr a week advice? it could simply be one multi-billion dollar idea said casually at lunch.
Sitting in front of the computer browsing websites bored out of your mind is not productive; hours mean nothing, results do.
More puff pieces from doufus journos. You claim more productivity by how many machines process your code? LOL
They have ONE product that earns cash, that doesn't speak well of their productivity or any other process but if you show the reporter your tatoos you get a few bonus points. They must have worn the "cute logo" articles out.
>>top OCR package on the planet
|does not include a page layout analysis module (yet), so it will perform poorly on multi-column material. It also doesn't do well on grayscale and color documents, and it's not nearly as accurate as some of the best commercial OCR packages out there. |
Thank you Google for an OCR POS.
|hours mean nothing, results do. |
Exactly. An important insight which has not enlightened some major US corporations.... Hours on the clock does not equal results when doing intellectual labor.
OCR developed by someone else versus
|Linux, Google engineers have solved data corruption and bottleneck problems, while increasing overall system reliability. |
wonder which would be more useful.
see the full quote in my earlier post.
|and it's not nearly as accurate as some of the best commercial OCR packages out there. |
If it's not a least *as good as* a shrinkwrap package, then it's not worth using in a business. Period. Full Stop.
|They just helped bring the top OCR package on the planet out of the dark bowels of forgotten code lockers at HP |
I would guess there was a reason it was forgotten.
Hey, if they want pr points, they can *buy* some of my source code and release it as open source too. They just won't get my *best* stuff. We all have code that is archived away.
Hmmm, as cool as I like to think I am I do not think it is cool what Google do anymore.
It is not innovative, clever, or of any use to me when Google are part of things like this - open source is just that (open), Google PLC is not that (anymore).
Anything Google does now is part of a bigger plan, which is great but don't wrap it up in a "Google is helping the Internet with it's open source type projects"
I for one can't stand PLC corporate crap. I have an even bigger contempt for the attitude of "lots of cool guys hanging out creating apps - blah blah, with cool campus, pool, nice clothes etc. etc."
Thats why I reacted against the "Microsoft Village" type of thing - my clients would be shocked if they turned up to my place and had that type of experimental crap. They would say:
"what the ... am I spending my money on? Your leisure time?"
"Why are they not improving the service to me rather than messing about with self indulgent crap!"
It is the equivalent of a small business owner turning up in a porsche to a meeting - what does the client think?
And you know what, Google are now just that. A Google Project is a mess about, a bunch of clever people exerting their self indulgent lives.
I for one am sick of Google this and that. Tell you what, why not redirect ALL resources into producing a good core search product because that is a pile of crap at the moment and has been since the launch of their many un-successful pieces of crap software.
I think it is a sign of the times - I for one have ended my affair with giving a damn what they are doing and spending more time scamming cash of the parasite which has become Google - which is trying to impersonate some sort of "for the people open source bull#*$!"
Googleguy is a distant memory. If MSN can get a "guy" to answer questions on a regular basis yet Google can't afford a dedicated guy to deal with "partners" on these forums (out of a billion quid turnover) then I feel content that whatever I do (customer support wise) as a business is genius! Yet I know it's not!
plumsauce, exactly - this is not an open source contribution.
If their engineers have even bothered a tiny bit on this - why? It does not serve the community and it is not in the spirit of it. If it is for internal purposes then so be it - but what the hell are they doing releasing it.
And Brett, who the hell cares about this OCR package by Google?
You could be Google's press officer of recent times - I know you are a techie but come on - Google's new OCR software!
Give me a break.
>>most of those "programmers are 50% to 100% more productive than their peers" seemed to spend a lot of time working with toys and walking around.
if you work 12 hours in a day, and 1 of those hours is playing and walking around, you still worked 11 hours... and we know you didn't take a long lunch, you had a 25 minute lunch in the Google cafeteria. Googlers rarely leave campus, which is a big plus for the company.
LOL, i feel like you took the words right out of my mouth.
Heres to the Smuoogel farce...
that would be "a pile of *steaming* ..."
> what has google contributed?
They submitted a huge number of patches to the WINE project when they ported Picasa from windows to linux.
Articles like that used to be called "press releases" intended to boost image and stock prices.
Google, Open Source, OCR. That's too funny.
Nice article but i am wondering,are Ebay,Amazon etc less smarter than Google to not save "as much as 4 times" on infrostructure?
All the really big hosts have to put an emphasis on cost-effectiveness, (in software, hardware, development, maintenance) that's really inconceivable even for most major software development companies, let alone independent webmasters.
My background is software development, with specific emphasis on efficient software. I've worked in, or interviewed with, a number of major software development companies, (I have in mind, but won't name, particular projects and design decisions.)
So some of the things mentioned in the article are not unique. Linux rather than Windows is a no-brainer, duh, Amazon and eBay are onboard there (which you could have figured out by just realizing neither company is brain-dead yet). Hand-tuned Linux kernels, not unusual: but Google may be talking about a level of scale of moderately-loosely-coupled processing unmatched by anyone else in the world (including three-letter agencies!) Very large matrix operations (page rank calculations) comparable to the largest geophysical simulations, but like nothing else on the web itself. It is fairly likely that Yahoo simply COULDN'T do anything like that, without investing heavily in specialty PhD's (like Google has been hiring for years--not rocket scientists, just the folk that rocket scientists bring their really hard problems to!) Of course, Yahoo may not think it's worthwhile to take that approach -- they may be optimizing in a different dimension.)
As for open-source OCR, that is something that I have been seriously looking at for my own personal suse, which is not something that I can say about any other software ever discussed in these forums. (Don't ever make the mistake of assuming that your technical interests and needs are shared by anyone outside your own clique -- I could get rich taking the other side of that bet against anyone.)
Who benefits from having the open source community working on improving an existing OCR package?
OTOH, while I'm no expert in the open source arena, doesn't Google's work with the Mozilla Foundation count for anything?
Going back to the article (i.e. Google technology)...
It's true that Google has great expertise in creating distributed parallel networks (or however you would like to describe their server network) however, there is a huge caveat with their technology - it is designed for search and as such synchronization of data is not critical. For just about any other application (e.g. Amazon and Ebay) data has to be perfectly synchronised.
I'm not sure that Google's confidence in its technical superiority is entirely justified. Many large companies develop techology for use in house. I believe one robot manufacturer actually began life making wheel barrows but needed an automated painting machine and so designed one themselves (in the 1970s).
I hope this is a pertinent question or observation.
Why is google, whom we are assured is one off the most profitable companys using the internet,
supposed to be an "open source" champion?
Secondly, I presume a majority off us here work for a living, either employed or running our own businesses,,,
is anyone offering their articles or product for free?
Apart from resisting the Microsoft giant, I canna figure out the open source everything free software folk,
I suspect the majority owners off Google aren't going to hand over their hard earned, entirely deserved rewards to anyone :-)
|Why is google... supposed to be an "open source" champion? |
The article states that Google uses a modified version of linux on its servers. By using an open source operating system, they are able to customise it to their particular needs and avoid divulging information in the form of purchasing licenses. Clearly, Google is benefitting in a big way from open source software.
Further, if Google were able to successfully promote open source software, etc. it could only harm Microsoft, so from Google's point of view, open source software is absolutely wonderful.