| 3:36 pm on Aug 23, 2007 (gmt 0)|
Flash will never replace magnetic disks just like L2 cache in CPU will never replace RAM: perhaps flash will take intermediate position between RAM and proper hard disks, and only that if they make throughput fast, then, maybe, key OS files and swap will be located on flash drives, with the rest placed on magnetic disks.
| 3:43 pm on Aug 23, 2007 (gmt 0)|
When you say never...
I have moved all my Internet-facing servers based at home (eg SMTP, DNS, all my static Apache Web servers, plus some Java stuff) and have it on a low-power machine that boots and runs from a 4GB SD memory card plugged into the side. So you can do it today for a few quid if you can boot off a memory card. (I'm running Ubuntu Linux in this case.)
The machine *does* also have a hard disc, but it's powered down almost all the time to save noise and power.
PS. Swap is a bad thing to put on flash because (depending on the exact type) it wears out quickly.
PPS. Mods, zap this link if you wish, for my setup: [earth.org.uk...]
[edited by: DamonHD at 3:45 pm (utc) on Aug. 23, 2007]
| 4:37 pm on Aug 23, 2007 (gmt 0)|
I was careful to use word "never" with another word "replace" :)
L2 cache will never replace RAM, this is a pretty close analogy when it comes to flash/magnetic devices - if you think about it starting from processor registers as the fastest possible storage available for processing, and at the same time it can hold the least data, then you move down the line with L1, L2 or even L3 caches - all bigger than the previous one but with increased latency and lower cost, so then you go to RAM, and normally it was magnetic hard disk after it - flash just fits in nicely between RAM and magnetic hard disks.
| 5:23 pm on Aug 23, 2007 (gmt 0)|
But in some cases you won't need anything after the flash in that storage hierarchy, since the reason that magnetic disc was there was for cheap, bulk, non-volatile storage at the bottom.
For me, for my application, the SD memory card *has* largely replaced the use of hard disc, and I'm not alone.
So I quibble with your use of "never" since it *has* replaced hard disc in at least one case, ie mine! (I had ~15 hard discs before, so I could further argue that this SD card has replaced about 14.5 of them for me!) B^>
But in general I think many people will be well served by one less moving part to break or go wrong in laptops and other mobile devices...
PS. I entirely agree with the notion that Flash will not replace all hard discs, at least not for a very long time. They are truely amazing things, and get more so every year. I remember explaining how one worked at some gathering, and realising that the 'Winchester' disc's co-inventor from IBM Winchester was standing listening. Apparently I did OK! We were looking at a 10MB (yes, megabyte) 'washtub' style disc stack at the time...
[edited by: DamonHD at 5:27 pm (utc) on Aug. 23, 2007]
| 9:56 am on Aug 24, 2007 (gmt 0)|
The technology exists today (or very soon will) to create an entire internet server no larger than a Mars bar (including the power supply and fan) and entirely solid-state. (Even cooling can be solid-state by using ion-wind). Such a server may only have a few Gig of storage and might not be able to to keep detailed logs forever, and it might not be the fastest server on the planet, but it would work adequately for the websites of more than 99% of businesses on the planet. Indeed, such a device could handle many typical websites.
Flash may not replace hard disks entirely but eventually a fast solid-state technology will emerge.
| 10:24 am on Aug 24, 2007 (gmt 0)|
|but it would work adequately for the websites of more than 99% of businesses on the planet. |
No it would not because 99% of businesses look at cost rather than speed - most of cheap web hosting deals have either unlimited or many gigabytes of storage offered to each client, if suddenly this storage costs 10 times more than magnetic hard disks that do this job just fine, then flash has no chance.
Correct me if I am wrong but currently production of flash is often done by manufacturers that also make RAM or other chips, and they often juggle production from one type to another depending on costs. This means that production of flash is more or less tied to something that will always be more expensive than disks (per GB) - RAM. I'd say there is no way flash storage will ever compare in price with magnetic disks. It is another matter however that some other interesting technologies are in development that should be much cheaper and faster than flash, but it remains to be seen if they ever have a chance with magnetic disks that have a pretty long roadmap ahead of them.
I am saying this sitting on over 50 TB worth of disks, I dread to think how much it would have cost me if they were flash based - if I had that kind of money I might as well have retired :)
| 7:43 pm on Aug 24, 2007 (gmt 0)|
I did not say that a "Mars-bar" server would be the most cost effective option for businesses, only that it would work adequately for most. However, given that such a device could be manufactured for $20 to $30 (if not now then very soon) I would have thought it would be cost-effective too.
| 8:10 pm on Aug 24, 2007 (gmt 0)|
Glad to see Seagate get involved it will be helpful in developing new mass storage devices. As someone who has seen quite a few computers scrapped by hard disk failures I would pay extra for something that was less effected by temperature and did not have the possibility of a catastrophic failure.
Edit: must be low power too.
[edited by: Tidal2 at 8:12 pm (utc) on Aug. 24, 2007]
| 9:26 pm on Aug 24, 2007 (gmt 0)|
|only that it would work adequately for most |
Ok, lets say you have this device and you offer web hosting plan of 100 MB for $5 per month. This is enough for most sites. However your competitor uses magnetic disks and offers 1000 MB for the same $5 per month. Both you and your competitor know that most customers won't use even half of the space offered. Who would get clients however, you or your competitor?
More importantly knowing that most people won't use up allocated space it allows to oversell it, so to maximise revenues from the same CPU/memory you need to whack as much as possible disk space into the system to host as many customers as you can.
Flash has got its niche, but there is no way it will replace magnetic disks anytime soon - I'd say never because there are new ideas of persistent non-magnetic storage that makes flash look bad.
| 10:48 am on Aug 25, 2007 (gmt 0)|
Cheap solid-state servers need not be shared. In a price war, that comparison might win.
Hard disk capacities are already greater than most people or businesses will ever need (unless they are into video editing and/or storage). 4GB flash drives are already common. In three years, we can expect to see 16GB flash drives - enough for a high definition movie in a package far smaller than a DVD or magnetic disk. Seagate are not one of my favourite companies, but they are probably right to move into this area.
| 11:42 am on Aug 25, 2007 (gmt 0)|
You won't win Kaled - CPU/memory/motherboard/power usage all these factors will mean that per GB cost of hosting on solid state will be much higher than using much bigger magnetic disks - we are talking here many times over rather than 20-30%.
I use a server with 16 TB of magnetic storage in it - this could host lots of sites and the price per site will be extremely low because processors, memory, motherboard and other things are all divided by a lot of gigabytes that gives very low cost per GB. In your case you won't even be able to achieve density of solid state storage anywhere near mine - this completely disregards cost that will be 5-10 times higher.
You say that in 3 years we will see 16 GB flash drives - but right now top disks are 1 TB in size, in 3 years we will see at least 2-3 TB sized ones, and new interesting developments in magnetic recording promise at least 10 times density growth - this means transfer speed will increase greatly too. Operationally it is only the seek time that is the problem, but this can be more or less solved with good caching on server side.
There will need to be a true miracle development to happen for solid state to have a chance to take over position currently enjoyed by magnetic disks - not only low price, but size comparable with the best in class magnetic disks. Not happening in the next 10 years - it's like exotic materials that can help make faster CPUs - these materials are sure better than silicon, but their cost is so high that they are not practical.
That said I, of course, welcome development of solid state stuff as it is very useful in some niche areas where magnetic disks can be replaced for good - MP3 players for example, or USB sticks replacing floppies etc :)
[edited by: Lord_Majestic at 11:44 am (utc) on Aug. 25, 2007]
| 12:04 pm on Aug 25, 2007 (gmt 0)|
|but right now top disks are 1 TB in size |
If the storage capacity of solid-state drives is adequate for the job and speed and reliability are at least as good as magnetic drives, then solid-state drives will ultimately replace magnetic drives in most applications. It matters not one jot that magnetic drives have capacities a hundred times greater if all that space is simply wasted.
I doubt flash technology will ultimately replace hard disks but some form of solid-state technology certainly will. Clearly, Seagate are of this opinion and I expect many other manufacturers are too.
Physical size, power consumption, and cost (absolute, not per MB) are not factors that can simply be ignored.
| 12:26 pm on Aug 25, 2007 (gmt 0)|
|If the storage capacity of solid-state drives is adequate for the job |
But my small hatchback car is adequate for me personally, however public transport is based around big buses, trains or even airplanes - all of which have much higher capacity than something that is adequate to me.
If you look at it from business point of view then having much lower cost of magnetic disks is going to win the day - you simply have no chance with same GB of storage costing 30 (!) times more.
If you look at who is making Flash you will see same manufacturers who make RAM. They often shift production from one to another to maximise their profits, in effect dramatic (order of magnitude) falls in price are simply not going to happen. It was the consumers that demanded bigger flash sticks that drove this production in the first place, but now the sizes are high enough for consumers to keep their data, which is why those manufacturers are turning to business applications of their stuff but it simply not going to work as easy because they will be competing with big magnetic disks directly now whereas before they were competing with much smaller (yet magnetic too) floppy disks.
Ain't going to happen - price competition in hosting business is such that even 50% higher storage price is bad, here we have 3000% bigger.
| 10:42 am on Aug 26, 2007 (gmt 0)|
You could fit a dozen solid-state servers on a single board (think mobile phone technology). That would dramatically cut the cost of dedicated servers. It's hard to see this not happening.
Marketeers like to talk big numbers (Gigabytes, GigaHertz, etc) but they'll happily switch to other things such as low power and the gullible will believe that instead.
| 11:05 am on Aug 26, 2007 (gmt 0)|
|You could fit a dozen solid-state servers on a single board (think mobile phone technology). That would dramatically cut the cost of dedicated servers |
Ok, you will fit 12 x 32 GB solid state disks, this is just over single 320 GB disk. Right now new 1 TB disks ship with just 3 platters - this means that it is possible (should the manufacturer wanted so) to ship disk with 1.6 TB capacity, and in 5 years this will be good 10 times bigger. Flash won't scale that well - doubling up from a high figure is much harder, consumers already got big enough solid state disks (MP3 players, USB sticks etc).
Maybe you can fill whole server with solid disk - but this will always cost you 7-10 times per GB when compared with the magnetic disks. How can you compete with this kind of price disadvantage? This approach has got only limited applications like in the military, but mass market for servers with full solid state disks has zero chance of commercial success due to way too high costs. It does not mean there won't be any, just they won't exceed 3-5% market share, if that.
No chance whatsoever to replace high capacity magnetic disks any time soon - not in the next decade.
| 4:27 pm on Aug 26, 2007 (gmt 0)|
So, if someone were to bring out a petabyte disk for the same price as today's terabyte disks, that would reduce the real-world cost of hosting by 1000... I don't think so. Or perhaps it would make the host a thousand times better, or perhaps it would make the websites on it a thousand times more interesting, or a thousand times faster, or bring in a thousand times as many users.
If the storage capacity is adequate, that's all you need, by definition. Also, if the CPU, etc. can keep up with speed of the network connection, that is good enough too.
There are probably arguments against the system I have suggested based on limited numbers of IP addresses, however, when 6Byte IP addresses are introduced, that argument will vanish.
Incidentally, when I introduced the idea of a miniature solid-state server, I only did so to provoke thought. In situations where physical size and power are more important than speed and storage capacity, solid-state storage will always win. Speed and capacity are likely to improve faster than the requirements of users, therefore, solid-state drives will edge-out magnetic hard drives. It may take ten years, but they have already seen off floppy disks - even high capacity ones. In ten years, solid-state terabyte drives might be achieved costing only $20. Do you really think joe-public will still need or want hard disks then?
| 4:42 pm on Aug 26, 2007 (gmt 0)|
|In ten years, solid-state terabyte drives might be achieved costing only $20. Do you really think joe-public will still need or want hard disks then? |
Joe Public will buy whatever mainstream computer builders (like Dell) are selling. This means that unless solid state is cheaper than magnetic disks it won't get into mainstream cost-sensitive builds.
Looking at perspective magnetic recording technologies and flash it appears to me that there is no way that flash will ever be cheaper than magnetic disks (I don't count here floppy disks since their development stopped 2 decades ago and they became obsolete). Important consideration is that flash based solid disks are made in volume by people who also make RAM. RAM is even faster (both seek and throughput) than flash, but it did not take over hard disks because of price. Guys who make flash often shift production from RAM to Flash depending on where they make most of money. There seems to be some kind of manufacturing similiarity between them, if that's the case (and it appears to me to be so), then flash is inherently going to be high price option that will never challenge dominance of magnetic disks.
There are new interesting solid state technologies in development however - they seem to be based on different principles than flash and it may help lower prices, but I doubt we will see cheap versions of them anytime soon.
Anyway, I think this discussion is turning into dead end - let's wait 10 years and then revisit this issue on the basis of reality that will exist in 2017 :)
| 5:27 pm on Aug 26, 2007 (gmt 0)|
I'll wait for mram based hard drives to come around and I do realize I'll probably waiting for some time. However, once mrams become popular and cheaply produced I believe the market for flash will die. Even so, I don't see magnetic disk storage becoming obsolete for many years to come.
It's a lot easier to recover lost data from a magnetic disc drive than a flash drive. I definitely wouldn't run a hosting server using flash drives to store and deliver client's websites.
DamonHD, good work on the low power laptop server and thanks for the link. I've also tinkered around with the same type of project. Have you done any field tests to test power consumption and server speed using a ram disk to store static files and scripts?
| 6:29 pm on Aug 26, 2007 (gmt 0)|
A few of the things that are nominally on hard disc (and on the SD card) are clearly being cached in memory by Linux, such as the cron script that I use to poll by USB digital inputs every 2 minutes to monitor battery/power status. The hard disc is simply NOT being woken every couple of minutes to fetch them.
Also, one of the main purposes in life of this server is to dedicate about more than 1GB of its 2GB to caching stuff that my mirror servers might need from it quickly. So creating an explicit RAM disc would conflict with that: the fewer firm partitions the better IMHO.
And today I just got ~7.3h off-grid with this set-up, so I'm pretty happy with it for the time being. B^>