It was a nice relaxing day I was planning to take some family photographs but unexpectedly I decided on writing something on our socio-techno lives. Since most of our photos make it to facebook somehow it would be interesting to share with all of you on how facebook is keeping up with humongous 60 billion image files on its system. Facebook eats up a lot of storage with its photo application alone. Members are adding 220 million new photos every week. Facebook currently has more than 1.5 petabytes of storage for its photos, and that is growing at a rate of 25 terabytes a week. Last year, Facebook spent an estimated $30 million on NetApp storage appliances alone just to keep up with the growth of photos and other uploaded content.
“Facebook is spending well over a $1 million per month on electricity alone. Bandwidth is likely another $500,000 or more per month on top of that. The company has earmarked $100 million to buy 50,000 servers this year and next. And sources say they’ve been buying one NetApp 3070 storage system per week just to keep up with all this user generated content. At up to $2 million each, that adds up quickly – we’ve heard estimates that they may have spent as much as $30 million this year alone with the company. And the icing on the cake – earmark another $15 million per year in office and datacenter rent payments”. Source: Facebook EBITDA
To reduce some of these costs, Facebook decided to engineer its own storage architecture called Haystack. On the whole, Haystack will allow Facebook to switch from expensive, commercial storage appliances to commodity off-the shelf hardware. It is going from a traditional network file system to something more parallel to stripped-down network application that does only what it needs to do. Not only will Facebook get the cost savings of going commodity, but they also get a 3X improvement in storage capacity. In other words, what used to take 30 disks to store, now will take only 10. It’s heard in the future, Facebook may even open-source the architecture so other companies can benefit from it. Not bad, for something that was built by three engineers (Doug Beaver, Peter Vajgel, and Jason Sobel).
