Berkeley’s School of Information Management Systems calculated how much unique information was produced in 1999 and again in 2002. The SIMS studies are regarded as the benchmarks in this arena, measuring the volume of unique data created in the world each year saved to film, disk, optical, and paper. The 1999 study estimated that between 2,132,238 terabytes ("TB") and 3,212,731 TB of unique information were produced that year. The later study estimated that between 3,416,281 TB and 5,609,121 TB were produced in 2002, so there was a ~19% annual growth rate between ‘99 and ‘02. Assuming the same growth rate to present, somewhere between 6,869,341 TB and 11,278,629 TB of new information were produced in 2006.Interesting numbers.
In November of 2006, Google released a paper called "Bigtable: A Distributed Storage System for Structured Data" which discussed Google’s proprietary distributed storage system designed to scale to petabytes of data across thousands of commodity servers. The paper also mentioned that Google’s web index at the time was 800 TB and Bigtable as a whole was ~1085 TB. If you take the SIMS numbers as valid, in 2006 Google had organized the equivalent of 0.02% of the unique information produced in that year alone.
Google organizes 0.02% of world's information
Posted on Monday, Nov 12 2007 @ 05:10 CET by Thomas De Maesschalck