{"id":23131,"date":"2015-02-04T04:58:08","date_gmt":"2015-02-04T12:58:08","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=23131"},"modified":"2021-08-13T05:03:28","modified_gmt":"2021-08-13T12:03:28","slug":"hard-drive-data-feb2015","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/hard-drive-data-feb2015\/","title":{"rendered":"Reliability Data Set for 41,000 Hard Drives Now Open-source"},"content":{"rendered":"

\"blog-stats-data\"<\/p>\n

Stats geeks: Now it’s your turn.<\/strong><\/p>\n

Backblaze has released the raw data collected from the more than 41,000 disk drives in our data center. To the best of our knowledge, this is the largest data set on disk drive performance ever to be made available publicly.<\/p>\n

Over the past 16 months, I have been posting information about hard drive reliability<\/a> based on the raw data that we collect in the Backblaze data center. I have been crunching those numbers to correlate drive failures with drive model numbers, with SMART statistics, and other variables.<\/p>\n

There are lots of smart people out there who like working with data, and you may be one of them. Now it\u2019s your turn to pore over the data and find hidden treasures of insight. All we ask is that if you find something interesting, that you post it publicly for the benefit of the computing community as a whole.<\/p>\n

What\u2019s in the Data?<\/h2>\n

The data that we have released is in two files, one containing the 2013 data and one containing the 2014 data. We\u2019ll add data for 2015 and so on in a similar fashion.<\/p>\n

Every day, the software that runs the Backblaze data center takes a snapshot of the state of every drive in the data center, including the drive\u2019s serial number, model number, and all of its SMART data. The SMART data includes the number of hours the drive has been running, the temperature of the drive, whether sectors have gone bad, and many more things. (I wrote a blog post<\/a> correlating SMART data with drive failures a few months ago.)<\/p>\n

Each day, all of the drive \u201csnapshots\u201d are processed and written to a new daily stats file. Each daily stats file has one row for every drive operational in the data center that day. For example, there are 365 daily stats files in the 2014 data package with each file containing a \u201csnapshot\u201d for each drive operational on any given day.<\/p>\n

What Does It Look Like?<\/h2>\n

Each daily stats file is in CSV (comma-separated value) format. The first line lists the names of the columns, and then each following line has all of the values for those columns. Here are the columns:<\/p>\n