{"id":112071,"date":"2025-05-01T10:29:48","date_gmt":"2025-05-01T17:29:48","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=112071"},"modified":"2025-12-12T13:02:29","modified_gmt":"2025-12-12T21:02:29","slug":"iceberg-on-backblaze-b2","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/iceberg-on-backblaze-b2\/","title":{"rendered":"Iceberg on Backblaze B2"},"content":{"rendered":"\r\n
\"A<\/figure>\r\n\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n

If you work with cloud storage and data lakes, you\u2019re likely hearing the word \u201cIceberg\u201d with increasing frequency, occasionally prefixed by \u201cApache\u201d. What is Apache Iceberg, and how can you leverage it to efficiently store data in object stores such as Backblaze B2 Cloud Storage? I\u2019ll answer both of those questions in this blog post.<\/p>\r\n\r\n\r\n\r\n

But, first, join me on a brief trip back in time to the beginning of the twenty-first century, a long-ago time before the emergence of big data and cloud computing.<\/p>\r\n\r\n\r\n\r\n

\r\n

A timely shoutout to the Data Council conference<\/h4>\r\nWe recently attended the 2025 Data Council<\/a> conference and caught Ryan Blue, co-creator of Apache Iceberg\u2019s excellent presentation (featuring some very entertaining slides). If you want to hear more about topics like this one, feel free to join us at Backblaze Weekly<\/a>, an ongoing webinar series where we discuss all things Backblaze.<\/div>\r\n\r\n\r\n
\r\n
\"An\r\n
Ryan Blue speaking at the 2025 Data Council conference. Note: His shirt says \u201cthe future is open\u201d. We agree!<\/figcaption>\r\n<\/figure>\r\n<\/div>\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n

CSV: The lingua franca of tabular data<\/h2>\r\n\r\n\r\n\r\n

In the early 2000s, if you were working with tabular data, you were likely using either a relational database management system (RDBMS)<\/a>, such as Oracle Database, or a spreadsheet, likely Microsoft Excel.<\/p>\r\n\r\n\r\n\r\n

Data stored in an RDBMS is highly structured, meaning that it MUST conform to a predefined schema. For example, you might create an employee table with columns such as first name, last name, date of birth, hire date, and so on. The database schema holds metadata such as the name and data type of each column, whether that column must have a value, relationships between tables, and so on.<\/p>\r\n\r\n\r\n\r\n

A spreadsheet, on the other hand, has some<\/em> structure\u2014data is arranged in rows and columns, similarly to an RDBMS\u2013but each cell can contain anything: text, a number, a formula referencing other cells, even an image in today\u2019s spreadsheets. We say that a spreadsheet is semi-structured data<\/em>.<\/p>\r\n\r\n\r\n\r\n

At the turn of the century, each database and spreadsheet had its own proprietary file format, optimized for its own requirements, and often not at all publicly documented, but the need to be able to exchange data between applications led to broad adoption of a file format to allow just that: comma-separated values, or CSV.<\/p>\r\n\r\n\r\n\r\n

Here\u2019s a simple example of some tabular data represented as CSV:<\/p>\r\n\r\n\r\n\r\n

employee_id,first_name,last_name,reports_to,job_title,is_manager
1,Gleb,Budman,,CEO,1
123,Patrick,Thomas,1,\"VP of Marketing\",1
45,Yev,Pusin,123,\"Head of Communications and Community\",1
678,Pat,Patterson,45,\"Chief Technical Evangelist\",0<\/pre>\r\n\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n
CSV is simple and flexible enough that it was easy for me to type that example up manually and import it into Microsoft Excel with no problems at all. Note that, as well as the commas, the double quotes in the CSV data are part of the file format, and do not appear in the imported data:<\/p>\r\n\r\n\r\n\r\n
\"A<\/figure>\r\n\r\n\r\n\r\n
CSV has a lot of advantages: It\u2019s simple; flexible; widely understood; the optional header line means that data can be somewhat self-describing; and it\u2019s not controlled by any single vendor.<\/p>\r\n\r\n\r\n\r\n
CSV does, however, also have a few disadvantages, including:<\/p>\r\n\r\n\r\n\r\n