Daltix<\/a> is a pioneer in providing complete, transparent, and high-quality retail data. With global industry leaders like GFK and Unilever depending on their pricing, product, promotion, and location data to build go-to market strategies and make critical decisions, maintaining a reliable data ecosystem is an imperative for Daltix.<\/p>\nAs the company has grown since its founding in 2016, the amount of data Daltix is processing has increased exponentially. They\u2019re currently managing around 250TB, but that amount is spread across billions of files, which soon created a massive drag on time and resources. With an infrastructure built almost entirely around AWS and billions of miniscule files to manage, Daltix started to outgrow AWS\u2019 storage options in both scalability and cost efficiency.<\/p>\nThe Daltix team in Belgium.<\/figcaption><\/figure>\nWe got to chat with Charlie Orford, Principal Software Engineer for Daltix, about how Daltix switched to Backblaze B2 Cloud Storage and their takeaways from that process. Here are some highlights:<\/p>\n
\nThey used a custom engine to migrate billions of files from AWS S3 to Backblaze B2.<\/li>\n Monthly costs reduced by $2,500 while increasing data portability and reliability.<\/li>\n Daltix established the infrastructure to automatically back up 8.4 million data objects every day.<\/li>\n<\/ul>\nRead on to learn how they did it.<\/p>\n
A Complex Data Pipeline Built Around AWS<\/h2>\n Most of the S3-based infrastructure Daltix built in the company\u2019s early days is still intact. Historically, the data pipeline started with web-scraped resources written directly to Amazon S3, which were then standardized by Lamba-based extractors before being sent back to S3. Then AWS Batch picked up the resources to be augmented and enriched using other data sources.<\/p>\n
All those steps took place before the data was ready for Daltix\u2019s team of analysts. In order to optimize the pipeline and increase efficiency, Orford started absorbing pieces of that process into Kubernetes. But there was still a data storage problem; Daltix generates about 300GB of compressed data per day, and that figure was growing rapidly. \u201cAs we\u2019d scaled up our data collection, we\u2019d had to sharpen our focus on cost control, data portability, and reliability,\u201d said Orford. \u201cThey\u2019re obvious, but at scale, they\u2019re extremely important.\u201d<\/p>\n
Cost Concerns Inspire The Search For Warm Archival Storage<\/h2>\n By 2020, Daltix had started to realize the limitations of building so much of their infrastructure in AWS. For example, heavy customization around S3 metadata made the ability to move objects entirely dependent on the target system\u2019s compatibility with S3. Orford was also concerned about the costs of permanently storing such a huge data lake in S3. As he puts it, \u201cIt was clear that there was no need to have everything in S3 forever. If we didn\u2019t do anything about it, our S3 costs were going to continue to rise and eventually dwarf virtually all of our other AWS costs.\u201d<\/p>\nSide-by-side comparison of server costs.<\/figcaption><\/figure>\nBecause Daltix works with billions of tiny files, using Glacier was out of the question as its pricing model is based around retrieval fees. Even using Glacier Instant Retrieval, the sheer number of files Daltix works with would have forced them to rack up an additional $200,000 in fees per year. So Daltix\u2019s data collection team\u2014which produces more than 85% of the company\u2019s overall data\u2014pushed for an alternative solution that could address a number of competing concerns:<\/p>\n
\nThe sheer size of the data lake.<\/li>\n The need to store raw resources as discrete files (which means that batching is not an option).<\/li>\n Limitations on the team\u2019s ability to invest time and effort.<\/li>\n A desire for simplicity to guarantee the solution\u2019s reliability.<\/li>\n<\/ul>\nDaltix settled on using Amazon S3 for hot storage and moving warm storage into a new archival solution, which would reduce costs while keeping priority data accessible\u2014even if the intention is to keep files stored away. \u201cIt was important to find something that would be very easy to integrate, have a low development risk, and start meaningfully eating into our costs,\u201d said Orford. \u201cFor us, Backblaze really ticked all the boxes.\u201d<\/p>\n
Initial Migration Unlocks Immediate Savings of $2,000 Per Month<\/h2>\n Before launching into a full migration, Orford and his team tested a proof of concept (POC) to make sure the solution addressed his key priorities:<\/p>\n
\nMaking sure the huge volume of data was migrated successfully.<\/li>\n Avoiding data corruption and checking for errors with audit logs.<\/li>\n Preserving custom metadata on each individual object.<\/li>\n<\/ul>\n\u201cEarly on, Backblaze worked with us hand-in-hand to come up with a custom migration tool that fit all our requirements,\u201d said Orford. \u201cThat\u2019s what gave us the confidence to proceed.\u201d Backblaze delivered a tailor-made engine to ensure that the migration process would transfer the entire data lake reliably and with object-level metadata intact. After the initial POC bucket was migrated successfully, Daltix had everything they needed to start modeling and forecasting future costs. \u201cAs soon as we started interacting with Backblaze, we stopped looking at other options,\u201d Orford said.<\/p>\n
In August 2021, Daltix moved a 120TB bucket of 2.2 billion objects from standard storage in S3 to Backblaze B2 cloud storage. That initial migration alone unlocked an immediate cost savings of $2,000 per month, or $24,000 per year.<\/p>\nA peaceful data lake.<\/figcaption><\/figure>\nQuadruple the Data, Direct S3 Compatibility, and $100,000 Cumulative Savings<\/h2>\n Today, Daltix is migrating about 3.2 million data objects (approximately 160GB of data) from Amazon S3 into Backblaze B2 every day. They keep 18 months of hot data in S3, and as soon as an object reaches 18 months and one day, it becomes eligible for archiving in B2. On the rare occasions that Daltix receives requests for data outside that 18-month window, they can pull data directly from Backblaze B2 into Amazon S3 thanks to Backblaze\u2019s S3-compatible API and ever-available data.<\/p>\n
Daily audit logs summarize how much data has been transferred, and the entire migration process happens automatically every day. \u201cIt runs in the background, there\u2019s nothing to manage, we have full visibility, and it\u2019s cost effective,\u201d Orford said. \u201cBackblaze B2 is an ideal solution for us.\u201d<\/p>\n
As daily data collection increases and more data ages out of the hot storage window, Orford expects more cost reductions. Orford expects it will take about a year and a half for daily migrations to nearly triple their current levels: that means Daltix will be backing up 9 million objects (about 450GB of data) to Backblaze B2 every day. Taking that long-term view, we see incredible cost savings for Daltix by switching from Amazon S3 to Backblaze B2. \u201cBy 2023, we forecast we will have realized a cumulative saving in the region of $75,000-$100,000 on our storage spend thanks to leveraging Backblaze B2, with expected ongoing savings of at least $30,000 per year,\u201d said Orford.<\/p>\n
\u201cIt runs in the background, there\u2019s nothing to manage, we have full visibility, and it\u2019s cost effective. Backblaze B2 is an ideal solution for us.\u201d \u2014Charlie Orford, Principal Software Engineer, Daltix<\/p><\/blockquote>\n
Crunch the Numbers and See for Yourself<\/h2>\n Want to find out what your business could do with an extra $30,000 a year? Check out our Cloud Storage Pricing Calculator<\/a> to see what you could save switching to Backblaze B2.<\/p>\n","protected":false},"excerpt":{"rendered":"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.<\/p>\n","protected":false},"author":152,"featured_media":106912,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":"","jetpack_post_was_ever_published":false},"categories":[7,9],"tags":[468],"class_list":["post-106911","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-storage","category-customer-stories","tag-b2cloud","entry"],"acf":[],"yoast_head":"\n
\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K<\/title>\n<meta name=\"description\" content=\"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.\" \/>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K\" \/>\n<meta property=\"og:description\" content=\"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/\" \/>\n<meta property=\"og:site_name\" content=\"Backblaze Blog | Cloud Storage & Cloud Backup\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/backblaze\" \/>\n<meta property=\"article:published_time\" content=\"2022-10-04T16:20:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-07-18T21:01:52+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"820\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Amrit Singh\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@backblaze\" \/>\n<meta name=\"twitter:site\" content=\"@backblaze\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Amrit Singh\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K","description":"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.","robots":{"index":"noindex","follow":"follow"},"og_locale":"en_US","og_type":"article","og_title":"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K","og_description":"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.","og_url":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/","og_site_name":"Backblaze Blog | Cloud Storage & Cloud Backup","article_publisher":"https:\/\/www.facebook.com\/backblaze","article_published_time":"2022-10-04T16:20:37+00:00","article_modified_time":"2025-07-18T21:01:52+00:00","og_image":[{"width":1440,"height":820,"url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","type":"image\/png"}],"author":"Amrit Singh","twitter_card":"summary_large_image","twitter_creator":"@backblaze","twitter_site":"@backblaze","twitter_misc":{"Written by":"Amrit Singh","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#article","isPartOf":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/"},"author":{"name":"Amrit Singh","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/fda33a5567bc2da33d68c452f790f8da"},"headline":"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K","datePublished":"2022-10-04T16:20:37+00:00","dateModified":"2025-07-18T21:01:52+00:00","mainEntityOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/"},"wordCount":1220,"commentCount":0,"publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","keywords":["B2Cloud"],"articleSection":["Cloud Storage","Customer Stories"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/","url":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/","name":"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K","isPartOf":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#primaryimage"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","datePublished":"2022-10-04T16:20:37+00:00","dateModified":"2025-07-18T21:01:52+00:00","description":"Meet Daltix: A Backblaze customer in the consumer goods space, they are a pioneer in providing complete, transparent, and high-quality retail data. Read more to see how they implemented Backblaze B2 Cloud Storage to easily and efficiently manage their growing data lake.","breadcrumb":{"@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#primaryimage","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","width":1440,"height":820},{"@type":"BreadcrumbList","@id":"https:\/\/www.backblaze.com\/blog\/an-ideal-solution-daltixs-automated-data-lake-archive-saves-100k\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/"},{"@type":"ListItem","position":2,"name":"\u201cAn Ideal Solution\u201d: Daltix\u2019s Automated Data Lake Archive Saves $100K"}]},{"@type":"WebSite","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","name":"Backblaze Cloud Solutions Blog","description":"Cloud Storage & Cloud Backup","publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization","name":"Backblaze","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","width":512,"height":512,"caption":"Backblaze"},"image":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/backblaze","https:\/\/x.com\/backblaze","https:\/\/www.youtube.com\/user\/Backblaze","https:\/\/en.wikipedia.org\/wiki\/Backblaze"]},{"@type":"Person","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/fda33a5567bc2da33d68c452f790f8da","name":"Amrit Singh","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2020\/03\/amrit-1-150x150.jpg","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2020\/03\/amrit-1-150x150.jpg","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2020\/03\/amrit-1-150x150.jpg","caption":"Amrit Singh"},"description":"Amrit Singh is a product marketer at Backblaze but an engineer at heart, helping developers build and grow their applications on the B2 Cloud Storage platform. He's always looking out for new and innovative ways developers utilize cloud storage to build next-gen tech stacks. So if you want to chat about anything cloud or technology, connect with him on Instagram: @akooms or Linkedin: Amrit Singh.","sameAs":["https:\/\/www.instagram.com\/akooms","https:\/\/www.linkedin.com\/in\/amritksingh"],"url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/author\/asingh\/"}]}},"jetpack_featured_media_url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png","_links":{"self":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/106911","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/users\/152"}],"replies":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/comments?post=106911"}],"version-history":[{"count":0,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/106911\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media\/106912"}],"wp:attachment":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media?parent=106911"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/categories?post=106911"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/tags?post=106911"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2022\/10\/bb-bh-Daltix-e1664844038407.png\")
<\/p>\n![\"\"](\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2022\/10\/Daltix21-Team-in-Belgian-office1-scaled.jpg\")
![\"\"](\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2022\/10\/Screen-Shot-2022-10-03-at-10.29.57-PM.png\")
![\"\"](\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2022\/10\/bb-bh-big-data-and-data-lakes.jpg\")