Desert To Data in 7 Days – Our New Phoenix Data Center

By | June 28th, 2017

We are pleased to announce that Backblaze is now storing some of our customers’ data in our newest data center in Phoenix. Our Sacramento facility was slated to store about 500 petabytes of data and was starting to fill up so it was time to expand. After visiting multiple locations in the US and Canada, we selected Phoenix as it had the right combination of power, networking, price and more that we were seeking. Let’s take you through the process of getting the Phoenix data center up and running.

Day 0 – Designing the Data Center

After we selected the Phoenix location as our next DC (data center), we had to negotiate the contract. We’re going to skip that part of the process because, unless you’re a lawyer, it’s a long, boring process. Let’s just say we wanted to be ready to move in once the contract was signed. That meant we had to gather up everything we needed and order a bunch of other things like networking equipment, racks, storage pods, cables, etc. We decided to use our Sacramento DC as the staging point and started gathering what was going to be needed in Phoenix.

In actuality, for some items we started the process several months ago as lead times for things like network switches, Storage Pods, and even hard drives can be measured in months and delays are normal. For example, depending on our move in date, the network providers we wanted would only be able to provide limited bandwidth, so we had to prepare for that possibility. It helps to have a procurement person who knows what they are doing, can work the schedule, and is creatively flexible – thanks Amanda.

So by Day 0, we had amassed multiple pallets of cabinets, network gear, PDUs, tools, hard drives, carts, Guido, and more. And yes, for all you Guido fans he is still with us and he now resides in Phoenix. Everything was wrapped and loaded into a 53-foot semi-truck that was driven the 755 miles (1,215 km) from Sacramento, California to Phoenix, Arizona.

Day 1 – Move In Day

We sent a crew of 5 people to Phoenix with the goal of going from empty space to being ready to accept data in one week. The truck from Sacramento arrived mid-morning and work started unloading and marshaling the pallets and boxes into one area, while the racks were placed near their permanent location on the DC floor.

Day 2 – Building the Racks

Day 2 was spent primarily working with the racks. First they were positioned to their precise location on the data center floor. They were then anchored down and tied together. We started with 2 rows of twenty-two racks each, with twenty being for storage pods and two being for networking equipment. By the end of the week there will be 4 rows of racks installed.

Day 3 – Networking and Power, Part 1

While one team continued to work on the racks, another team began the process a getting the racks connected to the electricty and running the network cables to the network distribution racks. Once that was done, networking gear and rack-based PDUs (Power Distribution Units) were installed in the racks.

Day 4 – Rack Storage Pods

The truck from Sacramento brought 100 Storage Pods, a combination of 45 drive and 60 drive systems. Why did we use 45 drives units here? It has to do with the size (in racks and power) of the initial installation commitment and the ramp (increase) of installations over time. Contract stuff: boring yes, important yes. Basically to optimize our spend we wanted to use as much of the initial space we were allotted as possible. Since we had a number of empty 45 drive chassis available in Sacramento we decided to put them to use.

Day 5 – Drive Day

Our initial set-up goal was to build out five Backblaze Vaults. Each Vault is comprised of twenty Storage Pods. Four of the Vaults were filled with 45 drive Storage Pods and one was filled with 60 drive Storage Pods. That’s 4,800 hard drives to install – thank goodness we don’t use those rubber bands around the drives anymore.

Day 6 – Networking and Power, Part 2

With the storage pods in place, Day 6 was spent routing network and power cables to the individual pods. A critical part of the process is to label every wire so you know where it comes from and where it goes too. Once labeled, wires are bundled together and secured to the racks in a standard pattern. Not only does this make things look neat, it standardizes where you’ll find each cable across the hundreds of racks that are in the DC.

Day 7 – Test, Repair, Test, Ready

With all the power and networking finished, it was time to test the installation. Most of the Storage Pods light up with no problem, but there were a few that failed. These failures are quickly dealt with, and one by one each Backblaze Vault is registered into our monitoring and administration systems. By the end of the day, all five Vaults were ready.

Moving Forward

The Phoenix data center was ready for operation except that the network carriers we wanted to use could only provide a limited amount of bandwidth to start. It would take a few more weeks before the final network lines would be provisioned and operational. Even with the limited bandwidth we kicked off the migration of customer data from Sacramento to Phoenix to help balance out the workload. A few weeks later, once the networking was sorted out, we started accepting external customer data.

We’d like to thank our data center build team for documenting their work in pictures and allowing us to share some of them with our readers.

Questions About Our New Data Center

Now that we have a second DC, you might have a few questions, such as can you store your data there and so on. Here’s the status of things today…

    Q: Does the new DC mean Backblaze has multi-region storage?
    A: Not yet. Right now we consider the Phoenix DC and the Sacramento DC to be in the same region.

    Q: Will you ever provide multi-region support?
    A: Yes, we expect to provide multi-region support in the future, but we don’t have a date for that capability yet.

    Q: Can I pick which data center will store my data?
    A: Not yet. This capability is part of our plans when we provide multi-region support.

    Q: Which data center is my data being stored in?
    A: Chances are that your data is in the Sacramento data center given it currently stores about 90% of our customer’s data.

    Q: Will my data be split across the two data centers?
    A: It is possible that one portion of your data will be stored in the Sacramento DC and another portion of your data will be stored in the Phoenix DC. This will be completely invisible to you and you should see no difference in storage or data retrieval times.

    Q: Can my data be replicated from one DC to the other?
    A: Not today. As noted above, your data will be in one DC or the other. That said files uploaded to the Backblaze Vaults in either DC are stored redundantly across 20 Backblaze Storage Pods within that DC. This translates to 99.999999% durability for the data stored this way.

    Q: Do you plan on opening more data centers?
    A: Yes. We are actively looking for new locations.

If you have any additional questions, please let us know in the comments or on social media. Thanks.

Andy Klein

Andy Klein

Director of Product Marketing at Backblaze
Andy has 20+ years experience in technology marketing. He has shared his expertise in computer security and data backup at the Federal Trade Commission, Rootstech, RSA and over 100 other events. His current passion is to get everyone to back up their data before it's too late.
Andy Klein

Latest posts by Andy Klein (see all)

Category:  Cloud Storage
  • John Schmitt

    You guys really know how to turn a guy ON!!!

  • Pingback: Startup Advantages: How to Win Against Bigger Established Competitors()

  • Adam Gossett

    Not sure if Andy or anyone at backblaze will read this but have you considered Ohio for a DC? If you are looking for Colo I would recommend PCM in New Albany, Ohio. I think they would have exactly what you are looking for. The New Albany DC is only a few years old and they are considering building another one at the same location!

  • hdtvguy

    Is there an option to choose which DC your data goes to? California DCs scare me regardless fo where in CA they are.

  • Pingback: 2017 Hard Drive Reliability By Manufacturer and Model()

  • Brian Essig

    Why are the bottom half of the racks empty? Is that for easier future additions?

  • I think Poland is gaining momentum with all those dozens of thousands of IT specialists working for many off- and nearshore companies. Unfortunately workforce still is very cheap here. So please feel invited ;-)

  • ulaganath KRISHANSAMY

    Do you have plans to open outside US like global presence.

  • Dirk

    When will there be a EU (or better NL) DC? think I would benefit a lot on my upload speeds.

    • Aitor Bleda

      I would also like to have my data in europe, and it is fundamental for EU companies.

      • Seems like market niche. EU-based company may provide a Backblaze-backed “european” backup service – adding extra encryption layer(s) to secure the data. In fact no bit of client data would leave EU – all in all it would be some ‘product’ of initial data and encryption process.
        Any lawyer here to support this concept?

        • Igor Kalders

          I think the whole concept of data protection is actually trying to make sure the data does not leave the eu in any form. The encryption wouldn’t change anything about that.

          Not sure how “niche” this is, since GDPR will basically force the whole of Europe to submit to this.

  • Pingback: Cloud Infrastructure Costs: A Peek Behind the Curtain()

  • bob

    The team looks like a landing party from classic Star Trek, complete with the red shirt guy! Nice writeup, hope you like hot weather.

    • Stefan Seidel

      Oh man, hope the redshirt is ok…

  • Anurag Bhatia

    Can you share which switches do you use in Phoenix datacenter? From one of picture it seems like a cat6 cable connecting the pod. Is that 1GigE or 10GE ?

  • Allen Nicoson

    You named your SeverLift Guido, I love it!
    Great Article. you have the right amount of detail w/o it being too much. Congrats on the new DC. Are you going to be offering tours?

  • Paul Tien 168818 G6PY52600195

    Thanks for the wonderful one-of-a-kind reporting.

  • Colin Stuart

    Great work!

  • Thomas Folkers

    Data Centers: Lots and lots of heat…
    Phoenix: Hottest city in the Southwest…

    Seems legit…

    • Well there’s a hot aisle, and a cold aisle…:D

    • jhersche

      Also has the cheapest electricity costs.

      • Aaron Wallentine

        lol, that’s what I was wondering about … is AZ electricity really much cheaper than elsewhere? Enough to offset the massive amounts of cooling needed? I guess it must be.

        • jhersche

          Yep. Also real estate is pretty affordable as well. Low state taxes too.

      • David Tapley

        Plenty of cheaper rates — including commercial — in other places:

    • Anibal Pereira

      Speaking of heat, hydro power based data center in sweden –
      Wonder why it isnt used more by USA cloud companys to aim to EU client.

  • CMS

    Congrats on the new DC! Excellent post and great pics to go with it… and those pics hosted on B2 to top it off!!

    Perhaps you’ll consider crossing the Atlantic for DC3. Ireland has the right climate for free data center cooling, good US and EU connectivity, strong privacy laws and post-Brexit, we’ll be the only English speaking country left in the EU