{"id":110292,"date":"2023-11-09T09:17:29","date_gmt":"2023-11-09T17:17:29","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=110292"},"modified":"2025-12-14T15:50:00","modified_gmt":"2025-12-14T23:50:00","slug":"ai-101-training-vs-inference","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/ai-101-training-vs-inference\/","title":{"rendered":"AI 101: Training vs. Inference"},"content":{"rendered":"\r\n
\"A<\/figure>\r\n\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n

What do Sherlock Holmes and ChatGPT have in common? Inference, my dear Watson!<\/p>\r\n\r\n\r\n\r\n

“We approached the case, you remember, with an absolutely blank mind, which is always an advantage. We had formed no theories. We were simply there to observe and to draw inferences<\/em> from our observations.”
\u2014Sir Arthur Conan Doyle, The Adventures of the Cardboard Box<\/a><\/p>\r\n\r\n\r\n\r\n

As we all continue to refine our thinking around artificial intelligence (AI), it\u2019s useful to define terminology that describes the various stages of building and using AI algorithms\u2014namely, the AI training stage and the AI inference stage. As we see in the quote above, these are not new concepts: they\u2019re based on ideas and methodologies that have been around since before Sherlock Holmes\u2019 time.\u00a0<\/p>\r\n\r\n\r\n\r\n

If you\u2019re using AI, building AI, or just curious about AI, it\u2019s important to understand the difference between these two stages so you understand how data moves through an AI workflow. That\u2019s what I\u2019ll explain today.<\/p>\r\n\r\n\r\n\r\n

The TL:DR<\/h2>\r\n\r\n\r\n\r\n

The difference between these two terms can be summed up fairly simply: first you train<\/strong> an AI algorithm, then your algorithm uses that training to make inferences<\/strong> from data. To create a whimsical analogy, when an algorithm is training, you can think of it like Watson\u2014still learning how to observe and draw conclusions through inference. Once it\u2019s trained, it\u2019s an inferring machine, a.k.a. Sherlock Holmes.\u00a0<\/p>\r\n\r\n\r\n\r\n

Whimsy aside, let\u2019s dig a little deeper into the tech behind AI training and AI inference, the differences between them, and why the distinction is important.\u00a0<\/p>\r\n\r\n\r\n\r\n

Obligatory Neural Network Recap<\/h2>\r\n\r\n\r\n\r\n

Neural networks have emerged as the brainpower behind AI, and a basic understanding of how they work is foundational when it comes to understanding AI.\u00a0\u00a0<\/p>\r\n\r\n\r\n\r\n

Complex decisions, in theory, can be broken down into a series of yeses and nos, which means that they can be encoded in binary. Neural networks have the ability to combine enough of those smaller decisions, weigh how they affect each other, and then use that information to solve complex problems. And, because more complex decisions require more points of information to come to a final decision, they require more processing power. Neural networks are one of the most widely used approaches to AI and machine learning (ML).\u00a0<\/p>\r\n\r\n\r\n

\r\n
<\/figure>\r\n<\/div>\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n

What Is AI Training?: Understanding Hyperparameters and Parameters<\/h2>\r\n\r\n\r\n\r\n

In simple terms, training an AI algorithm is the process through which you take a base algorithm and then teach it how to make the correct decision. This process requires large amounts of data, and can include various degrees of human oversight. How much data you need<\/a> has a relationship to the number of parameters you set for your algorithm as well as the complexity of a problem.\u00a0<\/p>\r\n\r\n\r\n\r\n

We made this handy dandy diagram to show you how data moves through the training process:<\/p>\r\n\r\n\r\n

\r\n
\"A\r\n
As you can see in this diagram, the end result is model data, which then gets saved in your data store for later use.<\/figcaption>\r\n<\/figure>\r\n<\/div>\r\n\r\n\r\n
\u00a0<\/div>\r\n\r\n\r\n\r\n

And hey\u2014we\u2019re leaving out a lot of nuance in that conversation because dataset size, parameter choice, etc. is a graduate-level topic on its own, and usually is considered proprietary information by the companies who are training an AI algorithm. It suffices to say that dataset size and number of parameters are both significant and <\/em>have a relationship to each other, though it\u2019s not a direct cause\/effect relationship. And, both the number of parameters and the size of the dataset affect things like processing resources\u2014but that conversation is outside of scope for this article (not to mention a hot topic in research).\u00a0<\/p>\r\n\r\n\r\n\r\n

As with everything, your use case determines your execution. Some types of tasks actually see excellent results with smaller datasets and more parameters, whereas others require more data and fewer parameters. Bringing it back to the real world, here\u2019s a very cool graph showing how many parameters different AI systems have. Note that they very helpfully identified what type of task each system is designed to solve:<\/p>\r\n\r\n\r\n\r\n