{"id":111976,"date":"2025-03-04T09:22:33","date_gmt":"2025-03-04T17:22:33","guid":{"rendered":"https:\/\/www.backblaze.com\/blog\/?p=111976"},"modified":"2025-03-04T09:51:07","modified_gmt":"2025-03-04T17:51:07","slug":"experimenting-with-deepseek-backblaze-b2-and-drive-stats","status":"publish","type":"post","link":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/","title":{"rendered":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats"},"content":{"rendered":"\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1440\" height=\"820\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png\" alt=\"A decorative image showing buildings of many sizes. \" class=\"wp-image-111978\"\/><\/figure>\n\n\n\n<div style=\"height:15px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">As we explained in our recent blog post, <a href=\"https:\/\/www.backblaze.com\/blog\/ai-reasoning-models-openai-o3-mini-o1-mini-and-deepseek-r1\/\" target=\"_blank\" rel=\"noreferrer noopener\">AI Reasoning Models: OpenAI o3-mini, o1-mini, and DeepSeek R1<\/a>, Chinese startup <a href=\"https:\/\/www.deepseek.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">DeepSeek<\/a> caused a stir when it released its R1 reasoning model in January of this year. Interestingly, DeepSeek R1 has an OpenAI-compatible API, so applications written for OpenAI <em>should<\/em> work with DeepSeek R1 with just a configuration change. Since I had a suitable sample app all ready to go, I decided to put their claim to the test.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why, and why not, use DeepSeek?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A major difference between DeepSeek and OpenAI is cost. At the time of writing, DeepSeek charges $0.55 per million input tokens and $2.19 per million output tokens for its R1 model. That\u2019s about 3.6% of OpenAI\u2019s $15.00 per million input tokens and $60.00 per million output tokens for its flagship <code>o1<\/code> reasoning model, and about half of o3-mini\u2019s $1.10 per million input tokens and $4.40 per million output tokens.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Set against this is the fact that, in using the DeepSeek platform\u2019s API, you are sending your data to a startup located in China that has been <a href=\"https:\/\/www.axios.com\/2025\/01\/29\/openai-deepseek-ai-models-data-training\" target=\"_blank\" rel=\"noreferrer noopener\">accused by OpenAI<\/a> of &#8220;inappropriately&#8221; basing its work on the output of OpenAI&#8217;s models. It\u2019s up to you, and your organizations\u2019 data governance policy, whether the trade-off is worthwhile.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another consideration is the ability to run DeepSeek\u2019s models locally, on your own infrastructure, or, more likely, your chosen provider\u2019s infrastructure, rather than sending requests to the DeepSeek platform. Spinning up my own DeepSeek instance was out of scope for this blog post, but I\u2019ll likely return to it in a future blog post.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Swapping OpenAI for DeepSeek<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Last month, I explained how you can <a href=\"https:\/\/www.backblaze.com\/blog\/building-an-ai-agent-with-backblaze-b2-langchain-and-drive-stats\/\" target=\"_blank\" rel=\"noreferrer noopener\">build an AI agent with Backblaze B2, LangChain, and Drive Stats<\/a>, walking you through a simple chatbot that can answer questions based on our <a href=\"https:\/\/www.backblaze.com\/drivestats\" target=\"_blank\" rel=\"noreferrer noopener\">Drive Stats<\/a> data set\u201411 years of metrics gathered from the Backblaze B2 Cloud Storage platform\u2019s fleet of hard drives. In that example, the chatbot accepted a natural language question, used <a href=\"https:\/\/openai.com\/index\/gpt-4o-mini-advancing-cost-efficient-intelligence\/\" target=\"_blank\" rel=\"noreferrer noopener\">OpenAI\u2019s GPT\u20114o mini<\/a> large language model (LLM) to generate a SQL query that might help provide an answer, executed the query against the Drive Stats data set via the Trino SQL engine, and then used OpenAI again to interpret the result set and either repeat the query-interpret cycle, or generate a natural language answer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I copied <a href=\"https:\/\/github.com\/backblaze-b2-samples\/ai-agent-demo\/blob\/main\/agent_demo.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">the Jupyter notebook from that example<\/a> and used it as the basis for investigating the feasibility of swapping out OpenAI for DeepSeek. The <a href=\"https:\/\/github.com\/backblaze-b2-samples\/ai-agent-demo\/blob\/main\/agent_demo_deepseek.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">DeepSeek version of the notebook<\/a> contains the full source code of my experiments; I\u2019ll include relevant extracts here, edited for clarity.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Since I used the LangChain AI framework, which provides a layer above a range of AI models, the only place that OpenAI surfaced in my code was in creating an instance of LangChain\u2019s ChatOpenAI wrapper:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># OPENAI_API_KEY must be defined in the .env file<br \/>load_dotenv()<br \/>llm = ChatOpenAI(model=\"gpt-4o-mini\")<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The ChatOpenAI class contains all the code required to communicate with OpenAI via its API.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">According to the <a href=\"https:\/\/api-docs.deepseek.com\/\">DeepSeek documentation<\/a>, all you should need to do is:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Provide your DeepSeek API key in the same <code>OPENAI_API_KEY<\/code> environment variable.<\/li>\n\n\n\n<li>Set the API base URL to <code>https:\/\/api.deepseek.com<\/code>.<\/li>\n\n\n\n<li>Provide a DeepSeek model name in place of the OpenAI one.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If this reminds you of the steps for using <a href=\"https:\/\/www.backblaze.com\/docs\/en\/cloud-storage-use-python-to-create-an-application#configure-aws-(python)\" target=\"_blank\" rel=\"noreferrer noopener\">Backblaze B2\u2019s S3-compatible API<\/a>, you\u2019re not alone. The OpenAI API has become a de facto standard for integrating with LLMs in much the same way as Amazon\u2019s S3 API allows an ecosystem of apps and tools to interoperate with object storage systems from a variety of vendors.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Looking at the <a href=\"https:\/\/api-docs.deepseek.com\/quick_start\/pricing\" target=\"_blank\" rel=\"noreferrer noopener\">DeepSeek documentation<\/a>, you can use one of two models, <code>deepseek-reasoner<\/code> (aka DeepSeek R1) or <code>deepseek-chat<\/code>. Let\u2019s see what the much-talked-about DeepSeek R1 came up with.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Using DeepSeek R1 in the AI agent<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To make it easy to use both the OpenAI and DeepSeek notebooks, I created a second entry in the .env file for the DeepSeek API key, and copied it to the OpenAI environment variable in the notebook code:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\"># The .env file needs at least DEEPSEEK_API_KEY, and may also contain<br \/># OPENAI_API_KEY. Move the DeepSeek API key to the OpenAI environment<br \/># variable<br \/>load_dotenv()<br \/><br \/>os.environ[\"OPENAI_API_KEY\"] = os.environ.pop(\"DEEPSEEK_API_KEY\")<br \/><br \/>llm = ChatOpenAI(model=\"deepseek-reasoner\", base_url='https:\/\/api.deepseek.com')<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">As I set about repeating the steps from <a href=\"https:\/\/github.com\/backblaze-b2-samples\/ai-agent-demo\/blob\/main\/agent_demo.ipynb\" target=\"_blank\" rel=\"noreferrer noopener\">the Jupyter notebook that supported my previous blog post<\/a>, I was disappointed to see DeepSeek fall at the very first hurdle: generating a SQL query for a simple natural language question. Here is the code:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">question = {\"question\": \"How many drives are there?\"}<br \/><br \/>write_query(question)<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Looking back at the original notebook, OpenAI\u2019s response was valid SQL, although it didn\u2019t have enough information to construct the correct query:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">{'query': 'SELECT COUNT(*) AS drive_count FROM drivestats'}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">DeepSeek, on the other hand, responded with a Python stack trace and this error:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">openai.UnprocessableEntityError: Failed to deserialize the JSON body into the target type: response_format: response_format.type `json_schema` is unavailable now at line 1 column 13827<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">What went wrong? Searching for the error turns up a <a href=\"https:\/\/github.com\/langchain-ai\/langchain\/issues\/29282#issuecomment-2599971695\" target=\"_blank\" rel=\"noreferrer noopener\">comment from a LangChain engineer<\/a> explaining that we should use BaseChatOpenAI rather than ChatOpenAI since it &#8220;[&#8230;] accommodates many APIs that are similar to OpenAI. It uses tool calling for structured output by default.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, we can redefine llm accordingly, and try generating a query again:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">llm = BaseChatOpenAI(model=\"deepseek-reasoner\", base_url='https:\/\/api.deepseek.com')<br \/><br \/>write_query(question)<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Unfortunately, DeepSeek returns another error:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">BadRequestError: Error code: 400 - {'error': {'message': 'The last message of deepseek-reasoner must be a user message, or an assistant message with prefix mode on (refer to https:\/\/api-docs.deepseek.com\/guides\/chat_prefix_completion).', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Looking back at the AI agent code, we can see that we used an off-the-shelf prompt from the LangChain Prompt Hub that provides the model with a single, system, message:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">================================ System Message ================================<br \/><br \/>Given an input question, create a syntactically correct {dialect} query to run to help find the answer. Unless the user specifies in his question a specific number of examples they wish to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.<br \/><br \/>Never query for all the columns from a specific table, only ask for a few relevant columns given the question.<br \/><br \/>Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.<br \/><br \/>Only use the following tables:<br \/>{table_info}<br \/><br \/>Question: {input}<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Does this mean that DeepSeek is not, in fact, API-compatible with OpenAI? I would argue that it does not. DeepSeek implements the same API request\/response syntax as OpenAI, but it is a different platform. Some variation in semantics is to be expected. We see similar variations between Backblaze B2 and Amazon S3; for example, the S3 <a href=\"https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/API\/API_PutObjectAcl.html\" target=\"_blank\" rel=\"noreferrer noopener\">PutObjectAcl<\/a> operation sets the access control list (ACL) for an object in a bucket. Amazon S3\u2019s access management model allows you to manipulate an object\u2019s ACL independently of its bucket\u2014for example, you can put a private object in a public bucket, and vice versa.<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">This flexibility comes with a cost: It becomes difficult to reason about the visibility of data. In fact, AWS <a href=\"https:\/\/docs.aws.amazon.com\/AmazonS3\/latest\/userguide\/acl-overview.html#:~:text=We%20recommend%20that%20you%20keep%20ACLs%20disabled%2C%20except%20in%20unusual%20circumstances%20where%20you%20need%20to%20control%20access%20for%20each%20object%20individually.\" target=\"_blank\" rel=\"noreferrer noopener\">now recommends<\/a> \u201cthat you keep ACLs disabled, except in unusual circumstances where you need to control access for each object individually.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Backblaze B2\u2019s model is much simpler: You control access at the bucket level, and all objects have the same ACL as their bucket. Backblaze B2 implements the <code>PutObjectAcl<\/code> operation, but, if you try to set an object\u2019s ACL to any other value than its bucket\u2019s ACL, the service responds with an error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Returning to the AI agent code, we can replace the single-system-message prompt with one that combines a system message with a user message:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">import textwrap<br \/>from langchain_core.prompts import ChatPromptTemplate<br \/><br \/>query_prompt_template = ChatPromptTemplate([<br \/>    (\"system\", textwrap.dedent(\"\"\"Given an input question, create a<br \/>    syntactically correct {dialect} query to run to help find the answer.<br \/>    Unless the user specifies in his question a specific number of examples<br \/>    they wish to obtain, always limit your query to at most {top_k} results.<br \/>    You can order the results by a relevant column to return the most<br \/>    interesting examples in the database.<br \/><br \/>    Never query for all the columns from a specific table, only ask for a the<br \/>    few relevant columns given the question.<br \/><br \/>    Pay attention to use only the column names that you can see in the schema<br \/>    description. Be careful to not query for columns that do not exist. Also,<br \/>    pay attention to which column is in which table.<br \/><br \/>    Only use the following tables:<br \/>    {table_info}\"\"\")),<br \/>    (\"human\", \"Question: {input}\"),<br \/>])<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Trying the write_query() call for a third time, this is the response:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">BadRequestError: Error code: 400 - {'error': {'message': 'deepseek-reasoner does not support Function Calling', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">A third error! What is this \u201cfunction calling\u201d that deepseek-reasoner does not support? <a href=\"https:\/\/huggingface.co\/docs\/hugs\/en\/guides\/function-calling\" target=\"_blank\" rel=\"noreferrer noopener\">A helpful article on the topic<\/a> at the <a href=\"https:\/\/huggingface.co\/\" target=\"_blank\" rel=\"noreferrer noopener\">Hugging Face<\/a> AI community explains:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">Function calling is a powerful capability that enables Large Language Models (LLMs) to interact with your code and external systems in a structured way. Instead of just generating text responses, LLMs can understand when to call specific functions and provide the necessary parameters to execute real-world actions.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unfortunately, that is exactly our use case. It\u2019s becoming clear that DeepSeek R1 is not the correct tool for implementing an AI agent\u2014we\u2019ve been trying to use a chisel as a screwdriver!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">DeepSeek-V3: A better fit<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">As its name suggests, the <code>deepseek-chat<\/code> model is more appropriate for this application. The DeepSeek documentation tells us that it is based on <a href=\"https:\/\/api-docs.deepseek.com\/news\/news1226\" target=\"_blank\" rel=\"noreferrer noopener\">DeepSeek-V3, released in December 2024<\/a>. DeepSeek-V3 is priced at $0.27 per million input tokens and $1.10 per million output tokens; this is actually more expensive than the GPT-4o mini model I used for the OpenAI agent example ($0.15 per million input tokens, $0.600 per million output tokens), but how does it compare? Let\u2019s take a look.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">First, we need to edit the LLM creation code again to set the model name:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">llm = BaseChatOpenAI(model=\"deepseek-chat\", base_url='https:\/\/api.deepseek.com')<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Now we can run write_query() again. It\u2019s immediately clear that it\u2019s a better fit than its \u201cbig brother:\u201d<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">{'query': 'SELECT COUNT(*) AS total_drives FROM drivestats LIMIT 10'}<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">As with the OpenAI agent, this query is well-formed SQL, but it\u2019s not answering the question we set\u2014it\u2019s giving us the total number of rows in the dataset, rather than the number of drives. Also, it\u2019s a little odd to have a <code>LIMIT<\/code> clause in a <code>SELECT COUNT(*)<\/code> query, but it\u2019s legal SQL, and the agent is following its instructions very literally: <code>always limit your query to at most {top_k} results<\/code>, where we set <code>top_k<\/code> to <code>10<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">question = {\"question\": \"Each drive has its own serial number. How many drives are there?\"}<br \/><br \/>query = write_query(question)<br \/><br \/>{'query': 'SELECT COUNT(DISTINCT serial_number) AS total_drives FROM drivestats'}<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">So far, so good!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I\u2019ll skip some intermediate steps here\u2014they are all in the Jupyter notebook if you want to review them, or run them for yourself\u2014and look at how a simple LangChain graph, built on the DeepSeek LLM, answered the question: \u201cEach drive has its own serial number. How many drives did each data center have on 9\/1\/2024?\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The OpenAI version generated an invalid query, comparing the date column with the string \u2019<code>2024-09-01<\/code>\u2019 without using the required <code>DATE<\/code> type identifier, but DeepSeek generates a correct SQL query and provides a useful natural language response:<\/p>\n\n\n\n<div class=\"wp-block-group has-purple-background-color has-background\"><div class=\"wp-block-group__inner-container is-layout-constrained wp-block-group-is-layout-constrained\">\n<p class=\"wp-block-paragraph\">\/SELECT datacenter, COUNT(DISTINCT serial_number) AS drive_count FROM drivestats WHERE date = DATE &#8216;2024-09-01&#8217; GROUP BY datacenter ORDER BY drive_count DESC LIMIT 10<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">[(&#8216;phx1&#8217;, 89477), (&#8216;sac0&#8217;, 78444), (&#8216;sac2&#8217;, 60775), (&#8221;, 24080), (&#8216;iad1&#8217;, 22800), (&#8216;ams5&#8217;, 16139)]<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p class=\"wp-block-paragraph\">On September 1, 2024, the data centers had the following number of drives:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>phx1<\/strong>: 89,477 drives<\/li>\n\n\n\n<li><strong>sac0<\/strong>: 78,444 drives<\/li>\n\n\n\n<li><strong>sac2<\/strong>: 60,775 drives<\/li>\n\n\n\n<li><strong>(empty datacenter)<\/strong>: 24,080 drives<\/li>\n\n\n\n<li><strong>iad1<\/strong>: 22,800 drives<\/li>\n\n\n\n<li><strong>ams5<\/strong>: 16,139 drives<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">These are the top data centers with the highest drive counts on that date.<\/p>\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">DeepSeek scores a point!<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Moving on to the ReAct AI Agent, which allows the LLM to perform multiple SQL queries in generating an answer to a question, DeepSeek performs similarly to OpenAI. Given the question, \u201cEach drive has its own serial number. What is the annualized failure rate of the ST4000DM000 drive model?\u201d, the DeepSeek agent provides the <em>overall<\/em> failure rate rather than the <em>annualized<\/em> failure rate (AFR).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When we provide explicit instructions for calculating AFR in its prompt, the DeepSeek agent provides the correct result, identical, in fact, to the OpenAI agent\u2019s response:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">The annual failure rate (AFR) for the ST4000DM000 drive model is approximately <strong>2.63%<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">However, when given the question, \u201cWhat was the annual failure rate of the ST8000NM000A drive model in Q3 2024?\u201d, the DeepSeek agent gives us:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">[(1.6100573445081607,)]<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">While OpenAI responds:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">The annual failure rate (AFR) of the ST8000NM000A drive model in Q3 2024 is approximately 1.61%.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Wrapping up the investigation, the final question from the OpenAI notebook is more complex:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">Considering only drive models which had at least 100 drives in service at the end of the quarter and which accumulated 10,000 or more drive days during the quarter, which drive had the most failures in Q3 2024, and what was its failure rate?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Impressively, the OpenAI agent constructed a well-formed SQL query and provided the correct response:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">The drive model with the most failures in Q3 2024 is the <strong>TOSHIBA MG08ACA16TA<\/strong>, which had <strong>181 failures<\/strong>. Its failure rate during this period was approximately <strong>1.84%<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Unfortunately, giving the same question to the DeepSeek AI agent resulted in an error message:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">BadRequestError: Error code: 400 - {'error': {'message': \"An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. (insufficient tool messages following tool_calls message)\", 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_request_error'}}<br \/>During task with name 'agent' and id '0aa26ba6-a3ee-ced1-de4d-b60ed7fbca99'<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The phrase \u201cinsufficient tool messages\u201d suggested that the DeepSeek LLM might need to be reconfigured to allow more tokens. According to <a href=\"https:\/\/api-docs.deepseek.com\/quick_start\/pricing\" target=\"_blank\" rel=\"noreferrer noopener\">the documentation on models and pricing<\/a>, the <code>deepseek-chat<\/code> model supports a maximum of 8K output tokens, but defaults to 4K if <code>max_tokens<\/code> is not specified.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Recreating the DeepSeek wrapper object and agent accordingly, I gave it the last question again:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">llm = BaseChatOpenAI(model=\"deepseek-chat\", base_url='https:\/\/api.deepseek.com', max_tokens=8192, **extra_kwargs)<br \/><br \/>agent_executor = create_react_agent(llm, tools, state_modifier=system_message)<br \/><br \/>response = agent_executor.invoke(<br \/>    {\"messages\": [{\"role\": \"user\", \"content\": \"Considering only drive models which had at least 100 drives in service at the end of the quarter and which accumulated 10,000 or more drive days during the quarter, which drive had the most failures in Q3 2024, and what was its failure rate?\"}]}<br \/>)<br \/><br \/># Show the SQL query sent to the database<br \/>print(response['messages'][-3].tool_calls[0]['args']['query'])<br \/><br \/># Show the final response message<br \/>display_markdown(response['messages'][-1].content, raw=True)<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This time, DeepSeek was able to generate a similar SQL query to OpenAI:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">WITH drive_counts AS (<br \/>    SELECT model, COUNT(DISTINCT serial_number) AS drive_count<br \/>    FROM drivestats<br \/>    WHERE date &gt;= DATE '2024-07-01' AND date &lt;= DATE '2024-09-30'<br \/>    GROUP BY model<br \/>    HAVING COUNT(DISTINCT serial_number) &gt;= 100<br \/>), drive_days AS (<br \/>    SELECT model, COUNT(*) AS total_drive_days<br \/>    FROM drivestats<br \/>    WHERE date &gt;= DATE '2024-07-01' AND date &lt;= DATE '2024-09-30'<br \/>    GROUP BY model<br \/>    HAVING COUNT(*) &gt;= 10000<br \/>), failures AS (<br \/>    SELECT model, COUNT(*) AS failure_count<br \/>    FROM drivestats<br \/>    WHERE date &gt;= DATE '2024-07-01' AND date &lt;= DATE '2024-09-30' AND failure = 1<br \/>    GROUP BY model<br \/>)<br \/>SELECT d.model,<br \/>       f.failure_count,<br \/>       100 * (CAST(f.failure_count AS DOUBLE) \/ (CAST(d.total_drive_days AS DOUBLE) \/ 365)) AS annual_failure_rate<br \/>FROM drive_days d<br \/>JOIN failures f ON d.model = f.model<br \/>JOIN drive_counts dc ON d.model = dc.model<br \/>ORDER BY f.failure_count DESC<br \/>LIMIT 1<br \/><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">With a correct response:<\/p>\n\n\n\n<p class=\"has-purple-background-color has-background wp-block-paragraph\">To answer the question:<br \/><br \/>The drive model with the most failures in Q3 2024 is <strong>TOSHIBA MG08ACA16TA<\/strong>, which had <strong>181 failures<\/strong>. The annualized failure rate (AFR) for this model during that quarter was <strong>1.84%<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Success! But, unfortunately, this isn\u2019t the whole story.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">DeepSeek Reliability<\/h2>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"936\" height=\"888\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-error-1.png\" alt=\"A screenshot of a DeepSeek error message. \" class=\"wp-image-111979\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-error-1.png 936w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-error-1-300x285.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-error-1-768x729.png 768w\" sizes=\"auto, (max-width: 936px) 100vw, 936px\" \/><\/figure>\n<\/div>\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">I originally set out to write this blog post at the end of January, but the <a href=\"https:\/\/platform.deepseek.com\/sign_in\" target=\"_blank\" rel=\"noreferrer noopener\">DeepSeek platform website<\/a> had gone offline by January 30, so I couldn\u2019t even start until I was able to sign up for an API key on February 5.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"936\" height=\"584\" src=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-availability.png\" alt=\"A screenshot of DeepSeek availability from December 2024 to Feburary 2025. \" class=\"wp-image-111980\" srcset=\"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-availability.png 936w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-availability-300x187.png 300w, https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/Backblaze_Deepseek-availability-768x479.png 768w\" sizes=\"auto, (max-width: 936px) 100vw, 936px\" \/><\/figure>\n<\/div>\n\n\n<div style=\"height:10px\" aria-hidden=\"true\" class=\"wp-block-spacer\"><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Given my shiny new API key, and DeepSeek\u2019s claims of OpenAI API compatibility, I na\u00efvely expected to be able to work through my earlier OpenAI notebook and write up the results in a couple of days. The reality was more like two <em>weeks<\/em>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In this blog post I\u2019ve detailed some of the error messages I encountered along the way, but I saw <em>many<\/em> more that pointed to the DeepSeek API simply being overwhelmed with traffic. For example, for over a day, when the status page reported no issues, most API requests to DeepSeek terminated after a minute with the error message:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">A time-consuming investigation revealed that this was caused by the DeepSeek API returning the 200 status code and headers as if the request was successful, then hanging for a minute before terminating the connection without returning any actual data. The calling code saw the 200 as success and tried to decode the non-existent API response body, resulting in the error.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I saw several more instances of intermittent errors that all seemed to point in the same direction: DeepSeek needs to add capacity to its API platform. Notably, the platform seemed faster and more stable on a Saturday morning, U.S. Pacific time, the early hours of Sunday morning in China.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Final thoughts<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">At present, I would have to classify the DeepSeek-V3 API as \u201cpromising, but somewhat flaky.\u201d An agent invocation that succeeds one minute could fail the next with any of a range of error messages. That\u2019s a shame, since when it does work, for instance, in creating the SQL query for the final question above, it tends to work very well.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One final caveat: This is a dynamic field; frameworks and services are literally being updated on a daily basis. For example, since yesterday, as I write this, <em>four<\/em> of the notebook\u2019s module dependencies have been updated. I encourage you to experiment for yourself as your mileage will almost certainly vary, hopefully in a positive direction.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Read about how to build a chatbot using Backblaze, DeepSeek, and the Drive Stats dataset. <\/p>\n","protected":false},"author":174,"featured_media":111978,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","footnotes":"","jetpack_post_was_ever_published":false},"categories":[7,434,438,483],"tags":[489,468],"class_list":["post-111976","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-storage","category-featured-1","category-featured-cloud-storage","category-tech-lab","tag-ai-ml","tag-b2cloud","entry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.1 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Experimenting with DeepSeek, Backblaze B2, and Drive Stats<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Experimenting with DeepSeek, Backblaze B2, and Drive Stats\" \/>\n<meta property=\"og:description\" content=\"Read about how to build a chatbot using Backblaze, DeepSeek, and the Drive Stats dataset.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/\" \/>\n<meta property=\"og:site_name\" content=\"Backblaze Blog | Cloud Storage &amp; Cloud Backup\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/backblaze\" \/>\n<meta property=\"article:published_time\" content=\"2025-03-04T17:22:33+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2025-03-04T17:51:07+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1440\" \/>\n\t<meta property=\"og:image:height\" content=\"820\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Pat Patterson\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@backblaze\" \/>\n<meta name=\"twitter:site\" content=\"@backblaze\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Pat Patterson\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/","og_locale":"en_US","og_type":"article","og_title":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats","og_description":"Read about how to build a chatbot using Backblaze, DeepSeek, and the Drive Stats dataset.","og_url":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/","og_site_name":"Backblaze Blog | Cloud Storage &amp; Cloud Backup","article_publisher":"https:\/\/www.facebook.com\/backblaze","article_published_time":"2025-03-04T17:22:33+00:00","article_modified_time":"2025-03-04T17:51:07+00:00","og_image":[{"width":1440,"height":820,"url":"https:\/\/www.backblaze.com\/blog\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","type":"image\/png"}],"author":"Pat Patterson","twitter_card":"summary_large_image","twitter_creator":"@backblaze","twitter_site":"@backblaze","twitter_misc":{"Written by":"Pat Patterson","Est. reading time":"16 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#article","isPartOf":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/"},"author":{"name":"Pat Patterson","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/a724a8aee97b6451107442747cd101a4"},"headline":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats","datePublished":"2025-03-04T17:22:33+00:00","dateModified":"2025-03-04T17:51:07+00:00","mainEntityOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/"},"wordCount":2281,"commentCount":0,"publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","keywords":["AI\/ML","B2Cloud"],"articleSection":["Cloud Storage","Featured","Featured-Cloud Storage","Tech Lab"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/","url":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/","name":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats","isPartOf":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#primaryimage"},"image":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#primaryimage"},"thumbnailUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","datePublished":"2025-03-04T17:22:33+00:00","dateModified":"2025-03-04T17:51:07+00:00","breadcrumb":{"@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#primaryimage","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","width":1440,"height":820,"caption":"A decorative image showing buildings of many sizes."},{"@type":"BreadcrumbList","@id":"https:\/\/www.backblaze.com\/blog\/experimenting-with-deepseek-backblaze-b2-and-drive-stats\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Experimenting with DeepSeek, Backblaze B2, and Drive Stats"}]},{"@type":"WebSite","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#website","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","name":"Backblaze Cloud Solutions Blog","description":"Cloud Storage &amp; Cloud Backup","publisher":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#organization","name":"Backblaze","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.backblaze.com\/blog\/wp-content\/uploads\/2017\/12\/backblaze_icon_transparent.png?fit=512%2C512&ssl=1","width":512,"height":512,"caption":"Backblaze"},"image":{"@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/backblaze","https:\/\/x.com\/backblaze","https:\/\/www.youtube.com\/user\/Backblaze","https:\/\/en.wikipedia.org\/wiki\/Backblaze"]},{"@type":"Person","@id":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/#\/schema\/person\/a724a8aee97b6451107442747cd101a4","name":"Pat Patterson","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/01\/PatPatterson1920px-150x150.png","url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/01\/PatPatterson1920px-150x150.png","contentUrl":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2022\/01\/PatPatterson1920px-150x150.png","caption":"Pat Patterson"},"description":"Pat Patterson is the former chief technical evangelist at Backblaze. Over his three decades in the industry, Pat has built software and communities at Sun Microsystems, Salesforce, StreamSets, and Citrix. In his role at Backblaze, he creates and delivers content tailored to the needs of the hands-on technical professional, acts as the \u201cvoice of the developer\u201d on the Product team, and actively participates in the wider technical community. Outside the office, Pat runs far, having completed ultramarathons up to the 50 mile distance. Catch up with Pat via Bluesky or LinkedIn.","url":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/author\/pat\/"}]}},"jetpack_featured_media_url":"https:\/\/backblazeprod.wpenginepowered.com\/wp-content\/uploads\/2025\/03\/bb-bh-Cost-of-Cloud-Storage_Final-1-e1741106460615.png","_links":{"self":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/111976","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/users\/174"}],"replies":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/comments?post=111976"}],"version-history":[{"count":0,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/posts\/111976\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media\/111978"}],"wp:attachment":[{"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/media?parent=111976"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/categories?post=111976"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/backblazeprod.wpenginepowered.com\/blog\/wp-json\/wp\/v2\/tags?post=111976"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}