Why are calculations in LLMs so unreliable to begin with?

Most Large Language Models (LLMs) are capable of performing some calculations on their own. While they handle simple arithmetic reasonably well and continue to improve, they frequently make mistakes—and worse, they're “unaware” of them. This means they present incorrect results with the same confidence as correct ones.

To address this limitation, ChatGPT for example includes an integrated Python runtime environment (Code Interpreter). When asked to perform calculations, it often writes Python code to execute them. These calculations are generally reliable, provided the language model correctly understands the user’s instructions. However, the types of calculations commonly found in spreadsheets can be difficult to describe in natural language and surprisingly tricky to replicate with traditional programming.

In many ways, spreadsheets act as a specialized programming language for business calculations, and a vast amount of business data and logic is already encoded in this format. Unfortunately, LLMs cannot directly access or understand spreadsheet models, including their formulas, references, and calculation methods. Worse still, when users upload spreadsheets to LLMs, they attempt to interpret them using code, essentially inventing logic on the fly.

By integrating GRID with LLMs, we combine strengths: LLMs excel at language, while GRID ensures reliable and accurate handling of numbers. Let LLMs handle the words and leave the calculations to us.

I can already upload spreadsheets to Chat GPT. Why do I need GRID?

ChatGPT will handle every XLSX file as a data file. It will ignore any formulas and simply read the values saved in the file. This means ChatGPT can work really well with tabular data stored in spreadsheets - think data that could just as well be saved in a CSV file - but will make dangerous errors with spreadsheet models and formulas.
GRID integrates a true spreadsheet engine with ChatGPT enabling reliable and verifiable spreadsheet calculations in natural language interactions.

What kind of questions can I expect my LLM to answer?

As a rule of thumb, the information you're asking about has to exist in your spreadsheet. For example: If you have a Revenue model that consists of monthly numbers of organic visitors, paid visitors, registrations and new subscriptions, the LLM will not be able to answer questions about churned users because that information is not a part of the spreadsheet. We encourage you to never ask an LLM for calculations that don't exist in your spreadsheet because that will result in hallucinations and wrong answers.

What is a custom GPT?

Custom GPTs are versions of ChatGPT that you can tailor for specific tasks or topics by combining instructions, knowledge and capabilities. They boost productivity in a significant way and are tailored to your business and use-case. Read more about how to create custom GPTs in this guide by OpenAI.

How can I make sure the calculations are performed by GRID, not the LLM?

It’s important that the calculations exist in the spreadsheet and that the LLM is aware of them. That means, if you ask your LLM to calculate a discount, you need to make sure the discount calculations live in your spreadsheet and that their labels are defined in your AI prompt, for example your GPT instructions. If these two things are not in place, the LLM might try to calculate the discount itself, resulting in incorrect calculations.

How can I verify the calculations from GRID?

GRID ensures precise and dependable calculations, and we've made it possible to ask GPTs to explain their calculations. To build trust and transparency, you can enable GRID-powered GPTs to explain their calculations directly. This feature, designed for debugging and internal use, provides step-by-step details of how results are derived. By default, this capability is off, offering control over whether to display calculations, especially for external users.