The DAX LLM Benchmark v2 shows that the latest frontier models write DAX very well, and that the model generation matters more than the vendor. This post describes how we built the benchmark, how we read its results, and what I think it means for DAX.

The second version of the DAX LLM Benchmark is online at www.daxbench.com. Its results show that the latest generations of frontier models write DAX very well, and they differ significantly from older generations, even those just a few months old. Open-source models still lag behind, but they follow the same trend.

How v2 started

Over the last few months, I worked with Maxim Anatsko to create a second, more advanced version of the DAX LLM Benchmark. Maxim released the first version in December 2025. When I saw the first version, I was impressed. It was the first time I saw a complete, reproducible, relatively broad set of tests to evaluate the ability of an LLM to “understand” and write DAX expressions correctly. I had been thinking for a while about how to implement tests to “measure” an LLM’s ability to write DAX; Maxim’s work seemed very close to what I had in mind.

I quickly contacted Maxim to better understand how it worked and which plans he had moving forward. It did not take long to conclude that working together would benefit both the Power BI community and us. This past summer, we improved the benchmark engine and created tests that covered many more real-world examples of varying complexity.

I do not think that the tests we created are perfect. Like a certification exam, tests can be valid up to a certain point. The usefulness of a model in the real world matters more. A perfect model could fail to handle a specific business requirement, and a below-average model could be more than enough for several users. We understand that, and I want to be the first to write a disclaimer: you should not judge a model just by its benchmarks; your personal experience matters more than any benchmark. However, this new benchmark gives us a lot of information.

How the benchmark works

We have a page describing the methodology used in the benchmark. Every test is a single-shot request with the default reasoning level provided for each given model. The model can reason, but we intentionally disabled the ability for the agent to iterate over solutions and retest them. Moreover, we do not provide any skills to the agents. These decisions make the benchmark far from the real world. However, this is the only way to evaluate the model’s “native” skill. We assume that a better “native” competence level generally corresponds to a better outcome of the final result. In short, we wanted a tool to help us choose a model and assess whether other models surpass our current choice in performance or cost.

Usually, a test requires writing one or more DAX formulas in an existing model where other DAX measures exist. In some cases, we need to fix an existing DAX measure to meet updated requirements. We also test DAX queries, calculated tables, calculated columns, and user-defined functions (UDF). Most of the semantic models are based on variations of the Contoso model we often use in SQLBI content.

We test the DAX code, not the semantic model. We know that a more comprehensive test should also include the data modeling side. We do not completely ignore the data model, because a few tests require understanding a non-trivial semantic model. However, we did not set data modeling as a specific goal, because older LLMs already struggle with relatively complex formulas and scenarios.

Every test runs on a semantic model loaded for that test. The response of the LLM alters the semantic model; the benchmark engine then runs a test query to validate the changes the LLM applied. If the response produces the expected results, we do a deeper analysis to verify best practices and the “distance” from the reference solution. We tried to avoid giving low scores to correct solutions just because the generated code was far from the reference one.

We compare the solution of each LLM run with the reviewed reference solution, and we compare its results with the expected ones. The final score combines evaluation metrics that measure the number of tasks completed successfully, output accuracy, and syntax correctness. However, we do not provide a score to a task that does not complete correctly, even though it is syntactically correct and well-formatted.

What the results show

At the time of writing (October 18, 2026), several models from different vendors share very similar scores. Small differences in the final score are not very important because they come from minor variations that could be irrelevant in daily use. More helpful is how close several LLMs are to a perfect score, which suggests that choosing a specific LLM vendor may matter less than choosing the model generation, regardless of vendor.

Another important outcome is the improvement curve across model generations. Because these tests do not rely on LLM memory or sessions, the only reason we see improvement is that the models have evolved.

Costs differ significantly, and we compute them using API pricing while ignoring subscription savings. For certain applications, the API cost could be minimal, so the choice between models could depend on quality alone; but with higher consumption and over the long run, you probably want to keep costs under control. The Value Analysis compares the models from that perspective.

What this means for DAX

If you had asked me one year ago (October 2025), I would never have thought that the improvement in DAX writing we have seen in one year would be possible. I was using (and still use) LLM tools for many other tasks, but the improvements in DAX did not keep pace with the speed of improvements in coding until then. This changed with the frontier models released starting in April 2026. Six models from three different vendors, released in August and September 2026, score between 90% and 96% on our new benchmark. I almost feel the need to create version 2.1 or 3.0 of the benchmark: every task was solved by at least one LLM. We will probably find where the bar is. However, it is already above the level that helps many users.

That is why, in September 2026, I wrote that using AI to write DAX has started to be effective: I was not confident in a similar statement six months earlier, but now an agent can save time writing DAX.

I expect the question: "If an LLM writes DAX, is it still worth learning DAX?" The short answer: yes, and not only because I love the language and part of my job is teaching it. At least being able to read DAX code gives you an intermediate view of how business rules are implemented. Descriptions of the specification in English (which we call contracts) can still be ambiguous about how a measure is interpreted when it involves unrelated attributes. DAX provides a clear and deterministic explanation of the algorithm. It could be wrong and must be fixed, but different clients will always get the same numbers for the same report. Being able to read the DAX code an LLM generates lets you validate the business rules the development team implemented.

LLMs can also create and automate validation tests, but it is harder to build a set of measures that will last without knowing the master plan in detail. That is why implementing a business rule in DAX also documents the intended behavior better.

Another note: complexity is no longer a limit. For years, I heard people say that DAX complexity limited its adoption. That limit is gone, and, in general, language complexity limits are gone. DAX was created because MDX was too hard, and it is simpler at the cost of a few MDX features, such as recursion and hierarchies. When an LLM writes the code, a language can be more complex if it is more powerful, because people no longer write every expression by hand. Compatibility with legacy reports and models remains a constraint, but migration costs could be lowered. This makes so many evolutions possible that it is extremely hard to predict what could happen in two to three years.

Conclusions

The latest generation of frontier models writes DAX well enough to save time, and model generation matters more than the vendor. At the time of writing, six models from three vendors score between 90% and 96% on the DAX LLM Benchmark v2.

One final recommendation: use the benchmark to choose a model generation and to keep costs under control, but validate the results with your own tests on your own semantic models.

Enjoy the ride!