Anybody using AI Assistant, and if so, what model(s)?

[dbcandle]

I'm a sporadic user of Tabular Editor 3, but wanted to give it another try. I thought the AI Assistant feature was very interesting, as an alternative to feeding .bim and DAX Script into a frontier AI model and asking questions.

Are many people really using AI Assistant though? There are only a few posts in the "AI Assistant" section on Community, and the doc page looks like a "work in progress".

If anyone is using AI Assistant, how rigorous are the requirements on a model? With "vibe coding" before the vendors all make usage-limits a major limitation a couple months back, I used Opus, Sonnet and GPT. To save costs though, I'd like to use something cheaper like a local model, or maybe Deepseek. Does AI Assistant require that the model used has full knowledge of DAX, M, and Tabular Editor 3? Or is it only needed as a conversational "glue", with AI Assistant having some sort of built-in RAG, vector-database, context hints/prompts, etc?

[dbcandle]

Note: I see this from the doc page in the announcement, but don't know if the model cited is really adequate. I have 12Gb VRAM, so I can't run the best models, but I do wonder if I can get-away with Ollama rather than needing to burn API $$$.

Example: Ollama
Download and install Ollama
Pull a model, for example: ollama pull llama3.1
Start the Ollama server (it runs automatically after installation, by default on port 11434)
In Tabular Editor, go to Tools > Preferences > AI Assistant > AI Provider
Set Choose provider to Custom (OpenAI-compatible)
Set Service Endpoint to http://localhost:11434/v1
Set Model name to the model you pulled (e.g. llama3.1)
The API Key field can be set to any non-empty value (e.g. ollama) — Ollama does not require authentication, but the field cannot be left blank

[otykier]

@dbcandle thanks for raising this. One obvious hurdle for a lot of folks with regard to the AI Assistant is indeed the "bring your own API-key", and as you correctly point out, most frontier models no longer offer subsidized pricing for API access, which can make this approach very expensive.

You can use the AI Assistant with local models, but our internal testing found that smaller models (1B, 4B) are essentially useless with it. 7B models have some usefulness for certain tasks, but you probably need at least 12B to achieve consistently good results. @eugman to keep me honest.

Regarding how the AI Assistant "harness" works internally, we have a collection of system prompts that are progressively disclosed to the LLM via tool calls. These prompts explain the various features of TE3 and the things the LLM can do. They also contain basic DAX and C# script examples.

Moreover, we include information about newer semantic model features (like UDFs, which postdate the knowledge cutoff of most frontier LLMs). In addition, we have a tool that lets the LLM search through an offline knowledgebase corpus, which is compiled from all of our blogs, docs, and GitHub comments from TE3 team members. This corpus ships with TE3 as a small SQLite database with a BM25 (full text) search index, and it works surprisingly well when the LLM needs to know more about a given topic. No RAG or vector databases.

For C# scripts specifically, there's also a scripting API lookup feature (i.e., so the LLM can learn about classes and their properties, method signatures, etc.) as and when needed. Also, we try to "close the loop" whenever we can. For example, when the LLM generates a C# script or DAX query, we'll run the generated code through our internal parser. If we detect any syntax or semantic errors, we feed those back to the LLM, giving it a chance to improve the generated code.

The last thing I'll say is that the AI Assistant is still a preview feature, so nothing is set in stone, and we're continuously looking for customer feedback and opportunities to improve the feature. So by all means, if you have any suggestions or if you'd like to share your experience with the AI Assistant on a local model, don't hesitate to reach out.

[dbcandle]

Thanks! Will see if I can try with 12B model locally, and Deepseek.

[dbcandle]

OK, configured for my working Ollama (model: gemma4:12b), but when I ask AI Assistant to describe my data model, or I just say hello, nothing happens. I've done some AI programming (Python, JavaScript) using agentic AI tools, and have used Google Gemini and Ollama quite a bit in the past. My Ollama is fine, but "nothing happens".

[dbcandle]

I tried other URLs:

Tool expects…	Use this
Ollama/native API base URL	http://localhost:11434
Ollama chat endpoint	http://localhost:11434/api/chat
Ollama generate endpoint	http://localhost:11434/api/generate
OpenAI-compatible base URL	http://localhost:11434/v1
OpenAI-compatible chat completions endpoint	http://localhost:11434/v1/chat/completions

Everything except http://localhost:11434/v1 gives me a 404 error. http://localhost:11434/v1 returns nothing, per screen-shot above. Was going to try DeepSeek (afordable), but with no 2FA and security concerns per ChatGPT "Deep Research", I thought the better of it...

[otykier]

Thanks for reporting this. We'll try to reproduce this with Ollama and gemma4:12b and get back to you.

[dbcandle]

FYI, with the latest Ollama on Windows 11, I find the model "mistral-nemo:latest" works with AI Assistant. Of course, all I did so far was prompt it to "describe my current data model". I'll play with it more.

These two models just return nothing though (still with Ollama). AI assistant thinks a bit, then returns nothing. These two models return nothing: "gemma4:12b" and "gemma4:latest".

[dbcandle]

FYI, I just signed-up for OpenRouter (free), and initial testing looks promising!

Sure better than the feeble models I can run locally on 12Gb VRAM!

[dbcandle]

I'm confused. I thought a screen would appear confirming before any data is sent (as opposed to metadata). Could the contents of my tables be sent to a cloud AI without my knowledge!? Ever!?

So far, the only prompt I used with a cloud-based AI was "describe my data model". Could any table data have been sent? Hopefully just metadata! Trying it with a local model (not cloud), the confirmation I see is:

Permission Required
The AI assistant needs to access model metadata. The AI would like to access the metadata of your semantic model.

Sorry to be dense, but before agentic AI costs went "through the roof" a couple months back, I had written my own AI-based interface to Power BI Desktop, and the distinction between metadata and table data was a key function. Unfortunately, I couldn't afford to maintain the application (AntiGravity first, then GitHub Copilot).

[otykier]

@dbcandle
You should see separate consent prompts for metadata and data.

For example, a prompt such as "describe my data model" would most likely cause the LLM to request model metadata:

Where as a prompt such as "analyze this table for data quality issues" would cause the LLM to write a DAX query, thereby requesting access to data:

You can click the "Copy" button to get the DAX query into the clipboard, if you want to inspect it and run it manually in TE3, before giving consent.

[mlonsk]

Hi @richardchen874-sys, we really appreciate your input and your taking part in the discussion. I've had to hide two of your comments about the API, as these fall under the self-promotion restriction in our Code of Conduct.

There's also one paragraph in your longer reply above, the part beginning "I’m currently", that falls under the same restriction. There are a lot of valid points in that comment, so rather than hide the whole thing, could you edit it yourself to remove just that paragraph?

[dbcandle]

I'm in the US. I think the Chinese models show a lot of promise, and chip technology there is advancing rapidly. My main concern is security. Ignoring prompts being saved/used for "training purposes", my initial concern is for 2-factor authentication (2FA). I was going to subscribe to Deepseek, but finding they don't even support 2FA, I decided not to subscribe.

[dbcandle]

@dbcandle You should see separate consent prompts for metadata and data.

For example, a prompt such as "describe my data model" would most likely cause the LLM to request model metadata:

Where as a prompt such as "analyze this table for data quality issues" would cause the LLM to write a DAX query, thereby requesting access to data: You can click the "Copy" button to get the DAX query into the clipboard, if you want to inspect it and run it manually in TE3, before giving consent.

My sincere personal thanks for this specific and EXTREMELY useful response! Feeling much better now.