AI Database Agent, Part 1: Naïve Text-to-SQL and Why It Fails Silently

Meta Description: Naive Text-to-SQL with a 7B model scored 53% on 32 questions, with 7 silently wrong answers and 2 unsafe statements sent to SQL Server. See what failed and why.

Suggested URL Slug: ai-database-agent-part-1-naive-text-to-sql

Secondary Keywords: naive Text-to-SQL, AI database agent, Text-to-SQL SQL Server, Text-to-SQL benchmark, execution accuracy, qwen2.5-coder, Ollama Text-to-SQL, ASP.NET Core AI agent, LLM SQL generation

The baseline every Text-to-SQL demo starts from: table names in, SQL out, measured against 32 real questions. 53% correct, seven silently wrong answers, and an UPDATE and a DROP TABLE sent to the database.

AI Engineering Foundry · September 27, 2026 · 3 min read

Part 1 of the AI Database Agent series: five versions, each measured on the same benchmark.

Give a code model the table and column names, ask the question, run whatever SQL comes back. This is the demo everyone builds first, and it's worth building properly, because it's the baseline every later version is measured against.

The setup for the whole series: an ASP.NET Core API, a SQL Server database called FoundryTrade (a seeded trading business with customers, orders, products, stock, and staff), and a local model, qwen2.5-coder:7b, served by Ollama.

system: You translate questions into SQL for a Microsoft SQL Server database. Reply with a single T-SQL query in a ``` sql code block and nothing else. user: Tables: - catalog.Categories(CategoryId, Name, ParentCategoryId) - sales.Orders(OrderId, CustomerId, SalesRepId, WarehouseId, OrderDate, Status,...)... Question: Which 5 customers spent the most in 2025? Enter fullscreen mode Exit fullscreen mode No types, no keys, no descriptions, no business rules.

From real runs against FoundryTrade with qwen2.5-coder:7b:

The benchmark is 32 questions with hand-written gold SQL, scored by execution accuracy: the agent's result set must match the gold result set. Six of the questions should be declined (writes, confidential data, prompt injection, data that doesn't exist). Scoring rules: eval/README.md.

Seven answers were silently wrong. They ran and returned numbers, just not the right ones. That is the real problem with naïve Text-to-SQL: failure doesn't look like failure.

The model isn't bad at SQL; it's missing context. It needs types, keys and join paths, what the columns mean, which values an enum can take, the dialect, the business rules, and permission to say "I can't answer that."

Source code, setup guide and all decision records: AfzaalLucky/ai-database-agent on GitHub.