Text-to-SQL on Oracle: two prompt rules took a light model from 35% to 64%

UniQL is a new benchmark that translates BIRD's 1,534 text-to-SQL questions into 16 SQL dialects. Its Oracle column is hard. The best published result is 49.61% (Claude Sonnet 4.5), and Gemini 2.5 Pro scores 26.14%.

I ran schemagate on it with a light model, Gemini 2.5 Flash. The scores below come from UniQL's own evaluator, on Oracle Free 23ai.

The first run looked good for the wrong reason. 69.9% was mostly schemagate's repair loop: run the query, show Oracle's error to the model, let it fix it. On a first try alone, Flash got 35%. That's below the bare models, and not a fair comparison with their single-try numbers.

Two mistakes caused almost all of it. Of the 694 failed first queries:

So schemagate now tells the model Oracle's rules whenever the target is Oracle. First-try accuracy went from 35.0% to 64.5%, about 15 points above the best published model, with a lighter one. Failed first queries dropped from 694 to 85. Other databases' prompts are unchanged.

The lesson I'd take: measure before you tune, and report the number that matches what others report. A repair loop is a fine design, but it doesn't belong in a column of single-try results.

Submitted to the UniQL leaderboard: JerryGao818/UniQL#2. Predictions, evaluator output and the workflow that reproduces it: https://github.com/ashishsinha1602/schemagate/tree/main/benchmarks/results/uniql_oracle