Prevent Hallucinated Bot Commands: Build an LLM Tool Loop for Telegram in Node.js

Posted on Oct 26 • Originally published at botservice.biz

When building a Telegram bot powered by a Large Language Model (LLM), developers often start by feeding the user's message directly to an inference API and piping the raw string output back to bot.sendMessage(). This works fine for generic Q&A, but breaks down completely when the user asks the bot to execute a concrete action, such as "Check the status of order #9482" or "Book a table for 2 people at 7 PM".

If you rely on prompt engineering alone—asking the LLM to return plain text or raw JSON formatted like {"action": "check_order", "id": 9482} —the system will inevitably fail in production. The model will hallucinate missing parameters, output conversational text around the JSON block, format output with unparseable Markdown fences, or generate fabricated database records when an order ID is missing. Even worse, attempting to resolve system tasks by feeding sensitive system context or credentials into the LLM system prompt risks exposing your Telegram bot token or internal database structure.

To build a reliable transactional bot, you must implement strict function calling (tool use) in Node.js. In this pattern, the LLM does not execute actions directly and never sees your API keys. Instead, the model outputs a structured intent JSON, your Node.js application executes the local code (such as querying PostgreSQL or calling a REST service), and the results are fed back into the model to generate a final human-readable response for the Telegram user.

A common anti-pattern in early AI Telegram bots is attempting to parse intended actions out of freeform assistant responses using regular expressions or JSON.parse.

Consider a typical customer support bot built with Node.js and a standard Telegram library like grammY or node-telegram-bot-api. The naive implementation tries to extract parameters directly from unstructured model text.

// BAD: Relying on unstructured text and regex extraction bot. on ( ' message:text ', async ( ctx ) => { const userText = ctx. message. text; const prompt = `You are a support bot. If the user wants order status, reply ONLY with JSON: {"action": "get_order", "id": 123}. User: ${ userText } `; const response = await openai. chat. completions. create ({ model: ' gpt-4o-mini ', messages: [{ role: ' user ', content: prompt }], }); const replyText = response. choices [ 0 ]. message. content; try { // This will fail when the model returns ``` { % endraw % } json... { % raw % } ``` or adds text like "Sure, here is..." const command = JSON.parse(replyText); if (command.action === 'get_order') { const order = await db.orders.find(command.id); await ctx.reply(` Order status: $ { order. status } `); return; } } catch (err) { // Falls back to sending raw text, which often leaks system instructions or fails completely await ctx.reply(replyText); } }); Enter fullscreen mode Exit fullscreen mode This approach breaks under predictable operational conditions:

To fix this, you must separate model intent from execution logic using explicit API schemas and a controlled tool loop.

Native function calling requires defining explicit JSON schemas for every utility function your Telegram bot supports. These schemas inform the LLM about available tools, required parameter types, and property descriptions.

Create a dedicated definitions file ( tools.js ) that defines two concrete actions: fetching an order status ( get_order ) and creating a table booking ( create_booking ).

// tools.js export const botTools = [ { type: ' function ', function: { name: ' get_order ', description: ' Fetch the status and delivery details of an existing customer order. ', parameters: { type: ' object ', properties: { order_id: { type: ' string ', description: ' The unique alphanumeric order identifier, e.g., ORD-5502. ', }, }, required: [ ' order_id ' ], }, }, }, { type: ' function ', function: { name: ' create_booking ', description: ' Reserve a table for a customer at the restaurant. ', parameters: { type: ' object ', properties: { guest_name: { type: ' string ', description: ' Full name of the primary guest making the reservation. ', }, party_size: { type: ' integer ', description: ' Number of guests attending (1 to 10). ', }, booking_time: { type: ' string ', description: ' ISO 8601 formatted date and time string for the reservation. ', }, }, required: [ ' guest_name ', ' party_size ', ' booking_time ' ], }, }, }, ]; Enter fullscreen mode Exit fullscreen mode By specifying required fields in the schema, the inference engine knows it must ask the user for clarification if essential values (like order_id or party_size ) are missing from the input message, rather than guessing them.

Never pass database connection strings, internal server tokens, or your Telegram Bot Token into tool descriptions or system messages. The model only receives function arguments; actual execution takes place within your Node.js runtime.

The core architecture of a reliable Telegram AI agent consists of a multi-turn conversation loop within a single HTTP request handler:

Here is the full implementation using Node.js, grammY, and the official @openai/api SDK.

// bot.js import { Bot } from ' grammY '; import OpenAI from ' openai '; import { botTools } from './tools.js '; const bot = new Bot ( process. env. TELEGRAM_BOT_TOKEN ); const openai = new OpenAI ({ apiKey: process. env. OPENAI_API_KEY }); // Local mock database handlers async function getOrderDetails ( orderId ) { const mockDb = { ' ORD-101 ': { status: ' Shipped ', carrier: ' FedEx ', tracking: ' FX-99281 ' }, ' ORD-102 ': { status: ' Processing ', carrier: ' Pending ', tracking: ' N/A ' }, }; return mockDb [ orderId ] || { error: ' Order not found in database. ' }; } async function createTableBooking ( guestName, partySize, bookingTime ) { // Validate and write to database const bookingId = `BK- ${ Math. floor ( 1000 + Math. random () * 9000 )} `; return { success: true, booking_id: bookingId, details: { guestName, partySize, bookingTime }, }; } bot. on ( ' message:text ', async ( ctx ) => { const userMessage = ctx. message. text; // Initialize short-term memory array for this request lifecycle const messages = [ { role: ' system ', content: ' You are an automated Telegram customer support agent.

Answer questions concisely using provided tools. Do not invent order statuses or fake reservations. ', }, { role: ' user ', content: userMessage }, ]; try { // Initial call to LLM with tool definitions let response = await openai. chat. completions. create ({ model: ' gpt-4o-mini ', messages: messages, tools: botTools, tool_choice: ' auto ', }); let responseMessage = response. choices [ 0 ]. message; // Process tool calls if requested by the model if ( responseMessage. tool_calls && responseMessage. tool_calls. length > 0 ) { // Append the assistant's request for tool execution to message history messages. push ( responseMessage ); for ( const toolCall of responseMessage. tool_calls ) { const functionName = toolCall. function. name; const functionArgs = JSON. parse ( toolCall. function. arguments ); let functionResult; if ( functionName === ' get_order ' ) { functionResult = await getOrderDetails ( functionArgs. order_id ); } else if ( functionName === ' create_booking ' ) { functionResult = await createTableBooking ( functionArgs. guest_name, functionArgs. party_size, functionArgs. booking_time ); } else { functionResult = { error: ' Unknown function call. ' }; } // Return tool output to the model conversation history messages. push ({ tool_call_id: toolCall. id, role: ' tool ', name: functionName, content: JSON. stringify ( functionResult ), }); } // Second call to LLM to process tool results into natural response response = await openai. chat. completions. create ({ model: ' gpt-4o-mini ', messages: messages, }); responseMessage = response. choices [ 0 ]. message; } // Send final synthesized text to Telegram user if ( responseMessage. content ) { await ctx. reply ( responseMessage. content ); } } catch ( error ) { console. error ( ' Execution Error: ', error ); await ctx. reply ( ' Sorry, I encountered an issue processing your request. Please try again later. ' ); } }); bot. start (); Enter fullscreen mode Exit fullscreen mode Production Notes When moving an LLM-powered Telegram bot to production, relying on basic functionality will leave you open to race conditions, billing spikes, and hanging Telegram updates.

Adhere to these key practices:

An LLM can occasionally loop infinitely by requesting tools back-to-back if a tool returns an error payload. Always wrap your execution loop in a strict iteration counter (e.g., maximum 3 tool turns per user update). If the limit is reached, terminate execution gracefully and inform the user.

Telegram messages hard-cap at 4,096 characters per message payload. When passing database results back to the LLM, truncate large data arrays on your Node.js backend before inserting them into the tool output message. If the final model response exceeds 4,000 characters, split the message before invoking ctx.reply().

Telegram Webhooks expect an HTTP 200 OK response within 10 to 60 seconds (depending on bot architecture). Running multiple LLM inference calls and external database queries in sequence can easily exceed webhook execution budgets. For long-running operations:

Treat arguments produced by JSON.parse(toolCall.function.arguments) as untrusted user input. Validate argument types and sanitize strings before feeding them directly into SQL parameters, external shell tools, or secondary API integrations.

Building transactional Telegram integrations requires strict tool boundary separation to guarantee stability under heavy production traffic. If you need assistance scaling complex custom workflows or mini-app backends, get in touch with BotCreator — studio that ships Telegram bots / Mini Apps.