From Chatbots to AI Agents: When Should Software Take Action?

For the past few years, much of the conversation around Artificial Intelligence has focused on chatbots.

We ask questions, receive answers, generate content, summarize documents, and interact with AI through conversational interfaces. These capabilities have already changed how people work with software.

AI is moving beyond simply answering questions. It is increasingly being used to perform tasks, interact with systems, execute workflows, and take actions on behalf of users.

This is where AI agents enter the picture.

However, there is an important engineering question that every software builder should consider:

Just because AI can take an action, does it mean we should allow it to?

A traditional chatbot primarily responds to user input.

"What were my total expenses last month?"

The chatbot processes the question, retrieves or interprets relevant information, and provides an answer.

The difference is not simply that one uses a better AI model.

The real difference is the ability to interact with external tools and systems.

An AI agent can communicate, reason through a workflow, and take permitted actions.

One common misconception is that building an AI agent simply means connecting an application to an LLM and giving it a prompt.

That is not enough for production software.

A reliable agent typically requires several components:

Interprets user intent and determines the next step.

Allow the agent to interact with external systems, databases, services, and application functionality.

Enforces the actual rules of the application. Financial calculations, permissions, validation, and transaction rules should not depend entirely on AI-generated decisions.

Help the agent understand what has already happened in a workflow and what information is still required.

Controls which actions the agent can perform independently and which require user confirmation.

Make it possible to understand what the agent did, which tools it called, and where something went wrong.

An agent is therefore not just a model. It is an engineered system built around a model.

Imagine an AI-powered business application.

"Show me customers who have outstanding payments."

This is a relatively low-risk request. The system can retrieve customer records and display the results.

"Send payment reminders to all customers who owe me money."

This request involves an external action.

The system needs to identify the correct customers, prepare appropriate messages, verify the communication channel, and potentially send messages on the user's behalf.

Should the AI send everything automatically?

This approach combines AI flexibility with deterministic software control.

The goal is not to eliminate human involvement. The goal is to reduce unnecessary work while keeping important decisions under control.

Automation is valuable, but uncontrolled automation can create serious problems.

These are not merely AI problems. They are software engineering, security, and product-design problems.

For this reason, production AI systems should follow clear permission boundaries.

Low-risk actions: Reading records, searching, filtering, summarizing, and generating draft reports.

Medium-risk actions: Preparing reminders, creating draft invoices, or proposing changes for review.

High-risk actions: Sending external communications, modifying financial records, deleting data, or initiating payments.

The higher the impact of an action, the stronger the validation and approval requirements should be.

One of the most important architectural decisions is knowing which responsibilities belong to AI and which belong to traditional software.

Traditional application logic should handle:

For example, an AI agent may understand that a user wants to record a payment of PKR 5,000.

But the actual balance calculation, transaction validation, database update, and audit trail should be handled by the application's trusted business logic.

AI should help decide what needs to happen. The application should control how it happens.

This separation makes systems more predictable, testable, secure, and maintainable.

AI agents are not limited to large enterprises.

Small businesses can benefit from carefully designed automation too.

Consider a shopkeeper managing customers, expenses, receivables, and daily transactions.

A future AI-powered business assistant could potentially help with workflows such as:

"Which customers have the highest outstanding balances?"

"Prepare a transaction entry for the payment I just received."

"Summarize today's income and expenses."

"Prepare reminders for customers whose payments are overdue."

"Compare this month's expenses with the previous month."

These are practical use cases because they address real business problems rather than introducing AI simply for marketing purposes.

At HisabDo, the broader product direction is focused on making everyday financial management easier for individuals and small businesses. AI-assisted workflows could become a useful extension of that direction, provided they are introduced with proper validation, permissions, and user control.

The objective should always be to solve a real problem.

When building an AI agent, it is tempting to connect every available API and allow the system to perform multiple operations.

A more reliable approach is incremental:

Allow the agent to search, retrieve, and summarize information.

Let it identify patterns and suggest possible actions.

Allow it to prepare records, messages, and reports without submitting them.

Introduce actions that require explicit user confirmation.

Only automate well-tested, low-risk tasks with clear boundaries, monitoring, and recovery mechanisms.

Each stage should be evaluated before moving to the next.

Measure accuracy, execution success, latency, cost, failure rates, and user trust.

A useful AI agent is not the one that performs the most actions. It is the one that performs the right actions reliably.

AI agents introduce new challenges that software teams must address.

Reliability: Models can misunderstand instructions or select inappropriate tools.

Security: Tool access must follow least-privilege principles. An agent should never receive broader permissions than its task requires.

Data privacy: Sensitive business and customer information must be protected throughout the workflow.

Cost and latency: Multiple model calls and API interactions can make agent workflows expensive or slow.

Observability: Developers need visibility into decisions, tool calls, failures, and execution history.

Idempotency: Repeated requests or retries should not accidentally create duplicate transactions or trigger the same external action multiple times.

These challenges remind us that building an AI agent is not just an AI development exercise.

It is a complete software engineering responsibility.

I believe the next generation of useful business applications will combine three capabilities:

We should not measure progress only by how much work AI can perform independently.

We should measure it by how effectively technology helps people complete their work.

For businesses, the real opportunity is not replacing every human decision with AI. It is removing repetitive work, making information easier to understand, and helping people make better decisions.

That is where AI agents can create meaningful value.

Moving from chatbots to AI agents is an important step in the evolution of software.

But the transition requires more than connecting an LLM to APIs.

It requires thoughtful architecture, secure integrations, reliable business logic, clear permissions, monitoring, and human oversight.

As software engineers and product builders, we must remember:

The future of AI is not just about building systems that can take action. It is about building systems that know when, how, and under whose authority to take action.

That is the difference between an impressive AI demo and a dependable AI-powered product.

In Edition 7 of The Builder’s Letter, we will explore:

"The Data Problem in AI: Why Better Data Often Matters More Than a Bigger Model."

Because even the most capable AI model cannot consistently produce reliable results when the information behind it is incomplete, inaccurate, or poorly structured.

Mian Usman Khalid Founder & CEO, XICTEK Systems

The Builder’s Letter — Building practical technology for real-world problems.