Von der Fehlermeldung bis zum Production-Fix: So nutzt ein QA-Engineer KI

Signup Write New Story From Bug Report to Production: How I Fix Bugs with AI by inDrive.Tech by inDrive.Tech | @ indrivetech Team of inDrive developers who know how to experiment and learn from their mistakes for growth.

Team of inDrive developers who know how to experiment and learn from their mistakes for growth.

Hi! I’m Tim, QA Engineer at inDrive. Some time ago, when I was updating tests for new endpoints, one of them behaved strangely. The service rejected a request, but on the next attempt it said the action had already been completed. In fact, nothing had happened.

The problem was with an idempotency key: the service reserved it before validating the request. If validation failed, the key remained reserved anyway. I prepared a fix, went through code review, and shipped the change. After the release, duplicate-key errors on the affected endpoints dropped by about 30%.

Previously, my work on a bug usually ended with a detailed report and retesting someone else's fix. Now, with Claude, I can go through the whole process by myself: understand unfamiliar code, check a hypothesis against logs, prepare a patch and tests, submit the review change, and take it through to release. Today, I independently ship an average of three to five small fixes and improvements a week.

I distinguish three levels of this approach based on my experience at inDrive, which I'll describe below. It's important to note that this is not a mandatory career path for every QA engineer, nor does it mean handing responsibility over to an agent. In this approach, AI helps investigate systems faster, but the engineer still checks the facts, assesses the risks, and takes responsibility for the result.

A typical bug report answers the question, “What is broken?” But the developer still has to find the cause, understand the related parts of the code, and choose a solution. To save developers time, I started adding the results of that investigation to my bug reports.

First, I ask an AI agent to trace the flow of data and find a possible cause. Then I check every conclusion in the repository: whether that piece of code actually runs in the scenario, how similar cases are handled elsewhere, and what other parts of the system the patch might affect. If a hypothesis isn't confirmed, I don't present it as a fact.

As a result, each ticket includes six sections:

This kind of report doesn't guarantee that the proposed solution is correct. But it gives the developer a verified starting point instead of a task that still needs to be investigated from scratch.

The inDrive order service has separate ride options for an assistive device, a service animal, and a wheelchair-accessible car. When the service built its response, all three values were mistakenly read from the same field: assistiveDevice.

As a result, the driver might not see a request for an accessible car if the passenger hadn't selected an assistive device. And when the device option was selected, its description appeared under the other options as well. No tests covered this case.

response.assistiveDevice = describe(order.assistiveDevice) response.serviceAnimals = describe(order.assistiveDevice) response.wheelchairCar = describe(order.assistiveDevice) After the fix:

response.assistiveDevice = describe(order.assistiveDevice) response.serviceAnimals = describe(order.serviceAnimals) response.wheelchairCar = describe(order.wheelchairCar) The cause was two copy-and-paste mistakes. In the ticket, I pointed to the exact location and specified the necessary changes and checks. The first available developer quickly made the fix, and the ticket was closed a couple of days later.

What matters here is not only the size of the patch but also the consequences of the bug: because of two incorrect field references, passengers who needed special ride arrangements might not get the service they required. That’s frustrating, and it had to be fixed fast.

In another investigation, the agent helped find an endless recursive call in an error handler: for an unknown error, the function called itself with the same arguments. This could cause a stack overflow and crash the process. I explicitly noted in the ticket that the defect had been found by reading the code, not reproduced on a running instance. It's essential to preserve that distinction.

Even when you've found the cause, a question remains: how serious is the problem? How many requests or inDrive users does it affect? Can you confirm it not just from the code but also from how the system behaves?

For this, I analyze logs and metrics. The agent helps identify which events the relevant code writes, build queries, collect results across clusters, and group errors by cause. Where access to logging and metrics tools is available, MCP connectors are a convenient option.

But the results still need to be checked: the wrong filter or denominator can easily turn a convincing number into a false conclusion.

While analyzing logs from the inDrive order service, I found a large number of 500 errors occurring during calls to the bidding service. To understand the causes, I used AI to build log queries and collect one week of statistics across 13 clusters.

In total, I found about 1.86 million such errors. Preliminary analysis showed that roughly 96.6% of them—about 1.79 million—were caused by clients cancelling requests.

The cause was in how errors were handled between services.

When a client closed the connection during a call to the bidding service, that service returned gRPC Canceled. The order service converted it into a generic 500 server error and wrote it to the logs.

Yet other parts of the service already handled these situations correctly: if the client closed the connection, the server did not try to send a response or log an error.

I checked whether these cancellations were caused by slow responses from the bidding service. They weren't: calls completed in milliseconds. For example, the median duration of an unsuccessful bid retrieval call was 7.5 ms, compared with 5.6 ms for a successful one.

In other words, slow responses were not the problem. The client had already stopped waiting, while the server continued to count the cancelled request as its own failure.

As a result, monitoring showed errors where the server had not actually failed. Including related scenarios, client cancellations generated several million ERROR-level log entries in a week. This noise made it harder to distinguish real failures from expected system behavior.

Based on the investigation, I proposed changing the error handling:

According to a preliminary estimate, this change could eliminate about 96% of the 500 errors in question. That's the expected effect calculated from logs, not the result of a deployed fix. The findings also need to be reviewed with the bidding team.

This example shows why QA engineers should analyze production data. A high number of errors in monitoring doesn't necessarily mean that users experienced the same number of failures. Before assigning priority and proposing a fix, you need to understand what exactly the system measures and how it classifies errors.

While analyzing another scenario, I found that about 3% of requests to the ride-status transition endpoint failed over the course of a day. The logs showed that 90% of those errors occurred when the driver requested a status the ride was already in. And 55% of the errors involved a transition to “completed” for a ride that was already completed.

Timing data and traces pointed to repeated taps, stale screen state, and bursts of requests from an older ride screen. The next day, I repeated the analysis and got the same 90%.

Before putting metrics in a ticket, I check the time period the data covers, which clusters are included, what counts as an error, and what denominator is used to calculate each percentage. Then I check whether a spike can be explained by one instance or a rare scenario and, where possible, repeat the analysis over a different period.

It's also important to distinguish three things:

They shouldn't be presented together as one proven fact.

When the cause is clear, the risks are limited, and the change is small enough, a QA engineer can prepare the patch by himself instead of handing the task to a developer. Although AI helps write code and tests, this in no way means that a patch can be shipped without the usual engineering checks.

At inDrive, I started with simple changes reviewed by developers. Over time, I began taking on more substantial tasks. Now my process includes code analysis, testing, code review, the standard release process, and verifying the impact after release.

Let's return to the bug from the beginning of the article. Some endpoints reserved the request key before validating its contents. If validation failed or subsequent processing ended unsuccessfully, the key stayed reserved. The user's next attempt received a response saying an operation had been repeated, even though it hadn't happened in the first place.

In simplified form, the original sequence looked like this:

reserve(key) // the key is reserved before validation validate(request) process(request) After the fix:

validate(request) reserve(key) // only a valid request reserves the key try: process(request) onFailure: release(key) // the request can be retried after failure For an operation that really has already completed, the conflict response remains. But the response now also includes an explicit reason that can be tracked in monitoring.

After the release, I checked the metrics. Duplicate-key errors on these endpoints were down by about 30%; the decrease held every day after the release and across every cluster with traffic. Median response time, the 99th percentile, and other errors showed no noticeable change. The additional load on the shared cache—about 0.01%—had been assessed before the merge and considered acceptable.

That result matters more than the fact that AI helped write a few lines of code: the fix reached production, and we could verify its expected effect.

One service was crashing at startup with an unclear error. At first, it looked like a broken response parser. By tracing execution with AI, I followed the issue beyond our service into the shared authentication library. When the auth service returned a plain-text error page, the client tried to parse it as JSON and lost the actual response status code.

While investigating the source of that error further, I found a more serious problem: a background job that was supposed to refresh keys regularly ran only once and then stopped. After the next key rotation, services using an older version of the library could keep using stale keys, causing token validation to fail.

We reproduced the scenario. The fix was reviewed by the owners of the shared library, and then a new version was released. This case shows why it's important to investigate a root cause all the way through, including beyond the service where the error first appeared.

The agent is good at reading unfamiliar code, tracing connections between services, and drafting log queries, patches, and tests. But it can also confidently explain a cause that doesn't actually exist.

That's why I keep four decisions for myself:

These aren't special rules for AI. They're ordinary engineering discipline, and they become essential when a QA engineer at inDrive starts changing production code.

For me, the biggest change in QA work isn't that an agent can write code. I no longer have to stop at reporting a bug: I can find the cause, confirm it with data, prepare a fix, and—when the risk is acceptable—take it all the way to production. Responsibility for quality still stays with the engineer.

My growth over the past few months has been possible thanks to the support of my colleagues: they review my code, ask questions, and help me grow and push me further. Thank you to the Driver Experience team and to all the colleagues at inDrive who share their practices and support this process.

How We Scaled to 530 Aurora Clusters With a 4.5-Person DBA Team