ai newsMo., 26. Okt.
EilmeldungGPT-6.1 Terra halves OpenAI's API prices and still gains ground on its larger models

What TestSprite Taught Me About Testing Concurrency Bugs in FlowGuard AI

A Passing API Response Doesn't Always Mean Correct Behavior Imagine two refund requests arriving at almost the same time. Both requests read the same state. Both believe the refund hasn't been processed yet. Both proceed. Without proper concurrency protection, one logical refund operation could pro

dev.toMontag 21:11 · 26.10.5 Min. Lesezeit
Bild: dev.to

Imagine two refund requests arriving at almost the same time.

Both requests read the same state. Both believe the refund hasn't been processed yet. Both proceed.

Without proper concurrency protection, one logical refund operation could produce two committed side effects.

This is one of the engineering problems that inspired me to build FlowGuard AI, my project for TestSprite Hackathon Season 4.

I wanted to explore an important question:

What if our code works, but the business rule it is supposed to protect fails?

FlowGuard AI was built to help developers investigate these risks through business-rule analysis and controlled verification.

But building the application also taught me another lesson: a reliability-focused product needs a reliable development and testing workflow.

That's where TestSprite became an important part of my journey.

FlowGuard AI is a browser-based behavioral-risk analysis and verification playground.

The application helps developers investigate code snippets and public GitHub pull requests against business requirements.

Extracts important business invariants and expected behavior from the requirements provided by developers.

Examines the available code and highlights potential paths where those invariants might be violated.

Proposes test assertions and verification scenarios that developers can use to investigate the potential problems.

One of the most important principles behind FlowGuard AI is:

A potential risk is not the same as a verified bug.

That's why external code analysis findings are explicitly labeled POTENTIAL.

The application does not execute arbitrary third-party GitHub code. Its Verification Lab uses controlled, first-party synthetic fixtures.

The Verification Lab includes two refund implementations.

In the deliberately vulnerable simulation:

These demonstrations use synthetic, in-memory fixtures rather than real banking transactions.

They illustrate why testing business invariants matters, especially when concurrency and asynchronous operations are involved.

One of the most valuable experiences during this hackathon was integrating TestSprite with GitHub pull requests.

What I really appreciated was the ability to connect automated testing with actual code changes.

Instead of treating testing as something to perform only at the end of development, I could use TestSprite's automated feedback as part of the development workflow.

For me, this made testing feel more structured and meaningful.

TestSprite's GitHub integration was one of the features I appreciated most.

Having automated test results associated with a pull request made it easier to connect development changes with verification.

It encouraged me to pay closer attention to how the application behaved after each iteration.

I think this is particularly valuable for developers who want automated quality checks to become part of their everyday workflow.

During an intermediate execution on FlowGuard AI's pull request #14, TestSprite reported:

The blocked case concerned keyboard navigation and user feedback.

Instead of treating the result as a complete success, I investigated the application's accessibility and interaction behavior.

This led to improvements involving keyboard focus management, form interactions, and feedback for users navigating the application.

What I appreciated most was that TestSprite's feedback encouraged further investigation.

Testing wasn't just about collecting passing results. It became part of improving the actual product.

Another TestSprite capability I appreciated was its integration with GitHub's required checks.

A blocked test shouldn't automatically be considered a successful test.

Connecting automated testing with pull request merge requirements helps encourage developers to investigate unresolved results before merging.

For my project, this supported a more disciplined approach to software verification.

After further development and testing, the final head of PR #14 achieved:

Seeing all 97 TestSprite tests pass on the final pull request commit was one of the most rewarding moments of the hackathon.

The successful official TestSprite V3 execution was associated with the final PR head before merging.

https://www.testsprite.com/dashboard-v3/o/28a6d5ec-2a40-54eb-9bf6-0c87db224fde/projects/570eab99-607a-49fa-96b2-fe5ef724f2d9/execution/901e7027-1c4a-477f-adc9-ca0d17b16635

https://github.com/allfridhosaragii/FlowGuard-AI/pull/14

Alongside TestSprite, I used GitHub Actions to run independent automated tests.

After the final pull request was merged, the project successfully completed:

✅ 70/70 Playwright Chromium tests passed

These results provided additional evidence for the tested application.

Of course, passing automated tests does not guarantee that software is completely free of bugs.

But having multiple verification methods helps developers better understand which behaviors have been tested.

https://github.com/allfridhosaragii/FlowGuard-AI/actions/runs/40273469302

After using TestSprite during Hackathon Season 4, several features stood out to me.

I appreciated how TestSprite brought automated verification into the GitHub development workflow.

The distinction between Passed, Failed, and Blocked encouraged me to interpret testing results more carefully.

The earlier blocked case encouraged investigation and improvements to keyboard accessibility.

Connecting automated testing with pull request checks made quality verification part of the development process.

Being able to connect test results with a specific pull request and tested commit made it easier to explain what had actually been verified.

For someone developing a reliability-focused project like FlowGuard AI, these capabilities were especially meaningful.

My experience with TestSprite throughout this hackathon was positive.

I genuinely appreciated how its automated testing workflow helped me pay more attention to verification, accessibility, and software quality.

If I could suggest one improvement, it would be even richer diagnostics for blocked browser tests.

Additional execution traces, screenshots, environment information, and clearer explanations of where a scenario stopped could make debugging easier.

When a browser test becomes blocked, developers sometimes need to investigate whether the issue is related to application behavior, test configuration, or the testing environment.

More detailed diagnostic information would help developers investigate those possibilities more efficiently.

I believe this would make an already valuable testing workflow even more useful.

Looking back on this journey, I realized that the most valuable result wasn't simply achieving a green testing dashboard.

It was the development process behind it.

This is what made TestSprite valuable to me: it helped make verification a meaningful part of the development process.

The experience reinforced something I want to carry into future projects:

Good software engineering isn't only about writing code that works. It's about understanding what the code must guarantee and collecting evidence that those guarantees hold.

I want to express my appreciation to the TestSprite team for organizing Hackathon Season 4.

This hackathon gave me an opportunity to build a real project, explore software reliability problems, improve my engineering workflow, and learn more about automated testing.

I'm grateful for the tools, the challenge, and the opportunity to improve.

Achieving 97/97 passing TestSprite tests on FlowGuard AI's final pull request was an exciting milestone.

But more importantly, the experience helped reinforce a testing mindset that I want to continue developing.

Thank you, TestSprite, for being part of this journey!

https://github.com/allfridhosaragii/FlowGuard-AI

https://flowguard-ai-smoky.vercel.app/lab

https://medium.com/@pedoprimasaragi/how-testsprite-helped-me-build-a-better-flowguard-ai-from-blocked-tests-to-97-97-passed-77aec2fb0828

FlowGuard AI — Evidence Before Verdicts.

Built for TestSprite Hackathon Season 4.

Quelle: dev.to (dev.to). Original öffnen