Vibe Coding... Vibe Testing?

AI Gravity Testing
Julien Botella

Julien Botella

Published on 26.03.25

Recently, I embarked on an unusual experiment: developing a React Native app from scratch using Cursor. But with a twist… I wanted to test “vibe coding.” But what exactly does that mean?

It’s an approach where you let AI generate as much code as possible, simply providing high-level instructions and quickly stacking features together. The goal? To see how far it can go on its own and assess the effectiveness (or limitations) of this method. So, I kicked off my project with this mindset..

I handed everything over to AI—client, server, database—approaching it as if I were a complete beginner with little to no development knowledge. The pace was impressive. The most surprising part? AI didn’t just generate code—it also suggested architectural choices, database strategies, and even solutions to problems I hadn’t anticipated yet.

Then, inevitably… a bug appeared.

The AI attempted to fix it. Then suggested another solution. And another. Only to eventually… return to the first one. Problem? None of these fixes actually worked.

I had to step in and investigate. And that’s when frustration hit: I had no real understanding of what was happening under the hood. The AI had generated so much code that I lacked the context to pinpoint the root cause. Everything felt chaotic, and I found myself more of a spectator than an active participant.

Time for a new approach: this time, I opted for a structured Test-Driven Development (TDD) methodology.

  1. I started by writing unit and UI tests.
  2. The AI generated them, and I reviewed, adjusted, and refined them.
  3. Once satisfied, I instructed the AI to write only the necessary code to make the tests pass—without modifying the tests themselves.
  4. I structured the code clearly, organizing files logically.
  5. Then, I iterated, refactored, and moved forward incrementally.

The result? Much better!

I had a clear understanding of my code, confidence in my tests, and—most importantly—full control over the development process.

Dario Amodei (CEO of Anthropic) recently claimed:

“AI will write 90% of the code for software engineers within the next three to six months and every line of code within the next year.”

Well… I don’t entirely agree. (Not that he’s waiting for my opinion 😆).

Of course, there’s a marketing angle here. He’s promoting his technology and trying to captivate (or scare) investors. But in the real world, things are more nuanced.

We often hear this phrase:

“AI won’t replace developers, but developers who use AI will replace those who don’t.”

And I’m convinced this is exactly what’s going to happen. I don’t even consider myself a “real” developer, yet by leveraging AI, I can be productive. Now imagine an expert using these tools to their full potential!

At Gravity, we focus heavily on end-to-end testing by analyzing anonymized production sessions. Some tools claim to generate end-to-end tests automatically from production sessions. Well… almost.
In reality:

  • The setup is often complex.
  • Test data doesn’t magically generate itself—especially from anonymized production data.
  • And when a test fails, someone has to investigate.

And here we go again: too many false positives, unclear context, and difficulty understanding the intent behind each test.

Our vision with Gravity is that AI should assist testers, not replace them.

Our objectives:

  • Provide reliable and objective metrics to help prioritize testing efforts.
  • Reduce the cost of designing both automated and manual test scripts.
  • Guide testers while ensuring they remain in control of the process.

AI alone can’t (yet?) do everything. But a tester or developer who knows how to use it wisely will always outperform someone who ignores it… or blindly relies on it.

“AI won’t replace testers, but testers who use AI will replace those who don’t.”

This “Vibe Coding” experiment confirmed that AI can be a powerful ally—but it can also quickly get out of hand if you don’t maintain control.

By following a structured approach (like TDD), we can enjoy the best of both worlds:

  • The speed and assistance of AI
  • Human oversight and understanding

What if we applied this philosophy not just to testing, but to all technical tasks?

The future of development and testing won’t be AI vs. Humans, but AI + Humans. And that sounds like good news, doesn’t it? 😃

If you want to read a text in French, visit Julien’s LinkedIn.

Stay tuned!

Gravity product update january 2025

Gravity product update January 2025

Gravity News Gravity

Hello ! ❄️ Winter has settled in, and we’re excited to introduce the latest updates to Gravity! Over the past…

Aligning Continuous Testing with Customer Journeys through AI

Aligning Continuous Testing with Customer Journeys through AI

Gravity Testing

Businesses are under unprecedented pressure to innovate quickly and deploy new features while ensuring high standards through continuous testing. Development…

Boost Your Agile Testing Efficiency in Jira with Yest and Xray

Boost Your Agile Testing Efficiency in Jira with Yest and Xray

Testing Yest

In agile development, efficient testing and collaboration are key. Although methodologies and ceremonies guide the process, choosing the best tools…