Key Takeaways: Ship Over Hype - Scaling AI Across 60 Android Engineers

AI has become a buzzword, and many people use it in articles, LinkedIn posts, and conferences without explaining how it actually helps in day-to-day work. Because of that, in our conversation between Ilya and our CTO, Aleksa, the focus was on how engineering teams can integrate AI into their workflow without disrupting productivity.

Meet Ilya Savin

Ilya Savin is an engineer Manager at Qonto, a company that provides a finance solution that creates the freedom for SMEs to succeed in business, where he leads a big team of 60 Android developers.

He started as an Android developer, and his last 10 years have been spent building and scaling apps from idea to 100+ dev orgs. Now, he is a big AI supporter, trying and testing tools, and figuring out what actually works!

For this webinar, Ilya prepared a presentation where he explained in detail what is the initial challenge of adopting AI in development for companies such as Qonto, how everything looked in practice, how they measured the success of implementation, and a lot more.

So let’s start:

Q: What was the initial challenge for a banking app like Qonto when using AI tools?

For Banking ups like ours, we need to be sure that user data is protected, and we were extremely concerned about sending corporate and customer data to external, public AI models, so we

have found more compliance solutions - Dust that works like a company in the middle that proxies and stores our data in a secure way.

(Dust is a tool/platform that enables teams to use different AI models (such as OpenAI, Anthropic, Llama…) in a secure, centralized, and controlled way - without giving individual API keys to every developer.)

To make AI more aligned with our needs, in June 2024, we integrated it with Slack and Notion to have a better context about us. This system makes onboarding easy.

In this moment, it was just reading opt-in, not writing.

Later, in September, we started using bots.

Q: How did AI help you with coding?

Dust helps us integrate a lot of things, and one of them is GitHub. Our developer was prompting, asking ChatGPT many things about the code, but we got generic answers. But after implementation, we can ask the AI with full-repo context and start getting Qonto-specific code insights.

Something like what you can do now with Cursor, but much earlier.

About 50% of our developers tried Dust for explaining architecture patterns, onboarding, standards, and processes..

Q: Have you experimented with generative AI?

In the process of adopting AI, we ran several experiments. Some of them were successful and some were not, but we learned a lot from each of them.

We once let an AI generate code based on our Figma screenshots by connecting it to Cursor using MCP. Unfortunately, the experiment failed. Some of the reasons are:

• Our Design System was too big for LLM, and AI was hallucinating
• MCP servers were not stable
• Low adoption among developers because it was Cursor - AI outside of Android Studio

In another experiment, we have a problem when our models contain many parts with more than 300 files. Not clear ownerships and a long time to build for any minor change. So we try with Claud Code, an Agentic CLI tool that “lives” in the terminal and has access rights in Android Studio and XCode. And we want to separate the module, speeding up building time.

Results:We estimate it to be done in 3 weeks with handwork, but with ClaudCode we successfully finished the task in 5 days.

Q: What would you recommend for someone who doesn't have time to experiment with different approaches

We made an AI starter guide that can be implemented by any company in any development team, Android or iOS.

Some tools that we recommend for the start and work fine are:

• Claude Code
• Gemini Code Assist and
• Dust

Q: How to measure the impact of AI tools.

Spoilers - nobody cracked it yet.

Measurement of AI impact is still the hardest part.

Metrics such as lines of code were never useful, because only about 40% of the cycle time was actually spent on coding.

For proper testing, you need A/B testing. A team that uses AI tools and a team that doesn’t.

There are some objective metrics that we track, like the level of adoption of AI tools and their activity.

But also, we ask our developer about their experience and their perceptions

Q: What are the Key takeaways from this presentation

The key takeaways are:

• Pick a champion - Choose the right persons who will lead AI adoption, and test their abilities. Someone who will drive the AI initiative. We in Qonto call them champions
• Build a safe sandbox early  - For us, it was a vendor, Dust, and we experimented in read-only mode. It is an essential step. If I were starting AI adoption in another organization, I would still start from this, not from Cloud Code, not from Coursor.
• Start with boring work - Don’t start with features. Be realistic and don’t set false expectations.
• Keep the menu short - There are a lot of tools in the market, and you can be easily overwhelmed.
• Embrace wins, learn from failure - You have to be there to experiment and fail. It’s normal, but the key is to learn from it.

This was a key question Ilya implicitly answered during his presentation. At the end of the presentation, we left a QR code from where you can access to presentation, experiments, and other interesting material.

‍