November 19, 2026
GenAI Days Nantes 2026: In Production, the Model Is No Longer the Focus
9 minutes reading

At Premier Octet, generative AI is embedded in our daily life, both in our development tools and the products we build for clients. On November 17, we attended GenAI Days in Nantes, following the “Those Who Implement” track.
The common thread of the day can be summed up in one sentence: the model is no longer the focus, because models evolve too quickly. What makes the real difference in production is everything built around them: review processes, the harness, measurement and security.

Here’s a look back at six talks that stood out to us.
1. Shipping Faster, but What, and at What Cost?
As AI accelerates development, the bottleneck shifts: it’s no longer production, but rather deciding what to build and verifying what you deliver.
Jean-Yves Simon – I Ship Faster Than Ever, but No Longer Know Why
Chief Product Officer at Mirakl after a stint at AB Tasty, Jean-Yves Simon opened the day with an observation every product team can relate to. Thanks to AI, the whole chain has sped up: user stories, prototyping, and now, about 80% of code generated by AI. The result: roughly 30% more features released each year. Yet, customers don’t “feel” this acceleration. What they keep asking for is fixing daily pain points.
Speed also brings new challenges. Internal requests explode (prototypes from support, sales, management), parallel projects multiply, and saying yes to everything threatens product consistency: the risk of a “Frankenstein effect.” At Mirakl, the answer comes through annual themes set with the CTO and clear investment allocation:
- 50% for run, maintaining and improving what exists;
- 30% for adjacent, extending the product to neighboring use cases;
- 20% for breakthrough, exploring new directions.
Roles are becoming “fungible”: PMs prototype, devs write specs, designers open PRs. This causes friction, but above all creates a new bottleneck: reviews. Production is no longer what slows teams down; it's their ability to review and validate new output.
Florian Bruniaux – I Don’t Trust My AI-Generated Pull Requests
Florian Bruniaux, independent Chief AI Officer and founding member of RTK, took this further on the code side. With agents, PRs pile up, CI runs nonstop, and each fix sparks a new cycle. He highlighted a hidden cost: while token usage might be budgeted, CI runner costs can soar. In one example, the bill rose from €500 to €3,000 in two months.
His solution: the harness. Practically, this means:
- a qualifying gate before CI to trigger only relevant checks;
- a trio of review bots providing cross-checked feedback;
- regular analysis of 50-100 PRs to spot recurring patterns and continuously improve the harness.
He emphasized a tool he thinks is underrated: Claude Code hooks. These are scripts that trigger at specific moments in an agent session, for instance before or after every action. Automatically linting after each modification, blocking dangerous commands, enforcing test passes before handover: where a prompt instruction can be ignored, a hook executes every time, adding determinism to an inherently non-deterministic system. For him, this reshapes the developer’s job: they no longer write every line, but design the framework the agent works in, with its rules, checks and safeguards that guarantee the quality of what is produced.
Lastly, instead of hiding AI intervention, he advocates for maximum documentation: each commit logs the model, its parameters, skills, and hooks used. If issues arise, you can always replay exactly what happened to understand the error and validate the fix.
Key takeaway: on the product side as on the code side, the conclusion is the same. Writing code costs almost nothing now; what matters is knowing what to build and being able to trust what you deliver.

2. Shedding Light on the Black Box for Reliability
Faced with unpredictable models, the first instinct is often to pile on more rules, sub-agents, and intelligence layers, hoping to get better results. The following two case studies show the contrary: reliability comes from transparency and tight framing, making every step explicit, observable, and recoverable if errors occur.
Charli Idrac – My Agent Never Responds the Same Way Twice
Chief Data & AI Officer at Brevo, Charli Idrac shared a refreshingly candid review. The goal: automate customer support ticket handling with an AI agent. The bar was set high: every ticket had to match the quality of the best human support advisors. The result of the first version? A 0% success rate.
Adding rules, more data, sub-agents, or even an LLM-as-a-judge changed nothing; every fix created a new problem. So, the team started over with an ultra-deterministic agent. Each ticket category is coded with its handling in a clear decision tree, and direct API calls are preferred over sub-agents or MCP servers. Their reasoning: stacking agentic components adds around 2% error risk per layer, and these compound.
For tooling, Brevo relies on shadow testing, replaying past tickets, and Langfuse.
Langfuse is an open source observability platform for LLM applications. It records every execution step (prompts, responses, tool calls, latency, cost), so you can precisely see why the agent made a decision, and where it went wrong.
This approach also shifted team priorities: it’s less about getting the latest model, more about measuring the agent’s concrete weekly impact.
This solid foundation brought a new possibility: allowing the customer service team to directly build their own tools. With Cockpit, support advisors describe their needs to a coding agent, which delivers new features in less than two hours. Real-world feedback is incorporated instantly, co-creating features that meet actual team needs. In two weeks, 200 new features launched, and support productivity jumped by 40%.
Roch Molléro – My Model Is Allowed to Fail. My Robot Is Not
While a software agent’s mistake can often be fixed, reliability is much more critical once AI acts in the real world. This is where Roch Molléro, CTO at Gobano Robotics, operates, working on “Physical AI”: robots that learn to manipulate real-world objects, including soft materials like textiles.
After outlining major milestones (reinforcement learning, diffusion policies, vision-language-action models), he explained their approach. Reinforcement learning means letting the robot try, fail, and adjust its behavior based on results, across thousands of attempts. Doing this on a physical robot is far too slow, and current simulators struggle to mimic how, say, fabric folds or crumples.
Instead, Gobano collects real-world data, then trains a “world model”: a simulator that learns to predict how the world reacts to a robot’s motions. This learned simulator becomes the training ground, where robots can make large-scale attempts before acting for real.
In industry, reliability beats speed. Even with 99% success, a robot performing hundreds of actions fails regularly. Rather than seeking perfect models, Gobano assumes failure will happen, and surrounds it with a multi-level recovery harness:
- A timeout aborts actions that drag on too long.
- Anomaly detection spots unusual situations.
- A supervising LLM attempts to debug edge cases.
- A human steps in as a last resort, when none of these mechanisms is enough.
Result: robots run autonomously overnight in two-hour blocks, with very few failures.
Key takeaway: at Brevo as at Gobano, reliability doesn’t come from a better model but from everything around it. Decision trees, observability, recovery mechanisms: this framework is what takes over when the model gets it wrong.
3. Keeping Control: Measuring and Securing Your Agents
Rolling out an agent is easy. Knowing whether it’s working well, and making sure it does only what you expect, is far harder.
Tristan Fransen – I Can't Tell if My Agents Are Performing Well
Senior Machine Learning Engineer at Pennylane, Tristan Fransen works on agents handling about 40,000 requests per day: reading invoices, extracting VAT, matching invoices to transactions. His key insight: very few teams track regular performance indicators for their agents, even though that’s the only way to know if things are improving or degrading over time.
User feedback is too sparse: only about one response for every 400 messages, mostly negative. So Pennylane focuses on “weak signals,” which are more abundant:
- a user rephrasing their request three times is frustrated;
- a click on ‘copy’ means the answer was useful.
All traces, whether local, sandbox or production, are stored in the same table. This simple choice matters: analysis tools work everywhere, and production behaviors can be directly compared to development. These traces are analyzed by micro-evaluators: small models, each checking just one criterion: is the extracted VAT correct; did it match the right transaction; does the tone fit? Rather than one big judge, this delivers more reliable, cost-effective scores, and helps pinpoint exactly what aspect of the agent is degrading.
One problem remains: evaluation models have biases, too. To minimize this, several different models score each trace, like an “LLM jury.” When they agree, you can trust the verdict. When they disagree, the team reviews manually. These human annotations are then used to regularly recalibrate the jury, keeping scores aligned with what the team actually considers a good answer.
Antoine Brossault – My AI Agents Are the New Attack Vector
Solutions Engineer at Cloudflare, Antoine Brossault explained how they equipped their entire company with AI agents in less than a year, from developers to sales. Such broad deployment raises a key governance question: who uses which model, with which data and at what cost? Instead of handing out API keys to every user, at the risk of uncontrolled sprawl, Cloudflare routes all traffic through a single AI Gateway. This centralizes keys, logs, and costs, broken down by user, team, and model. The same logic applies for agent tools: an MCP Portal distributes MCP servers centrally, with Zero Trust authentication and least-privilege access: each agent gets only the tools it needs.
Widespread agent use also increases the attack surface. Antoine illustrated this with “Bob in Sales”: Bob isn’t technical, but now has an agent that can act on his behalf, and could, by mistake, publish sensitive customer data without realizing it. Another, subtler risk: prompt injection. An agent that browses the web may encounter hidden instructions on a page, executing them as if they came from its user.
To address these risks, Cloudflare applies several principles:
- Agents in the cloud instead of local machines, so a compromised agent can’t access employee files or credentials.
- Isolated sandboxes to run generated code, preventing compromised agents from exfiltrating data to external servers.
- Gatekeepers that grant agents narrow, task-specific resource access instead of broad, ongoing privileges.
Key takeaway: Pennylane and Cloudflare offer different angles, but share a core belief: an agent in production should never be a black box. Knowing what it does, how well it’s performing, and what it can access is essential for safe, large-scale deployment.
Conclusion
Across every talk, the message was clear: the performance of an AI system in production depends less on the model, and more on the ecosystem built around it. Review processes, harnesses, evaluation, security: these are the enduring foundations, even as models change.
And you, where are you with your AI projects? Whether you want to explore your first use case or enhance an existing tool, contact us to connect.


