Blog

Beyond the chat window: AI that works inside your research environment

· 8 min read · 75bit

Access to AI is no longer the scarce thing. Anyone on a research team can open a chat window, paste in a problem and get a fluent answer in seconds. Coding agents go further: they read a repository, run commands and edit files. Most teams now use some of this, and many are quietly unsure what it is actually doing for them.

We think the interesting question has moved. It is no longer whether a team can use AI. It is whether the AI can work inside the team's real environment, with its data, its tools and its rules, in a way that is repeatable and that someone can audit.

Adoption is high. Trust is not.

Stack Overflow's 2025 developer survey found that 84% of respondents were using or planning to use AI tools, up from 76% the year before. In the same survey, more developers said they distrust the accuracy of AI output (46%) than said they trust it. The coverage disagrees on the exact trust figure, so we quote only the distrust number here. The pattern is clear enough: widespread use, uneasy confidence.

Using AI tools, and distrusting their accuracy

Share of survey respondents, 2025

  • Using or planning to use AI tools84%
  • Distrust the accuracy of AI output46%
Stack Overflow, 2025 Developer Survey results. Figures as reported in coverage of the survey; the two shares may have different bases.

Faster is not the same as better

The evidence on productivity is mixed, and worth reading closely. In 2025, METR ran a randomised trial with 16 experienced open-source developers across 246 real tasks in repositories they already knew well. When AI tools were allowed, they took 19% longer to finish. Before starting, they had expected to be 24% faster. Afterwards, they still believed they had been about 20% faster.

Developers' expectations against the measured result

Speed with AI tools
Expected before the tasks24% faster
Believed afterwardsAbout 20% faster
Measured by the trial19% slower
METR, Measuring the impact of early-2025 AI on experienced open-source developer productivity. 16 developers, 246 tasks.

Two cautions. METR itself says this is one setting and a snapshot of early-2025 tools, so it does not settle the question for every team or for today's tools. And it measured coding, not research. But it does show something that matters for any organisation: how productive an AI tool feels is not a reliable measure of how productive it is. You need a way to check.

Why a chat is a weak system

A chat is an excellent interface and a thin system. It knows only the text you paste into it. It cannot see your data, run your code or check an answer against your baseline. It does not remember how your team works. And it leaves no record: when a result matters six months later, there is no log of what was asked, what was run or who approved it.

That is fine for a quick question. It is the wrong foundation for research that an enterprise has to rely on.

What the coding agents got right

Tools such as Claude Code and Codex work because they are more than a conversation. They sit inside the environment, with access to files, a terminal and tests. They act, check the result and try again, within permissions a person has set. Anthropic's guidance on building effective agents draws a useful line between workflows, where models and tools follow paths defined in code, and agents, where the model directs its own process. It also advises starting with the simplest approach that works and adding autonomy only when it earns its place.

That advice is the heart of how we approach research. Most research tasks are better served by a well-designed workflow with a model at the right steps than by an open-ended agent left to improvise.

One question, or a workflow that runs inside your environment

Single chat

  1. Paste context into a window
  2. Model answers from the text it was given
  3. A person copies results back by hand
  4. Nothing is recorded or repeatable

Integrated workflow

  1. Runs in your environment, on pinned data
  2. Calls tools, runs code and tests
  3. Checks results against a baseline
  4. Pauses for human approval, logs every step
Illustrative comparison of the two approaches. Conceptual, not measured.

What a research workflow needs on top

A coding agent checks its work with tests. Research needs its own equivalents. These are the pieces we build so that AI can be trusted inside a research environment:

The workflow needsWhy it matters
Pinned data and environmentsA run can be reproduced exactly, so results are comparable (see our post on reproducibility)
Baselines and checksThe workflow tests its own output against known-good results instead of asking you to take its word
Scoped tools and permissionsIt can reach the data and systems it needs and nothing more
Human approval gatesPeople decide at the points that matter: methods, data changes, anything irreversible
A complete run logEvery prompt, tool call, input version and output is recorded, so any result can be audited
Cost and quality trackingYou can see what a workflow spends and whether it is actually helping

Where it goes wrong, and what stops it

  • Plausible but wrong results. Checking against baselines and held-out cases catches errors that read well.
  • Silent changes to data or tests. Read-only access by default, with changes proposed for review rather than applied.
  • Runs nobody can repeat. Pinned inputs and a full log make every run reproducible.
  • Leaking sensitive data. The workflow runs inside your environment, under your access rules, rather than sending data into a shared chat.

None of this is exotic. It is the same discipline as good research engineering, applied to a new kind of collaborator.

What we build

We do not build another agent. Your teams already use coding agents, and they will keep changing them. What those tools lack is a shared place to put what the work produces. So we built Cairn.

Cairn is a layer that works with the agents you already use. Claude Code and Codex both read from it and write back to it, and it holds the work of all your company projects in one store. The goal is simple: understand what a piece of research has produced, keep it in one place, and let other tools use the final result and the knowledge behind it.

A preview of the Cairn home screen. Illustrative, with sample data. Click to enlarge.
  • Works with your agents. Claude Code and Codex use the same store, so one tool can build on what the other found.
  • Understands the work. Captures findings, decisions, data and environment versions, and run logs as the work happens, instead of leaving them in a chat that scrolls away.
  • One store for every project. A single, searchable record across the company that belongs to your organisation and sits under your access rules.
  • Lets other tools use the result. The final result and the knowledge behind it are available to the next agent, tool or person, so nobody starts from a blank prompt.

We start narrow: one team, one workflow, one measurable outcome. We make it reproducible and auditable, put approval gates where your people want them, and measure whether it helps before widening the scope.

If your team is using AI in chat windows today and wants what it learns to stay useful, we would be glad to talk.

Sources