Autonomous Research Engines: The ISA Artifacts Pipeline

Most "AI research assistants" are chat loops with a search tool bolted on. They work when a human is present to steer, retry, and sanity-check. They fail — quietly and expensively — the moment you ask them to run unattended: to accept a stream of research requests overnight, work through them with bounded concurrency, survive process crashes mid-task, and produce artifacts a professional can cite without re-verifying every sentence.

We built the ISA artifacts engine because we needed exactly that: a system that ingests research intents (a question, a scope, a required output shape), executes them autonomously, and emits durable artifacts — literature-grounded briefs, comparative analyses, eligibility assessments — whose every claim is traceable to retrieved evidence. This page describes the architecture and the reasoning behind it. The core insight is unglamorous: autonomous research is 20% model capability and 80% workflow engineering. The model was never the hard part. Durability, idempotency, grounding discipline, and calibrated self-assessment were.

Research as a queue, not a conversation

The first design decision was to abandon the request/response framing entirely. A research task worth automating takes minutes to hours: multiple retrieval rounds, several synthesis passes, verification. Holding that inside an HTTP request or a chat session couples the work's lifetime to a connection's lifetime, which is how you get half-finished analyses and no record of why.

Instead, every research request becomes an intent — a persistent record capturing the question, constraints (source classes permitted, recency windows, output schema), priority, and provenance (who asked, under what authority). Intents land in a queue and the requester immediately gets a handle, not an answer. Workers — autonomous engine instances — pull from the queue and drive intents through a state machine: queued → claimed → in-pipeline (staged) → review → published or failed-with-diagnosis.

This framing buys three things that chat cannot:

• Durability. An intent survives worker crashes, deploys, and restarts. The question is never lost; at worst, work on it is repeated from the last checkpoint.
• Backpressure and prioritization. When demand exceeds capacity, the queue makes the trade-off explicit and policy-driven rather than letting requests silently time out.
• Auditability. Every state transition is recorded. Months later you can reconstruct not just what the engine concluded, but what it retrieved, what it discarded, and why.

Concurrency-safe claiming

Run more than one worker against a shared queue and you meet the classic distributed-systems problem: two workers grabbing the same intent, doubling cost and — worse — publishing two subtly different artifacts for one question. The failure is more damaging in research workloads than in ordinary job processing, because retrieval is time-sensitive: two runs hours apart can ground on different evidence and reach different conclusions, and now your artifact store contradicts itself.

Our claiming discipline is lease-based, a pattern well established in job-queue literature that we adapted for long-running cognitive work:

• A worker claims an intent atomically — the claim either succeeds entirely or fails entirely; there is no observable intermediate state in which two workers both believe they own the work.
• A claim is a lease with an expiry, not permanent ownership. Long-running pipelines renew the lease at stage boundaries. A worker that dies mid-pipeline simply stops renewing; the lease lapses and the intent becomes claimable again.
• Every pipeline stage is idempotent against checkpointed state. A reclaimed intent resumes from its last completed stage rather than restarting, so at-least-once execution semantics never become at-least-once publication semantics. Publication itself is guarded by a final atomic transition, giving exactly-once artifact emission on top of at-least-once processing.

The subtle design point is lease duration. Too short, and a slow-but-healthy synthesis stage gets its work stolen mid-flight; too long, and a dead worker blocks an intent for hours. We resolved this by making lease renewal a stage-boundary obligation rather than a timer: the pipeline's own progress is the heartbeat. A worker that stops progressing stops renewing, definitionally.

The pipeline: perceive, plan, retrieve, synthesize, reflect

Inside a claim, the engine runs a staged cognitive pipeline. It descends from the same brain-engine loop that drives our production personas (PERCEIVE→REASON→PLAN→ACT→REFLECT), specialized for research work:

 intent
   │
   ▼
 PERCEIVE ── parse the question; classify domain, scope,
   │         required artifact shape; detect ambiguity
   ▼
 PLAN ────── decompose into sub-questions; choose source
   │         classes; budget retrieval rounds and cost
   ▼
 RETRIEVE ── multi-round evidence gathering; dedupe;
   │         relevance filtering; build the evidence set
   ▼                                (checkpoint: evidence frozen)
 SYNTHESIZE─ compose the artifact under citation
   │         restriction — claims bind to evidence
   ▼
 REFLECT ─── verify claim/evidence bindings; score
   │         coverage & confidence; decide publish /
   ▼         re-plan / abstain
 artifact + confidence report

Two properties matter more than the stage names, which any agent framework can imitate.

First, stages are checkpointed contracts, not prompt phases. Each stage consumes a typed input, produces a typed, persisted output, and can be re-executed in isolation. This is what makes lease reclamation safe, but it also makes the pipeline debuggable: when an artifact is wrong, we can pinpoint whether perception misread scope, planning under-decomposed, retrieval missed a source class, or synthesis overreached its evidence. In an undifferentiated agent loop, all four failures look identical — "the model was wrong."

Second, the evidence set is frozen before synthesis begins. Retrieval and synthesis are deliberately separated by a hard checkpoint. The synthesizer cannot go fetch "one more source" mid-composition. This feels restrictive — sometimes it is restrictive, and REFLECT compensates by looping back to PLAN with an explicit gap diagnosis — but it is the foundation of the grounding guarantee described next. A synthesizer that can extend its own evidence set mid-flight can also launder a hallucination into a "retrieved fact," and no post-hoc audit can cleanly distinguish the two.

Citation-restricted synthesis

The central discipline of the engine is closed-world synthesis: the model composing the artifact may only assert what the frozen evidence set supports, and every substantive claim must bind to specific evidence items. Anything the model "knows" from pretraining but cannot ground is either omitted or explicitly marked as background context, never presented as a finding.

Mechanically (described at the architecture level — the implementation is proprietary and patent-pending):

• Synthesis operates over an enumerated evidence set with stable identifiers. The output format requires claim-to-evidence bindings inline, so grounding is produced during composition, not reconstructed afterward.
• A separate verification pass — distinct model invocation, adversarial framing — checks each binding: does the cited evidence actually entail the claim, or merely mention the same nouns? Bindings that fail entailment checking are demoted or the claim is struck. Verification is deliberately asymmetric: it can only remove or weaken claims, never add them, so the check cannot itself introduce ungrounded content.
• Claims with no surviving binding trigger one of two outcomes: the claim is cut, or REFLECT routes the intent back to PLAN with the unsupported claim recast as a retrieval gap. Abstention is a first-class outcome. An artifact that says "insufficient evidence within scope" is a success state, not a failure state.

This is more expensive than free composition — typically several times the token cost of a naive single-pass answer — and it produces artifacts that read as more cautious than a fluent chat response. We consider both costs well spent. The relevant comparison for a research artifact is not "how impressive does it sound" but "how much human verification does it need before someone acts on it." Citation restriction moves verification cost from every reader to one automated pass.

We pair this with deterministic, versioned scoring engines for the evaluative portions of artifacts (eligibility determinations, threshold checks, fit scores). Where a judgment can be computed from rules, it is computed — reproducibly, from a versioned rule set — and the language model's role is confined to evidence extraction feeding those rules. AI passes are evidence-gated: the model may propose, but only grounded evidence admitted through the gate may influence a score. Determinism where determinism is possible; grounded generation only where it is not.

Honest confidence

Most systems emit confidence as decoration — a vibe expressed as a percentage. We treat confidence as a contract about downstream handling, assembled from measurable components rather than asked of the model directly:

• Coverage: what fraction of planned sub-questions produced grounded findings; which source classes were reachable versus silent.
• Binding strength: the distribution of entailment-check outcomes across the artifact's claims — an artifact whose claims all survived adversarial verification is categorically different from one where a third were demoted.
• Evidence agreement: whether independent evidence items corroborate or conflict, with conflicts surfaced in the artifact rather than averaged away.
• Staleness and scope caveats: explicit statements of what was not searched and what may have changed since retrieval.

The composite maps to routing, which is what makes it honest: high-confidence artifacts publish autonomously; mid-band artifacts publish flagged for human review; low-confidence runs abstain and report their gap diagnosis. A confidence signal that never changes anyone's behavior is theater. Ours decides whether a human is summoned — so miscalibration has an immediate, visible operational cost, which in turn is what keeps pressure on calibration. When flagged artifacts sail through review repeatedly, thresholds are loosened; when autonomous publications draw corrections, they tighten. Calibration becomes a feedback loop over review outcomes, not a one-time tuning exercise.

Failure modes, candidly

Running this in production has taught us where autonomous research engines actually break:

• Poison intents. An ambiguous or internally contradictory question can loop PLAN→RETRIEVE→REFLECT indefinitely, each pass diagnosing a "gap" no retrieval can fill. We bound re-plan cycles and force abstention-with-diagnosis rather than letting a worker burn budget. The diagnosis often reveals the intent needed a human conversation before it needed an engine.
• Retrieval drift. The same intent run a month apart yields different evidence and possibly different conclusions. This is not a bug — the world changed — but it obligates artifact versioning: artifacts carry their evidence snapshot, and contradiction between an old artifact and a new run is surfaced explicitly rather than silently overwritten.
• Over-abstention. Strict entailment checking can strike claims a domain expert would accept, because evidence entails them only through a reasoning step the verifier won't take. Tuning verifier strictness is a genuine precision/recall trade-off with no free lunch; we bias toward abstention and accept the reduced yield, but we do not pretend the dial has a correct setting.
• Reflection echo. A REFLECT stage powered by a model similar to the synthesizer shares its blind spots — it grades fluent errors as sound. Structural checks (binding entailment, coverage accounting, deterministic scoring) carry more weight in our confidence composite than model self-assessment for exactly this reason.
• Queue pathologies. Priority schemes starve low-priority intents; retry policies turn one flaky source into a thundering herd. Ordinary queueing theory applies and is routinely ignored in agent systems; we stopped ignoring it after it bit us.

The engine's own memory

The engine is itself governed by the discipline it serves. Every change to pipeline stages, claiming semantics, verification thresholds, or scoring rules ships with a Codex entry — an executable blueprint/playbook/runbook triple in our institutional knowledge corpus (440+ entries and growing). The dependency and knowledge graphs at ticket.lucaexpress.com let us trace, for any published artifact, the engine version, scoring-rule version, and operational doctrine in force when it was produced. For a system whose product is trustworthy claims, the ability to reconstruct its own state at publication time is not bookkeeping — it is part of the trust argument.

Where this is heading

Three directions occupy us now. First, fine-tuned stage specialists: through Model Forge we are moving individual pipeline stages — entailment verification and evidence extraction especially — onto small fine-tuned models, trading generalist breadth for calibrated, cheap, fast judgment on narrow contracts; the staged architecture makes each stage independently swappable and independently evaluable. Second, cross-artifact reasoning: treating the published artifact corpus as an evidence class of its own, with contradiction detection between an engine's past and present conclusions promoted to a first-class signal. Third, richer intent contracts: letting requesters specify not just questions but verification budgets and abstention preferences, making the cost/caution trade-off a caller-visible parameter instead of an engine constant.

The pattern generalizes well beyond research artifacts: any workload of the form "unattended cognitive work whose outputs must be trusted" — compliance review, due-diligence screening, technical monitoring — fits the intent-queue, staged-pipeline, evidence-gated shape.

We take on external research engagements in this space — engine architecture reviews, grounding and calibration design, and autonomous-workflow builds. If your team is fighting these problems, talk to us.