Summary
Why ad-hoc AI research breaks at scale
Ad-hoc AI research looks productive until you try to repeat it. A researcher opens a chat interface, asks a series of questions, copies the outputs into a document, and edits them into a deliverable. The process works once. The second time, a different researcher asks slightly different questions, gets different outputs, and produces a deliverable that is inconsistent with the first. By the tenth iteration, no one can explain why two research reports on the same topic reached different conclusions, because the process that produced them was never recorded.
The core problem is that ad-hoc AI research has no structure. There is no defined sequence of steps, no consistent set of instructions for the AI, no record of what the AI was asked or what it returned, and no human review gate that catches errors before they propagate into the final output. Every run is a one-off, and one-offs cannot be improved systematically.
Structured AI research workflows solve this by making the process explicit. Each research step is a named, versioned agent with a specific purpose and a locked system prompt. The sequence of steps is a workflow graph that runs the same way every time. Every AI call and every human decision is recorded in the audit trail. When the output is wrong, you can trace exactly which step produced the error and fix it without touching the rest of the workflow.
DAVE is built for this kind of structured AI work. The agent system gives you reusable, versioned AI components. The workflow engine sequences them reliably. The human review gates let you put a person in the loop at exactly the right moments. And the audit trail records everything, so the process is reproducible and improvable.
Designing agents as research specialists
The foundation of a structured research workflow is a set of agents, each designed for a specific research function. In DAVE, an agent is a named, reusable AI component that wraps a provider configuration (system prompt, model, temperature, and other parameters) in a versioned, immutable snapshot. When you reference an agent in a workflow, you select a specific version. That version's configuration is used every time the workflow runs, producing consistent behavior across all instances.
For a research and synthesis workflow, the natural agent specializations are:
- Source Summarizer. Takes a raw source document or URL content as input and produces a structured summary: key claims, supporting evidence, and source metadata. System prompt is tightly scoped to extraction and summary, with a low temperature setting (around 0.2 to 0.3) to minimize hallucination and maximize fidelity to the source material.
- Claim Extractor. Takes a set of summaries and identifies discrete, attributable claims. Output is a structured list of claims, each linked to its source summary. This agent should be configured with a model that handles long context well, since it may receive multiple summaries at once.
- Synthesis Writer. Takes the extracted claims and produces a coherent narrative synthesis. This agent can use a higher temperature setting (around 0.5 to 0.7) since creative coherence matters more than strict extraction at this stage. The system prompt should specify the output format, target length, and any style or tone requirements.
- Consistency Checker. Takes the synthesis draft and checks it against the extracted claims for factual consistency. Flags any statement in the synthesis that is not supported by an extracted claim. This agent acts as a pre-human-review quality gate, reducing the burden on the human reviewer.
Each of these is a separate agent in DAVE, with its own name, description, tags, and version history. When you refine the Synthesis Writer's system prompt, you create a new version of that agent. The Source Summarizer and Claim Extractor are unaffected. You can test the new Synthesis Writer version in isolation before deploying it to the production workflow.
The Config JSON for a Source Summarizer agent might look like this:
{"systemPrompt": "You are a research source summarizer. Given the text of a source document, extract: (1) the main thesis or argument, (2) the key supporting claims, (3) any quantitative evidence cited, and (4) the source type and apparent credibility signals. Format your output as structured JSON with keys: thesis, claims (array), evidence (array), sourceType, credibilityNotes.", "model": "gpt-4", "temperature": 0.2, "maxTokens": 2048}Naming agents by function rather than by model is a deliberate best practice. "Source Summarizer" tells a workflow designer what the agent does. "GPT-4 Agent" tells them nothing useful and becomes misleading the moment you switch models in a new version.
Using the Prompt Library to standardize research instructions
DAVE's Prompt Library is a collection of reusable prompt templates that can be created, managed, and assigned to agents. For a research team, the Prompt Library is where institutional knowledge about how to prompt AI for research tasks lives. Instead of each researcher maintaining their own private prompt collection, the team maintains a shared library of tested, categorized prompts that can be assigned to agents and locked.
Locking a prompt to an agent is a governance decision. When a library prompt is locked to an agent, the agent always uses that prompt when invoked in any workflow. In the workflow editor, the prompt selector for that agent is greyed out: workflow designers cannot override it. This means the research team's carefully tested summarization instructions cannot be accidentally replaced by a workflow designer who thinks they have a better prompt. The prompt is the agent's behavior specification, and locking it makes that specification stable.
The Prompt Library supports six categories: Content Generation, Review, Safety, Analysis, Transformation, and Custom. For a research workflow, the natural category assignments are:
- Source Summarizer prompt: Analysis
- Claim Extractor prompt: Analysis
- Synthesis Writer prompt: Content Generation
- Consistency Checker prompt: Review
Tags on prompts work the same way as tags on agents: they are comma-separated labels for filtering and organization. A consistent tagging scheme across both agents and prompts (for example, using research, synthesis, review, extraction) makes it easy to find related components when building or auditing a research workflow.
DAVE also offers AI-assisted prompt generation in the Prompt Library. When creating or editing a prompt, click AI Assistant, enter a narrative description of what you want the prompt to accomplish, and click Generate. The system uses your tenant's default AI provider and model to produce a structured prompt. This is useful for drafting initial versions of research prompts, which you then refine and test before locking to an agent.
Structuring the research workflow graph
With agents defined and prompts locked, the research workflow graph is the sequence that connects them. A well-designed research workflow for a typical synthesis task has five stages:
- Input collection (User Interaction node). The workflow starts by collecting the research brief from the user: the topic, the sources to analyze, the output format, and any specific constraints. The User Interaction node presents a structured form to the person initiating the research run. The submitted data becomes the workflow's initial context, passed forward to every subsequent node.
- Source summarization (Agent Interaction nodes). One or more Agent Interaction nodes call the Source Summarizer agent, one per source. If the workflow is designed to handle a variable number of sources, a Routing node can branch the workflow into parallel summarization paths and then merge the outputs before the next stage.
- Claim extraction (Agent Interaction node). A single Agent Interaction node calls the Claim Extractor agent, passing it the combined summaries from the previous stage. The output is a structured list of attributable claims.
- Synthesis and consistency check (Agent Interaction nodes). Two sequential Agent Interaction nodes: first the Synthesis Writer produces a draft, then the Consistency Checker reviews it against the extracted claims and flags any unsupported statements.
- Human review gate (Human Review node). A Human Review node presents the synthesis draft, the consistency check report, and the original extracted claims to a designated reviewer. The reviewer has three options: Approve (the synthesis is accepted and the workflow proceeds to output delivery), Reject (the synthesis is discarded and the workflow ends or loops back), or Request Changes (the reviewer's comment is passed back to the Synthesis Writer for revision). The decision is recorded in the audit trail with the reviewer's identity, timestamp, and any comment they provided.
This five-stage structure is a starting point, not a prescription. A research workflow for a short-form briefing note might collapse stages 2 and 3 into a single agent. A workflow for a long-form research report might add a second human review gate after the claim extraction stage, before synthesis begins. The workflow graph is yours to design: the node types and agent system give you the building blocks.
The key design principle is that every AI step is an Agent Interaction node referencing a specific, versioned agent. No AI configuration is embedded inline in the workflow graph. When you need to change how the Synthesis Writer behaves, you create a new agent version and update the workflow to reference it. The change is tracked in the agent's version history, and the previous behavior is preserved in the retired version.
Governance, version control, and the audit trail
A structured research workflow is not just more reliable than ad-hoc prompting: it is auditable. Every element of the process is recorded in DAVE's audit trail.
At the agent level, every create, update, and version change is logged. When you create a new version of the Synthesis Writer agent, the audit trail records who created it, when, and what the new configuration contains. When you retire an old version, the retirement is logged. If a research output is later questioned, you can trace exactly which version of each agent was used in the workflow instance that produced it.
At the workflow instance level, every node execution is recorded: which agent version was called, what context was passed to it, and what it returned. The human review gate records the reviewer's identity, their decision (Approve, Reject, or Request Changes), and any comment they provided. This is a complete chain of custody for the research output, from the initial brief to the final approved synthesis.
When SOC 2 or GDPR compliance modes are enabled on the tenant, additional read-access logging is applied. Agent configuration reads are logged as security events. Prompt injection detection, which scans every assembled prompt before it is sent to the AI provider, is logged under SOC 2 CC7.2 (anomaly monitoring). Agent version changes are tracked as configuration changes under SOC 2 CC8.1 (change management).
For research teams operating in regulated industries, this audit trail is not a nice-to-have: it is the difference between a research process that can be evidenced and one that cannot. The ability to show an auditor exactly what AI was asked, what it returned, and who approved the output before it was used is a significant governance advantage over ad-hoc AI research.
Version control at the agent level also supports continuous improvement. When the research team identifies a pattern of errors in synthesis outputs (for example, the Synthesis Writer consistently overstates the strength of evidence), they can create a new agent version with a revised system prompt, run a set of test instances, compare the outputs, and deploy the new version to production. The old version remains available for comparison. The improvement is documented in the version history. The process gets better over time, and the improvement is traceable.
Team roles for a research workflow
DAVE's six default roles map cleanly onto the different functions in a research workflow team. Assigning roles correctly ensures that each team member can do their job without having access to configuration they do not need.
- Admin. Sets up AI providers, configures tenant settings, manages users and roles. One or two people on the team. Does not need to build workflows or run research instances day-to-day.
- Create. Builds and maintains agents, creates new agent versions, manages the Prompt Library. The research team's AI specialists: the people who design and refine the agent configurations that power the research workflow. Create role users can view agents and prompts but cannot build or edit workflow graphs.
- Curate. Builds and edits workflow graphs in the Workflow Editor. References agents created by the Create role. Can view agents and the Prompt Library but cannot create or modify them. The workflow designers on the team.
- Use. Initiates workflow instances (submitting the research brief at the User Interaction node) and completes tasks (acting as the human reviewer at the Human Review node). The researchers and reviewers who use the workflow day-to-day. Cannot see agent configurations or workflow graphs.
- Reporting. Views analytics and audit logs. Useful for a research team lead or compliance officer who needs to monitor workflow performance and review the audit trail without having operational access.
- Financial. Manages billing and plan details. Typically assigned to one person on the team with budget responsibility.
A single user can hold multiple roles simultaneously. A senior researcher who both designs workflows and runs research instances can hold both the Curate and Use roles on a single seat. Multiple roles do not require multiple seats.
Role access to agents is worth noting specifically: the Use role cannot access the Agents section at all. Researchers running workflow instances never see the agent configurations that power the workflow. They interact only with the User Interaction node's input form and the Task Inbox for human review decisions. This separation between workflow design and workflow use is a deliberate governance feature, not a limitation.