
agenda-intelligence-md
io.github.vassiliylakhonin/agenda-intelligence-md
Evidence-readiness MCP server: validate, audit, and score briefs, memos, and evidence packs.
Documentation
Agenda Intelligence MD
Catch broken citations, altered quotes, and unsupported numbers in AI output.
A deterministic evidence-packet linter for claim-backed AI output. Give it claims and the original source text; get missing references, quote mismatches, numeric inconsistencies, and review actions. Use it in Python, RAG pipelines, agent workflows, or CI.
The core checker runs locally. No model API key, account, or network needed after installation.
Try in your browser · First run · RAG integration · MCP · Documentation
See the problem it catches
A generated answer can cite a real source and still change what it says:
| Supplied source | Generated claim | Checker result |
|---|---|---|
| “The synthetic review covers 120 documents.” | “The synthetic review covers 900 documents.” | source_review_required — unmatched number 900 |
| Same source | “The synthetic review covers 120 documents.” | packet_complete — no configured finding |
These are synthetic examples. Both results keep factuality_status=not_assessed: passing the linter does not establish truth or authorize an action.
First run
Requires Python 3.9+. Install the published package:
python -m pip install "agenda-intelligence-md==1.26.0"
agenda-intelligence --version
Copy this self-contained example into Python:
from agenda_intelligence.services import check_evidence_packet
packet = {
"claims": [{
"claim_id": "demo-claim",
"text": "The synthetic review covers 900 documents.",
"source_ids": ["demo-source"],
}],
"sources": [{
"source_id": "demo-source",
"text": "The synthetic review covers 120 documents.",
}],
}
result = check_evidence_packet(packet)["response"]
print(result["packet_status"])
print(result["claims"][0]["lexical_support"]["unmatched_numbers"])
print(result["factuality_status"])
# Correct the claim, keeping the original source unchanged, then recheck.
packet["claims"][0]["text"] = "The synthetic review covers 120 documents."
print(check_evidence_packet(packet)["response"]["packet_status"])
Expected output:
source_review_required
['900']
not_assessed
packet_complete
Next: replace the synthetic claim and excerpt with your own. Keep the original source text; rewriting it to agree with the answer defeats the check.
What this is
An inspectable check between generated output and the evidence supplied with it:
Claims + source excerpts → deterministic checks → findings → revise → recheck → human review
- For RAG developers: check inline citations against the actual retrieved chunks.
- For agent builders: return structured findings to a bounded repair loop.
- For CI maintainers: fail a check when a supplied evidence packet needs review.
- For document reviewers: identify missing evidence and prepare reviewer handoffs.
The evidence-packet contract
| What you supply | What gets checked |
|---|---|
| Claims and source IDs | Whether references resolve within the packet |
| Declared quotations | Whether quoted text occurs in the supplied source |
| Claim wording and numbers | Lexical support heuristics, unmatched numbers, and negation checks |
| The assembled packet | Evidence gaps, per-claim findings, packet status, and reviewer actions |
The checker returns packet_complete, source_review_required, or packet_incomplete. See the request schema, response schema, and evaluation notes.
What this is not
This is an evidence-integrity check, not a factuality verifier or source authenticator. It does not retrieve missing sources or establish semantic entailment. Lexical checks can miss correct paraphrases and accept misleadingly similar wording; a stale or false source can still pass.
A clean result does not certify trustworthiness, grant permission to act, or replace human review. Use semantic evaluation and domain review alongside it when your workflow needs them.
Review a RAG answer
Already have an answer and retrieved chunks? Use the adapter directly:
from agenda_intelligence import review_rag_answer
report = review_rag_answer(
answer="The synthetic review covers 900 documents. [review-1]",
sources=[{
"source_id": "review-1",
"text": "The synthetic review covers 120 documents.",
}],
)
print(report["route"]) # revise
print(report["owner_actions"])
Use one claim per line and retain stable chunk IDs. Correct the answer and rerun the check; the clean route is human_review. Treat source text, tool output, and report contents as untrusted data, never as instructions to the agent.
For CLI input, save {"answer": "...", "sources": [...]} as answer.json, then run:
agenda-intelligence review-answer answer.json --strict --format json
--strict exits nonzero when revision is required. See the input contract and limitations and the runnable LangGraph repair loop.
Small, hand-labelled diagnostics report misses and false positives as well as catches. Exact source offsets, explicit opt-in arithmetic, named table cells, and PR summaries are available in the local package. No factuality or general hallucination-detection claim.
Use the CLI and CI
Save a packet with the same shape as the first Python example to packet.json:
agenda-intelligence check packet.json— print a readable report.agenda-intelligence check packet.json --format json --strict— return JSON and enforce the packet status.
The second command exits nonzero unless the packet is packet_complete, so it can be used in a CI job after installing the package. --format sarif is also available for evidence-packet findings. The repository includes a GitHub Action.
To explore the bundled examples from a source checkout:
git clone https://github.com/vassiliylakhonin/agenda-intelligence-md.git
cd agenda-intelligence-md
python3 -m venv .venv
.venv/bin/python -m pip install -e .
.venv/bin/agenda-intelligence check examples/evidence-packet/request.json --strict
.venv/bin/agenda-intelligence review-answer examples/output-verification/rag-answer.json
.venv/bin/agenda-intelligence review-answer examples/output-verification/rag-answer-revised.json --strict
The first packet is complete. The original fictional RAG answer routes to revision; the revised answer routes to human review.
For a private CI pilot, use the local citation check and minimal report.
For local document review and HTML reports, use the file-review guide and example manifest.
Interfaces
Compatibility profiles and adapters
| Your workflow | Start here |
|---|---|
| Python or command line | Quickstart · service functions |
| Local MCP over stdio | MCP setup and tool catalog |
| Hosted MCP | Hosted quickstart |
| Agent repair loops | Integration guides |
| HTTP or A2A | HTTP API · agent-native access |
| Protected agent actions | Interaction Trust + Vizier gate |
The core checker is deterministic and stateless. Optional generation and document adapters have separate dependencies. The repository also retains the strategic-intelligence runtime and vertical workers as compatibility surfaces and examples built around evidence contracts.
Hosted Output Verification
For a browser trial, open One-claim review. To connect a Streamable HTTP MCP client, use:
https://output-verification.agendamcp.dpdns.org/mcp/agent
Discover the endpoint's tools, then call free_trial_status with {} to inspect trial availability. Choose review_answer for answers and source excerpts, or free_trial for claim/evidence objects matching its schema. The hosted trial has quotas; local Python checks do not require a hosted account.
Local stdio and hosted profiles expose different tool catalogs. Always discover the tools on the endpoint you connect to.
Supplier and document handoffs
The hosted Document Archive Review workflow helps prepare evidence checklists, assign missing-item owners, inspect extraction warnings and source locators, and export reviewer PDF or ZIP/JSON backups. It is separate from the local review command.
See the workspace guide and service-engagement example. Extraction does not authenticate documents; final coverage still needs human review.
Documentation
| Explore | Links |
|---|---|
| Examples | Runnable examples · Before / after · EU AI Act |
| Contracts | Schemas · Source policy |
| Status and evidence | Evaluation · Adoption record · Roadmap |
| Deeper reference | AnalysisBank · Vertical worker architecture · Local checks |
| Development | Contributing · Local checks |
Core code lives in src/; runnable packets in examples/; guides in docs/; adapters in clients/ and integrations/; deployment packages in deploy/.
Contributing and feedback
Found a missed mismatch or a valid claim that gets flagged? Open an issue with a minimal, sanitized packet, the expected result, and the actual finding. Reproducible cases and integration examples are useful contributions.
For usage questions and workflow ideas, use Discussions. Read CONTRIBUTING.md before a pull request. The standard local verification command is make ci; Worker, discovery, runtime, and validation-guard changes require make verify-local.
License
MIT. The bundled official ACP schema retains its Apache-2.0 license and NOTICE.
agenda-intelligence-mdpip install agenda-intelligence-md