CorpusKit is the open source research portal for Progress Agentic RAG. It is licensed under Apache 2.0 and maintained by Noice, which also runs it as a hosted service, CorpusKit Cloud.
Search the collection, ask questions with sources you can inspect, and follow connections between documents. The tools below support that work, from reading a single report to administering a portal.
At a glance
The specifics behind the catalogue: what a portal takes in, how it answers, what it produces and what it runs on.
Content it takes in
PDF reports, including scanned and image-only pages read by a vision language model
Word, PowerPoint, Excel and OpenDocument files
Web pages, whole sites and sitemaps, re-synced on a schedule
Video and audio, transcribed and searchable
Pasted text as a resource of its own
Tables extracted as tables with the table-aware method
Uploads up to 100 MB per file, singly or in batches
Originals never altered; generated fields sit beside them
How it answers
Hybrid, semantic and keyword retrieval with a reranking pass
Neighbouring-paragraph or full-document grounding, chosen per question type
Knowledge graph relations in the retrieval loop
Deep research that maps a broad question into sub-questions
Answers stream as they are written, then the checked answer replaces the stream
Nothing from general knowledge or the internet; every sentence cites its passage
How it checks
Figures, populations, named studies, years and safety wording checked sentence by sentence
Plain-text comparison against the extracted documents, with no language model in the loop
Unsupported sentences removed, and the removal declared under the answer
Reference lists cut out before the check, so a bibliography cannot stand in for a finding
Confidence shown as high, moderate or low, led by the check
Platform quality scores can lower that label but never raise it
What comes out
Briefings, comparisons, timelines, FAQs and pros and cons, each with numbered references
Knowledge checks of 3, 5 or 10 questions at foundational, intermediate or advanced level
Word exports of Ask sessions, investigations and generated material
A print-ready view for saving a briefing as a PDF
Audit records as CSV or JSON, and enrichment records as JSON
Watches that re-run a saved search daily and flag new results
How it connects
An MCP server over Streamable HTTP with four read-only tools: search, answer, get document and browse catalogue
Bearer keys scoped to one portal, with a role, an optional expiry and instant revocation
Microsoft Entra sign-in with app roles and group mappings
Public, authenticated and restricted access modes
Six roles across portal and platform scopes, over 22 declared permissions
Custom hostnames per portal
How it looks
Five library palettes, one of them dark, or the portal’s own seeded colours
Six typeface pairings, or your own uploaded fonts
Square, rounded or soft shapes, three text sizes and four densities
Logo, hero and banner images per portal
Every palette passes an automated WCAG contrast check before it ships
Responsive from a phone to a wide display
What it runs on
Cloudflare Workers, with application state in SQLite-backed Durable Objects
One Progress Agentic RAG knowledge box per portal, with an Australian region option
Deno 2 and TypeScript throughout, with React on the front end
A typecheck, lint, format and test gate before any deploy
Knowledge box credentials held on the server and never sent to the browser
No mock mode: the portal always works against a live knowledge box
Apache 2.0 licence
Discovery and search
Explore the collection
Start with suggested questions and topic rows drawn from the portal’s own research collection.
Questions written from the documents themselves, not a static list
Topic rows built from the taxonomy the portal designed for your domain
Search three ways
Use hybrid, semantic or keyword search. Narrow results with facets and read the matched passages alongside them.
Hybrid, semantic or keyword, with a reranking pass
Facets for topics, document kinds and any label set you define
A short cited answer over the results, or the results alone
Save a search as a watch and be told when its results change
Question routing
Ask routes questions to stored search configurations suited to the task, such as an exact lookup or an evidence review. Administrators manage those configurations.
Rule and classifier stages pick the configuration; the choice shows as a chip you can change
Each configuration sets retrieval features, grounding strategy, graph use and prompt variant
Content sets such as documentation are kept apart in stored configurations, not per-request filters
Answers you can check
Cited conversations
Ask questions across the collection and follow up in the same conversation. Numbered citations connect the answer to its sources.
Stages shown as the answer streams: interpreting, retrieving, writing, checking
An “Interpreted as” line shows how the question was read
Follow-ups keep the context; deep research splits broad questions into sub-questions
Evidence checks
The trust layer checks claims against cited text, including figures and their context. It can remove unsupported statements or withhold an answer, and tells you when it does.
Figures, populations, named studies, years and safety wording, sentence by sentence
Plain-text comparison with no language model in the loop
A thinly grounded answer offers a one-tap deep re-answer over full documents
Visible confidence
See confidence information for sources and answers, including warnings when evidence is thin. These signals help you review an answer; they are not a guarantee of correctness.
High, moderate or low, led by the evidence check
Platform quality scores can lower the label but never raise it
Each source carries a relevance score, and a Supports, Partial or Not relevant verdict on request
Footnotes and passages
Footnote references are bound to retrieved source passages on the server. Open a citation to inspect its evidence and read the original document.
A citation opens the PDF at the cited page with the passage highlighted
“Also retrieved” keeps visible the passages the answer did not use
Reference lists are cut out before the check, so a bibliography cannot count as evidence
Knowledge graph and investigations
Entity relationships
Explore entities and relationships extracted from the research. Follow connections and filter the graph using the portal’s entity and relation types.
Entity and relation types chosen for your domain during provisioning
A legend to show or hide types, a most-connected list and find by name
Click a node for its evidence, or trace how two entities connect
Optionally include the platform’s built-in people, dates and places
Concept map
See connections between categories that share resources. This offers a different view of the collection from the graph of extracted entities.
Nodes are topics and document kinds; shared resources pull them closer
Pick a category to see what it pairs with
Investigations
Keep a named research question and build its evidence over time. Saved evidence retains its provenance so you can trace where it came from.
Mark evidence Supports, Partial, Contradicts or Not relevant, with tags and notes
Synthesis is grounded only on the evidence you kept, and lists what it excluded
Export the whole case to Word
Library and reading
A browsable library
Browse, sort and filter the collection. Resources show curated titles, summaries and key takeaways to help you decide what to read.
Sort by newest, oldest or title, and page through large corpora
Search within the library using real retrieval rather than title matching
Failed ingests and junk entries hidden automatically
Questions about a document
Open a resource and ask questions scoped to that document. Use the source material and citations to check the response.
A PDF reader, web page, video or audio player with transcript, or extracted text
Extracted text for every document, with the matches listed
A related-work rail to the next document
Saved research sessions
Return to earlier questions, answers and their sources. A saved session keeps the research trail together.
Start, rename, reopen and delete sessions from the sidebar
Export a session to Word with questions, answers, sources and scores
Mark answers helpful or not helpful
Generate and assess
Research formats
With analyst access, turn a research question into a briefing, comparison, timeline, FAQ or pros and cons. Generated material includes references for review.
A section nothing in the corpus supports is left out and listed as omitted
Comparison cells name the source each score came from
Save to an investigation, export to Word or print to PDF
Knowledge checks
With analyst access, generate an assessment on a topic in the collection. Answer the questions and review feedback against the source material.
3, 5 or 10 questions at foundational, intermediate or advanced level
Choose a corpus topic or type your own
Every question links to the source it was written from
Content, taxonomy and enrichment
Portal provisioning
Platform administrators can create a portal from a domain brief and a knowledge box. Provisioning sets up the taxonomy, graph strategy, suggested questions and branding.
Bind an existing knowledge box, or create and provision a new one from the app
Entity types, label sets and suggested questions designed from the corpus itself
Bindings and credentials stay on the server
Content management
Curators and administrators can upload resources and review corpus health. Manage the collection from the portal’s administration area.
Upload files up to 100 MB, add a link, paste text, or register a site or sitemap as a source
Sources re-sync on a schedule; back-pressure from the box is reported, never hidden
Corpus health finds failed ingests, thin extraction and bot-challenge pages, and purges them after confirmation
Label sets
Curators and administrators define categories, labels and their descriptions. Separate label sets can describe topics, document types and audiences for browsing and filtering.
Single-value or multi-value sets, each label with a definition the classifiers apply
Editing a set restarts its labellers for new resources without relabelling the existing corpus
Definitions double as the vocabulary reference on the Taxonomy page
Enrichment and presentation
Curators and administrators manage agents that classify content and generate titles, summaries and key takeaways. These fields make documents easier to discover and recognise.
A real title, summary, key takeaways and quotes of interest for every document
Each enrichment is an agent plus a schema; new fields render without code changes
Runs automatically on a cadence from hourly to monthly, and exports as JSON
Extraction Lab
An administrative lab profiles documents and compares extraction methods in a sandbox knowledge box. Administrators can set routing rules for how new uploads are extracted.
Default, table-aware and visual methods compared side by side on the same file
Documents profiled as prose, tables, image-only, garbled text or long scan
Routing rules send each class to the cheapest method that recovers it
Access and security
Roles and permissions
Portal roles range from viewer and analyst to curator and portal administrator. Separate platform administrator and owner roles govern the wider installation.
22 permissions; every route and MCP tool declares exactly one
Roles are cumulative within a scope
A portal role grants nothing on any other portal
Portal access modes
Administrators choose public, authenticated or restricted access. Restricted portals require an explicit access assignment.
Public: anyone can read
Authenticated: signed-in users of your Entra tenant
Restricted: only explicit assignments; everyone else sees sign-in metadata only
Mode changes are audited and take effect on the next request
Microsoft Entra sign-in
Sign in through the configured Microsoft Entra tenant. Role assignments and supported group mappings determine access to portal capabilities.
App roles for owner and platform administrator
Group mappings per portal once group claims are verified
Highest role wins per scope across app roles, groups and assignments
Scoped keys
Administrators issue keys for one portal, with a role and optional expiry. Keys can be revoked and cannot grant more access than their creator currently holds.
One portal, one role, optional expiry
Capped at the creator’s current role at every use
Revoke at any time; clients stop immediately
Audit log
Authorised administrators can read and export audit records for privileged actions, access denials and emergency access. Records identify the actor, action, scope and outcome.
Portal-scoped and platform-scoped views
Export as CSV or JSON
Emergency access
An explicitly enabled break-glass path gives an operator emergency access for a single request. Its use is audited, with lockouts after repeated failed attempts.
Off unless explicitly enabled
Single-request access, audited every time
Lockout after repeated failed attempts
Integration and tools
MCP server
Connect a compatible Model Context Protocol client to search the corpus, browse the catalogue, retrieve documents and ask questions. Access uses a key scoped to that portal.
Streamable HTTP endpoint with bearer keys
Four read-only tools: search the corpus, answer a question, get a document, browse the catalogue
A ready-to-paste JSON configuration for any MCP client
Research exports
With analyst access, export Ask sessions and investigations to Word. Generate also offers Word exports and a print-ready view that you can save as PDF.
Word: Ask sessions, investigations and generated material, references included
PDF: a print-ready view of a briefing
CSV or JSON: audit records; JSON: enrichment records
Custom domains
Administrators can connect a custom hostname to a portal. Its branding and knowledge box remain configured for that portal.
Connect a hostname from the administration area
Branding, knowledge box and access mode follow the portal
Platform and deployment
Cloudflare deployment
CorpusKit runs on Cloudflare Workers. SQLite-backed Durable Objects hold application state such as portal configuration, sessions and investigations.
Deploys only from a green gate: typecheck, lint, format, tests and build
Deno 2 and TypeScript, with no npm install step
One knowledge box per portal
Each portal connects to its own Progress Agentic RAG knowledge box. The box handles content extraction, indexing, retrieval and generation.
Reranking, answer scoring and entity extraction also run on the platform
One typed retrieval interface; no screen knows which model answered
No mock mode: always a live knowledge box
Australian region option
Administrators can select an Australian region for the Progress Agentic RAG knowledge box. Application hosting and model provider settings are configured separately.
Knowledge box created in the Australian region when selected
Corpus, index and graph stay with the box
Open source
CorpusKit is available under Apache-2.0, so you can run, inspect and adapt the code. Noice maintains the project and welcomes contributions.
The provisioning engine, retrieval layer, design system and every screen are yours to fork
Maintained by Noice, a Melbourne digital agency and Progress partner
Frequently asked questions
What CorpusKit is, how its answers are made and checked, and what running it takes.
What is CorpusKit?
CorpusKit is the open source research portal for Progress Agentic RAG. It turns a collection of reports, papers and other documents into a portal where people search, ask questions with cited answers and follow connections between sources. It is licensed under Apache 2.0.
Where do answers come from?
From the portal’s own collection. Every sentence that states a finding cites the passage it came from, and opening a citation shows that passage in its document.
How are answers checked?
Figures, populations, named studies and years are checked against the cited text, sentence by sentence, by plain-text comparison with no language model in the loop. Unsupported sentences are removed and the removal is declared under the answer, and confidence is shown as high, moderate or low.
PDF reports; Word, PowerPoint, Excel and OpenDocument files; and web pages and whole sites, re-synced on a schedule. On a Progress Agentic RAG plan that supports them, scanned pages are read too, and video and audio are transcribed and searchable. Uploads can be up to 100 MB per file.
Can AI assistants use a portal?
Yes. Each portal has an MCP server over Streamable HTTP with four read-only tools: search, answer, get document and browse catalogue. Access uses a key scoped to that portal, which an administrator can revoke at any time.
A Progress Agentic RAG account, with one knowledge box per portal. CorpusKit builds with Deno 2 and deploys to Cloudflare Workers, or runs locally or in a container, and administrators of the reference deployment sign in with Microsoft Entra. The repository lists every prerequisite.
Yes. Noice, which maintains CorpusKit, runs it as a hosted service called CorpusKit Cloud, at corpuskit.cloud. You can also run it yourself from the repository.
Through pull requests on GitHub. The contribution guide explains how, and every change passes typechecking, linting, formatting, tests and a full build before it merges.