Operator console for an automatic pipeline — health, exceptions, rare review, emergency rollback, costs.
Portal · Graph · Dashboard · System Status · Graphiti Health · API Reference
Most DarkHerd work runs automatically. Use these five panels when something needs attention.
Sign in to load.
Failed / stuck worker jobs.
Pending staging claims needing promote/reject.
Snapshot ontology facets before risky changes; restore only if needed.
SQLite cost / usage slice.
Sign in to load dual-gate status. Live auto_apply soak stays off unless you set DARKHERD_ONTOLOGY_PATCH_AUTO_APPLY_ENABLED=true.
Need ingest, communities, whitepapers, or queue prune? Switch to Advanced.
Save and restore ontology catalog snapshots. Use checkpoints before making ontology changes so you can roll back if needed.
Saves the current ontology catalog state as a new checkpoint. Optionally captures a full snapshot for later restore.
Restores the ontology catalog from a saved checkpoint snapshot. Use dry-run first to preview the change.
Each ingest can create multiple SQLite rows per coin (one per URL tried). Use One row per asset to show a single “current best” row per coin (prefers ok over failed). Sort by fetch time matches ingest order. Files live under the directory shown; errors are from HTTP or text extraction.
| ID | Asset | Status | Disk | Chars | URL | Error / note | Local path |
|---|
Checks which CoinGecko-sourced assets have been delisted and archives them. CoinGecko rate limits (HTTP 429) may stop the batch early.
Pushes archived asset status changes to the Neo4j knowledge graph. Use after archiving assets that were previously synced.
Review AI-extracted claims from whitepapers. Claims start as pending and can be promoted (added to the knowledge graph) or rejected. The Evidence and LLM excerpt columns let you compare what the AI extracted vs the source text. Promote requires a Neo4j connection.
Runs Mistral AI to extract claims from document chunks and saves them as pending rows for review. Use Asset offset to page through assets. Corpus sweep processes all assets automatically. Check Queue to worker to run in the background instead of blocking this page.
Extraction fields (including Queue to worker, Include archived, Corpus sweep) persist in sessionStorage for this tab (debounced; also after each extract run) — P2al v6 / P2vcg / P2vda.
This tab remembers staging list filters, include toggles, the Graphiti / detach checkboxes, and P2ap Filter loaded rows (same sessionStorage key — P2aq) (debounced on edit; also after each list load) — P2ak. On reload, #sc-msg may combine P2ak + P2al + revision panel (P2am) restores. Clear saved staging UI drops those keys and clears the row-filter input (P2an).
Bulk (page scope): actions use only rows currently loaded in the table (status / Kind / limit). They do not scan the full database.
Server bulk by filter — same status / claim_kind / limit / offset as the list query (newest first; offset skips newest N before the cap). JSON includes total_filtered (P2u), has_more (P2w), and optional prev_offset / next_offset (P2y, same step as list P2x). After a Preview dry-run, P2aa buttons apply those offsets and re-run the same preview. Previews default to dry_run; execute affects up to limit matching rows after offset (not only the visible table).
Claim revisions (P2m / P2n / P2o / P2p / P2ab / P2ac / P2ad / P2ae / P2af / P2ag / P2ah): append-only text snapshots; optional unified diff vs prior (GET). After Load revisions, a timeline lists each revision with expandable diff blocks; raw JSON stays under Raw JSON. P2ab: Live diff vs canonical compares your New claim text to canonical_claim_text from the same GET (line-unified, capped). P2ac: after a successful Append operator revision, the page re-runs the same GET (timeline + JSON + live-diff baseline) and clears New claim text. P2af / P2ag: Use claim from selection copies the first checked row’s claim text + Claim id; P2ag also fills Optional kind / Optional conf. when those boxes are still empty. P2ah: ID cell click checks that row (supports P2af without an extra checkbox click). Operator edit: pending / failed — SQLite + revision only (P2n). promoted — check If row is promoted, update Neo4j so the matching :Claim text/kind updates with SQLite (P2o; needs Bolt).
P2am: Claim id, Diff vs prior, Live diff vs canonical, Optional kind/conf, and Neo4j sync persist in sessionStorage for this tab (debounced on edit; also after Load revisions / Append / ID-cell or selection fills). New claim text is intentionally not persisted.
Load revisions to set the canonical baseline.
P2ad: click an ID cell to fill Claim id and check that row (P2ah). P2ae: double-click the same cell to run Load revisions immediately (checkbox-only clicks still skip ID handlers).
| ID | Asset | Status | Kind | Conf. | Hints | Model | Chunk | Source | Evidence (chunk) | LLM excerpt | Created | Promoted | Claim | Note |
|---|
Groups related assets into communities using the Louvain algorithm based on shared tags. After running, use sync_graph or a targeted sync so the Neo4j graph picks up community labels. Use Leiden preview buttons to compare algorithms without writing to the database.
Summaries (Mistral): one short blurb per community for a persist batch; stored in tag_overlap_community_summaries. Requires MISTRAL_API_KEY. Leave run id empty to use the latest batch on assets.
When Post-ingest is checked, new assets are automatically synced to the knowledge graph. Replay Graphiti sends asset data to the temporal knowledge graph (disable if assets were already sent to avoid duplicates). Auto chunk + tags generates tags and embeddings using Mistral. Ontology review checks if new assets introduce concepts not yet in the ontology.
Monitor and manage background worker jobs: check health, view SLO rates, retry/cancel jobs, or prune old entries.
Shows all available background job types that can be enqueued.
Shows the scheduled refresh cadence and active corpus ceiling from the system configuration. This panel is read-only.
Queues a multi-step job: Louvain community detection → optional Neo4j sync → optional Mistral summaries. Requires a background worker to be running.
Queues a full community rebuild job (Louvain + optional Neo4j sync + optional Mistral summaries). Requires a background worker to be running.
Shows whether Neo4j graph sync jobs were automatically enqueued after the last ontology patch. Read-only diagnostic.
Shows recent global refresh job runs (community detection, summarization, etc.) to help correlate scheduled tasks with their outcomes.
Shows process-local counters for auto-scheduled refresh tasks (outcomes and tick exceptions).
Read-only database_dialect and marker p7zaoi on
GET /api/v1/insights/summary
— parity GET /api/v1/ready (P7zaoe)
and GET /metrics darkherd_database_up (P7zaoh).
Matches /portal Summary strip, /graph + /dashboard footers, and lockdown 403 HTML (P7zaok).
Insight fetch panels on this page use sameOriginApiPath so HyperCharts (P7xq) hits /api/darkherd/insights/summary.
Shows which API providers have keys configured and lets you preview usage/cost data for each provider.
Preview cost/usage data for each configured provider (24-hour window):
View the history of admin job runs (ingest, community refresh, etc.).