ISAC Reports — Architecture & Data Flow

How data flows in, gets processed, and flows out

1. Data Ingestion — External Sources

APIs
CISA KEV
Known Exploited Vulns JSON
cisa.gov
NVD
CVE database (paginated)
nvd.nist.gov
MalwareBazaar
Malware samples
abuse.ch (auth key)
AbuseIPDB
Malicious IP blacklist
abuseipdb.com (API key)
ThreatFox
IOCs (IPs, domains, hashes)
abuse.ch (auth key)
OTX
Threat intel pulses
alienvault.com (API key)
RSS
CISA Advisories
1 feed: all advisories
Vendor Blogs (11 feeds)
MSRC · Google TAG · CISA ICS · CISA ICS Medical · NCSC UK
Siemens · Cisco Talos · Fortinet · Palo Alto · Rockwell · CERT-EU
General News (9 feeds)
BleepingComputer · Hacker News · Dark Reading · SecurityWeek
InfoSec Mag · Help Net Security · The Record · CyberScoop · Ars Technica
→ filtered by transport/cyber relevance
Each source class: fetch() → normalize() → NormalizedItem[]

2. Source Class Hierarchy

OOP
BaseSource (abstract)
run() = fetch() + normalize() — template method pattern
ApiSource
Extends BaseSource
Auth check → rateLimitedFetch → parse → validate → extractItems
Used by: CISA KEV, NVD, MalwareBazaar, AbuseIPDB, ThreatFox, OTX
RssSource
Extends BaseSource
Loop feeds → rss-parser → filter by lookback → optional relevance filter
Used by: CISA Advisories, Vendor Blogs, General News
All items collected into a single array

3. Pipeline Processing (pipeline.ts)

Steps
① Filter Stale
Remove old CVEs
Remove items outside
lookback window
② Deduplicate
Single-pass O(n)
Match by: CVE, SHA-256,
IP, domain, title
③ Classify
Tag items: ransomware,
apt, zeroday, ics,
transport, rce, etc.
④ Truncate
Cap descriptions
at 300 chars
⑤ Domain Classify
Tag each ref URL
primary / media / other
⑥ Metadata
Build source &
category breakdowns
PipelineResult { items, fetchErrors, metadata }

4. Storage

DB
SQLite Database (db.ts)
output/isac-reports.db via node:sqlite DatabaseSync

Tables:
• items — all normalized items (JSON columns for indicators, tags, refs)
• fetch_errors — source errors from last run
• metadata — key/value (generatedAt, reportDate, breakdowns)

Shared queries: searchItems(filters), getItemsByRange(range)
Indexes: severity, source, category, published_at
Three entry points serve data from cache

5. Entry Points

Entry
server.ts — HTTP Server (:3000)
GET / → Dashboard (cache status, sources, range links, search link)
GET /report?range=daily|weekly|monthly|full → HTML report with range selector
GET /search?query=...&severity=... → Search page with filters + pagination
GET /api/status → JSON cache status
/mcp → StreamableHTTP MCP transport (sessions)
Pipeline runs on startup + cron (3 AM / 3 PM)
mcp-server.ts
Stdio transport
For Claude Desktop,
Cursor, Windsurf, etc.
index.ts — CLI
Run pipeline once
Write JSON + HTML
to output/ directory
Data served via multiple formats

6. Output Formats

Out
MCP Tools (4 tools)
get_report — Markdown report (daily/weekly/monthly, filterable by source)
search — Structured filters: query, severity, category, source, tag, date range, pagination
query_db — Raw read-only SQL for advanced analysis
list_sources — Enabled/disabled sources
HTML Report
Range selector
Interactive table
Search + filter
Severity badges
CVE links to NVD
HTML Search
Filter form (dropdowns)
Server-side query
Pagination
Same dark theme
JSON Report
output/report-*.json
Machine-readable
Includes LLM prompt
MCP Prompt
generate_bulletin
daily/weekly/monthly
Transport ISAC bulletin
API Sources
RSS Sources
Pipeline Processing
Storage / Cache
Entry Points
Outputs