Summary
hdf-libs currently converts Splunk and Nessus/Tenable file formats (JSON events, .nessus XML) to HDF, but has no API client code to fetch data from those services. Heimdall2 maintains its own raw axios-based API clients for both (~700 lines total) with hand-rolled auth, error handling, CORS workarounds, and retry logic. This duplicates effort — any tool that wants to pull from Splunk or Tenable has to rewrite the same integration.
Proposal
Add thin, typed API clients to hdf-libs that fetch data from Splunk and Tenable.SC REST APIs and pipe it through the existing converters. The client handles auth + fetch, the converter handles format transformation — clean separation.
Splunk API Client
What Heimdall2 does today (source: heimdall2 libs/hdf-converters/src/splunk-mapper.ts + splunk-tools.ts):
- Auth:
POST /services/auth/login with username/password → session key
- Search:
POST /services/search/v2/jobs with SPL query → poll for completion → GET .../results
- Uses raw axios with manual session key header injection
- ~350 lines of code
What already exists:
Suggested API:
interface SplunkClientConfig {
host: string;
port?: number;
scheme?: 'http' | 'https';
username: string;
password: string;
}
async function fetchSplunkToHdf(config: SplunkClientConfig, index: string): Promise<HDFResults>
async function pushHdfToSplunk(config: SplunkClientConfig, data: HDFResults, index: string): Promise<void>
async function verifySplunkCredentials(config: SplunkClientConfig): Promise<boolean>
Tenable.SC API Client
What Heimdall2 does today (source: heimdall2 apps/frontend/src/utilities/tenable_util.ts):
- Auth: API key header (
x-apikey: accesskey=...;secretkey=...)
- List scans:
GET /rest/scanResult?fields=name,description,...
- Download results:
POST /rest/scanResult/{id}/download?downloadType=v2 → zip → unzip → .nessus XML
- ~385 lines including error handling and CORS management
What already exists:
Suggested API:
interface TenableClientConfig {
host: string;
accessKey: string;
secretKey: string;
}
async function listTenableScans(config: TenableClientConfig, options?: { startTime?: number; endTime?: number }): Promise<TenableScanResult[]>
async function fetchTenableScanToHdf(config: TenableClientConfig, scanId: string): Promise<HDFResults>
async function verifyTenableCredentials(config: TenableClientConfig): Promise<boolean>
Why This Belongs in hdf-libs
- The converter already exists —
convertNessusToHdf() and convertSplunkToHdf() handle format transformation. Adding a fetch layer is the natural extension.
- Dual TS + Go — both implementations benefit. The Go CLI could gain
hdf fetch splunk and hdf fetch tenable commands.
- Single maintenance point — Heimdall2, saf-cli, and any future consumer get the integration for free.
- Auth patterns consolidated — one place to handle session keys, API keys, token refresh, error classification.
Additional Considerations
- HTTP client choice: Consider
ky (browser-first, small, typed) or native fetch (Node 18+, browsers, Cloudflare Workers) for maximum portability.
- Browser vs server: Tenable.SC requires CORS proxy in browser contexts. The client should support both direct and proxied modes.
- The splunk-sdk question: Worth evaluating whether
splunk-sdk@2.0.2 handles enough complexity to adopt vs maintaining a thin REST wrapper.
Contribution Offer
We're migrating Heimdall2 from embedded libs/inspecjs + libs/hdf-converters v2 to @mitre/hdf-libs v3 packages. As part of this, we'd build the initial TypeScript API client implementations and contribute them as a PR, since we already have working implementations in Heimdall2 to port from.
Summary
hdf-libs currently converts Splunk and Nessus/Tenable file formats (JSON events, .nessus XML) to HDF, but has no API client code to fetch data from those services. Heimdall2 maintains its own raw axios-based API clients for both (~700 lines total) with hand-rolled auth, error handling, CORS workarounds, and retry logic. This duplicates effort — any tool that wants to pull from Splunk or Tenable has to rewrite the same integration.
Proposal
Add thin, typed API clients to hdf-libs that fetch data from Splunk and Tenable.SC REST APIs and pipe it through the existing converters. The client handles auth + fetch, the converter handles format transformation — clean separation.
Splunk API Client
What Heimdall2 does today (source: heimdall2
libs/hdf-converters/src/splunk-mapper.ts+splunk-tools.ts):POST /services/auth/loginwith username/password → session keyPOST /services/search/v2/jobswith SPL query → poll for completion →GET .../resultsWhat already exists:
splunk-sdk@2.0.2on npm — official Splunk Enterprise SDK for JavaScript. Handles auth, search jobs, results pagination. Unclear browser compatibility.@splunk/cloud-sdk— Splunk Cloud SDK (different platform)Suggested API:
Tenable.SC API Client
What Heimdall2 does today (source: heimdall2
apps/frontend/src/utilities/tenable_util.ts):x-apikey: accesskey=...;secretkey=...)GET /rest/scanResult?fields=name,description,...POST /rest/scanResult/{id}/download?downloadType=v2→ zip → unzip → .nessus XMLWhat already exists:
Suggested API:
Why This Belongs in hdf-libs
convertNessusToHdf()andconvertSplunkToHdf()handle format transformation. Adding a fetch layer is the natural extension.hdf fetch splunkandhdf fetch tenablecommands.Additional Considerations
ky(browser-first, small, typed) or nativefetch(Node 18+, browsers, Cloudflare Workers) for maximum portability.splunk-sdk@2.0.2handles enough complexity to adopt vs maintaining a thin REST wrapper.Contribution Offer
We're migrating Heimdall2 from embedded
libs/inspecjs+libs/hdf-convertersv2 to@mitre/hdf-libsv3 packages. As part of this, we'd build the initial TypeScript API client implementations and contribute them as a PR, since we already have working implementations in Heimdall2 to port from.