Skip to content

Feature: Add Splunk + Tenable.SC API clients alongside existing file-format converters #86

Description

@aaronlippold

Summary

hdf-libs currently converts Splunk and Nessus/Tenable file formats (JSON events, .nessus XML) to HDF, but has no API client code to fetch data from those services. Heimdall2 maintains its own raw axios-based API clients for both (~700 lines total) with hand-rolled auth, error handling, CORS workarounds, and retry logic. This duplicates effort — any tool that wants to pull from Splunk or Tenable has to rewrite the same integration.

Proposal

Add thin, typed API clients to hdf-libs that fetch data from Splunk and Tenable.SC REST APIs and pipe it through the existing converters. The client handles auth + fetch, the converter handles format transformation — clean separation.

Splunk API Client

What Heimdall2 does today (source: heimdall2 libs/hdf-converters/src/splunk-mapper.ts + splunk-tools.ts):

  • Auth: POST /services/auth/login with username/password → session key
  • Search: POST /services/search/v2/jobs with SPL query → poll for completion → GET .../results
  • Uses raw axios with manual session key header injection
  • ~350 lines of code

What already exists:

Suggested API:

interface SplunkClientConfig {
  host: string;
  port?: number;
  scheme?: 'http' | 'https';
  username: string;
  password: string;
}

async function fetchSplunkToHdf(config: SplunkClientConfig, index: string): Promise<HDFResults>
async function pushHdfToSplunk(config: SplunkClientConfig, data: HDFResults, index: string): Promise<void>
async function verifySplunkCredentials(config: SplunkClientConfig): Promise<boolean>

Tenable.SC API Client

What Heimdall2 does today (source: heimdall2 apps/frontend/src/utilities/tenable_util.ts):

  • Auth: API key header (x-apikey: accesskey=...;secretkey=...)
  • List scans: GET /rest/scanResult?fields=name,description,...
  • Download results: POST /rest/scanResult/{id}/download?downloadType=v2 → zip → unzip → .nessus XML
  • ~385 lines including error handling and CORS management

What already exists:

Suggested API:

interface TenableClientConfig {
  host: string;
  accessKey: string;
  secretKey: string;
}

async function listTenableScans(config: TenableClientConfig, options?: { startTime?: number; endTime?: number }): Promise<TenableScanResult[]>
async function fetchTenableScanToHdf(config: TenableClientConfig, scanId: string): Promise<HDFResults>
async function verifyTenableCredentials(config: TenableClientConfig): Promise<boolean>

Why This Belongs in hdf-libs

  1. The converter already existsconvertNessusToHdf() and convertSplunkToHdf() handle format transformation. Adding a fetch layer is the natural extension.
  2. Dual TS + Go — both implementations benefit. The Go CLI could gain hdf fetch splunk and hdf fetch tenable commands.
  3. Single maintenance point — Heimdall2, saf-cli, and any future consumer get the integration for free.
  4. Auth patterns consolidated — one place to handle session keys, API keys, token refresh, error classification.

Additional Considerations

  • HTTP client choice: Consider ky (browser-first, small, typed) or native fetch (Node 18+, browsers, Cloudflare Workers) for maximum portability.
  • Browser vs server: Tenable.SC requires CORS proxy in browser contexts. The client should support both direct and proxied modes.
  • The splunk-sdk question: Worth evaluating whether splunk-sdk@2.0.2 handles enough complexity to adopt vs maintaining a thin REST wrapper.

Contribution Offer

We're migrating Heimdall2 from embedded libs/inspecjs + libs/hdf-converters v2 to @mitre/hdf-libs v3 packages. As part of this, we'd build the initial TypeScript API client implementations and contribute them as a PR, since we already have working implementations in Heimdall2 to port from.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions