Better Steerability, Authentication Improvements, and the Migration to the Pi Harness

[keygraphVarun]

Better Steerability, Authentication Improvements, and the Migration to the Pi Harness

Over the past month, we have been quietly working on foundational improvements to Shannon.

These updates are about making autonomous pentesting more controllable, more reliable, and more flexible in real-world environments: clearer run scoping, better authentication preflight checks, and the groundwork for multi-model support.

1. Steerability

The first major June update is a much richer configuration layer for controlling how Shannon scopes and executes a run.

Before this release, config was mostly limited to environment description, authentication setup, and simple path-based focus and avoid rules. That worked for basic targeting, but it was not expressive enough for real pentest scoping.

The new config-driven run scope lets teams specify which vulnerability classes should run, whether exploitation is allowed, which URLs or code paths are in scope, and how the final report should be filtered:

# Limit which vulnerability classes run end-to-end (optional, default: all five)
vuln_classes: [injection, xss, auth, authz, ssrf]

# Skip the exploitation phase for analysis-only runs
exploit: "false"

# Free-form rules of engagement
rules_of_engagement: |
  - No password brute-force; cap login attempts at 5 per account.
  - Throttle to under 5 requests per second per endpoint; back off 60s on any 429.
  - Use placeholders like [order_id] in deliverables -- no real data values.

rules:
  # Supported types: url_path, subdomain, domain, method, header, parameter, code_path
  avoid:
    - description: "AI should avoid testing logout functionality"
      type: url_path
      value: "/logout"

    - description: "Out-of-scope vendored libraries"
      type: code_path
      value: "src/vendor/**"

  focus:
    - description: "AI should emphasize testing API endpoints"
      type: url_path
      value: "/api"

report:
  min_severity: low
  min_confidence: low
  guidance: |
    Drop findings about missing security headers and rate-limit gaps.

Under the hood, Shannon now validates run scope before resuming, checks that code_path rules match at least one repo entry, tags missing code path entries by avoid or focus kind, and emits documented deny rules for scoped file access. Report generation also understands scoped runs better, including per-class findings when exploitation is disabled.

What changed in practice is that run scoping is now a first-class part of the workflow instead of a loose instruction passed to agents. Shannon parses the config into an explicit run scope, applies that scope before agents start work, and carries the same constraints through analysis, exploitation, and reporting. URL, domain, method, header, parameter, and code-path rules can now be treated consistently as focus or avoid signals, and code-path rules are enforced against repo-relative files and globs.

Report filtering is also part of the same scope. Teams can now choose to suppress findings below a severity or confidence threshold, or add report-specific guidance for classes of findings they do not want included. That means a run can be narrower at execution time and cleaner at reporting time.

Summary:

• richer config-driven run scoping
• vulnerability-class selection with vuln_classes
• analysis-only runs with exploit: "false"
• focus and avoid rules across URL, domain, method, header, parameter, and code path
• repo-relative code_path enforcement for files and globs
• report filtering by severity, confidence, and custom guidance

Launch note: feat: add config-driven run scoping and report filtering (#326)

2. Authentication Improvements

Authentication is one of the most important parts of autonomous pentesting. If agents cannot log in reliably, the rest of the run suffers.

We have added an authentication validation step to the preflight checklist, so you no longer have to wait through a long multi-hour run just to discover that Shannon was unable to authenticate.

Before a full run begins, Shannon now checks whether the provided credentials and login flow can actually get an agent into the application. When authentication fails, the preflight step reports where the failure happened and masks sensitive values in the failure detail, so teams get useful debugging context without leaking credentials.

We also made the browser side of authentication more robust. Shannon now provisions a Playwright stealth configuration for browser auto-discovery and uses a more realistic browser fingerprint for identity-provider flows. We have been testing and tuning this against common providers like Google, Microsoft, and Okta, with manual overrides available for more complex login paths.

This release also expands the kinds of login flows Shannon can handle. Passwords are now optional for passwordless flows, and email_login credentials let the agent sign into a mailbox for magic-link and email-OTP authentication. Login-flow placeholders can reference those email credentials, which makes it possible to describe multi-step authentication flows more naturally in config.

The goal is simple: catch auth issues before the expensive part of the run starts, and make Shannon work against more of the login systems teams actually use.

Summary:

• authentication validation now runs during preflight
• failures are reported earlier with masked sensitive values
• browser stealth and fingerprinting are improved for identity-provider flows
• authentication has been tested and tuned against Google, Microsoft, and Okta
• passwordless flows are supported
• email_login credentials enable magic-link and email-OTP authentication
• login-flow placeholders now support email credential substitution

Launch note: feat(auth): auth-validation preflight + email_login credentials (#335)

3. The Big One: Migrating to the Pi Harness

Our main focus for June is a larger architectural shift: we are rewriting Shannon to use the Pi harness and moving away from claude-agent-sdk.

This has been a long time coming. We have heard the community feedback, and the number one request has been clear: support for multiple models.

Earlier on, we had workarounds like claude-code-router, and later we added support for overriding the base URL. Those helped, but they were still workarounds. The deeper limitation was that Shannon was fundamentally built on claude-agent-sdk.

After a lot of evaluation, we have settled on the Pi harness, the same harness behind OpenClaw.

This gives us a better foundation for multi-model support and a more flexible agent runtime going forward. We have a private preview running now, and we will be releasing more in the coming weeks.

More to come on this!