Prevent authenticated scans from setting canonicalized headers

[halcyondream]

Description

Authenticated scans will set headers with their canonical names (ie, Authtoken rather than authToken). This makes it challenging to test web apps and APIs that are not compliant with the case-insensitive requirement of the HTTP standard.

I propose a fix in PR 6479. I'm open to a better fix here, but it worked for me to finish an assessment.

Example config:

$ cat config.yaml

env-vars: true 
templates:
  - "/templates/test.yaml"
exclude-tags:
  - noscan
secret-file:
  - "/templates/secret.yaml"
disable-update-check: true 
json: true 
target:
  - "https://webhook.site/13cd0b09-9f9b-4a1b-a06f-8773a4b3f7ed"
store-responses: true 
prefetch-secrets: true

$ cat secret.yaml

id: auth-tokens 
info:
  name: Get and set an authorization token 
  author: me
  severity: info 
  tags: noscan 
dynamic:
  - template: /templates/auth.yaml
    input: "https://webhook.site/13cd0b09-9f9b-4a1b-a06f-8773a4b3f7ed"
    variables:
      - key: x
        value: y
    domains:
      - "webhook.site"
    type: header
    headers:
      - key: barAuthToken
        value: "{{barAuthToken}}"

$ cat auth.yaml

id: auth 
info:
  name: Actually get the auth token from the API 
  author: me
  severity: info 
  tags: noscan 
requests:
  - method: POST
    path:
      - "{{BaseURL}}/check"
    headers:
      Content-Type: application/json
      Accept: application/json 
    body: |
      {"hello": "world"}
    matchers-condition: and 
    matchers:
      - type: dsl 
        dsl:
          - "status_code == 200"
    extractors:
      - type: json 
        part: body 
        name: barAuthToken
        json: [".bar"]$ 

The token from .bar is found and set:

HTTP/1.1 200 OK
...

{"bar": "SomeBarValue"}
[auth:dsl-1] [http] [info] https://webhook.site/13cd0b09-9f9b-4a1b-a06f-8773a4b3f7ed/check ["SomeBarValue"]

The header barAuthToken is transformed to Barauthtoken:

[test] Test the case sensitivity of the header keys (@me) [low]
[INF] [test] Dumped HTTP request for https://webhook.site/13cd0b09-9f9b-4a1b-a06f-8773a4b3f7ed/test

POST /13cd0b09-9f9b-4a1b-a06f-8773a4b3f7ed/test HTTP/1.1
Host: webhook.site
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36
Connection: close
Content-Length: 22
Accept: application/json
Accept-Language: en
Barauthtoken: SomeBarValue
Content-Type: application/json
Accept-Encoding: gzip

This looks looks much nicer, and shouldn't pose an issue. But, it will fail against non-compliant servers that expect barAuthToken as-is.

[black-leg-nameko]

Thanks for raising this. What you’re seeing is the Go net/http behavior: header keys are canonicalized (e.g., barAuthToken → Barauthtoken) as they’re written, because HTTP/1.1 field-names are case-insensitive by spec. Nuclei inherits this behavior when using the standard HTTP client. See prior reports confirming the canonicalization on the client path.

As a practical workaround today, you can avoid canonicalization by using raw/unsafe HTTP in your template so Nuclei uses the rawhttp client and preserves whatever you put on the wire:

• Raw HTTP template docs: https://docs.projectdiscovery.io/templates/protocols/http/raw-http
• Unsafe HTTP (enable unsafe: true): https://docs.projectdiscovery.io/templates/protocols/http/unsafe-http

Example sketch:

id: case-sensitive-header-check
info:
  name: Case sensitive header test
  severity: info
requests:
  - raw:
      - |
        POST /test HTTP/1.1
        Host: {{Hostname}}
        barAuthToken: {{some_token}}
        Content-Type: application/json
        Content-Length: 22

        {"hello":"world"}
    unsafe: true        # use rawhttp
    matchers:
      - type: status
        status:
          - 200

This path keeps the exact casing (barAuthToken) for non-compliant targets that break otherwise.

That said, I like your proposal in PR #6479 to prevent canonicalization for authenticated scans specifically. Even if HTTP header names are case-insensitive by spec, real-world targets aren’t always compliant, and having a first-class knob to opt out (without switching the whole request to raw mode) would be helpful.

Happy to test a build or share more examples if needed.