Add option to treat all input targets as default scope for entire crawl session

[ehsandeep]

Discussed in #1383

^{Originally posted by mrschyte August 10, 2025}

Introduce a new optional boolean flag that treats all input targets (from file, CLI, or stdin) as explicit scope roots for the entire crawling session.
This differs from the current default host-based scoping (per-host) and from -fs normalization (dn/rdn/fqdn).

Details

Flag name
-ais, -all-input-scope treat all input targets as in-scope for the full crawling session

Behavior

• Each input line (stdin/file/CLI) is used as-is as a scope root.
• Ignores -fs (field-scope). No dn/rdn/fqdn normalization is applied.
• Compatible with -cs (in-scope regex) and -cos (out-of-scope regex) for additional filtering.
• Conflicts with -ns (no-scope). If both are set, -ais should take precedence and -ns ignored, or validation should error.
• Works seamlessly with stdin input: every line piped in is considered part of the global scope.

Example

Input file targets.txt:

• https://app.foo.example.com
• https://bar.test.com
• https://example.org

Run:

katana -list targets.txt -ais

Scope interpretation:

• All three targets are treated as independent scope roots.
• Crawling will consider links under each, without normalizing to rdn/fqdn/dn.

Why Needed

• Simplifies use cases where users want all provided targets in-scope at once, without needing to craft regex or rely on -fs.
• Useful for multi-target crawling sessions across different domains.

Note:

• -ais and -fs can't be used togather and should return error as validation error.