Introduce Spicy parser framework with HTTP parser implementation

[Boolean-Autocrat]

Implements 2/3 of #184

This PR introduces Spicy protocol parser integration with C++ into Glutton, along with a concrete Spicy HTTP parser and handler which mimics the existing functionality of the pure-Go HTTP parser.

Spicy parsers can be enabled via Viper config (spicy.enabled: true) and will seamlessly replace the Go parsers for the supported protocols (only HTTP for now).

Architecture

C++ Bridge Layer (protocols/spicy/bridge.{h,cpp})

The core integration uses a C++ bridge that interfaces between Spicy/HILTI runtimes and Go:

• Runtime Management: Thread-safe initialization and cleanup of Spicy/HILTI runtimes
• Generic Parsing Interface: spicy_parse_generic() accepts any parser name and raw data
• Memory Management: Safe allocation/deallocation with a lot of OOM protection (added in later commits)
• Data Structure Flattening: Recursive traversal of HILTI type system to flatten nested structures into flat key-value pairs using dot notation (e.g., version.number, headers[0].name)

Go Worker Thread Model (protocols/spicy/parser.go)

This was a fairly important design decision I had to make. Due to Spicy/HILTI's thread-local storage requirements, I decided that parsing should operate through a dedicated worker:

• Single Worker Thread: Locked to OS thread with runtime.LockOSThread()
• Command Channel: Thread-safe communication via workerCmd messages
• Parser Registry: Automatic discovery and registration of available Spicy parsers
• Timeout Protection, among other things

Key C++ Parsing Function

The dump_value() function recursively processes HILTI's type system:

• Containers: Arrays/vectors become field[0], field[1], etc.
• Maps: Key-value pairs become prefix.key
• Structs: Fields become prefix.fieldname
• Depth Limiting: Prevents stack overflow with 64-level limit
• Binary Data: Automatically detects and preserves binary vs. text data

Implementation

The new HTTP grammar file (protocols/spicy/parsers/http.spicy) shows the integration of Spicy into the Glutton system. I've also written a corresponding handler (protocols/spicy/handlers/http.go) which tries to maintain behavioral compatibility with the existing pure-Go HTTP handler (while leveraging Spicy under the hood).

As for the integration points, in protocols/protocols.go I have added a simple viper.GetBool("spicy.enabled") within the "poor man's detection" for HTTP and the entire Spicy runtime management (init and shutdown) happens in glutton.go after checking the same Viper boolean.

Adding New Parsers

I have tried to keep the addition of new parsers as seamless as possible. The flow looks something like this:

• Write Spicy Grammar: Create .spicy file in protocols/spicy/parsers/
• Build Parser: make spicy automatically compiles Spicy grammars to C++
• Create Handler: Implement Go handler in protocols/spicy/handlers/
• Auto-Discovery: Parser is automatically registered and available

Other than the usual change for registering in protocols/protocols.go everything can basically remain untouched. The only change required in protocols/spicy/bridge.cpp will be the addition of an include statement for the new parser's generated header file.

Notes

• For now, I am not shipping any of the .cc or .h files generated by Spicy. In the main Glutton directory, you can simply run make spicy which will trigger their creation.
• As for testing, I have added two simple test files protocols/spicy/parser_test.go and protocols/spicy/handlers/http_test.go which verify the workings of both the parser and the handler with some basic edge cases.
• Build times for the Go binary are a bit longer now due to the inclusion of Spicy and its generated C++ files
• A lot of the intuition for this implementation was derived from the Silk PoC, so thanks for that :)
• I've tried to leave relevant comments in all critical places explaining the "why" of the way specific things were done
• I've also tried my best to cover all C++ edge cases and implement measures at points of failure, but I may have missed something so I'm open to making any changes to the implementation.

Screenshots (showing the Spicy HTTP parser in action)

Commands ran:

curl http://localhost:5000/test

# triggers handleEthereumRPC
curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' http://localhost:5000/rpc

# triggers handleWallet
curl http://localhost:5000/wallet/balance

# triggers handleDockerAPIVersion
curl http://localhost:5000/v1.16/version

# triggers handleCitrixSMB
curl http://localhost:5000/vpn/index.html

# triggers handleVMwareSend
curl -X POST -H "Content-Type: text/plain" -d "test data with 127.0.0.1 22 embedded" http://localhost:5000/sdk/hyper/send

[Boolean-Autocrat]

@glaslos @furusiyya Here is my PR for the protocol parser implementation.

I believe the GitHub action will need to be changed to accommodate the Spicy runtime. I've left spicy.enabled to true by default. The cgo flags in protocols/spicy/parser.go are Linux specific, so I hope that isn't an issue.

We might need to include some documentation about the installation of the Spicy runtime and clang too.

[Boolean-Autocrat]

@glaslos @furusiyya I have sorted out the GitHub actions checks too. The checks are passing successfully after including the Spicy runtime in the actions runner.

[Copilot]

Pull Request Overview

This PR adds the Spicy HTTP parser and its Go/C++ integration to optionally replace the existing pure-Go HTTP parser when enabled in configuration. Key changes include:

• Exporting and using the embedded TCP resources FS as Res instead of res
• Introducing the full C++ bridge layer and Go worker thread model for Spicy parsers
• Wiring up Spicy initialization, shutdown, and TCP handler selection via a Viper config flag

Reviewed Changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
protocols/tcp/resources.go	Export embedded FS as Res
protocols/tcp/http.go	Switch to using Res.ReadFile for embedded files
protocols/spicy/parsers/http.spicy	New Spicy HTTP grammar
protocols/spicy/parser.go	Go worker model and generic parse/cleanup functions
protocols/spicy/helpers.go	Utility functions for nested flat maps
protocols/spicy/handlers/http.go	Spicy-based HTTP handler, mirroring existing behavior
Makefile	Updated build command to use Clang and dropped LDFLAGS logic

Comments suppressed due to low confidence (2)

protocols/spicy/helpers.go:82

• This helper is marked unused and lacks tests. Either remove it or add unit tests to ensure it behaves correctly when converting flat maps to nested structures.

func NestedFromFlat(flat map[string]interface{}) map[string]interface{} {

protocols/spicy/handlers/http.go:84

• The pure-Go handler used strings.Contains, matching URIs with query parameters. Switching to HasPrefix may miss valid version endpoints (/v1.16/version?foo=bar). Consider reverting to Contains or explicitly handling query strings.

	if !strings.HasPrefix(uri, "/v1.16/version") {

[Boolean-Autocrat]

@glaslos I've resolved all of Copilot's concerns; they were fairly minor.

I hope the overall structure of the implementation is fine. It would be great if you could go through protocols/spicy/bridge.cpp once; it's the main bridge layer and Copilot didn't comment on it either.

[glaslos]

Some initial thoughts.

[furusiyya]

@glaslos just for your info:

Mid-term Review Meeting Minutes:
Suyash will integrate Lukas' comments and work on the Spicy DNS parser and its test cases next.

July Week 5 Sync:
Will be wrapping it up this week.

[glaslos]

@furusiyya what is needed to get this over the finish line? In case @Boolean-Autocrat is not available anymore, I can get the PR ready to be merged.

[furusiyya]

@glaslos I was waiting for @Boolean-Autocrat to push spicy DNS parser but he is not available. I will try to get this over the finish line in next month.