fix: increase verbosity logs and tracing

[TheRayquaza]

• adding tracing instrument to enrich trace spans
• adding some debug logs
• making info default log level (is it a good idea ?)

[Copilot]

Pull request overview

This PR increases observability in the containerd-shim-spin runtime by adding tracing spans (#[instrument]) and additional debug/info logs, and by raising the shim’s default log level.

Changes:

• Add tracing::instrument annotations and additional debug events across cache init, source loading, trigger execution, and compilation.
• Increase log detail (e.g., layer sizes, trigger/component counts) during OCI/app load and runtime execution.
• Change the shim default log level from error to info and add tracing dependencies/features.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
containerd-shim-spin/src/utils.rs	Adds #[instrument] and a debug event when initializing the cache; instruments archive unpack helper.
containerd-shim-spin/src/trigger.rs	Instruments trigger runner to add span context for trigger type and errors.
containerd-shim-spin/src/source.rs	Instruments source loading and locked-app creation; adds more detailed logs while processing OCI layers and loaded app structure.
containerd-shim-spin/src/main.rs	Changes default shim log level from error to info.
containerd-shim-spin/src/engine.rs	Instruments sandbox execution and compilation; adds debug logs for env injection, trigger selection, AOT enabling, and precompile metrics.
containerd-shim-spin/Cargo.toml	Enables tracing feature on containerd-shim-wasm and adds a direct tracing dependency; removes spin-telemetry.
Cargo.lock	Updates lockfile for the dependency changes (adds tracing, removes spin-telemetry).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kate-goldenring]

@TheRayquaza have you tested the tracing locally at all? I cannot seem to get the shim startup traces to be captured. My set up is the following:

### 1. Build and install the shim

```sh
cargo build --release
sudo cp target/release/containerd-shim-spin-v2 /usr/bin/containerd-shim-spin-v2

2. Start a local Jaeger instance

Jaeger's all-in-one image includes an OTLP collector and a web UI:

docker run -d --name jaeger \
  --network host \
  -e COLLECTOR_OTLP_ENABLED=true \
  jaegertracing/all-in-one:latest

The Jaeger UI is available at http://localhost:16686.

3. Configure containerd to enable tracing

Set the OTLP endpoint and increase log verbosity on the containerd service:

sudo systemctl stop containerd
sudo systemctl edit containerd

Add to the override file:

[Service]
Environment="OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318"
Environment="RUST_LOG=debug"

Port note: Use port 4318 (HTTP/protobuf) not 4317 (gRPC). The shim defaults to http/protobuf — pointing it at the gRPC port causes an invalid HTTP version error. To use port 4317 instead, also add Environment="OTEL_EXPORTER_OTLP_PROTOCOL=grpc".

Restart containerd:

sudo systemctl restart containerd

4. Pull and run a Spin workload

sudo ctr image pull ghcr.io/spinkube/spin-operator/cpu-load-gen:20240311-163328-g1121986
sudo ctr run --env OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 --rm --net-host --runtime io.containerd.spin.v2 \
  ghcr.io/spinkube/spin-operator/cpu-load-gen:20240311-163328-g1121986 \
  my-spin-task bogus-arg

@Mossaka is likely the best to look into these tracing enhancements

[TheRayquaza]

Hey @kate-goldenring sorry for reaching late, I had a couple of issues and struggled a lot understanding the different OTEL exporters context (spin shim vs runwasi one). I finally made it work. Note that I have a similar env as yours, except that I am using our product datadog (which should not change anything actually):

Btw I ran into an issue which I believe was due to the runwasi exporter not handling https endpoints but in fact this shim has its own exporter with the spin-telemetry crate also used for spin app (correct me if I am wrong).
Turns out, spin also seems to have some issues with https endpoint, I have prepared another PR in the telemetry lib to fix that (spinframework/spin#3462)