Skip to content

fix: stitch multi-line stack traces into one log event in observability-logs-openobserve #281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TemRevil wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: stitch multi-line stack traces into one log event in observability-logs-openobserve #281

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions observability-logs-openobserve/helm/templates/fluent-bit/config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,15 @@ data:
Skip_Long_Lines On
Tag kube.*

[FILTER]
Name multiline
Match kube.*
multiline.key_content log
mode parser
multiline.parser go, java, python, dotnet
buffer On
flush_ms 2000

[FILTER]
Name kubernetes
Buffer_Size 15MB
Expand Down Expand Up @@ -62,4 +71,21 @@ data:
Time_Keep Off
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L

# The go, java and python multiline parsers referenced by the multiline
# filter above are built into Fluent Bit. .NET is not, so it is defined
# here as a custom parser. A .NET stack trace starts with a line that
# ends in "Exception: <message>" and continues with " at ..." frames and
# "--- End of ... stack trace ---" markers.
#
# To support another log format not covered by the built-ins, add a
# [MULTILINE_PARSER] here (type regex) and append its name to
# `multiline.parser` on the multiline filter. See
# https://docs.fluentbit.io/manual/data-pipeline/filters/multiline-stacktrace
[MULTILINE_PARSER]
name dotnet
type regex
flush_timeout 2000
rule "start_state" "/^.*Exception[:,].*$/" "cont"
rule "cont" "/^\s+(at|--- End of).*$/" "cont"
Comment on lines +89 to +90

@coderabbitai coderabbitai Bot Jul 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Add a word boundary after at in the continuation regex to prevent false-positive matches.

The cont rule /^\s+(at|--- End of).*$/ matches any line starting with whitespace followed by the literal at, including non-stack-frame lines like atmosphere or attached. Since this only applies after an exception start line, real-world risk is low — but adding at\s (or \bat\b) is a one-character fix that prevents incorrect log grouping in edge cases.

🛡️ Proposed fix 
-        rule          "cont"        "/^\s+(at|--- End of).*$/" "cont"
+        rule          "cont"        "/^\s+(at\s|--- End of).*$/" "cont"
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
rule "start_state" "/^.*Exception[:,].*$/" "cont"
rule "cont" "/^\s+(at|--- End of).*$/" "cont"
rule "start_state" "/^.*Exception[:,].*$/" "cont"
rule "cont" "/^\s+(at\s|--- End of).*$/" "cont"
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@observability-logs-openobserve/helm/templates/fluent-bit/config.yaml` around
lines 89 - 90, Update the continuation regex in the “cont” rule to require a
word boundary or whitespace after “at” (for example, `at\s` or `\bat\b`), while
preserving the existing `--- End of` matching.

{{- end }}