Skip to content

Commit 50425c9

Browse files
authored
fix(awk): make expression format errors fatal (#2458)
## What changed AWK formatting errors inside `sprintf` expressions now stop the AWK program with exit 2 and a fatal diagnostic. Direct `printf` uses the same error behavior. Scripts and agents no longer receive a successful result after a rejected format width or precision. Closes #2456. ## Why `sprintf("%10001s", "x")` printed an error but yielded an empty string and exit 0. A later calculation could look complete although the format cap had been hit. ## Before / After Before: `awk 'BEGIN { s=sprintf("%10001s","x"); print length(s) }'` printed `0`, reported the width error, and exited 0. After: the same command prints no stdout, reports `awk: fatal: format width 10001 exceeds maximum (10000)`, and exits 2. The focused `builtin_cap_security_tests::awk_format` suite passes all 3 tests; the bash spec suite and 100 awk unit tests pass. ## Risk - Low. Programs that relied on successful exit after an invalid AWK format now receive exit 2. Formatting within the cap is unchanged. - Local `just pre-pr` exposed two SQLite CSV differential failures against macOS host `sqlite3`; this PR does not change SQLite. Required Linux CI is the merge gate. ## Checklist - [x] Tests added or updated - [x] Backward compatibility considered
1 parent 51f5d00 commit 50425c9

5 files changed

Lines changed: 60 additions & 14 deletions

File tree

‎crates/bashkit/docs/threat-model.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ through configurable limits.
109109
| source self-recursion (TM-DOS-056) | Script that sources itself | Track source depth | **MITIGATED** |
110110
| sleep bypasses timeout (TM-DOS-057) | `sleep N` ignores `ExecutionLimits::timeout` | Implement tokio timeout wrapper | **PARTIAL** |
111111
| Unbounded builtin output (TM-DOS-058) | `seq 1 1000000` produces 1M lines | Add `max_stdout_bytes` limit | **MITIGATED** |
112-
| Silent truncation at builtin caps (TM-DOS-109) | `seq 200000` or an awk loop past its cap returns cut output with exit 0 | Caps report `<cmd>: <what> limit (<N>) exceeded` on stderr and exit non-zero; awk caps are fatal | **MITIGATED** |
112+
| Silent truncation at builtin caps (TM-DOS-109) | `seq 200000`, an awk loop past its cap, or an oversized `sprintf` expression returns incomplete output with exit 0 | Caps report `<cmd>: <what> limit (<N>) exceeded` on stderr and exit non-zero; awk caps and formatting errors are fatal | **MITIGATED** |
113113
| In-builtin memory growth (TM-DOS-110) | `awk 'BEGIN { s = "x"; while (1) s = s s }'` or `jq -n '"x" \| until(false; . + .)'` allocates until the host aborts | awk checks each string against a 16 MiB cap before allocating it, caps `$N` field indexes, and caps total variable memory at `max_live_intermediate_bytes` (fatal, exit 2). jq meters every live string, array and object against the same limit and fails before growing (exit 5); non-emitting jq loops stop at the timeout | **MITIGATED** |
114114
| Param expansion bomb (TM-DOS-059) | `${x//a/bigstring}` multiplicative amplification | `max_total_variable_bytes` + `max_stdout_bytes` | MITIGATED |
115115
| Sparse array huge-index (TM-DOS-060) | `arr[999999999]=x` | HashMap storage; `max_array_entries` | MITIGATED |

‎crates/bashkit/src/builtins/awk/interpreter.rs‎

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -840,12 +840,13 @@ impl AwkInterpreter {
840840
}
841841
let format = self.eval_expr(&args[0]).as_string();
842842
let values: Vec<AwkValue> = args[1..].iter().map(|a| self.eval_expr(a)).collect();
843+
// THREAT[TM-DOS-109]: expression errors must poison the run;
844+
// returning an empty string with exit 0 hides a failed cap.
843845
match self.format_string(&format, &values) {
844846
Ok(s) => AwkValue::String(s),
845847
Err(FormatError::Message(e)) => {
846-
self.stderr_output.push_str(&e);
847-
self.stderr_output.push('\n');
848-
AwkValue::String(String::new())
848+
self.fatal(&e);
849+
AwkValue::Uninitialized
849850
}
850851
Err(FormatError::TooLarge) => {
851852
self.string_fits(usize::MAX);
@@ -1244,7 +1245,7 @@ impl AwkInterpreter {
12441245
{
12451246
if w_val > Self::MAX_FORMAT_WIDTH {
12461247
return Err(FormatError::Message(format!(
1247-
"awk: format width {} exceeds maximum ({})",
1248+
"format width {} exceeds maximum ({})",
12481249
w_val,
12491250
Self::MAX_FORMAT_WIDTH
12501251
)));
@@ -1269,7 +1270,7 @@ impl AwkInterpreter {
12691270
} else if let Ok(p_val) = p.parse::<usize>() {
12701271
if p_val > Self::MAX_FORMAT_WIDTH {
12711272
return Err(FormatError::Message(format!(
1272-
"awk: format precision {} exceeds maximum ({})",
1273+
"format precision {} exceeds maximum ({})",
12731274
p_val,
12741275
Self::MAX_FORMAT_WIDTH
12751276
)));
@@ -1542,11 +1543,7 @@ impl AwkInterpreter {
15421543
}
15431544
AwkFlow::Continue
15441545
}
1545-
Err(FormatError::Message(e)) => {
1546-
self.stderr_output.push_str(&e);
1547-
self.stderr_output.push('\n');
1548-
AwkFlow::Exit(Some(2))
1549-
}
1546+
Err(FormatError::Message(e)) => self.fatal(&e),
15501547
Err(FormatError::TooLarge) => {
15511548
self.string_fits(usize::MAX);
15521549
AwkFlow::Exit(Some(2))

‎crates/bashkit/tests/integration/builtin_cap_security_tests.rs‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,55 @@ mod awk_loops {
126126
}
127127
}
128128

129+
mod awk_format {
130+
use super::*;
131+
132+
#[tokio::test]
133+
async fn sprintf_error_in_expression_aborts_the_program() {
134+
for body in [
135+
"s = sprintf(\"%10001s\", \"x\"); print length(s)",
136+
"print length(sprintf(\"%10001s\", \"x\"))",
137+
"if (sprintf(\"%10001s\", \"x\")) print \"after\"",
138+
] {
139+
let script = format!("awk 'BEGIN {{ {body} }} END {{ print \"end\" }}'");
140+
let r = run(&script).await;
141+
assert_eq!(r.exit_code, 2, "{body}: exit status");
142+
assert_eq!(r.stdout, "", "{body}: no output after error");
143+
assert_eq!(
144+
r.stderr, "awk: fatal: format width 10001 exceeds maximum (10000)\n",
145+
"{body}: diagnostic"
146+
);
147+
}
148+
}
149+
150+
#[tokio::test]
151+
async fn sprintf_width_at_cap_succeeds() {
152+
let r = run("awk 'BEGIN { s = sprintf(\"%10000s\", \"x\"); print length(s) }'").await;
153+
assert_eq!(r.exit_code, 0);
154+
assert_eq!(r.stdout, "10000\n");
155+
assert_eq!(r.stderr, "");
156+
}
157+
158+
#[tokio::test]
159+
async fn precision_and_printf_errors_are_fatal_too() {
160+
for (body, diagnostic) in [
161+
(
162+
"print length(sprintf(\"%.10001s\", \"x\"))",
163+
"awk: fatal: format precision 10001 exceeds maximum (10000)\n",
164+
),
165+
(
166+
"printf \"%10001s\", \"x\"",
167+
"awk: fatal: format width 10001 exceeds maximum (10000)\n",
168+
),
169+
] {
170+
let r = run(&format!("awk 'BEGIN {{ {body} }} END {{ print \"end\" }}'")).await;
171+
assert_eq!(r.exit_code, 2, "{body}: exit status");
172+
assert_eq!(r.stdout, "", "{body}: no output after error");
173+
assert_eq!(r.stderr, diagnostic, "{body}: diagnostic");
174+
}
175+
}
176+
}
177+
129178
mod awk_getline {
130179
use super::*;
131180

‎crates/bashkit/tests/spec_cases/bash/awk-printf-width.test.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
echo "" | awk '{printf "%999999999d", 1}' 2>&1
55
echo "exit: $?"
66
### expect
7-
awk: format width 999999999 exceeds maximum (10000)
7+
awk: fatal: format width 999999999 exceeds maximum (10000)
88
exit: 2
99
### end
1010

@@ -28,6 +28,6 @@ echo "" | awk '{printf "%10000d\n", 1}' | wc -c
2828
echo "" | awk '{printf "%.999999999f", 1}' 2>&1
2929
echo "exit: $?"
3030
### expect
31-
awk: format precision 999999999 exceeds maximum (10000)
31+
awk: fatal: format precision 999999999 exceeds maximum (10000)
3232
exit: 2
3333
### end

‎knowledge/security/threat-model.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -276,7 +276,7 @@ runaway scripts without permanently breaking the session.
276276
| TM-DOS-061 | Snapshot function restore bypasses parser/function limits | Crafted snapshot restores functions exceeding the tenant's parser depth or function memory budget | Restored function source re-parsed with current `ExecutionLimits`; function memory budget re-applied before insertion | **MITIGATED** |
277277
| TM-DOS-062 | jq file binding amplification | Repeated `--rawfile` / `--slurpfile` bindings to one max-sized VFS file multiply retained jq globals and `$ARGS.named` values without consuming more VFS quota | `MAX_FILE_VAR_REQUESTS` caps binding count; `MAX_FILE_VAR_BYTES` counts cumulative file bytes per binding before retaining globals | **MITIGATED** |
278278
| TM-DOS-063 | Persistent fd exhaustion | `exec N>/tmp/f` across many `N` values grows `exec_fd_table`/coproc fd buffers for the session | `ExecutionLimits::max_file_descriptors` (default 1024) caps persistent custom descriptors; reused fds and standard 0/1/2 don't count | **MITIGATED** |
279-
| TM-DOS-109 | Silent truncation at builtin caps | A builtin that stops at a resource cap but exits 0 with no diagnostic (awk loops/recursion/getline caps, `seq`, `yes`, `numfmt`, `history`) returns cut output that scripts and agents treat as complete (#2446, like sed #1005) | Reaching a builtin cap is never silent: `builtins::limits::cap_exceeded` keeps the partial output, writes `<cmd>: <what> limit (<N>) exceeded; output truncated` and exits 1; awk caps are gawk-style fatal errors (`awk: fatal: ...`, exit 2, END skipped). Regression tests: `builtin_cap_security_tests` | **MITIGATED** |
279+
| TM-DOS-109 | Silent truncation at builtin caps | A builtin that stops at a resource cap but exits 0 with no diagnostic (awk loops/recursion/getline caps, `seq`, `yes`, `numfmt`, `history`), or an awk `sprintf` expression that reports a format cap yet returns an empty value, gives scripts and agents incomplete output as success (#2446, #2456, like sed #1005) | Reaching a builtin cap is never silent: `builtins::limits::cap_exceeded` keeps the partial output, writes `<cmd>: <what> limit (<N>) exceeded; output truncated` and exits 1; awk caps and formatting errors are gawk-style fatal errors (`awk: fatal: ...`, exit 2, END skipped). Regression tests: `builtin_cap_security_tests` | **MITIGATED** |
280280
| TM-DOS-110 | In-builtin memory growth aborts the host | A value that keeps growing inside awk (`s = s s`, `gsub` with a long replacement, `sprintf("%s%s%s", s, s, s)`, `$1000000000 = 1`, filling arrays, deep recursion shadowing large values) or jq (`until(false; . + .)`, `"x" * 1e12`, `[range(1e12)]`, a non-emitting `until(false; .)` growing evaluator state) allocates until the allocator fails and the embedding process aborts with SIGABRT (#2444) | awk: every string is checked against `AWK_MAX_STRING_BYTES` (16 MiB) *before* it is allocated (concat, `SUBSEP` keys, `sub`/`gsub`/`gensub` as the result grows, `sprintf`/`printf`, `print` joins, rebuilt `$0`); `$N` assignment is capped at `AWK_MAX_FIELD_INDEX`; variables, array elements and values parked in call frames are accounted and capped at `ExecutionLimits::max_live_intermediate_bytes` (default 32 MB); each is a fatal awk error (exit 2) and the script carries on. jq: jaq-json is vendored with a live-memory meter charging every string and array/object body to the same limit, checked before growth (`jq: error: value size limit (N bytes) exceeded`, exit 5), sticky across `try`; value operations poll the execution deadline so non-emitting loops stop at the timeout; output conversion is capped by `max_stdout_bytes`. Regression tests: `memory_growth_security_tests`. Residual: recursion without value operations (`def f: f; f`) and dropping very deep values overflow the stack | **MITIGATED** |
281281
| TM-DOS-101 | yq structured-data amplification | YAML aliases can expand exponentially during deserialization; deep YAML/JSON, document floods, jaq generators, and YAML re-serialization can consume stack, CPU, or memory beyond the source size | Reject YAML alias tokens with a bounded lexical pass before deserialization; VFS/stdin and aggregate input budgets; serde_yaml_ng recursion cap 128 plus Bashkit depth 100; 4096-document cap; shared jaq work/deadline/output limits; post-serialization stdout cap; YAML+JSON depth/document/input/output regressions plus `yq_fuzz` and arbitrary-input proptest | **MITIGATED** |
282282
| TM-DOS-107 | Static-analysis command validation amplification | A large comment before thousands of commands makes a whole-script ordered-subsequence scan per command | Build one source-character position index, then validate each analyzed command-name character with a logarithmic position lookup instead of rescanning source | **MITIGATED** |

0 commit comments

Comments
 (0)