Skip to content

Poison-pill SQS message can crash ingestion handler before partial-batch failure handling #824

Description

@m75gyqbsrw-boop

Summary

A malformed SQS message body can throw an uncaught exception in ingestion-lambda, causing the whole invocation to fail before it returns batchItemFailures. This enables a poison-pill DoS where one bad message repeatedly retries and blocks/co-retries healthy messages in the same batch.

Evidence in code

ingestion-lambda/src/handler.ts

  1. processSQSRecord does raw parse with no try/catch:
  • const { externalId, objectKey } = JSON.parse(record.body) as ...
  1. main processes records via Promise.all(records.map(async ...)).
  • If processSQSRecord throws, the mapped promise rejects.
  • Promise.all(...) rejects immediately.
  • Handler exits without constructing per-item batchItemFailures response.
  1. Existing per-record failure pathway (failureWith(...)) only handles returned OperationResult failures, not thrown parse exceptions from processSQSRecord.

Why this matters

  • One malformed SQS body can fail the entire Lambda invocation.
  • Good records in the same batch are retried unnecessarily.
  • In sustained conditions, this can create queue backlog and ingestion delay (availability impact).

Repro (conceptual)

Send a record with non-JSON body (e.g. body: "{") into ingestion queue.
Expected with current code path:

  • JSON.parse throws in processSQSRecord
  • invocation fails instead of returning { batchItemFailures: [{ itemIdentifier: badMessageId }] }

Recommended fix

  1. Make processSQSRecord exception-safe:
    • Wrap JSON.parse in try/catch and return status: 'failure' with reason for malformed body.
  2. Keep per-record isolation:
    • Ensure all record-level parsing/validation errors become BatchItemFailure entries, not invocation-level exceptions.
  3. Add regression tests:
    • mixed batch (1 malformed + 1 valid) should return only malformed message in batchItemFailures.
    • valid record should still process successfully in same invocation.

Severity

Medium-High (availability / queue processing reliability).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions