Skip to content

strkey: avoid unnecessary allocation in decodeString by not reusing srcBytes for decode #5936

Description

@leighmcculloch

What problem does your feature solve?

In strkey/main.go, the decodeString function converts the input string to a byte slice early on:

srcBytes := []byte(src)

Then later reuses that same slice as both source and destination for base32 decoding:

n, err := base32.StdEncoding.WithPadding(base32.NoPadding).Decode(srcBytes, srcBytes)

Since Go 1.22, []byte(string) does not allocate if the resulting byte slice is never mutated. However, because srcBytes is mutated by the in-place Decode call, the compiler is forced to allocate a copy of the string upfront.

If the decode step used a separate, right-sized destination buffer instead of reusing srcBytes, the initial []byte(src) conversion could avoid allocation entirely on Go 1.22+. The new destination buffer would also be smaller (decoded size < encoded size), reducing total memory usage.

Related: #5935 (adds an early length check to decodeString, but doesn't address this allocation concern).

What would you like to see?

Change decodeString to allocate a separate destination slice for the Decode call, sized to the decoded length, instead of reusing srcBytes. For example:

srcBytes := []byte(src)
// ... validation ...
decodedLen := encoding.DecodedLen(len(srcBytes))
decoded := make([]byte, decodedLen)
n, err := encoding.Decode(decoded, srcBytes)
if err != nil {
    return nil, errors.Wrap(err, "base32 decode failed")
}
return decoded[:n], nil

This way:

  1. []byte(src) becomes a no-op on Go 1.22+ (no allocation, no copy) since srcBytes is only read, never written.
  2. The decoded buffer is allocated at exactly the size needed for the decoded output, which is smaller than the encoded input.
  3. Total memory allocated per call is reduced.

What alternatives are there?

Keep the current in-place decode. It works correctly and the allocation cost is small for typical strkey sizes (~56 bytes encoded). The optimization mainly matters for high-throughput paths that decode many strkeys.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions