struct-compile

Installation

npm i struct-compile --save

Overview

This project is used internally by over-the-wire. I'm not sure if it has any use outside its main purpose.

The goal of this project is to generate efficient binary parsers and serializers in JavaScript directly from C-style structure declarations.

Two modes are available:

• Class mode (compile) – returns a Struct class per declaration. The class lazily reads fields on access through Object.defineProperty getters; great when only a few fields of a large struct are inspected at runtime.
• Fast mode (compileFast) – returns plain { size, decode, encode, decodeLE, decodeBE, encodeLE, encodeBE } objects per declaration. The decode/encode functions are emitted as inline new Functions that produce monomorphic object literals with constant-offset Buffer reads. This is roughly an order of magnitude faster than the class-based path on hot loops, at the cost of allocating the full record on every decode and not supporting bit fields or arrays.

System Requirements

• Node.js >= 16.18.0

Getting Started — Class Mode

compile(string, [arch], [BufferImpl]) => { [StructName]: StructClass }

import { compile } from 'struct-compile';
// CommonJS: const { compile } = require('struct-compile');

const { Data, PDU } = compile(`
  // simple example
  struct Data {
    uint8_t c;
    int v;
    unsigned long da;
  };

  //@NE Network endianness for all members of this struct
  struct __attribute__((__packed__)) PDU {
    // Some useful comment
    char name /*in-between comment*/ [16];
    double dbl;
    int p;
  };
`);

// Construct an empty record
const obj = new PDU();
obj.name = 'seva';
obj.dbl  = 1.1;

console.log('PDU size:   ', obj.length);
console.log('PDU buffer: ', obj.buffer);

// Parse raw binary
const parsed = new PDU(Buffer.from([
  0x73, 0x65, 0x76, 0x61, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x3f, 0xf1, 0x99, 0x99, 0x99, 0x99, 0x99, 0x9a,
  0x00, 0x00, 0x00, 0x00,
]));

console.log(parsed.name.toString());
console.log(parsed.dbl);

The struct syntax follows C alignment rules; auxiliary comments control endianness. See the references on Microsoft's padding and alignment guide and GCC type attributes.

Getting Started — Fast Mode

compileFast(string, [arch]) => { [StructName]: { size, decode, encode, decodeLE, decodeBE, encodeLE, encodeBE } }

import { compileFast } from 'struct-compile';
// CommonJS: const { compileFast } = require('struct-compile');

const { PacketHeader } = compileFast(`
  //@LE
  struct __attribute__((__packed__)) PacketHeader {
    uint32_t tv_sec;
    uint32_t tv_usec;
    uint32_t caplen;
    uint32_t len;
  };
`);

console.log(PacketHeader.size); // 16

// Decode — returns a plain object with a fixed shape, no class instance
const buf = Buffer.from('64fa125e000000006000000060000000', 'hex');
const record = PacketHeader.decode(buf, 0);
// → { tv_sec: 1578302052, tv_usec: 0, caplen: 96, len: 96 }

// Encode — writes scalars into a caller-provided Buffer
const out = Buffer.allocUnsafe(PacketHeader.size);
PacketHeader.encode(out, 0, { tv_sec: 1, tv_usec: 2, caplen: 60, len: 60 });

// Endianness can be overridden at the call site
PacketHeader.encodeBE(out, 0, { tv_sec: 1, tv_usec: 2, caplen: 60, len: 60 });
const r = PacketHeader.decodeBE(out, 0);

decode / encode honour the per-field endianness pinned via //@LE / //@BE comments (or the struct-level default); decodeLE/encodeLE/decodeBE/encodeBE force a single endianness across all fields, which is convenient when the same wire layout is used in both byte orders (e.g. classic pcap).

When to pick which

Scenario	Recommended
Tight parse loops on packed binary protocols	Fast mode
Random read of a few fields out of a large record	Class mode (lazy)
Mutation of fields on a backing buffer in place	Class mode
Need bit fields or arrays	Class mode
You only need { size, decode, encode } from an AST	fastFromConfig

fastFromConfig(config, [arch]) accepts a single struct AST (the same shape compileFast produces internally) and is useful when you want to skip the C-source parser entirely and build the layout programmatically.

Compatibility and Project Checklist

The input syntax is a subset of C; current grammar can be viewed here.

Feature	Class mode (compile)	Fast mode (compileFast)	Notes
C structure parsing	✅	✅
Binary data creation	✅	✅
Endianness setting	✅	✅	Via //@LE / //@BE / //@NE comments
Bit fields	✅	❌	Throws at compile time in fast mode
Arrays	✅	❌	Throws at compile time in fast mode
Nested structures	❌	❌
Enums	❌	❌
Browser support	❌	❌	Node.js Buffer API is required

Contributions or suggestions for the unsupported features are welcome.

Questions or Suggestions

If you have any ideas, or something is not working correctly, feel free to open an issue.