Nanfeng

Notes on software development, code, and curious ideas

Bits, Bytes, Packing, and Unpacking in Lua

In
Working with binary data through bitwise operators, byte strings, string.pack, and string.unpack.

Lua strings can contain arbitrary bytes, including zero. That makes them suitable for binary payloads, while Lua 5.3 and later provide native bitwise operators and packing functions.

Bitwise operations

1
2
3
4
5
6
7
local flags = 0
flags = flags | 0x04       -- set a bit
local enabled = flags & 0x04 ~= 0
flags = flags & ~0x04      -- clear a bit

local high = (0xABCD >> 8) & 0xFF
local low = 0xABCD & 0xFF

Operators include AND (&), OR (|), exclusive OR (~), NOT (~), shifts (<<, >>), and integer floor division. Exact integer width depends on how Lua was compiled. Lua 5.2 commonly uses the bit32 library instead.

Bytes in strings

1
2
3
local payload = string.char(0x41, 0x00, 0x42)
print(#payload)                    -- 3
print(string.byte(payload, 1, 3)) -- 65 0 66

Do not use APIs that assume zero-terminated C strings when transferring such data through native code.

Packing structured binary data

1
2
local packet = string.pack(">I2I4c4", 7, 1024, "PING")
local kind, size, label, nextIndex = string.unpack(">I2I4c4", packet)

The > prefix selects big-endian order; < selects little-endian. I2 and I4 are unsigned integers of two and four bytes, while c4 is a fixed four-byte string. Always specify byte order and widths in a protocol instead of relying on native layout.

Validate lengths before unpacking data received from outside the process, place upper bounds on allocations, and treat decoding errors as normal input failures. Binary formats are compact, but they require an explicit schema and versioning strategy.

+