Nanfeng

Notes on software development, code, and curious ideas

Data Files and Serialization in Lua

In
Reading data safely and serializing primitive values, tables, shared references, and cycles in Lua.

Writing a data file is easier than reading one because the writer controls its format. A robust reader must reject malformed, oversized, or unexpected input without leaving partially applied state.

Lua as a data format

A trusted configuration file can return a table:

1
2
3
4
5
6
-- settings.lua
return {
    width = 1280,
    height = 720,
    fullscreen = false
}

Load it with a minimal environment:

1
2
3
4
5
local chunk, err = loadfile("settings.lua", "t", {})
assert(chunk, err)
local ok, settings = pcall(chunk)
assert(ok, settings)
assert(type(settings) == "table", "settings must return a table")

An empty environment reduces available capabilities, but executing data as code is still a poor choice for hostile input. Use JSON or another dedicated parser with limits when data crosses a trust boundary.

Serializing primitive values

string.format("%q", value) produces a quoted representation for strings. Numbers, booleans, and nil can use tostring, with special handling if non-finite numbers must round-trip.

1
2
3
4
5
6
7
8
local function serializePrimitive(value)
    local kind = type(value)
    if kind == "string" then return string.format("%q", value) end
    if kind == "number" or kind == "boolean" or kind == "nil" then
        return tostring(value)
    end
    error("unsupported type: " .. kind)
end

Tables without cycles

1
2
3
4
5
6
7
8
9
10
11
12
13
local function serialize(value, seen)
    if type(value) ~= "table" then return serializePrimitive(value) end
    seen = seen or {}
    if seen[value] then error("cycle detected") end
    seen[value] = true

    local fields = {}
    for key, item in pairs(value) do
        fields[#fields + 1] = "[" .. serialize(key, seen) .. "]=" .. serialize(item, seen)
    end
    seen[value] = nil
    return "{" .. table.concat(fields, ",") .. "}"
end

This handles tree-shaped tables, but iteration order is not deterministic and metatables are intentionally omitted.

Shared references and cycles

To preserve a general table graph, assign every table an ID, emit constructors first, and then emit assignments between IDs. This two-pass representation can rebuild shared references and cycles. It also needs explicit policies for table keys, metatables, functions, userdata, maximum depth, and output size.

Serialization is a protocol, not just a recursive function. Version the format, validate the decoded structure, write files atomically when possible, and never assume syntactically valid input is semantically safe.

+