Schema Definition

Binary data encoding/decoding in Byteform is done using schemas. A schema is a set of rules that define how data is encoded and decoded. Schemas are defined using a simple, human-readable syntax:

import { Struct, List, text, f32, u8 } from '@evenstar/byteform';

const schema = new Struct({
  name: text, // string
  age: u8, // Unsigned 8-bit integer, e.g. number between 0 and 255
  scores: new List(f32), // List of 32-bit floating point numbers
});

In the example above, we define a schema with three fields: name, age, and scores.

info

As you might have noticed, some types are defined using a constructor function, while others are defined using a shorthand. This is because some field types have additional configuration options that can be passed to the constructor.

However, looking ahead, constructable types start with the Uppercase letter, while shorthand types start with the lowercase letter.

Byteform provides a set of built-in types that can be used to define schemas. We will cover the different types and their configuration options in the following sections.

---

Struct

Struct is a collection of fields. It's a synonym for a JavaScript object.

It is used to group multiple fields together. A struct can contain any number of fields of any type.

The constructor takes an object as an argument, where each key-value pair represents a field in the struct. There are a few examples of how to define a struct:

Simple

import { Struct, text, u8 } from '@evenstar/byteform';

/**
 * Define a schema with two fields: name and age.
 * Similar to JavaScript object: { name: 'Alice', age: 25 }
 */
const schema = new Struct({
  name: text, // string
  age: u8, // Unsigned 8-bit integer, e.g. number between 0 and 255
});

Nested

import { Struct, text, u8 } from '@evenstar/byteform';

/**
 * Define a schema with three fields: name, age, and address.
 * Similar to JavaScript object:
 * { 
 *    name: 'Alice',
 *    age: 25,
 *    address: {
 *      street: '123 Main St',
 *      city: 'New York',
 *      zip: '10001' 
 *    }
 *  }
 */
const schema = new Struct({
  name: text, // string
  age: u8, // Unsigned 8-bit integer, e.g. number between 0 and 255
  address: new Struct({ // Nested struct
    street: text,
    city: text,
    zip: text,
  }),
});

With TypeScript

import { Struct, text } from '@evenstar/byteform';

// Define a TypeScript type
type Person = {
  name: string;
  age: number;
};

/**
 * Validate the schema against the TypeScript type.
 * If the schema definition does not match the TypeScript type, a type error will be thrown.
 */
const schema = new Struct<Person>({
  name: text,
  age: text // TypeError: age should have a type that implements number encoding/decoding.
});

---

List

List is a collection of elements of the same type. It's a synonym for a JavaScript array.

Like a struct, list is used to group multiple elements together. A list can contain any number of elements.

The constructor takes a single argument, which is the type of elements in the list. There are a few examples of how to define a list:

Simple

import { List, u8 } from '@evenstar/byteform';

/**
 * Define a schema with a list of unsigned 8-bit integers.
 * Similar to JavaScript array: [0, 1, 2, 3, 4]
 */
const schema = new List(u8); // Unsigned 8-bit integer, e.g. number between 0 and 255

Nested

import { List, u8 } from '@evenstar/byteform';

/**
 * Define a schema with a nested list of unsigned 8-bit integers.
 * Similar to JavaScript array: [[0, 1, 2], [3, 4, 5]]
 */
const schema = new List(new List(u8)); // Nested list of unsigned 8-bit integers

With TypeScript

import { List, u8 } from '@evenstar/byteform';

/**
 * Validate the schema against the TypeScript type.
 * If the schema definition does not match the TypeScript type, a type error will be thrown.
 */
const names = new List<string>(u8); // TypeError: List data type should implement string encoding/decoding.

---

Dictionary

Dictionary is a collection of key-value pairs. It's a synonym for a JavaScript Map.

It is used to store associative data where each value is accessed by a unique key. A dictionary can contain any number of key-value pairs of any type.

The constructor takes three arguments:

• keySchema: The schema for the keys in the dictionary
• valueSchema: The schema for the values in the dictionary
• sizeSchema: The schema for the size of the dictionary (optional, defaults to u32)

Simple

import { Dictionary, u8, f64 } from '@evenstar/byteform';

/**
 * Define a schema for a dictionary with number keys and float values.
 * Similar to JavaScript Map: new Map([[1, 1.5], [2, 2.4], [3, 3.6]])
 */
const schema = new Dictionary(u8, f64);

Mixed Types

import { Dictionary, text, f64, u8 } from '@evenstar/byteform';

/**
 * Define a schema for a dictionary with string keys and float values.
 * Similar to JavaScript Map: new Map([['x', 1.5], ['y', 2.4], ['z', 3.6]])
 */
const schema = new Dictionary(text, f64);

/**
 * Define a schema for a dictionary with number keys and string values.
 * Similar to JavaScript Map: new Map([[1, 'one'], [2, 'two'], [3, 'three']])
 */
const stringValues = new Dictionary(u8, text);

Custom Size Schema

import { Dictionary, u8, f64, u16 } from '@evenstar/byteform';

/**
 * Define a schema for a dictionary with custom size encoding.
 * Uses u16 for size instead of the default u32, useful for smaller dictionaries.
 */
const schema = new Dictionary(u8, f64, u16);

Nested

import { Dictionary, text, Struct, u8, f64 } from '@evenstar/byteform';

/**
 * Define a schema for a dictionary with string keys and struct values.
 * Similar to JavaScript Map: new Map([
 *   ['player1', { id: 1, score: 100.5 }],
 *   ['player2', { id: 2, score: 85.2 }]
 * ])
 */
const playerSchema = new Struct({
  id: u8,
  score: f64,
});

const schema = new Dictionary(text, playerSchema);

With TypeScript

import { Dictionary, text, f64 } from '@evenstar/byteform';

// Define a TypeScript type
type Config = Map<string, number>;

/**
 * Validate the schema against the TypeScript type.
 * The schema will encode/decode Map<string, number> data.
 */
const schema = new Dictionary<Config>(text, f64);

---

Optional

Optional is a wrapper type that can represent a value that may or may not be present. It's similar to the concept of null in JavaScript.

The constructor takes a single argument, which is the type of elements in the list. There are a few examples of how to define an optional:

Simple

import { Optional, Struct, f32 } from '@evenstar/byteform';

const vec3 = new Struct({
  x: f32,
  y: f32,
  z: f32,
});

/**
 * Define a schema with an optional vector3.
 * Similar to JavaScript vec3 | null
 */
const schema = new Optional(vec3);

Nested

import { Optional, List, Struct, f32 } from '@evenstar/byteform';

const vec3 = new Struct({
  x: f32,
  y: f32,
  z: f32,
});

/**
 * Define a schema with a nested optional vector3.
 * Similar to JavaScript array: [{x: 0, y: 1, z: 2}, null, null, {x: 3, y: 4, z: 5}]
 */
const schema = new List(new Optional(vec3)); // Nested optional vector3

With TypeScript

import { Optional, Struct, f32 } from '@evenstar/byteform';

const vec3 = new Struct({
  x: f32,
  y: f32,
  z: f32,
});

/**
 * Validate the schema against the TypeScript type.
 * If the schema definition does not match the TypeScript type, a type error will be thrown.
 */
const names = new Optional<string>(vec3); // TypeError: Optional data type should implement vec3 | null encoding/decoding.

---

Str

Str is a type that represents a fixed-length string. It's a synonym for a JavaScript string.

The constructor takes a single argument, which is the byte length of the string. There are a few examples of how to define a fixed-length string:

import { Str } from '@evenstar/byteform';

const schema = new Str(5); // Fixed-length string of 5 bytes

schema.write(writer, "Hello, World!"); // Writes -> Hello (5 bytes)
schema.read(reader); // Reads -> Hello

schema.write(writer, "Hi"); // Writes -> Hi\0 (2 bytes + null terminator)
schema.read(reader); // Reads -> Hi

---

Text

Byteform provides a text schema type to represent a variable-length string. It's a synonym for a JavaScript string.

import { text } from '@evenstar/byteform';

const schema = text;

---

Number Types

Byteform provides an extensive set of number types.

Shortcuts

Type	Description	Size in bytes	Value Range
u8	Unsigned 8-bit integer	1	0 to 255
u16	Unsigned 16-bit little-endian integer	2	0 to 65535
u32	Unsigned 32-bit little-endian integer	4	0 to 4294967295
u64	Unsigned 64-bit little-endian integer	8	0 to 2^264 - 1
i8	Signed 8-bit integer	1	-128 to 127
i16	Signed 16-bit little-endian integer	2	-32768 to 32767
i32	Signed 32-bit little-endian integer	4	-2147483648 to 2147483647
i64	Signed 64-bit little-endian integer	8	-2^63 to 2^63 - 1
f32	32-bit floating point little-endian number	4	-3.4e38 to 3.4e38
f64	64-bit floating point little-endian number	8	-1.8e308 to 1.8e308

Endianness

There are also endian-specific versions of the number types:

Type	Description	Endianness
u16le	Unsigned 16-bit integer	little-endian
u16be	Unsigned 16-bit integer	big-endian
u32le	Unsigned 32-bit integer	little-endian
u32be	Unsigned 32-bit integer	big-endian
u64le	Unsigned 64-bit integer	little-endian
u64be	Unsigned 64-bit integer	big-endian
i16le	Signed 16-bit integer	little-endian
i16be	Signed 16-bit integer	big-endian
i32le	Signed 32-bit integer	little-endian
i32be	Signed 32-bit integer	big-endian
i64le	Signed 64-bit integer	little-endian
i64be	Signed 64-bit integer	big-endian
f32le	32-bit floating point	little-endian
f32be	32-bit floating point	big-endian
f64le	64-bit floating point	little-endian
f64be	64-bit floating point	big-endian

---

TypedArray Types

Byteform provides a set of types for TypedArray data structures.

Type	Description
u8Array	Unsigned 8-bit integer array
u16Array	Unsigned 16-bit little-endian integer
u32Array	Unsigned 32-bit little-endian integer
u64Array	Unsigned 64-bit little-endian integer
i8Array	Signed 8-bit integer
i16Array	Signed 16-bit little-endian integer
i32Array	Signed 32-bit little-endian integer
i64Array	Signed 64-bit little-endian integer
f32Array	32-bit floating point little-endian number
f64Array	64-bit floating point little-endian number

---

ArrayBuffer Types

Byteform provides an arrayBuffer schema type to represent ArrayBuffer data. It's a synonym for a JavaScript ArrayBuffer.

import { arrayBuffer } from '@evenstar/byteform';

const schema = arrayBuffer;