Add a system

How to run your own database system in DB Olympics: what a system consists of, how the benchmark talks to it, and the rules it plays by.

What a system is

A self-contained directory systems/<name>/ in the repository: the scripts that install and start the system, and its query program, through which the benchmark talks to it. The benchmark does the rest: it builds a container image, starts it with the machine's cores and memory, creates the schema, loads the data, runs and times the queries, and checks the results.

File	Purpose
`system.toml`	the statement that loads a Parquet file, the system's spellings of the placeholders and its type names, session settings, and containers to start next to the system's own
`install`	installs the system in its container (Ubuntu 26.04, as root, with network, with every file of the directory) and builds the query program; runs once, when the image is built
`start`	starts the system's servers, if it has any, and its query program; it gets the run's cores and memory in `DBO_THREADS` and `DBO_MEMORY_BYTES`
`query.rs`	the query program, in Rust (with its `Cargo.toml` and `Cargo.lock`): runs for the whole run, serves the protocol below and runs every statement on the system, or holds the system embedded; `main` hands the system to `dbo::serve` (`systems/dbo`: the protocol, the loop and the timer)
`stop`	optional: stops the query program and the servers at the end
`logo.svg`	optional: the system's mark on this website, any SVG, fitted into a square; drawn in `currentColor`, it takes the page's colours, also when the system is highlighted

A system that comes as a container image needs only a [[container]] entry, and a query program that connects to it. A system file, for Umbra:

name = "umbra"
version = "experimental"

[load]
statement = "COPY {table} FROM '{file}' WITH (FORMAT parquet)"

[dialect]
settings = ["SET debug.semana.pgdatesub = true"]

[[container]]
name = "db"
image = "docker.io/umbradb/umbra:experimental"
env = { PARALLEL = "{threads}", MEMORY = "{memory_mb}M", PGHOST = "/run/dbo" }

The protocol

The benchmark sends every statement to the query program over a Unix socket, one at a time, on one connection that stays open for the whole run. It measures every request itself, from sending it to having the whole answer; nothing the system reports counts.

Request: a u32 length of the rest, a u32 timeout in milliseconds (0: none), then one SQL statement in UTF-8. All integers are little-endian. Settings, the schema, the loads (one statement per table) and the queries all come this way.
Answer: a u32 length of the rest, a u8 status (0 ok, 1 error, 2 timeout), then the statement's result as CSV without a header (NULL as an empty field, the empty string as ""; nothing for a statement without one) or the error text.
Timeouts: a statement still running at its timeout is cancelled, and the system answers the next request as usual.

Each system decides how its query program starts and connects. The query programs are Rust programs with the system's own Rust client: the postgres crate for the systems that speak the PostgreSQL protocol, ureq for HTTP interfaces, Arrow Flight for Arrow Flight SQL. The protocol code is the same in every one, so a new one copies it and only has to run a statement on the system and cancel the running one. An embedded engine lives inside its query program, as DuckDB, chDB and DataFusion do. The system's install builds it.

Rules

The same schema for every system. A system may change type names and append text to CREATE TABLE (a table engine, for example); no indexes and no other tables.
The same queries. A system may spell the placeholders its own way, under the criteria of the methodology, and use its own settings to follow the SQL standard more closely; nothing else.
The query program passes the SQL through unchanged. A difference of dialect goes into the system file, never into the query program.
The system loads the data itself, from read-only Parquet files: one file and one statement per table and load.
The system lives for the whole run. It is never restarted, and nothing starts per statement: the query program keeps one connection to the system.
No result caches. Queries repeat on unchanged data; a cached result would not be a measurement.
No network. Whatever the system needs must be in its image after install.

Try it

On Linux with Podman and Rust, after the host setup of the repository:

cargo build --release --workspace
./target/release/dbo-bench render --system <name> --out rendered/<name>   # what the system gets
./target/release/dbo-bench run --benchmark bench/benchmark_sf1.toml --system <name> --config sf1-c8a.2xlarge --keep

Results go to systems/<name>/<date>/; every error and every difference from a reference result is in bench/out/<date>/sf1-c8a.2xlarge/<name>/errors.log (kept with --keep). When the system runs, send the directory as a pull request to the repository.