Get started

Three lines

#include "smart2raw.h"

S2RPool p;
s2r_pool_init(&p, s2r_classify_array(values, n), n);   /* 1. classify */
for (size_t i = 0; i < n; i++) s2r_push(&p, values[i]); /* 2. store    */
size_t k = s2r_count_gt_fast(&p, 100);                  /* 3. operate  */

Compile with anything:

cc -O2 -std=c11 -I include your_program.c -o your_program

Let the library choose the shape

The obvious entry point is not always the right one. On 4 million timestamps the flat pool is 15.26 MB and 0.73 ms where the block-wise form is 4.11 MB and 0.04 ms — so ask before deciding:

S2RAdvice a;
s2r_recommend(values, n, &a);
printf("best: %s, %zu bytes (baseline %zu)\n", a.best, a.best_bytes, a.raw_bytes);

Save and load

S2RBlocked b;
s2r_blocked_build_auto(&b, values, n);   /* block size is planned, not guessed */
s2r_blocked_save(&b, "column.s2r");

The file is canonical little-endian with a CRC32, so it is identical on any host and a corrupted byte is caught on load rather than returned as data.

Without writing C: the command-line tools

If you want to evaluate before writing code, the repository ships three programs. One make inside tools/ builds all three, and they need nothing beyond gcc.

s2r pack   data.txt column.s2r           # text → .s2r, classifying as it goes
s2r info   column.s2r                    # the chosen class, the count, the size
s2r agg    column.s2r count-gt 100       # the query, straight against the file
s2r verify column.s2r                    # magic, class, count and CRC32

s2r_verify exists separately for a reason: it exits with code 0 when the file is intact and non-zero when it is not, so it drops into a script or a CI job without anyone having to parse output. And s2r_convert closes the loop — convert, process in the compact form, unconvert — with an overflow ceiling that refuses rather than promoting past it:

s2r_convert data.txt out.txt --op mul --by 3 --cap 32

There are 19 checks in tools/test_cli.sh, and they run with everything else.

Downloads

s2r-probe.exe data.csv --column 3 --save column.s2r

It prints the class chosen, the size against int64 and against dictionary, RLE and bitmap, the timed query, and exits with status 0 only when every internal check passed.

Where it runs

The same file, with the same tests passing: x86-64 with SSE2 and AVX2, ARM with NEON and SVE2, RISC-V with RVV, big-endian machines, and microcontrollers in lean mode (-DS2R_NO_STDIO -DS2R_NO_MMAP -DS2R_NO_SIMD). And, as of this site, in WebAssembly and in a Windows PE linked without a C runtime.

It is worth saying how each one is verified, because they are not the same. One runs on real hardware: x86-64, which is the CI machine itself. ARM64 and s390x are the real ISA on an emulated machine (QEMU), on every commit — the instructions and the byte order are the target's; the silicon is not. And it is worth being specific about what that proves, because "big-endian" is usually a word rather than a measurement: before any test runs, the job compiles an s390x binary and executes it, and it prints __BYTE_ORDER__ : big and the value 0x01020304 laid out in memory as 01 02 03 04, exiting with an error if it is not; on top of that run 250,212 checks across 16 suites, 0 failures. The microcontroller mode (-DS2R_NO_STDIO -DS2R_NO_MMAP -DS2R_NO_SIMD) is a build configuration, not a board.

The RVV and SVE2 kernels run for real — the vector code that ships, not a reimplementation — checked element by element against a scalar reference, with the vector length swept from 128 to 1024 bits and the strip-mine boundaries and tails exercised explicitly. A real board would have given one length; the sweep covers the whole family, which is exactly what a length-agnostic kernel has to prove.

Licence

The published version is under AGPL-3.0-or-later. If you plan to build it into something you do not publish under the same licence, that needs a commercial licence — which is also how you reach the advanced version.