Persist an In-Memory Table to Date-Partitioned HDB

Question

Given the following in-memory table t, write a script to persist the rows for each date to a separate date partition in an HDB at `:/hdb.

N:1000
t:([] date:N?.z.d + til 3;
      time:N?100?23:59:59.999;
      sym:N?`A`B`C;
      price:N?10.0;
      qty:N?1000j)

Solution

N:1000
t:([] date:N?.z.d + til 3;
      time:N?100?23:59:59.999;
      sym:N?`A`B`C;
      price:N?10.0;
      qty:N?1000j)

path:`:/hdb
dts:exec distinct date from t

persistPerDate:{[d]
  data:delete date from select from t where date = d;
  tblPath:` sv path,`$string d,`t,`;
  tblPath set .Q.en[path;data]
 }
persistPerDate each dts

Explanation

Extracting distinct dates — exec distinct date from t returns a plain list of date atoms (not a table), which each can iterate over directly.

Path construction — ` sv path,`$string d,`t,` builds the full splayed path by joining symbols with backtick as separator:

` sv `:/hdb,`2025.01.15,`t,`
/ → `:/hdb/2025.01.15/t/

The trailing ` (empty symbol) is what produces the trailing / in the final path. That trailing slash is what tells kdb+ to write in splayed form — one file per column — rather than as a single flat binary file.

Removing the date column — the date column is not written to disk. In a partitioned database, the date is encoded in the directory structure itself. Including it in the column data would create a redundant date file in every partition.

.Q.en enumerates all symbol columns in data against the database's master sym file at path. This converts symbol atoms to integer indices shared across all partitions. The sym file is created automatically on the first call if it does not already exist.

set writes the table to disk at the constructed path. Because the path ends with /, kdb+ writes one binary file per column inside the directory, plus a .d file recording column order.