Sort and Apply Parted Attribute to On-Disk Partitions

Question

After persisting the table t to a date-partitioned HDB, sort each on-disk partition by sym and time, and apply the p# (parted) attribute to the sym column.

Assume path and dts are already defined from the persist step.

Solution

onDiskSort:{[d]
  tblPath:.Q.par[path;d;`t];
  @[`sym`time xasc tblPath;`sym;`p#]
 }
onDiskSort each dts

Explanation

.Q.par[path; date; tableName] constructs the full path to a table within a specific date partition. For path=`:/hdb and d=2025.01.15, it returns `:/hdb/2025.01.15/t. This is the standard way to reference an on-disk partitioned table without building the path string manually.

`sym`time xasc tblPath sorts the on-disk table in place. Unlike sorting an in-memory table (which returns a new table), xasc on a file path rewrites the column files on disk and returns the path. (It reads the column data into memory to reorder it, then writes it back.)

@[table; column; attribute] applies an attribute to a column. The triadic form used here:

@[tblPath; `sym; `p#]

applies p# to the sym column of the on-disk table at tblPath.

The p# (parted) attribute tells kdb+ that all rows sharing the same sym value are contiguous in the column — there are no gaps where a symbol group splits across non-adjacent rows. This enables a binary-search-style access pattern for where sym=`X filters, turning an O(n) scan into an O(log n) lookup, which makes historical queries significantly faster in practice.

The full expression chains these steps: sort the table on disk first, then apply the attribute to the now-sorted sym column.

@[`sym`time xasc tblPath; `sym; `p#]
/ Step 1: `sym`time xasc tblPath → sorts files on disk, returns tblPath
/ Step 2: @[tblPath; `sym; `p#] → applies parted attribute to sym column

p# is only valid when equal values in the column are contiguous. Applying it to an on-disk column that hasn't been sorted fails immediately with 'u-fail, so the sort must come first.