Gotchas & Edge Cases

Sorted Can Be Applied In Place — Others Are Not

KX's documentation notes that setting the sorted attribute on a list that is already sorted does not make a copy, whereas setting any other attribute does. In practice this means `s# applied to a sorted variable changes the variable itself, even without reassignment:

n: til 10
`s#n             / n is already sorted, so the attribute is set on n itself
attr n           / `s

m: til 10
`g#m             / returns a grouped copy — m itself is unchanged
attr m           / `

So for `u#, `p# and `g#, always assign the result back (m: g#m`), or use a form that updates the variable or table column in place:

trade: ([] sym:`AAPL`GOOG`AAPL; price:1 2 3f)
update `g#sym from `trade        / in place, by table name
@[`trade; `sym; `g#]              / the same, using amend

Removing an Attribute

To remove an attribute, apply the null symbol ` with # and assign the result:

n: `s# til 10
attr n           / `s
n: `#n
attr n           / `

/ In a table column:
update `#sym from `trade

`asc` Applies `s#` Automatically

Sorting a list with asc returns it with the sorted attribute already set:

n: 5 3 1 4 2
attr asc n       / `s

Error Messages When Applying Incorrectly

Applying an attribute to data that does not have the required structure fails immediately:

`s# 4 2 5 1 3    / 's-fail — not sorted
`u# 1 2 1 2 1    / 'u-fail — contains duplicates
`p# 1 1 2 1      / 'u-fail — the 1s are not contiguous

If you get no error, the attribute was applied. What fails silently is losing an attribute later, when the list is modified.

Plain Join Returns a New, Unattributed List

n: `s# til 10

/ Join creates a new list with no attribute — even though it is still sorted
attr n , 10 11 12            / `
n: n , 10 11 12
attr n                       / ` — reassigning the joined list doesn't help

/ Appending in place keeps the attribute if the property still holds
n: `s# til 10
n,: 10 11 12
attr n                       / `s

Appending to a Parted List Removes `p#`

KX's documentation states that "the parted attribute is removed by any operation on the list". That includes an in-place append, even one that would keep equal values contiguous:

x: `p# 1 1 1 2 2 3 4 4 4 4
x,: 4 4 5
attr x           / `

This is one reason p# is used on HDB data that is written once at end of day, and not on the RDB.

Modification Operations

Operations that produce a new list return it without an attribute, even when the property would still hold:

n: `s# til 10
attr n+1         / `
attr -1_n        / `

n[2]: 2          / even writing back the same value...
attr n           / ` — ...removes `s

The grouped attribute is the exception for item changes: it is kept when items are replaced, and updating other columns of a table does not affect it:

g: `g# `AAPL`GOOG`AAPL
g[0]: `MSFT
attr g           / `g

t: ([] sym:`g#`AAPL`GOOG`AAPL; price:100 200 100)
update price+1 from `t
attr t`sym       / `g — only the updated column is replaced

Dropping or deleting items, however, removes g# too:

attr -1_g                          / `
delete from `t where price=201
attr t`sym                         / `

Attributes Only Help in the First Where Constraint

The constraints in a where clause are applied left to right, each to the rows that passed the previous ones. Only the first constraint is applied to the full column, so it is the only one that can use that column's attribute:

t: ([] sym:`g#10000000?`3; price:10000000?100.0)

\t select from t where sym=`abc, price>0     / fast: uses g# on sym
\t select from t where price>-1, sym=`abc    / slow: the sym test runs on 10M rows that passed price>-1

Implication: put the constraint on the attributed column first — and in a partitioned database, put the partition column (date) before everything else.

Applying `s#` to Tables

Applying `s# to a whole table has a different effect depending on the kind of table:

/ Simple table: the first column gets the parted attribute
t: ([] col1: til 5; col2: `a`b`c`d`e)
meta `s#t        / col1 gets `p

/ Single-keyed table: the key column gets the sorted attribute
kt: ([col1: til 5] col2: `a`b`c`d`e)
meta `s#kt       / col1 gets `s

/ Multi-keyed table: the first key column gets the parted attribute
mkt: ([col1: til 5; col2: `a`b`c`d`e] col3: 1 2 3 4 5)
meta `s#mkt      / col1 gets `p

Assign the result (kt: s#kt`) to keep it.

Step Functions with Sorted Dictionaries

When `s# is applied to a dictionary (keys sorted ascending), it becomes a step function: looking up a key that does not exist returns the value for the largest key less than or equal to it:

d: `s# 10 20 30 40 50 ! `a`b`c`d`e

d 20             / `b — exact match
d 19             / `a — largest key ≤ 19 is 10
d 35             / `c — largest key ≤ 35 is 30
d 5              / `  — no key ≤ 5, so a null
d 10000          / `e — largest key ≤ 10000 is 50

`s# 20 10 30 40 50 ! `a`b`c`d`e    / 's-fail — keys not sorted

Step functions are useful for state that changes at known points and stays constant in between — a prevailing exchange rate, an instrument's trading status, a price band.

Step Functions with Sorted Keyed Tables

The same applies to a keyed table with `s# applied:

rates: ([time: 09:00:00 09:30:00 10:00:00 10:30:00] rate: 1.25 1.27 1.26 1.28)

/ Without s#: only exact keys match
rates 09:15:00          / a row of nulls — no exact match

/ With s#: the row for the largest key ≤ the lookup
kts: `s#rates
kts 09:15:00            / rate| 1.25
kts 10:30:00            / rate| 1.28
kts 11:00:00            / rate| 1.28

This is the same "as of" idea that aj applies across a whole table of lookups; a step-function keyed table is a lightweight way to do a single lookup.