Inspect Table Size in Memory

Question

Assume the following tables exist in memory:

trades:([] sym:`AAPL`MSFT`GOOG; qty:100 200 300)
quotes:([] sym:`AAPL`MSFT; bid:185.2 410.1; ask:185.3 410.2)
positions:([] sym:`AAPL`GOOG; pos:1000 500)

Write q expressions to:

1. Return the number of rows in each table as a dictionary.
2. Return the size of each table in bytes, ordered from largest to smallest.

Solution

Row Counts

tables[]!{exec count i from x} each tables[]

or

tables[]!{count value x}@'tables[]

Output:

positions| 2
quotes   | 2
trades   | 3

Table Sizes

desc tables[]!{-22!`. x} each tables[]

Output:

quotes   | 95
trades   | 82
positions| 69

Explanation

Getting the List of Tables

tables[]

returns the names of the tables in the root namespace, in alphabetical order:

`positions`quotes`trades

Counting Rows

exec count i from x returns the number of rows in table x — here x is a table name, which exec accepts as well as a table. Applying it to every table:

{exec count i from x} each tables[]

2 2 3

Then tables[]! pairs the names with the counts to make a dictionary:

positions| 2
quotes   | 2
trades   | 3

In the alternative, value `trades retrieves the table itself from its name, and count of a table is its number of rows.

Measuring Size

`. x looks up the global named x (it is the same as value x here), and -22! returns the number of bytes the object takes up when serialized — the size of the message it would make over IPC:

-22!`. `trades

82

kdb+ has no built-in that reports the exact memory a single object occupies, so the serialized size is a convenient estimate for comparing tables: it grows with the number of rows and columns and the column types. (The memory actually used is somewhat larger, since kdb+ allocates memory in blocks; .Q.w[] reports the memory use of the whole process.)

Applying it to every table and pairing with the names:

tables[]!{-22!`. x} each tables[]

positions| 69
quotes   | 95
trades   | 82

Finally, desc applied to a dictionary sorts it by value, largest first:

desc tables[]!{-22!`. x} each tables[]

quotes   | 95
trades   | 82
positions| 69

Summary

These are useful when checking the state of a running kdb+ process, for example to see which tables are growing.