Reclaim Memory with .Q.gc and Check Usage with .Q.w

Question

After deleting a large variable, .Q.w[] still shows high memory usage. Why hasn't the memory gone back to the operating system, and how can it be reclaimed?

Solution

.Q.w[]      / used, heap, peak, mmap stats for this process
.Q.gc[]     / run garbage collection, coalesce freed blocks

Explanation

Deleting a variable drops q's reference to it. If that was the last reference, the object is freed straight away — q uses reference counting — so used in .Q.w[] falls. But in the default (deferred) garbage-collection mode, q keeps the freed memory in its own heap for future reuse rather than handing it back to the OS, so heap in .Q.w[] — and the memory the OS sees — stays high.

.Q.gc[] asks q to return unused heap memory to the OS: it coalesces free blocks back into larger ones and returns any unused block of at least 64MB, reporting the number of bytes returned. The -g startup flag (or \g) controls whether q also does this automatically: in immediate mode (-g 1), freeing an individual object of at least 64MB returns its memory to the OS straight away, at some performance cost; the default deferred mode (-g 0) keeps it for reuse.

An object is only freed once its reference count drops to zero — q uses reference counting and copy-on-write, so an object shared by several variables only gets copied (and only costs new memory) the moment one of them is modified.

A related trick worth knowing: emptying a list in place with mylist:0#mylist drops its contents and releases that memory back to the heap, without deleting the variable itself — useful for a long-lived variable that needs to shrink, rather than disappear entirely.