Kdb+ Version 4.0 – Faster, More Secure

By Simon Garland

We are pleased to announce version 4.0 of kdb+, introducing significant new features for improving performance and security.

Multithreaded primitives

The parallelism implicit in loopless vector-programming languages, such as q, help them access the processing power of multi-processor machines with minimal coding effort. Q’s Parallel Each operator lets the developer simply assign expressions to be delegated to slave tasks.

Kdb+ V4.0 introduces finer-grained parallel processing, with no developer effort required. Language primitives now look for and exploit opportunities for parallel processing. Existing code run under V4.0 will make use of parallel processing where it will help. On our own tests, we have seen processing times drop by 50-70%.
Read more…

Intel’s Optane DC persistent memory

Intel Optane DC persistent memory (Optane Memory) is a new hardware technology from Intel, based on a new silicon technology, 3DX Point. It is significantly faster and more durable than existing storage media, and blurs familiar architectural distinctions between files and memory.

Earlier versions of kdb+ are already compatible with Optane Memory in Storage and Cached-memory modes, providing significant improvements for both in-memory and on-disk datasets. Kx has worked closely with Intel engineers, and kdb+ V4.0 supports App Direct Memory mode. Users can optimize their applications and schemas, keeping hot data in fast DRAM while still taking full advantage of the expanded memory capacity.
Read more…

Data-at-Rest Encryption

Transparent Disk Encryption (TDE) solves the problem of protecting data at rest, by encrypting database files on the hard drive and consequently also on backup media.

As TDE decrypts the data inside the kdb+ process, rather than at the OS/storage level, data remains encrypted when it comes across the wire from remote storage.
Read more…

The production version of kdb+ 4.0 is now available to licensed users; for users of the previous version the upgrade is seamless. We’d be interested to hear of your experiences with kdb+ V4.0.  If you have something to share, please write to me: simon@devweb.kx.com.