🗄️

ColumnarDB

Analytical Query Engine in C++17

A from-scratch columnar database with SIMD-accelerated vectorized execution

C++17 AVX2 SIMD MVCC B+ Tree Index Volcano Executor SQL Parser

5.4 GB/s

SIMD Scan Throughput

3.0×

Alloc Speedup

1024-row

Vector Batches

github.com/sidx007/ColumnarDB

Overview

What is ColumnarDB?

ColumnarDB is a complete analytical query engine built from scratch in C++17. It covers every layer — from a hand-written SQL parser down to custom memory allocators — with a focus on squeezing performance out of modern hardware.

SQL Compiler

Recursive-descent parser → AST → cost-based query planner with selectivity estimation, index selection, and join reordering.

Vectorized Executor

Volcano-model iterator with 1024-row batches. Operators: SeqScan (AVX2), IndexScan (B+ Tree), Filter, Aggregate, HashJoin.

Columnar Storage

RowGroups of 64K rows. Columns stored with Dictionary/RLE compression and 1-bit-per-row Null Bitmaps.

Concurrency

MVCC with Snapshot Isolation. Epoch-based GC. B+ Tree with hand-over-hand locking for concurrent reads.

Architecture

Stack Layers

SQL Interface — Lexer (FSA) → Recursive-Descent Parser → AST

↓

Cost-Based Query Planner — Selectivity Estimation · Index Selection · Join Reordering

↓

Vectorized Volcano Executor

SeqScan (AVX2) IndexScan (B+Tree) Filter Aggregate HashJoin

↓

Table · RowGroup · Column · Null Bitmap

B+ Tree Index
Lock-Free Reads

MVCC + Epoch GC
Snapshot Isolation

↓

Custom 64B-Aligned Slab Allocator — eliminates malloc contention

Benchmarks

Performance Numbers

Scan Throughput — 10M rows

Equality Scan — Scalar3.3 GB/s

Equality Scan — AVX25.4 GB/s

Range Scan — Scalar2.7 GB/s

Range Scan — AVX24.6 GB/s

Allocator — 100k cycles

Size	Slab	malloc	Speedup
32 B	5.97 ms	9.94 ms	1.7×
512 B	10.35 ms	30.57 ms	3.0×

Query Latency — 100k rows

Query	Latency
Full Scan	25.78 ms
COUNT Aggregate	8.85 ms
GROUP BY SUM	20.27 ms

Usage

SQL Interface

ColumnarDB exposes a SQL interface that covers CREATE, INSERT, SELECT with WHERE filters, GROUP BY aggregations, and JOINs — all executed through the vectorized pipeline.

Build with CMake, runs on x86_64 with AVX2

Extensive test suite — 100k+ assertions

Separate micro-benchmark binaries for scan and query latency

No external dependencies — standard C++17 + CMake only

-- Create and populate CREATE TABLE sales ( id INT, region VARCHAR(50), amount DOUBLE ); INSERT INTO sales VALUES (1,'North',450.50); INSERT INTO sales VALUES (2,'South',1200.00); -- Vectorized aggregation SELECT region, SUM(amount), COUNT(*) FROM sales WHERE amount > 400 GROUP BY region;