Vector Database Memory Sizing: RAM, Graph and Overhead

Vector database capacity planning is arithmetic, and it can be done before anything is installed. Almost every unpleasant surprise in a vector deployment traces back to one number that nobody calculated up front: bytes per vector, including the index structure, not just the embedding.

This is the number to work out first, because it decides the instance size, and the instance size decides the bill.

Start with the raw embedding

An embedding is a fixed length array of floats. At float32, each dimension costs 4 bytes, so raw storage is:

bytes = vector_count x dimensions x 4

That is the whole formula. For one million vectors:

Dimensions	Bytes per vector	Raw storage, 1M vectors
384	1,536	1.54 GB
768	3,072	3.07 GB
1536	6,144	6.14 GB
3072	12,288	12.29 GB

Gigabytes here are 10^9 bytes. Cloud providers bill in those, operating systems usually report GiB (2^30), and the 7 percent gap between the two is enough to make a “16 GB should be fine” plan fail at the margin.

The first lever is the one furthest upstream: dimensionality. A 1536-dimension model costs four times the memory of a 384-dimension model for the same corpus, forever, on every replica. Some model families now publish variants trained so that truncating the vector degrades quality gracefully, which turns dimension into a tuning knob rather than a fixed property of the model. If the retrieval quality holds at half the dimensions, that is a 50 percent cut nothing downstream can match.

The graph is a separate, dimension-independent cost

An HNSW index does not just hold vectors. It holds a navigable graph over them, and the graph has its own footprint.

The structure described in the HNSW paper gives each element a set of neighbour links per layer. The bottom layer contains every element and is built with roughly twice the connectivity of the layers above it, which the reference implementation exposes as the parameter M. Higher layers hold an exponentially shrinking share of elements, so their contribution is small. Links are stored as integer identifiers four bytes wide.

That gives a usable approximation for the bottom layer:

graph bytes per vector ~= 2 x M x 4

This is not a rule of thumb invented here. The Faiss index-selection guidance states the per-vector cost of an HNSW index directly as d * 4 + M * 2 * 4 bytes, which is the embedding term plus exactly this graph term. hnswlib’s parameter documentation arrives at the same place from the other direction, describing memory consumption as roughly M * 8-10 bytes per stored element, the extra allowance covering per-element bookkeeping.

At the common default of M = 16, that is about 128 bytes per vector, plus that small bookkeeping. At M = 32 it is about 256 bytes, and at M = 64 about 512 bytes.

The important property: this cost does not scale with dimensions. It scales with M and with vector count only. Which means the graph’s share of your index swings wildly depending on what the vectors themselves cost:

Configuration (1M vectors, M=16)	Vectors	Graph	Graph share
768-dim float32	3.07 GB	0.13 GB	4%
384-dim float32	1.54 GB	0.13 GB	8%
768-dim scalar-quantized (1 byte/dim)	0.77 GB	0.13 GB	14%
384-dim binary (1 bit/dim)	0.05 GB	0.13 GB	73%

A flat percentage rule of thumb for graph overhead is wrong in both directions. On uncompressed high-dimension vectors it wildly overstates the graph. On aggressively quantized low-dimension vectors the graph becomes the majority of the index, and cutting M becomes a bigger lever than compressing the vectors any further.

This is also why raising M is not free. It increases memory linearly and increases build time, in exchange for better recall at a given search effort. Treat it as a memory decision first.

Quantization changes the ratio, not just the total

Compression applies to the vector payload, and the documented reductions are straightforward: scalar quantization to one byte per dimension is roughly a 4x reduction against float32, and binary quantization to one bit per dimension is roughly 32x.

Two consequences get missed.

First, compression is lossy, and the standard remedy is rescoring: retrieve a larger candidate set using the compressed representation, then re-rank those candidates against higher-precision copies. That means the full-precision vectors still have to exist somewhere reachable. Vendors handle this by keeping originals on disk while the compressed form stays resident, so the plan needs a disk budget, not only a RAM budget, and the rescoring pass needs to be fast enough that reading originals does not dominate query time.

Second, oversampling to support rescoring means the engine fetches more candidates than the caller asked for. That is additional work per query, and it is the real price of the memory you saved.

Everything else that wants memory

The index is not the only resident object. A capacity plan that only counts vectors and graph links will be short.

• Payloads and metadata. Filters need indexed metadata, and a keyword or numeric index over the payload is its own structure. Text stored alongside vectors for display is often larger than the vectors.
• Build-time headroom. Index construction is more memory-hungry than serving it. In Postgres, pgvector’s HNSW build uses maintenance_work_mem, and if the graph does not fit in that budget the build spills to disk and slows down sharply. Size the build, not just the steady state.
• Deletes. Graph indexes generally tombstone deleted elements rather than removing them, so a collection with heavy churn holds memory for vectors that no longer answer queries until a compaction or rebuild reclaims it.
• Replicas and shards. Each replica holds a full copy of what it serves. Two replicas for availability doubles the memory bill, and this is where an otherwise correct calculation gets multiplied into the wrong instance class.
• Operating headroom. An index sized to exactly fill available RAM will page, and a paging graph index is catastrophically slow because graph traversal is random access by design.

Worked example

Five million documents, 768 dimensions, float32, M = 16, two replicas:

• Vectors: 5,000,000 x 768 x 4 = 15.36 GB
• Graph: 5,000,000 x 128 = 0.64 GB
• Index subtotal: ~16 GB per copy
• Two replicas: ~32 GB of index across the fleet
• Plus payload indexes, build headroom and operating margin, on every node

Read that per node, not as a fleet total, because each replica has to hold its own full copy resident. A 16 GB node cannot serve a 16 GB index: there is nothing left for payload indexes, the operating system, or the margin that keeps the graph from paging. A 32 GB node per replica is the honest answer, which is 64 GB of RAM bought in total for a corpus whose vectors are 15 GB.

Switch to scalar quantization and the per-copy index falls to roughly 4.5 GB, which changes the node class entirely and makes disk capacity for the full-precision originals the thing to check next.

The HNSW index sizer on this site runs exactly this arithmetic, including the M-dependent graph term rather than a flat percentage.

When RAM is the wrong answer

Above a certain corpus size, keeping everything resident stops being economical. Disk-resident graph indexes are designed for this case: they keep a compressed representation in memory for traversal and read full vectors from SSD, trading a modest latency increase for a much larger addressable corpus per node. The tradeoff is real and the design assumes NVMe-class storage; the same index on network storage behaves very differently.

The decision point is usually not “does it fit” but “does it fit at a price that makes sense”. Work out the resident cost per million vectors, multiply by growth over the planning horizon, and compare against the disk-based option before committing to a shape.

A sizing checklist

1. Fix the embedding dimension. Confirm whether the model supports truncation before assuming the full width is required.
2. Compute raw bytes: count x dimensions x bytes_per_value.
3. Add the graph: about 2 x M x 4 bytes per vector for HNSW.
4. Decide quantization, then add a disk budget for full-precision originals if rescoring is enabled.
5. Add payload and metadata indexes.
6. Multiply by replicas.
7. Add build-time headroom and operating margin.

Do this before choosing an index type, because the memory answer often decides the index. For how those types differ once the budget is known, see HNSW vs IVF: vector index tradeoffs compared. If the index is already built and returning the wrong neighbours, start with low vector search recall: causes and fixes. For the underlying concepts, vector search fundamentals covers what these indexes are doing and why they can be wrong.