pgvector IVFFlat Lists Tuning Guide: Probes and Recall

The short version: set lists to rows / 1000 for tables up to 1M rows and sqrt(rows) above that, start ivfflat.probes at sqrt(lists), build the index only after the data is loaded, then raise probes until recall@10 against exact search clears your target. The starting points come from the pgvector README. The sweep is the part teams skip.

The incident that forces this is usually quiet: a RAG eval drops a few points after a bulk ingest, or a LIMIT 10 query starts returning three rows. Nothing errored. ivfflat.probes defaults to 1, so each query scanned one cluster, against centroids trained when the table held a fraction of today’s rows.

What do lists and probes actually control?

lists is the number of k-means clusters pgvector computes when you run CREATE INDEX, and probes is how many of the nearest clusters each query scans. Lists is baked into the index (the default is 100 in the pgvector source); probes is a runtime setting you can change per transaction. Together they set the fraction of the table a query touches.

The build samples 50 rows per list, with a floor of 10,000 samples, clusters them, then assigns every row to its nearest centroid (ivfbuild.c). At query time the vector is compared against the centroids and the probes closest lists are scanned exhaustively. More lists means smaller lists and cheaper scans; the AWS Database Blog’s IVFFlat and HNSW deep dive notes that this “increases the likelihood of reduced recall.” More probes buys the recall back with more scanning.

That sample rate sits at the low end of the 30 to 256 training vectors per centroid in the Faiss index guidelines, so the rows present at build time carry a lot of weight.

The metric that matters: recall@k at a stated scan fraction

Tune on recall@k at a stated scan fraction, not on latency alone and not on the probes value alone.

recall@k        = |ANN top-k ∩ exact top-k| / k    (mean over a golden query set)
scan fraction   ≈ probes / lists
rows per query  ≈ rows × probes / lists

The approximations assume roughly balanced lists, which k-means does not guarantee. Latency alone says nothing: twice as fast at 80 percent recall has not beaten 97 percent. Probes alone does not transfer between indexes, because probes = 10 scans about 10 percent of the table at lists = 100 and 1 percent at lists = 1000. Report the pair. How to benchmark recall at k for ANN indexes covers ground truth and tie handling; the method carries over unchanged.

Starting values by table size

Applying the README’s two rules, rounded, gives this grid. The last column is arithmetic from the formulas above, not a measurement.

Two things fall out. The rules meet at 1M rows, where both give 1,000 lists, so there is no cliff at the boundary. And while scan fraction drops from 10 percent to 1 percent, absolute rows scanned keeps climbing, so per-query cost grows with the table even when you follow the guidance exactly.

pgvector’s rule also yields far fewer lists than Faiss, whose guidelines suggest 4√N to 16√N centroids below 1M vectors: roughly 2,800 to 11,300 at 500k rows against pgvector’s 500. If recall flattens below target at a probes value you can afford, rebuilding with more lists and proportionally more probes is a fair experiment.

For scale, AWS’s own test (a vendor benchmark) built a lists = 100 index on 58,634 vectors of 1,536 dimensions in about 15.5 seconds on pgvector 0.5.0. At that size, rebuilding while you tune is cheap.

Wiring it up

Build after the bulk load, with more maintenance_work_mem and parallel workers, and watch progress from a second session:

SET maintenance_work_mem = '4GB';
SET max_parallel_maintenance_workers = 7;  -- plus leader

CREATE INDEX items_embedding_ivf
  ON items USING ivfflat (embedding vector_cosine_ops)
  WITH (lists = 1000);

-- second session
SELECT phase,
       round(100.0 * tuples_done / nullif(tuples_total, 0), 1) AS "%"
FROM pg_stat_progress_create_index;

Phases run initializing, performing k-means, assigning tuples, loading tuples. The percentage only fills in during the last one, so a build sitting in k-means with an empty column is not stuck.

Then sweep probes against exact search. This uses psycopg 3 and pgvector-python, builds ground truth with index scans disabled (the README’s own recall-monitoring recipe), and logs one MLflow step per probes value:

import time

import mlflow
import numpy as np
import psycopg
from pgvector.psycopg import register_vector

K, LISTS = 10, 1000
PROBES = [1, 4, 8, 16, 32, 48, 64, 128]
SQL = "SELECT id FROM items ORDER BY embedding <=> %s LIMIT %s"


def topk(cur, q):
    cur.execute(SQL, (q, K))
    return [row[0] for row in cur.fetchall()]


with psycopg.connect("dbname=app") as conn:
    register_vector(conn)
    cur = conn.cursor()
    cur.execute("SELECT embedding FROM golden_queries")
    queries = [row[0] for row in cur.fetchall()]

    cur.execute("SET LOCAL enable_indexscan = off")  # exact search
    truth = [set(topk(cur, q)) for q in queries]
    conn.commit()

    with mlflow.start_run(run_name=f"ivfflat-lists-{LISTS}"):
        mlflow.log_param("lists", LISTS)
        for p in PROBES:
            cur.execute("SELECT set_config('ivfflat.probes', %s, true)", (str(p),))
            hits, short, lat_ms = 0, 0, []
            for q, gt in zip(queries, truth):
                t0 = time.perf_counter()
                got = topk(cur, q)
                lat_ms.append((time.perf_counter() - t0) * 1000)
                hits += len(gt.intersection(got))
                short += len(got) < K
            conn.commit()  # SET LOCAL scope ends here
            mlflow.log_metric("recall_at_10", hits / (K * len(queries)), step=p)
            mlflow.log_metric("p95_ms", float(np.percentile(lat_ms, 95)), step=p)
            mlflow.log_metric("short_results", short, step=p)

set_config(name, value, true) is the function form of SET LOCAL, which lets you bind the value as a parameter.

What you’ll see

Plot recall_at_10 and p95_ms against step. Healthy: recall climbs steeply over the first few probes values, then flattens, while p95 keeps rising because scan work is linear in probes. The operating point is the knee, the smallest probes that clears your recall target with margin. short_results stays at zero on unfiltered queries.

Unhealthy patterns:

• Recall plateaus below target at high probes. Centroids trained on unrepresentative data, often an index built after the first tenant’s load. Rebuild before touching probes.
• short_results above zero. Per the README’s troubleshooting, the index was likely built with too little data for its lists, or probes is capping results. Filters make it worse.
• Recall at fixed probes slides over weeks. Stale centroids.
• Recall matches exact search and p95 matches a sequential scan. Probes equals lists, and the README notes the planner then skips the index.

When should you rebuild an IVFFlat index?

Rebuild when golden-set recall at your production probes value falls below target, or when the table has grown enough that the sizing rule calls for materially more lists; a doubling of row count is a reasonable trigger. IVFFlat does not retrain on insert. New rows go to the nearest existing centroid, so a drifting corpus gets packed into lists drawn for older data.

The Supabase IVFFlat docs put it plainly: “Anytime the distribution changes significantly, consider rebuilding indexes.” REINDEX INDEX CONCURRENTLY reruns k-means on current rows at the same lists value. Changing lists means CREATE INDEX CONCURRENTLY with the new value, confirming with EXPLAIN that queries use it, then DROP INDEX CONCURRENTLY on the old one.

The build warning will not save you. pgvector emits ivfflat index created with little data only when sampled rows number fewer than lists, so a 20,000-row table with lists = 1000 builds silently at about 20 rows per list.

Caveats

• Filtered queries come back short. Filtering is applied after the index scan, so WHERE tenant_id = 42 ... LIMIT 10 at low probes can return fewer than ten rows. Since pgvector 0.8.0, SET ivfflat.iterative_scan = relaxed_order keeps probing until the limit is met, capped by ivfflat.max_probes. Relaxed order can return rows slightly out of distance order, so re-sort in an outer query if that matters. For very selective filters, a B-tree or partial index beats any probes value; low vector search recall: causes and fixes covers the diagnosis.
• Ground truth is expensive. Each exact query is a sequential scan, so on large tables compute it on a replica and cache it.
• Golden sets sampled from the table flatter recall. A query that is itself a row always lands in its own list at probes = 1, which inflates recall at small k. Use logged production queries.
• Some rows never appear. NULL vectors, and zero vectors under cosine distance, are not indexed.

FAQ

Should I use IVFFlat or HNSW in pgvector?

Use HNSW unless build time or memory is the binding constraint. The pgvector README says IVFFlat builds faster and uses less memory but has a worse speed-recall tradeoff, and Supabase recommends HNSW because it holds up better as data changes. IVFFlat suits bulk-loaded tables rebuilt on a schedule; see HNSW vs IVF tradeoffs.

How do I set ivfflat.probes for a single query?

Run the query inside a transaction and use SET LOCAL ivfflat.probes = 32, which reverts at commit, as the pgvector README describes. Avoid session-level SET behind PgBouncer in transaction pooling mode: its feature map lists SET as never compatible, so the value can stick to a server connection another client reuses.

How much maintenance_work_mem does an IVFFlat build need?

At minimum, enough for the k-means sample: lists × 50 vectors, or 10,000 if larger, capped at the table’s row count, at about 4 bytes per dimension each. For 1,536-dimension embeddings and lists = 1000, that is roughly 300 MB before centroids. Too little and the build fails with “memory required is N MB”.

What is the maximum number of lists in pgvector IVFFlat?

The maximum is 32,768, set by IVFFLAT_MAX_LISTS in the pgvector source; the minimum is 1 and the default is 100. Under the sqrt(rows) rule you would not reach the cap until about 1.07 billion rows. IVFFlat also caps indexed vector columns at 2,000 dimensions, or 4,000 with halfvec.

Related across the network

• RAG Pipeline Architecture: Components and Build Order — ragstackguide.com
• RAG Chunking Strategy: Picking Chunk Size and Overlap — ragstackguide.com