Embeddings are managed. Chunking is yours to tune.
CogniWeave handles the embedding model, dimensions, and vector index automatically—no model picker, no dimensionality decision. What you can configure per workspace is OCR, chunk size, and chunk overlap. Here's how those choices affect retrieval quality, and how to tune them on your own corpus.
What CogniWeave manages for you
Embedding model, vector dimensions, normalization, query vs document encoding, the vector index, and reranking are all handled by the platform. You don't pick an embedding provider or a dimensionality—those decisions are owned by CogniWeave so retrieval quality stays consistent across upgrades and across workspaces.
What you can configure (Workspace Settings)
- • OCR: on/off, plus language hints for scanned documents.
- • Chunk size: the target length of each retrievable unit.
- • Chunk overlap: how much adjacent chunks share to preserve context across boundaries.
- • Available in the workspace UI and via PUT /api/settings/workspace.
Why embeddings and dimensions matter (background)
Embeddings turn text into vectors whose distances reflect meaning. Higher-dimensional vectors can capture more nuance but cost more to store and search. CogniWeave picks a dimensionality that balances recall and latency so you don't have to make—and later re-make—that trade-off yourself.
Why chunking matters (background)
Retrieval quality depends as much on how documents are split as on the embedding model. Chunks that are too small fragment ideas; chunks that are too large dilute relevance. Adjusting chunk size and overlap is the most direct lever you have over recall on your own corpus.
How to tune what you can change
- • Start with the defaults—they're tuned for typical mixed corpora.
- • Build a small held-out set of representative questions with known correct passages.
- • Change one variable at a time (OCR, chunk size, or overlap) and re-measure.
- • Use recall@k as the headline metric; precision@k as a tiebreaker.
- • Re-ingest affected documents after changing chunking so new settings take effect.
Common questions
Do I pick the embedding model in CogniWeave?
No. CogniWeave manages the embedding model, dimensions, and normalization for you. There is no model picker in the workspace—those choices are handled by the platform so retrieval stays consistent across upgrades.
What can I actually configure per workspace?
OCR (on/off and language), chunk size, and chunk overlap. These are exposed in Workspace Settings and via PUT /api/settings/workspace. Everything else—embedding model, vector index, retrieval and reranking—is managed by CogniWeave.
Why does chunk size matter?
Smaller chunks make retrieval more precise but can split context across boundaries. Larger chunks preserve context but dilute relevance scores. The default works for most corpora; tune only when you can measure recall on a held-out QA set.
When should I turn OCR on?
Enable OCR when your corpus includes scanned PDFs or image-only pages. For born-digital documents (exported from a word processor or generated programmatically), OCR adds ingest time with no quality benefit.