feat(knowledge): add Ollama embedding provider support

[teedonk]

Summary

• Adds Ollama as an embedding provider for knowledge bases, enabling fully offline/private RAG workflows
• Implements per-KB dynamic pgvector tables to handle variable embedding dimensions across Ollama models
• Auto-detects embedding dimension and context length from Ollama's /api/show endpoint
• Adds provider selection UI with preset models (nomic-embed-text, mxbai-embed-large, all-minilm, etc.)

Fixes (#3037)

• Per-KB embedding tables solve the variable dimension problem
• Context-aware chunking and smart batching prevent context length overflow
• Cross-provider search normalizes distance scores across OpenAI and Ollama KBs
• All existing OpenAI/Azure paths remain untouched

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation
• Other: ___________

Testing

Tested with Ollama (nomic-embed-text) on local setup:

• Created KB with Ollama provider — dimension auto-detected
• Uploaded small and large documents (40+ pages) — all chunks processed successfully
• Search returns relevant results across both OpenAI and Ollama KBs
• All 4267 existing tests pass, zero TypeScript errors, Biome lint clean

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the [Contributor License Agreement (CLA)](./CONTRIBUTING.md#contributor-license-agree

Video.Project.1.mp4

ment-cla)

[vercel]

@teedonk is attempting to deploy a commit to the Sim Team on Vercel.

A member of the Team first needs to authorize it.

[cursor]

PR Summary

High Risk
High risk because it adds a new embedding provider, introduces dynamic per-knowledge-base pgvector tables with raw SQL, and changes document ingestion + search paths that affect core data storage and retrieval behavior.

Overview
Adds Ollama as an embedding provider for knowledge bases, including creation-time validation against Ollama’s /api/show, optional ollamaBaseUrl handling, and UI controls to pick preset/custom models and dimensions.

Introduces lib/knowledge/dynamic-tables.ts to manage per-KB pgvector tables (create/drop, insert/delete, and vector/tag search), and wires this into the system: KB deletion drops the per-KB table, document processing routes embeddings into either the shared embedding table (OpenAI) or the KB-specific table (Ollama), and search now splits requested KBs by provider, generates provider-specific query embeddings, queries both backends, then normalizes/merges scores before returning results.

Updates embedding/chunking to be Ollama-aware: generateEmbeddings/generateSearchEmbedding can call Ollama’s /api/embed with smart batching/truncation, and document chunk sizing/overlap are capped based on Ollama model context length; token estimation for chunking becomes model-dependent.

^{Written by Cursor Bugbot for commit 075b005. This will update automatically on new commits. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds Ollama as a fully local/private embedding provider for knowledge bases, implementing per-KB dynamic pgvector tables to handle variable embedding dimensions across different Ollama models. The overall architecture is well-reasoned — auto-detecting dimensions from /api/show, context-aware chunking, smart batching, and cross-provider search normalization — and the OpenAI/Azure paths are left untouched.

Key issues found:

• Data loss on Ollama document reprocessing (service.ts:611): The Ollama path deletes existing embeddings then inserts new ones without a transaction. A failure mid-insert leaves the document with no embeddings and no recovery path. The OpenAI path correctly uses db.transaction; Ollama should too.
• Orphaned KB row on table creation failure (route.ts:136): If createKBEmbeddingTable throws after createKnowledgeBase has already committed, the KB row is left in the database with no corresponding embedding table. The catch block should delete the orphaned KB row before re-throwing.
• Document status update regression (service.ts:628): The refactor extracted processingStatus: 'completed' from the db.transaction to a standalone call. For the OpenAI path this is a regression — if the embedding transaction succeeds but the status update fails, the document is permanently stuck in 'processing' state.
• sql.raw vector interpolation (dynamic-tables.ts:238): The query vector is interpolated into a sql.raw template string. While the value is always JSON.stringify(number[]) today, this pattern bypasses Drizzle's parameterization and is fragile for future changes. A sql tagged template with a placeholder is safer.
• Native <select> in the UI (create-base-modal.tsx): The model-picker uses a plain HTML <select> with hand-written Tailwind classes instead of the project's Select component, which won't inherit dark-mode tokens or future design-system updates.
• Module-level cache in serverless (embeddings.ts:39): ollamaModelInfoCache won't persist across serverless invocations, making the TTL cache a no-op in typical deployment targets.

Confidence Score: 3/5

• Not yet safe to merge — two P1 data-integrity bugs (non-atomic Ollama delete+insert and orphaned KB row) need to be fixed before this lands.
• The feature design is solid and well-scoped, but service.ts introduces a data-loss scenario for Ollama document processing (no transaction around delete+insert) and a regression for OpenAI (status update pulled out of its transaction). The KB creation flow also leaves orphaned DB rows if pgvector table creation fails. These three issues touch the core data path and would cause observable bugs in production. The fixes are small and targeted, but they need to land before merge.
• apps/sim/lib/knowledge/documents/service.ts (data-loss and regression), apps/sim/app/api/knowledge/route.ts (orphaned KB row), apps/sim/lib/knowledge/dynamic-tables.ts (sql.raw interpolation)

Important Files Changed

Filename	Overview
apps/sim/lib/knowledge/dynamic-tables.ts	New file implementing per-KB dynamic pgvector tables for Ollama embeddings. Core logic is sound (kbId → table name, create/drop/insert/search), but the vector literal is interpolated via sql.raw rather than parameterized, and the search functions don't wrap delete+insert atomically.
apps/sim/lib/knowledge/documents/service.ts	Two bugs introduced: (1) Ollama delete + insert embeddings are not wrapped in a transaction — partial insert failure leaves document with no embeddings (data-loss); (2) the document status update was extracted from the OpenAI transaction, meaning status can get stuck in 'processing' if the update call fails after embeddings are successfully inserted (regression).
apps/sim/app/api/knowledge/route.ts	Adds Ollama KB creation flow with dimension auto-detection. Bug: if createKBEmbeddingTable throws after createKnowledgeBase succeeds, the KB row is left orphaned in the DB with no corresponding embedding table and no cleanup.
apps/sim/lib/knowledge/embeddings.ts	Adds getOllamaModelInfo (with TTL cache), getOllamaModelContextLength, callOllamaEmbeddingAPI, and Ollama branches in generateEmbeddings/generateSearchEmbedding. Logic is clean; module-level cache is a minor concern in serverless environments.
apps/sim/app/api/knowledge/search/route.ts	Refactored to split OpenAI and Ollama KB IDs, generate separate query embeddings per provider, and merge results. The per-provider routing logic is clean and correctly handles mixed KB searches.
apps/sim/app/workspace/[workspaceId]/knowledge/components/create-base-modal/create-base-modal.tsx	Adds provider toggle (OpenAI/Ollama) with preset model selector and custom model/dimension inputs. Functional, but uses a raw HTML instead of the project's component library Select, which won't inherit dark-mode or design-system updates automatically.

Sequence Diagram

sequenceDiagram
    participant UI as CreateBaseModal
    participant API as POST /api/knowledge
    participant OllamaAPI as Ollama /api/show
    participant DB as Database
    participant DynTable as Per-KB Table

    UI->>API: POST {name, embeddingModel: "ollama/nomic-embed-text", ollamaBaseUrl}
    API->>OllamaAPI: POST /api/show {name: "nomic-embed-text"}
    OllamaAPI-->>API: {embedding_length: 768, context_length: 2048}
    API->>DB: createKnowledgeBase(data)
    DB-->>API: newKnowledgeBase {id}
    API->>DynTable: CREATE TABLE kb_embeddings_{id} (embedding vector(768))
    DynTable-->>API: OK

    Note over UI,DynTable: Document Upload Flow
    UI->>API: POST /api/knowledge/{id}/documents
    API->>OllamaAPI: GET context length (cached)
    OllamaAPI-->>API: contextLength=2048
    API->>API: chunk document (effectiveChunkSize = 0.3 × contextLength)
    API->>OllamaAPI: POST /api/embed {model, input: chunks[]}
    OllamaAPI-->>API: embeddings[]
    API->>DynTable: DELETE WHERE document_id=X
    API->>DynTable: INSERT embeddings (batched, no transaction)

    Note over UI,DynTable: Search Flow
    UI->>API: POST /api/knowledge/search
    API->>DB: SELECT embeddingModel FROM knowledgeBase WHERE id IN (...)
    DB-->>API: [{id, embeddingModel: "ollama/..."}]
    API->>OllamaAPI: POST /api/embed {model, input: [query]}
    OllamaAPI-->>API: queryVector[]
    API->>DynTable: SELECT ... ORDER BY embedding <=> queryVector LIMIT topK
    DynTable-->>API: SearchResult[]
    API-->>UI: merged results

Loading

_{Reviews (1): Last reviewed commit: "test(knowledge): update search tests for..." | Re-trigger Greptile}

[greptile-apps]

Non-atomic delete + insert risks data loss

The Ollama path deletes all existing embeddings for the document and then inserts new ones without a wrapping transaction. If insertKBEmbeddings fails mid-way (e.g., after a partial batch insert), the old embeddings have already been deleted but the new ones are only partially written — leaving the document with fewer embeddings or none at all, with no way to recover automatically.

The OpenAI path below correctly wraps both operations in a db.transaction. Consider wrapping the Ollama path similarly. Since deleteKBDocumentEmbeddings and insertKBEmbeddings both use db.execute / db.insert, they should participate in a transaction too.

[greptile-apps]

Orphaned KB row when embedding table creation fails

createKnowledgeBase persists the KB row to the database at line 136 before createKBEmbeddingTable is called. If createKBEmbeddingTable throws (e.g. the pgvector extension isn't enabled or a naming collision occurs), the error is re-thrown and the request returns 500 — but the KB record is left behind in the database without a corresponding embedding table. Any subsequent document upload to this KB will fail with a table-not-found error and the user has no way to fix it from the UI.

A safe fix is to delete the orphaned KB row in the catch block before re-throwing:

} catch (tableError) {
  logger.error(...)
  // Clean up orphaned KB row
  await deleteKnowledgeBase(newKnowledgeBase.id, requestId).catch(() => {})
  throw tableError
}

[greptile-apps]

Vector literal injected via sql.raw — fragile and risky pattern

queryVector is constructed by the caller as JSON.stringify(emb) where emb is a number[], so in practice it contains no single quotes. However, interpolating it directly into a sql.raw string template is fragile: a future refactor that passes a different value could silently introduce a SQL injection or at minimum a syntax error.

Drizzle's parameterized approach is safer and just as performant:

// Instead of:
const vecLiteral = sql.raw(`'${queryVector}'::vector`)

// Prefer:
const vecLiteral = sql`${queryVector}::vector`

Using a Drizzle placeholder ensures the value is always parameterized correctly, regardless of its content.

[greptile-apps]

Document status update moved outside the transaction (regression)

In the original code, both the embedding inserts and the processingStatus: 'completed' update were inside a single db.transaction, so they were atomic. This PR extracts the status update to a separate await db.update(document)... call that runs after the transaction.

For the OpenAI path, if the transaction (embedding inserts) succeeds but the subsequent status update fails, the document stays permanently in 'processing' state — even though its embeddings are fully in place. There's no retry that would recover this (the catch block at line 656 sets status to 'failed', which is also incorrect since the embeddings are already there).

This is a regression for the OpenAI path that was introduced by the refactor to support the Ollama code path. The processingStatus update should remain inside the db.transaction for the OpenAI path.

[greptile-apps]

Native <select> instead of the component library's Select

The rest of the form uses components from the existing @/components/ui library (e.g. Input, Label, Button). This <select> is a raw HTML element with hand-written Tailwind classes for the border/background/text, which won't automatically respect dark mode tokens or any future design-system updates.

Consider replacing it with the project's Select / SelectContent / SelectItem / SelectTrigger pattern used elsewhere in the codebase.

_{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

[greptile-apps]

Module-level cache ineffective in serverless/edge deployments

ollamaModelInfoCache is a module-level Map. In serverless environments (Next.js API routes deployed to Vercel, AWS Lambda, etc.) each invocation may spawn a fresh process where the module is re-initialized, so the cache is effectively a no-op. Additionally, multiple concurrent invocations will bypass the cache and issue redundant /api/show calls.

If the project already uses a shared cache layer (e.g. Redis via lib/core/storage), consider using it here. Otherwise, this is worth documenting as a known limitation so future maintainers aren't surprised.

[cursor]

Cross-provider normalization skips single-result sets causing unfair ranking

Low Severity

normalizeScores returns single-item arrays unchanged (if (items.length <= 1) return items). In cross-provider search, this means if one provider returns only 1 result, its raw distance (e.g., 0.5) is sorted alongside the other provider's normalized 0–1 distances. The raw score from one embedding space isn't comparable to normalized scores from another, producing incorrect ranking when merged.

 

[cursor]

Returned chunkingConfig omits stored ollamaBaseUrl field

Low Severity

createKnowledgeBase stores ollamaBaseUrl inside chunkingConfig JSONB in the database (line 104–106), but the returned object uses the original data.chunkingConfig without ollamaBaseUrl (line 123). This means the API response from KB creation reflects a different chunkingConfig than what's persisted, which could cause inconsistencies if consumers cache or rely on the returned value.

 

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

There are 4 total unresolved issues (including 2 from previous reviews).

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

[cursor]

OpenAI search embedding ignores KB's configured model

High Severity

The search route generates a single OpenAI query embedding using the hardcoded default text-embedding-3-small, ignoring each KB's actual embeddingModel. This diff newly allows text-embedding-3-large in the creation schema, but search always calls generateSearchEmbedding(query, undefined, workspaceId) where undefined defaults to text-embedding-3-small. A KB created with text-embedding-3-large (3072 dimensions) would fail at search time because the query vector (1536 dimensions) can't be compared against stored vectors via pgvector's <=> operator. The Ollama path correctly generates per-model embeddings, but the OpenAI path does not.

Additional Locations (1)

• apps/sim/app/api/knowledge/route.ts#L35-L37

 

[cursor]

Stored token count uses wrong ratio for Ollama

Low Severity

The tokenCount in embedding records is always calculated as Math.ceil(chunk.text.length / 4), using the OpenAI token estimation ratio. For Ollama models, the TextChunker uses a ratio of 3 characters per token, so the stored tokenCount is inconsistent with how the chunker actually estimated tokens. This affects any downstream analytics or display that relies on per-chunk token counts.