Knowhere Node.js SDK

Official Node.js/TypeScript SDK for the Knowhere document parsing API.

Features

• 🚀 TypeScript-first - Full type safety with comprehensive type definitions
• 📦 Stream-based uploads - Efficient handling of large files
• 🔄 Automatic retries - Exponential backoff for transient failures
• 📊 Adaptive polling - Smart waiting for job completion
• 🎯 Progressive API - High-level convenience methods + low-level control
• ⚡ Modern JavaScript - ESM and CommonJS support

Installation

npm install @ontos-ai/knowhere-sdk

Requirements:

• Node.js >= 22.13.0
• npm >= 10.0.0
• TypeScript >= 5.0 (optional, for type checking)

Quick Start

import Knowhere from '@ontos-ai/knowhere-sdk';

// Initialize client
const client = new Knowhere({
  apiKey: process.env.KNOWHERE_API_KEY,
});

// Parse a document from URL
const result = await client.parse({
  url: 'https://example.com/document.pdf',
});

// Access parsed content
console.log(`Found ${result.textChunks.length} text chunks`);
console.log(`Found ${result.imageChunks.length} images`);
console.log(`Found ${result.tableChunks.length} tables`);
console.log(`Found ${result.pageChunks.length} page chunks`);

// Work with chunks — worker metadata is in chunk.metadata
result.textChunks.forEach((chunk) => {
  console.log(chunk.content);
  console.log(chunk.metadata.keywords);
  console.log(chunk.metadata.summary);
});

result.pageChunks.forEach((chunk) => {
  console.log(chunk.contentSource); // "summary"
  console.log(chunk.content); // page-level summary
  console.log(chunk.metadata.pageNums); // citation pages
});

// Save results to disk
await result.save('./output/');

Configuration

Environment Variables

KNOWHERE_API_KEY=sk_...                       # Required
KNOWHERE_BASE_URL=https://api.knowhereto.ai   # Optional

Client Options

const client = new Knowhere({
  apiKey: 'sk_...', // API authentication key
  baseURL: 'https://...', // API base URL
  timeout: 60000, // Request timeout (ms)
  uploadTimeout: 600000, // Upload timeout (ms)
  maxRetries: 5, // Max retry attempts
});

Usage Examples

Parse from File

// From file path (recommended)
const result = await client.parse({
  file: './document.pdf',
});

// From Buffer
const buffer = await fs.readFile('./document.pdf');
const result = await client.parse({
  file: buffer,
  fileName: 'document.pdf',
});

// From Stream
const stream = fs.createReadStream('./document.pdf');
const result = await client.parse({
  file: stream,
  fileName: 'document.pdf',
});

fileName is inferred automatically when file is a local file path. When file is a Buffer, Uint8Array, or a stream without path metadata, provide fileName explicitly.

Advanced Options

const result = await client.parse({
  url: 'https://example.com/doc.pdf',
  model: 'advanced', // 'base' | 'advanced'
  ocr: true, // Enable OCR
  docType: 'pdf', // Document type hint
  smartTitleParse: true, // Smart title detection
  summaryImage: true, // Generate image summaries
  summaryTable: true, // Generate table summaries
  summaryText: true, // Generate text summaries
  addFragDesc: 'Custom context', // Additional fragment description
  kbDir: 'project_docs', // Knowledge base directory
  pollInterval: 10000, // Polling interval (ms)
  pollTimeout: 1800000, // Max wait time (ms)
  verifyChecksum: true, // Verify ZIP checksum (default: true)
  webhook: {
    // Webhook for completion
    url: 'https://...',
  },
  onUploadProgress: (progress) => {
    console.log(`Upload: ${progress.percent}%`);
  },
  onPollProgress: (status) => {
    console.log(`Status: ${status.status}`);
  },
});

Bring Your Own Key (BYOK)

Pass per-request LLM credentials via llmConfig on parse/job create and retrieval queries. Flat root applies to both channels; use models for different model ids on the same endpoint, or text / vision for different provider endpoints. Use camelCase in TypeScript; the HTTP client serializes to snake_case on the wire.

// Multimodal shorthand — one model for text + vision
const llmConfig = {
  apiKey: process.env.OPENAI_API_KEY,
  model: 'gpt-4o',
  baseUrl: 'https://api.openai.com/v1',
};

// Same endpoint, different models per channel
const modelsConfig = {
  apiKey: process.env.OPENAI_API_KEY,
  baseUrl: 'https://api.openai.com/v1',
  models: { text: 'gpt-4o-mini', vision: 'gpt-4o' },
};

// Or two different endpoints
const splitLlmConfig = {
  text: {
    apiKey: process.env.OPENAI_API_KEY,
    model: 'gpt-4o-mini',
    baseUrl: 'https://api.openai.com/v1',
  },
  vision: {
    apiKey: process.env.DASHSCOPE_API_KEY,
    model: 'qwen-vl-max',
    baseUrl: 'https://dashscope.aliyuncs.com/compatible-mode/v1',
  },
};

const result = await client.parse({
  url: 'https://example.com/doc.pdf',
  llmConfig: modelsConfig,
});

const response = await client.retrieval.query({
  namespace: 'support-center',
  query: 'What is the refund policy?',
  useAgentic: true,
  llmConfig: splitLlmConfig,
});

Page Citation Assets

Page citation assets are generated by Knowhere during page-memory parsing. The SDK does not render PDF pages locally; it preserves the page image descriptors returned by the server on page chunks and retrieval metadata.

import Knowhere from '@ontos-ai/knowhere-sdk';

const client = new Knowhere({ apiKey: process.env.KNOWHERE_API_KEY });

const result = await client.parse({
  file: './manual.pdf',
});

for (const chunk of result.pageChunks) {
  console.log(chunk.metadata.pageAssets);
}

metadata.pageAssets entries point at concrete page images stored with the Knowhere result. artifactRef is the durable result artifact path; assetUrl is an optional server-generated access URL.

type PageCitationAsset = {
  pageNum: number;
  artifactRef: string;
  assetUrl?: string;
  contentType: 'image/png' | 'image/jpeg';
  width?: number;
  height?: number;
  source: 'knowhere-rendered-page-citation-source';
};

Document chunk APIs can include signed page image URLs when requested:

const chunks = await client.documents.listChunks('doc_123', {
  chunkType: 'page',
  includeAssetUrls: true,
});

console.log(chunks.chunks[0]?.metadata.pageAssets?.[0]?.assetUrl);

Low-Level API

For granular control over the job lifecycle:

// 1. Create job
const job = await client.jobs.create({
  sourceType: 'file',
  fileName: 'document.pdf',
  documentMetadata: {
    createdByClient: 'cli',
    sourceFileName: 'document.pdf',
  },
  parsingParams: { model: 'advanced', ocrEnabled: true },
});

// 2. Upload file
await client.jobs.upload(job, {
  file: './document.pdf',
  onProgress: ({ percent }) => console.log(`${percent}%`),
});

// 3. Wait for completion
const jobResult = await client.jobs.wait(job.jobId, {
  pollInterval: 10000,
});

// 4. Load results
const result = await client.jobs.load(jobResult);

Retrieval and Document Lifecycle

Published documents are queryable through the retrieval API after a job finishes. client.jobs.create(...) may return a planned documentId; persist jobResult.documentId after publication as the canonical value if you need to update or archive the same document later.

const job = await client.jobs.create({
  sourceType: 'url',
  sourceUrl: 'https://example.com/manual.pdf',
  namespace: 'support-center',
  documentMetadata: {
    createdByClient: 'notebook',
    title: 'Support manual',
  },
});

const jobResult = await client.jobs.wait(job.jobId);
const documentId = jobResult.documentId ?? job.documentId;

if (!documentId) {
  throw new Error('Expected documentId after successful publication.');
}

console.log(documentId);

// Read all initially recognized chapters without waiting for Fine or copying assets.
const directory = await client.documents.getStructure(documentId, {
  namespace: 'support-center',
});
if (directory.directoryVersion === 1) {
  console.log(directory.sections); // physical pageNums are 1-based
}

// Agentic mode (LLM navigation + answer synthesis)
const response = await client.retrieval.query({
  namespace: 'support-center',
  query: 'How do I reset Bluetooth pairing?',
  chunkTypes: ['page'],
  topK: 5,
  useAgentic: true,
});

console.log(response.answerText); // LLM-generated answer
console.log(response.referencedChunks); // cited evidence chunks
console.log(response.evidence); // grouped [E n] parts (text/HTML and inline images)
console.log(response.evidenceText); // deprecated; always empty. Use evidence.
console.log(response.stopReason); // agentic termination reason, when returned
console.log(response.failureReason); // no-answer reason, when returned

for (const result of response.results) {
  console.log(result.content); // raw path chunk for debug
  console.log(result.contentSource);
  console.log(result.score);
  console.log(result.metadata?.pageNums);
  console.log(result.source.sourceFileName, result.source.sectionPath);
}

Retrieval results use one canonical source object:

result.content;
result.chunkId;
result.chunkType;
result.contentSource;
result.score;
result.assetUrl;
result.metadata;
result.source.documentId;
result.source.sourceFileName;
result.source.sectionPath;

Agentic references expose the current retrieval citation fields:

const reference = response.referencedChunks[0];

reference.chunkId;
reference.documentId;
reference.chunkType;
reference.contentSource;
reference.sectionPath;
reference.filePath;
reference.jobId;
reference.assetUrl;
reference.metadata;

Use documentId to update or archive a document:

const updateJob = await client.jobs.create({
  sourceType: 'url',
  sourceUrl: 'https://example.com/manual-v2.pdf',
  documentId,
});

const documents = await client.documents.list({
  namespace: 'support-center',
  page: 1,
  pageSize: 50,
});
const document = await client.documents.get(documentId);
const chunks = await client.documents.listChunks(documentId, {
  page: 1,
  pageSize: 50,
  chunkType: 'page',
  includeAssetUrls: true,
});
const archived = await client.documents.archive(documentId);

console.log(documents.documents.length);
console.log(documents.pagination.totalPages);
console.log(document.status);
console.log(chunks.pagination.total);
if (chunks.chunks[0]) {
  const chunk = await client.documents.getChunk(documentId, chunks.chunks[0].id, {
    includeAssetUrls: true,
  });
  console.log(chunk.chunk.content);
  console.log(chunk.chunk.contentSource);
  console.log(chunk.chunk.metadata.pageNums);
  console.log(chunk.chunk.assetUrl);
}
console.log(archived.status);

Progressive parsing delivers job.document.directory_ready once logically per parsed revision, independently of baseline and final completion. Retries retain the same event_id; consumers should apply them idempotently and pass the event's job_result_id and namespace to getStructure. Version zero means the directory is pending. Version one includes every recognized descendant, including an empty directory when no hierarchy was found. A structure read does not trigger Fine or download page images. Raw webhook types are exported as DocumentDirectoryReadyEvent; SDK responses use camelCase and parse the directory timestamp into a Date.

Local Knowledge Tools

The SDK can also run exact inspection tools over parsed results. Local import helpers still write under the SDK cache directory by design, while published documentId reads use configured parsed storage first and fall back to Knowhere's remote chunk API without importing the result ZIP into local disk. Server workflows can configure parsed storage with client.knowledge.withParsedStorage(...) so Notebook, MCP, and CLI surfaces share the same committed result-layout model.

const parsed = await client.knowledge.parseToLocalCache({
  file: './manual.pdf',
  localDocumentId: 'manual',
});

const outline = await client.knowledge.getDocumentOutline(parsed.document.localDocumentId);

const read = await client.knowledge.readChunks({
  localDocumentId: parsed.document.localDocumentId,
  sectionPath: outline.sections[0]?.sectionPath,
  limit: 5,
});

const grep = await client.knowledge.grepChunks({
  localDocumentId: parsed.document.localDocumentId,
  pattern: 'warranty',
  maxResults: 10,
});
// grep.matches include pageNumbers when the source chunk has them.

const serverSearch = await client.knowledge.search({
  query: 'battery warranty',
  localDocumentIds: [parsed.document.localDocumentId],
  topK: 5,
});

console.log(read.chunks);
console.log(grep.matches);
console.log(serverSearch.references);

Local grep and reads use the cached parse result. Published documentId grep streams documents.listChunks(...) page by page when parsed storage is missing or stale. Search uses the Knowhere API retrieval query; local document IDs only help map returned server document IDs back to local cache IDs when available. When knowledge.parseToLocalCache(...), knowledge.importJobResult(...), or knowledge.loadJobResult(...) runs with configured parsed storage, the SDK writes the expanded Knowhere result layout (manifest.json, chunks.json, sidecars, and assets), sync progress, and a final .knowhere-sdk/commit.json before returning. Partial storage writes are ignored until the commit marker is present.

const knowledge = client.knowledge.withParsedStorage({
  storage: myParsedDocumentStorage,
  scheduler: myBackgroundScheduler,
  limits: { remotePageSize: 100, maxPagesPerSync: 10 },
});

Remote syncParsedDocument({ documentId }) and syncParsedDocument({ jobId }) stream ZIP entries without loading a full ParseResult. Add the optional ParsedDocumentStorage.writeObjectStream sink for memory usage independent of large asset sizes. Each invocation defaults to 100 entries or a 120-second budget checked between entries; re-enqueue while completed is false. Signed URLs are refreshed on every retry, and persisted checkpoints pin the revision and archive. retainResultArchive: true optionally streams the original ZIP to result.zip after committing the expanded snapshot. Servers without HTTP Range support need temporary disk space for ZIP integrity validation. See the streaming snapshot guide for the adapter contract, retry semantics, and memory validation harness.

For immutable snapshots that support object delivery, establish readability before copying every image:

const prepared = await knowledge.prepareReadableSnapshot({
  jobId: 'job_123',
  revisionKey: 'snapshot_baseline_123', // Always the snapshot ID, not the Job Result ID.
  documentId: 'doc_123',
  signal,
});

// Persist the selected revision in your durable workflow and publish your ready state.
const page = await knowledge.readChunks({
  documentId: prepared.documentId,
  revisionKey: prepared.revisionKey,
  page: 1,
});

// Resume image replication in a subsequent durable queue invocation.
const copied = await knowledge.syncParsedDocument({
  jobId: prepared.snapshot.jobId,
  revisionKey: prepared.revisionKey,
  signal,
});
// Re-enqueue the same revision while copied.completed is false.

prepareReadableSnapshot validates and stores every essential inventory object, including manifest.json, chunks.json, doc_nav.json, and source.pdf. It checks the pinned object inventory against document references and writes a separate .knowhere-sdk/readable.json descriptor. Interrupted core transfers reuse valid persisted objects on retry. Preparation does not write a full-copy commit, advance archive checkpoints, or change the current displayed snapshot. Readers must pass the selected revisionKey until the application publishes its own snapshot pointer.

readChunks supplies local essential-object URLs after the readable barrier. Image URLs use fresh capabilities through the authenticated immutable snapshot while replication is incomplete; local image URLs are preferred after a full-copy commit certifies the same document, revision, and snapshot checksum. Conflicting commit identities are rejected without reading every image body to resolve metadata URLs. Applications can also call getParsedDocumentObjectUrl({ documentId, revisionKey, path, signal }) or getParsedDocumentObjectUrls({ documentId, revisionKey, paths, signal }) directly. The plural method returns a readonly record keyed by requested paths, with null for unavailable objects. It deduplicates paths and authorizes partial-copy image URLs in sequential batches of at most 200 through the authenticated snapshot API. readChunks batches only assets referenced by its selected chunks; URL resolution does not download image bodies. Legacy full copies use their existing local URLs. Applications can use readParsedDocumentObject(...) for bytes. Object reads prefer stored bytes and verify origin image length and SHA-256 before returning uncopied bytes. Origin reads do not copy images or certify replication. Capability URLs expire and must be refreshed rather than stored as permanent identities. The application must authorize its own document owner before exposing bytes or URLs to its readers.

Use knowledge.readChunks({ documentId, revisionKey, includeAssetUrls: false }) when the application resolves stored asset references through its own authorized resource reader. URL generation is enabled by default.

Use knowledge.getDocumentSummary(reference) to build document navigation before loading any selected chunk body:

const summary = await knowledge.getDocumentSummary({ documentId, revisionKey });
console.log(summary.chunks); // Complete ordered metadata index, including unloaded chunks

The method reads only cached manifest.json and chunks.json after verifying a stored commit or readable snapshot identity. It does not construct rich chunks, read image/table files or navigation bodies, generate asset URLs, or scan remote chunks. A local cache ID (including a string reference) is also supported. A remote-only reference or unavailable stored revision rejects with ValidationError; prepare or synchronize the snapshot first. Job IDs alone do not locate a complete stored snapshot.

For a complete local or stored snapshot, readChunks also returns documentSummary: the resolved revision, full chunk/type counts, trusted manifest page count (or null), sorted original physical page numbers, presentation flags, and an ordered chunks index containing IDs, positions, types, section/source paths, physical page numbers, and optional file paths. Index entries contain no body text or asset URLs. The summary covers the complete snapshot even when the returned chunk batch is filtered or paginated. Remote-only chunk windows omit it. Descriptors also expose optional embeddedChunkIds for explicit embeds connections, preserving chapter ownership for unloaded image/table entries. Unversioned local caches report revisionKey: null; a caller-supplied revision does not certify their contents. Stored snapshots report their verified revision.

Use sourcePageNumber to read one original physical page without hydrating URLs for other pages:

const page = await knowledge.readChunks({
  documentId: prepared.documentId,
  revisionKey: prepared.revisionKey,
  sourcePageNumber: 137,
  page: 1,
  pageSize: 20,
});

sourcePageNumber must be a positive safe integer. It filters before display pagination and URL resolution, retaining original chunk IDs and global positions. Multi-page chunks retain their content while their page-asset list is narrowed to the requested physical page. totalChunks and totalPages describe matching chunks and display batches; documentSummary.pageNumbers preserves the original physical page domain, including sparse pages. The stored JSON is still read and lightly scanned in full; only selected chunks receive rich normalization and URL hydration. Physical-page reads require a complete local or stored snapshot and throw ValidationError for remote-only documents; prepare or synchronize the pinned snapshot first. Ordinary remote reads keep their existing behavior.

Only syncParsedDocument(...).completed and its existing full commit certify the complete archive copy and integrity checks. An origin or replication failure does not remove the readable descriptor. Historical snapshots without immutable object delivery return the API's unsupported/not-found error from preparation; callers can retain the existing full-archive synchronization path for those snapshots.

If a search result only has a published documentId, read-oriented helpers can accept that remote identifier directly. readChunks supports display paging; remote fallback requests asset URLs from the Knowhere API, and storage hits use stored object URLs when the configured storage can resolve them:

const remoteRead = await knowledge.readChunks({
  documentId: 'doc_123',
  page: 1,
  pageSize: 20,
  chunkType: 'page',
});

const remoteOutline = await knowledge.getDocumentOutline({
  documentId: 'doc_123',
});

const grep = await knowledge.grepChunks({
  documentId: 'doc_123',
  pattern: 'warranty',
  maxResults: 10,
});

const jobRead = await knowledge.readChunks({
  jobId: jobResult.jobId,
  localDocumentId: 'manual',
  limit: 5,
});

The MCP package is a wrapper over this SDK interface; install it only when an agent host needs an MCP server. See the MCP package README for Codex, Claude Code, Claude Desktop, and generic stdio MCP host configuration examples.

For longer parses, use the non-blocking SDK flow and cache the result after the job completes:

const started = await client.knowledge.startParse({
  file: './manual.pdf',
  localDocumentId: 'manual',
});

const status = await client.knowledge.getJobStatus(started.job.jobId);

if (status.job.isDone && status.cache.document) {
  console.log(status.cache.document.localDocumentId);
}

When the job was started through client.knowledge.startParse(...), getJobStatus(...) automatically caches the completed result locally the first time it observes status.job.isDone. Use importJobResult(...) to recover a completed job into the local cache when it was not started through the local knowledge helper, or to retry a local import step explicitly. Use loadJobResult(...) for server workflows that should load a completed result without creating SDK local-disk cache state. Use syncParsedDocument(...) to explicitly resume or retry parsed-storage sync for an existing documentId, jobId, or local parsed result.

Retrieval queries can limit documents and exclude documents or sections for one request. includeDocumentIds?: string[] restricts retrieval to the supplied document IDs. Omitting it leaves documents unrestricted by inclusion; passing [] matches no documents. Exclusions take precedence over inclusions, including when a document ID appears in both includeDocumentIds and excludeDocumentIds.

const followUp = await client.retrieval.query({
  namespace: 'support-center',
  query: 'battery charging',
  includeDocumentIds: ['doc_123', 'doc_old'],
  excludeDocumentIds: ['doc_old'],
  excludeSections: [{ documentId: 'doc_123', sectionPath: 'Appendix / Legal' }],
});

Error Handling

import {
  BadRequestError,
  AuthenticationError,
  RateLimitError,
  PollingTimeoutError,
  JobFailedError,
  ValidationError,
  InvalidStateError,
} from '@ontos-ai/knowhere-sdk';

try {
  const result = await client.parse({ url: '...' });
} catch (error) {
  if (error instanceof ValidationError) {
    console.error('Invalid parameters:', error.message);
  } else if (error instanceof RateLimitError) {
    // Wait and retry
    await sleep(error.retryAfter * 1000);
  } else if (error instanceof AuthenticationError) {
    console.error('Invalid API key');
  } else if (error instanceof PollingTimeoutError) {
    console.error('Processing timeout');
  } else if (error instanceof JobFailedError) {
    console.error('Job failed:', error.jobResult.error);
  } else if (error instanceof InvalidStateError) {
    console.error('Invalid state:', error.message);
  }
}

Documentation

For complete documentation, visit https://docs.knowhereto.ai

Examples

Check out the examples directory for more usage examples:

• Basic Usage
• File Upload
• Error Handling
• Low-Level API

Development

# Install dependencies
npm ci

# Run tests
npm test

# Run tests with coverage
npm run test:ci

# Lint code
npm run lint

# Format code
npm run format

# Type check
npm run typecheck

# Build
npm run build

Release Workflow

See docs/release-workflow.md for the Changesets-based stable and beta release process.

Community

• Contributing guide: CONTRIBUTING.md
• Security policy: SECURITY.md
• Code of conduct: CODE_OF_CONDUCT.md

License

MIT

Support

• 📧 Email: team@knowhereto.ai
• 🐛 Issues: GitHub Issues
• 📚 Documentation: https://docs.knowhereto.ai

Changelog

See CHANGELOG.md for release history.