Type alias IndexOptions

Type declaration

• Optional batchSize?: number

  The number of documents to index in one batch.

• Optional cleanup?: CleanupMode

  The cleanup mode to use. Can be "full", "incremental" or undefined.

  • Incremental: Cleans up all documents that haven't been updated AND that are associated with source ids that were seen during indexing. Clean up is done continuously during indexing helping to minimize the probability of users seeing duplicated content.
  • Full: Delete all documents that haven to been returned by the loader. Clean up runs after all documents have been indexed. This means that users may see duplicated content during indexing.
  • undefined: Do not delete any documents.

• Optional cleanupBatchSize?: number

  Batch size to use when cleaning up documents.

• Optional forceUpdate?: boolean

  Force update documents even if they are present in the record manager. Useful if you are re-indexing with updated embeddings.

• Optional sourceIdKey?: StringOrDocFunc

  Optional key that helps identify the original source of the document. Must either be a string representing the key of the source in the metadata or a function that takes a document and returns a string representing the source. Required when cleanup is incremental.