SentenceSplitterParams
SentenceSplitterParams =
object
Defined in: packages/core/src/schema/params.ts:6
Parameters for the SentenceSplitter
Properties
Section titled “Properties”chunkSize
Section titled “chunkSize”chunkSize:
number
Defined in: packages/core/src/schema/params.ts:8
The token chunk size for each chunk
chunkOverlap
Section titled “chunkOverlap”chunkOverlap:
number
Defined in: packages/core/src/schema/params.ts:10
The token overlap of each chunk when splitting
separator
Section titled “separator”separator:
string
Defined in: packages/core/src/schema/params.ts:12
Default separator for splitting into words
paragraphSeparator
Section titled “paragraphSeparator”paragraphSeparator:
string
Defined in: packages/core/src/schema/params.ts:14
Separator between paragraphs
secondaryChunkingRegex
Section titled “secondaryChunkingRegex”secondaryChunkingRegex:
string
Defined in: packages/core/src/schema/params.ts:16
Backup regex for splitting into sentences
extraAbbreviations
Section titled “extraAbbreviations”extraAbbreviations:
string[]
Defined in: packages/core/src/schema/params.ts:18
Extra abbreviations to consider while splitting into sentences