Split long text into chunks an AI model will actually accept

Anyone who has tried to put a real document through a language model has met the context limit, and usually met it by having the answer quietly get worse rather than by getting an error. The fix is chunking, and chunking badly is why so many retrieval systems return confident nonsense.

The first thing to get right is the unit. Models count tokens, not characters or words, and the ratio is not stable: ordinary English runs about four characters to the token, but code, names, numbers and anything non-English run much shorter. Splitting on character count means some chunks are half the size they could be and others quietly overflow, and the overflow is the one you will not notice.

The second is overlap, which is the part most people skip. If a paragraph explains something in its first half and names it in its second, a hard split puts the explanation in one chunk and the name in another, and neither chunk answers the question. A small overlap means a fact that straddles a boundary still appears intact somewhere. It costs a little duplication and it prevents a whole class of confidently wrong answers.

The third is where to split, and this is judgement rather than arithmetic. Splitting mid-sentence is worse than splitting slightly early at a paragraph break, because the model reads a chunk as a passage rather than as a slice. Respecting the document's own structure beats hitting the size target exactly, more or less every time.

This is also the point in an AI project where cost and quality are actually decided, which is why it is worth doing deliberately rather than accepting a framework default. Chunk too small and you retrieve fragments without context. Chunk too large and you pay for tokens that were never relevant to the question, on every single call, forever.

It runs in your browser, which matters because the documents that need chunking are usually internal: contracts, policies, support histories, the knowledge base. Those are exactly the documents that should not be pasted into a free web service on the way to being pasted into a model.