✂️ Chunking

Concept

LLMs have a context window limit — you can't pass an entire book as a prompt. The solution is to split the document into overlapping chunks. The RecursiveCharacterTextSplitter tries to split at natural boundaries (paragraphs, then sentences, then words) instead of cutting mid-word.

Overlapping chunks (chunk_overlap) ensures context at the boundary is never lost — this is the "context glue" that makes retrieval accurate.

Theory & Concepts

When you paste a long document into this Chunking demo, you're addressing a core limitation of Large Language Models (LLMs) and embedding models: the context window limit. You cannot feed an entire book into an API and expect it to process it in one go. Instead, the text must be broken down into smaller segments, or "chunks". In this demo, when you submit your text via the browser, the backend uses LangChain's RecursiveCharacterTextSplitter to intelligently divide your input.

Rather than simply slicing your text into arbitrary 100-character blocks (which could cut a word or sentence in half, destroying its meaning), this demo's splitter is recursive. It first tries to split your text by paragraphs. If a paragraph is still larger than the chunk_size, it drops down to splitting by sentences, and then by words. This ensures that the chunks you see rendered in the result retain their natural linguistic boundaries and semantic meaning.

Additionally, you might notice that the end of one chunk in the demo's output often repeats at the beginning of the next. This is governed by the chunk_overlap parameter (set to 10 characters in this code). This overlap acts as a contextual glue. If a critical concept in your submitted text spans across the boundary of a split, the overlap ensures that neither chunk loses the full context. This sliding window approach is what makes downstream retrieval in a full RAG pipeline accurate and reliable.

Request flow

Code flow

Backend

# Import LangChain text splitter designed to maintain semantic boundaries
from langchain_text_splitters import RecursiveCharacterTextSplitter


def chunk_text(
    text: str,
    chunk_size: int = 100,   # target character length per segment
    chunk_overlap: int = 10,  # character overlap window between adjacent chunks to preserve context
) -> list[str]:
    """Split text into overlapping chunks."""
    # Step 1: Initialize RecursiveCharacterTextSplitter with target size and overlap
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
    )
    # Step 2: Split text hierarchically (paragraphs > sentences > words) before hard-cutting
    return splitter.split_text(text)

API route

@bp.route("/chunk", methods=["POST"])
def chunk_route():
    """Split text into overlapping chunks.

    Request body (JSON): ``{ "text": "<long text>" }``
    Response (JSON):     ``{ "result": { "chunks": ["...", ...], "count": 3 } }``
    Error response:      ``{ "error": "<message>" }`` with HTTP 400 or 500
    """
    # Step 1: Parse incoming JSON request
    data = request.get_json(force=True)
    text = (data.get("text") or "").strip()

    # Step 2: Validate input boundary (reject empty or whitespace-only text)
    validation = validate_textarea(text, "Text")
    if validation:
        return validation

    # Step 3: Execute chunking and return chunk list with count
    try:
        chunks = chunk_text(text)
        return jsonify({"result": {"chunks": chunks, "count": len(chunks)}})
    except Exception as e:
        # Return error message on failure
        return jsonify({"error": str(e)}), 500