🔢 Sentence Embedding#

Sentence Embeddings demonstrates how AI systems measure semantic similarity (meaning) rather than keyword overlap. Using the local HuggingFace SentenceTransformer("all-MiniLM-L6-v2") model, the service encodes two input sentences into 384-dimensional dense floating-point vectors and computes their cosine similarity.

Because the embedding model maps concepts with similar meanings to nearby directions in 384-dimensional space, "A cat is sleeping on the couch" and "A kitten is napping on the sofa" produce a high similarity score (~0.75+) despite sharing almost no content words. The response returns the rounded similarity score, the vector dimensionality (384), and the first 5 coordinates of each vector so you can inspect the numerical representation.

1. From Discrete Words to Dense Semantic Vectors

Traditional search engines (like BM25 or SQL LIKE) match exact tokens: searching for "sofa" misses documents that only say "couch". An embedding model is a neural network trained on millions of sentence pairs to project text into a continuous 384-dimensional geometric space (R384) where distance corresponds to semantic relatedness.

2. Cosine Similarity Mathematics

To compare two embedding vectors v1 and v2, the service computes the cosine of the angle between them using NumPy's dot and norm:

similarity(v1, v2) = dot(v1, v2) / (||v1|| × ||v2||)

• Near 1.0: Vectors point in the same direction (synonyms or paraphrases).
• Near 0.0: Vectors are orthogonal (unrelated topics).
• Below 0.0: Vectors point in opposite directions.

3. Polysemy and Context Awareness

Unlike static word tables, transformer encoders read the whole sentence before producing the vector. The word "bank" in "deposit money at the bank" vs. "sit by the river bank" yields very different 384-number vectors because surrounding words shift the attention representations.

4. Startup Model Caching for Low-Latency Inference

Loading transformer weights from disk takes several hundred milliseconds. By instantiating model = SentenceTransformer("all-MiniLM-L6-v2") once at module import time, every subsequent HTTP request encodes sentences in a few milliseconds on CPU with zero external API cost.

src/python/02_embedding.py converts natural-language sentences into 384-dimensional dense numeric vectors using a local HuggingFace SentenceTransformer model and measures their semantic alignment using cosine similarity.

1. Core AI Concepts in Dense Embeddings

• Dense Vector Representations: An embedding model projects variable-length text into a fixed-size coordinate space (384 dimensions for all-MiniLM-L6-v2) where texts with similar meanings land close together even when they share zero keywords.
• Cosine Similarity: Measures the cosine of the angle between two vectors $\mathbf{v}_1$ and $\mathbf{v}_2$ via $\frac{\mathbf{v}_1 \cdot \mathbf{v}_2}{\|\mathbf{v}_1\| \|\mathbf{v}_2\|}$. Scores near 1.0 indicate strong semantic equivalence, scores near 0.0 indicate unrelated topics, and negative values indicate opposing directions.

2. Key Entities in src/python/02_embedding.py

• EmbeddingRequest(BaseModel) & EmbeddingResponse(BaseModel): Pydantic schemas defining the two input texts (text1, text2) and the output structure (text1: str, text2: str, similarity: float, dimensions: int, preview_text1: list[float], preview_text2: list[float]).
• model = SentenceTransformer("all-MiniLM-L6-v2") (Step ①): Loads the compact 384-dimensional bi-encoder model once at startup so its weights remain warm in memory for fast inference.
• compute_similarity(text1: str, text2: str) -> dict:
  • Step ② — Encode both texts into dense vectors: Calls v1 = model.encode(text1) and v2 = model.encode(text2) to produce two 384-dimensional NumPy arrays.
  • Step ③ — Compute cosine similarity: Calculates float(dot(v1, v2) / (norm(v1) * norm(v2))) using NumPy's dot product and L2 vector norms.
  • Step ④ — Format similarity metrics and vector previews: Returns a dictionary containing the rounded 4-decimal similarity score, vector dimensionality (len(v1)), and the first 5 rounded floating-point values of each vector (v1[:5] and v2[:5]).

3. End-to-End Embedding & Similarity Flow

1. compute_similarity() receives two sentences (text1 and text2).
2. The all-MiniLM-L6-v2 SentenceTransformer tokenizes each sentence and pools its transformer hidden states into a 384-dimensional vector.
3. NumPy computes the normalized dot product between v1 and v2, returning the cosine similarity score alongside 5-element vector previews and the embedding dimension count.

Sentence Embedding & Similarity Service.

"""Sentence Embedding & Similarity Service.

This service demonstrates how text embeddings work:
1. Encodes text into dense numeric vectors using SentenceTransformer ("all-MiniLM-L6-v2", 384 dims).
2. Computes cosine similarity between two sentences using dot product and vector norms.
3. Provides dimension count and vector preview values.
"""

from numpy import dot
from numpy.linalg import norm
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer


# ---------------------------------------------------------------------------
# Request & Response Schemas (Pydantic)
# ---------------------------------------------------------------------------
class EmbeddingRequest(BaseModel):
    """Input payload for text embedding and similarity comparison."""

    text1: str = "A cat is sleeping on the couch"
    text2: str = "A kitten is napping on the sofa"


class EmbeddingResponse(BaseModel):
    """Output payload containing similarity score and vector previews."""

    text1: str
    text2: str
    similarity: float
    dimensions: int
    preview_text1: list[float]
    preview_text2: list[float]


# ---------------------------------------------------------------------------
# Core Service Logic
# ---------------------------------------------------------------------------
# ① Load the compact 384-dimensional SentenceTransformer embedding model once at startup
model = SentenceTransformer("all-MiniLM-L6-v2")


def compute_similarity(text1: str, text2: str) -> dict:
    """Compute embeddings for two texts and calculate their cosine similarity.

    Args:
        text1: First input text.
        text2: Second input text.

    Returns:
        Dictionary matching EmbeddingResponse fields.
    """
    # ② Encode both texts into 384-dimensional dense vectors
    v1 = model.encode(text1)
    v2 = model.encode(text2)

    # ③ Compute cosine similarity using dot product divided by the product of vector norms
    #   similarity = dot(v1, v2) / (||v1|| * ||v2||)
    similarity = float(dot(v1, v2) / (norm(v1) * norm(v2)))

    # ④ Return similarity score, vector dimensionality, and 5-value vector previews
    return {
        "text1": text1,
        "text2": text2,
        "similarity": round(similarity, 4),
        "dimensions": len(v1),
        "preview_text1": [round(float(x), 4) for x in v1[:5]],
        "preview_text2": [round(float(x), 4) for x in v2[:5]],
    }

Flask Blueprint server and route handler (embedding_service) in src/python/app.py.

"""Flask server for 03-ai-rag-chroma."""

import importlib.util
import os
import sys
from pathlib import Path

from dotenv import find_dotenv, load_dotenv
from flask import Blueprint, Flask, jsonify, request
from flask_cors import CORS
from rate_limiter import check_rate_limit

load_dotenv(find_dotenv())

PATH_PREFIX = os.environ.get("PATH_PREFIX", "")
STATIC_DIR = Path(__file__).resolve().parents[1]
app = Flask(__name__, static_folder=str(STATIC_DIR))
CORS(app)

bp = Blueprint("main", __name__)

_CUR_DIR = Path(__file__).resolve().parent


def _load_service(filename: str, alias: str):
    # ① Resolve the numbered service file path and create its import spec
    file_path = _CUR_DIR / filename
    spec = importlib.util.spec_from_file_location(f"services.{alias}", file_path)
    if spec is None or spec.loader is None:
        raise ImportError(f"Cannot load module from {file_path}")
    # ② Register and execute the module so its functions can be called by routes
    mod = importlib.util.module_from_spec(spec)
    sys.modules[f"services.{alias}"] = mod
    spec.loader.exec_module(mod)
    return mod


mod_chunk = _load_service("01_chunking.py", "chunking")
mod_embed = _load_service("02_embedding.py", "embedding")
mod_rag = _load_service("03_rag.py", "rag")


@bp.before_request
def enforce_rate_limit():
    # ① Only rate-limit mutating POST requests
    if request.method == "POST":
        # ② Check whether this client has exceeded 30 requests per hour
        blocked, msg, retry_after = check_rate_limit(
            request, max_requests=30, window_seconds=3600
        )
        if blocked:
            # ③ Return HTTP 429 with Retry-After header when rate-limited
            resp = jsonify({"error": msg})
            resp.status_code = 429
            resp.headers["Retry-After"] = str(retry_after)
            return resp


@bp.route("/")
def index():
    # ① Read index.html and inject the runtime PATH_PREFIX into data-api-base
    with open(os.path.join(app.static_folder, "index.html"), encoding="utf-8") as f:
        html = f.read()
    html = html.replace('data-api-base=""', f'data-api-base="{PATH_PREFIX}"')
    # ② Return the patched HTML page
    return app.response_class(html, mimetype="text/html")


@bp.route("/css/<path:filename>")
def css(filename):
    return app.send_static_file(os.path.join("css", filename))


@bp.route("/js/<path:filename>")
def js(filename):
    return app.send_static_file(os.path.join("js", filename))


@bp.route("/explanation/<path:filename>")
@bp.route("/info/<path:filename>")
def explanation_pages(filename):
    return app.send_static_file(os.path.join("explanation", filename))


@bp.route("/theory/<path:filename>")
def theory_pages(filename):
    """Serve the theory documentation pages from src/theory."""
    return app.send_static_file(os.path.join("theory", filename))


@bp.route("/api/chunk", methods=["POST"])
def chunking_service():
    # ① Parse text, chunk_size, and chunk_overlap from the JSON request body
    data = request.get_json(force=True, silent=True) or {}
    text = (data.get("text") or "").strip()
    chunk_size = int(data.get("chunk_size") or 250)
    chunk_overlap = int(data.get("chunk_overlap") or 40)
    if not text:
        return jsonify({"error": "Text cannot be empty"}), 400
    try:
        # ② Split the text into overlapping chunks via 01_chunking.py and return JSON
        chunks = mod_chunk.chunk_text(text, chunk_size, chunk_overlap)
        return jsonify({
            "result": f"Split document into {len(chunks)} chunks (size {chunk_size}, overlap {chunk_overlap}).",
            "total_chunks": len(chunks),
            "chunk_size": chunk_size,
            "chunk_overlap": chunk_overlap,
            "chunks": chunks,
        })
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


@bp.route("/api/embedding", methods=["POST"])
def embedding_service():
    # ① Parse both input strings and validate that neither is empty
    data = request.get_json(force=True, silent=True) or {}
    text1 = (data.get("text1") or "").strip()
    text2 = (data.get("text2") or "").strip()
    if not text1 or not text2:
        return jsonify({"error": "Both text1 and text2 are required"}), 400
    try:
        # ② Compute vector embeddings and cosine similarity via 02_embedding.py
        res = mod_embed.compute_similarity(text1, text2)
        res["result"] = f"Cosine Similarity: {res['similarity']:.4f} (Dimension: {res['dimensions']})"
        return jsonify(res)
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


@bp.route("/api/rag", methods=["POST"])
def rag_service():
    # ① Parse the source paragraph and user question from the JSON request body
    data = request.get_json(force=True, silent=True) or {}
    paragraph = (data.get("paragraph") or "").strip()
    question = (data.get("question") or data.get("message") or "").strip()
    if not paragraph or not question:
        return jsonify({"error": "Both paragraph and question are required"}), 400
    try:
        # ② Execute the Chroma RAG pipeline in 03_rag.py and return answer + sources
        res = mod_rag.answer_rag(paragraph, question)
        res["result"] = res["answer"]
        return jsonify(res)
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


app.register_blueprint(bp, url_prefix=PATH_PREFIX)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

Use these numbered takeaways to quickly revise dense embeddings and semantic similarity:

1. What an Embedding Is: An embedding model maps variable-length text into a fixed-length dense vector of floating-point numbers (384 dimensions for all-MiniLM-L6-v2) where each coordinate captures latent semantic features learned during training.
2. Semantic vs. Lexical Matching: Traditional keyword search fails when two sentences share zero words (e.g., "reset my password" vs. "forgot my login credentials"), whereas dense embeddings map both to nearby points in vector space.
3. Cosine Similarity Formula: Computed as (v1 · v2) / (||v1|| * ||v2||), cosine similarity measures the directional alignment between two vectors regardless of their magnitude, ranging from -1.0 (opposite) through 0.0 (orthogonal/unrelated) to +1.0 (identical meaning).
4. Local Bi-Encoder Efficiency (all-MiniLM-L6-v2): Sentence-Transformers bi-encoders encode texts independently on CPU in milliseconds without requiring an external API key or per-token cloud billing.
5. Batch Encoding (model.encode([text1, text2])): Passing a list of strings to model.encode() vectorizes them in a single batched forward pass, which is significantly faster than encoding one string at a time.
6. Module-Level Model Caching: Instantiating SentenceTransformer at module scope (and caching the loaded module in app.py) avoids paying the multi-second disk and PyTorch initialization cost on every HTTP request.
7. Foundation of Vector Databases: Vector stores like ChromaDB store these exact 384-D vectors alongside document chunks and run fast approximate nearest-neighbor (ANN) cosine searches at query time.