🧠 Memory Chat

Concept

LLMs have no built-in memory — they only "remember" because we re-send past turns with every request. The MessagesPlaceholder is the slot in the prompt where that conversation history is injected. The web layer is stateless: the browser keeps the running history and sends it as JSON with each message.

Theory & Concepts

The Memory Chat demo illustrates how to give a Large Language Model the illusion of memory. By default, LLMs like GPT-4o-mini are completely stateless—they do not remember the question you asked them just seconds ago. To have a continuous conversation, the application must manage the history and resend the entire transcript every time.

This demo uses Stateless Client-Side Memory combined with LangChain's prompt structuring to achieve this conversational flow:

• Client-Side State: Instead of storing the chat history in a backend database, the web browser keeps track of the conversation. Every time you type a new message, the browser sends your latest question along with the entire array of past user and assistant messages as JSON.
• Role Delineation: The backend translates this JSON history into strict LangChain objects (HumanMessage and AIMessage). This clear role separation ensures the LLM knows exactly what it said previously versus what the user asked.
• Dynamic Prompting: The core of this demo is the ChatPromptTemplate. It starts with a fixed System Persona ("You are a friendly assistant"). Then, it uses a MessagesPlaceholder("history") to dynamically inject the variable-length list of past messages. Finally, it appends your newest question at the very end.
• Execution: The LangChain chain passes this fully assembled conversation to the model. The model reads the context, generates a reply, and the backend sends it to the browser, which appends it to its local history for the next turn.

Request flow

Code flow

Backend

from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

from config import get_chat_model

# The prompt has three parts: a system persona, the history placeholder where
# past turns slot in, and the newest human question.
PROMPT = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a friendly assistant. Use the conversation history to "
            "stay consistent.",
        ),
        MessagesPlaceholder("history"),
        ("human", "{question}"),
    ]
)


def _build_chain():
    """Assemble the chat chain: prompt | model.

    Built lazily inside a function (rather than at import time) so importing
    this module never requires an API key — the model client is only created
    when a reply is actually requested.  A slightly warmer temperature keeps
    replies conversational; the parser is omitted because reply() reads
    ``.content`` from the returned message directly.
    """
    return PROMPT | get_chat_model(temperature=0.7)


def _to_messages(history: list[dict]) -> list:
    """Convert plain {role, content} dicts into LangChain message objects.

    The browser sends history as JSON dicts (``{"role": "user"|"assistant",
    "content": "..."}``).  LangChain's placeholder expects ``HumanMessage`` /
    ``AIMessage`` objects, so we translate here at the boundary.

    Args:
        history: Prior turns as a list of ``{"role", "content"}`` dicts.

    Returns:
        A list of alternating ``HumanMessage`` / ``AIMessage`` objects.
    """
    messages = []
    for turn in history:
        if turn.get("role") == "assistant":
            messages.append(AIMessage(turn.get("content", "")))
        else:
            messages.append(HumanMessage(turn.get("content", "")))
    return messages


def reply(question: str, history: list[dict] | None = None) -> str:
    """Answer a question with the conversation history for context.

    Args:
        question: The newest user message.
        history:  Prior turns as ``{"role", "content"}`` dicts.  Defaults to
                  an empty conversation.

    Returns:
        The assistant's reply text.
    """
    messages = _to_messages(history or [])
    response = _build_chain().invoke({"history": messages, "question": question})
    return response.content

API route

@bp.route("/chat", methods=["POST"])
def chat():
    """Reply to a message using the conversation history for memory."""
    data = request.get_json(force=True)
    message = (data.get("message") or "").strip()
    # history is optional; default to an empty conversation for the first turn.
    history = data.get("history") or []
    if not message:
        return jsonify({"error": "A message is required."}), 400
    try:
        text = reply(message, history)
        response = jsonify({"result": text})
        # Chat replies are turn-specific — never cache them.
        response.headers["Cache-Control"] = "no-store"
        return response
    except Exception as e:
        return jsonify({"error": str(e)}), 500