😂 Joke Generator#

Joke Generator demonstrates one of the most practical architectural patterns in modern AI engineering: provider portability via OpenAI-compatible APIs. Instead of installing a proprietary SDK for every model host, this demo reuses the exact same openai.OpenAI client from the Travel Guide demo but points its base_url at Groq's high-speed inference cloud (https://api.groq.com/openai/v1).

By swapping only three configuration values — api_key (GROQ_API_KEY), base_url (GROQ_BASE_URL), and model (llama-3.3-70b-versatile) — the service routes requests to an open-weights LLaMA 3.3 70B model running on Groq's LPU (Language Processing Unit) hardware, generating topical jokes with sub-second latency.

1. The OpenAI Chat Completions Standard as a Universal Contract

The OpenAI /v1/chat/completions JSON request/response specification has become the de facto industry wire protocol. Providers such as Groq, Google Gemini, Together AI, Fireworks, OpenRouter, Ollama, and vLLM all expose OpenAI-compatible endpoints. Writing against this single client interface prevents vendor lock-in and lets teams benchmark or fail over between cloud providers without rewriting application logic.

2. Open-Weights Models on Specialized Inference Hardware

While proprietary models (like GPT-4o) are tied to a single vendor, open-weights models (such as Meta's llama-3.3-70b-versatile or OpenAI's gpt-oss series) can be hosted by multiple competing inference providers. Groq runs these models on deterministic SRAM LPU chips, delivering hundreds of tokens per second — ideal for interactive user-facing features.

3. Decoupling Code from Provider Credentials

Notice that the message construction (system persona + user topic), the client.chat.completions.create() call, and the response extraction (response.choices[0].message.content) are line-for-line identical to the OpenAI Travel Guide service. Only the client initialization parameters change, loaded cleanly from .env.

4. Boundary Validation with Pydantic

The service defines JokeRequest and JokeResponse Pydantic schemas to document the expected JSON payload contract, supply sensible defaults (topic="programming"), and keep input/output types explicit alongside the Flask route handler.

Core AI Module (src/python/02_chat_groq.py)

src/python/02_chat_groq.py demonstrates provider portability using the OpenAI-compatible Chat Completions protocol. It reuses the standard openai.OpenAI SDK client but redirects inference to Groq's LPU cloud (https://api.groq.com/openai/v1) to run Meta's open-weights llama-3.3-70b-versatile model.

Pydantic Schemas and AI Service Function

• class JokeRequest(BaseModel): Defines the input schema with topic: str = "programming" specifying the subject of the joke.
• class JokeResponse(BaseModel): Defines the output schema pairing the requested topic: str with the generated joke: str.
• get_groq_joke(topic: str = "programming") -> str: Generates a topical joke from Groq through four numbered steps:
  1. Step ① — Initialize the OpenAI SDK Client for Groq: Instantiates OpenAI(api_key=os.getenv("GROQ_API_KEY"), base_url=os.getenv("GROQ_BASE_URL", "https://api.groq.com/openai/v1"), timeout=15.0) and selects model = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile").
  2. Step ② — Build Role-Tagged Prompt Messages: Constructs a two-element messages array with a system persona ("You are a funny joke teller.") and a user instruction (f"Tell me a joke about {topic}.").
  3. Step ③ — Call the Chat Completions Endpoint: Invokes response = client.chat.completions.create(model=model, messages=messages), dispatching an OpenAI-formatted request to Groq's inference endpoint.
  4. Step ④ — Extract and Return the Joke Text: Reads response.choices[0].message.content or "" from the completion envelope and returns the generated string.

How the AI Components Connect End-to-End

1. A topic string (modeled by JokeRequest) is passed into get_groq_joke(topic).
2. The OpenAI client is pointed at Groq's OpenAI-compatible base_url and configured with llama-3.3-70b-versatile.
3. The system persona and user topic prompt are sent together via client.chat.completions.create().
4. Groq's model generates the joke completion, and response.choices[0].message.content is extracted to populate the JokeResponse contract (topic and joke).

Groq Chat Completion Service - Joke Generator.

"""Groq Chat Completion Service - Joke Generator.

This service demonstrates using an alternative high-speed LLM provider (Groq)
via the OpenAI-compatible client interface:
1. Groq hosts open-weights models (such as LLaMA 3.3 70B) with high inference speed.
2. The standard `openai.OpenAI` client is reused by setting `base_url` to Groq's endpoint
   and authenticating with `GROQ_API_KEY`.
3. Pydantic models (JokeRequest, JokeResponse) validate and serialize API payloads.
"""

import os

from dotenv import find_dotenv, load_dotenv
from openai import OpenAI
from pydantic import BaseModel

# ---------------------------------------------------------------------------
# Configuration & Environment Setup
# ---------------------------------------------------------------------------
# Automatically locate and load the nearest .env file (e.g. at the repo or project root)
load_dotenv(find_dotenv())


# ---------------------------------------------------------------------------
# Request & Response Schemas (Pydantic)
# ---------------------------------------------------------------------------
class JokeRequest(BaseModel):
    """Input payload specifying the joke topic."""

    topic: str = "programming"


class JokeResponse(BaseModel):
    """Output payload containing the topic and generated joke."""

    topic: str
    joke: str


# ---------------------------------------------------------------------------
# Core Service Logic
# ---------------------------------------------------------------------------
def get_groq_joke(topic: str = "programming") -> str:
    """Tell a funny joke on a topic using Groq.

    Args:
        topic: Topic for the joke (defaults to "programming").

    Returns:
        A humorous joke generated by the Groq model.
    """
    # ① Initialize the OpenAI SDK client configured for Groq's base URL
    #   Read endpoint and model configuration (falling back to sensible defaults)
    client = OpenAI(
        api_key=os.getenv("GROQ_API_KEY"),
        base_url=os.getenv("GROQ_BASE_URL", "https://api.groq.com/openai/v1"),
        timeout=15.0,
    )
    model = os.getenv("GROQ_MODEL", "llama-3.3-70b-versatile")

    # ② Build the messages (system = persona, user = topic request)
    messages = [
        {"role": "system", "content": "You are a funny joke teller."},
        {"role": "user", "content": f"Tell me a joke about {topic}."},
    ]

    # ③ Call the chat completions endpoint with system persona and user topic
    response = client.chat.completions.create(model=model, messages=messages)

    # ④ Extract and return the joke text
    return response.choices[0].message.content or ""

Flask Blueprint server and route handler (joke_generator) in src/python/app.py.

"""Flask server for 01-ai-basics."""

import importlib.util
import os
import sys
from pathlib import Path

from dotenv import find_dotenv, load_dotenv
from flask import Blueprint, Flask, jsonify, request
from flask_cors import CORS
from rate_limiter import check_rate_limit

load_dotenv(find_dotenv())

PATH_PREFIX = os.environ.get("PATH_PREFIX", "")
STATIC_DIR = Path(__file__).resolve().parents[1]
app = Flask(__name__, static_folder=str(STATIC_DIR))
CORS(app)

bp = Blueprint("main", __name__)

_CUR_DIR = Path(__file__).resolve().parent


def _load_service(filename: str, alias: str):
    # ① Resolve the numbered service file path and create its import spec
    file_path = _CUR_DIR / filename
    spec = importlib.util.spec_from_file_location(f"services.{alias}", file_path)
    if spec is None or spec.loader is None:
        raise ImportError(f"Cannot load module from {file_path}")
    # ② Register and execute the module so its functions can be called by routes
    mod = importlib.util.module_from_spec(spec)
    sys.modules[f"services.{alias}"] = mod
    spec.loader.exec_module(mod)
    return mod


mod_travel = _load_service("01_chat_openai.py", "chat_openai")
mod_joke = _load_service("02_chat_groq.py", "chat_groq")


@bp.before_request
def enforce_rate_limit():
    # ① Only rate-limit mutating POST requests
    if request.method == "POST":
        # ② Check whether this client has exceeded 30 requests per hour
        blocked, msg, retry_after = check_rate_limit(
            request, max_requests=30, window_seconds=3600
        )
        if blocked:
            # ③ Return HTTP 429 with Retry-After header when rate-limited
            resp = jsonify({"error": msg})
            resp.status_code = 429
            resp.headers["Retry-After"] = str(retry_after)
            return resp


@bp.route("/")
def index():
    # ① Read index.html and inject the runtime PATH_PREFIX into data-api-base
    with open(os.path.join(app.static_folder, "index.html"), encoding="utf-8") as f:
        html = f.read()
    html = html.replace('data-api-base=""', f'data-api-base="{PATH_PREFIX}"')
    # ② Return the patched HTML page
    return app.response_class(html, mimetype="text/html")


@bp.route("/css/<path:filename>")
def css(filename):
    return app.send_static_file(os.path.join("css", filename))


@bp.route("/js/<path:filename>")
def js(filename):
    return app.send_static_file(os.path.join("js", filename))


@bp.route("/explanation/<path:filename>")
@bp.route("/info/<path:filename>")
def explanation_pages(filename):
    return app.send_static_file(os.path.join("explanation", filename))


@bp.route("/theory/<path:filename>")
def theory_pages(filename):
    """Serve the theory documentation pages from src/theory."""
    return app.send_static_file(os.path.join("theory", filename))


@bp.route("/api/travel", methods=["POST"])
def travel_suggestion():
    # ① Parse city from JSON payload and reject empty values
    data = request.get_json(force=True, silent=True) or {}
    city = (data.get("city") or data.get("message") or "").strip()
    if not city:
        return jsonify({"error": "City cannot be empty"}), 400
    try:
        # ② Call get_travel_suggestion() in 01_chat_openai.py and return JSON
        suggestion = mod_travel.get_travel_suggestion(city)
        return jsonify({"result": suggestion, "city": city, "suggestion": suggestion})
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


@bp.route("/api/joke", methods=["POST"])
def joke_generator():
    # ① Parse topic from JSON payload and reject empty values
    data = request.get_json(force=True, silent=True) or {}
    topic = (data.get("topic") or data.get("message") or "").strip()
    if not topic:
        return jsonify({"error": "Topic cannot be empty"}), 400
    try:
        # ② Call get_groq_joke() in 02_chat_groq.py and return JSON
        joke = mod_joke.get_groq_joke(topic)
        return jsonify({"result": joke, "topic": topic, "joke": joke})
    except Exception as exc:
        return jsonify({"error": str(exc)}), 500


app.register_blueprint(bp, url_prefix=PATH_PREFIX)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

A numbered reference of the core AI concepts, SDK patterns, and engineering choices demonstrated in Joke Generator:

1. OpenAI-Compatible Wire Protocol: The /v1/chat/completions JSON specification is a universal industry contract supported by Groq, Gemini, OpenRouter, Together AI, Ollama, and vLLM, enabling zero-rewrite provider switching.
2. Client Portability via base_url: Passing base_url="https://api.groq.com/openai/v1" and api_key=GROQ_API_KEY to the standard openai.OpenAI constructor redirects all SDK calls to Groq without installing a vendor-specific package.
3. Open-Weights Models (llama-3.3-70b-versatile): Unlike closed frontier models tied to a single vendor, open-weights models like Meta's LLaMA 3.3 70B can be hosted on specialized inference hardware across multiple cloud providers.
4. LPU (Language Processing Unit) Inference Speed: Groq runs transformer weights on deterministic SRAM-based LPU chips rather than HBM-bound GPUs, delivering hundreds of tokens per second with minimal time-to-first-token latency.
5. System vs. User Role Separation: The system message ("You are a funny joke teller.") sets persistent behavioral instructions, while the user message supplies the dynamic topic per request.
6. Explicit Request Timeout (timeout=15.0): Configuring a 15-second client timeout ensures slow or stalled upstream network calls fail fast instead of hanging Flask worker threads until the 60-second Nginx gateway timeout.
7. Environment-Driven Configuration (.env): Reading API keys, base URLs, and model identifiers via os.getenv() with sensible defaults follows Twelve-Factor App principles and keeps secrets out of source control.
8. Pydantic Boundary Schemas: JokeRequest and JokeResponse document input defaults and output field types alongside the service function.