🥊 LLM Arena

GPT-4o mini (OpenAI) vs. GPT OSS 120B (OpenAI API)

Concept

This demo sends the exact same prompt to a proprietary model (GPT-4o mini via OpenAI) and an open-source model (GPT OSS 120B via OpenAI API), then displays both replies side by side. It's a simple way to compare how two different providers respond to identical input.

Both models are called through the same helper function because they both use the OpenAI SDK — only the client and model name differ.

Theory & Concepts

Standardized Interfaces and A/B Testing

The LLM Arena demo highlights the flexibility of standard APIs and the power of side-by-side comparison:

  • Blind A/B Testing in Practice: Evaluating language models is inherently subjective. By taking your single input prompt and sending it simultaneously to both GPT-4o mini and GPT OSS 120B, this demo allows you to directly compare their reasoning, tone, and formatting in real-time. This side-by-side evaluation is exactly how industry leaderboards like LMSYS Chatbot Arena rank models based on human preference.
  • API Standardization: Notice how the code uses a single _ask() helper function for both OpenAI and the open-source Groq endpoint. Because many open-source inference providers have adopted the OpenAI API specification, we can swap between a proprietary model (GPT-4o mini) and an open-weights model (GPT OSS 120B) without rewriting the orchestration code. It's a simple matter of changing the base URL and the model name parameter.

Request flow

🧑 Browser Type a prompt, click “Battle”
↓
POST /arena Flask route
↓
battle() Sends the same prompt to both providers
↓
OpenAI API GPT-4o mini replies
OpenAI API GPT OSS 120B replies
↓
🧑 Browser Both replies rendered side by side

Code flow

flowchart TD A[Browser
prompt] -->|POST /arena| B[app.py
arena route] B -->|prompt| C[arena.py
battle] C -->|prompt| D[OpenAI API
GPT-4o mini] C -->|prompt| E[OpenAI API
GPT OSS 120B] D -->|reply A| C E -->|reply B| C C -->|both replies| B B -->|JSON result| A

Backend

Sends one prompt to two models and returns both replies. 
OPENAI_MODEL = "gpt-4o-mini"          # proprietary model
GROQ_MODEL = "openai/gpt-oss-120b"    # open-source model, served fast by OpenAI API


def _ask(client, model: str, prompt: str) -> str:
    # Both providers accept the same request format, so one helper covers both.
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content


def battle(prompt: str) -> dict:
    # Call OpenAI first (typically higher latency due to a more capable model).
    openai_reply = _ask(get_openai_client(), OPENAI_MODEL, prompt)
    # Then call the OSS endpoint via OpenAI API.
    groq_reply = _ask(get_groq_client(), GROQ_MODEL, prompt)

    # Label each reply with its model so the UI can tell them apart.
    return {
        "model_a": {"model": OPENAI_MODEL, "reply": openai_reply},
        "model_b": {"model": GROQ_MODEL, "reply": groq_reply},
    }

API route

Exposes the arena over HTTP as POST /arena. 
@bp.route("/arena", methods=["POST"])
def arena():
    data = request.get_json(force=True)
    prompt = (data.get("prompt") or "").strip()
    if not prompt:
        return jsonify({"error": "A prompt is required."}), 400  # validate at the boundary
    try:
        result = battle(prompt)
        return jsonify({"result": result})
    except Exception as e:
        return jsonify({"error": str(e)}), 500