🎯 InterviewIQ#

Hiring teams often want a fast, first-pass read on a candidate's interview answers before a human reviews them. Your task is to build InterviewIQ: an AI agent that acts as a mock-interview coach with a working browser interface. Given a candidate's spoken answer to an interview question, the agent evaluates it using a set of tools, produces short feedback, remembers the whole session well enough to answer questions about it at any point, and surfaces all of that through a UI — not just a terminal.

Ask 1 — Build a tool-calling evaluator agent.
Implement three rule-based tools the agent can call, each returning a structured dict:

1. detect_filler_words — flags filler words like "um", "like", "basically".
2. check_star_structure — checks whether a behavioral answer covers Situation, Task, Action, Result.
3. score_relevance — scores (0–100) how many expected keywords/concepts for the question appear in the answer.

The agent should read the candidate's answer, decide which tool(s) are relevant, call them, and turn the results into short, encouraging feedback.

Ask 2 — Session memory with a real aggregation, verified by a memory check.
The agent must remember every answer's evaluation across the session — not just the current one — and be able to answer meta-questions at any point mid-session, such as "how am I doing so far?" or "what's my weakest area?", not only produce a report at the very end. The report must do real aggregation: an average relevance score and a named weakest area (the question with the lowest score), not a flat recap of everything scored so far.

Run memory_check.py against your finished agent: it scores one strong answer and one deliberately weak one, then asks the meta-question. If your agent correctly names the weak answer's question as the weakest area — not whichever question was asked most recently — your memory and aggregation are working. If it gets this wrong, your agent is only looking at the last turn.

Ask 3 — A working web interface (required).
Wrap the agent in a plain HTML/JS web app (app.py acting as an API, with a index.html frontend) with: the current question and an answer box, a feedback panel, a live scorecard showing every question answered so far, a free-form box to ask the coach meta-questions, and a button to pull the final report. This is the interface you'll demo — not optional, and not just a CLI print loop.

Ask 4 (bonus, not graded) — a second agent.
None of the six classes actually built a multi-agent system — Class 6's guide only introduces the vocabulary (an "orchestrator" coordinating "specialized agents") and explicitly scopes out building one as "a larger topic for later." So this ask sits outside the core project on purpose: interviewer_agent.py and run_multi_agent.py are included fully worked, not as a TODO — a second agent (the "Interviewer") whose only job is to decide the next question's category based on how the candidate has done so far, handed off to the Evaluator agent from Asks 1–3 (completely unchanged) by a small orchestrator loop. Run python run_multi_agent.py to try it. Read it, run it, and extend it if you want to go further — there's nothing here you're required to build or that gets graded, since it goes beyond what was actually taught.

1. Get the plumbing working first (API key, one plain LLM call) before writing any tool logic.
2. Write and test each tool as a standalone Python function — don't wire it to the agent until you're confident it works on its own.
3. Wire the tools to the agent and get one full evaluated turn working via main.py before touching memory.
4. Add memory and the aggregation report, then run memory_check.py to prove it actually uses more than the last turn.
5. Only once main.py and memory_check.py both work should you move to app.py — the UI should be the last layer on top of an agent you already trust.

1. Environment setup — venv, pip install -r requirements.txt, .env configured with your API key.
2. First plain LLM call working (no tools yet) — sanity check that your key and provider are correct.
3. detect_filler_words implemented and tested directly.
4. check_star_structure implemented and tested directly.
5. score_relevance implemented and tested directly.
6. All three tools wired into the agent's tool-calling loop; one full evaluated turn works end-to-end via main.py.
7. Session memory added — running a few questions shows it accumulating.
8. generate_final_report implemented with real aggregation (average relevance + named weakest area).
9. ask_agent implemented so meta-questions work mid-session, not just at the end.
10. python memory_check.py passes — the weakest area named matches the deliberately weak answer, not the most recent turn.
11. app.py built — the full flow (question → answer → feedback → scorecard → meta-question → final report) works in the browser.
12. (Bonus, ungraded) Run python run_multi_agent.py and read interviewer_agent.py — a second agent handed off to your Evaluator by a small orchestrator.

InterviewIQ is an AI-powered mock-interview coach engineered with a multi-turn ReAct (Reasoning + Acting) tool-calling architecture. Instead of relying on a single subjective prompt that risks arithmetic hallucinations and inconsistent grading, InterviewIQ embeds the LLM as a cognitive orchestrator inside an autonomous loop.

The agent dynamically decides which specialized diagnostic tools to invoke (filler-word density, STAR framework coverage, keyword relevance scoring), inspects the structured outputs, and synthesizes nuanced, actionable coaching advice. In parallel, an append-only session memory ledger tracks candidate performance across all turns, applying deterministic mathematical ranking to eliminate recency bias and power mid-session meta-coaching.

A second, much smaller Interviewer agent (Ask 4, the bonus) sits beside the Evaluator. Its only job is to read the shared session memory and pick the category of the next question. It never scores answers. A small orchestrator hands work between the two agents: the GET /next-question route behind the Interviewer: Pick Next button, or the run_multi_agent.py loop on the command line. The Evaluator stays exactly as it was.

1. Autonomous ReAct Agent Loop vs. Static Pipelines

Traditional LLM applications operate in a rigid, single-turn sequence: User Prompt → LLM → Output. This passive paradigm struggles in complex evaluation tasks because language models suffer from cognitive overload, hallucination during precise mathematical counting, and an inability to inspect intermediate diagnostic data.

InterviewIQ implements the ReAct (Reasoning + Acting) framework (Yao et al., 2022) via the OpenAI Function Calling protocol. The LLM operates in an autonomous loop:

• Tool Schema Contract: The application exposes a JSON schema (TOOLS_SCHEMA) defining available Python evaluation functions, their parameter specifications, and natural-language descriptions.
• Autonomous Tool Selection: Given a candidate's answer and question metadata, the LLM acts as an autonomous planner. It decides which tools are necessary (e.g., invoking detect_filler_words, check_star_structure, and score_relevance in parallel or sequentially) and extracts the appropriate arguments.
• Environment Execution & Feedback: The runtime dispatches tool calls to deterministic Python functions, captures the structured outputs, and appends them back to the message transcript under the tool role.
• Iterative Convergence: The agent loops (up to MAX_TOOL_ROUNDS) until all necessary observations are gathered, then switches from tool calling to generating its final synthesized coaching critique.

2. Separation of Concerns: Deterministic NLP vs. Generative Synthesis

A core principle in robust AI engineering is never asking an LLM to do what deterministic code does with 100% precision. Large Language Models are probabilistic next-token predictors; they are inherently unreliable at exact character counting, regex matching, and statistical calculations.

• Exact Filler-Word Density (detect_filler_words): Uses compiled regex word-boundary patterns (\b(um|uh|like|basically|actually|literally|you know)\b) to compute exact word frequencies and normalized density per 100 words ((fillers / word_count) * 100).
• STAR Structural Parsing (check_star_structure): Heuristically parses behavioral responses across the 4 pillars of the STAR framework (Situation, Task, Action, Result) using contextual phrases, temporal indicators, ownership verbs, and quantifiable impact metrics (e.g., \d+% or \$\d+).
• Inflectional Keyword Relevance (score_relevance): Generates morphological regex stems to match expected domain keywords across grammatical inflections (e.g., -tion, -ment, -e, -y), scoring coverage against a calibrated tiered curve (0–100).
• Empathetic Cognitive Synthesis: By delegating mathematical and structural analysis to deterministic tools, the LLM is freed from arithmetic. It uses its language capability solely to interpret raw metrics, contextualize trade-offs, and deliver encouraging, constructive feedback.

3. Structured Session Memory & Anti-Recency Bias

Standard conversational AI setups pass entire chat transcripts into the prompt context. This introduces severe recency bias and context dilution ("lost-in-the-middle" phenomenon), where the model overemphasizes recent turns and forgets early answers.

InterviewIQ overcomes this using an append-only structured session ledger (InterviewSessionMemory):

• Immutable Turn Ledgers: Every interview turn stores the question metadata, candidate answer, and raw structured dicts from all tool executions.
• Mathematical Composite Ranking: Weakest and strongest performance areas are determined globally across all turns using a deterministic composite sort key:
  rank_key(turn) = (relevance_score ↑, star_score ↑, -total_filler_count ↓)
  The globally weakest turn is computed as min(turns, key=rank_key), ensuring an objective assessment even if the candidate aced the most recent question.
• Dynamic Meta-Tool Invocation: The agent exposes generate_final_report as an executable tool. When a candidate asks a mid-session meta-question (e.g., "What is my weakest area so far?"), the LLM autonomously calls this tool, reads the aggregated memory state, and provides an evidence-based progress review.

4. Orchestrator + Specialised Agents (Ask 4)

A multi-agent system does not need a protocol to start with. Here it is just two agents, each with its own system prompt and one job, plus a loop that decides who acts next:

• Interviewer agent (InterviewerAgent.choose_next): reads a short recap of every turn and returns JSON {"category", "reason"}. Its rule of thumb: after a weak answer (relevance below 60), stay in the same family (people = Behavioral, Leadership; engineering = Technical, System Design, Problem-Solving) so the candidate gets another try at similar skills. After a strong answer, switch family to test breadth.
• Evaluator agent (EvaluatorAgent.evaluate_answer): the tool-calling agent from Asks 1–3, unchanged.
• Shared memory as the hand-off: the Evaluator writes each turn into InterviewSessionMemory and the Interviewer only reads it, so neither agent calls the other directly.
• Guard rails: the Interviewer may only pick a category that still has an unanswered question. If the model's reply is unusable, or there is no API key, the same rule of thumb runs in plain Python. The UI labels the decision as an "AI decision" or a "rule-based fallback".

Interviewer hand-off (Ask 4)

InterviewIQ's core AI architecture is organized across four focused modules that combine an autonomous ReAct tool-calling loop, deterministic NLP diagnostic tools, anti-recency-bias session memory, and a secondary Interviewer agent:

• interview_bank.py — Defines the 5 categorized interview questions (QUESTIONS), domain keyword rubrics (expected_keywords), and reference strong/weak answers used to evaluate candidate responses.
• agent.py — Implements EvaluatorAgent (the ReAct tool-calling loop that evaluates candidate answers and answers mid-session meta-questions) and InterviewSessionMemory (the append-only session ledger with composite anti-recency ranking and report aggregation).
• tools.py — Implements the three deterministic, rule-based diagnostic tools invoked by EvaluatorAgent during its ReAct loop: detect_filler_words, check_star_structure, and score_relevance (with morphological stem builder _build_keyword_pattern).
• interviewer_agent.py — Implements InterviewerAgent (Ask 4), a second specialized agent that reads shared InterviewSessionMemory and selects the next question's category based on candidate performance.

1. Question Bank & Keyword Rubrics (interview_bank.py)

• QUESTIONS: Defines 5 curated interview questions across Behavioral, Technical, Problem-Solving, Leadership, and System Design, each storing id, category, question, expected_keywords, sample_strong_answer, and sample_weak_answer.
• Lookup Helpers: get_all_questions() returns a shallow list copy of the question bank; get_question_by_id(question_id) iterates through QUESTIONS (Steps ①–②) to return the matching rubric dict or None (Step ③); and get_question_categories() returns the deduplicated category list in bank order.

2. ReAct Evaluator Agent & Anti-Recency Session Memory (agent.py)

• class InterviewSessionMemory: Append-only ledger (self._turns: list[dict]) storing every evaluated turn across the interview:
  • turns & turn_count properties: Return a shallow copy of recorded turns and the total turn count.
  • add_turn(question, answer, results, category="", question_id=0, expected_keywords=None): Step ① appends a structured turn record containing turn_id, question_id, question, category, expected_keywords, answer, and the raw results dictionary returned by the evaluation tools.
  • get_weakest_area() -> dict | None: Eliminates LLM recency bias by ranking all turns globally. Step ① returns None if empty; Step ② defines a 3-tuple composite sort key sort_key(turn) = (rel, star, -fillers) where lower relevance, lower STAR percentage, and higher filler count rank as weaker; Step ③ selects min(self._turns, key=sort_key); Step ④ returns the weakest turn's question, category, scores, and unmatched_keywords.
  • get_strongest_area() -> dict | None: Uses the exact same composite sort_key (Steps ①–②) with max(self._turns, key=sort_key) (Step ③) to return the strongest turn's summary (Step ④).
  • get_average_relevance() -> float: Collects relevance scores across all turns (Step ①) and returns their rounded mean or 0.0 (Step ②).
  • get_scorecard() -> list[dict]: Loops through self._turns (Step ①), extracts relevance, STAR, and filler metrics (Step ②), and formats compact turn rows (Turn, Category, truncated Question, Relevance Score, STAR Score, Fillers) (Step ③).
  • get_category_breakdown() -> dict: Groups relevance and STAR scores by question category (Step ①), computes per-category averages and question counts (Step ②), and returns the breakdown dict (Step ③).
  • generate_final_report_dict() -> dict: Aggregates all recorded turns into a structured assessment report. Returns an empty-session message if no turns exist (Step ①); computes session-wide totals, averages, STAR mastery rate, and strongest/weakest areas (Step ②); formats top-line metrics (Step ③), strongest/weakest highlights with missed keywords (Step ④), category breakdown bullets (Step ⑤), and threshold-based coaching recommendations (Step ⑥); then returns both machine-readable metrics and report_text (Step ⑦).
  • clear(): Resets self._turns for a fresh session.
• TOOLS_SCHEMA, SYSTEM_PROMPT, MAX_TOOL_ROUNDS = 5, MAX_GLITCH_RETRIES = 3:
  • TOOLS_SCHEMA exposes 4 OpenAI function-calling schemas to the LLM: detect_filler_words(answer), check_star_structure(answer), score_relevance(answer, expected_keywords), and generate_final_report() (which takes no arguments and lets the LLM answer mid-session meta-questions).
  • SYSTEM_PROMPT instructs the model to call the evaluation tools before giving short, encouraging feedback, and to invoke generate_final_report whenever the candidate asks how they are doing or what their weakest area is.
• class EvaluatorAgent:
  • __init__(self, memory=None): Step ① attaches InterviewSessionMemory and initializes the LLM client and model (openai/gpt-oss-20b via Groq or gpt-4o-mini via OpenAI, with offline fallback); Step ② binds self._tool_functions mapping each schema name to its Python callable.
  • evaluate_answer(self, question_data: dict, answer: str) -> dict: Step ① unpacks question, expected_keywords, category, and id; Step ② falls back to _deterministic_evaluate() if no LLM client is configured; Step ③ constructs the user prompt with the question, candidate answer, and expected keywords; Step ④ initializes messages with SYSTEM_PROMPT; Step ⑤ runs self._run_loop(messages, results_out=results) to execute tools and synthesize coaching feedback, then logs the turn via self.memory.add_turn(...); Step ⑥ returns turn, feedback, relevance_evaluation, star_evaluation, and filler_evaluation.
  • ask_agent(self, user_message: str) -> str: Handles free-form coaching meta-questions (e.g., "What is my weakest area so far?"). Uses generate_final_report() directly in offline mode (Step ①) or sends user_message through self._run_loop(messages) (Step ②) where the LLM autonomously invokes the generate_final_report tool.
  • generate_final_report(self) -> str: Formats a plain-text session summary (Steps ①–⑤) listing every turn's filler, STAR, and relevance metrics plus the session-wide average relevance, globally weakest question from self.memory.get_weakest_area(), and total fillers.
  • _create_with_retry(self, **kwargs): Calls self._client.chat.completions.create(**kwargs) up to MAX_GLITCH_RETRIES (3) times (Step ①), retrying if the model leaks a formatting token into a tool name (code == "tool_use_failed") (Step ②), and re-raising on final failure (Step ③).
  • _run_loop(self, messages: list, results_out: dict | None = None) -> str: The core ReAct loop running up to MAX_TOOL_ROUNDS (5) turns. Step ① calls _create_with_retry() with tools=TOOLS_SCHEMA and tool_choice="auto"; Step ② returns msg.content immediately once msg.tool_calls is empty; Step ③ parses JSON arguments and dispatches each requested tool in self._tool_functions; Step ④ saves per-answer tool output dicts into results_out; Step ⑤ appends each tool result to messages with role="tool" and tool_call_id=call.id; Step ⑥ requests a final synthesis without tools if all 5 rounds are exhausted.
  • _deterministic_evaluate(...): Offline fallback (Steps ①–⑤) that calls all three diagnostic tools directly, records the turn in self.memory, and formats their summary messages when no LLM client is available.

3. Deterministic Diagnostic Tools (tools.py)

• FILLER_PATTERNS & detect_filler_words(answer: str) -> dict:
  • FILLER_PATTERNS defines 12 word-boundary regexes for conversational crutches ("you know", "sort of", "kind of", "i mean", "um", "uh", "like", "basically", "actually", "literally", "honestly", "right").
  • Steps ①–⑥: Rejects empty answers (Step ①), lowercases text (Step ②), counts matches per pattern with re.findall() (Step ③), computes total word count and filler_density_per_100_words = round((total_count / max(word_count, 1)) * 100, 1) (Step ④), formats a diagnostic message (Step ⑤), and returns the structured metrics dict (Step ⑥).
• STAR_PATTERNS & check_star_structure(answer: str) -> dict:
  • STAR_PATTERNS maps each of the 4 STAR pillars (Situation, Task, Action, Result) to regex lists matching contextual openers, ownership verbs, and quantifiable outcomes (e.g., \d+%, \$\d+).
  • Steps ①–⑥: Handles empty input (Step ①), normalizes text (Step ②), checks whether each pillar matches at least one pattern (Step ③), computes star_score = round((len(covered) / 4) * 100, 1) and is_star_complete (Step ④), builds a coaching tip naming any missing_components (Step ⑤), and returns the full breakdown dict (Step ⑥).
• _build_keyword_pattern(kw_clean: str) -> str & score_relevance(answer: str, expected_keywords: list[str]) -> dict:
  • _build_keyword_pattern() constructs inflection-tolerant regexes: handles multi-word/hyphenated terms (Step ①) and expands -tion (Step ②), -ment (Step ③), -e (Step ④), -y (Step ⑤), and standard plural/verb suffixes (Step ⑥) so "resolved" matches "resolution" and "caching" matches "cache".
  • score_relevance() handles empty answers (Step ①) or empty rubrics (Step ②), normalizes text (Step ③), tests each expected keyword with _build_keyword_pattern() (Step ④), computes coverage_ratio = total_matched / max(total_expected, 1) (Step ⑤), maps coverage onto a calibrated 4-tier curve (>=70% → 90–100, >=45% → 75–89, >=25% → 50–74, <25% → 5–49) (Step ⑥), writes a summary message listing top missed keywords (Step ⑦), and returns the structured score dict (Step ⑧).

4. Specialized Interviewer Agent (interviewer_agent.py)

• Adaptive Category Routing (WEAK_THRESHOLD = 60, CATEGORY_FAMILIES, SYSTEM_PROMPT): Groups categories into "people" (Behavioral, Leadership) and "engineering" (Technical, System Design, Problem-Solving). After a weak answer (relevance < 60), the Interviewer stays in the same family to reinforce the skill; after a strong answer, it switches family to test breadth.
• class InterviewerAgent:
  • __init__(self, client=None, model=None): Stores an injected or configured LLM client and model identifier (Step ①).
  • choose_next(self, memory) -> dict | None: Step ① computes unanswered questions and remaining categories from memory.turns (returning None when all 5 bank questions are answered); Step ② queries _ask_llm(memory, available) if an LLM client exists and multiple categories remain, falling back to _rule_based(memory, available) otherwise; Step ③ attaches the first unanswered question matching the chosen category and returns {"category", "reason", "source", "question"}.
  • build_summary(memory, available): Formats a concise plain-English recap of all turns so far (Steps ①–②), highlights the most recent answer's score and weak/strong verdict (Step ③), and lists allowed categories (Step ④).
  • _ask_llm(self, memory, available) & _parse_reply(content, available): Makes a single tool-free chat completion call (Step ①) and extracts (category, reason) from JSON or text while validating that the chosen category is genuinely available (Step ②).
  • _rule_based(memory, available): Deterministic policy fallback that opens with Behavioral (Step ①), stays in the same family after a score below 60 or switches family after >= 60 (Step ②), and generates a human-readable explanation labeled source="rules" (Step ③).

5. End-to-End Multi-Agent & ReAct Flow

1. When a candidate submits an answer for a question from interview_bank.py, EvaluatorAgent.evaluate_answer() in agent.py starts a ReAct conversation with SYSTEM_PROMPT and TOOLS_SCHEMA.
2. Inside EvaluatorAgent._run_loop(), the LLM autonomously requests tool calls for detect_filler_words, check_star_structure, and score_relevance.
3. agent.py executes each deterministic diagnostic function in tools.py, records the structured dictionaries in results, and appends them as role="tool" messages so the LLM can synthesize grounded, constructive coaching feedback.
4. evaluate_answer() appends the turn and raw tool dicts to InterviewSessionMemory via self.memory.add_turn(...).
5. When the candidate asks a mid-session meta-question (e.g., "What is my weakest area so far?"), EvaluatorAgent.ask_agent() runs the same ReAct loop; the LLM calls generate_final_report, which queries self.memory.get_weakest_area() (using the anti-recency 3-tuple composite sort key across all turns) to return an accurate global assessment.
6. To select the next question, InterviewerAgent.choose_next() in interviewer_agent.py reads the shared InterviewSessionMemory ledger and picks a category from the same skill family after a weak score (< 60) or switches families after a strong score.

Interviewer hand-off (Ask 4)

Flask server exposing InterviewIQ evaluation endpoints.

"""Flask server exposing InterviewIQ evaluation endpoints.

Architecture notes
------------------
- All routes are attached to a Blueprint (``bp``) instead of directly to
  ``app``.  This lets us register the entire Blueprint under a runtime URL
  prefix (``PATH_PREFIX``) without touching individual route strings.
- In local development PATH_PREFIX is empty, so routes are at "/", "/evaluate",
  etc.  In production Nginx forwards ``/09-interviewiq/...`` traffic to the
  container and PATH_PREFIX is set to "/09-interviewiq", keeping every URL
  consistent.
- flask-cors adds ``Access-Control-Allow-Origin: *`` headers so the HTML
  page can call the API even if it is served from a different origin during
  development.
"""

import os
from pathlib import Path

from flask import Blueprint, Flask, jsonify, request
from flask_cors import CORS

from agent import EvaluatorAgent, InterviewSessionMemory
from interview_bank import get_all_questions, get_question_by_id
from interviewer_agent import InterviewerAgent
from rate_limiter import check_rate_limit

# ---------------------------------------------------------------------------
# Configuration
# ---------------------------------------------------------------------------

# PATH_PREFIX is set by the deployment environment (e.g. "/09-interviewiq") so
# the app works correctly behind an Nginx location block.  Locally it is an
# empty string, which mounts all routes at the root.
PATH_PREFIX = os.environ.get("PATH_PREFIX", "")

# app.py lives in src/python, while index.html, css/, and js/ live in src/.
STATIC_DIR = Path(__file__).resolve().parents[1]
app = Flask(__name__, static_folder=str(STATIC_DIR))

# Allow cross-origin requests from any origin.  In production you would
# restrict this to the specific front-end domain.
CORS(app)

# A Blueprint groups related routes.  We register it once at the bottom with
# the runtime PATH_PREFIX, avoiding any hardcoded path strings in the routes.
bp = Blueprint("main", __name__)


@bp.before_request
def enforce_rate_limit():
    """Enforce strict 30 requests per hour limit on all POST endpoints."""
    # ① only rate-limit mutating API requests
    if request.method == "POST":
        # ② ask the shared rate limiter whether this client is blocked
        blocked, msg, retry_after = check_rate_limit(
            request, max_requests=30, window_seconds=3600
        )
        if blocked:
            # ③ return a 429 response with a retry hint for the browser
            resp = jsonify({"error": msg})
            resp.status_code = 429
            resp.headers["Retry-After"] = str(retry_after)
            return resp


# Shared session memory and evaluator agent (single-process, not multi-user).
_memory = InterviewSessionMemory()
_agent = EvaluatorAgent(memory=_memory)
# Ask 4: a second agent that reads the same memory to pick the next category.
_interviewer = InterviewerAgent()



# ---------------------------------------------------------------------------
# Routes
# ---------------------------------------------------------------------------


@bp.route("/")
def index():
    """Serve index.html, injecting the correct API base URL for the environment."""
    # ① read the frontend HTML template from the static folder
    with open(os.path.join(app.static_folder, "index.html"), encoding="utf-8") as f:
        html = f.read()
    # ② replace the empty data-api-base with the deployment path prefix
    # The HTML file ships with 'data-api-base=""' (empty = relative URL, works
    # locally).  For production we replace it with the actual path prefix so
    # all fetch() calls in the browser target the right endpoint.
    html = html.replace('data-api-base=""', f'data-api-base="{PATH_PREFIX}"')
    # ③ return the patched HTML response to the browser
    return app.response_class(html, mimetype="text/html")


@bp.route("/css/<path:filename>")
def css(filename):
    """Serve stylesheets from the src/css directory."""
    return app.send_static_file(os.path.join("css", filename))


@bp.route("/js/<path:filename>")
def js(filename):
    """Serve scripts from the src/js directory."""
    return app.send_static_file(os.path.join("js", filename))


@bp.route("/explanation/<path:filename>")
@bp.route("/info/<path:filename>")
def explanation_pages(filename):
    """Serve explanation pages from src/explanation."""
    return app.send_static_file(os.path.join("explanation", filename))


@bp.route("/logo.svg")
def logo():
    """Serve logo.svg for favicons and site branding."""
    return app.send_static_file("logo.svg")


@bp.route("/questions", methods=["GET"])
def questions():
    """Return all interview questions from the bank.

    Response (JSON): ``[{"id": 1, "category": "...", "question": "...", ...}, ...]``
    """
    return jsonify(get_all_questions())


@bp.route("/evaluate", methods=["POST"])
def evaluate():
    """Evaluate a candidate's answer to an interview question.

    Request body (JSON): ``{"question_id": 1, "answer": "..."}``
    Response (JSON): ``{"turn": 1, "feedback": "...", "relevance_evaluation": {...},
        "star_evaluation": {...}, "filler_evaluation": {...}}``
    """
    # ① parse the JSON payload and normalize the answer text
    data = request.get_json(force=True)
    question_id = data.get("question_id")
    answer = (data.get("answer") or "").strip()

    # ② reject empty answers before doing any scoring
    if not answer:
        return jsonify({"error": "An answer is required."}), 400

    # ③ find the requested question in the interview bank
    q = get_question_by_id(question_id)
    if not q:
        return jsonify({"error": f"Question ID {question_id} not found."}), 400

    try:
        # ④ let the evaluator agent score the answer and update memory
        result = _agent.evaluate_answer(q, answer)
        return jsonify(result)
    except Exception as e:
        # ⑤ return scoring errors as JSON so the frontend can display them
        return jsonify({"error": str(e)}), 500


@bp.route("/coach", methods=["POST"])
def coach():
    """Handle a free-form meta-question from the candidate.

    Request body (JSON): ``{"query": "How am I doing?"}``
    Response (JSON): ``{"response": "..."}``
    """
    # ① parse the candidate's coaching question from either supported field
    data = request.get_json(force=True)
    message = (data.get("query") or data.get("message") or "").strip()

    # ② reject empty coaching questions
    if not message:
        return jsonify({"error": "A question is required."}), 400

    try:
        # ③ ask the agent to answer using current session memory
        reply = _agent.ask_agent(message)
        return jsonify({"response": reply})
    except Exception as e:
        # ④ return agent errors as JSON for the frontend
        return jsonify({"error": str(e)}), 500


@bp.route("/next-question", methods=["GET"])
def next_question():
    """Let the Interviewer agent pick the next question (Ask 4 orchestrator).

    A GET because it only reads session memory; that also keeps it outside the
    POST rate limits (Flask and Nginx), so a full 5-question run with the
    Interviewer still leaves room for ``/coach``.

    Response (JSON): ``{"done": false, "question_id": 5, "category": "...",
        "reason": "...", "source": "llm" | "rules"}`` or ``{"done": true,
        "message": "..."}`` once every question has been answered.
    """
    try:
        # ① hand the shared session memory to the Interviewer agent
        pick = _interviewer.choose_next(_memory)
    except Exception as e:
        # ② return agent errors as JSON for the frontend
        return jsonify({"error": str(e)}), 500

    # ③ signal the end of the bank so the UI can suggest the final report
    if pick is None:
        return jsonify({
            "done": True,
            "message": "Every question has been answered — pull the final report.",
        })
    # ④ return just the decision; the browser already has the question text
    return jsonify({
        "done": False,
        "question_id": pick["question"]["id"],
        "category": pick["category"],
        "reason": pick["reason"],
        "source": pick["source"],
    })


@bp.route("/scorecard", methods=["GET"])
def scorecard():
    """Return live session scorecard and running average.

    Response (JSON): ``{"scorecard": [...], "average_relevance": 75.5,
        "total_questions": 3, "weakest_area": {...}}``
    """
    # ① gather live session metrics into one scorecard response
    return jsonify({
        "scorecard": _memory.get_scorecard(),
        "average_relevance": _memory.get_average_relevance(),
        "total_questions": _memory.get_total_questions(),
        "weakest_area": _memory.get_weakest_area(),
    })


@bp.route("/report", methods=["GET"])
def report():
    """Generate the aggregated final assessment report.

    Response (JSON): ``{"report_text": "...", "total_questions": 3, ...}``
    """
    try:
        # ① ask memory to aggregate all turns into a final report
        return jsonify(_memory.generate_final_report_dict())
    except Exception as e:
        # ② return report-generation errors as JSON for the frontend
        return jsonify({"error": str(e)}), 500


@bp.route("/reset", methods=["POST"])
def reset():
    """Clear session memory and start a fresh interview.

    Response (JSON): ``{"status": "ok", "message": "Session reset successfully."}``
    """
    # ① clear session state in memory and the evaluator agent
    _agent.reset()
    # ② confirm the fresh session to the browser
    return jsonify({"status": "ok", "message": "Session reset successfully."})


# ---------------------------------------------------------------------------
# Blueprint registration + server entry point
# ---------------------------------------------------------------------------

# Register all Blueprint routes under the optional path prefix.  This single
# line is the only place where PATH_PREFIX is applied — every route above is
# written as a relative path (e.g. "/evaluate") and the prefix is prepended.
app.register_blueprint(bp, url_prefix=PATH_PREFIX)


if __name__ == "__main__":
    # Run the development server.  0.0.0.0 makes the app reachable from
    # outside the container; port 5000 is mapped to host port 8085 by
    # docker-compose.yml.
    app.run(host="0.0.0.0", port=5000)

Interview Question Bank for InterviewIQ.

"""Interview Question Bank for InterviewIQ.

Contains categorized interview questions along with expected key concepts,
and sample strong & weak answers for testing and demonstration.
"""

QUESTIONS = [
    {
        "id": 1,
        "category": "Behavioral",
        "question": "Tell me about a time you had a major conflict or disagreement with a teammate and how you handled it.",
        "expected_keywords": [
            "conflict", "disagreement", "perspective", "listen", "communication",
            "compromise", "resolution", "collaborate", "respect", "team",
            "alignment", "outcome",
        ],
        "sample_strong_answer": (
            "In my previous role as a senior engineer, our team encountered a major "
            "conflict and technical disagreement regarding our API architecture. My "
            "task was to facilitate open communication and achieve a win-win "
            "resolution. I respected every teammate's perspective and actively "
            "listened during technical review meetings. I encouraged both sides to "
            "collaborate and propose a practical compromise. As a result, we achieved "
            "full team alignment, successfully resolved the dispute, and delivered a "
            "positive business outcome with a 35% performance gain."
        ),
        "sample_weak_answer": (
            "Um, basically, like, someone disagreed with my PR and was arguing with "
            "me. I know I was right so I just told them to look at the code again. "
            "Eventually they gave up and approved it, you know."
        ),
    },
    {
        "id": 2,
        "category": "Technical",
        "question": "How do you design a scalable microservices architecture and manage reliable inter-service communication?",
        "expected_keywords": [
            "microservices", "api gateway", "rest", "grpc", "async",
            "message queue", "kafka", "rabbitmq", "event-driven",
            "load balancer", "circuit breaker", "resilience",
            "database per service", "caching", "idempotency", "scalability",
        ],
        "sample_strong_answer": (
            "To design a high-scalability microservices architecture, I decouple "
            "services using the database per service pattern. For external client "
            "ingress, I deploy an API Gateway with load balancer and rate limiting. "
            "For high-throughput inter-service communication, I implement "
            "asynchronous event-driven message queue systems using Kafka or RabbitMQ "
            "with idempotency guarantees. For low-latency synchronous RPC calls, I "
            "use gRPC protected by circuit breaker patterns to guarantee system "
            "resilience. Finally, distributed Redis caching and REST APIs ensure "
            "high performance and loose coupling."
        ),
        "sample_weak_answer": (
            "Well, microservices are just splitting code into multiple servers. You "
            "just make REST API calls between them and use a single database that "
            "everyone connects to so data stays in sync."
        ),
    },
    {
        "id": 3,
        "category": "Problem-Solving",
        "question": "Describe a critical production incident or bug you diagnosed and resolved under pressure.",
        "expected_keywords": [
            "incident", "production", "monitoring", "logs", "metrics",
            "root cause", "debug", "reproduce", "patch", "rollback",
            "post-mortem", "alert", "latency",
        ],
        "sample_strong_answer": (
            "During a high-severity production incident with elevated latency, our "
            "monitoring alerts triggered on payment failures. As incident commander, "
            "I analyzed APM metrics and distributed server logs to debug and isolate "
            "the root cause: an unindexed database query causing connection pool "
            "deadlocks. I immediately initiated a safe rollback to restore production "
            "stability within 8 minutes. Once stable, we reproduced the issue in "
            "staging, deployed a tested patch, and published a blameless post-mortem "
            "with new monitoring alerts."
        ),
        "sample_weak_answer": (
            "Uh, yeah, one day the website was down. Basically I restarted the "
            "server and it worked again. I don't know exactly what happened, but "
            "restarting fixed it, like, right away."
        ),
    },
    {
        "id": 4,
        "category": "Leadership",
        "question": "Tell me about a high-stakes project you led where you had to manage tight deadlines and shifting requirements.",
        "expected_keywords": [
            "leadership", "prioritize", "delegate", "deadline", "milestones",
            "stakeholders", "scope", "communication", "delivery", "risk",
            "alignment", "impact",
        ],
        "sample_strong_answer": (
            "When tasked with leadership on an enterprise compliance initiative with "
            "an aggressive 60-day deadline, my objective was on-time delivery without "
            "compromising security. I established 2-week sprint milestones, "
            "prioritized mission-critical security controls, and delegated "
            "specialized tasks across three engineering teams. I maintained "
            "transparent communication and weekly alignment sessions with executive "
            "stakeholders to manage scope and mitigate delivery risks. As a result of "
            "this leadership, we passed the audit with zero defects ahead of deadline "
            "and achieved a $1.2M revenue impact."
        ),
        "sample_weak_answer": (
            "Like, we had a really short deadline from our manager. We worked late "
            "hours every day and pushed everyone to finish. It was stressful but we "
            "got it done, you know."
        ),
    },
    {
        "id": 5,
        "category": "System Design",
        "question": "How would you design a distributed URL shortening service like TinyURL capable of handling billions of redirects?",
        "expected_keywords": [
            "url shortener", "hash", "base62", "encoding", "collisions",
            "database", "nosql", "redis", "cache", "load balancing", "cdn",
            "throughput", "capacity", "zookeeper",
        ],
        "sample_strong_answer": (
            "To design a high-throughput distributed url shortener service handling "
            "billions of redirects, I estimate capacity for a 100:1 read-to-write "
            "ratio. I utilize Base62 encoding on unique 64-bit integer IDs generated "
            "by a distributed counter with ZooKeeper coordination to guarantee zero "
            "collisions without hash retry penalties. For persistence, I use a "
            "distributed NoSQL database partitioned by short key. To maximize "
            "throughput, I place a multi-tier Redis cache cluster behind Anycast CDN "
            "and load balancing tiers."
        ),
        "sample_weak_answer": (
            "You take a long URL, run MD5 hash on it, take the first 6 letters, and "
            "save it in a MySQL table. When someone requests it, query the table and "
            "redirect."
        ),
    },
]


def get_all_questions() -> list[dict]:
    """Return all available questions."""
    return QUESTIONS


def get_question_by_id(question_id: int) -> dict | None:
    """Retrieve a specific question by its ID."""
    # ① scan the question bank for the requested id
    for q in QUESTIONS:
        # ② return the first matching question record
        if q["id"] == question_id:
            return q
    # ③ return None when the id is not in the bank
    return None


def get_question_categories() -> list[str]:
    """Return unique categories across the question bank."""
    return list(dict.fromkeys(q["category"] for q in QUESTIONS))

InterviewIQ Agent — tool-calling evaluator with session memory.

"""InterviewIQ Agent — tool-calling evaluator with session memory.

Architecture
------------
- **InterviewSessionMemory**: append-only session store with on-demand
  aggregation methods and anti-recency-bias weakest-area detection.
- **EvaluatorAgent**: wraps the LLM client and the genuine tool-calling loop
  (the LLM *chooses* which tools to invoke, just like the class-notes
  reference) plus session memory.

The key design choice: tools return structured *dicts*, and the LLM decides
dynamically which tools to call via the OpenAI function-calling API.  This
is the authentic ReAct-style agent pattern taught in the class.
"""

import json

from openai import BadRequestError

from config import get_openai_client
from tools import check_star_structure, detect_filler_words, score_relevance

# ---------------------------------------------------------------------------
# Session memory
# ---------------------------------------------------------------------------


class InterviewSessionMemory:
    """Append-only session store for interview turns.

    Each turn records the question, answer, and the structured dict results
    returned by each evaluation tool.
    """

    def __init__(self):
        self._turns: list[dict] = []

    @property
    def turns(self) -> list[dict]:
        return list(self._turns)

    @property
    def turn_count(self) -> int:
        return len(self._turns)

    def add_turn(
        self, question: str, answer: str, results: dict,
        category: str = "", question_id: int = 0,
        expected_keywords: list | None = None,
    ) -> None:
        # ① store this answer and every tool result as the next interview turn
        self._turns.append({
            "turn_id": len(self._turns) + 1,
            "question_id": question_id,
            "question": question,
            "category": category,
            "expected_keywords": expected_keywords or [],
            "answer": answer,
            "results": results,
        })

    def get_weakest_area(self) -> dict | None:
        """Return the turn with the lowest composite score.

        Uses a 3-key sort (relevance ASC, star_score ASC, filler_count DESC)
        to explicitly avoid recency bias — the weakest area is the globally
        worst turn, not the most recent one.
        """
        # ① stop early when no answers have been recorded
        if not self._turns:
            return None

        # ② define how to rank turns from weakest to strongest
        def sort_key(turn):
            r = turn["results"]
            rel = r.get("score_relevance", {}).get("score", 50)
            star = r.get("check_star_structure", {}).get("star_score", 50)
            fillers = r.get("detect_filler_words", {}).get("total_filler_count", 0)
            # ① lower relevance, lower STAR, higher fillers = weaker
            return (rel, star, -fillers)

        # ③ pick the globally weakest turn and unpack its tool scores
        weakest = min(self._turns, key=sort_key)
        rel_data = weakest["results"].get("score_relevance", {})
        star_data = weakest["results"].get("check_star_structure", {})
        filler_data = weakest["results"].get("detect_filler_words", {})
        # ④ return only the weakness details needed by the UI and report
        return {
            "turn_id": weakest["turn_id"],
            "question_id": weakest["question_id"],
            "question": weakest["question"],
            "category": weakest["category"],
            "relevance_score": rel_data.get("score", 0),
            "star_score": star_data.get("star_score", 0),
            "filler_count": filler_data.get("total_filler_count", 0),
            "unmatched_keywords": rel_data.get("unmatched_keywords", []),
        }

    def get_strongest_area(self) -> dict | None:
        """Return the turn with the highest composite score."""
        # ① stop early when no answers have been recorded
        if not self._turns:
            return None

        # ② define how to rank turns from weakest to strongest
        def sort_key(turn):
            r = turn["results"]
            rel = r.get("score_relevance", {}).get("score", 50)
            star = r.get("check_star_structure", {}).get("star_score", 50)
            fillers = r.get("detect_filler_words", {}).get("total_filler_count", 0)
            return (rel, star, -fillers)

        # ③ pick the strongest turn and unpack its tool scores
        strongest = max(self._turns, key=sort_key)
        rel_data = strongest["results"].get("score_relevance", {})
        star_data = strongest["results"].get("check_star_structure", {})
        filler_data = strongest["results"].get("detect_filler_words", {})
        # ④ return only the strength details needed by the UI and report
        return {
            "turn_id": strongest["turn_id"],
            "question_id": strongest["question_id"],
            "question": strongest["question"],
            "category": strongest["category"],
            "relevance_score": rel_data.get("score", 0),
            "star_score": star_data.get("star_score", 0),
            "filler_count": filler_data.get("total_filler_count", 0),
        }

    def get_average_relevance(self) -> float:
        # ① collect relevance scores from turns that have relevance data
        scores = [
            t["results"].get("score_relevance", {}).get("score", 0)
            for t in self._turns
            if "score_relevance" in t["results"]
        ]
        # ② average available scores, or return zero before any scoring
        return round(sum(scores) / len(scores), 1) if scores else 0.0

    def get_total_questions(self) -> int:
        return len(self._turns)

    def get_scorecard(self) -> list[dict]:
        """Return a structured summary list for the scorecard table."""
        # ① start an empty list of rows for the scorecard table
        card = []
        for t in self._turns:
            # ② pull the three visible scores from this turn's tool results
            r = t["results"]
            rel = r.get("score_relevance", {}).get("score", 0)
            star = r.get("check_star_structure", {}).get("star_score", 0)
            fillers = r.get("detect_filler_words", {}).get("total_filler_count", 0)
            q_text = t["question"]
            # ③ append a compact learner-facing row for this question
            card.append({
                "Turn": t["turn_id"],
                "Category": t["category"],
                "Question": (q_text[:55] + "...") if len(q_text) > 55 else q_text,
                "Relevance Score": f"{rel}/100",
                "STAR Score": f"{star}%",
                "Fillers": fillers,
            })
        return card

    def get_category_breakdown(self) -> dict:
        """Compute category-wise average performance."""
        # ① collect relevance and STAR scores under each question category
        cat_stats: dict[str, list] = {}
        for t in self._turns:
            cat = t["category"]
            if cat not in cat_stats:
                cat_stats[cat] = []
            r = t["results"]
            cat_stats[cat].append({
                "relevance_score": r.get("score_relevance", {}).get("score", 0),
                "star_score": r.get("check_star_structure", {}).get("star_score", 0),
            })
        # ② average the collected scores for each category
        breakdown = {}
        for cat, items in cat_stats.items():
            avg_rel = round(sum(i["relevance_score"] for i in items) / len(items), 1)
            avg_star = round(sum(i["star_score"] for i in items) / len(items), 1)
            breakdown[cat] = {"count": len(items), "avg_relevance": avg_rel, "avg_star": avg_star}
        # ③ return the category summary for reports and scorecards
        return breakdown

    def generate_final_report_dict(self) -> dict:
        """Generate a comprehensive report as a dict with report_text markdown."""
        # ① return an empty-session report before calculating aggregates
        if not self._turns:
            return {
                "total_questions": 0,
                "average_relevance": 0.0,
                "weakest_area": None,
                "strongest_area": None,
                "total_fillers": 0,
                "report_text": "No questions have been answered yet in this session.",
            }

        # ② calculate session-wide counts, averages, and strongest/weakest areas
        total_questions = len(self._turns)
        avg_relevance = self.get_average_relevance()
        weakest = self.get_weakest_area()
        strongest = self.get_strongest_area()
        total_fillers = sum(
            t["results"].get("detect_filler_words", {}).get("total_filler_count", 0)
            for t in self._turns
        )
        avg_fillers = round(total_fillers / total_questions, 1)
        star_compliant = sum(
            1 for t in self._turns
            if t["results"].get("check_star_structure", {}).get("star_score", 0) >= 75
        )
        star_rate = round((star_compliant / total_questions) * 100, 1)
        category_breakdown = self.get_category_breakdown()

        # ③ start the markdown report with top-line metrics
        lines = [
            "# 🎯 InterviewIQ Final Assessment Report",
            f"**Total Questions Answered:** {total_questions} | "
            f"**Average Relevance Score:** {avg_relevance}/100",
            f"**Total Filler Words:** {total_fillers} (avg {avg_fillers}/turn) | "
            f"**STAR Framework Mastery:** {star_rate}%",
            "",
            "## 📊 Key Highlights & Aggregations",
        ]
        # ④ add strongest and weakest highlights when available
        if strongest:
            lines.append(
                f"1. **Strongest Area:** {strongest['category']} "
                f"(Score: {strongest['relevance_score']}/100)\n"
                f"   - Question: *\"{strongest['question']}\"*"
            )
        if weakest:
            missed = ", ".join(weakest.get("unmatched_keywords", [])[:5]) or "None"
            lines.append(
                f"2. **Weakest Area (Needs Focus):** {weakest['category']} "
                f"(Score: {weakest['relevance_score']}/100)\n"
                f"   - Question: *\"{weakest['question']}\"*\n"
                f"   - Missing Concepts: {missed}"
            )
        # ⑤ append per-category averages to show topic-level patterns
        lines.append("\n## 📈 Category Breakdown")
        for cat, stats in category_breakdown.items():
            lines.append(
                f"- **{cat}**: {stats['count']} question(s) | "
                f"Avg Relevance: {stats['avg_relevance']}/100 | "
                f"Avg STAR: {stats['avg_star']}%"
            )
        # ⑥ add coaching recommendations based on the aggregate scores
        lines.append("\n## 💡 Coach Recommendations")
        if avg_relevance >= 80:
            lines.append("- **Knowledge Depth**: Excellent domain grasp and comprehensive keyword coverage.")
        elif avg_relevance >= 60:
            lines.append("- **Knowledge Depth**: Solid baseline; focus on articulating deeper architectural trade-offs.")
        else:
            weak_cat = weakest["category"] if weakest else "all areas"
            lines.append(f"- **Knowledge Depth**: Review core concepts, particularly in {weak_cat}.")
        if total_fillers > total_questions * 2:
            lines.append(f"- **Delivery**: High filler word frequency ({total_fillers} total). Practice deliberate pauses.")
        else:
            lines.append("- **Delivery**: Clean, articulate delivery with minimal filler words.")
        if star_rate < 75:
            lines.append("- **Structure**: Strengthen STAR structure, particularly quantifiable Results.")
        else:
            lines.append("- **Structure**: Consistently strong STAR narrative with measurable Results.")

        # ⑦ return structured fields plus the rendered markdown report
        return {
            "total_questions": total_questions,
            "average_relevance": avg_relevance,
            "weakest_area": weakest,
            "strongest_area": strongest,
            "total_fillers": total_fillers,
            "star_compliance_rate": star_rate,
            "category_breakdown": category_breakdown,
            "report_text": "\n".join(lines),
        }

    def clear(self) -> None:
        self._turns.clear()


# ---------------------------------------------------------------------------
# Tool schema (the "menu" the LLM reads)
# ---------------------------------------------------------------------------

TOOLS_SCHEMA = [
    {
        "type": "function",
        "function": {
            "name": "detect_filler_words",
            "description": (
                "Count filler words (um, like, basically, etc.) in a "
                "candidate's answer and compute density per 100 words."
            ),
            "parameters": {
                "type": "object",
                "properties": {"answer": {"type": "string"}},
                "required": ["answer"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "check_star_structure",
            "description": (
                "Check whether a behavioral answer covers Situation, Task, "
                "Action, Result (the STAR framework) and return a percentage "
                "score."
            ),
            "parameters": {
                "type": "object",
                "properties": {"answer": {"type": "string"}},
                "required": ["answer"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "score_relevance",
            "description": (
                "Score how many expected keywords/concepts for the question "
                "appear in the answer, using a 0-100 calibrated scale."
            ),
            "parameters": {
                "type": "object",
                "properties": {
                    "answer": {"type": "string"},
                    "expected_keywords": {
                        "type": "array",
                        "items": {"type": "string"},
                    },
                },
                "required": ["answer", "expected_keywords"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "generate_final_report",
            "description": (
                "Summarise the whole interview session so far, including the "
                "weakest area. Call this whenever the candidate asks how "
                "they're doing, what their weakest area is, or for an overall "
                "report — at any point in the session, not only at the end."
            ),
            "parameters": {"type": "object", "properties": {}},
        },
    },
]

SYSTEM_PROMPT = (
    "You are InterviewIQ, an AI mock-interview coach.\n"
    "For every candidate answer, call the relevant evaluation tools "
    "(filler words, STAR structure, relevance) before giving feedback.\n"
    "Keep feedback short, specific, and encouraging.\n"
    "If the candidate asks how they're doing, what their weakest area is, "
    "or for an overall report — at any point in the session — call "
    "generate_final_report and answer using what it returns."
)

MAX_TOOL_ROUNDS = 5
MAX_GLITCH_RETRIES = 3


# ---------------------------------------------------------------------------
# Evaluator agent
# ---------------------------------------------------------------------------


class EvaluatorAgent:
    """Tool-calling evaluator agent with session memory.

    Uses the authentic ReAct-style tool-calling loop: the LLM dynamically
    decides which tools to invoke via the function-calling API.
    """

    def __init__(self, memory: InterviewSessionMemory | None = None):
        # ① reuse supplied memory and choose the configured LLM provider
        self.memory = memory or InterviewSessionMemory()
        self._client, self._model = get_openai_client()

        # ② register the tool functions; generate_final_report is a method on
        # this instance so it can access self.memory.
        self._tool_functions = {
            "detect_filler_words": lambda args: detect_filler_words(args["answer"]),
            "check_star_structure": lambda args: check_star_structure(args["answer"]),
            "score_relevance": lambda args: score_relevance(
                args["answer"], args["expected_keywords"]
            ),
            "generate_final_report": lambda _args: self.generate_final_report(),
        }

    # -- Public API ----------------------------------------------------------

    def evaluate_answer(
        self, question_data: dict, answer: str
    ) -> dict:
        """Evaluate one interview answer and record the turn.

        Args:
            question_data: Dict with keys ``question``, ``expected_keywords``,
                ``category``, ``id``.
            answer: The candidate's answer text.

        Returns a structured dict with ``turn``, ``feedback``,
        ``relevance_evaluation``, ``star_evaluation``, ``filler_evaluation``
        that the UI's ``renderEvaluationResult`` expects.
        """
        # ① unpack question fields needed by tools and session memory
        question = question_data["question"]
        expected_keywords = question_data.get("expected_keywords", [])
        category = question_data.get("category", "")
        question_id = question_data.get("id", 0)

        # ② fall back to deterministic scoring when no LLM provider is configured
        if not self._client:
            return self._deterministic_evaluate(
                question, answer, expected_keywords, category, question_id,
            )

        # ③ build the prompt with the question, answer, and scoring rubric
        user_msg = (
            f"Question: {question}\n"
            f"Candidate's answer: {answer}\n"
            f"Expected keywords for this question: {expected_keywords}\n"
            f"Evaluate this answer using the available tools, then give "
            f"short feedback."
        )
        # ④ start the tool-calling conversation with system and user messages
        messages = [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_msg},
        ]

        # ⑤ run the agent loop and save the evaluated turn in memory
        results: dict = {}
        feedback = self._run_loop(messages, results_out=results)
        self.memory.add_turn(
            question, answer, results,
            category=category,
            question_id=question_id,
            expected_keywords=expected_keywords,
        )

        # ⑥ return the UI-shaped evaluation payload
        return {
            "turn": self.memory.turn_count,
            "feedback": feedback,
            "relevance_evaluation": results.get("score_relevance", {}),
            "star_evaluation": results.get("check_star_structure", {}),
            "filler_evaluation": results.get("detect_filler_words", {}),
        }

    def ask_agent(self, user_message: str) -> str:
        """Handle a free-form meta-question ("How am I doing?").

        Uses the same tool-calling loop — the LLM decides whether to call
        generate_final_report to answer the question.
        """
        # ① use the deterministic report when no LLM provider is configured
        if not self._client:
            return self.generate_final_report()

        # ② send the meta-question through the same tool-calling loop
        messages = [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_message},
        ]
        return self._run_loop(messages)

    def generate_final_report(self) -> str:
        """Aggregate session memory into a formatted report.

        Called as a tool by the LLM, or directly by the UI.
        """
        # ① read recorded turns and stop if nothing has been scored
        turns = self.memory.turns
        if not turns:
            return "No answers scored yet."

        # ② prepare report lines and running totals
        lines = [f"Interview Report — {len(turns)} question(s) answered so far\n"]
        relevance_scores: list[tuple[int, str]] = []
        total_fillers = 0

        # ③ summarize each turn's tool results into readable report lines
        for t in turns:
            results = t["results"]
            lines.append(f"Q{t['turn_id']}: {t['question']}")

            if "detect_filler_words" in results:
                r = results["detect_filler_words"]
                count = r.get("total_filler_count", 0)
                total_fillers += count
                density = r.get("filler_density_per_100_words", 0)
                lines.append(f"   - Filler words: {count} ({density}% density)")

            if "check_star_structure" in results:
                r = results["check_star_structure"]
                star = r.get("star_score", 0)
                if r.get("is_star_complete"):
                    note = "all stages present"
                else:
                    note = f"missing {', '.join(r.get('missing_components', []))}"
                lines.append(f"   - STAR structure: {star}% — {note}")

            if "score_relevance" in results:
                r = results["score_relevance"]
                score = r.get("score", 0)
                relevance_scores.append((score, t["question"]))
                missed = r.get("unmatched_keywords", [])
                missed_str = ", ".join(missed[:4]) if missed else "none"
                lines.append(f"   - Relevance: {score}/100 (missed: {missed_str})")

        # ④ add average relevance and weakest area when relevance scores exist
        if relevance_scores:
            avg = round(sum(s for s, _ in relevance_scores) / len(relevance_scores))
            weakest = self.memory.get_weakest_area()
            lines.append(f"\nAverage relevance score: {avg}/100")
            if weakest:
                w_score = weakest.get("relevance_score", "?")
                lines.append(
                    f'Weakest area: "{weakest["question"]}" '
                    f"(relevance {w_score}/100)"
                )

        # ⑤ append the delivery total and return the final report text
        lines.append(f"Total filler words across the session: {total_fillers}")
        return "\n".join(lines)

    def reset(self) -> None:
        """Clear session memory for a fresh start."""
        self.memory.clear()

    # -- Internal helpers ----------------------------------------------------

    def _create_with_retry(self, **kwargs):
        """Call ``client.chat.completions.create`` with retry for Groq glitches.

        Groq's gpt-oss-20b occasionally leaks a format token into the tool
        name (e.g. ``score_relevance<|channel|>commentary``), which Groq then
        rejects as unknown.  Regenerating almost always fixes it.
        """
        last_error = None
        # ① try the chat completion a few times in case the provider glitches
        for _ in range(MAX_GLITCH_RETRIES):
            try:
                return self._client.chat.completions.create(**kwargs)
            except BadRequestError as e:
                # ② retry only the known Groq tool-call formatting glitch
                if getattr(e, "code", None) != "tool_use_failed":
                    raise
                last_error = e
        # ③ raise the final retryable error after all attempts fail
        raise last_error

    def _run_loop(self, messages: list, results_out: dict | None = None) -> str:
        """Shared tool-calling loop (the authentic ReAct pattern).

        Sends messages to the LLM, executes whichever tools the LLM requests,
        appends the results back, and loops until the model stops calling
        tools (up to ``MAX_TOOL_ROUNDS``).

        If ``results_out`` (a dict) is passed, tool results are recorded into
        it by tool name — used by ``evaluate_answer`` to build a session turn.
        """
        for _ in range(MAX_TOOL_ROUNDS):
            # ① ask the model whether to call tools or answer directly
            response = self._create_with_retry(
                model=self._model,
                messages=messages,
                tools=TOOLS_SCHEMA,
                tool_choice="auto",
            )
            msg = response.choices[0].message

            # ② return the final reply once the model stops requesting tools
            if not msg.tool_calls:
                return msg.content or ""

            messages.append(msg)
            for call in msg.tool_calls:
                # ③ parse arguments and execute the requested Python tool
                args = (
                    json.loads(call.function.arguments)
                    if call.function.arguments
                    else {}
                )
                fn = self._tool_functions[call.function.name]
                result = fn(args)

                # ④ record tool results for session logging (except the report
                # itself, which is meta-data, not per-answer evaluation).
                if (
                    results_out is not None
                    and call.function.name != "generate_final_report"
                ):
                    results_out[call.function.name] = result

                # ⑤ append tool output so the model can continue reasoning
                messages.append({
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": (
                        result if isinstance(result, str) else json.dumps(result)
                    ),
                })

        # ⑥ ran out of rounds — ask for a final answer without offering tools
        response = self._create_with_retry(
            model=self._model, messages=messages
        )
        return response.choices[0].message.content or ""

    def _deterministic_evaluate(
        self, question: str, answer: str, expected_keywords: list[str],
        category: str = "", question_id: int = 0,
    ) -> dict:
        """Fallback evaluation when no LLM client is available.

        Runs all three tools deterministically and formats a plain-text
        summary.  No LLM synthesis.
        """
        # ① run all deterministic tools against the same answer
        filler = detect_filler_words(answer)
        star = check_star_structure(answer)
        relevance = score_relevance(answer, expected_keywords)

        # ② package results under the same names used by tool calls
        results = {
            "detect_filler_words": filler,
            "check_star_structure": star,
            "score_relevance": relevance,
        }
        # ③ record the deterministic turn in shared session memory
        self.memory.add_turn(
            question, answer, results,
            category=category,
            question_id=question_id,
            expected_keywords=expected_keywords,
        )

        # ④ format concise feedback from the three tool messages
        lines = [
            f"Relevance: {relevance['message']}",
            f"STAR: {star['message']}",
            f"Fillers: {filler['message']}",
        ]
        # ⑤ return the same UI payload shape as the LLM path
        return {
            "turn": self.memory.turn_count,
            "feedback": "\n".join(lines),
            "relevance_evaluation": relevance,
            "star_evaluation": star,
            "filler_evaluation": filler,
        }

Shared configuration: load .env, build API clients for Groq / OpenAI.

"""Shared configuration: load .env, build API clients for Groq / OpenAI.

This module is the single place that knows about secrets and model names.
Every other module imports from here instead of reading ``os.environ`` or
constructing API clients itself, so key and model management stays in one
place.

Provider priority:
    1. Groq (free tier) — ``GROQ_API_KEY``
    2. OpenAI — ``OPENAI_API_KEY``
    3. None — deterministic-only mode (tools still work, LLM feedback disabled)
"""

import os

from dotenv import load_dotenv

# Read key=value pairs from the .env file and inject them into os.environ.
# Safe to call at import time — if no .env exists (e.g. in production where
# env vars are set directly), python-dotenv simply does nothing.
load_dotenv()


def get_env(name: str, default: str = "") -> str:
    """Return an environment variable, falling back to ``default``."""
    return os.environ.get(name, default)


# ---------------------------------------------------------------------------
# LLM provider configuration
# ---------------------------------------------------------------------------

GROQ_API_KEY = os.environ.get("GROQ_API_KEY", "")
OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY", "")


def get_openai_client():
    """Return an OpenAI-compatible client using Groq first, OpenAI second.

    Returns:
        A tuple of ``(client, model_name)`` or ``(None, None)`` when no API
        key is available.
    """
    # ① import the client only when a provider lookup is needed
    from openai import OpenAI

    # ② use Groq first because it is the preferred free-tier provider
    if GROQ_API_KEY:
        return (
            OpenAI(
                api_key=GROQ_API_KEY,
                base_url="https://api.groq.com/openai/v1",
            ),
            "openai/gpt-oss-20b",
        )

    # ③ fall back to OpenAI when no Groq key is configured
    if OPENAI_API_KEY:
        return OpenAI(api_key=OPENAI_API_KEY), "gpt-4o-mini"

    # ④ signal deterministic-only mode when no API keys are available
    return None, None

Evaluation tools for InterviewIQ.

"""Evaluation tools for InterviewIQ.

Deterministic, rule-based tools for evaluating candidate interview answers:
1. detect_filler_words: Identifies and counts speech filler words/phrases.
2. check_star_structure: Analyzes answers for Situation, Task, Action, Result.
3. score_relevance: Evaluates answer relevance (0-100) against expected keywords.

Each returns a structured dict (not a pre-formatted string) so the agent can
use the raw numbers for session aggregation and the final report.
"""

import re
from typing import Any

# ---------------------------------------------------------------------------
# Filler-word detection
# ---------------------------------------------------------------------------

# Standard filler words and conversational crutch phrases.
FILLER_PATTERNS = [
    ("you know", r"\byou know\b"),
    ("sort of", r"\bsort of\b"),
    ("kind of", r"\bkind of\b"),
    ("i mean", r"\bi mean\b"),
    ("um", r"\bum+\b"),
    ("uh", r"\buh+\b"),
    ("like", r"\blike\b"),
    ("basically", r"\bbasically\b"),
    ("actually", r"\bactually\b"),
    ("literally", r"\bliterally\b"),
    ("honestly", r"\bhonestly\b"),
    ("right", r"\bright\b"),
]


def detect_filler_words(answer: str) -> dict[str, Any]:
    """Count filler words (um, like, basically, etc.) in a candidate's answer.

    Returns a structured dict with detected fillers, total count, density per
    100 words, and a verdict message.
    """
    # ① reject empty answers before running pattern matching
    if not answer or not answer.strip():
        return {
            "detected_fillers": {},
            "total_filler_count": 0,
            "has_fillers": False,
            "filler_density_per_100_words": 0.0,
            "message": "No answer provided or answer is empty.",
        }

    # ② normalize the answer and prepare filler counters
    text_lower = answer.lower()
    detected_fillers: dict[str, int] = {}
    total_count = 0

    # ③ count each filler pattern and accumulate totals
    for name, pattern in FILLER_PATTERNS:
        matches = re.findall(pattern, text_lower, flags=re.IGNORECASE)
        count = len(matches)
        if count > 0:
            detected_fillers[name] = count
            total_count += count

    # ④ compute word count for density calculation
    words = re.findall(r"\b\w+\b", text_lower)
    word_count = len(words)
    density = round((total_count / max(word_count, 1)) * 100, 1)

    # ⑤ build a learner-friendly message based on detected fillers
    if total_count > 0:
        details = ", ".join(
            f"{k} ({v})"
            for k, v in sorted(detected_fillers.items(), key=lambda x: -x[1])
        )
        message = f"Found {total_count} filler word(s) ({density}% density): {details}."
    else:
        message = "Excellent! No filler words detected in this answer."

    # ⑥ return structured score data for the agent and UI
    return {
        "detected_fillers": detected_fillers,
        "total_filler_count": total_count,
        "has_fillers": total_count > 0,
        "filler_density_per_100_words": density,
        "word_count": word_count,
        "message": message,
    }


# ---------------------------------------------------------------------------
# STAR structure detection
# ---------------------------------------------------------------------------

# Cue words and phrase patterns for STAR framework detection.
STAR_PATTERNS = {
    "Situation": [
        r"\b(?:in my (?:previous|former|past|last) role|when i was (?:at|working|leading)|at my previous company)\b",
        r"\b(?:the situation was|the context was|our company|our team was facing|we had a project)\b",
        r"\b(?:during a|during the|faced with|at the time|background|scenario)\b",
    ],
    "Task": [
        r"\b(?:my (?:task|goal|objective|role|responsibility|duty) was)\b",
        r"\b(?:i (?:was assigned to|was tasked with|needed to|had to|was responsible for))\b",
        r"\b(?:the challenge was|our target was|required to|aimed to)\b",
    ],
    "Action": [
        r"\b(?:i (?:decided|implemented|built|created|led|developed|analyzed|organized|proposed|initiated|designed|coordinated|refactored|migrated|reached out))\b",
        r"\b(?:my approach was|steps i took|i set up|i established|i wrote|i introduced)\b",
        r"\b(?:to resolve this, i|i worked with|i facilitated)\b",
    ],
    "Result": [
        r"\b(?:as a result|the outcome was|we achieved|we delivered|we improved|successfully)\b",
        r"\b(?:increased|reduced|decreased|saved|boosted|resolved|prevented)\b",
        r"\b(?:\d+%(?: reduction| increase| improvement| latency| failure)?|\$\d+[\d,.]*(?:k|m|b)?|\d+ (?:days|weeks|months|minutes|hours))\b",
        r"\b(?:impact was|outcome|post-mortem|non-conformities|contract closed)\b",
    ],
}


def check_star_structure(answer: str) -> dict[str, Any]:
    """Check whether a behavioral answer covers Situation, Task, Action, Result.

    Returns a structured dict with per-component coverage, a percentage score,
    and a recommendation message.
    """
    # ① reject empty answers before checking STAR components
    if not answer or not answer.strip():
        return {
            "situation": False,
            "task": False,
            "action": False,
            "result": False,
            "covered_components": [],
            "missing_components": ["Situation", "Task", "Action", "Result"],
            "star_score": 0.0,
            "is_star_complete": False,
            "message": "Answer is empty. Please provide a detailed response.",
        }

    # ② normalize the answer and prepare component tracking buckets
    text_lower = answer.lower()
    covered: list[str] = []
    missing: list[str] = []
    components_status: dict[str, bool] = {}

    # ③ scan each STAR component's cue patterns in the answer
    for component, patterns in STAR_PATTERNS.items():
        found = any(
            re.search(pattern, text_lower, flags=re.IGNORECASE)
            for pattern in patterns
        )
        components_status[component.lower()] = found
        if found:
            covered.append(component)
        else:
            missing.append(component)

    # ④ convert covered components into a percentage completion score
    star_score = round((len(covered) / 4) * 100, 1)
    is_complete = len(covered) == 4

    # ⑤ choose a coaching message based on STAR coverage
    if is_complete:
        message = "Outstanding! All 4 STAR components are clearly present."
    elif len(covered) >= 2:
        message = f"Good structure ({star_score}% STAR). Strengthen: {', '.join(missing)}."
    else:
        message = f"Weak structure ({star_score}% STAR). Missing: {', '.join(missing)}."

    # ⑥ return per-component status for feedback and aggregation
    return {
        "situation": components_status.get("situation", False),
        "task": components_status.get("task", False),
        "action": components_status.get("action", False),
        "result": components_status.get("result", False),
        "covered_components": covered,
        "missing_components": missing,
        "star_score": star_score,
        "is_star_complete": is_complete,
        "message": message,
    }


# ---------------------------------------------------------------------------
# Keyword relevance scoring
# ---------------------------------------------------------------------------


def _build_keyword_pattern(kw_clean: str) -> str:
    """Build a regex pattern that matches a keyword and common inflections."""
    # ① handle multi-word phrases and hyphenated/underscored terms
    if " " in kw_clean or "-" in kw_clean or "_" in kw_clean:
        tokens = [re.escape(t) for t in re.split(r"[\s\-_]+", kw_clean) if t]
        return r"\b" + r"[\s\-_]+".join(tokens) + r"\b"

    # ② expand -tion into verb forms (resolution → resolve, resolving, …)
    if kw_clean.endswith("tion") and len(kw_clean) > 5:
        stem = re.escape(kw_clean[:-4])
        return rf"\b(?:{re.escape(kw_clean)}s?|{stem}t(?:e|es|ed|ing)?|{stem[:-1]}v(?:e|es|ed|ing)?)\b"
    # ③ expand -ment into verb forms (alignment → align, aligning, …)
    if kw_clean.endswith("ment") and len(kw_clean) > 5:
        stem = re.escape(kw_clean[:-4])
        return rf"\b(?:{re.escape(kw_clean)}s?|{stem}(?:s|ed|ing|e)?)\b"
    # ④ expand -e endings (cache → caching, cached, …)
    if kw_clean.endswith("e") and len(kw_clean) > 3:
        stem = re.escape(kw_clean[:-1])
        return rf"\b(?:{re.escape(kw_clean)}s?|{stem}(?:ing|ed|es|able|ability)?)\b"
    # ⑤ expand -y endings (delivery → deliveries, …)
    if kw_clean.endswith("y") and len(kw_clean) > 3:
        stem = re.escape(kw_clean[:-1])
        return rf"\b(?:{re.escape(kw_clean)}|{stem}(?:ies|ied|ying))\b"
    # ⑥ default to base plus common suffixes
    base = re.escape(kw_clean)
    return rf"\b(?:{base}|{base}(?:s|es|ed|ing)?)\b"


def score_relevance(answer: str, expected_keywords: list[str]) -> dict[str, Any]:
    """Score relevance of an answer against expected keywords (0-100).

    Uses regex-based matching with grammatical inflection support and a
    calibrated tiered scoring curve.
    """
    # ① reject empty answers and mark all expected keywords as unmatched
    if not answer or not answer.strip():
        return {
            "score": 0,
            "matched_keywords": [],
            "unmatched_keywords": list(expected_keywords),
            "keyword_coverage_ratio": 0.0,
            "total_expected": len(expected_keywords),
            "total_matched": 0,
            "message": "Answer is empty. Relevance score is 0.",
        }

    # ② give full credit when no keyword rubric was supplied
    if not expected_keywords:
        return {
            "score": 100,
            "matched_keywords": [],
            "unmatched_keywords": [],
            "keyword_coverage_ratio": 1.0,
            "total_expected": 0,
            "total_matched": 0,
            "message": "No expected keywords specified.",
        }

    # ③ normalize the answer and prepare matched/unmatched buckets
    text_lower = answer.lower()
    matched_keywords: list[str] = []
    unmatched_keywords: list[str] = []

    # ④ match each expected keyword using an inflection-aware regex
    for kw in expected_keywords:
        kw_clean = kw.lower().strip()
        if not kw_clean:
            continue
        pattern = _build_keyword_pattern(kw_clean)
        if re.search(pattern, text_lower, flags=re.IGNORECASE):
            matched_keywords.append(kw)
        else:
            unmatched_keywords.append(kw)

    # ⑤ compute coverage totals used by the scoring curve
    total_expected = len(expected_keywords)
    total_matched = len(matched_keywords)
    coverage_ratio = total_matched / max(total_expected, 1)

    # ⑥ apply the calibrated scoring curve
    #   >= 70% coverage -> 90-100 score
    #   >= 45% coverage -> 75-89 score
    #   >= 25% coverage -> 50-74 score
    #   <  25% coverage -> 0-49 score
    if total_expected == 0:
        score = 100
    elif total_matched == 0:
        score = 0
    elif coverage_ratio >= 0.70:
        score = min(100, int(round(90 + ((coverage_ratio - 0.70) / 0.30) * 10)))
    elif coverage_ratio >= 0.45:
        score = int(round(75 + ((coverage_ratio - 0.45) / 0.25) * 14))
    elif coverage_ratio >= 0.25:
        score = int(round(50 + ((coverage_ratio - 0.25) / 0.20) * 24))
    else:
        score = max(5, int(round((coverage_ratio / 0.25) * 45)))

    # ⑦ write the feedback message for the score band
    if score >= 80:
        message = f"High relevance ({score}/100). Hit {total_matched}/{total_expected} expected concepts."
    elif score >= 50:
        tops = ", ".join(unmatched_keywords[:3])
        message = f"Moderate relevance ({score}/100). Hit {total_matched}/{total_expected}. Consider: {tops}."
    else:
        tops = ", ".join(unmatched_keywords[:4])
        message = f"Low relevance ({score}/100). Hit only {total_matched}/{total_expected}. Missed: {tops}."

    # ⑧ return structured relevance data for the agent and UI
    return {
        "score": score,
        "matched_keywords": matched_keywords,
        "unmatched_keywords": unmatched_keywords,
        "keyword_coverage_ratio": round(coverage_ratio, 2),
        "total_expected": total_expected,
        "total_matched": total_matched,
        "message": message,
    }

Interviewer agent — Ask 4 (bonus): a second, specialised agent.

"""Interviewer agent — Ask 4 (bonus): a second, specialised agent.

The Evaluator agent in ``agent.py`` scores answers. This agent has a single,
different job: look at how the candidate has done so far and decide which
**category** the next question should come from. It never evaluates answers
and never touches the Evaluator's tools — it only *reads* the shared session
memory.

Kept deliberately simple (no A2A protocol, no agent cards, no network hops
between agents): a second agent with its own system prompt, called in
sequence by a small orchestrator (``run_multi_agent.py`` on the CLI, the
``GET /next-question`` route in ``app.py`` for the web UI). That is the
"orchestrator + specialised agents" pattern in its smallest working form.

When no LLM key is configured, a deterministic rule of thumb makes the same
kind of decision so the feature still works offline.
"""

import json
import re

from config import get_openai_client
from interview_bank import QUESTIONS

# A relevance score below this counts as a weak answer (sample weak answers
# score 0–22, sample strong answers score 98–100).
WEAK_THRESHOLD = 60

# Categories are grouped into two families so "stay close" vs "switch for
# breadth" has a meaning even though the bank has one question per category.
CATEGORY_FAMILIES = {
    "Behavioral": "people",
    "Leadership": "people",
    "Technical": "engineering",
    "System Design": "engineering",
    "Problem-Solving": "engineering",
}

SYSTEM_PROMPT = (
    "You are the Interviewer half of InterviewIQ's two-agent mock interview "
    "system. A separate Evaluator agent scores every answer; you never score "
    "answers yourself.\n"
    "Your only job: choose the category of the NEXT question from the list of "
    "categories that are still available, based on how the candidate has done "
    "so far.\n"
    "Category families: people = Behavioral, Leadership; engineering = "
    "Technical, System Design, Problem-Solving.\n"
    "Rule of thumb: after a weak answer (relevance below "
    f"{WEAK_THRESHOLD}/100), stay in the same family so the candidate gets "
    "another shot at similar skills; after a strong answer, switch family to "
    "test breadth. For the very first question, start with Behavioral if it "
    "is available.\n"
    'Reply with JSON only: {"category": "<one available category>", '
    '"reason": "<one short sentence addressed to the candidate>"}'
)


class InterviewerAgent:
    """Chooses the next question's category from shared session memory."""

    def __init__(self, client=None, model: str | None = None):
        # ① reuse an injected client (tests) or the configured provider
        if client is not None:
            self._client, self._model = client, model
        else:
            self._client, self._model = get_openai_client()

    # -- Public API ----------------------------------------------------------

    def choose_next(self, memory) -> dict | None:
        """Pick the next unanswered question for the candidate.

        Args:
            memory: The Evaluator's ``InterviewSessionMemory`` (read-only).

        Returns:
            ``{"category", "reason", "source", "question"}`` where ``source``
            is ``"llm"`` or ``"rules"``, or ``None`` when every question in the
            bank has already been answered.
        """
        # ① work out which questions and categories are still unanswered
        answered_ids = {t["question_id"] for t in memory.turns}
        remaining = [q for q in QUESTIONS if q["id"] not in answered_ids]
        if not remaining:
            return None
        available = list(dict.fromkeys(q["category"] for q in remaining))

        # ② ask the LLM first; fall back to the rule of thumb on any problem
        decision = None
        if self._client and len(available) > 1:
            decision = self._ask_llm(memory, available)
        if decision is None:
            decision = self._rule_based(memory, available)

        # ③ hand back the first unanswered question in the chosen category
        question = next(q for q in remaining if q["category"] == decision["category"])
        return {**decision, "question": question}

    # -- Internal helpers ----------------------------------------------------

    @staticmethod
    def build_summary(memory, available: list[str]) -> str:
        """Plain-English recap of the session that the LLM reads."""
        # ① describe the opening move when nothing has been answered yet
        turns = memory.turns
        lines = []
        if not turns:
            lines.append("This is the first question of the session.")
        else:
            # ② list each turn's category and headline scores in order
            lines.append(f"Answers so far ({len(turns)}):")
            for t in turns:
                r = t["results"]
                rel = r.get("score_relevance", {}).get("score", 0)
                star = r.get("check_star_structure", {}).get("star_score", 0)
                fillers = r.get("detect_filler_words", {}).get("total_filler_count", 0)
                lines.append(
                    f"- Turn {t['turn_id']} [{t['category']}]: relevance {rel}/100, "
                    f"STAR {star}%, fillers {fillers}"
                )
            # ③ call out the most recent answer, which drives the rule of thumb
            last = turns[-1]
            last_rel = last["results"].get("score_relevance", {}).get("score", 0)
            verdict = "weak" if last_rel < WEAK_THRESHOLD else "strong"
            lines.append(
                f"Most recent answer: {last['category']} ({last_rel}/100, {verdict})."
            )
        # ④ list the only categories the agent is allowed to pick
        lines.append(f"Available categories: {', '.join(available)}.")
        return "\n".join(lines)

    def _ask_llm(self, memory, available: list[str]) -> dict | None:
        """Let the LLM choose; return ``None`` if the reply is unusable."""
        messages = [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": self.build_summary(memory, available)},
        ]
        try:
            # ① one plain chat call — this agent needs no tools
            response = self._client.chat.completions.create(
                model=self._model, messages=messages,
            )
            content = response.choices[0].message.content or ""
        except Exception:
            return None

        # ② accept only a category that is actually still available
        category, reason = self._parse_reply(content, available)
        if not category:
            return None
        return {
            "category": category,
            "reason": reason or f"The Interviewer chose {category} next.",
            "source": "llm",
        }

    @staticmethod
    def _parse_reply(content: str, available: list[str]) -> tuple[str | None, str]:
        """Pull ``(category, reason)`` out of a JSON-ish or plain-text reply."""
        lookup = {c.lower(): c for c in available}
        # ① prefer a JSON object, even if the model wrapped it in extra text
        match = re.search(r"\{.*\}", content, re.DOTALL)
        if match:
            try:
                data = json.loads(match.group(0))
                category = lookup.get(str(data.get("category", "")).strip().lower())
                if category:
                    return category, str(data.get("reason", "")).strip()
            except (json.JSONDecodeError, AttributeError):
                pass
        # ② otherwise accept the first available category named in the text
        lowered = content.lower()
        for key, category in lookup.items():
            if key in lowered:
                return category, ""
        return None, ""

    @staticmethod
    def _rule_based(memory, available: list[str]) -> dict:
        """Deterministic fallback that follows the same rule of thumb."""
        turns = memory.turns
        # ① open with Behavioral when possible
        if not turns:
            category = "Behavioral" if "Behavioral" in available else available[0]
            return {
                "category": category,
                "reason": f"Opening with a {category} question to warm up.",
                "source": "rules",
            }

        # ② stay in the family after a weak answer, switch after a strong one
        last = turns[-1]
        last_rel = last["results"].get("score_relevance", {}).get("score", 0)
        last_family = CATEGORY_FAMILIES.get(last["category"])
        weak = last_rel < WEAK_THRESHOLD
        if weak:
            preferred = [c for c in available if CATEGORY_FAMILIES.get(c) == last_family]
        else:
            preferred = [c for c in available if CATEGORY_FAMILIES.get(c) != last_family]
        category = (preferred or available)[0]

        # ③ explain the choice in one short sentence
        if weak and preferred:
            reason = (
                f"Your {last['category']} answer scored {last_rel}/100, so staying "
                f"with {last_family} skills — try {category} next."
            )
        elif not weak and preferred:
            reason = (
                f"Strong {last['category']} answer ({last_rel}/100) — switching "
                f"to {category} to test breadth."
            )
        else:
            reason = f"{category} is the next unanswered category."
        return {"category": category, "reason": reason, "source": "rules"}

run_multi_agent.py — Ask 4 (bonus): two agents, one orchestrator.

"""run_multi_agent.py — Ask 4 (bonus): two agents, one orchestrator.

A small orchestrator loop that hands work between two specialised agents:

- ``InterviewerAgent.choose_next`` (interviewer_agent.py) decides WHICH
  category to ask next, based on the session so far.
- ``EvaluatorAgent.evaluate_answer`` (agent.py — the agent from Asks 1–3,
  completely unchanged) scores the answer and records it in session memory.

Both agents share one ``InterviewSessionMemory``: the Evaluator writes to it,
the Interviewer reads from it.

Usage (from ``src/python``, or ``docker compose run --rm web python
src/python/run_multi_agent.py``)::

    python run_multi_agent.py                 # type your own answers
    python run_multi_agent.py --auto mixed    # replay bank sample answers
    python run_multi_agent.py --turns 5 --auto weak
"""

import argparse

from agent import EvaluatorAgent, InterviewSessionMemory
from config import GROQ_API_KEY, OPENAI_API_KEY
from interviewer_agent import InterviewerAgent

NUM_TURNS = 3  # keep it short — this is a bonus demo, not the full bank


def pick_sample_answer(question: dict, mode: str, turn: int) -> str:
    """Return a bank sample answer for ``--auto`` mode."""
    # ① "mixed" alternates strong / weak so the Interviewer sees both cases
    if mode == "mixed":
        mode = "strong" if turn % 2 == 1 else "weak"
    return question[f"sample_{mode}_answer"]


def read_answer() -> str | None:
    """Prompt until a non-empty answer is typed; ``None`` on Ctrl-D."""
    while True:
        try:
            answer = input("Your answer: ").strip()
        except EOFError:
            return None
        if answer:
            return answer
        print("(Please type an answer, or press Ctrl-D to stop.)")


def main() -> None:
    parser = argparse.ArgumentParser(description="InterviewIQ two-agent mode")
    parser.add_argument("--turns", type=int, default=NUM_TURNS,
                        help=f"number of questions to ask (default {NUM_TURNS})")
    parser.add_argument("--auto", choices=["strong", "weak", "mixed"],
                        help="answer automatically with the bank's sample answers")
    args = parser.parse_args()

    # ① one shared memory: the Evaluator writes it, the Interviewer reads it
    memory = InterviewSessionMemory()
    evaluator = EvaluatorAgent(memory=memory)
    interviewer = InterviewerAgent()

    print("=== InterviewIQ — Two-Agent Mode (Interviewer + Evaluator) ===")
    if not (GROQ_API_KEY or OPENAI_API_KEY):
        print("(No API key found — both agents run in deterministic mode.)")
    print()

    for turn in range(1, args.turns + 1):
        # ② hand-off 1: the Interviewer decides what to ask next
        pick = interviewer.choose_next(memory)
        if pick is None:
            print("Every question in the bank has been answered — ending early.")
            break
        q = pick["question"]
        print(f"[Interviewer → {pick['category']} ({pick['source']})] {pick['reason']}")
        print(f"Q{turn}: {q['question']}")

        if args.auto:
            answer = pick_sample_answer(q, args.auto, turn)
            print(f"Your answer: {answer}")
        else:
            answer = read_answer()
            if answer is None:
                print("\nStopping early.")
                break

        # ③ hand-off 2: the Evaluator scores the answer and updates memory
        result = evaluator.evaluate_answer(q, answer)
        score = result["relevance_evaluation"].get("score", 0)
        print(f"\nCoach (relevance {score}/100): {result['feedback']}\n")
        print("-" * 60)

    # ④ the Evaluator's aggregated report covers every turn of the session
    print("\n=== Final Report ===")
    print(evaluator.generate_final_report())


if __name__ == "__main__":
    main()

Use these key architectural and AI engineering takeaways for quick revision of InterviewIQ:

1. Autonomous ReAct Tool-Calling Loop (_run_loop): Instead of a rigid single-shot prompt, EvaluatorAgent exposes TOOLS_SCHEMA with tool_choice="auto" and loops up to MAX_TOOL_ROUNDS = 5 turns — allowing the LLM to decide which diagnostic tools to call, inspect their structured outputs, and synthesize grounded feedback.
2. Deterministic NLP vs. Probabilistic Synthesis Boundary: LLMs are unreliable at exact word counting and arithmetic. InterviewIQ delegates exact counting and scoring to deterministic Python functions in tools.py and uses the LLM strictly for empathetic natural-language synthesis and tool orchestration.
3. Word-Boundary Regex Filler Detection (detect_filler_words): Scans answers against 12 compiled word-boundary patterns (...) so substrings inside normal words are never false-flagged, and normalizes counts into filler_density_per_100_words.
4. Heuristic STAR Structural Parsing (check_star_structure): Evaluates behavioral answers across Situation, Task, Action, and Result using contextual phrase patterns, first-person ownership verbs, and quantifiable impact regexes (percentages, dollar amounts, time durations) to compute a 0–100% completion score and pinpoint missing stages.
5. Inflection-Aware Keyword Relevance & Tiered Curve (score_relevance): _build_keyword_pattern() expands nominalizations and suffixes (-tion, -ment, -e, -y) into regex stems so grammatical variations match fairly, then maps raw coverage ratio onto a calibrated 0–100 scoring curve.
6. Anti-Recency-Bias Session Memory (InterviewSessionMemory): Passing raw chat history to an LLM causes recency bias where the model forgets early weak turns. Storing structured tool outputs in self._turns and ranking them with a deterministic composite key (relevance_score, star_score, -filler_count) via min(self._turns, key=sort_key) guarantees the true globally weakest area is identified.
7. Mid-Session Meta-Tool Invocation (generate_final_report): By exposing generate_final_report as a callable tool in TOOLS_SCHEMA, the same _run_loop() engine handles both per-answer evaluation and mid-session meta-questions ("How am I doing so far?") without separate routing code.
8. Orchestrator + Specialized Multi-Agent Pattern (Ask 4): InterviewerAgent (category selection) and EvaluatorAgent (answer scoring) have separate system prompts and responsibilities and coordinate through shared state (InterviewSessionMemory) — the Evaluator writes turns, and the Interviewer reads them to stay in the same skill family after weak answers (< 60) or switch families after strong answers.
9. Multi-Tier Provider Resilience & Offline Fallback: config.py prioritizes Groq (openai/gpt-oss-20b), falls back to OpenAI (gpt-4o-mini), and degrades gracefully to deterministic-only mode (_deterministic_evaluate and _rule_based) when no API key is present, while _create_with_retry() recovers from transient tool_use_failed model glitches.