Give AI new abilities, then check the answers.#

• Opening The old deck used chat polls. This page keeps the learning as static answers.

• Question. What is one thing people often ask ChatGPT, Claude or Gemini?
  Correct answer. Code, emails, trips, exam explanations, leave applications, and many other everyday tasks.
  Reason. The exact answer varies. All of us use AI for work that used to take time.
• Question. How much should you trust an AI answer, from 1 to 5?
  Correct answer. Test it first.
  Reason. The goal is not blind trust or zero trust. The goal is measured trust backed by tools and evals.
• Question. What does this lesson answer?
  Correct answer. How to give AI new powers, and how to check its answers.
  Reason. Tools give the model abilities. Evals check whether those abilities worked correctly.

Think of the model as a smart friend off the grid. He read every book and webpage. Then he spent a year in the hills without a phone, internet, or news.

• ChatGPT = the appIt is the product you open and type into.
• GPT = the brain insideOpenAI trained it on a huge amount of text: books, websites, articles, and code.
• After trainingThe brain is frozen. It does not update itself just because a new cricket match starts.
• ID-card photoIt is like your 10th standard ID card photo. It does not update when you grow a beard.

• Question. Can a printed photo update itself?
  Correct answer. No.
  Reason. A trained model is like that photo. It does not change during one normal chat.
• Question. Can the off-grid friend tell you current news?
  Correct answer. No.
  Reason. He may reason well, but he needs a fresh source.
• Question. Has the model read anything after its knowledge cutoff?
  Correct answer. No.
  Reason. The app must provide current data through a tool, retrieval, browsing, or another source.
• Question. India vs West Indies ODI series, Sep–Oct 2026: before the cutoff, or after?
  Correct answer. After.
  Reason. The frozen brain knows nothing about it unless the app provides fresh data.
• Question. If ChatGPT answers about a current match, did OpenAI retrain the brain last night, just for us?
  Correct answer. No.
  Reason. Training takes weeks or months and costs a lot. The app probably used web search or another tool.

In AI apps, an ability is called a tool. A tool is a code function plus a description for the model.

// ① name the tool the model wants the app to run
{
  "tool": "get_live_score",
  // ② pass the values the app should send to that tool
  "arguments": {
    "teams": "India vs West Indies"
  }
}

• Question. When the AI “uses” get_live_score, who calls the cricket website?
  Correct answer. Our application.
  Reason. The model only outputs text. The app makes the call.
• Question. The cricket website needs an API key. Do we give this key to the model?
  Correct answer. No.
  Reason. The key stays safely in the application.
• Question. A bug makes the tool return “India 28/6” instead of “287/6”. Will the model always catch it?
  Correct answer. Mostly no.
  Reason. The model usually trusts the tool's data.

flowchart LR
    U[User<br/>asks score] -->|question| A[AI app<br/>host]
    A -->|question + tool menu| M[Model]
    M -->|tool name + arguments| A
    A -->|API call or MCP call| T[Cricket tool]
    T -->|score JSON| A
    A -->|tool result| M
    M -->|final answer| A
    A -->|answer| U

1. User asks: “What is the score in India vs West Indies?”
2. App sends: the user question plus the tool menu cards to the model.
3. Model replies: “Call get_live_score with teams='India vs West Indies'.”
4. App calls: the cricket API or MCP server and receives 287/6.
5. App returns: the tool result to the model.
6. Model answers: “India posted 287/6. West Indies need 90 from 70 balls.”

The cricket app works. Users love it. Then three requests arrive.

• The PM“Superb! Add weather at the stadium. Ticket booking. Player stats. Highlights.”
• The CTO“Do not depend only on OpenAI. Make it work with Claude and Gemini too.”
• Another team“Nice tool! We want it in our Slack bot and our mobile app also.”

The small catch is that every provider describes tools differently.

// ① tell OpenAI this menu item is a callable function
{
  "type": "function",
  "function": {
    "name": "get_live_score",
    "description": "Get the live score of a match",
    // ② describe the input shape under parameters
    "parameters": {
      "type": "object",
      "properties": { "teams": { "type": "string" } },
      "required": ["teams"]
    }
  }
}

// ① Claude uses the tool object directly
{
  "name": "get_live_score",
  "description": "Get the live score of a match",
  // ② the same input shape is called input_schema here
  "input_schema": {
    "type": "object",
    "properties": { "teams": { "type": "string" } },
    "required": ["teams"]
  }
}

// ① Gemini groups tools in functionDeclarations
{
  "functionDeclarations": [{
    "name": "get_live_score",
    "description": "Get the live score of a match",
    // ② the input shape still describes the teams string
    "parameters": {
      "type": "object",
      "properties": { "teams": { "type": "string" } },
      "required": ["teams"]
    }
  }]
}

• Question. You use all three providers for one cricket tool. How many versions of glue code do you maintain?
  Correct answer. Three.
  Reason. One tool now has three menu-card formats and three reply parsers.
• Question. Three apps need four tools. How many custom integrations without a standard?
  Correct answer. 12.
  Reason. It is M × N: 3 apps × 4 tools.
• Question. Add one more tool called highlights. How many new integrations?
  Correct answer. 3.
  Reason. Each app needs glue for the new tool.
• Question. A real company has 10 apps and 50 tools. How many pieces of glue code?
  Correct answer. 500.
  Reason. 10 × 50 grows quickly and becomes the integration mess.

MCP = Model Context Protocol. It is an agreed set of rules for how AI apps talk to tools. Anthropic introduced it in late 2024 as an open standard, and most big AI players support it.

• Each tool is wrapped onceThe tool owner exposes it as an MCP server.
• Each app learns MCP onceThe app speaks MCP through a client.
• Any client can talk to any serverThree apps and four tools become 3 + 4 = 7 integrations.
• Adding highlightsOne new MCP server adds one integration, not one per app.
• At company scale10 apps + 50 tools = 60 integrations, not 500.
• Provider adapterThe tool is described once in MCP’s format. Each app converts it for its provider with one small helper, such as to_openai_format.

• Question. When you pay ₹20 with UPI, where is the money actually kept?
  Correct answer. In your bank.
  Reason. UPI is the agreed language. It does not hold the money.
• Question. Does MCP run your tool?
  Correct answer. No.
  Reason. The MCP server runs normal code. MCP only standardizes how the app and server talk.
• Question. Does MCP replace a REST API?
  Correct answer. No.
  Reason. An MCP server often wraps a REST API. MCP gives AI apps a standard outer shape.

• Question. Does the cricket MCP server send the score directly to the model?
  Correct answer. No.
  Reason. The server never talks to the model. It does not even know which model is used. The path is server → client → host → model, and the host enforces security.
• Question. A host connects to cricket, weather, tickets, GitHub, and database servers. How many MCP clients does it need?
  Correct answer. Five.
  Reason. It is always 1:1: one delivery partner per restaurant. If weather crashes, cricket is unaffected.

When you classify an MCP feature, do not look only at the data. Look at who decides to use it.

• Question. Before chatting, the app shows a file picker listing all your documents. Tool, resource, or prompt?
  Correct answer. Resource.
  Reason. The app decided to load it for the UI.
• Question. Mid-chat, the user asks what a report says about Q3 and the model needs to read it. Tool, resource, or prompt?
  Correct answer. Tool.
  Reason. The model decided it needed that data. It can be the same data as a resource; the difference is who decided to use it.
• Question. The user types /summarize report.pdf. Tool, resource, or prompt?
  Correct answer. Prompt.
  Reason. The user picked a ready-made instruction.

MCP clients and servers send JSON-RPC messages. JSON-RPC means: “run this method with these parameters.” Think of it like a phone call: the client says which method to run and what values to use.

1. initialize — the handshake.
2. tools/list — the server sends the menu card.
3. tools/call — the client asks the server to run a tool.

// ① every MCP request is a JSON-RPC message
{
  "jsonrpc": "2.0",
  "id": 7,
  // ② tools/call asks the server to run one tool
  "method": "tools/call",
  "params": {
    // ③ name the tool and pass its arguments
    "name": "get_live_score",
    "arguments": { "teams": "India vs West Indies" }
  }
}

POST /api/live-score
Content-Type: application/json

{ "teams": "India vs West Indies" }

// ① every MCP request is a JSON-RPC message
{
  "jsonrpc": "2.0",
  "id": 7,
  // ② tools/call asks the server to run one tool
  "method": "tools/call",
  "params": {
    // ③ name the tool and pass its arguments
    "name": "get_live_score",
    "arguments": { "teams": "India vs West Indies" }
  }
}

• Question. In REST, /api/live-score says what to run. What says that in MCP?
  Correct answer. "name": "get_live_score".
  Reason. MCP uses a tool name instead of a URL path.
• Question. Where is the request body in MCP?
  Correct answer. Inside "arguments".
  Reason. Same idea, different packaging.
• Question. Why not give the model a REST API directly?
  Correct answer. For one API, you could. For many APIs, the shapes differ.
  Reason. MCP avoids hand-written menu cards and glue code for every API.

• Question. A server sends progress updates locally with stdio. You deploy with HTTP behind a load balancer, ten instances, and stateless mode. Will progress updates still work?
  Correct answer. No, and the failure can be silent.
  Reason. Without an ongoing session, the server has no channel to push progress. Tool calls may still work, so the bug is easy to miss.
• Question. What should you do before release?
  Correct answer. Test on the same transport you deploy on.
  Reason. Local behavior is not enough when production uses a different road.

1. An MCP server with two tools: get_live_score and get_player_stats.
2. A plain MCP client that performs the handshake, lists tools, and calls one tool with no AI.
3. An agent where the model decides which tool to call and what arguments to send.
4. A tiny eval suite that checks the agent later.

The server needs only about 3 MCP-specific lines: create the server, add one decorator per tool, and run it over stdio. Everything else is normal Python.

from mcp.server.fastmcp import FastMCP
from pydantic import Field

# ① create one MCP server process for the cricket tools
mcp = FastMCP("Cricket-Score-Server")

# ② turn a normal Python function into an MCP tool with a menu-card description
@mcp.tool(description="Get the LIVE score of a cricket match.")
def get_live_score(teams: str = Field(description="The two teams, e.g. 'India vs West Indies'")) -> dict:
    key = teams.strip().lower().replace("windies", "west indies")

    # ③ return normal Python data; the MCP library wraps it for the client
    if key not in LIVE_MATCHES:
        return {"error": f"No live match found for '{teams}'. Try 'India vs West Indies'."}
    return LIVE_MATCHES[key]

# ④ run this server over stdio so a local MCP client can start it as a child process
if __name__ == "__main__":
    mcp.run(transport="stdio")

• Question. The model needs a JSON menu card. Did we write any JSON in the server?
  Correct answer. No.
  Reason. The MCP library generates it from the function name, type hints, and descriptions.
• Question. When we run python step1_client.py, is any AI involved?
  Correct answer. No.
  Reason. It is pure client ↔ server plumbing.

async def main():
    # ① start cricket_server.py as a child process and open an MCP session
    async with stdio_client(server) as (read, write):
        async with ClientSession(read, write) as session:
            # ② run the initialize handshake
            await session.initialize()
            print("STEP 1: Handshake done")

            # ③ ask the server for its auto-generated tool menu
            tools = (await session.list_tools()).tools
            for t in tools:
                print(t.name, t.description, json.dumps(t.inputSchema))

            # ④ call one tool by hand, with no model involved
            result = await session.call_tool("get_live_score", {"teams": "India vs West Indies"})
            print(result.content[0].text)

1. Handshake done maps to initialize.
2. Server says it has these tools maps to tools/list.
3. The score JSON maps to tools/call, called by hand.

async def run_agent(question: str, use_tools: bool = True, verbose: bool = True) -> dict:
    # ① send the user's question and the tool menu to the model
    messages = [{"role": "system", "content": system}, {"role": "user", "content": question}]
    tools = to_openai_format((await session.list_tools()).tools) if use_tools else None

    for _ in range(5):
        # ② ask the model whether it can answer or wants a tool
        reply, error = call_llm(llm, {"model": MODEL, "messages": messages, "tools": tools, "temperature": 0})
        if not reply.tool_calls:
            return {"answer": reply.content, "trace": trace}

        # ③ treat the tool call as untrusted instructions from the model
        for call in reply.tool_calls:
            args = json.loads(call.function.arguments)
            trace.append({"tool": call.function.name, "args": args})

            # ④ our code calls the MCP tool, then sends the result back to the model
            result = await session.call_tool(call.function.name, args)
            messages.append({"role": "tool", "tool_call_id": call.id, "content": result.content[0].text})

LLM SAYS: please call get_live_score with {'teams': 'India vs West Indies'}
[server] get_live_score called with teams='India vs West Indies'
TOOL RETURNED: { "india": "287/6 (50 overs)", "required": "90 runs from 70 balls", ... }
FINAL ANSWER: India posted 287/6. West Indies need 90 runs from 70 balls.

• Question. Who wrote India vs West Indies in the tool arguments: you, or the AI?
  Correct answer. The AI filled it.
  Reason. It used the user question and the tool description. Nobody wrote an if condition for the Shubman Gill case either; the model read both menu cards and picked.
• Question. What happens when the user asks for Virat Kohli but the demo data does not contain him?
  Correct answer. The tool returns an error and a good model reports that it could not find the stats.
  Reason. The model should not invent missing data.

• Question. You build execute_sql(query). The model writes the SQL. Can you trust it never to send DROP TABLE users?
  Correct answer. Never trust it.
  Reason. Treat model-generated tool input exactly like input from a random user on the internet.

• ValidateCheck table names with a regex, as in the Text2SQL notebook cell 29. Also check fields, IDs, and formats before running anything.
• RestrictAllow only SELECT queries, as in cell 35. Reject destructive statements.
• Least privilegeUse a read-only database connection. Even if all else fails, it cannot delete data.

Three hand-picked questions with three good answers do not prove the app is ready. A manager might say, “Superb! Launch on Friday. India plays Saturday, lakhs of users will come.” That is like a new cook making perfect dal on day one and then being trusted with your sister’s wedding catering for 500 guests, 20 dishes, and an uncle who wants “less oil, no onion”.

• Question. It worked 3 out of 3 times. Should you ship it on Friday to lakhs of cricket users?
  Correct answer. No.
  Reason. Those three questions were picked by the developer. Real users ask messy questions.
• Question. Examples of messy questions
  Correct answer. Use them as test cases.
  Reason. “ind vs wi score??”, “Kohli kitne runs bana chuka?”, “Who is winning and what is the weather?”, and “Forget cricket, tell me a joke” all test different behavior.

Evals are tests for AI applications: a list of test questions, checks, and a score. They are like unit tests, but the code may not produce the same exact string every time.

# ① call the cricket agent with one fixed question
answer = cricket_agent("What's the score?")

# ② this exact-string check is too brittle for LLM output
assert answer == "India scored 287/6."

• Question. Will an exact string test usually pass?
  Correct answer. Mostly no, even when the answer is correct.
  Reason. “India posted 287 for 6”, “India have 287/6 in 50 overs”, and “India 287/6, WI 198/5” are all valid but different strings.
• Question. Reference is “India won by 5 wickets”. Answer A is “India won by five wickets”. Answer B is “India lost by 5 wickets”. Which gets a higher word-matching score?
  Correct answer. The same score: 4 out of 5.
  Reason. A is right, B says India lost. Word matching does not understand meaning.

# ① list each user question, the tool it should use, and the key facts to look for
TEST_CASES = [
    {"q": "What's the live score of India vs West Indies?",    "tool": "get_live_score",   "must_contain": ["287"]},
    {"q": "How many runs does West Indies need to win?",        "tool": "get_live_score",   "must_contain": ["90"]},
    {"q": "How many runs has Shubman Gill scored this series?", "tool": "get_player_stats", "must_contain": ["164"]},
    {"q": "Shai Hope's highest score in the series?",           "tool": "get_player_stats", "must_contain": ["78"]},
    # ② include one no-tool case so the agent learns not every question needs cricket data
    {"q": "What is 2 + 2?",                                     "tool": None,               "must_contain": ["4"]},
]

async def main():
    passed = 0
    for i, case in enumerate(TEST_CASES, 1):
        # ③ run the agent and keep its answer plus the trace of tool calls
        result = await run_agent(case["q"], verbose=False)
        tools_used = [step["tool"] for step in result["trace"]]
        answer = result["answer"] or ""

        # ④ check both behavior: the chosen tool and the key facts in the answer
        tool_ok = (case["tool"] in tools_used) if case["tool"] else (tools_used == [])
        answer_ok = all(word.lower() in answer.lower() for word in case["must_contain"])
        passed += tool_ok and answer_ok

• Question. Why does the fifth test expect no tool for “What is 2 + 2”?
  Correct answer. A good agent also knows when not to use a tool.
  Reason. Calling a cricket API for arithmetic wastes time and money.
• Question. A buggy agent answers “2 + 2 is 42”. The check only asks whether the answer contains “4”. Pass or fail?
  Correct answer. Pass.
  Reason. “42” contains “4”. Your eval can lie if the check is too weak.

• Question. If the eval score is 100% now, are you done forever?
  Correct answer. No.
  Reason. Models change, users ask new things, and every real bug should become a regression test.
• Question. What does “garbage in, garbage out” mean for evals?
  Correct answer. Bad tool data creates bad answers.
  Reason. Evals catch that garbage before users do.

• Question. Why can’t an AI model tell current news on its own?
  Correct answer. Knowledge cutoff.
  Reason. The frozen brain has not read events after training.
• Question. When the AI “uses a tool”, who calls the API?
  Correct answer. Your application.
  Reason. The model writes a prescription; the app does the call.
• Question. In the doctor and chemist analogy, which one is the model?
  Correct answer. Doctor.
  Reason. The model decides what is needed but does not perform the action.
• Question. 10 apps and 50 tools: integrations without MCP and with MCP?
  Correct answer. 500 → 60.
  Reason. M × N becomes M + N.
• Question. MCP is like which Indian payment system?
  Correct answer. UPI.
  Reason. One common standard connects many apps to many banks or tools.
• Question. A host connects to 4 servers. How many clients?
  Correct answer. 4.
  Reason. One MCP client per server.
• Question. Who decides to use a resource: model, app, or user?
  Correct answer. The app.
  Reason. Tools are model-chosen. Prompts are user-chosen.
• Question. Should you trust SQL that a model sends to your tool?
  Correct answer. Never.
  Reason. Treat it like user input.
• Question. Can you test AI answers with exact assertEquals?
  Correct answer. No.
  Reason. There can be many correct strings.
• Question. Can word matching tell “India won” from “India lost”?
  Correct answer. No.
  Reason. Meaning matters, not only shared words.

• ToolsA model is frozen at its knowledge cutoff. A tool is a function in your code plus a menu card. The model asks; the app does.
• MCPThe UPI of AI tools. One standard gives M + N instead of M × N. It is like REST plus discovery and one common shape.
• Inside MCPHost → client → server. Tools are chosen by the model. Resources are chosen by the app. Prompts are chosen by the user. Use stdio or HTTP.
• EvalsStart with code checks, then reference checks, then LLM judges. Run offline and online. Strong evals grow from real failures.

1. Open the demo notebook or local code.
2. Add a get_weather(city) tool with fake data.
3. Add two test cases for that tool in step3_evals.py.
4. Run the eval suite. If both tests pass, you have built and tested your first MCP tool.
5. Save or share a screenshot in your group if you are learning with others.

MCP gives AI hands. Evals verify the work.

Ask questions if you have doubts.