About This Course

LLM API and Orchestration is for engineers who can call a language model from a notebook and now need to run model calls in production: across several providers, with users waiting, budgets to respect and data to protect. It moves you from single, hand-written API calls to a deliberate integration layer that streams, caches, routes, falls back and enforces guardrails.

The course pairs free external material with original Factored deep dives and hands-on follow-alongs. Anthropic's Claude Platform 101 and Building with the Claude API, a streaming tutorial from MachineLearningPlus, DataCamp's LiteLLM guide, and two verified videos on AI gateways give you a broad on-ramp. The original units go further, connect every topic to how we build systems for clients, and walk you through building your own multi-provider gateway with Claude and free models.

What You Will Learn

In the first half of the course you build directly on the Claude API. You learn what a production call looks like beyond the answer text: choosing a model by cost and latency, reading stop reasons and usage, counting tokens, and handling tool use at the request level, including how its format differs between providers. You learn which errors to retry and which to fix, how Claude's rate limits work, and how to size retries and timeouts. You then stream responses end to end, from the Claude event sequence to a FastAPI endpoint and a browser, and cut cost and latency with prompt caching, comparing how Anthropic, OpenAI and Gemini cache and measuring your own hit rate.

In the second half you learn why LLM gateways exist, when they are not worth it, and how to choose between LiteLLM, Portkey, OpenRouter and cloud options. You build a gateway with LiteLLM and again with Portkey, using Claude through OpenRouter alongside free Gemini, Groq and local Ollama models, and deliberately break a provider to watch fallback work. You then learn where guardrails belong, how rules, classifiers and LLM judges compare, and add PII masking and an injection check to your gateway. The course ends with a mission to build a production-grade gateway that brings all of it together.

Who This Course Is For

This course is for software engineers and data engineers in the Jed-AI Academy who are comfortable with Python, virtual environments, JSON and REST APIs, and who already understand at a basic level what a language model, a token and a context window are. No prior experience with the Claude API, streaming, prompt caching, gateways or guardrails is assumed; every one of those topics is taught inside the course before it is used or tested.

How the Course Is Structured

A Welcome section explains the course, its prerequisites and how to request model access. Section 1, Building on the Claude API, covers the platform, core API skills and tool use, and what to do when calls fail. Section 2, Streaming and Prompt Caching, covers streaming end to end and caching across providers. Section 3, LLM Gateways: Why and How, explains the problems gateways solve and their costs, then runs two follow-alongs with LiteLLM and Portkey and a guide to choosing between options. Section 4, Guardrails, covers a Factored workshop recording, where guardrails live, which kinds of checks to use, and a third follow-along that adds them to your gateway. Each of these four sections ends with a short scenario-based Section Check. Section 5 is the Jed-AI Mission.

Time Commitment and Completion

The course is self-paced and follows the Week 3 block of the Jed-AI calendar. External courses and videos are taken on their own platforms; the follow-alongs are meant to be run on your own machine, and they are where most of the learning happens, so give them unhurried time. Open edX tracks your progress as you view units and submit the Section Checks, and the mission is submitted through the channel your cohort lead announces.