Production-Grade Generative AI and Agentic AI Engineer Program

You can explain what a language model is doing when it answers, in terms a sceptical engineer will accept, and say what it cannot do.

1. From n-grams to neural language models1 hr
   • statistical language models
   • word embeddings
   • sequence models
   • the scaling story
   • what changed
2. Tokenisation and why it matters1 hr
   • byte pair encoding
   • vocabulary size
   • token boundaries
   • non-English penalty
   • token counting
3. Next-token prediction and sampling1 hr
   • logits and softmax
   • temperature
   • top-p and top-k
   • greedy decoding
   • determinism limits
4. Training stages: pretraining, SFT, alignment1 hr
   • pretraining corpora
   • supervised fine-tuning
   • RLHF and alternatives
   • instruction following
   • refusal behaviour
5. Capabilities, limits and hallucination1 hr
   • knowledge cutoff
   • confabulation mechanics
   • arithmetic weakness
   • context grounding
   • calibration
6. Choosing a model for a task1 hr
   • frontier vs small models
   • open weights
   • latency and cost
   • licence constraints
   • benchmark scepticism

Hands-on project: Write a technical explainer for your team covering tokenisation, sampling and why the same prompt gives different answers, with three worked examples.

You can read a transformer diagram and say what each block does, which is what makes the rest of the course something you reason about rather than copy.

1. Embeddings and positional encoding1 hr
   • token embeddings
   • learned positions
   • rotary embeddings
   • context length
   • extrapolation limits
2. Self-attention mechanics2 hrs
   • query key value
   • scaled dot product
   • attention weights
   • causal masking
   • intuition for heads
3. Multi-head attention and feed-forward blocks1 hr
   • head splitting
   • head specialisation
   • MLP block
   • activation functions
   • residual stream
4. Normalisation, residuals and stability1 hr
   • layer normalisation
   • pre-norm vs post-norm
   • residual connections
   • gradient flow
   • training stability
5. Efficient attention and long context1 hr
   • KV cache
   • grouped query attention
   • flash attention
   • sliding window
   • context cost curve
6. Reading a model card and config1 hr
   • parameter count
   • hidden size
   • layer count
   • vocab size
   • licence terms

Hands-on project: Implement a minimal decoder-only transformer from scratch in PyTorch and train it on a small corpus until it produces coherent text.

You can get reliable, structured output from a model and know which failures are prompt problems and which are not.

1. System prompts and instruction hierarchy1 hr
   • role setting
   • constraint phrasing
   • instruction conflicts
   • priority ordering
   • refusal conditions
2. Few-shot and in-context learning1 hr
   • example selection
   • format anchoring
   • ordering effects
   • when examples hurt
   • dynamic examples
3. Structured output and schema enforcement2 hrs
   • JSON mode
   • schema constraints
   • parse failure recovery
   • nested objects
   • validation layer
4. Chain-of-thought and reasoning control1 hr
   • step-by-step prompting
   • extended thinking
   • verbosity budgets
   • hidden reasoning
   • cost trade-off
5. Context window management1 hr
   • token budgeting
   • truncation strategy
   • position effects
   • compaction
   • overflow handling
6. Prompt versioning and measurement1 hr
   • prompt registry
   • A/B testing
   • regression sets
   • metric choice
   • rollback

Hands-on project: Build a classification service that returns schema-valid JSON for a thousand varied inputs with a measured failure rate under one percent.

You can choose an embedding model and index type for a corpus and justify both on recall and cost rather than popularity.

1. What an embedding represents1 hr
   • semantic similarity
   • dimensionality
   • cosine vs euclidean
   • anisotropy
   • domain mismatch
2. Embedding model selection1 hr
   • general vs domain models
   • multilingual embeddings
   • dimension trade-off
   • truncation behaviour
   • benchmark reading
3. Vector index structures2 hrs
   • flat index
   • HNSW
   • IVF
   • product quantisation
   • recall vs speed
4. Vector databases in practice1 hr
   • Qdrant
   • pgvector
   • Pinecone
   • metadata filtering
   • operational cost
5. Chunking strategy2 hrs
   • fixed-size
   • semantic chunking
   • heading-aware splits
   • overlap tuning
   • metadata attachment
6. Measuring retrieval quality1 hr
   • labelled query set
   • recall at k
   • MRR
   • latency percentiles
   • regression tracking

Hands-on project: Benchmark three embedding models and two index types over a real corpus and publish a recommendation with measured recall and latency.

You can build a retrieval-augmented system that grounds answers in your documents and cites them, and explain every stage of the pipeline.

1. Ingestion: parsing and normalising documents2 hrs
   • PDF extraction
   • table handling
   • OCR fallback
   • metadata extraction
   • deduplication
2. Indexing pipelines and incremental updates2 hrs
   • batch indexing
   • incremental upsert
   • deletion handling
   • versioned documents
   • reindex strategy
3. Retrieval and context assembly2 hrs
   • top-k selection
   • context ordering
   • token budget
   • deduplication
   • source diversity
4. Generation with grounding and citation2 hrs
   • citation prompting
   • span attribution
   • refusal when ungrounded
   • quote fidelity
   • answer formatting
5. Handling no-answer and low-confidence cases1 hr
   • confidence thresholds
   • honest refusal
   • clarifying questions
   • escalation
   • user messaging
6. Baseline measurement and failure analysis1 hr
   • accuracy baseline
   • failure taxonomy
   • retrieval vs generation faults
   • error budget
   • iteration loop

Hands-on project: Build a RAG service over a real document corpus with citation rendering, a measured accuracy baseline and a documented failure analysis.

You can take a mediocre RAG system and make it measurably better, knowing which lever to pull for which failure mode.

1. Hybrid dense and lexical retrieval2 hrs
   • BM25
   • reciprocal rank fusion
   • weighting
   • sparse-dense trade-offs
   • filter pushdown
2. Reranking with cross-encoders2 hrs
   • bi-encoder vs cross-encoder
   • rerank depth
   • latency cost
   • model choice
   • score calibration
3. Query transformation and expansion2 hrs
   • query rewriting
   • HyDE
   • multi-query
   • decomposition
   • intent classification
4. Structured and graph retrieval2 hrs
   • text-to-SQL
   • knowledge graphs
   • entity linking
   • hybrid structured search
   • schema grounding
5. Contextual and hierarchical chunking1 hr
   • parent-child chunks
   • contextual prefixes
   • summary indexes
   • hierarchical retrieval
   • window expansion
6. Attributing each improvement1 hr
   • ablation testing
   • component isolation
   • measured deltas
   • cost per point
   • diminishing returns

Hands-on project: Improve a baseline RAG system's answer accuracy by at least twenty points through hybrid retrieval, reranking and query transformation, with each gain attributed.

You can prove a change helped with numbers an engineering manager will accept, instead of vibes from a handful of prompts.

1. Building a golden evaluation set1 hr
   • case sampling
   • labelling guidelines
   • annotator agreement
   • edge cases
   • set maintenance
2. Faithfulness and groundedness metrics2 hrs
   • claim extraction
   • support checking
   • hallucination rate
   • citation accuracy
   • unsupported spans
3. Answer relevance and completeness1 hr
   • relevance scoring
   • completeness checks
   • verbosity penalty
   • format compliance
   • rubric design
4. LLM-as-judge and its biases2 hrs
   • judge prompt design
   • position bias
   • verbosity bias
   • self-preference
   • human calibration
5. Regression testing in CI1 hr
   • eval pipeline
   • pass thresholds
   • cost ceilings
   • flake handling
   • historical tracking
6. Online evaluation and feedback loops1 hr
   • implicit signals
   • thumbs feedback
   • sampled human review
   • drift detection
   • feeding the eval set

Hands-on project: Build an evaluation suite with a hundred cases covering faithfulness, relevance and retrieval recall, running in CI.

You can give a model tools and have it select them correctly, which is the single step that turns a chatbot into a system that does work.

1. Tool definitions and JSON schemas1 hr
   • schema design
   • parameter descriptions
   • enums and constraints
   • required fields
   • naming discipline
2. Tool selection and disambiguation2 hrs
   • overlapping tools
   • tool count effects
   • forced tool choice
   • parallel calls
   • selection debugging
3. Executing tools safely1 hr
   • argument validation
   • timeout handling
   • error surfaces
   • side-effect gating
   • idempotency
4. The agent loop2 hrs
   • observe act repeat
   • termination conditions
   • step budgets
   • loop detection
   • state carry-over
5. Model Context Protocol servers1 hr
   • MCP concepts
   • server implementation
   • resource exposure
   • transport
   • client wiring
6. When tools are better than fine-tuning1 hr
   • knowledge vs capability
   • freshness
   • auditability
   • cost comparison
   • decision rubric

Hands-on project: Build a six-tool assistant with typed schemas and a test proving each tool is chosen for the right intent and rejected for the wrong one.

You can build a multi-agent workflow with explicit state and durable checkpoints, and say honestly when one agent would have been better.

1. Graph state, nodes and conditional routing2 hrs
   • state schema
   • node functions
   • conditional edges
   • cycles
   • compilation
2. Supervisor, router and specialist patterns2 hrs
   • supervisor design
   • handoff contracts
   • specialisation boundaries
   • loop prevention
   • termination
3. CrewAI roles, tasks and process models2 hrs
   • agent roles
   • task definition
   • sequential process
   • hierarchical process
   • delegation
4. Checkpointing, interrupts and human approval2 hrs
   • checkpointer backends
   • thread state
   • interrupt points
   • approval gates
   • resume semantics
5. Parallel branches and result aggregation1 hr
   • fan-out
   • fan-in
   • partial failure
   • ordering
   • token multiplication
6. Measuring whether multi-agent earned its cost1 hr
   • single-agent baseline
   • quality delta
   • token cost delta
   • latency delta
   • honest conclusion

Hands-on project: Build a research-and-report pipeline with four specialised agents, persistent checkpoints, human approval gates and a measured comparison against a single-agent baseline.

You can decide between prompting, RAG and fine-tuning on evidence, and run a fine-tune that actually improves a measured metric.

1. The decision: prompt, retrieve or tune1 hr
   • capability vs knowledge
   • data volume needed
   • cost model
   • maintenance burden
   • decision rubric
2. Dataset construction and quality2 hrs
   • instruction formatting
   • data volume
   • deduplication
   • contamination checks
   • train-test split
3. Supervised fine-tuning mechanics2 hrs
   • loss masking
   • learning rate
   • epochs and overfitting
   • batch size
   • gradient accumulation
4. Evaluating a fine-tune honestly1 hr
   • held-out set
   • baseline comparison
   • regression on general ability
   • overfit signals
   • cost per point
5. Preference tuning and alignment methods1 hr
   • preference pairs
   • DPO
   • reward modelling
   • when it is overkill
   • safety implications
6. Lifecycle: versioning, drift and retraining1 hr
   • model registry
   • version pinning
   • drift detection
   • retraining triggers
   • rollback

Hands-on project: Fine-tune a small open model for a narrow task, measure it against a prompted baseline, and write up whether it was worth doing.

You can adapt an open model on a single GPU and explain the memory arithmetic that makes it possible.

1. Memory arithmetic for training1 hr
   • parameter memory
   • optimiser states
   • activations
   • gradient checkpointing
   • budget calculation
2. LoRA and low-rank adaptation2 hrs
   • rank selection
   • target modules
   • alpha scaling
   • adapter size
   • merge semantics
3. QLoRA and quantised training1 hr
   • 4-bit quantisation
   • NF4
   • double quantisation
   • paged optimisers
   • quality cost
4. Adapter management and composition1 hr
   • multiple adapters
   • hot swapping
   • adapter registry
   • per-tenant adapters
   • conflict behaviour
5. Quantisation for inference2 hrs
   • GPTQ
   • AWQ
   • GGUF
   • quality measurement
   • hardware fit
6. Distillation and small-model strategies1 hr
   • teacher-student
   • synthetic data
   • task-specific distillation
   • cost savings
   • quality ceiling

Hands-on project: Train a LoRA adapter on a domain dataset within a 24 GB memory budget, merge it, and serve the result.

You can serve an open model at a throughput and latency you can defend, and explain where every millisecond and gigabyte goes.

1. The inference loop and KV cache2 hrs
   • prefill vs decode
   • KV cache sizing
   • memory per sequence
   • cache eviction
   • context cost
2. Batching strategies2 hrs
   • static batching
   • continuous batching
   • scheduling
   • throughput vs latency
   • queue behaviour
3. Serving with vLLM2 hrs
   • paged attention
   • tensor parallelism
   • server configuration
   • OpenAI-compatible API
   • health checks
4. Local and edge deployment with Ollama1 hr
   • model pulling
   • GGUF quantisations
   • resource limits
   • offline use
   • when local wins
5. Load testing and capacity planning2 hrs
   • token throughput
   • concurrency ceiling
   • p95 latency
   • GPU utilisation
   • cost per million tokens
6. Caching and request shaping1 hr
   • prompt caching
   • semantic cache
   • dedup of identical calls
   • streaming
   • early termination

Hands-on project: Deploy a model with vLLM, load-test it, and publish a capacity model giving tokens per second per GPU at a target p95 latency.

You can run a generative system as a service with versioning, tracing and a cost line someone in finance will accept.

1. Packaging and deployment topology1 hr
   • container builds
   • GPU scheduling
   • autoscaling signals
   • warm pools
   • graceful shutdown
2. Versioning prompts, models and indexes together1 hr
   • version manifest
   • index versioning
   • compatibility matrix
   • atomic rollout
   • rollback bundle
3. Tracing with Arize Phoenix and OpenTelemetry2 hrs
   • span structure
   • retrieval spans
   • token attribution
   • PII redaction
   • sampling
4. Cost attribution and budget control2 hrs
   • per-request cost
   • per-tenant attribution
   • budget alerts
   • model mix
   • anomaly detection
5. Drift, quality regression and alerting1 hr
   • embedding drift
   • answer quality drift
   • index staleness
   • alert thresholds
   • review cadence
6. Canary rollout and automated rollback1 hr
   • traffic splitting
   • quality gates
   • automated rollback
   • kill switch
   • incident runbook

Hands-on project: Deploy a RAG service with full tracing, a cost dashboard, canary rollout and an automated rollback trigger.

You can pass a security and compliance review for a generative system, with evidence rather than assurances.

1. Prompt injection and the data boundary2 hrs
   • direct injection
   • indirect via documents
   • instruction vs data
   • delimiter defences
   • red-team suite
2. Data privacy, PII and retention1 hr
   • PII detection
   • redaction pipelines
   • retention windows
   • deletion requests
   • residency constraints
3. Access control and tenant isolation1 hr
   • per-tenant indexes
   • row-level filtering
   • leakage testing
   • credential scoping
   • audit trail
4. Output safety and content filtering1 hr
   • unsafe output classes
   • filtering layers
   • refusal design
   • escalation
   • false positive cost
5. Governance, audit and model policy2 hrs
   • usage policy
   • approval workflow
   • audit logging
   • model inventory
   • incident reporting
6. Regulatory landscape and documentation1 hr
   • EU AI Act basics
   • sector rules
   • model cards
   • data sheets
   • evidence index

Hands-on project: Produce a governance package: threat model, prompt-injection test results, PII handling design, audit logging and a model usage policy.

You can deliver a complete generative AI system to production standard: grounded, evaluated, observable, secured and costed.

1. Scoping, architecture and success metrics2 hrs
   • use case definition
   • architecture diagram
   • metric selection
   • non-goals
   • cost ceiling
2. Ingestion, indexing and retrieval layer2 hrs
   • parsing pipeline
   • chunking choice
   • hybrid index
   • reranking
   • retrieval evaluation
3. Agentic query layer and tools2 hrs
   • tool inventory
   • routing logic
   • state handling
   • fallbacks
   • step budgets
4. Serving, optimisation and capacity2 hrs
   • vLLM deployment
   • batching config
   • load test
   • capacity model
   • cost per query
5. Evaluation, tracing and dashboards2 hrs
   • eval suite
   • CI gating
   • trace wiring
   • cost dashboard
   • drift alerts
6. Security review and production defence2 hrs
   • injection testing
   • PII handling
   • audit logging
   • governance pack
   • panel defence

Hands-on project: Build and deploy a production platform over a real corpus: ingestion and indexing pipeline, hybrid retrieval with reranking, an agentic query layer with tools, a fine-tuned or adapted component where justified, vLLM serving, full tracing and cost dashboard, a hundred-case evaluation suite in CI, prompt-injection test results and a governance pack, defended in review.