Generative AI Course

You can distinguish generative architectures from discriminative models and set up an isolated Python environment with PyTorch and Hugging Face.

1. Generative vs discriminative models and where GenAI fits90 min
   • conditional probability distributions
   • maximum likelihood estimation
   • discriminative classification boundaries
   • latent space generative trajectories
   • likelihood-based model training
2. The modern GenAI landscape: LLMs, diffusion, multimodal and audio90 min
   • autoregressive language models
   • latent diffusion models
   • vision-encoder architectures
   • speech-to-text transformer pipelines
   • multimodal alignment layers
3. Probability, sampling and learning a data distribution2 hrs
   • softmax probability vectors
   • categorical sampling
   • Kullback-Leibler divergence
   • temperature scaling parameters
   • entropy in probability distributions
4. Neural network and deep learning refresher2 hrs
   • backpropagation in PyTorch
   • tensor operations and shape broadcasting
   • activation functions like GELU and SiLU
   • optimizer states in AdamW
   • gradient accumulation steps
5. Environment setup: Python, PyTorch and the Hugging Face ecosystem2 hrs
   • pip and conda virtual environments
   • torch.cuda device management
   • transformers and datasets libraries
   • huggingface-cli login and token setup
   • accelerate library configuration
6. GPU compute, VRAM management and mixed precision training2 hrs
   • FP16 and BF16 precision formats
   • autocast context manager in PyTorch
   • gradient checkpointing for memory saving
   • VRAM fragmentation debugging
   • CUDA out-of-memory error resolution

Hands-on project: Write a Python script that loads a pre-trained tokenizer from Hugging Face, tokenizes a raw text corpus, and outputs vocabulary statistics and token frequency histograms.

You can explain multi-head self-attention, token embeddings, and positional encodings mathematically and inspect attention weight matrices in PyTorch.

1. Tokenization: BPE, WordPiece and SentencePiece2 hrs
   • Byte-Pair Encoding merge rules
   • subword tokenization algorithms
   • special tokens like BOS, EOS, and PAD
   • vocabulary size optimization trade-offs
   • token-to-id mapping tables
2. Word and subword embeddings2 hrs
   • high-dimensional vector spaces
   • nn.Embedding lookup tables
   • semantic geometric relationships
   • weight tying between embedding and LM head
   • embedding normalization techniques
3. The attention mechanism: scaled dot-product and multi-head attention3 hrs
   • query, key, and value projections
   • scaling factor division by square root of d_k
   • multi-head tensor reshaping
   • attention score matrix multiplication
   • causal masking triangular matrices
4. The Transformer: encoders, decoders and positional encodings3 hrs
   • sinusoidal vs learned positional encodings
   • residual connections and LayerNorm
   • masked causal attention layers
   • feed-forward network expansion ratios
   • RMSNorm vs LayerNorm formulations
5. Pre-training objectives: causal and masked language modelling2 hrs
   • next-token prediction loss calculation
   • cross-entropy loss over vocabulary logits
   • autoregressive sequence generation
   • masked language modelling span corruption
   • perplexity evaluation metrics
6. Scaling laws, emergent abilities and model families2 hrs
   • compute-optimal training parameters
   • Chinchilla scaling laws
   • parameter count vs dataset size trade-offs
   • emergent capabilities in large models
   • architectural shifts in recent LLMs

Hands-on project: Implement a scaled dot-product attention block from scratch using PyTorch tensors and verify output shapes against a multi-head reference layer.

You can configure decoding hyperparameters, manage KV cache memory footprints, and serve open-source weights locally via Ollama.

1. Model landscape: GPT, LLaMA, Mistral, Gemma and Qwen2 hrs
   • open-weights vs proprietary model ecosystems
   • architectural variations like grouped-query attention
   • context length limits across families
   • sliding window attention in Mistral
   • mixture of experts routing mechanisms
2. Open vs closed models and how to choose90 min
   • data privacy compliance constraints
   • hosting cost vs API fee calculations
   • latency benchmarks
   • fine-tuning flexibility of open weights
   • vendor lock-in considerations
3. Decoding strategies: temperature, top-k, top-p and beam search2 hrs
   • temperature scaling on logits
   • nucleus sampling top-p probability mass
   • repetition penalty parameters
   • frequency and presence penalties
   • beam search token pruning
4. Context windows, KV cache and inference cost2 hrs
   • Key-Value cache memory allocation
   • flash attention optimization
   • context window saturation bottlenecks
   • PagedAttention memory management
   • KV cache quantization techniques
5. Running open models locally with Ollama and Hugging Face2 hrs
   • Modelfile creation syntax
   • transformers pipeline API usage
   • GPU memory offloading with device_map
   • GGUF quantization file formats
   • ollama run and pull CLI commands
6. API integration patterns for commercial model providers2 hrs
   • OpenAI client SDK initialization
   • rate limit handling and exponential backoff
   • async request concurrency patterns
   • token usage accounting headers
   • fallback provider architectures

Hands-on project: Deploy an open-source 7B parameter model locally using Ollama and write an evaluation script that benchmarks token generation latency across different temperature settings.

You can design robust system prompts, enforce strict JSON output schemas, and manage context window stuffing for complex tasks.

1. Zero-shot, few-shot and chain-of-thought prompting2 hrs
   • in-context learning mechanics
   • few-shot exemplar selection
   • step-by-step reasoning triggers
   • least-to-most prompt decomposition
   • self-consistency prompting loops
2. System, user and assistant roles and prompt templates90 min
   • chat message array structures
   • Jinja2 prompt template rendering
   • persona boundary enforcement
   • system instruction priority weight
   • multi-turn dialogue history formatting
3. Context engineering: deciding what belongs in the context window2 hrs
   • token budget allocation
   • sliding window history compression
   • noise reduction in retrieved snippets
   • lost-in-the-middle attention mitigation
   • dynamic context pruning strategies
4. Structured outputs: JSON mode and function / tool schemas2 hrs
   • response_format JSON parameters
   • Pydantic schema translation to JSON Schema
   • handling malformed model outputs
   • Instructor library validation retries
   • constrained generation with Guidance
5. Prompt evaluation, iteration and versioning2 hrs
   • automated assertion testing on prompts
   • version tracking in code repositories
   • regression testing prompt changes
   • prompt playground experiment logs
   • evaluation dataset creation
6. Advanced prompting patterns: tree of thoughts and skeleton-of-thought2 hrs
   • Tree of Thoughts search algorithms
   • Skeleton-of-Thought parallel generation
   • reflexion self-correction prompts
   • constitutional AI critique loops
   • meta-prompting generation scripts

Hands-on project: Build a structured data extraction pipeline using OpenAI function schemas that converts unstructured invoice text into a validated Pydantic model.

You can build end-to-end Retrieval-Augmented Generation systems featuring semantic chunking, hybrid search, and cross-encoder re-ranking.

1. Text embeddings and semantic similarity search2 hrs
   • cosine similarity vs dot product metrics
   • embedding dimension normalization
   • dense vector representation of intent
   • MTEB benchmark evaluation scores
   • embedding model selection criteria
2. Vector databases: FAISS, Chroma, Pinecone and pgvector3 hrs
   • HNSW indexing algorithms
   • persistent storage configurations
   • SQL integration with pgvector extensions
   • IVFFlat index parameter tuning
   • metadata filtering in vector queries
3. Document loading, chunking and indexing strategies2 hrs
   • recursive character text splitters
   • metadata enrichment on chunks
   • overlap sizing trade-offs
   • semantic chunking based on embedding distance
   • table and image parsing in PDFs
4. Building an end-to-end Retrieval-Augmented Generation pipeline3 hrs
   • retriever-generator decoupling
   • prompt context injection formatting
   • source attribution citation handling
   • asynchronous document ingestion workers
   • handling empty retrieval results
5. Advanced RAG: hybrid search, re-ranking and query rewriting3 hrs
   • BM25 sparse search integration
   • cross-encoder re-ranking models
   • HyDE hypothetical document embeddings
   • multi-query expansion techniques
   • Reciprocal Rank Fusion algorithms
6. Evaluating RAG: faithfulness, relevance and groundedness2 hrs
   • Ragas evaluation framework metrics
   • context precision and recall testing
   • automated test set generation
   • answer relevance scoring prompts
   • faithfulness hallucination checks

Hands-on project: Construct a production RAG pipeline over a 500-page technical manual using Chroma, recursive text splitting, and an external re-ranker, achieving over 90% groundedness scores.

You can fine-tune large language models efficiently using PEFT, LoRA, and QLoRA on consumer cloud GPUs without memory overflow.

1. Prompt vs RAG vs fine-tuning: choosing the right approach90 min
   • knowledge updates vs behavior modification
   • cost-benefit analysis of training runs
   • maintenance overhead comparisons
   • catastrophic forgetting risks
   • domain adaptation trade-offs
2. Parameter-efficient fine-tuning: LoRA and QLoRA3 hrs
   • low-rank matrix decomposition
   • target module selection for attention weights
   • 4-bit NormalFloat quantization types
   • double quantization memory savings
   • peft configuration dictionaries
3. Instruction tuning and preparing training datasets2 hrs
   • Alpaca and ShareGPT data formatting standards
   • sequence padding and loss masking
   • dataset token length validation
   • data cleaning and deduplication pipelines
   • train-validation split strategies
4. Preference alignment: RLHF and DPO overview2 hrs
   • Reward model scoring mechanics
   • Direct Preference Optimization loss functions
   • policy gradient stability issues
   • Bradley-Terry preference models
   • ORPO odds ratio preference optimization
5. Quantization for efficient inference (bitsandbytes, GPTQ, AWQ)2 hrs
   • post-training quantization algorithms
   • activation outlier handling
   • VRAM footprint reduction calculations
   • GPTQ calibration dataset setup
   • AWQ activation-aware weight clipping
6. Training monitoring, checkpointing and experiment tracking2 hrs
   • Weights & Biases training loss logging
   • learning rate warmup and cosine schedules
   • saving adapter weights vs merged models
   • evaluating checkpoints on validation sets
   • resuming interrupted training jobs

Hands-on project: Fine-tune a LLaMA 3 8B model on a custom JSON dataset using QLoRA, bitsandbytes 4-bit quantization, and Hugging Face Trainer with Weights & Biases tracking.

You can construct conversational applications with persistent memory, tool integration, and streaming responses using LangChain and LlamaIndex.

1. LangChain and LlamaIndex core concepts2 hrs
   • LangChain Expression Language (LCEL) syntax
   • LlamaIndex query engine abstractions
   • composable data connectors
   • Runnable interface methods
   • index structure abstractions
2. Chains, prompt templates, memory and tools2 hrs
   • RunnablePassthrough and pipe operators
   • ConversationBufferMemory management
   • tool decorator implementation
   • RunnableLambda custom transformations
   • history injection in LCEL chains
3. Building chatbots with retrieval and conversation memory2 hrs
   • history-aware retriever chains
   • condense question sub-graphs
   • session persistence handlers
   • ChatMessageHistory store backends
   • redis chat message history adapters
4. Streaming responses, callbacks and observability with LangSmith2 hrs
   • async iterator token streaming
   • LangSmith trace run collection
   • token usage audit logging
   • custom callback handler implementation
   • tracing multi-step chain execution
5. Tool calling and structured output in real applications2 hrs
   • binding tools to chat models
   • handling tool execution exceptions
   • parsing multi-tool response loops
   • structured output parsing with Pydantic
   • dynamic tool registration patterns
6. Error recovery, fallback chains and retry policies2 hrs
   • with_fallbacks chain configuration
   • rate limit retry interceptors
   • parsing error fallback prompts
   • circuit breaker patterns for LLM APIs
   • graceful degradation on timeout

Hands-on project: Build an interactive CLI chatbot in Python that maintains conversation state across sessions, queries a SQL database via LangChain tool abstractions, and streams responses.

You can build autonomous multi-agent systems using LangGraph and the Model Context Protocol for secure external tool execution.

1. What makes an agent: reasoning, tools and memory2 hrs
   • autonomous decision loops
   • perceive-plan-execute cycles
   • state management in graphs
   • working memory vs episodic memory
   • reflection and self-correction loops
2. The ReAct pattern, planning and tool use3 hrs
   • Reason-Action-Observation loop traces
   • handling tool execution failures
   • loop termination conditions
   • plan-and-solve execution strategies
   • scratchpad memory formatting
3. Function calling and the Model Context Protocol (MCP)3 hrs
   • MCP server and client architecture
   • standardized resource and prompt exposure
   • secure local subprocess transport
   • JSON-RPC protocol over stdio
   • exposing database tools via MCP
4. Multi-agent orchestration: orchestrator–worker–reviewer patterns3 hrs
   • state handoff between specialized agents
   • consensus and review loops
   • cyclic graph routing logic
   • supervisor agent delegation patterns
   • shared message bus architectures
5. Agent frameworks: LangGraph, CrewAI and AutoGen2 hrs
   • StateGraph compilation and checkpointing
   • CrewAI hierarchical process flows
   • multi-agent conversation turns
   • MemorySaver checkpoint persistence
   • human-in-the-loop approval nodes
6. Guardrails, verification and agent evaluation2 hrs
   • infinite loop detection safeguards
   • action authorization checkpoints
   • deterministic output validation
   • agent trajectory benchmarking
   • sandbox execution security isolation

Hands-on project: Build an autonomous code refactoring agent team using LangGraph state graphs and MCP server tools that inspects a local git repository, runs tests, and opens pull requests.

You can generate and manipulate images using Stable Diffusion pipelines, ControlNet conditioning, and DreamBooth subject fine-tuning.

1. Generative image modelling and the diffusion process (DDPM)2 hrs
   • forward noise addition schedules
   • reverse denoising neural networks
   • U-Net noise prediction objectives
   • DDIM accelerated sampling steps
   • variance schedule parameterization
2. Latent diffusion and the Stable Diffusion architecture2 hrs
   • variational autoencoder latent space compression
   • text encoder conditioning with CLIP
   • classifier-free guidance scaling
   • cross-attention layers in image generation
   • UNet vs DiT transformer backbones
3. Text-to-image: prompts, negative prompts and guidance scale2 hrs
   • guidance scale parameter tuning
   • negative prompt token weighting
   • scheduler selection (Euler, DPM++)
   • prompt engineering for visual styles
   • seed reproducibility control
4. ControlNet, inpainting, outpainting and image LoRAs3 hrs
   • zero convolution weight locking
   • canny edge and depth map conditioning
   • masked latent inpainting pipelines
   • image outpainting canvas expansion
   • loading style LoRA weights
5. Customizing image models with DreamBooth2 hrs
   • rare token identifier association
   • class-specific prior preservation loss
   • subject-driven generation training
   • instance prompt formulation
   • preserving background generalization
6. Deploying image generation models for production2 hrs
   • TensorRT optimization for Stable Diffusion
   • memory efficient attention slicing
   • async generation queue architecture
   • image caching and S3 storage
   • API wrapper implementation in FastAPI

Hands-on project: Build a Python image generation service using the Diffusers library that applies ControlNet pose conditioning and generates custom subject variants via DreamBooth weights.

You can process images, transcribe audio with Whisper, and build multimodal AI applications that ingest visual and speech data.

1. Vision-language models: CLIP, LLaVA and multimodal LLMs2 hrs
   • image patch projection into embedding space
   • cross-attention between vision and text tokens
   • multimodal chat template structures
   • vision encoder quantization
   • patch grid resizing strategies
2. Image understanding and image-grounded generation2 hrs
   • visual question answering pipelines
   • bounding box coordinate extraction
   • chart and document OCR parsing
   • multi-image conversation turns
   • visual grounding evaluation
3. Speech-to-text with Whisper and text-to-speech systems2 hrs
   • audio spectrogram preprocessing
   • Whisper encoder-decoder transcription
   • neural vocoder waveform generation
   • timestamp alignment in transcription
   • streaming audio chunk processing
4. Music, audio and an overview of video generation2 hrs
   • latent audio diffusion models
   • temporal consistency in video generation
   • multimodal token stream synchronization
   • text-to-audio conditioning prompts
   • video frame interpolation models
5. Building end-to-end multimodal assistant applications2 hrs
   • handling mixed input payloads in FastAPI
   • multimodal chat history storage
   • streaming image and text responses
   • audio recording frontend integration
   • error handling for corrupt media files
6. Multimodal agent integration and tool calling2 hrs
   • passing images as tool arguments
   • speech-driven agent invocation loops
   • multimodal model function calling schemas
   • audio streaming token exchange protocols
   • latency optimization for vision pipelines

Hands-on project: Build an audio-to-text-to-action pipeline that transcribes meeting voice notes using Whisper and extracts action items into a JSON payload using a vision-language model.

You can implement LLM-as-a-judge evaluation frameworks, defend against prompt injection, and enforce safety guardrails on generative endpoints.

1. Evaluating generative outputs and LLM-as-a-judge2 hrs
   • pairwise comparison evaluation
   • rubric-based LLM scoring prompts
   • automated test dataset validation
   • position bias mitigation in judging
   • correlation with human raters
2. Hallucination, bias and toxicity mitigation2 hrs
   • fact-checking retrieved context
   • demographic bias testing
   • toxicity classifier integration
   • uncertainty estimation in generation
   • self-critique filtering loops
3. Guardrails, content moderation and PII handling2 hrs
   • Regex and NER PII redacting filters
   • NeMo Guardrails flow control
   • content safety classification models
   • input sanitization pipelines
   • output policy enforcement hooks
4. Prompt injection, jailbreaks and LLM security2 hrs
   • direct vs indirect prompt injection
   • delimiter isolation strategies
   • dual-model verification patterns
   • adversarial suffix attacks
   • jailbreak pattern detection regex
5. Responsible AI, data privacy and model licensing90 min
   • open-source license compliance (MIT, Apache 2.0, LLaMA community)
   • data provenance tracking
   • copyright and IP considerations
   • GDPR right to be forgotten in AI
   • ethical AI deployment checklists
6. Enterprise compliance, audit trails and governance90 min
   • audit logging for LLM interactions
   • SOC 2 compliance for AI systems
   • data residency and regional hosting
   • model card documentation standards
   • incident response for AI hallucinations

Hands-on project: Build a security middleware proxy in FastAPI that inspects incoming user prompts for jailbreak patterns and filters toxic model outputs using content moderation APIs.

You can serve an LLM feature within a latency and cost budget and show the numbers.

1. Measuring what a request costs90 min
   • Token accounting
   • Cost per thousand requests
   • Latency breakdown by stage
2. Making it feel fast90 min
   • Streaming and first-token latency
   • Speculative and parallel calls
   • Perceived vs actual latency
3. Spending less per answer90 min
   • Prompt and context trimming
   • Exact and semantic caching
   • Small-model routing
4. Operating within budget90 min
   • Per-user rate limits
   • Timeout and fallback chain
   • Alerting on cost spikes

Hands-on project: Define a p95 latency and cost-per-request budget for the capstone feature, instrument token usage, then meet the budget with streaming, caching, prompt trimming and routing simple requests to a smaller model, reporting before and after.

You can serve LLMs with vLLM, build asynchronous FastAPI inference microservices, monitor token usage, and deploy a production GenAI capstone to the cloud.

1. Serving LLMs efficiently with vLLM and TGI2 hrs
   • PagedAttention memory allocation in vLLM
   • continuous batching of inference requests
   • OpenAI-compatible REST API endpoints
   • tensor parallelism across multiple GPUs
   • throughput and latency benchmarking
2. Building GenAI APIs with FastAPI and streaming2 hrs
   • StreamingResponse generators in FastAPI
   • async request handling and concurrency
   • API authentication and token-bucket rate limiting
   • CORS and middleware configuration
   • Pydantic request validation models
3. Cost, latency, caching and token-usage optimization2 hrs
   • semantic caching layers with Redis
   • token usage tracking and cost attribution
   • quantized inference performance profiling
   • prompt caching strategies
4. Containerization and local orchestration with Docker2 hrs
   • multi-stage Dockerfiles for Python runtimes
   • GPU passthrough configuration in Docker
   • docker-compose multi-container setup
   • environment variable secret management
   • image size reduction techniques
5. Cloud deployment, monitoring and observability2 hrs
   • cloud GPU instance provisioning on AWS or GCP
   • Prometheus metrics scraping for LLM servers
   • Grafana dashboard setup for request latency
   • structured JSON logging in production
   • health check and liveness probe configuration
6. Capstone project build, code review and interview prep3 hrs
   • integrating all modules into a unified platform
   • end-to-end integration test suites
   • peer code review and refactoring
   • portfolio documentation and GitHub README
   • mock LLM engineering interview questions

Hands-on project: Containerise a complete multi-agent RAG application with vLLM inference and FastAPI using Docker, configure token-bucket rate limiting, and push the artifact to a cloud VM with automated health checks.