Deep Learning

You can calculate gradients, decode matrix operations, and trace the exact linear algebra powering neural networks.

1. Scalars, vectors, matrices and tensors2 hrs
   • ndarray shape and strides
   • tensor broadcasting rules
   • memory layout and contiguous arrays
   • np.ndim and np.squeeze operations
   • dtype casting and numerical precision
2. Matrix operations, dot products and matrix multiplication2.5 hrs
   • np.dot vs np.matmul vs @ operator
   • inner and outer products
   • matrix transposition and inversion
   • determinant calculation and rank
   • einsum notation for multi-axis contractions
3. Eigenvalues, eigenvectors and matrix decomposition2.5 hrs
   • np.linalg.eig
   • Principal Component Analysis math
   • singular value decomposition (SVD)
   • eigenvalue spectral radius
   • positive semi-definite matrices
4. Calculus: derivatives, partial derivatives and the chain rule3 hrs
   • scalar-by-vector differentiation
   • Jacobian matrices
   • chain rule for composite functions
   • total derivative calculation
   • numerical gradient approximation
5. Gradients, Jacobians and the intuition behind gradient descent2.5 hrs
   • convex vs non-convex loss surfaces
   • learning rate selection
   • local minima and saddle points
   • vanishing gradient symptoms
   • Hessian matrix curvature analysis
6. Probability, distributions, Bayes' theorem and Maximum Likelihood2.5 hrs
   • Gaussian distributions
   • Maximum Likelihood Estimation (MLE)
   • Bayes theorem in classification
   • Kullback-Leibler divergence math
   • entropy and cross-entropy derivation

Hands-on project: Write a pure Python vectorised backpropagation engine from scratch using NumPy and verify analytic gradients against finite-difference approximations with 10^-5 tolerance.

You can configure GPU environments on Google Colab, manipulate tensors with NumPy, and build initial pipelines using TensorFlow and PyTorch.

1. Python essentials for deep learning2.5 hrs
   • list comprehensions vs generators
   • object-oriented data iterators
   • memory profiling with memory_profiler
   • context managers for resource cleanup
   • type hinting with typing module
2. NumPy for fast tensor operations2.5 hrs
   • vectorised array indexing
   • axis-wise reductions
   • avoiding Python loops with vectorisation
   • np.where conditional assignment
   • memory views and slicing efficiency
3. Pandas for data handling and Matplotlib for visualization2.5 hrs
   • pd.DataFrame groupby and apply
   • Matplotlib subplots and loss curves
   • Seaborn confusion matrix heatmaps
   • missing value imputation strategies
   • vectorised string operations in Pandas
4. Jupyter, Google Colab and GPU environment setup2.5 hrs
   • nvidia-smi monitoring
   • Colab runtime disk mounting
   • CUDA device allocation in PyTorch
   • pip package caching and virtualenvs
   • jupyter magic commands like %timeit
5. Introduction to TensorFlow, Keras and PyTorch2.5 hrs
   • tf.Tensor and torch.Tensor syntax
   • autograd computation graphs
   • Keras Sequential API basics
   • tensor device transfer with .to('cuda')
   • inspecting gradients via tensor.grad
6. Writing custom PyTorch Datasets and DataLoaders2.5 hrs
   • torch.utils.data.Dataset subclassing
   • __getitem__ and __len__ implementation
   • custom collate_fn for variable padding
   • pin_memory and num_workers tuning
   • iterable dataset patterns for streaming data

Hands-on project: Build an exploratory data analysis pipeline using Pandas and NumPy to clean a 100k-row image metadata CSV, generating train/val splits and loading them efficiently via PyTorch DataLoader.

You can evaluate models using cross-validation, diagnose overfitting, and select appropriate loss functions for regression and classification tasks.

1. Supervised vs unsupervised learning2.5 hrs
   • regression vs classification targets
   • clustering algorithms overview
   • dimensionality reduction with t-SNE
   • semi-supervised learning paradigms
   • label noise handling techniques
2. Train/validation/test splits, overfitting and underfitting2.5 hrs
   • data leakage prevention
   • learning curves analysis
   • model capacity tuning
   • train_test_split stratification
   • cross-validation fold selection
3. The bias-variance trade-off2.5 hrs
   • bias squared vs variance decomposition
   • regularisation impact on variance
   • ensemble smoothing effects
   • complex model penalty tuning
   • double descent phenomenon overview
4. Loss functions and evaluation metrics2.5 hrs
   • Mean Squared Error vs Mean Absolute Error
   • binary cross-entropy formulation
   • F1-score and macro averaging
   • ROC curve and AUC interpretation
   • precision-recall tradeoffs in imbalance
5. Gradient descent variants: batch, stochastic and mini-batch2.5 hrs
   • SGD stability trade-offs
   • mini-batch sizing implications
   • epoch vs iteration calculations
   • learning rate decay schedules
   • momentum accumulation mechanics
6. Evaluating classifiers on imbalanced datasets2.5 hrs
   • SMOTE oversampling pipeline
   • class weight adjustment in loss
   • Cohen's Kappa coefficient
   • balanced accuracy metric
   • cost-sensitive training loops

Hands-on project: Train a baseline scikit-learn classifier on tabular sensor data, execute stratified K-fold cross-validation, and log evaluation metrics including precision, recall, and ROC-AUC.

You can construct multilayer perceptrons from scratch, implement forward and backward propagation, and configure activation functions.

1. The perceptron and its biological inspiration3.5 hrs
   • McCulloch-Pitts neuron model
   • decision boundary geometry
   • XOR problem and multi-layer necessity
   • step activation function limits
   • perceptron convergence theorem
2. Activation functions: sigmoid, tanh, ReLU, Leaky ReLU, softmax4 hrs
   • derivative formulas for activations
   • dying ReLU problem
   • numerical stability in softmax
   • GELU and SiLU modern activations
   • monotonicity and range constraints
3. Loss functions: MSE and cross-entropy3.5 hrs
   • log-likelihood connection
   • gradient combination simplification
   • multi-class cross-entropy implementation
   • Huber loss for robust regression
   • binary focal loss for hard negatives
4. Backpropagation and the maths behind it4 hrs
   • computation graph traversal
   • local vs global gradients
   • weight gradient matrix equations
   • jacobian vector products
   • gradient accumulation across batches
5. Weight initialization strategies3.5 hrs
   • Xavier / Glorot normal init
   • He initialization for ReLU networks
   • zero initialization pitfalls
   • orthogonal weight initialization
   • variance preservation across layers
6. Building your first neural network from scratch3.5 hrs
   • forward pass matrix chaining
   • backward pass delta propagation
   • parameter update loop in Python
   • numerical gradient check assertions
   • mini-batch training loop structuring

Hands-on project: Implement a 3-layer feedforward neural network from scratch in NumPy with configurable activation functions, achieving over 95 percent accuracy on the MNIST digits dataset without deep learning frameworks.

You can apply modern optimizers, batch normalisation, and regularisation techniques to stabilise and accelerate deep network convergence.

1. Optimizers: SGD, Momentum, RMSProp and Adam4 hrs
   • exponentially weighted averages
   • first and second moment estimates
   • AdamW weight decay fix
   • AdaGrad historical gradient scaling
   • Nesterov accelerated gradient update
2. Learning rate scheduling and warm-up3.5 hrs
   • cosine annealing schedule
   • ReduceLROnPlateau callback
   • linear warm-up phases
   • exponential decay configuration
   • cyclic learning rate policies
3. Regularization: L1, L2, dropout and early stopping4 hrs
   • weight decay penalty calculation
   • nn.Dropout probability scaling
   • EarlyStopping patience and delta config
   • stochastic depth regularisation
   • label smoothing regularisation
4. Batch normalization and layer normalization4 hrs
   • running mean and variance tracking
   • training vs inference mode behavior
   • normalization placement in residual blocks
   • RMSNorm for efficient transformer scaling
   • group normalization for small batches
5. Vanishing and exploding gradients3.5 hrs
   • gradient clipping with torch.nn.utils.clip_grad_norm_
   • residual skip connections
   • activation saturation diagnosis
   • gradient norm monitoring
   • mixed precision overflow avoidance
6. Hyperparameter tuning and experiment tracking2.5 hrs
   • Optuna hyperparameter sweeps
   • Weights & Biases logging integration
   • reproducibility with torch.manual_seed
   • hyperband pruning algorithm
   • parallel trial configuration

Hands-on project: Build an experimental tracking script in PyTorch that compares Adam, RMSProp, and SGD with momentum on a deep classifier, using TensorBoard to log learning rates, weight norms, and validation loss.

You can build, fine-tune, and deploy convolutional and object-detection architectures for advanced computer vision applications.

1. The convolution operation, filters and feature maps3 hrs
   • kernel size, stride and padding formulas
   • receptive field growth calculation
   • channel depth transformations
   • dilated convolutions for wider context
   • separable convolutions for efficiency
2. Pooling layers: max and average pooling3 hrs
   • spatial dimension reduction
   • translation invariance properties
   • global average pooling layers
   • fractional max pooling
   • adaptive pooling output sizing
3. Classic architectures: LeNet, AlexNet, VGG, ResNet, Inception4 hrs
   • VGG uniform block design
   • ResNet identity shortcut connections
   • Inception multi-scale filter concatenation
   • DenseNet feature reuse connections
   • MobileNet depthwise separable blocks
4. Transfer learning and fine-tuning pretrained models4 hrs
   • freezing backbone layers
   • replacing classification heads
   • torchvision.models pretrained weights loading
   • discriminative learning rates per layer
   • unfreezing schedule strategies
5. Image classification project3 hrs
   • Albumentations data augmentation pipeline
   • CrossEntropyLoss label smoothing
   • top-k accuracy evaluation metrics
   • Mixup and CutMix data regularisation
   • test-time augmentation (TTA)
6. Object detection: R-CNN family and YOLO3 hrs
   • bounding box regression targets
   • Intersection over Union (IoU) calculation
   • Non-Max Suppression (NMS)
   • anchor box generation logic
   • mean Average Precision (mAP) metric
7. Image segmentation with U-Net2 hrs
   • encoder-decoder skip connections
   • pixel-wise cross-entropy loss
   • Dice coefficient evaluation
   • transposed convolutions for upsampling
   • patch-based inference handling

Hands-on project: Fine-tune a pretrained ResNet50 model using PyTorch on a custom multi-class image dataset, achieving over 92% validation accuracy with data augmentation and mixed-precision training.

You can model sequential time-series and text data using Recurrent Neural Networks, LSTMs, and gated units.

1. Sequential data and time-series problems2.5 hrs
   • sliding window sequence generation
   • autoregressive forecasting setup
   • stationarity in sequential data
   • lag feature engineering
   • temporal train-test split splits
2. Recurrent Neural Networks (RNNs) and their limitations2.5 hrs
   • hidden state recurrence equations
   • Backpropagation Through Time (BPTT)
   • exploding gradients in long sequences
   • vanishing gradients across time steps
   • nn.RNN module shape handling
3. Long Short-Term Memory (LSTM) networks3 hrs
   • forget, input and output gates
   • cell state conveyor belt
   • nn.LSTM module configuration
   • hidden and cell state initialisation
   • gradient flow in LSTM cells
4. Gated Recurrent Units (GRUs)2.5 hrs
   • reset and update gates
   • parameter efficiency vs LSTMs
   • nn.GRU usage patterns
   • comparison of training convergence speed
   • when to select GRU over LSTM
5. Bidirectional and stacked RNNs2.5 hrs
   • bidirectional hidden state concatenation
   • stacked multi-layer recurrence
   • dropout application in RNNs
   • bidirectional sequence masking
   • multilayer hidden tensor dimensions
6. Sequence-to-sequence modelling2 hrs
   • encoder-decoder architectural split
   • context vector bottleneck
   • teacher forcing during training
   • scheduled sampling technique
   • sequence evaluation with BLEU score

Hands-on project: Train an LSTM network in PyTorch to predict multi-step financial time-series stock prices, handling sequence padding and evaluating root mean squared error on test splits.

You can preprocess text, generate dense word embeddings, and implement attention mechanisms for natural language processing tasks.

1. Text preprocessing and tokenization2.5 hrs
   • subword tokenization algorithms
   • vocabulary indexing and padding
   • torchtext data pipelines
   • lowerercasing and punctuation stripping
   • special tokens like <pad> and <unk>
2. Word embeddings: Word2Vec and GloVe2.5 hrs
   • Skip-gram vs CBOW architectures
   • cosine similarity in vector space
   • loading pretrained embedding matrices
   • freezing vs fine-tuning embedding layers
   • handling out-of-vocabulary words
3. The attention mechanism explained3 hrs
   • Bahdanau additive attention
   • Luong multiplicative attention
   • attention weight visualization heatmaps
   • alignment score calculation
   • context vector weighting
4. Encoder-decoder architectures2.5 hrs
   • sequence alignment handling
   • decoder hidden state propagation
   • beam search decoding strategy
   • length penalty in beam search
   • greedy decoding vs sampling
5. Sentiment analysis and text classification project2.5 hrs
   • embedding layer freezing
   • packed sequence utility for variable lengths
   • classification head design
   • nn.Embedding padding_idx configuration
   • evaluation across test folds
6. Evaluating text generation and classification2 hrs
   • token-level accuracy metrics
   • perplexity calculation for language models
   • ROUGE score for summarisation
   • precision and recall in tagging tasks
   • confusion matrix for multi-class NLP

Hands-on project: Build a sentiment analysis classifier using PyTorch embedding layers and bidirectional LSTMs on a customer review dataset, achieving over 89% test set accuracy.

You can implement self-attention, fine-tune transformer models with Hugging Face, and build applications powered by LLMs.

1. The Transformer architecture and self-attention4 hrs
   • scaled dot-product attention formula
   • Query, Key, and Value projections
   • attention score matrix calculations
   • causal masking for autoregressive models
   • memory complexity of self-attention
2. Multi-head attention and positional encoding4 hrs
   • concatenating multiple attention heads
   • sinusoidal positional encodings
   • learned positional embeddings
   • RoPE (Rotary Position Embedding) overview
   • head dimension partitioning
3. BERT, GPT and the modern LLM landscape3.5 hrs
   • encoder-only vs decoder-only differences
   • masked language modeling objective
   • causal autoregressive generation
   • sequence-to-sequence T5 architecture
   • mixture of experts (MoE) routing
4. Fine-tune pretrained transformers with Hugging Face4 hrs
   • transformers.AutoModelForSequenceClassification
   • Trainer API and TrainingArguments
   • peft and LoRA parameter-efficient tuning
   • bitsandbytes 4-bit quantization setup
   • datasets library map and filter methods
5. Prompt engineering fundamentals3.5 hrs
   • few-shot prompting patterns
   • system vs user instruction tuning
   • temperature and top-p sampling control
   • chain-of-thought reasoning prompts
   • guardrails and output validation
6. Building applications powered by LLMs3 hrs
   • LangChain LLM wrapper integration
   • structured output parsing with Pydantic
   • prompt template management
   • retrieval-augmented generation (RAG) wiring
   • token cost tracking and budget limits

Hands-on project: Fine-tune a BERT model using Hugging Face Transformers and PyTorch for multi-label text classification on an enterprise support ticket dataset, and save the model artifact.

You can design and train generative models including variational autoencoders, generative adversarial networks, and diffusion pipelines.

1. Autoencoders and Variational Autoencoders (VAEs)3 hrs
   • encoder-decoder bottlenecks
   • reparameterization trick
   • Kullback-Leibler divergence loss
   • latent space interpolation
   • sparse autoencoder regularisation
2. Generative Adversarial Networks (GANs)3 hrs
   • minimax game objective
   • DCGAN architectural guidelines
   • mode collapse diagnosis
   • Wasserstein GAN gradient penalty
   • spectral normalization in discriminators
3. Introduction to diffusion models3 hrs
   • forward noise addition process
   • reverse denoising network U-Net
   • DDPM sampling algorithm
   • classifier-free guidance scaling
   • latent diffusion model architecture
4. Image generation project2.5 hrs
   • training loop balancing in GANs
   • fid score evaluation metric
   • saving intermediate generated epochs
   • inception score calculation
   • conditional image generation conditioning
5. Evaluating generative model fidelity2 hrs
   • precision and recall for distributions
   • Fréchet Inception Distance (FID)
   • Kernel Inception Distance (KID)
   • visual artifact inspection methods
   • mode drop detection strategies
6. Advanced generative workflows1.5 hrs
   • text-to-image pipeline integration
   • controlnet spatial conditioning
   • LoRA adapters for image models
   • prompt weighting and negative prompts
   • inference optimization with TensorRT

Hands-on project: Train a Deep Convolutional GAN (DCGAN) on a custom image dataset using PyTorch, monitoring generator and discriminator loss curves and generating synthetic sample grids.

You can serialise model weights, build serving APIs with FastAPI, containerise with Docker, and track experiments with MLflow.

1. Saving, serializing and versioning models2.5 hrs
   • torch.save state_dict best practices
   • ONNX export format conversion
   • model registry versioning
   • TorchScript graph tracing
   • safetensors format security benefits
2. Serving models with Flask, FastAPI and TensorFlow Serving2.5 hrs
   • FastAPI async request validation with Pydantic
   • batching inference requests
   • handling CPU/GPU memory in production
   • health check and readiness endpoints
   • uvicorn worker process scaling
3. Containerizing models with Docker2.5 hrs
   • multi-stage builds for small images
   • nvidia-container-toolkit setup
   • environment variable configuration
   • base image selection with CUDA runtimes
   • volume mounting for model weight caching
4. Experiment tracking and monitoring with MLflow2.5 hrs
   • mlflow.log_param and log_metric
   • artifact store configuration
   • model signature logging
   • tracking server database backend setup
   • model promotion workflows in registry
5. Deploying to the cloud (AWS / GCP)2.5 hrs
   • container deployment on AWS ECS
   • cloud GPU instance provisioning
   • health check endpoint configuration
   • autoscaling rules based on GPU memory
   • load balancer configuration for inference
6. Model optimization: quantization and pruning for edge devices2.5 hrs
   • Post-Training Quantization (PTQ)
   • weight magnitude pruning
   • latency benchmarking on CPU
   • INT8 precision conversion workflows
   • pruned model sparsity measurement

Hands-on project: Containerise a fine-tuned PyTorch image classification model inside a multi-stage Dockerfile, expose it via a FastAPI prediction endpoint, and track model metrics with MLflow.

You can train a model that does not fit comfortably on one GPU and explain the trade-offs you took.

1. Why training runs out of memory90 min
   • Activations vs parameters
   • Batch size and memory
   • Profiling GPU memory
2. Mixed precision and checkpointing90 min
   • fp16 and bf16
   • Loss scaling
   • Gradient checkpointing cost
3. Data and model parallelism90 min
   • DistributedDataParallel
   • Gradient accumulation
   • Sharded optimisers
4. Keeping runs reproducible90 min
   • Seeding and determinism
   • Experiment tracking
   • Checkpoint and resume

Hands-on project: Take a training run near the memory limit and make it fit and finish faster using mixed precision, gradient accumulation and gradient checkpointing, then scale it across devices with DDP, reporting throughput and final accuracy against the baseline.

You can architect, build, and deploy an end-to-end deep learning system, interpret research papers, and clear technical hiring loops.

1. End-to-end deep learning capstone, deployed live3 hrs
   • end-to-end system architecture design
   • integrating model inference with web APIs
   • live cloud deployment and validation
   • monitoring error rates and latency
   • handling graceful degradation under load
2. Reading and implementing research papers3 hrs
   • decoding network architecture diagrams
   • translating math notations into PyTorch code
   • reproducing baseline paper results
   • identifying core novel contributions
   • debugging shape mismatches from papers
3. Kaggle competitions and leaderboard practice3 hrs
   • exploratory data strategy for competitions
   • ensemble modeling and cross-validation
   • submission file generation and validation
   • leakage detection in tabular and image data
   • post-processing prediction thresholds
4. Portfolio, GitHub and Kaggle profile building3 hrs
   • writing professional README files
   • structuring GitHub repositories for recruiters
   • showcasing Kaggle tier achievements
   • documenting reproducibility steps
   • hosting live inference demos
5. Mock interviews, DSA refresh and HR rounds3 hrs
   • deep learning system design interview
   • algorithmic coding preparation for ML roles
   • behavioral interview structuring
   • salary negotiation and offer evaluation
   • mock interview feedback review
6. Career transition planning and portfolio defense3 hrs
   • resume positioning for AI roles
   • defending model design choices in interviews
   • handling live coding debugging tests
   • open-source contribution strategy
   • networking in the machine learning community

Hands-on project: Build, containerize, and deploy a full-stack deep learning application combining a Transformer NLP model and a CNN vision classifier, integrated with a FastAPI backend and a cloud deployment.