A vector is simply a list of numbers — but in ML it always represents something meaningful: a word embedding, a user profile, or a data sample. Think of it as a point in n-dimensional space, or a direction from the origin.

• Dot product: measures similarity between two vectors — foundation of attention, cosine similarity, linear regression
• Magnitude (L2 norm): how "long" a vector is — used in normalization and regularization
• Unit vector: vector scaled to length 1 — used whenever direction matters more than scale
• Column vs row vectors: knowing which orientation a framework expects saves hours of debugging

A matrix is a 2D grid of numbers. In ML, matrices represent datasets (rows = samples, cols = features), weight matrices in neural nets, and linear transformations that rotate/scale/project data.

• Matrix multiply: the core of every forward pass — output = W·x + b. Shape rules: (m×n)·(n×p) → (m×p)
• Transpose: flip rows and columns — critical for getting dot products right, especially in attention
• Identity matrix: the "do nothing" matrix — useful for proofs and understanding invertibility
• Inverse: A⁻¹ such that A·A⁻¹ = I — used conceptually in linear regression closed-form solution (θ = (XᵀX)⁻¹Xᵀy)
• Determinant: single number describing how a matrix scales space — zero determinant = non-invertible

For a matrix A, an eigenvector v is a special direction that doesn't change when transformed — it only gets scaled. That scale factor is the eigenvalue λ. Formally: Av = λv. You don't need to compute these by hand — you need to understand what they represent.

• Eigenvectors point in the "principal directions" of a transformation
• Larger eigenvalue = direction with more variance in the data
• PCA works by finding the eigenvectors of the covariance matrix — the top eigenvectors become your principal components
• Spectral decomposition: a symmetric matrix can be written as A = QΛQᵀ where columns of Q are eigenvectors

SVD generalizes eigendecomposition to any matrix (not just square ones). It breaks any matrix A into three parts: A = UΣVᵀ. Think of it as: find directions in input space (V), stretch/compress them (Σ), and map them to output space (U). The singular values in Σ tell you "how important" each direction is.

• Truncated SVD: keep only top-k singular values → low-rank approximation (compress the matrix)
• Recommendation systems classically use matrix factorization, which SVD underlies
• More numerically stable than computing eigenvalues directly — libraries prefer it
• Relation to PCA: PCA on centered data = truncated SVD on the data matrix

A derivative measures how much a function changes when its input changes. In ML, this tells us: "if I tweak this weight by a tiny amount, how much does the loss change?" A partial derivative ∂f/∂x does the same but holds all other variables constant.

• Derivative of f(x) = slope of f at point x — write it as f'(x) or df/dx
• Partial derivative ∂L/∂w: how much loss L changes when weight w changes, all else fixed
• Chain rule: d/dx[f(g(x))] = f'(g(x))·g'(x) — this is the mathematical engine of backprop
• Common derivatives to memorize: d/dx[xⁿ] = nxⁿ⁻¹, d/dx[eˣ] = eˣ, d/dx[ln x] = 1/x, d/dx[sigmoid] = σ(1-σ)

The gradient is the vector of all partial derivatives of a function — it points in the direction of steepest increase. Gradient descent flips it: move in the opposite direction of the gradient to minimize loss. Picture a ball rolling down a hill to find the lowest point.

• ∇L = [∂L/∂w₁, ∂L/∂w₂, ...] — one partial derivative per parameter
• Update rule: w ← w − η·∇L, where η is the learning rate
• Learning rate too high: overshoot, diverge. Too low: slow convergence
• Variants: SGD (one sample at a time), mini-batch SGD, Adam (adaptive learning rates per parameter)
• Adam maintains a running mean and variance of past gradients to smooth updates

Backprop is just the chain rule applied repeatedly through a computational graph. You compute the forward pass (prediction), compute the loss, then propagate gradients backward layer by layer — each layer's gradient depends on the next layer's gradient multiplied by the local derivative.

• Forward pass: compute activations layer by layer, store intermediate values
• Backward pass: starting from loss, compute ∂L/∂w for each weight using chain rule
• ∂L/∂w₁ = (∂L/∂output) × (∂output/∂hidden) × (∂hidden/∂w₁) — chain of multiplications
• Vanishing gradient: gradients shrink as they propagate through many sigmoid/tanh layers → use ReLU or residual connections
• Exploding gradient: gradients grow exponentially → use gradient clipping

A convex function has only one minimum — any bowl-shaped curve is convex. If your loss surface is convex, gradient descent is guaranteed to find the global minimum. Neural nets are non-convex, so you settle for good local minima. This distinction comes up constantly in interviews.

• Convex function: a line segment between any two points on the curve lies above or on the curve
• Global minimum: the single lowest point (only guaranteed for convex functions)
• Local minimum: lowest in the neighborhood — neural nets typically have many
• Saddle point: gradient is zero but it's not a minimum — gradient descent can get stuck
• Linear regression has a convex loss (MSE); neural networks do not

Probability assigns numbers between 0 and 1 to events. Three core concepts you must know fluently: joint probability (both events happen), conditional probability (one event given another), and marginal probability (one event regardless of others).

• P(A ∩ B) — joint: probability A and B both happen
• P(A|B) = P(A ∩ B) / P(B) — conditional: probability of A given B already happened
• P(A) = Σ P(A|B)·P(B) — marginal: sum over all possible values of B (law of total probability)
• Independence: P(A ∩ B) = P(A)·P(B) — knowing B gives no information about A
• Mutually exclusive vs independent — these are different and often confused in interviews

Bayes' theorem tells you how to update your belief about something when you get new evidence. It connects prior knowledge (what you believed before), the likelihood of the evidence, and the posterior (updated belief). It's the foundation of an entire approach to ML.

• P(H|E) = P(E|H)·P(H) / P(E) — posterior = likelihood × prior / evidence
• Prior P(H): belief before seeing data (e.g. 1% of emails are spam)
• Likelihood P(E|H): how probable is this evidence if H is true (e.g. "win" appears in spam 80% of the time)
• Posterior P(H|E): updated belief after seeing evidence — what the model outputs
• Naive Bayes classifier: assumes all features are independent given the class → P(class|features) ∝ P(class) × ∏P(featureᵢ|class)

Different types of data follow different distributions. Knowing which distribution to reach for — and what properties it has — is essential for model design, loss function choice, and understanding what your model is actually predicting.

• Gaussian (Normal): bell curve, defined by mean μ and variance σ². Many natural phenomena. Used in linear regression noise assumption
• Bernoulli: single binary event with probability p. Models a coin flip. Output of sigmoid in binary classification
• Categorical: extension of Bernoulli to K classes. Output of softmax in multi-class classification
• Uniform: all outcomes equally likely — used in random initialization and sampling
• Poisson: count of events in a fixed interval (e.g. requests per second) — relevant in NLP and event modeling

MLE (Maximum Likelihood Estimation) finds the parameters that make the observed data most probable. MAP (Maximum A Posteriori) is MLE plus a prior — it's a Bayesian extension. Both are frameworks that explain why common loss functions work the way they do.

• MLE: argmax_θ P(data|θ) — find θ that makes data most likely
• In practice: maximize log-likelihood (log is monotonic, converts products to sums, numerically stable)
• Linear regression + Gaussian noise → MLE gives you MSE loss
• Logistic regression + Bernoulli likelihood → MLE gives you cross-entropy loss
• MAP: argmax_θ P(θ|data) = P(data|θ)·P(θ) — adds prior. L2 regularization = MAP with Gaussian prior

Before any modelling, you need to describe and understand your data. These are the numbers interviewers expect you to reach for immediately when asked "how would you explore this dataset?"

• Mean: average value — sensitive to outliers. Median is more robust for skewed data
• Variance σ²: average squared deviation from mean — measures spread. Std deviation σ = √variance, in original units
• Covariance Cov(X,Y): how X and Y vary together. Positive → increase together; negative → one increases as the other decreases
• Covariance matrix: captures pairwise relationships between all features — central to PCA
• Skewness: asymmetry of distribution. Kurtosis: "tailedness" — how heavy the tails are

Every model error breaks into bias (wrong assumptions → underfitting) and variance (too sensitive to training data → overfitting) and irreducible noise. Reducing one tends to increase the other — the tradeoff is the core tension in model selection.

• High bias: model is too simple, misses real patterns. Example: linear model on non-linear data
• High variance: model memorizes training data, fails on new data. Example: deep tree with no pruning
• Total error = Bias² + Variance + Irreducible noise
• Regularization (L1/L2), dropout, early stopping — all tools to reduce variance at slight bias cost
• More data reduces variance but doesn't fix bias — a biased model needs rethinking, not more data

Hypothesis testing lets you decide whether an observed pattern is real or just noise. You define a null hypothesis (H₀: "no effect"), run an experiment, and check whether the result is statistically significant. This comes up heavily in A/B testing interviews.

• Null hypothesis H₀: your default assumption (e.g. "new model performs the same as old model")
• p-value: probability of seeing results this extreme if H₀ were true. Low p → evidence against H₀
• Significance level α (usually 0.05): threshold below which you reject H₀ — not a magic number
• Type I error (false positive): rejecting H₀ when it's true. Type II (false negative): failing to reject when it's false
• Confidence interval: range of values within which the true parameter likely falls with 95% confidence
• Statistical vs practical significance: a tiny p-value with huge data may not mean a meaningful difference

Correlation measures the linear relationship between two variables. It ranges from −1 to +1. Causation means one variable directly causes the other — much harder to establish. Confusing the two is one of the most common errors in applied ML and data science.

• Pearson r: measures linear correlation. r=1 → perfect positive linear, r=0 → no linear relation
• Spearman: rank-based correlation — works for monotonic non-linear relationships too
• Correlation does not imply causation: ice cream sales and drowning rates both rise in summer (confound: temperature)
• Confounders: hidden variables that cause both X and Y, creating a spurious correlation
• To establish causation: randomized controlled trials (RCT), or causal inference techniques (do-calculus, IV)