additive noise
Additive noise is random error or disturbance that enters a model by being added to the underlying quantity.
Additive noise is a random term added to a signal, measurement, or model output. It represents variability that shifts the observed value up or down.
It is a standard way to model measurement error and uncertainty in statistical and signal-processing problems. Many derivations in the lecture assume observations are corrupted by additive noise.
In this lecture, additive noise appears in the observation model for measurements and parameters. It helps explain why the observed data differ from the underlying quantity of interest.
status: candidate · lectures: 1
alternative hypothesis
alternative hypotheses
An alternative hypothesis is a competing statement about the unknown quantity or model being tested against the null hypothesis.
An alternative hypothesis is a statement that represents a possible departure from the null hypothesis. In this lecture, it refers to one of a finite set of candidate explanations or parameter values.
It defines the possibilities that Bayesian inference compares when updating beliefs from data. The posterior distribution is used to assess which alternative is more plausible.
The lecture uses the term when describing a finite, usually small, number of possible hypotheses to consider. These alternatives form the set over which inference is carried out.
status: candidate · lectures: 1
Bayes' rule / Bayes rule
Bayes rule, Bayes' rule
Bayes' rule is the probability identity used to update a prior with observed data to obtain a posterior distribution.
Bayes' rule is a probability identity that relates a posterior quantity to a prior quantity and a likelihood term. It is the basic updating rule used throughout Bayesian inference.
It is the mechanism that turns prior beliefs and observed data into a posterior distribution. This makes it central to estimation, prediction, and hypothesis comparison.
status: candidate · lectures: 2
Bayesian estimation problem
A Bayesian estimation problem is an estimation task where the unknown parameter is modeled probabilistically and updated using observed data.
A Bayesian estimation problem is a statistical problem in which an unknown parameter is represented by a prior distribution and then updated after observing data. The goal is to infer the parameter using the posterior distribution.
It provides the framework for Bayesian decision-making and estimator construction. The choice of estimator depends on the posterior and the loss or error criterion.
The lecture introduces this phrase when moving from general Bayesian inference to parameter estimation. It frames the later discussion of posterior-based estimators and error measures.
status: candidate · lectures: 1
Bayesian inference
Bayesian inference framework, Bayesian inference methodology, Bayesian inference problem
Bayesian inference is a framework for updating beliefs about unknown quantities by combining prior information with observed data.
Bayesian inference is a statistical approach in which unknown quantities are described probabilistically and updated after data are observed. The result is a posterior distribution that summarizes the updated uncertainty.
It provides the main framework for deriving posterior distributions and estimators. It also supports reasoning and decision-making under uncertainty.
status: candidate · lectures: 2
beta distribution
Beta distributions, beta family
The beta distribution is a probability distribution on the interval from 0 to 1, often used to model uncertainty about a probability.
The beta distribution is a family of continuous probability distributions defined on the interval from 0 to 1. Its shape can vary widely, which makes it useful for modeling probabilities.
It is a common prior distribution in Bayesian inference when the unknown parameter is a probability. Its flexibility makes posterior updating especially convenient in simple binary-data examples.
The lecture identifies the posterior in the example as a beta distribution and refers to the beta family. This appears in the discussion of the updated distribution for the unknown parameter.
status: candidate · lectures: 1
binomial probability
binomial probabilities
Binomial probability is the probability of a particular number of successes in a fixed number of independent Bernoulli trials.
Binomial probability gives the chance of observing exactly k successes in n independent trials with the same success probability. It is the basic probability model for counts of heads in repeated coin flips.
It is the likelihood component in the lecture’s example and helps connect observed counts to the unknown parameter. It also underlies the posterior update in the binomial-beta setting.
The lecture uses binomial probabilities when describing the probability of getting k heads. This is part of the example used to build the Bayesian update.
status: candidate · lectures: 1
candidate models
Candidate models are the possible models under consideration in a model-selection or inference problem.
Candidate models are the alternative models or hypotheses that are being compared using data. One of them is eventually favored based on the chosen criterion.
They define the options among which inference or selection is made. Understanding them is necessary for interpreting how data support one model over another.
The lecture uses the phrase when describing the decision to favor one model over another. It appears in the setup for Bayesian comparison of alternatives.
status: candidate · lectures: 1
completing the square
Completing the square is an algebraic method for rewriting a quadratic expression in a more useful form.
Completing the square is a manipulation that rewrites a quadratic expression as a squared term plus or minus a constant. It is often used to simplify optimization and probability calculations.
It is useful for finding maxima, minima, and Gaussian forms in derivations. In estimation problems, it often reveals the value that optimizes a quadratic expression.
The lecture names completing the square as the trick used in the derivation. It appears as a step in simplifying the expression for the estimator or posterior form.
status: candidate · lectures: 1
conditional density
conditional density of X, conditional density of X given Theta, conditional distribution of X given Theta, conditional PDF of Xi
A conditional density is the density of a continuous random variable after conditioning on another variable.
A conditional density describes how a continuous random variable is distributed after conditioning on another variable. It is the continuous analogue of a conditional probability mass function.
It is needed when the unknown parameter or observed variable is continuous. In Bayesian inference, it appears in posterior calculations and conditional distributions.
status: candidate · lectures: 2
conditional distribution
conditional distribution of X
A conditional distribution is the distribution of a random variable given the value of another variable or some observed information.
A conditional distribution describes the probability law of one random variable after conditioning on another variable or event. It can be discrete or continuous depending on the setting.
It is a basic building block for Bayesian updating and for describing posterior uncertainty. Many estimators in the lecture are defined through conditional distributions.
The lecture refers to the conditional distribution of X and of the unknown parameter Theta. This is used when moving from general probability statements to posterior calculations.
status: candidate · lectures: 1
conditional expectation
expected value of theta given X
Conditional expectation is the expected value of a random variable given observed data or another conditioning variable.
Conditional expectation is the average value of a random variable after conditioning on known information. In Bayesian settings, it is often the posterior mean.
It is a common estimator because it summarizes the posterior distribution with a single number. The lecture uses it as one of the main Bayesian estimates.
The lecture explicitly describes it as the expected value of theta given X and uses it as an estimator. It appears alongside the MAP estimate in the comparison of Bayesian estimators.
status: candidate · lectures: 1
conditional expectation estimator / conditional expectation estimate
conditional expectation estimate, conditional expectation estimator
A conditional expectation estimator is an estimator obtained by taking the expected value of the unknown parameter given the data.
A conditional expectation estimator is a rule that estimates an unknown quantity by its conditional expectation given the observed data. In Bayesian inference, this is typically the posterior mean.
It is a standard Bayesian estimator and often serves as a benchmark for comparing other estimators. Its performance is evaluated using the posterior distribution and a loss criterion.
The lecture names this estimator when comparing it with the MAP estimate. It is presented as one of the main Bayesian point estimates for the unknown parameter.
status: candidate · lectures: 1
conditional mean squared error
Conditional mean squared error is the expected squared estimation error computed after conditioning on observed data or other information.
Conditional mean squared error is the expected value of the squared difference between an estimator and the unknown quantity, computed conditionally on observed information. It measures error after conditioning on the data.
It provides a way to assess estimator performance when uncertainty is updated by data. In Bayesian analysis, it helps compare estimators under the posterior distribution.
status: candidate · lectures: 2
conditional probability of error
conditional probabilities of error
Conditional probability of error is the probability of an incorrect decision or estimate given the information available.
Conditional probability of error is the probability that a decision or estimate is wrong after conditioning on the observed information. It measures error under the current data and model assumptions.
It is a natural criterion for evaluating decision rules and estimators. In the lecture, it helps connect Bayesian updating with performance assessment.
The lecture defines it as the probability of making an error given the information available. It is used in the discussion of how to judge an estimator or decision rule.
status: candidate · lectures: 1
conditional variance
Conditional variance is the variance of a random variable given another variable or observed data.
Conditional variance measures the spread of a random variable after conditioning on known information. It is the variance counterpart of conditional expectation.
It quantifies uncertainty that remains after observing data. In estimation problems, it often appears in formulas for mean squared error and posterior spread.
The lecture uses conditional variance while deriving the estimator and comparing error expressions. It is treated as a quantity that can be computed from the conditional distribution.
status: candidate · lectures: 1
confidence interval / confidence intervals
confidence interval, confidence intervals, interval
A confidence interval is an interval estimate for an unknown parameter, paired with a specified coverage level.
A confidence interval is a range of values computed from data and intended to estimate an unknown parameter. The stated confidence level describes the long-run coverage behavior of the construction.
It is a standard way to report uncertainty about an estimated quantity. The lecture uses it as a familiar comparison point when discussing probabilistic intervals.
The lecture mentions confidence intervals while discussing how much probability or coverage lies inside an interval. It serves as a contrast to the Bayesian quantities derived earlier.
status: candidate · lectures: 1
continuous random variable / continuous random variables
continuous random variable, continuous random variables
A continuous random variable is a random variable whose possible values form a continuum and are described by a density.
A continuous random variable can take any value in an interval or other continuous set. Its behavior is described by a probability density function rather than by probabilities at individual points.
Continuous random variables are the setting for densities, integrals, and continuous versions of Bayes' rule. The lecture relies on this framework throughout the derivation.
The lecture explicitly switches to the case of continuous random variables when writing Bayes' rule and conditional densities. This allows the model to be expressed with PDFs and integrals.
status: candidate · lectures: 1
distance function
A distance function is a measure of discrepancy between two values, used to quantify estimation error.
A distance function assigns a nonnegative measure of separation between two values or objects. In estimation, it is used to quantify how far an estimator is from the unknown quantity.
It determines how estimation error is measured and therefore influences which estimator is preferred. Different distance functions can lead to different optimal rules.
The lecture introduces it when defining the performance objective for estimation problems. It serves as the error measure underlying the comparison of estimators.
status: candidate · lectures: 1
distribution of the vector of measurements given Theta
The distribution of the vector of measurements given Theta is the conditional distribution of the observed measurements for a fixed parameter value.
This is the probability model for the measurement vector after conditioning on the unknown parameter Theta. It describes how the observed data are distributed when Theta is treated as fixed.
It is the likelihood model used to connect the unknown parameter to the observed data. The lecture needs it to write Bayes' rule and derive the posterior.
The lecture introduces this distribution as part of the observation model for the vector of measurements. It is the starting point for the Bayesian derivation that follows.
status: candidate · lectures: 1
estimation problem
estimation problems
An estimation problem is a statistical problem of inferring an unknown quantity from data.
An estimation problem is a statistical task in which the goal is to infer an unknown numerical quantity from observed data. The unknown quantity may be a parameter, mean, probability, or other value.
It is the setting in which estimators, loss functions, and error measures are defined. It also provides a contrast with hypothesis testing and model selection.
status: candidate · lectures: 2
estimator
estimators
An estimator is a rule that uses observed data to produce an estimate of an unknown quantity.
An estimator is a function or rule that maps observed data to an estimated value of an unknown quantity. It can be a point estimate or part of a broader decision rule.
Estimators are the main objects compared in estimation problems, and their quality is judged by an error criterion. Bayesian inference often defines estimators from the posterior distribution.
status: candidate · lectures: 2
expected squared error / expected value of the squared error
expected squared error, expected value of the squared error
Expected squared error is the expected value of the squared difference between an estimator and the true quantity.
Expected squared error is the average of the squared difference between an estimator and the unknown quantity under the relevant probability model. It is a standard loss measure in estimation.
It quantifies estimator accuracy in a way that penalizes larger errors more strongly. In Bayesian settings, it is often used to compare estimators under the posterior distribution.
The lecture introduces it as the expected value of the squared error when discussing performance criteria for estimators. It provides the basis for later conditional error measures.
status: candidate · lectures: 1
expected value
Expected value is the probability-weighted average of a random variable.
Expected value is the average outcome of a random variable under its distribution. It summarizes the center of the distribution in a single number.
It is a basic quantity in probability and statistics and appears in definitions of estimators and error measures. The lecture uses it in the derivation of the least mean squares estimator.
The lecture notes that the expected value is the same as the least mean squares estimator in the relevant setting. It appears in the discussion of conditional expectation.
status: candidate · lectures: 1
genomic data
Genomic data are data derived from genomes, such as DNA sequences or related biological measurements.
Genomic data are measurements or sequences obtained from genomes and related biological sources. They are often large, complex, and high-dimensional.
They provide a motivating example of data that may require statistical modeling and inference. In the lecture, they illustrate the kind of large data set that can be processed statistically.
The lecture mentions genomic data as an example of large-scale data to be processed. It serves as a motivating application for statistical inference methods.
status: candidate · lectures: 1
hypothesis testing
hypothesis testing problems, hypothesis-testing context
Hypothesis testing is the process of using data to decide between competing hypotheses, often by controlling the chance of error.
Hypothesis testing is a statistical framework for making a decision between alternative statements about a parameter, model, or system based on observed data.
It is one of the two main problem types in inference discussed in the lecture, and it focuses on making decisions with a small probability of error.
In this lecture, hypothesis testing is presented as a core inference problem class alongside estimation. The discussion emphasizes choosing a rule that minimizes the overall probability of error.
status: candidate · lectures: 1
independence / independence assumptions
independence, independence assumptions, independent
Independence means that random variables do not influence each other's distributions.
Independence is a property of random variables in which the joint distribution factors into separate distributions. Knowing one variable gives no information about the others under the model.
It simplifies probability calculations and is often assumed in measurement and noise models. The lecture uses independence to factor joint densities and justify the model structure.
The lecture states that Theta and W are independent and that the random variables involved are independent of each other. This assumption is used when writing the joint PDF as a product.
status: candidate · lectures: 1
independent normal random variables
Independent normal random variables are Gaussian random variables that are independent of one another.
These are random variables that each follow a normal distribution and are independent across variables. Their joint density is the product of the individual normal densities.
This assumption makes many derivations tractable and is common in linear estimation models. It allows the lecture to express the observation model in a simple probabilistic form.
The lecture describes the basic variables in the model as independent normal random variables. This supports the later use of Bayes' rule and factorized joint densities.
status: candidate · lectures: 1
inference
Inference is the use of data and a model to draw conclusions about unknown parameters, states, or hypotheses.
Inference is the general statistical process of learning about unknown quantities from observed data. It includes methods for estimation, testing, and related decision problems.
It is the broad theme of the lecture and the field in which the main problem types are defined. Understanding inference helps organize the course’s methods and goals.
The lecture frames its topic as a study of inference and then narrows to specific inference methods and problem types. It uses inference as the umbrella term for the course’s decision and estimation tasks.
status: candidate · lectures: 1
inference methods
Inference methods are statistical procedures for learning about unknown quantities from observed data.
Inference methods are the techniques used to extract conclusions from data under a statistical model. They include procedures for estimation, hypothesis testing, and related tasks.
They provide the practical tools for turning observations into decisions or numerical estimates. The lecture uses them as the broader category containing the main problem types.
The lecture refers to its discussion as a study of inference methods and contrasts them with the specific problem types that appear in the field. The phrase is used as a general label for the procedures being developed.
status: candidate · lectures: 1
joint PDF / joint density
joint density, joint PDF, joint PDF of the vector of thetas, joint probability density function
Joint PDF means joint probability density function, the density of several random variables taken together.
A joint PDF describes the combined density of two or more continuous random variables. It gives the density of the variables as a group rather than separately.
It is used to model dependence or independence among variables and to derive marginal and conditional densities. The lecture uses it in the Bayesian factorization of the model.
The lecture refers to the joint PDF of the measurements and to the joint density of the Theta parameters. It also notes when the joint PDF factors into a product of individual PDFs.
status: candidate · lectures: 1
linear function / linear function of the measurements
linear function, linear function of the measurements, linear function of the observations, linear function of the Xi's, linear functions, linear functions of the observations
A linear function is a function formed from a linear combination of variables, often used to build simple estimators.
A linear function combines variables using multiplication by constants and addition. In estimation, it often means a weighted sum of measurements, sometimes with an added constant.
Linear functions are easy to analyze and are common in estimation and signal models. The lecture uses them when discussing simple estimators built from observations.
The lecture refers to linear functions of the measurements, observations, and Xi's as candidate estimator forms. These appear in the discussion of how the estimator processes the data.
status: candidate · lectures: 1
linear functions of normal random variables
Linear functions of normal random variables are weighted sums or affine combinations of Gaussian variables.
These are expressions formed by taking linear combinations of normal random variables, possibly with a constant term. Under standard assumptions, such combinations are themselves normal.
They are important because many estimation models are built from Gaussian variables and linear transformations. This makes the resulting distributions easier to analyze.
The lecture mentions linear functions of normal random variables in the context of the model setup and estimator form. This supports the Gaussian derivations used later.
status: candidate · lectures: 1
linear model
A linear model expresses an outcome as a linear combination of predictors and unknown parameters, usually with random error.
A linear model is a statistical model in which the mean or expected value of a variable is written as a linear function of unknown parameters and observed inputs. In this lecture, it is the starting point for the Bayesian setup.
It provides a simple framework for describing relationships between variables and for deriving estimators. Many later results in the lecture are built from this model form.
The lecture begins by saying one starts with a linear model before adding normality assumptions for the parameters and noise. It serves as the basic structure for the estimation problem discussed here.
status: candidate · lectures: 1
linear normal models
Linear normal models are statistical models with a linear form and Gaussian noise or errors.
Linear normal models are models in which the observed quantity is expressed as a linear function of unknown parameters plus normally distributed randomness. They are a common setting for estimation and inference.
They provide a tractable example where posterior and estimator formulas can often be written explicitly. This makes them useful for illustrating core inference ideas.
status: candidate · lectures: 2
linear regression
Linear regression is a linear model used to describe how a response variable depends on one or more predictors.
Linear regression is a statistical method that models a response as a linear function of explanatory variables plus an error term. It is one of the most common forms of linear modeling.
It is a basic tool for estimation and prediction, and it provides a familiar example of the broader linear model class. The lecture uses the term as the common name for the model under discussion.
The lecture notes that the model goes under the name of linear regression. Here it is treated as the standard label for the linear setup being analyzed.
status: candidate · lectures: 1
linear relation
linear relations
A linear relation is a relationship between variables that can be written as a linear expression in the unknowns.
A linear relation is a dependence between variables that is expressed as a sum of terms with coefficients, rather than through powers, products, or other nonlinear forms. In this lecture, it describes the structural form of the model.
Recognizing linear relations helps identify when a model belongs to the linear class and when standard estimation formulas apply. It is part of the lecture’s basic description of the model assumptions.
The lecture repeatedly refers to models involving linear relations and normal random variables. The term is used to emphasize the simple algebraic structure of the model.
status: candidate · lectures: 1
LMS / least mean squares estimate
least mean squares, least mean squares estimate, least mean squares estimator, least-mean-squares estimator, LMS
LMS is the least mean squares estimator, a rule that selects the estimate with the smallest mean squared error.
LMS is an estimation rule that chooses the value minimizing the expected squared difference between the estimate and the unknown quantity. In the course, it is described as the estimate equal to a conditional expectation.
It is a standard optimality criterion for point estimation when squared error is the performance measure. It gives a precise target for comparing estimators.
status: candidate · lectures: 2
LMS rule
least mean squares rule
The LMS rule is the rule that produces the least mean squares estimate.
The LMS rule is an estimation rule that selects the estimate minimizing mean squared error, often expressed as a conditional expectation. It is the decision rule associated with the LMS estimator.
It formalizes how to choose an estimate when squared error is the criterion of interest. This makes it a standard benchmark in estimation problems.
In the lecture, the LMS rule is introduced as the rule that calculates an estimate equal to the conditional expectation. It appears in the discussion of how to derive point estimates from a posterior.
status: candidate · lectures: 1
log posterior / logarithm of the posterior
log posterior, logarithm of the posterior
The log posterior is the logarithm of the posterior distribution or density.
The log posterior is the posterior written on a logarithmic scale. Because the logarithm is monotonic, maximizing the log posterior gives the same maximizer as maximizing the posterior itself.
It is often easier to work with than the posterior directly, especially when products become sums. This makes optimization and derivations simpler.
The lecture uses the log posterior when describing how to find the MAP estimate by maximizing the posterior. It appears as the quantity whose maximizer is the desired estimate.
status: candidate · lectures: 1
MAP / MAP estimate
MAP, MAP estimate, MAP estimates, MAP methodology, MAP rule, maximum a posteriori, maximum a posteriori estimate, maximum a posteriori probability estimate, maximum a posteriori probability estimator, maximum a posteriori probability rule
MAP is the maximum a posteriori estimate, the value that maximizes the posterior distribution or density.
MAP is an estimation rule that selects the parameter value with the highest posterior probability or density. It is obtained by maximizing the posterior, or equivalently the log posterior.
It is one of the central estimation rules in Bayesian inference and gives a principled way to turn a posterior into a single estimate. It is especially useful when a single best value is needed.
status: candidate · lectures: 2
mean squared error
mean-squared error
Mean squared error is the average squared difference between an estimate and the true value.
Mean squared error is a performance measure that averages the squared difference between an estimate and the quantity being estimated. Squaring makes larger errors count more heavily.
It is a standard criterion for comparing estimators and is the objective minimized by the LMS rule. It provides a clear numerical measure of estimation quality.
The lecture uses mean squared error when discussing how to judge estimators and why the LMS estimate is defined the way it is. It appears as the key error metric in the estimation discussion.
status: candidate · lectures: 1
negative exponential of a quadratic function of x
A negative exponential of a quadratic function is an expression like exp(−ax²−bx−c), which has the shape of a normal density.
This is a function whose exponent is a quadratic polynomial with a negative sign in front, so the value decreases away from its peak. Such expressions are the standard algebraic form of a normal density up to constants.
Recognizing this form makes it possible to identify normal distributions from their algebraic expression. The lecture uses this observation in the derivation of the posterior.
The lecture says that an expression of this form is always a normal PDF. This is used as a quick way to recognize the distributional shape in the derivation.
status: candidate · lectures: 1
noise
random noise
Noise is random variation that distorts a signal, measurement, or observation.
Noise is the random component in a signal or measurement that is not explained by the intended message or model. It is often modeled as a random variable drawn from some distribution.
It is a basic source of uncertainty in statistical and engineering models. Accounting for noise is essential for inference, estimation, and detection problems.
The lecture uses noise in a signal-transmission example, where a pilot signal is sent and the received observation includes random disturbance. It is described as random noise coming from a distribution.
status: candidate · lectures: 1
noise random variables
Noise random variables are the random terms in a model that represent disturbance, error, or unexplained variation.
Noise random variables are the random components added to a model to capture variability not explained by the main signal or predictors. They are often assumed to have a specific distribution.
They determine how uncertain the observations are and strongly affect the posterior and estimator formulas. The lecture uses them as part of the Bayesian model setup.
The lecture introduces a model for the noise random variables before discussing their variance and distributional assumptions. They are part of the uncertainty structure of the linear model.
status: candidate · lectures: 1
noise term Wi
Wi is the noise term for the i-th observation in the model.
Wi denotes the random noise associated with the i-th observation. It is the part of the model that captures random deviation from the linear signal.
Noise terms control the variability of the observations and enter directly into the likelihood and posterior calculations. Their size affects how strongly the data influence the estimate.
The lecture mentions Wi when discussing how large or small a particular noise term can be. It is used as the observation-level error component in the model.
status: candidate · lectures: 1
noise variance
noise variances
Noise variance is the variance of the random error terms in a model.
Noise variance measures the spread of the noise random variables around their mean, usually zero. A smaller variance means the noise is more tightly concentrated.
It controls the amount of uncertainty in the observations and influences the shape of the posterior distribution. It is a key parameter in the lecture’s normal model.
The lecture refers to the corresponding noise variance being small and to one noise variance being very small. This is part of the discussion of how the model behaves under different uncertainty levels.
status: candidate · lectures: 1
normal distribution / normal
normal, normal distribution
The normal distribution is a symmetric bell-shaped distribution described by a mean and a variance.
The normal distribution is a continuous probability distribution with a symmetric bell shape centered at its mean. Its spread is controlled by the variance.
It is one of the most important distributions in statistics and appears throughout the lecture’s model and posterior calculations. Many formulas become especially simple when variables are normal.
The lecture uses normality to recognize PDFs, describe random variables, and show that the posterior is also normal. It also notes that the peak of a normal distribution occurs at the mean.
status: candidate · lectures: 1
normal random variable
normal random variables, standard normal random variables
A normal random variable has a normal distribution.
A normal random variable is a random variable whose distribution is normal. It may have any mean and variance, and in the standard case it has mean 0 and variance 1.
Normal random variables are central to the lecture’s model assumptions and to the form of the posterior distribution. Their algebraic properties make the estimation problem tractable.
The lecture repeatedly works within the world of normal random variables and notes cases where Theta and W are standard normal. These assumptions drive the posterior and MAP calculations.
status: candidate · lectures: 1
normality assumptions
Normality assumptions are modeling assumptions that specified random variables are normally distributed.
Normality assumptions state that certain random variables in the model follow normal distributions. They are often made for the parameter, the noise, or both.
These assumptions simplify the form of the likelihood and posterior and make the estimation problem easier to analyze. They are a key part of the lecture’s Bayesian setup.
The lecture says that one makes certain normality and independence assumptions. These assumptions define the probabilistic structure used in the derivation.
status: candidate · lectures: 1
normalizing constant
A normalizing constant is a factor that scales a function so it becomes a valid probability distribution or density.
A normalizing constant is the multiplicative factor needed to make an unnormalized function integrate or sum to one. It ensures that the result is a proper probability distribution or density.
It appears whenever a posterior or other probability expression is written up to proportionality. Knowing that such a constant exists helps separate the shape of the distribution from its scale.
status: candidate · lectures: 2
overall probability of error
Overall probability of error is the total probability that a hypothesis test or decision rule makes an incorrect decision.
Overall probability of error is a performance measure for a decision rule that combines the probabilities of all wrong outcomes. It summarizes how often the rule is expected to make a mistake.
It is the main criterion used in the lecture to compare hypothesis-testing rules. Minimizing it is a natural goal when the task is to make the best possible decision.
In the hypothesis-testing part of the lecture, performance is evaluated in terms of overall probability of error. The lecture emphasizes choosing a rule whose error probability is as small as possible.
status: candidate · lectures: 1
PDF / conditional PDF
conditional PDF, conditional PDF of X, marginal density, marginal PDF, marginal PDF of Theta, marginal PDFs, marginal posterior PDF, normal PDF, PDF, probability density function, probability distribution, shifted PDF
PDF stands for probability density function, the function used to describe a continuous probability distribution.
A PDF, or probability density function, is a function whose integral over an interval gives the probability that a continuous variable falls in that interval. It is not itself a probability distribution, but a function that represents one.
PDFs are the basic objects used to write likelihoods, priors, and posteriors in the course. Understanding them is necessary for following the derivations.
status: candidate · lectures: 2
performance metrics
Performance metrics are measures used to evaluate how good an estimate or decision rule is.
Performance metrics are quantitative criteria used to assess the quality of an estimator or decision rule. They may measure error, accuracy, or the chance of making a wrong decision.
They provide the basis for comparing methods and choosing among competing rules. In inference, the metric determines what counts as a good solution.
The lecture introduces performance metrics when explaining how to judge the quality of an estimate. It uses them to motivate the criteria later applied to hypothesis testing and estimation.
status: candidate · lectures: 1
pilot signal
A pilot signal is a known signal sent for observation, calibration, or estimation purposes.
A pilot signal is a predetermined signal transmitted through a system so that the received output can be analyzed. It is often used as a known input in a measurement or communication model.
It provides a controlled reference for studying noise, distortion, or channel behavior. This makes it useful in estimation and signal-processing examples.
The lecture uses a pilot signal in its motivating communication example, where a known signal is sent and then observed after noise is added. It helps illustrate how inference is performed from noisy data.
status: candidate · lectures: 1
PMF / conditional PMF
conditional PMF, conditional PMF of Theta, PMF, probability mass function, probability mass functions
PMF stands for probability mass function, and in this lecture it is used for a conditional mass function when the variable is discrete.
PMF stands for probability mass function. In this lecture, it is used in the conditional sense to describe the probability of a discrete random variable given another variable.
It is the discrete counterpart of a density and is needed for Bayesian updating with discrete variables. It appears when the unknown parameter or observation takes countable values.
The lecture refers to a conditional PMF of Theta and distinguishes it from the continuous case. This notation is used in the derivation of the posterior distribution for discrete variables.
status: candidate · lectures: 1
point estimate
point estimates
A point estimate is a single numerical value used to represent an unknown parameter or quantity.
A point estimate is a single value chosen to summarize an unknown quantity, such as a parameter or state. It is the output of an estimation rule.
Point estimates are the basic output of many inference procedures. They provide a compact summary when a full distribution is not needed.
status: candidate · lectures: 2
posterior
The posterior is the distribution of an unknown quantity after the data have been observed.
The posterior is the probability distribution or density for an unknown parameter after combining prior information with observed data. It represents the updated belief about the unknown quantity.
It is the central object in Bayesian inference and the basis for MAP estimation. It summarizes what is learned from the data.
The lecture uses posterior to describe the distribution that results after observing data and then summarizes it with a single estimate. It appears in the discussion of how the posterior takes a particular form.
status: candidate · lectures: 1
posterior distribution / posterior density
posterior density, posterior distribution, posterior probability distribution
The posterior distribution is the updated probability distribution for an unknown parameter after observing data.
The posterior distribution is the probability distribution for an unknown parameter after the prior has been updated by the observed data. In continuous settings, it is often described as a posterior density.
It is the main Bayesian object from which estimates and decisions are derived. It captures the information gained from the data in a complete probabilistic form.
The lecture repeatedly refers to the posterior distribution of Theta and gives formulas for the posterior density. It is the distribution that is later summarized by the MAP estimate.
status: candidate · lectures: 1
prior distribution
prior, prior density, prior distributions, prior probability distribution, priors
A prior distribution is the probability distribution assigned to an unknown parameter before data are observed.
A prior distribution expresses initial uncertainty about an unknown parameter or quantity before observing the current data. It is combined with the likelihood to form the posterior.
It is a defining ingredient of Bayesian inference and influences the resulting posterior and estimates. It encodes information available before the new observation is made.
status: candidate · lectures: 2
prior mean
The prior mean is the expected value of a parameter before data are observed.
The prior mean is the center or average value of a prior distribution for an unknown quantity. It represents the model’s initial belief about that quantity before seeing data.
It helps determine how the prior influences the posterior estimate. In Bayesian calculations, it is one of the key inputs that gets combined with the observed data.
In this lecture, the prior mean is treated as a numerical input in the Gaussian prior model, sometimes set to 0 or written as X0. It appears in the algebra that leads to the posterior form.
status: candidate · lectures: 1
prior variance
The prior variance is the spread of a prior distribution before observing data.
The prior variance measures how uncertain the prior distribution is about an unknown parameter. A larger prior variance means a wider, less concentrated prior.
It controls how strongly the prior influences the posterior compared with the data. Together with the prior mean, it shapes the starting belief in the model.
Here it appears as a parameter in the Gaussian prior for Theta, with a specific numerical value used in the derivation. It is part of the variance terms that are combined in the posterior calculation.
status: candidate · lectures: 1
probabilistic model
A probabilistic model describes a system using random variables and probability distributions.
A probabilistic model is a mathematical description of a system in which uncertainty is represented with random variables and probability distributions. It specifies how observed data and unknown quantities are related.
It lets the course treat estimation and inference as problems of reasoning under uncertainty. This makes it possible to compare estimates using probabilities and expected error.
In this lecture, the unknown quantity is modeled in probabilistic terms so that inference can be based on observed data. The phrase appears as part of the setup for Bayesian estimation.
status: candidate · lectures: 1
probability of error
The probability of error is the chance that an estimate or decision differs from the true value.
The probability of error measures how likely an estimator or decision rule is to produce an incorrect result. It is a performance metric defined under the assumed probabilistic model.
The lecture uses it as a main criterion for judging estimation quality. It captures correctness directly, rather than only average numerical closeness.
Here it is presented as the relevant metric for comparing estimation methods. The lecture uses it repeatedly when discussing how to evaluate a particular estimate.
status: candidate · lectures: 1
quadratic function
quadratic function of time, quadratic in Theta
A quadratic function is a function with squared terms, such as a polynomial of degree two.
A quadratic function is a function that includes terms squared in the variable, and may also include linear and constant terms. In one variable, it has the general shape of a parabola.
Quadratic forms are easy to analyze and minimize, which makes them common in estimation and probabilistic derivations. They also appear naturally in Gaussian models.
In this lecture, quadratic expressions appear when the log-density is rewritten in terms of Theta and when the result is minimized. The same idea is also used for time-dependent and parameter-dependent expressions.
status: candidate · lectures: 1
random variable
random variables, relation between random variables
A random variable is a numerical quantity whose value is determined by chance and modeled probabilistically.
A random variable is a variable that can take different values according to a probability distribution. It is the basic object used to model uncertainty in probability and statistics.
Random variables allow the course to describe unknown parameters, observations, and noise in a unified way. They are essential for defining priors, likelihoods, and estimators.
status: candidate · lectures: 2
recursive posterior updating / recursive ways of updating the posterior
recursive posterior updating, recursive ways of updating the posterior
Recursive posterior updating is the process of revising the posterior distribution repeatedly as new data are observed.
Recursive posterior updating means using the current posterior as the new prior and updating it again when additional observations become available. It is a step-by-step Bayesian updating procedure.
It provides an efficient way to incorporate data sequentially instead of starting over each time. This is especially useful when inference is built from repeated observations.
The lecture mentions recursive ways of updating the posterior of Theta as part of the Bayesian estimation setup. The idea is used to show how inference can be updated incrementally.
status: candidate · lectures: 1
sigma squared
sigma 0 squared, sigma 1 squared, sigma i squared, sigma0 squared
Sigma squared is a notation for variance, especially in normal models.
Sigma squared denotes a variance parameter, usually the spread of a random variable or noise term. Indexed forms such as sigma 0 squared or sigma 1 squared refer to different variances in the model.
Variance controls how dispersed a distribution is and how much uncertainty is present. It is a central quantity in Gaussian calculations and posterior formulas.
The lecture uses sigma squared in several variance terms, including prior and noise variances. It also distinguishes indexed versions when different parts of the model have different spreads.
status: candidate · lectures: 1
squared distance
Squared distance is the squared difference between two values, often used as a loss measure.
Squared distance is a measure of how far apart two values are, computed by taking their difference and squaring it. It penalizes larger deviations more strongly than smaller ones.
It is a standard way to define estimation error and loss in statistical problems. Squaring makes the criterion mathematically convenient and sensitive to large mistakes.
In this lecture, squared distance is used when discussing the criterion for evaluating an estimate. It appears in the derivation of the error measure.
status: candidate · lectures: 1
squared error
Squared error is the square of the difference between an estimate and the true value.
Squared error is an error measure obtained by subtracting the true value from an estimate and squaring the result. It is always nonnegative and grows quickly with larger mistakes.
It is a common loss function in estimation and learning because it gives a simple numerical measure of accuracy. It also connects naturally to mean squared error.
The lecture refers to a certain squared error when discussing how to evaluate an estimate. It is part of the comparison between different estimation criteria.
status: candidate · lectures: 1
symmetry argument
A symmetry argument is a proof or reasoning step that uses symmetry to justify a conclusion.
A symmetry argument is a way of reasoning that relies on the symmetry of a problem to narrow down or determine the correct form of a result. It often shows that no option is preferred over another.
Symmetry can simplify derivations and reveal the structure of an optimal estimator or distribution. It is a common tool for making inference arguments more concise.
The lecture says to choose a quantity using a symmetry argument, indicating that symmetry helps determine the form of the solution. It is used as part of the reasoning in the estimation setup.
status: candidate · lectures: 1
Theta
theta_j, thetas
Theta is the unknown parameter or quantity that the course aims to infer.
Theta is a symbol used for an unknown quantity in the model, often treated as a parameter or random variable. Its value is not directly observed and must be inferred from data.
It is the central unknown in the estimation problem, so the inference task is about learning Theta from observations. Understanding its role is necessary for following the Bayesian setup.
status: candidate · lectures: 2
theta components / theta 0
theta 0, theta 1, theta 2, theta components
Theta components are the separate parameters that make up the full Theta vector.
Theta components are the individual entries of a parameter vector, each representing a different unknown quantity. In the lecture’s example, they include values such as initial height and initial velocity.
Breaking Theta into components makes the model easier to interpret and estimate. Each component can have its own meaning and prior distribution.
The lecture uses component notation to describe the parameters of a trajectory model, including theta 0 and theta 1. These components are assigned meanings such as height and velocity in the example.
status: candidate · lectures: 1
Theta hat
Theta hat is the estimate of Theta obtained from data, rather than the true unknown parameter itself.
Theta hat denotes an estimated value of Theta. The hat notation indicates that it is computed from data and represents the inferred quantity rather than the unknown true parameter.
This notation cleanly separates the estimate from the parameter being estimated, which is essential in estimation theory and statistical derivations. It helps keep formulas and interpretations unambiguous.
status: candidate · lectures: 2
total expectation theorem
The total expectation theorem says that an expectation can be found by taking a conditional expectation and then averaging it.
The total expectation theorem is a result that relates an overall expectation to an expectation taken conditionally on another variable. It is often written as the expectation of a conditional expectation.
It simplifies calculations by breaking a difficult expectation into smaller conditional pieces. This is especially useful in Bayesian and probabilistic derivations.
The lecture invokes the theorem to simplify an expectation in the posterior analysis. It is used as a standard step in the algebraic derivation.
status: candidate · lectures: 1
total probability theorem
total probability or expectation theorem
The total probability theorem expresses an overall probability by averaging over mutually exclusive cases or conditioning events.
The total probability theorem is a rule for finding an overall probability by combining probabilities across a partition of cases. It can be written as a sum or integral depending on whether the variables are discrete or continuous.
It is a basic tool for marginalizing over unknown quantities in probabilistic models. The lecture uses it to connect conditional and unconditional quantities.
The lecture invokes the theorem when deriving expressions involving the posterior and the unconditional error criterion. It is part of the calculation that combines different possible values of Theta.
status: candidate · lectures: 1
trajectory
trajectory estimation problem
A trajectory is the path an object follows over time.
A trajectory is the time-dependent path or motion of an object. In modeling, it can be represented as a function of time or as a sequence of positions.
Trajectory estimation is a common problem in applied statistics and signal processing. It connects latent motion models with noisy measurements.
In this lecture, the trajectory is the underlying path the model tries to estimate from observations at certain times. The discussion returns to the trajectory estimation problem as the example develops.
status: candidate · lectures: 1
unconditional mean squared error
Unconditional mean squared error is the average squared error taken over the full probability model.
Unconditional mean squared error is the expected value of the squared estimation error without conditioning on a particular parameter value or event. It averages the error over the full distribution of the model.
It is a standard way to measure overall estimator performance. Because it averages over all randomness, it gives a single summary of expected accuracy.
The lecture uses this quantity when comparing estimation criteria and evaluating a particular estimate. It is presented as the relevant average error measure.
status: candidate · lectures: 1
uniform distribution
uniform on the unit interval
A uniform distribution gives equal probability density across its support.
A uniform distribution is a probability distribution that is constant over a specified interval or set and zero outside it. In the continuous case, all values in the support are equally likely in density terms.
It is a simple model for uncertainty when no value is preferred over another within a range. It also serves as a common prior in Bayesian inference.
The lecture refers to estimating the parameter of a uniform distribution and to a prior that is uniform on the unit interval. This makes the uniform distribution part of the Bayesian setup.
status: candidate · lectures: 1
uniform prior
A uniform prior is a prior distribution that is constant over the allowed range of parameter values.
A uniform prior is a Bayesian prior that gives equal density to all values in a specified range. It represents initial uncertainty without favoring one value over another within that range.
It provides a simple starting point for Bayesian inference when little prior information is assumed. The choice of prior affects the posterior and the resulting estimate.
The lecture says that the setup gives a uniform prior, using it as the starting distribution for Theta. It is part of the Bayesian model used in the derivation.
status: candidate · lectures: 1
unknown bias of a coin
The unknown bias of a coin is the unknown probability that the coin lands heads.
The unknown bias of a coin is the parameter that determines the probability of heads in repeated tosses. It is a standard example of an unknown quantity to be estimated from data.
It illustrates how a hidden parameter can be inferred from observations. The example makes abstract estimation ideas concrete and familiar.
The lecture uses the coin-bias example when discussing Theta as an unknown quantity. It serves as an intuitive illustration of Bayesian inference from observations.
status: candidate · lectures: 1
unknown parameter
An unknown parameter is a quantity in the model whose true value is not known in advance.
An unknown parameter is a model quantity with an unspecified true value. It is the target of inference in estimation problems.
Identifying unknown parameters is the main purpose of statistical modeling and estimation. The course uses probability to represent uncertainty about them.
The lecture describes Theta as an unknown parameter that is modeled as a random variable. This framing supports the Bayesian setup used in the derivation.
status: candidate · lectures: 1
unobserved variables
Unobserved variables are variables whose values are not directly seen in the observed data.
Unobserved variables are quantities in a model that are not directly measured or recorded. Their values must be inferred indirectly from the observed data and the model structure.
They are central to probabilistic modeling because many important quantities are hidden rather than directly observed. Inference often focuses on estimating or integrating over them.
The lecture notes that there may be variables that are unknown and unobserved, motivating the need for probabilistic inference. This sets up the treatment of hidden quantities like Theta.
status: candidate · lectures: 1
variance
variance of the posterior distribution, variances
Variance measures how spread out a random variable or distribution is around its mean.
Variance is a numerical measure of dispersion around the mean. Larger variance means values are typically more spread out, while smaller variance means they are more concentrated.
Variance is a core way to describe uncertainty in probability and statistics. It is also used in noise modeling, posterior uncertainty, and error measures such as mean squared error.
status: candidate · lectures: 2
W_i
Wi
W_i is a noise term in the observation model.
W_i denotes a random noise term added to the underlying signal or parameter in the model. It represents random error or disturbance in the observation process.
Noise terms explain why observed data differ from the underlying true quantity. They are essential for writing realistic probabilistic models.
In this lecture, W_i appears in the equation relating observations to Theta. The lecture explicitly identifies W_i as the noise in the model.
status: candidate · lectures: 1
weighted average
A weighted average is an average where some values count more than others.
A weighted average combines several values using weights that determine each value’s contribution. Values with larger weights have more influence on the result.
Weighted averages often appear in estimation formulas and posterior means. They provide a compact way to combine prior information and data.
The lecture describes the resulting estimator as a weighted average of the observed Xi values. This interpretation helps explain the structure of the posterior mean.
status: candidate · lectures: 1
X of t
X of t is the true position of the object at time t.
X of t denotes the underlying true position or state at time t. It is a time-indexed quantity that describes the object’s actual motion.
It serves as the latent target in the trajectory estimation problem. Observations are compared against it to infer the object’s path.
The lecture uses X of t to name the true position in the trajectory model. It is the quantity that the noisy observations are trying to reveal.
status: candidate · lectures: 1
x0
x0 is a model parameter used as the prior mean or initial value.
x0 is a notation for a specific parameter value in the model. In the lecture, it is used as the prior mean or as an initial value in the setup.
It provides a concrete reference point for the prior distribution or model initialization. Such parameters help anchor the probabilistic description.
The lecture introduces x0 when defining the prior mean and later refers to X0 in the same role. It is part of the notation used in the Gaussian prior model.
status: candidate · lectures: 1
Xi / X1 up to Xn
X sequence, X1 up to Xn, X_i, Xi
Xi is the i-th observation in a sequence of random variables.
Xi denotes the i-th random variable in an indexed sequence of observations. In some contexts, the notation refers to the whole collection X1 through Xn.
Indexed observations are the data used to estimate unknown parameters and latent states. They are the bridge between the model and the measured information.
The lecture uses Xi when writing the observation model and when discussing the sequence X1 up to Xn. These variables are treated as normal random variables in the example.
status: candidate · lectures: 1
zero mean
Zero mean means that the expected value of a random variable is equal to zero.
A zero-mean random variable has expected value equal to zero. It may still vary around zero, but its average level is centered at zero.
Zero-mean noise is a common simplifying assumption in statistical models. It helps separate systematic effects from random fluctuations.
The lecture states that the noise has zero mean when describing the model assumptions. This supports the interpretation of the noise as centered random variation.
status: candidate · lectures: 1