Categorical Distribution

Categorical distributions over finite sets.

This module provides: - Categorical: Distribution over n states with probabilities summing to 1 - Bernoulli: Special case for binary variables (equivalent to Categorical(2)) - Bernoullis: Product of n independent Bernoulli distributions

Class Hierarchy

Categorical

class Categorical(n_categories: int)

Bases: Analytic

Categorical distribution over \(n\) states.

The categorical distribution describes discrete probability distributions over \(n\) states with probabilities \(\eta_i\) where \(\sum_{i=0}^n \eta_i = 1\).

\[p(k; \eta) = \eta_k\]

As an exponential family:

• Base measure: \(\mu(k) = 0\)

• Sufficient statistic: One-hot encoding for \(k > 0\)

• Log partition: \(\psi(\theta) = \log(1 + \sum_{i=1}^d e^{\theta_i})\)

• Negative entropy: \(\phi(\eta) = \sum_{i=0}^d \eta_i \log(\eta_i)\)

n_categories: int

Number of categories.

property dim: int

Dimension \(d\) is n_categories - 1 due to the sum-to-one constraint.

property data_dim: int

Dimension of the data space.

from_probs(probs: Array) → Array

Construct the mean parameters from the complete probabilities, dropping the first element.

to_probs(means: Array) → Array

Return the probabilities of all labels.

sufficient_statistic(x: Array) → Array

Compute the sufficient statistic \(\mathbf{s}(x)\) of an observation.

log_base_measure(x: Array) → Array

Compute \(\log \mu(x)\) for an observation.

log_partition_function(params: Array) → Array

Compute the log-partition function \(\psi\) at the given natural parameters.

negative_entropy(means: Array) → Array

Compute negative entropy \(\phi\) at the given mean parameters.

sample(key: Array, params: Array, n: int = 1) → Array

Draw n samples from the distribution with the given natural parameters.