Graphical Harmoniums¶
Hierarchical harmoniums enable modeling complex, multi-level dependencies by stacking harmonium models. This stacked structure supports deep representations where information flows between multiple levels of latent variables, while maintaining analytical tractability through conjugate relationships.
In theory, a hierarchical harmonium is a conjugated harmonium where the latent manifold is itself a conjugated harmonium. This creates a structured joint distribution:
A lower harmonium provides likelihood \(p(x|y)\) between observable \(x\) and first latent \(y\)
An upper harmonium provides likelihood \(p(y|z)\) between first latent \(y\) and second latent \(z\)
Together they form a joint distribution \(p(x,y,z) = p(x|y)p(y|z)p(z)\)
Key algorithms (conjugation, natural parameters, sampling) are implemented recursively through the model hierarchy, enabling efficient inference and learning despite the model’s complexity.
Note: Implementation of these models pushes the boundaries of Python’s type system, requiring careful handling of generic types to maintain proper relationships between layers.
Class Hierarchy¶
Embeddings¶
- class ObservableEmbedding(hrm_man: Harmonium[Observable, Posterior])[source]¶
Bases:
TupleEmbedding[Observable,Harmonium],GenericEmbedding of the observable manifold of a harmonium into the harmonium itself.
This embedding projects a harmonium coordinates onto the observable component, or embeds an observable point into the full harmonium space by setting interaction and latent parameters to zero. Generally speaking, projection is only safe on mean coordinates, while embedding is only safe on natural coordinates.
- property sub_man: Observable¶
The submanifold.
- class LatentHarmoniumEmbedding(pst_obs_emb: LinearEmbedding[Any, Any], pst_hrm: PostHarmonium, prr_hrm: PriorHarmonium)[source]¶
Bases:
LinearEmbedding,GenericEmbedding of one harmonium into another via observable space embedding.
This embedding connects two harmoniums that share the same interaction and latent structure but differ in their observable parameterization. It’s used in hierarchical models where the posterior uses a restricted observable representation (e.g., diagonal covariance) for computational efficiency, while the prior uses a fuller representation for conjugation parameter computation.
Usage in hierarchical models:
In a hierarchical harmonium, the lower harmonium’s latent manifold equals the upper harmonium’s observable manifold. When the posterior and prior use different observable parameterizations, this embedding maps between them by:
Projecting prior parameters down to the restricted posterior observable space
Embedding posterior observable parameters up to the fuller prior observable space
Structure:
Both harmoniums must have:
The same interaction matrix representation and latent manifold
Observable manifolds \(\text{Obs}_1 \subseteq \text{Obs}_2\) where the posterior uses the restricted \(\text{Obs}_1\) and the prior uses the fuller \(\text{Obs}_2\)
An embedding \(e: \text{Obs}_1 \to \text{Obs}_2\) provided via pst_obs_emb
Parameters:
For harmonium points \((o, i, l)\) where \(o\) is observable, \(i\) is interaction, and \(l\) is latent:
Projection: \((o_2, i, l) \mapsto (e^{-1}(o_2), i, l)\) where \(e^{-1}\) projects to \(\text{Obs}_1\)
Embedding: \((o_1, i, l) \mapsto (e(o_1), i, l)\) where \(e\) embeds into \(\text{Obs}_2\)
Runtime requirements:
pst_obs_emb embeds pst_hrm.obs_man into prr_hrm.obs_man
pst_hrm and prr_hrm have matching interaction and latent structures
- pst_obs_emb: LinearEmbedding[Any, Any]¶
The embedding of the constrained observable manifold into the unconstrained observable manifold.
- pst_hrm: PostHarmonium¶
The harmonium that contains the constrained observable manifold.
- prr_hrm: PriorHarmonium¶
The harmonium that contains the unconstrained observable manifold.
- property sub_man: PostHarmonium¶
The submanifold.
- property amb_man: PriorHarmonium¶
The ambient manifold.