Generalised Linear Models in Deep Bayesian RL with Learnable Basis Functions

Researchers introduce GLiBRL, a novel deep Bayesian reinforcement learning method that combines generalized linear models with learnable basis functions to improve task generalization. The approach achieves fully tractable Bayesian inference over task parameters and demonstrates up to 1.8x performance improvements over existing meta-RL methods on benchmark tasks.

GLiBRL addresses a fundamental challenge in Bayesian reinforcement learning: the tension between model flexibility and inference tractability. Traditional deep BRL methods rely on variational inference when applying neural networks to joint data and task parameters, often producing unclear task representations that degrade policy performance. This new framework resolves that limitation by adopting generalized linear models with learnable basis functions, enabling exact Bayesian inference rather than approximate variational methods.

The technical innovation centers on maintaining closed-form solutions throughout the inference process while learning transition and reward models. This tractability permits exact marginal likelihood evaluation, a capability absent in most competing approaches. The permutation-invariance property of the method's exact Bayesian inference makes it compatible with both on-policy and off-policy reinforcement learning algorithms, expanding its practical applicability across different training paradigms.

GLiBRL's empirical validation demonstrates substantial improvements over recent meta-RL baselines on standard benchmarks including MuJoCo and MetaWorld, with performance gains reaching 1.8x in certain tasks. More notably, the researchers establish the first known structural relationship between task representation geometry and kernel-based task correspondence in online deep BRL, providing theoretical grounding for the approach.

This advancement matters for AI systems requiring rapid adaptation across varied environments. The method's efficiency and interpretability could accelerate deployment of RL agents in robotics and control applications where generalization across task distributions is critical. The theoretical contributions also provide foundations for future research in combining classical statistical approaches with modern deep learning architectures.

• →GLiBRL enables fully tractable Bayesian inference in deep RL by using generalized linear models with learnable basis functions instead of variational approximations
• →The method achieves up to 1.8x performance improvements over state-of-the-art meta-RL approaches on MuJoCo and MetaWorld benchmarks
• →Exact Bayesian inference permits seamless integration with both on-policy and off-policy RL algorithms through permutation-invariant inference
• →First theoretical result connecting task representation distance to empirical kernel-based task correspondence in online deep Bayesian RL
• →Improved task representation clarity through exact inference addresses a key limitation of variational deep BRL methods