Adapting, Fast and Slow: On Few-Shot Transportability of Compositions

Researchers present a framework for cross-domain generalization in machine learning that extends causal transportability theory to handle sequential prediction tasks. The work introduces module and circuit transportability, enabling models to compose learned mechanisms from source domains to make zero-shot predictions on target domains, with practical few-shot learning methods requiring minimal target domain data.

This research addresses a fundamental challenge in machine learning: how to effectively generalize learned patterns across different domains without extensive retraining. The work builds on causal transportability theory by introducing a compositional approach where target predictors are represented as circuits combining causal mechanisms learned from source data. This represents a meaningful advance in domain adaptation, moving beyond single-mechanism transfer to multi-module composition that enables predictions even when no individual source mechanism directly solves the target task.

The framework progresses through increasingly practical assumptions. Initially, it requires explicit causal knowledge of both source and target domains to identify composable circuits. The authors then relax these requirements by incorporating limited target domain data, developing a supervised adaptation scheme that works without explicit causal structure specification. This progression reflects real-world constraints where perfect causal knowledge is rarely available. The few-shot guarantees tie achievable error rates to the complexity of the smallest circuit composable from learned modules, providing theoretical grounding for practical expectations.

The gradient-based relaxation of symbolic circuit search with empirical validation demonstrates that theoretical predictions align with actual learning dynamics, including distinctions between fast adaptation with intermediate supervision and slower adaptation when source mechanisms don't directly transfer. For machine learning practitioners, this work offers principled guidance on when and how to compose learned models across domains. The framework's ability to function with minimal target data makes it particularly relevant for applications where data collection is expensive or time-consuming, spanning healthcare, finance, and scientific discovery domains.

• →Module and circuit transportability enable zero-shot cross-domain prediction by composing mechanisms learned from multiple source domains.
• →The framework progresses from requiring explicit causal knowledge to practical few-shot methods that learn without specified causal structures.
• →Achievable error rates are bounded by the minimum circuit complexity composable from source-learned modules, providing theoretical performance guarantees.
• →Gradient-based circuit search validates theoretical predictions of fast and slow adaptation regimes depending on source mechanism alignment.
• →Limited target domain data significantly improves adaptation by enabling circuit composition refinement without large-scale retraining.