Naiyu Yin

Causal Markov Blanket Representation Learning for Out-of-distribution Generalization

This research addresses the poor out-of-distribution (OOD) generalization issue in the realm of machine learning and computer vision. Current methods aim to secure invariant representations by either harnessing domain expertise or leveraging data from multiple domains. In this paper, we introduce a novel approach that involves acquiring Causal Markov Blanket (CMB) representations to improve prediction performance in the face of distribution shifts. Causal Markov Blanket representations comprise the direct causes and effects of the target variable. Theoretical analyses have demonstrated their capacity to harbor maximum information about the target, resulting in minimal Bayes error during prediction. To elaborate, our approach commences with the introduction of a novel structural causal model (SCM) equipped with latent representations, designed to capture the underlying causal mechanisms governing the data generation process. Subsequently, we propose a CMB representation learning framework that derives representations conforming to the proposed SCM. In comparison to state-of-the-art domain generalization methods, our approach exhibits robustness and adaptability under distribution shifts. Please refer to the [PDF]

Integrating Markov Blanket Discovery into Causal Representation Learning

Identifying low-dimensional, semantic latent causal representations for high-dimensional data has become a dynamic field in computer vision and machine learning. Causal domain generalization methods aim to identify latent causal variables that generate input data and build invariant causal mechanisms for prediction tasks, thereby improving out-of-distribution (OOD) prediction performance. However, there is no consensus on the best approach for selecting causal variables for prediction. Existing methods typically choose causal or anti-causal variables, excluding other invariant, discriminative features. In this paper, we propose using Markov Blanket features due to their property of being the minimal set that possesses the maximum mutual information with the target. To achieve this, we establish a Causal Markov Blanket Representation Learning (CMBRL) framework, which allows for Markov Blanket discovery in the latent space. We then construct an invariant prediction mechanism using the identified Markov Blanket features, making it suitable for predictions across domains. Compared to state-of-the-art domain generalization methods, our approach exhibits robustness and adaptability under distribution shifts. Please refer to the [PDF]

Effective and Identifiable Causal Discovery under Heteroscedastic Noise Model

This research addresses the problem of performing causal discovery under the challenging Heteroscedastic noise data. Causal DAG learning via the continuous optimization framework has recently achieved promising performance in terms of both accuracy and efficiency. However, most methods make strong assumptions of homoscedastic noise, i.e., exogenous noises have equal variances across variables, observations, or even both. The noises in real data usually violate both assumptions due to the biases introduced by different data collection processes. To address the issue of heteroscedastic noise, we introduce relaxed and implementable sufficient conditions, proving the identifiability of a general class of SEM subject to these conditions. Based on the identifiable general SEM, we propose a novel formulation for DAG learning that accounts for the variation in noise variance across variables and observations. We then propose an effective two-phase iterative DAG learning algorithm to address the increasing optimization difficulties and to learn a causal DAG from data with heteroscedastic variable noise under varying variance. We show significant empirical gains of the proposed approaches over state-of-the-art methods on both synthetic data and real data. Please refer to the [PDF]

An Efficient Differentiable Global Causal Discovery Method

This research addresses the time-consuming computation and low-efficiency issues raised by solving the constrained continuous optimization in functional causal model-based methods. It proposes a novel formulation to model and learn the weighted adjacency matrices of the causal DAG directly. This work theoretically proves that the set of causal DAGs is equivalent to the set of weighted gradients of graph potential functions, and hence one may perform causal DAG learning by searching the equivalent space of DAGs without explicitly posing the acyclicity constraint. Hence a two-step procedure is proposed to first obtain the cyclic solution through a few iterations of soft-constrained optimization. The acyclic graph can then be obtained by projecting the cyclic solution to the equivalent space. The novelties of this work lie in 1) theoretically demonstrating that the weighted adjacency matrix of a DAG can be represented as the product of a skew-symmetric matrix and the gradient of a potential function on graph variables. 2) proposing the novel formulation and algorithm for learning the causal DAG without explicit constraint. 3) achieving substantial improvement in efficiency with comparable accuracy compared to SOTA functional causal model-based causal discovery methods. Please refer to the published paper for further details.