Naiyu Yin

Causal Markov Blanket Representation Learning for Out-of-distribution Generalization

This research addresses the poor out-of-distribution (OOD) generalization issue in the realm of machine learning and computer vision. Current methods aim to secure invariant representations by either harnessing domain expertise or leveraging data from multiple domains. In this paper, we introduce a novel approach that involves acquiring Causal Markov Blanket (CMB) representations to improve prediction performance in the face of distribution shifts. Causal Markov Blanket representations comprise the direct causes and effects of the target variable. Theoretical analyses have demonstrated their capacity to harbor maximum information about the target, resulting in minimal Bayes error during prediction. To elaborate, our approach commences with the introduction of a novel structural causal model (SCM) equipped with latent representations, designed to capture the underlying causal mechanisms governing the data generation process. Subsequently, we propose a CMB representation learning framework that derives representations conforming to the proposed SCM. In comparison to state-of-the-art domain generalization methods, our approach exhibits robustness and adaptability under distribution shifts. Please refer to the [PDF]

Causal Representation Learning and Inference for Generalizable Cross-Domain Predictions

This research addresses the poor cross domain generalization issue for machine learning and computer vision tasks. Current methods utilize data from multiple domains and seek to transfer invariant representations to new and unseen domains. This paper proposes to perform causal inference on a transportable, invariant interventional distribution to improve prediction performance under distribution shifts. To do so, we first propose an identifiable structural causal model (SCM) to capture the underlying causal mechanism that underpins the data generation process. Subject to the proposed SCM model, we then introduce a latent representation learning framework, allowing us to discover latent variables and capture the underlying data generation mechanisms. Next, we propose an inference procedure to estimate the invariant, transportable interventional distribution that can account for confounding effects between input and label. Furthermore, we empirically demonstrate the robustness of our proposed method under distribution shifts across multiple benchmark real datasets. Empirical results show that our proposed method outperforms the majority of domain generalization baselines, achieving state-of-the-art performance.

Effective and Identifiable Causal Discovery under Heteroscedastic Noise Model

This research addresses the problem of performing causal discovery under the challenging Heteroscedastic noise data. Causal DAG learning via the continuous optimization framework has recently achieved promising performance in terms of both accuracy and efficiency. However, most methods make strong assumptions of homoscedastic noise, i.e., exogenous noises have equal variances across variables, observations, or even both. The noises in real data usually violate both assumptions due to the biases introduced by different data collection processes. To address the issue of heteroscedastic noise, we introduce relaxed and implementable sufficient conditions, proving the identifiability of a general class of SEM subject to these conditions. Based on the identifiable general SEM, we propose a novel formulation for DAG learning that accounts for the variation in noise variance across variables and observations. We then propose an effective two-phase iterative DAG learning algorithm to address the increasing optimization difficulties and to learn a causal DAG from data with heteroscedastic variable noise under varying variance. We show significant empirical gains of the proposed approaches over state-of-the-art methods on both synthetic data and real data. Please refer to the [PDF]

An Efficient Differentiable Global Causal Discovery Method

This research addresses the time-consuming computation and low-efficiency issues raised by solving the constrained continuous optimization in functional causal model-based methods. It proposes a novel formulation to model and learn the weighted adjacency matrices of the causal DAG directly. This work theoretically proves that the set of causal DAGs is equivalent to the set of weighted gradients of graph potential functions, and hence one may perform causal DAG learning by searching the equivalent space of DAGs without explicitly posing the acyclicity constraint. Hence a two-step procedure is proposed to first obtain the cyclic solution through a few iterations of soft-constrained optimization. The acyclic graph can then be obtained by projecting the cyclic solution to the equivalent space. The novelties of this work lie in 1) theoretically demonstrating that the weighted adjacency matrix of a DAG can be represented as the product of a skew-symmetric matrix and the gradient of a potential function on graph variables. 2) proposing the novel formulation and algorithm for learning the causal DAG without explicit constraint. 3) achieving substantial improvement in efficiency with comparable accuracy compared to SOTA functional causal model-based causal discovery methods. Please refer to the published paper for further details.