![\"SymptomsDisease\"](\"https://user-images.githubusercontent.com/33357428/210627825-c1e11271-a609-4bd7-b679-42dca0a40aee.jpg\")
This post is the second post of the series on Causal Machine Learning. In the last blog post i have given a brief introduction on Causal Machine Learning. In this we will uncover the WHY !. This blog post is based on the work of Judea Pearl. I will discuss how naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference). As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
\n\nCorrellation Is Not Causation !!!
\n
I know you are bored of this like every other, hearing this oft-repeated saying “correlation does not imply causation”. I will formally illustrate why this is in this post.
In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of statistical correlations (that are due to chance) with no causal implications. One of them is listed below.
![\"Spurious](\"https://user-images.githubusercontent.com/33357428/210524097-c10421ba-b2f5-44fd-93c8-311a365e78c7.jpg\")
\nWe clearly know No of people who drowned while in a swimming pool has nothing to do with the power generated by US nuclear power plants !!. You can find more interesting “Spurious Correlations” from Tyler Vigen website.
Spurious correlations are well-known in statistics, so it’s easy (Somewhat !) to be on the lookout for it. Lets see a little known paradox in statistics.
Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. Lets look at an example of Simpson's paradox from Wikipedia itself.
![\"Simpson's](\"https://user-images.githubusercontent.com/33357428/210533109-6f2a4803-ae3a-4cbb-82fa-38dd43b1a6a6.png\")
\nIn both 1995 and 1996, Justice had a higher batting average (in bold type) than Jeter did. However, when the two baseball seasons are combined, Jeter shows a higher batting average than Justice.
\n\nSame data give contradictory conclusions depending on how you look at them! .
\n
Simpson’s paradox highlights that how you look at your data matters. So the question becomes, how do we partition data? Although there is no standard method in statistics for this, causal inference provides a formalism to handle this problem. It all boils down to causal effects, which quantify the impact a variable has on another variable after adjusting for the appropriate confounders.
Let us look at an example from “The Book of Why: The New Science of Cause and Effect” by Judea Pearl , and a post the legend himself posted in his twitter handle.
Consider the below study that measures weekly exercise and cholesterol in various age groups. When we plot exercise on the X-axis and cholesterol on the Y-axis and segregate by age, as in left side of Fig , we see that there is a general trend downward in each group; the more young people exercise, the lower their cholesterol is, and the same applies for middle-aged people and the elderly. If, however, we use the same scatter plot, but we don’t segregate by gender (as in right side of Fig), we see a general trend upward; the more a person exercises, the higher their cholesterol is.
![\"Book](\"https://user-images.githubusercontent.com/33357428/210539152-32dac61b-7722-41c8-8d1f-6e3fea1d33f3.jpg\")
\n\n\nExcercise appears to be beneficial in each age group but harmful in the population as a whole !!!
\n
To resolve this problem, we once again turn to the story behind the data. If we know that older people, who are more likely to exercise are also more likely to have high cholesterol regardless of exercise, then the reversal is easily explained, and easily resolved. Age is a common cause of both treatment (exercise) and outcome (cholesterol). So we should look at the age-segregated data in order to compare same-age people and thereby eliminate the possibility that the high exercisers in each group we examine are more likely to have high cholesterol due to their age, and not due to exercising.
However, please do not get confused , segregated data does not always give the correct answer.
Lets look at another example from the Causal Inference In Statistics book by Pearl.
In the classical example used by Simpson (1951), a group of sick patients are given the option to try a new drug. Among those who took the drug, a lower percentage recovered than among those who did not. However, when we partition by gender, we see that more men taking the drug recover than do men are not taking the drug, and more women taking the drug recover than do women are not taking the drug!
![\"Book](\"https://user-images.githubusercontent.com/33357428/210541750-32d1eddf-be44-4dab-bea7-3246d893baad.jpeg\")
\nIn other words, the drug appears to help men and women, but hurt the general population. It seems nonsensical, or even impossible—which is why, of course, it is considered a paradox. Some people find it hard to believe that numbers could even be combined in such a way.
\n\nThe data seem to say that if we know the patient’s gender male or female we can prescribe the drug, but if the gender is unknown we should not! Obviously, that conclusion is ridiculous. If the drug helps men and women, it must help anyone; our lack of knowledge of the patient’s gender cannot make the drug harmful.
\n
Given the results of this study, then, should a doctor prescribe the drug for a woman? A man? A patient of unknown gender? Or consider a policy maker who is evaluating the drug’s overall effectiveness on the population. Should he/she use the recovery rate for the general population? Or should he/she use the recovery rates for the gendered sub-populations?
The answer is nowhere to be found in simple statistics.
In order to decide whether the drug will harm or help a patient, we first have to understand the story behind the data , the causal mechanism that led to, or generated, the results we see. For instance, suppose we knew an additional fact: Estrogen has a negative effect on recovery, so women are less likely to recover than men, regardless of the drug. In addition, as we can see from the data, women are significantly more likely to take the drug than men are. So, the reason the drug appears to be harmful overall is that, if we select a drug user at random, that person is more likely to be a woman and hence less likely to recover than a random person who does not take the drug. Put differently, being a woman is a common cause of both drug taking and failure to recover. Therefore, to assess the effectiveness, we need to compare subjects of the same gender, thereby ensuring that any difference in recovery rates between those who take the drug and those who do not is not ascribable to estrogen.
In fact, as statistics textbooks have traditionally (and correctly) warned students, correlation is not causation, so there is no statistical method that can determine the causal story from the data alone. Consequently, there is no statistical method that can aid in our decision.
The problems with traditional statistics when thinking about causality stems from a fundamental property of algebra, symmetry . The left-hand side of an equation equals the right-hand side (that’s the point of algebra). The equal sign implies symmetry. However, causality is fundamentally asymmetric i.e. causes lead to effects and not the other way around.
This distinction further implies that causal relations cannot be expressed in the language of probability and, hence, that any mathematical approach to causal analysis must acquire new notation – probability calculus is insufficient. To illustrate, the syntax of probability calculus does not permit us to express the simple fact that “symptoms do not cause diseases,” let alone draw mathematical conclusions from such facts. All we can say is that two events are dependent—meaning that if we find one, we can expect to encounter the other, but we cannot distinguish statistical dependence, quantified by the conditional probability P(disease|symptom) from causal dependence, for which we have no expression in standard probability calculus.
Let’s look at a simple example taken from Judea Pearl book itself . Suppose we model the relationship between a disease and the symptoms it produces, with the expression below. Y represents the severity of the symptoms, X the severity of the disease, m is the connection between the two, and b represents all other factors.
Using the rules of algebra we can invert the equation above to get the following expression.
![\"SymptomDisease2\"](\"https://user-images.githubusercontent.com/33357428/210628024-cf038674-9241-4076-9422-1e45b6d1199e.jpg\")
Here’s the problem, if we interpret the first equation as diseases cause symptoms, then we have to interpret the second equation as symptoms cause diseases! Which is of course not true.
Note : Linear relations are used here for illustration purposes only; they do not represent typical disease-symptom relations but illustrate the historical development of path analysis.
Before moving to the next set of blog posts , I should precisely define what a correlation is. I know you all are bored with listening this oft-repeated saying \"Correlation is not causation\" , so am i. So lets sort this out before moving to anything else!
Before moving ahead lets clarify one more thing : “Correlation” is often colloquially used as a synonym for statistical dependence. However, “correlation” is technically only a measure of linear statistical dependence. We will largely be using the term association to refer to statistical dependence from now on.
Lets take an example from Brady Neal's Causal Course book.
Say you happen upon some data that relates wearing shoes to bed and
waking up with a headache, as one does. It turns out that most times
that someone wears shoes to bed, that person wakes up with a headache.
And most times someone doesn’t wear shoes to bed, that person doesn’t
wake up with a headache. It is not uncommon for people to interpret
data like this (with associations) as meaning that wearing shoes to bed
causes people to wake up with headaches, especially if they are looking
for a reason to justify not wearing shoes to bed.
We can explain how wearing shoes to bed and headaches are associated
without either being a cause of the other. It turns out that they are
both caused by a common cause: drinking the night before. This kind of variables are called \"confounder\" or lurking variable. We will call this kind of association confounding association since the association is facilitated by a confounder.
![\"Headache](\"https://user-images.githubusercontent.com/33357428/210763817-267c4af8-6ac6-4392-8dcb-3196a94ed19a.jpg\")
The main problem motivating causal inference is that association is not causation.
If the two were the same, then causal inference would be easy. Traditional statistics and machine learning would already have causal inference solved, as measuring causation would be as simple as just looking at measures such as correlation and predictive performance in data.
Lets look at another example. Lets attempt to determine the causal effect of vitamin C intake on resistance to sickness. Let X be defined as a binary indicator representing if this subject intakes vitamin C and let Y be a binary indicator of being healthy (not getting sick). X is also referred to as the ‘treatment’ in a more general setting. Now, let C1 be the value of Y if X=1 (vitamin C is taken) and C0 be the value of Y if X=0 (vitamin C is not taken). We call C0 and C1 the potential outcomes of this experiment.
![\"Association](\"https://user-images.githubusercontent.com/33357428/210771012-3809777d-83cb-4027-a73c-495690ec5876.jpg\")
\nFor a single person, the causal effect of taking vitamin C in this context would be the difference between the expected outcome of taking vitamin C and the expected outcome of not taking vitamin C.
Causal Effect = E(C1) – E(C0)\nUnfortunately, we can only ever observe one of the possible outcomes C0 or C1. We cannot perfectly reset all conditions to see the result of the opposite treatment. Instead, we can use multiple samples and calculate the association between Vitamin C and being healthy.
Association = E(Y|X=1) – E(Y|X=0)\nAssociation as being (1+1+1+1)/4 – (0+0+0+0)/4 = 1
Causal effect, using the unobserved outcomes*, as being (4*0 + 4*1)/4 – (4*0 + 4*1)/4 = 0
We just calculated that, in this case, association does not equal causation. Observationally, there seems to be a perfect association between taking Vitamin C intake and being healthy. However, we can see there is no causal effect because we are privileged with the values of the unobserved outcomes. This inequality could be explained by considering that the people that stayed healthy practiced healthy habits which included taking Vitamin C.
Okay, one more motivating example :
In response to a large study that studied the relationship between income and life expectancy, Vox published an article titled “Want to live longer, even if you’re poor? Then move to a big city in California” (Klein, 2016). However, as is implied by the title of the study “The Association Between Income and Life Expectancy in the United States, 2001-2014”, the study did not presume to make this recommendation and in fact the closest statement made to the Vox recommendation was “… the strongest pattern in the data was that low-income individuals tend to live longest (and have more healthful behaviors) in cities with highly educated populations, high incomes, and high levels of government expenditures, such as New York, New York, and San Francisco, California.” (Chetty et al., 2016).
Similar to the example regarding vitamin C and health, this study only found associative effects. However, just like it is incorrect to say that vitamin C causes a person to be healthy, it is also incorrect to say that moving to California will cause you to live longer.
That's all for now ! , Hope you enjoyed reading so far !
In the next blog post, we will further investigate the differences between association and causation, by starting with Pearl’s three-level causal hierarchy. That will be much interesting to watch out for !
This post is the second post of the series on Causal Machine Learning. In the last blog post i have given a brief introduction on Causal Machine Learning. In this we will uncover the WHY !. This blog post is based on the work of Judea Pearl. I will discuss how naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference). As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
\n\nCorrellation Is Not Causation !!!
\n
I know you are bored of this like every other, hearing this oft-repeated saying “correlation does not imply causation”. I will formally illustrate why this is in this post.
In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of statistical correlations (that are due to chance) with no causal implications. One of them is listed below.
\nWe clearly know No of people who drowned while in a swimming pool has nothing to do with the power generated by US nuclear power plants !!. You can find more interesting “Spurious Correlations” from Tyler Vigen website.
Spurious correlations are well-known in statistics, so it’s easy (Somewhat !) to be on the lookout for it. Lets see a little known paradox in statistics.
Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. Lets look at an example of Simpson's paradox from Wikipedia itself.
\nIn both 1995 and 1996, Justice had a higher batting average (in bold type) than Jeter did. However, when the two baseball seasons are combined, Jeter shows a higher batting average than Justice.
\n\nSame data give contradictory conclusions depending on how you look at them! .
\n
Simpson’s paradox highlights that how you look at your data matters. So the question becomes, how do we partition data? Although there is no standard method in statistics for this, causal inference provides a formalism to handle this problem. It all boils down to causal effects, which quantify the impact a variable has on another variable after adjusting for the appropriate confounders.
Let us look at an example from “The Book of Why: The New Science of Cause and Effect” by Judea Pearl , and a post the legend himself posted in his twitter handle.
Consider the below study that measures weekly exercise and cholesterol in various age groups. When we plot exercise on the X-axis and cholesterol on the Y-axis and segregate by age, as in left side of Fig , we see that there is a general trend downward in each group; the more young people exercise, the lower their cholesterol is, and the same applies for middle-aged people and the elderly. If, however, we use the same scatter plot, but we don’t segregate by gender (as in right side of Fig), we see a general trend upward; the more a person exercises, the higher their cholesterol is.
\n\n\nExcercise appears to be beneficial in each age group but harmful in the population as a whole !!!
\n
To resolve this problem, we once again turn to the story behind the data. If we know that older people, who are more likely to exercise are also more likely to have high cholesterol regardless of exercise, then the reversal is easily explained, and easily resolved. Age is a common cause of both treatment (exercise) and outcome (cholesterol). So we should look at the age-segregated data in order to compare same-age people and thereby eliminate the possibility that the high exercisers in each group we examine are more likely to have high cholesterol due to their age, and not due to exercising.
However, please do not get confused , segregated data does not always give the correct answer.
Lets look at another example from the Causal Inference In Statistics book by Pearl.
In the classical example used by Simpson (1951), a group of sick patients are given the option to try a new drug. Among those who took the drug, a lower percentage recovered than among those who did not. However, when we partition by gender, we see that more men taking the drug recover than do men are not taking the drug, and more women taking the drug recover than do women are not taking the drug!
\nIn other words, the drug appears to help men and women, but hurt the general population. It seems nonsensical, or even impossible—which is why, of course, it is considered a paradox. Some people find it hard to believe that numbers could even be combined in such a way.
\n\nThe data seem to say that if we know the patient’s gender male or female we can prescribe the drug, but if the gender is unknown we should not! Obviously, that conclusion is ridiculous. If the drug helps men and women, it must help anyone; our lack of knowledge of the patient’s gender cannot make the drug harmful.
\n
Given the results of this study, then, should a doctor prescribe the drug for a woman? A man? A patient of unknown gender? Or consider a policy maker who is evaluating the drug’s overall effectiveness on the population. Should he/she use the recovery rate for the general population? Or should he/she use the recovery rates for the gendered sub-populations?
The answer is nowhere to be found in simple statistics.
In order to decide whether the drug will harm or help a patient, we first have to understand the story behind the data , the causal mechanism that led to, or generated, the results we see. For instance, suppose we knew an additional fact: Estrogen has a negative effect on recovery, so women are less likely to recover than men, regardless of the drug. In addition, as we can see from the data, women are significantly more likely to take the drug than men are. So, the reason the drug appears to be harmful overall is that, if we select a drug user at random, that person is more likely to be a woman and hence less likely to recover than a random person who does not take the drug. Put differently, being a woman is a common cause of both drug taking and failure to recover. Therefore, to assess the effectiveness, we need to compare subjects of the same gender, thereby ensuring that any difference in recovery rates between those who take the drug and those who do not is not ascribable to estrogen.
In fact, as statistics textbooks have traditionally (and correctly) warned students, correlation is not causation, so there is no statistical method that can determine the causal story from the data alone. Consequently, there is no statistical method that can aid in our decision.
The problems with traditional statistics when thinking about causality stems from a fundamental property of algebra, symmetry . The left-hand side of an equation equals the right-hand side (that’s the point of algebra). The equal sign implies symmetry. However, causality is fundamentally asymmetric i.e. causes lead to effects and not the other way around.
This distinction further implies that causal relations cannot be expressed in the language of probability and, hence, that any mathematical approach to causal analysis must acquire new notation – probability calculus is insufficient. To illustrate, the syntax of probability calculus does not permit us to express the simple fact that “symptoms do not cause diseases,” let alone draw mathematical conclusions from such facts. All we can say is that two events are dependent—meaning that if we find one, we can expect to encounter the other, but we cannot distinguish statistical dependence, quantified by the conditional probability P(disease|symptom) from causal dependence, for which we have no expression in standard probability calculus.
Let’s look at a simple example taken from Judea Pearl book itself . Suppose we model the relationship between a disease and the symptoms it produces, with the expression below. Y represents the severity of the symptoms, X the severity of the disease, m is the connection between the two, and b represents all other factors.
Using the rules of algebra we can invert the equation above to get the following expression.
Here’s the problem, if we interpret the first equation as diseases cause symptoms, then we have to interpret the second equation as symptoms cause diseases! Which is of course not true.
Note : Linear relations are used here for illustration purposes only; they do not represent typical disease-symptom relations but illustrate the historical development of path analysis.
Before moving to the next set of blog posts , I should precisely define what a correlation is. I know you all are bored with listening this oft-repeated saying \"Correlation is not causation\" , so am i. So lets sort this out before moving to anything else!
Before moving ahead lets clarify one more thing : “Correlation” is often colloquially used as a synonym for statistical dependence. However, “correlation” is technically only a measure of linear statistical dependence. We will largely be using the term association to refer to statistical dependence from now on.
Lets take an example from Brady Neal's Causal Course book.
Say you happen upon some data that relates wearing shoes to bed and
waking up with a headache, as one does. It turns out that most times
that someone wears shoes to bed, that person wakes up with a headache.
And most times someone doesn’t wear shoes to bed, that person doesn’t
wake up with a headache. It is not uncommon for people to interpret
data like this (with associations) as meaning that wearing shoes to bed
causes people to wake up with headaches, especially if they are looking
for a reason to justify not wearing shoes to bed.
We can explain how wearing shoes to bed and headaches are associated
without either being a cause of the other. It turns out that they are
both caused by a common cause: drinking the night before. This kind of variables are called \"confounder\" or lurking variable. We will call this kind of association confounding association since the association is facilitated by a confounder.
The main problem motivating causal inference is that association is not causation.
If the two were the same, then causal inference would be easy. Traditional statistics and machine learning would already have causal inference solved, as measuring causation would be as simple as just looking at measures such as correlation and predictive performance in data.
Lets look at another example. Lets attempt to determine the causal effect of vitamin C intake on resistance to sickness. Let X be defined as a binary indicator representing if this subject intakes vitamin C and let Y be a binary indicator of being healthy (not getting sick). X is also referred to as the ‘treatment’ in a more general setting. Now, let C1 be the value of Y if X=1 (vitamin C is taken) and C0 be the value of Y if X=0 (vitamin C is not taken). We call C0 and C1 the potential outcomes of this experiment.
\nFor a single person, the causal effect of taking vitamin C in this context would be the difference between the expected outcome of taking vitamin C and the expected outcome of not taking vitamin C.
Causal Effect = E(C1) – E(C0)\nUnfortunately, we can only ever observe one of the possible outcomes C0 or C1. We cannot perfectly reset all conditions to see the result of the opposite treatment. Instead, we can use multiple samples and calculate the association between Vitamin C and being healthy.
Association = E(Y|X=1) – E(Y|X=0)\nAssociation as being (1+1+1+1)/4 – (0+0+0+0)/4 = 1
Causal effect, using the unobserved outcomes*, as being (4*0 + 4*1)/4 – (4*0 + 4*1)/4 = 0
We just calculated that, in this case, association does not equal causation. Observationally, there seems to be a perfect association between taking Vitamin C intake and being healthy. However, we can see there is no causal effect because we are privileged with the values of the unobserved outcomes. This inequality could be explained by considering that the people that stayed healthy practiced healthy habits which included taking Vitamin C.
Okay, one more motivating example :
In response to a large study that studied the relationship between income and life expectancy, Vox published an article titled “Want to live longer, even if you’re poor? Then move to a big city in California” (Klein, 2016). However, as is implied by the title of the study “The Association Between Income and Life Expectancy in the United States, 2001-2014”, the study did not presume to make this recommendation and in fact the closest statement made to the Vox recommendation was “… the strongest pattern in the data was that low-income individuals tend to live longest (and have more healthful behaviors) in cities with highly educated populations, high incomes, and high levels of government expenditures, such as New York, New York, and San Francisco, California.” (Chetty et al., 2016).
Similar to the example regarding vitamin C and health, this study only found associative effects. However, just like it is incorrect to say that vitamin C causes a person to be healthy, it is also incorrect to say that moving to California will cause you to live longer.
That's all for now ! , Hope you enjoyed reading so far !
In the next blog post, we will further investigate the differences between association and causation, by starting with Pearl’s three-level causal hierarchy. That will be much interesting to watch out for !
\n\nTo Build Truly Intelligent Machines, Teach Them Cause and Effect.
\n
\n\nCorrelation is not Causation!.
\n
This post is the first of the series on Causal Machine Learning. I will start with the very basics of causal inference , Provide some basic background in Bayesian networks/graphical models , Show how graphical models can be used in causal inference and Describe application scenarios , case studies and the practical difficulties in Causal Inference. I will follow step by step approach and keep the things as simple as possible with lot of practical tutorials. In each of the session i will provide additional learning materials and references , which should help you to keep the flow. In this blog post , i will present Why we need causality in Machine Learning! . Enjoy the reading!
Deep learning techniques do a good job at building models by correlating data points. But many AI researchers believe that more work needs to be done to understand causation and not just correlation. The field causal deep learning -- useful in determining why something happened -- is still in its infancy, and it is much more difficult to automate than neural networks.
\n\nEven though we can observe correlation, it does not prove causation.
\n
Correlation: measures the relationship between two things.\nCausation: means that one thing will cause the other to happen.\nI know that when I see X, I will see Y (Association/Correlation)\nI know that X causes Y (Causality)\nThe distinctions between the two can have important implications. In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of correlations that are due to chance. One of the interesting spurious correlation is listed below.
![](\"https://user-images.githubusercontent.com/33357428/209831536-c820c96b-3347-4a30-b0f0-dca9e409e6f0.jpeg\")
While these two variables have a correlation of 95.23%, it is highly unrealistic to think that a degree in mathematics caused an increase in the amount of Uranium stored in the United States!
![](\"https://user-images.githubusercontent.com/33357428/208087796-ad8b5727-8db3-4d71-9881-8a4f56169964.png\")
\n\n“If Correlation Doesn’t Imply Causation, Then What Does?”
\n
We will uncover this in the upcoming blog posts.
The excitement in the field has been kindled by Judea Pearl, a professor at UCLA, who did some of the formative work on implementing Bayesian networks for statistical analysis. More recently he has been developing a framework for diagramming causation and teasing apart the factors that contribute to observed events in a computable framework.
![](\"https://user-images.githubusercontent.com/33357428/208084630-fd654f0e-0bc2-4576-844a-5d398c75d5a8.jpg\")
In his latest book, “The Book of Why: The New Science of Cause and Effect,” he argues that artificial intelligence has been handicapped by an incomplete understanding of what intelligence really is. In his new book, Pearl, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. Instead of the mere ability to correlate fever and malaria, machines need the capacity to reason that malaria causes fever. Pearl expects that causal reasoning could provide machines with human-level intelligence.
![\"The](\"https://user-images.githubusercontent.com/33357428/208090292-28654bde-487b-4f4e-b1ea-dd957c8ae7f0.jpg\")
Understanding why requires understanding of the whats, the wheres, and the whens. The hows, however, seem to be an implementation of the whys !!
Machine learning is a type of artificial intelligence that involves training algorithms to recognise patterns in data and make predictions or decisions based on those patterns. The goal of machine learning is to build models that can generalise to new data and make accurate predictions or decisions.
The main difference between machine learning and causal machine learning is the focus of the analysis. While traditional machine learning aims to predict an outcome based on patterns in the data, causal machine learning aims to identify the specific variables that are causing an outcome and how they contribute to it. This involves identifying and modelling the causal relationships between variables, rather than just the statistical relationships.
Causal machine learning is a type of machine learning that focuses on identifying the cause-and-effect relationships between variables. In other words, it aims to identify the factors that influence a particular outcome and how they influence it.
Causal machine learning can be used to improve the accuracy of predictive models by taking into account the underlying causes of the outcomes being predicted. It is also useful in a variety of applications, such as evaluating the effectiveness of interventions, predicting the impact of policy changes, and identifying the factors that contribute to certain outcomes.
Explainable AI (XAI) is a type of artificial intelligence (AI) that is designed to be transparent and interpretable, meaning that it can provide explanations for its decisions and actions. The goal of XAI is to make AI systems more transparent and understandable to humans, so that they can be trusted and used more effectively.
Causal AI, on the other hand, is a type of AI that focuses on identifying the cause-and-effect relationships between variables. It is a sub field of machine learning that aims to identify the specific factors that influence a particular outcome and how they contribute to it. One key difference between explainable AI and causal AI is their focus. Explainable AI is concerned with making AI systems more transparent and interpretable, while causal AI is focused on identifying the causes and effects of various phenomena.
Another difference is the techniques used. Explainable AI often employs techniques such as feature importance, sensitivity analysis, and model interpretation methods to provide insights into the decision-making process of AI systems. Causal AI, on the other hand, typically uses techniques such as structural equation modelling, instrumental variables, and counterfactual analysis to identify the causal relationships between variables.
While XAI and causal AI are related in that they both aim to increase our understanding of how AI systems work, they are distinct concepts. XAI focuses on providing explanations for the decisions and predictions made by AI systems, while causal AI focuses on identifying the underlying causes of outcomes.
One way in which XAI and causal AI can be related is that XAI methods, such as counterfactual analysis, can be used to identify the causes of particular outcomes or decisions. However, it is important to note that not all XAI methods are causal in nature, and not all causal AI systems are necessarily explainable. Overall, both XAI and causal AI are important tools for improving our understanding of how AI systems work and for increasing the transparency and trustworthiness of AI systems.
Causal machine learning can be applied in many different fields and industries to identify and understand the causal relationships between variables. Here are a few examples of the application of causal machine learning:
These are just a few examples of the many ways in which causal machine learning can be applied. There are many more applications in various fields and industries.
In this post I have provided an introduction to Causal machine learning. I have tried to be as compact as possible. Below you can find some additional resources if you want to know more about Causal machine learning. Hope you enjoyed reading so far!.
Course : Machine Learning & Causal Inference: A Short Course
\nTexbook : Causal inference in statistics:An overview. Judea Pearl
\nIndustry : Causality and Machine Learning: Microsoft Research
\n\n\nTo Build Truly Intelligent Machines, Teach Them Cause and Effect.
\n
\n\nCorrelation is not Causation!.
\n
This post is the first of the series on Causal Machine Learning. I will start with the very basics of causal inference , Provide some basic background in Bayesian networks/graphical models , Show how graphical models can be used in causal inference and Describe application scenarios , case studies and the practical difficulties in Causal Inference. I will follow step by step approach and keep the things as simple as possible with lot of practical tutorials. In each of the session i will provide additional learning materials and references , which should help you to keep the flow. In this blog post , i will present Why we need causality in Machine Learning! . Enjoy the reading!
Deep learning techniques do a good job at building models by correlating data points. But many AI researchers believe that more work needs to be done to understand causation and not just correlation. The field causal deep learning -- useful in determining why something happened -- is still in its infancy, and it is much more difficult to automate than neural networks.
\n\nEven though we can observe correlation, it does not prove causation.
\n
Correlation: measures the relationship between two things.\nCausation: means that one thing will cause the other to happen.\nI know that when I see X, I will see Y (Association/Correlation)\nI know that X causes Y (Causality)\nThe distinctions between the two can have important implications. In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of correlations that are due to chance. One of the interesting spurious correlation is listed below.
While these two variables have a correlation of 95.23%, it is highly unrealistic to think that a degree in mathematics caused an increase in the amount of Uranium stored in the United States!
\n\n“If Correlation Doesn’t Imply Causation, Then What Does?”
\n
We will uncover this in the upcoming blog posts.
The excitement in the field has been kindled by Judea Pearl, a professor at UCLA, who did some of the formative work on implementing Bayesian networks for statistical analysis. More recently he has been developing a framework for diagramming causation and teasing apart the factors that contribute to observed events in a computable framework.
In his latest book, “The Book of Why: The New Science of Cause and Effect,” he argues that artificial intelligence has been handicapped by an incomplete understanding of what intelligence really is. In his new book, Pearl, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. Instead of the mere ability to correlate fever and malaria, machines need the capacity to reason that malaria causes fever. Pearl expects that causal reasoning could provide machines with human-level intelligence.
Understanding why requires understanding of the whats, the wheres, and the whens. The hows, however, seem to be an implementation of the whys !!
Machine learning is a type of artificial intelligence that involves training algorithms to recognise patterns in data and make predictions or decisions based on those patterns. The goal of machine learning is to build models that can generalise to new data and make accurate predictions or decisions.
The main difference between machine learning and causal machine learning is the focus of the analysis. While traditional machine learning aims to predict an outcome based on patterns in the data, causal machine learning aims to identify the specific variables that are causing an outcome and how they contribute to it. This involves identifying and modelling the causal relationships between variables, rather than just the statistical relationships.
Causal machine learning is a type of machine learning that focuses on identifying the cause-and-effect relationships between variables. In other words, it aims to identify the factors that influence a particular outcome and how they influence it.
Causal machine learning can be used to improve the accuracy of predictive models by taking into account the underlying causes of the outcomes being predicted. It is also useful in a variety of applications, such as evaluating the effectiveness of interventions, predicting the impact of policy changes, and identifying the factors that contribute to certain outcomes.
Explainable AI (XAI) is a type of artificial intelligence (AI) that is designed to be transparent and interpretable, meaning that it can provide explanations for its decisions and actions. The goal of XAI is to make AI systems more transparent and understandable to humans, so that they can be trusted and used more effectively.
Causal AI, on the other hand, is a type of AI that focuses on identifying the cause-and-effect relationships between variables. It is a sub field of machine learning that aims to identify the specific factors that influence a particular outcome and how they contribute to it. One key difference between explainable AI and causal AI is their focus. Explainable AI is concerned with making AI systems more transparent and interpretable, while causal AI is focused on identifying the causes and effects of various phenomena.
Another difference is the techniques used. Explainable AI often employs techniques such as feature importance, sensitivity analysis, and model interpretation methods to provide insights into the decision-making process of AI systems. Causal AI, on the other hand, typically uses techniques such as structural equation modelling, instrumental variables, and counterfactual analysis to identify the causal relationships between variables.
While XAI and causal AI are related in that they both aim to increase our understanding of how AI systems work, they are distinct concepts. XAI focuses on providing explanations for the decisions and predictions made by AI systems, while causal AI focuses on identifying the underlying causes of outcomes.
One way in which XAI and causal AI can be related is that XAI methods, such as counterfactual analysis, can be used to identify the causes of particular outcomes or decisions. However, it is important to note that not all XAI methods are causal in nature, and not all causal AI systems are necessarily explainable. Overall, both XAI and causal AI are important tools for improving our understanding of how AI systems work and for increasing the transparency and trustworthiness of AI systems.
Causal machine learning can be applied in many different fields and industries to identify and understand the causal relationships between variables. Here are a few examples of the application of causal machine learning:
These are just a few examples of the many ways in which causal machine learning can be applied. There are many more applications in various fields and industries.
In this post I have provided an introduction to Causal machine learning. I have tried to be as compact as possible. Below you can find some additional resources if you want to know more about Causal machine learning. Hope you enjoyed reading so far!.
Course : Machine Learning & Causal Inference: A Short Course
\nTexbook : Causal inference in statistics:An overview. Judea Pearl
\nIndustry : Causality and Machine Learning: Microsoft Research
\nThis post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
![\"causaldiscoveryalgo\"](\"https://user-images.githubusercontent.com/33357428/219991483-bd875dbc-d04d-4bca-a1db-1b553162f199.png\")
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
![\"pcalgo\"](\"https://user-images.githubusercontent.com/33357428/219992079-8d7af535-e4bd-48e0-93a6-a993f65b84cb.png\")
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.
This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.