![\"causaldiscoveryalgo\"](\"https://user-images.githubusercontent.com/33357428/219991483-bd875dbc-d04d-4bca-a1db-1b553162f199.png\")
This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
![\"pcalgo\"](\"https://user-images.githubusercontent.com/33357428/219992079-8d7af535-e4bd-48e0-93a6-a993f65b84cb.png\")
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.
This post is the fifth post of the series on Causal Machine Learning. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful. In this post Causal Discovery will be discussed in detail. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
Causal inference focuses on estimating the causal effect of a specific intervention or exposure on an outcome.\nCausal discovery focuses on identifying the underlying causal relationships between variables in a system.\nCausal inference, which aims to answer questions involving cause and effect. As stated before, the starting point for all causal inference is a causal model. Usually, however, we don’t have a good causal model in hand. This is where Causal Discovery can be helpful.
Causal discovery aims to infer causal structure from data. In other words, given a dataset, derive a causal model that describes it.
Finding causal relationships is one of the fundamental tasks in science. A widely used approach is randomized experiments. For example, to examine whether a recently developed medicine is useful for cancer treatment, researchers recruit subjects and randomly divide subjects into two groups. One is the control group, where the subjects are given placebo, and the other is the treatment group, where the subjects are given the newly developed drug. The reason of randomization is to remove possible effects from confounders. For example, age can be one of the possible confounders which affects both taking the drug or not and the treatment effect. Thus, in practical experiments, we should keep the distribution of ages in the two groups almost the same.
However, in many cases, randomized experiments are very expensive and hard to implement, and sometimes it may even involve ethical issues. In recent decades, inferring causal relations from purely observational data, known as the task of causal discovery, has drawn much attention in machine learning, philosophy, statistics, and computer science.
Causal discovery is an example of an inverse problem. This is like predicting the shape of an ice cube based on the puddle it left on the kitchen counter. Clearly, this is a hard problem, since any number of shapes could generate the same puddle. Connecting this to causality, the puddle of water is like statistical associations embedded in data, and the ice cube is the like underlying causal model.
The usual approach to solving inverse problems is to make assumptions about what you are trying to uncover. This narrows down the possible solutions and hopefully makes the problem solvable. There are four common assumptions made across causal discovery algorithms.
👉 Acyclicity — Causal structure can be represented by DAG (G)
\n👀 Markov Property — All nodes are independent of their non-descendants when conditioned on their parents
\n🙃 Faithfulness — All conditional independences in true underlying distribution p are represented in G
\n👍 Sufficiency — Any pair of nodes in G has no common external cause
\nAlthough these assumptions help narrow down the number of possible models, they do not fully solve the problem. This is where a few tricks/tests for causal discovery are helpful. There is no single method for causal discovery that dominates all others. Although most methods use the assumptions above (perhaps even more), the details of different algorithms can vary tremendously. A taxonomy of algorithms based on the following tricks is given in the figure below.
One of these earliest causal discovery algorithms is the PC algorithm named after its authors Peter Spirtes and Clark Glymour. This algorithm (and others like it) use the idea that two statistically independent variables are not causally linked. The PC algorithm is illustrative of this first trick. An outline of the algorithm is given in the figure below.
The first step is to form a fully connected, undirected graph using every variable in the dataset. Next, edges are deleted if the corresponding variables are independent. Then, connected edges undergo conditional independence testing e.g. independence test of the bottom and far right node conditioned on the middle node in the figure above (step 2).
If conditioning on a variable kills the dependence, that variable is added to the Separation set for those two variables. Depending on the size of the graph, conditional independence testing will continue (i.e. condition on more variables) until there are no more candidates for testing.
Next, colliders (i.e. X → Y ← Z) are oriented based on the Separation set of node pairs. Finally, the remaining edges are directed based on 2 constraints, 1) no new v-structures, and 2) no directed cycles can be formed.
A greedy search is a way to navigate a space such that you always move in a direction that seems most beneficial based on the local surroundings. Although greedy searches cannot guarantee an optimal solution, for most problems the space of possible DAGs is so big that finding a true optimal solution is intractable. The Greedy Equivalence Search (GES) algorithm uses this trick. GES starts with an empty graph and iteratively adds directed edges such that the improvement in a model fitness measure (i.e. score) is maximized. An example score is the Bayesian Information Criterion (BIC)
\n\nA fundamental property of causality is asymmetry. A could cause B, but B may not cause A. There is a large space of algorithms that leverage this idea to select between causal model candidates.
\n
Functional asymmetry assumes models that better fit a relationship are better candidates. For example, given two variables X and Y, the nonlinear additive noise model (NANM) performs a nonlinear regression between X and Y, e.g. y = f(x) + n, where n = noise/residual, in both directions. The model (i.e. causation) is then accepted if the potential cause (e.g. x) is independent of the noise term (e.g. n).
There is no way I could fit a comprehensive review of causal discovery in a short blog post. Despite being young, causal discovery is a promising field that may help bridge the gap between machine and human knowledge.
Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models.
","url":"http://localhost:2368/explaining-bayesian-neural-networks/","canonical_url":null,"uuid":"6bef7972-5e59-4ea0-8ccd-99292bb992ab","codeinjection_foot":null,"codeinjection_head":null,"codeinjection_styles":null,"comment_id":"63eeea8172a3c427182edcc0","reading_time":0,"send_email_when_published":null,"email_subject":null,"childHtmlRehype":{"html":"Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models.
","htmlAst":{"type":"root","children":[{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Explainable AI refers to models that can easily be interpreted by humans. Bayesian inferencing is a natural choice to obtain explanations of trained deep learning models."}]}],"data":{"quirksMode":false}},"tableOfContents":[]},"featureImageSharp":{"base":"photo-1646583288948-24548aedffd8.jpg","publicURL":"/static/e6860d2b93d53230bfd7e05394764440/photo-1646583288948-24548aedffd8.jpg","imageMeta":{"width":2000,"height":1333},"childImageSharp":{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAANABQDASIAAhEBAxEB/8QAGAAAAwEBAAAAAAAAAAAAAAAAAAIDAQT/xAAVAQEBAAAAAAAAAAAAAAAAAAAAAf/aAAwDAQACEAMQAAABTnWKSMD/xAAYEAEBAQEBAAAAAAAAAAAAAAABAAIDEP/aAAgBAQABBQJbrlyS2lfP/8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAwEBPwE//8QAFBEBAAAAAAAAAAAAAAAAAAAAEP/aAAgBAgEBPwE//8QAGBAAAgMAAAAAAAAAAAAAAAAAAAERIEH/2gAIAQEABj8CJe1//8QAGRAAAgMBAAAAAAAAAAAAAAAAAAEQETEh/9oACAEBAAE/IeYkoD0ZRqOP/9oADAMBAAIAAwAAABDPP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQMBAT8QP//EABQRAQAAAAAAAAAAAAAAAAAAABD/2gAIAQIBAT8QP//EABwQAQEBAAEFAAAAAAAAAAAAAAERABAhMUFRYf/aAAgBAQABPxAaswSgie+peDzmEip9blb33//Z","aspectRatio":1.4957264957264957,"src":"/static/e6860d2b93d53230bfd7e05394764440/ea4ab/photo-1646583288948-24548aedffd8.jpg","srcSet":"/static/e6860d2b93d53230bfd7e05394764440/477ba/photo-1646583288948-24548aedffd8.jpg 175w,\n/static/e6860d2b93d53230bfd7e05394764440/06776/photo-1646583288948-24548aedffd8.jpg 350w,\n/static/e6860d2b93d53230bfd7e05394764440/ea4ab/photo-1646583288948-24548aedffd8.jpg 700w,\n/static/e6860d2b93d53230bfd7e05394764440/3055e/photo-1646583288948-24548aedffd8.jpg 1050w,\n/static/e6860d2b93d53230bfd7e05394764440/eff08/photo-1646583288948-24548aedffd8.jpg 1400w,\n/static/e6860d2b93d53230bfd7e05394764440/4e5f3/photo-1646583288948-24548aedffd8.jpg 2000w","srcWebp":"/static/e6860d2b93d53230bfd7e05394764440/89afa/photo-1646583288948-24548aedffd8.webp","srcSetWebp":"/static/e6860d2b93d53230bfd7e05394764440/9fca7/photo-1646583288948-24548aedffd8.webp 175w,\n/static/e6860d2b93d53230bfd7e05394764440/37a4e/photo-1646583288948-24548aedffd8.webp 350w,\n/static/e6860d2b93d53230bfd7e05394764440/89afa/photo-1646583288948-24548aedffd8.webp 700w,\n/static/e6860d2b93d53230bfd7e05394764440/78e7a/photo-1646583288948-24548aedffd8.webp 1050w,\n/static/e6860d2b93d53230bfd7e05394764440/03d34/photo-1646583288948-24548aedffd8.webp 1400w,\n/static/e6860d2b93d53230bfd7e05394764440/49d6b/photo-1646583288948-24548aedffd8.webp 2000w","sizes":"(max-width: 700px) 100vw, 700px"}}}},"next":{"id":"Ghost__Post__63b57b53bdc6867fe35526d6","title":"Causal Machine Learning - Part 4","slug":"causal-machine-learning-part-4","featured":false,"feature_image":"https://images.unsplash.com/photo-1535378620166-273708d44e4c?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDI0fHxyb2JvdHxlbnwwfHx8fDE2NzI4Mzc3Nzg&ixlib=rb-4.0.3&q=80&w=2000","excerpt":"This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !","custom_excerpt":"This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !","visibility":"public","created_at_pretty":"4 Jan 2023","published_at_pretty":"4 Jan 2023","updated_at_pretty":"20 Feb 2023","created_at":"2023-01-04T18:42:51.000+05:30","published_at":"2023-01-04T18:44:43.000+05:30","updated_at":"2023-02-20T06:14:06.000+05:30","meta_title":null,"meta_description":null,"og_description":null,"og_image":null,"og_title":null,"twitter_description":null,"twitter_image":null,"twitter_title":null,"authors":[{"slug":"amaljith","url":"http://localhost:2368/author/amaljith/","name":"Amaljith","bio":"Research Scholar @ IIT Kharagpur","cover_image":null,"profile_image":"http://localhost:2368/content/images/2022/09/Screenshot-from-2022-09-07-18-00-00.png","location":null,"website":null,"twitter":null,"facebook":null,"meta_title":null,"meta_description":null,"coverImageSharp":null,"profileImageSharp":null}],"primary_author":{"slug":"amaljith","url":"http://localhost:2368/author/amaljith/","name":"Amaljith","bio":"Research Scholar @ IIT Kharagpur","cover_image":null,"profile_image":"http://localhost:2368/content/images/2022/09/Screenshot-from-2022-09-07-18-00-00.png","location":null,"website":null,"twitter":null,"facebook":null,"meta_title":null,"meta_description":null,"coverImageSharp":null,"profileImageSharp":{"base":"Screenshot-from-2022-09-07-18-00-00.png","publicURL":"/static/28e31bfedd96b4afe90237d2c1f700c3/Screenshot-from-2022-09-07-18-00-00.png","imageMeta":{"width":316,"height":237},"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAAPCAYAAADkmO9VAAAACXBIWXMAAAsTAAALEwEAmpwYAAAEJElEQVQ4yyXO21fTBQDA8d9fUh5LQAW1eEImbIPJbSB3kZtijjSBxNvAGSAqlJZ64GAHIQ0VjkKICCiCQ8cYsYFjYwhsYxeGjJsopmEnH76dYw+f94/w2jmKfaSX9wtTrC1a+bBk5d83dlbnzFhHehnq62TBZeLjGyfvFqf4a9nGstuAx/oM96Qai76NieFu5u0jLLrNCCsOI47nat56J1hbtLH60kTfg5tUV6hQFeSSFLGTgtwcBtUteGc0zLoGmXNpmXU8Yd6txTOlxvq8hzmr/hNhyTaC06hmya3nn6VxHKMPKS08RFF+HgWK/aiO5JP3TQ4xMgn3mqt599bC8pyBRY+OxRkdXkc/rrE+POZ+nMY+hFXHKPPWP5m3a/i4Yqax9kfCREEkyKPJSk3mQFYmOelpRISGEBMuwahv4/WC4dNuwa3l1Us906O9jGs7sWg7EHQdv6NurWVmvJv33iHKTuSyft06tm3xJygwEIlIxDb/zUhFIvw2+HDrxkX+XjUx5x7A6+pn3qXBZnyIoacFTftNhKarlTxprsMz3sPbWR2jmiYiJSICt27lqwB/tmza+L8Af5KkYsYNHXhcvbidahy2x0ya2jFqm7lxqYxrF04jvLQMYB16jKazHrOulRWHjiJFOus/+5yAjX5s8vNhs58vfj5foooO43LJMaovncI1/ZhpWxcTlk6mRjrpa7uOrqsJYcaiY25imKftv6F52IBx4D61Z46y6Yv1+PpsYLOvD36+G0gJDuRcrIS0nTvY8XUAl66osDq60A3e5sHtXzCoW3CY+hHeeKdZdL5gytDDgsOAe1zL9KiawowEsoK2URi+HWWMmKvZuzibKuNcqoTiWBEXShToLHfo7K6h+9413DYzS/MehJnJYZxjOowDHdjHnjE53M2kUU1/zVmu742lOlNOkyKRmtzd1KkUtObFcacgmUetVTT0/kTDvQpemPoxDWmoralG8NrNvJqZwmbS8Xp5FrfNhMtuxtNei75Kia3xAmtdVZgaf6axvprKvXH8kB5FXd1lFMcyuVp7Hnd9E0+bb5Cdvgfhw4oXPq5hGR5kzGzkUVcPVeWlPK8tx/BrCfcrjnI+bx+KPSmcPJxP2QklFUolRfmHyf/uEEXK49y9XMndaxdJjpQirCy4WLCPoe38A4Nez8XySjISU5CFhCIO3k5wUDASkZiUuHj2Z2ahyN5HvuIgRUeOcUZZTOlJJeUlKgbU7Txta0AYbLnFqewMrpSepvhIITm700nblYg8IpJIWRjyiAjiI6JJjJKTGBtHcnwCqYlJpCelkJOWwfe533Jw337Kik9g1N5HqC9RcfyAgvTUPcRHyYmWypCFiJGKQggNDkYsEiEJEhEmCiE8REyYWIJMKiVCEoY8fCcJUTHEx8QgCw0l/8Be/gOTiTiUD46AHAAAAABJRU5ErkJggg==","aspectRatio":1.3333333333333333,"src":"/static/28e31bfedd96b4afe90237d2c1f700c3/6ccb0/Screenshot-from-2022-09-07-18-00-00.png","srcSet":"/static/28e31bfedd96b4afe90237d2c1f700c3/7d89d/Screenshot-from-2022-09-07-18-00-00.png 28w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/f4091/Screenshot-from-2022-09-07-18-00-00.png 55w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/6ccb0/Screenshot-from-2022-09-07-18-00-00.png 110w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/30481/Screenshot-from-2022-09-07-18-00-00.png 165w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/33bd6/Screenshot-from-2022-09-07-18-00-00.png 220w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/d398b/Screenshot-from-2022-09-07-18-00-00.png 316w","srcWebp":"/static/28e31bfedd96b4afe90237d2c1f700c3/8678c/Screenshot-from-2022-09-07-18-00-00.webp","srcSetWebp":"/static/28e31bfedd96b4afe90237d2c1f700c3/59cda/Screenshot-from-2022-09-07-18-00-00.webp 28w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/7da75/Screenshot-from-2022-09-07-18-00-00.webp 55w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/8678c/Screenshot-from-2022-09-07-18-00-00.webp 110w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/f282e/Screenshot-from-2022-09-07-18-00-00.webp 165w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/a7b21/Screenshot-from-2022-09-07-18-00-00.webp 220w,\n/static/28e31bfedd96b4afe90237d2c1f700c3/fb2b8/Screenshot-from-2022-09-07-18-00-00.webp 316w","sizes":"(max-width: 110px) 100vw, 110px"}}}},"primary_tag":{"slug":"machine-learning","url":"http://localhost:2368/tag/machine-learning/","name":"Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},"tags":[{"slug":"machine-learning","url":"http://localhost:2368/tag/machine-learning/","name":"Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},{"slug":"causal-machine-learning","url":"http://localhost:2368/tag/causal-machine-learning/","name":"Causal Machine Learning","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null},{"slug":"artificial-intelligence","url":"http://localhost:2368/tag/artificial-intelligence/","name":"Artificial Intelligence","visibility":"public","feature_image":null,"description":null,"meta_title":null,"meta_description":null,"featureImageSharp":null}],"plaintext":"This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. We have so far discussed about Causal AI , How to represent causality using SCMs , How naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference) , Pearl's Causal Hierarchy and gave flavors of Causal Inferences & Causal Discovery etc. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !\n\n\nHow are Causal AI models different from Bayesian networks?\n\n\nAt first glance there, there could be some ambiguities you feel while trying to explore Bayesian Networks and Causal Networks. This is completely normal. Lets try to uncover the differences before going ahead!.\n\nBayesian networks and Causal AI models appear similar. But Causal AI models capture underlying causal relationships; Bayesian networks just describe patterns of correlations.\n\nWhat Are Bayesian Networks ?\n\n\nYou remember the example we discussed earlier about the data describing people’s sleeping habits. We found that there’s a strong correlation between falling asleep with shoes on and waking up with a headache.\n\n\nA Bayesian network representing this is given below.\n\n\nThe BN tells us that both sleeping with shoes on and waking up with a migraine are correlated with drinking the night before, since there is a path between the variables. It also says that conditional on us knowing that someone was drinking the night before, knowing that they slept with their shoes on tells us absolutely nothing extra about whether they have a headache the next morning (this is called “conditional independence”). We can read off the conditional independence relationship by noticing that drinking alcohol “blocks” the pathway from shoe- sleeping to headache. BNs help to draw conclusions when more data becomes available (via “Bayes’ theorem”)\n\nWhat Are Causal Networks ?\n\n\nBNs sound useful! What’s the catch? The core problem is that Bayesian networks are blind to causality. This key, missing ingredient makes BNs very limited when it comes to more sophisticated reasoning and decision-making challenges.\n\nCausal AI is a new category of machine intelligence. Causal AI builds models that are able to capture cause- effect relationships while also retaining the benefits of BNs.\n\n\n\nMany BNs are all statistically compatible with the data, but only one BN corresponds to the genuine causal relationships in the system. As a result, it’s always left ambiguous whether your BN is a good causal model or not. And the overwhelming chances are that your BN is not a good causal model. The number of possible BNs grows exponentially as the number of features increases. With, say, 20 variables in your data, there’s effectively zero chance of randomly stumbling across the true causal model. This means you’re using a BN that’s making bad modeling decisions.\n\nHope this clarifies the differences and ambiguities between Bayesian and Causal Networks.\n\n\nCausal Inferences\n\n\nCausal inference refers to the process of drawing conclusions about the causal relationships between variables. In other words, it involves making judgments about whether changes in one variable are responsible for changes in another variable.\n\nHere are some examples of causal inference questions:\n\n 1. Does taking medication X reduce blood pressure in people with hypertension?\n 2. Does increasing the price of cigarettes lead to a decrease in smoking rates?\n 3. Does participating in an exercise program improve physical fitness?\n 4. Does attending preschool lead to better academic outcomes in school?\n 5. Does exposure to air pollution increase the risk of respiratory problems?\n 6. Did the treatment directly help those who took it?\n 7. Was it the marketing campaign that lead to increased sales this month or the holiday?\n 8. How big of an effect would increased wages have on productivity?\n\n\nThese are just a few examples, but causal inference questions can be asked in many different fields and contexts. The key is that they are trying to understand whether a particular intervention or exposure causes a change in some outcome or dependent variable.\n\n\nDo Calculus & Do Operator\n\n\nCausality - Formal Definition\n\n\nBefore going ahead , lets again define Causality in terms of interventions.\n\nIn the context of interventions, causality refers to the relationship between an intervention (also known as a treatment or exposure) and the resulting effect on an outcome or dependent variable. A causal relationship between an intervention and an outcome means that the intervention is responsible for the observed change in the outcome. In other words, if we change the intervention, we expect to see a corresponding change in the outcome.\n\nFor example, if a medication is found to reduce blood pressure in people with hypertension, we can say that there is a causal relationship between taking the medication and lowering blood pressure. This is because we expect that if we give the medication to a group of people with hypertension, their blood pressure will decrease as a result of the intervention. However, it is important to note that there may be other factors that could have influenced the relationship between the intervention and the outcome.\n\nWhat is Do-Calculus ?\n\n\nIn the context of causal AI, do-calculus can be used to reason about the effects of interventions on a causal model. In causal AI, a causal model represents the relationships between different variables in a system, and the do-calculus can be used to reason about how changing the value of one variable (the \"intervention\") will affect the values of other variables in the system. This can be useful for understanding the potential consequences of interventions in a real-world system, or for identifying the most effective intervention to achieve a particular outcome.\n\nHere is Judea Pearl’s canonical primer on do-calculus—a short PDF with lots of math and proofs (Pearl 2012).\n\nWhat do-operator does ?\n\n\nHowever, how does that fit into causality’s mathematical representation?\n\nThe do-operator is a mathematical representation of a physical intervention.\n\nIf we start with the model Z → X → Y, we can simulate an intervention in X by deleting all the incoming arrows to X, and manually setting X to some value x0\n\n\nBelow is the illustration how do-operator works.\n\nRules Of Do Calculus\n\n\n\n\nBeneath this scary math, each rule has specific intuition and purpose behind it! Here’s what each rule actually does:\n\nRule 1: Decide if we can ignore an observation\nRule 2: Decide if we can treat an intervention as an observation\nRule 3: Decide if we can ignore an intervention\n\n\n\nWhoa! That’s exceptionally logical. Each rule is designed to help simplify and reduce nodes in a DAG by either ignoring them (Rules 1 and 3) or making it so interventions like do(⋅)do(⋅) can be treated like observations instead (Rule 2).\n\n\nReferences\n\n\n 1. Judea Pearl’s canonical primer on do-calculus\n","html":"This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. We have so far discussed about Causal AI , How to represent causality using SCMs , How naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference) , Pearl's Causal Hierarchy and gave flavors of Causal Inferences & Causal Discovery etc. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
At first glance there, there could be some ambiguities you feel while trying to explore Bayesian Networks and Causal Networks. This is completely normal. Lets try to uncover the differences before going ahead!.
Bayesian networks and Causal AI models appear similar. But Causal AI models capture underlying causal relationships; Bayesian networks just describe patterns of correlations.
You remember the example we discussed earlier about the data describing people’s sleeping habits. We found that there’s a strong correlation between falling asleep with shoes on and waking up with a headache.
![\"Headache](\"https://user-images.githubusercontent.com/33357428/211162715-e934cca9-50aa-4d08-a259-c04c4112fa11.png\")
\nA Bayesian network representing this is given below.
![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211162842-a197dc73-3cd8-4597-a12b-d93ad49fad82.png\")
\nThe BN tells us that both sleeping with shoes on and waking up with a migraine are correlated with drinking the night before, since there is a path between the variables. It also says that conditional on us knowing that someone was drinking the night before, knowing that they slept with their shoes on tells us absolutely nothing extra about whether they have a headache the next morning (this is called “conditional independence”). We can read off the conditional independence relationship by noticing that drinking alcohol “blocks” the pathway from shoe- sleeping to headache. BNs help to draw conclusions when more data becomes available (via “Bayes’ theorem”)
BNs sound useful! What’s the catch? The core problem is that Bayesian networks are blind to causality. This key, missing ingredient makes BNs very limited when it comes to more sophisticated reasoning and decision-making challenges.
Causal AI is a new category of machine intelligence. Causal AI builds models that are able to capture cause- effect relationships while also retaining the benefits of BNs.
![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211164237-265f8d20-5b8c-4a31-95cc-897989888817.png\")
\n![\"Bayesian](\"https://user-images.githubusercontent.com/33357428/211164446-257a87fc-975c-4cef-accc-1fac10ba489f.png\")
\nMany BNs are all statistically compatible with the data, but only one BN corresponds to the genuine causal relationships in the system. As a result, it’s always left ambiguous whether your BN is a good causal model or not. And the overwhelming chances are that your BN is not a good causal model. The number of possible BNs grows exponentially as the number of features increases. With, say, 20 variables in your data, there’s effectively zero chance of randomly stumbling across the true causal model. This means you’re using a BN that’s making bad modeling decisions.
Hope this clarifies the differences and ambiguities between Bayesian and Causal Networks.
Causal inference refers to the process of drawing conclusions about the causal relationships between variables. In other words, it involves making judgments about whether changes in one variable are responsible for changes in another variable.
Here are some examples of causal inference questions:
These are just a few examples, but causal inference questions can be asked in many different fields and contexts. The key is that they are trying to understand whether a particular intervention or exposure causes a change in some outcome or dependent variable.
Before going ahead , lets again define Causality in terms of interventions.
In the context of interventions, causality refers to the relationship between an intervention (also known as a treatment or exposure) and the resulting effect on an outcome or dependent variable. A causal relationship between an intervention and an outcome means that the intervention is responsible for the observed change in the outcome. In other words, if we change the intervention, we expect to see a corresponding change in the outcome.
For example, if a medication is found to reduce blood pressure in people with hypertension, we can say that there is a causal relationship between taking the medication and lowering blood pressure. This is because we expect that if we give the medication to a group of people with hypertension, their blood pressure will decrease as a result of the intervention. However, it is important to note that there may be other factors that could have influenced the relationship between the intervention and the outcome.
In the context of causal AI, do-calculus can be used to reason about the effects of interventions on a causal model. In causal AI, a causal model represents the relationships between different variables in a system, and the do-calculus can be used to reason about how changing the value of one variable (the \"intervention\") will affect the values of other variables in the system. This can be useful for understanding the potential consequences of interventions in a real-world system, or for identifying the most effective intervention to achieve a particular outcome.
Here is Judea Pearl’s canonical primer on do-calculus—a short PDF with lots of math and proofs (Pearl 2012).
However, how does that fit into causality’s mathematical representation?
The do-operator is a mathematical representation of a physical intervention.
If we start with the model Z → X → Y, we can simulate an intervention in X by deleting all the incoming arrows to X, and manually setting X to some value x0
\nBelow is the illustration how do-operator works.
![\"Illustration](\"https://user-images.githubusercontent.com/33357428/211196318-0f07a704-ff54-4eff-82a7-d2b31913406c.png\")
![\"Do](\"https://user-images.githubusercontent.com/33357428/211195804-e99bff5b-b622-4184-9b21-99e96e9c9495.png\")
Beneath this scary math, each rule has specific intuition and purpose behind it! Here’s what each rule actually does:
Rule 1: Decide if we can ignore an observation\nRule 2: Decide if we can treat an intervention as an observation\nRule 3: Decide if we can ignore an intervention\nWhoa! That’s exceptionally logical. Each rule is designed to help simplify and reduce nodes in a DAG by either ignoring them (Rules 1 and 3) or making it so interventions like do(⋅)do(⋅) can be treated like observations instead (Rule 2).
![\"Do](\"https://user-images.githubusercontent.com/33357428/213162724-cbc40a1b-71eb-42c0-bb7e-c6f0f4f5dc27.jpg\")
This post is the fourth post of the series on Causal Machine Learning. This blog post is based on the work of Judea Pearl. We have so far discussed about Causal AI , How to represent causality using SCMs , How naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference) , Pearl's Causal Hierarchy and gave flavors of Causal Inferences & Causal Discovery etc. As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
At first glance there, there could be some ambiguities you feel while trying to explore Bayesian Networks and Causal Networks. This is completely normal. Lets try to uncover the differences before going ahead!.
Bayesian networks and Causal AI models appear similar. But Causal AI models capture underlying causal relationships; Bayesian networks just describe patterns of correlations.
You remember the example we discussed earlier about the data describing people’s sleeping habits. We found that there’s a strong correlation between falling asleep with shoes on and waking up with a headache.
\nA Bayesian network representing this is given below.
\nThe BN tells us that both sleeping with shoes on and waking up with a migraine are correlated with drinking the night before, since there is a path between the variables. It also says that conditional on us knowing that someone was drinking the night before, knowing that they slept with their shoes on tells us absolutely nothing extra about whether they have a headache the next morning (this is called “conditional independence”). We can read off the conditional independence relationship by noticing that drinking alcohol “blocks” the pathway from shoe- sleeping to headache. BNs help to draw conclusions when more data becomes available (via “Bayes’ theorem”)
BNs sound useful! What’s the catch? The core problem is that Bayesian networks are blind to causality. This key, missing ingredient makes BNs very limited when it comes to more sophisticated reasoning and decision-making challenges.
Causal AI is a new category of machine intelligence. Causal AI builds models that are able to capture cause- effect relationships while also retaining the benefits of BNs.
\n\nMany BNs are all statistically compatible with the data, but only one BN corresponds to the genuine causal relationships in the system. As a result, it’s always left ambiguous whether your BN is a good causal model or not. And the overwhelming chances are that your BN is not a good causal model. The number of possible BNs grows exponentially as the number of features increases. With, say, 20 variables in your data, there’s effectively zero chance of randomly stumbling across the true causal model. This means you’re using a BN that’s making bad modeling decisions.
Hope this clarifies the differences and ambiguities between Bayesian and Causal Networks.
Causal inference refers to the process of drawing conclusions about the causal relationships between variables. In other words, it involves making judgments about whether changes in one variable are responsible for changes in another variable.
Here are some examples of causal inference questions:
These are just a few examples, but causal inference questions can be asked in many different fields and contexts. The key is that they are trying to understand whether a particular intervention or exposure causes a change in some outcome or dependent variable.
Before going ahead , lets again define Causality in terms of interventions.
In the context of interventions, causality refers to the relationship between an intervention (also known as a treatment or exposure) and the resulting effect on an outcome or dependent variable. A causal relationship between an intervention and an outcome means that the intervention is responsible for the observed change in the outcome. In other words, if we change the intervention, we expect to see a corresponding change in the outcome.
For example, if a medication is found to reduce blood pressure in people with hypertension, we can say that there is a causal relationship between taking the medication and lowering blood pressure. This is because we expect that if we give the medication to a group of people with hypertension, their blood pressure will decrease as a result of the intervention. However, it is important to note that there may be other factors that could have influenced the relationship between the intervention and the outcome.
In the context of causal AI, do-calculus can be used to reason about the effects of interventions on a causal model. In causal AI, a causal model represents the relationships between different variables in a system, and the do-calculus can be used to reason about how changing the value of one variable (the \"intervention\") will affect the values of other variables in the system. This can be useful for understanding the potential consequences of interventions in a real-world system, or for identifying the most effective intervention to achieve a particular outcome.
Here is Judea Pearl’s canonical primer on do-calculus—a short PDF with lots of math and proofs (Pearl 2012).
However, how does that fit into causality’s mathematical representation?
The do-operator is a mathematical representation of a physical intervention.
If we start with the model Z → X → Y, we can simulate an intervention in X by deleting all the incoming arrows to X, and manually setting X to some value x0
\nBelow is the illustration how do-operator works.
Beneath this scary math, each rule has specific intuition and purpose behind it! Here’s what each rule actually does:
Rule 1: Decide if we can ignore an observation\nRule 2: Decide if we can treat an intervention as an observation\nRule 3: Decide if we can ignore an intervention\nWhoa! That’s exceptionally logical. Each rule is designed to help simplify and reduce nodes in a DAG by either ignoring them (Rules 1 and 3) or making it so interventions like do(⋅)do(⋅) can be treated like observations instead (Rule 2).
This post is the second post of the series on Causal Machine Learning. In the last blog post i have given a brief introduction on Causal Machine Learning. In this we will uncover the WHY !. This blog post is based on the work of Judea Pearl. I will discuss how naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference). As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
\n\nCorrellation Is Not Causation !!!
\n
I know you are bored of this like every other, hearing this oft-repeated saying “correlation does not imply causation”. I will formally illustrate why this is in this post.
In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of statistical correlations (that are due to chance) with no causal implications. One of them is listed below.
![\"Spurious](\"https://user-images.githubusercontent.com/33357428/210524097-c10421ba-b2f5-44fd-93c8-311a365e78c7.jpg\")
\nWe clearly know No of people who drowned while in a swimming pool has nothing to do with the power generated by US nuclear power plants !!. You can find more interesting “Spurious Correlations” from Tyler Vigen website.
Spurious correlations are well-known in statistics, so it’s easy (Somewhat !) to be on the lookout for it. Lets see a little known paradox in statistics.
Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. Lets look at an example of Simpson's paradox from Wikipedia itself.
![\"Simpson's](\"https://user-images.githubusercontent.com/33357428/210533109-6f2a4803-ae3a-4cbb-82fa-38dd43b1a6a6.png\")
\nIn both 1995 and 1996, Justice had a higher batting average (in bold type) than Jeter did. However, when the two baseball seasons are combined, Jeter shows a higher batting average than Justice.
\n\nSame data give contradictory conclusions depending on how you look at them! .
\n
Simpson’s paradox highlights that how you look at your data matters. So the question becomes, how do we partition data? Although there is no standard method in statistics for this, causal inference provides a formalism to handle this problem. It all boils down to causal effects, which quantify the impact a variable has on another variable after adjusting for the appropriate confounders.
Let us look at an example from “The Book of Why: The New Science of Cause and Effect” by Judea Pearl , and a post the legend himself posted in his twitter handle.
Consider the below study that measures weekly exercise and cholesterol in various age groups. When we plot exercise on the X-axis and cholesterol on the Y-axis and segregate by age, as in left side of Fig , we see that there is a general trend downward in each group; the more young people exercise, the lower their cholesterol is, and the same applies for middle-aged people and the elderly. If, however, we use the same scatter plot, but we don’t segregate by gender (as in right side of Fig), we see a general trend upward; the more a person exercises, the higher their cholesterol is.
![\"Book](\"https://user-images.githubusercontent.com/33357428/210539152-32dac61b-7722-41c8-8d1f-6e3fea1d33f3.jpg\")
\n\n\nExcercise appears to be beneficial in each age group but harmful in the population as a whole !!!
\n
To resolve this problem, we once again turn to the story behind the data. If we know that older people, who are more likely to exercise are also more likely to have high cholesterol regardless of exercise, then the reversal is easily explained, and easily resolved. Age is a common cause of both treatment (exercise) and outcome (cholesterol). So we should look at the age-segregated data in order to compare same-age people and thereby eliminate the possibility that the high exercisers in each group we examine are more likely to have high cholesterol due to their age, and not due to exercising.
However, please do not get confused , segregated data does not always give the correct answer.
Lets look at another example from the Causal Inference In Statistics book by Pearl.
In the classical example used by Simpson (1951), a group of sick patients are given the option to try a new drug. Among those who took the drug, a lower percentage recovered than among those who did not. However, when we partition by gender, we see that more men taking the drug recover than do men are not taking the drug, and more women taking the drug recover than do women are not taking the drug!
![\"Book](\"https://user-images.githubusercontent.com/33357428/210541750-32d1eddf-be44-4dab-bea7-3246d893baad.jpeg\")
\nIn other words, the drug appears to help men and women, but hurt the general population. It seems nonsensical, or even impossible—which is why, of course, it is considered a paradox. Some people find it hard to believe that numbers could even be combined in such a way.
\n\nThe data seem to say that if we know the patient’s gender male or female we can prescribe the drug, but if the gender is unknown we should not! Obviously, that conclusion is ridiculous. If the drug helps men and women, it must help anyone; our lack of knowledge of the patient’s gender cannot make the drug harmful.
\n
Given the results of this study, then, should a doctor prescribe the drug for a woman? A man? A patient of unknown gender? Or consider a policy maker who is evaluating the drug’s overall effectiveness on the population. Should he/she use the recovery rate for the general population? Or should he/she use the recovery rates for the gendered sub-populations?
The answer is nowhere to be found in simple statistics.
In order to decide whether the drug will harm or help a patient, we first have to understand the story behind the data , the causal mechanism that led to, or generated, the results we see. For instance, suppose we knew an additional fact: Estrogen has a negative effect on recovery, so women are less likely to recover than men, regardless of the drug. In addition, as we can see from the data, women are significantly more likely to take the drug than men are. So, the reason the drug appears to be harmful overall is that, if we select a drug user at random, that person is more likely to be a woman and hence less likely to recover than a random person who does not take the drug. Put differently, being a woman is a common cause of both drug taking and failure to recover. Therefore, to assess the effectiveness, we need to compare subjects of the same gender, thereby ensuring that any difference in recovery rates between those who take the drug and those who do not is not ascribable to estrogen.
In fact, as statistics textbooks have traditionally (and correctly) warned students, correlation is not causation, so there is no statistical method that can determine the causal story from the data alone. Consequently, there is no statistical method that can aid in our decision.
The problems with traditional statistics when thinking about causality stems from a fundamental property of algebra, symmetry . The left-hand side of an equation equals the right-hand side (that’s the point of algebra). The equal sign implies symmetry. However, causality is fundamentally asymmetric i.e. causes lead to effects and not the other way around.
This distinction further implies that causal relations cannot be expressed in the language of probability and, hence, that any mathematical approach to causal analysis must acquire new notation – probability calculus is insufficient. To illustrate, the syntax of probability calculus does not permit us to express the simple fact that “symptoms do not cause diseases,” let alone draw mathematical conclusions from such facts. All we can say is that two events are dependent—meaning that if we find one, we can expect to encounter the other, but we cannot distinguish statistical dependence, quantified by the conditional probability P(disease|symptom) from causal dependence, for which we have no expression in standard probability calculus.
Let’s look at a simple example taken from Judea Pearl book itself . Suppose we model the relationship between a disease and the symptoms it produces, with the expression below. Y represents the severity of the symptoms, X the severity of the disease, m is the connection between the two, and b represents all other factors.
![\"SymptomsDisease\"](\"https://user-images.githubusercontent.com/33357428/210627825-c1e11271-a609-4bd7-b679-42dca0a40aee.jpg\")
Using the rules of algebra we can invert the equation above to get the following expression.
![\"SymptomDisease2\"](\"https://user-images.githubusercontent.com/33357428/210628024-cf038674-9241-4076-9422-1e45b6d1199e.jpg\")
Here’s the problem, if we interpret the first equation as diseases cause symptoms, then we have to interpret the second equation as symptoms cause diseases! Which is of course not true.
Note : Linear relations are used here for illustration purposes only; they do not represent typical disease-symptom relations but illustrate the historical development of path analysis.
Before moving to the next set of blog posts , I should precisely define what a correlation is. I know you all are bored with listening this oft-repeated saying \"Correlation is not causation\" , so am i. So lets sort this out before moving to anything else!
Before moving ahead lets clarify one more thing : “Correlation” is often colloquially used as a synonym for statistical dependence. However, “correlation” is technically only a measure of linear statistical dependence. We will largely be using the term association to refer to statistical dependence from now on.
Lets take an example from Brady Neal's Causal Course book.
Say you happen upon some data that relates wearing shoes to bed and
waking up with a headache, as one does. It turns out that most times
that someone wears shoes to bed, that person wakes up with a headache.
And most times someone doesn’t wear shoes to bed, that person doesn’t
wake up with a headache. It is not uncommon for people to interpret
data like this (with associations) as meaning that wearing shoes to bed
causes people to wake up with headaches, especially if they are looking
for a reason to justify not wearing shoes to bed.
We can explain how wearing shoes to bed and headaches are associated
without either being a cause of the other. It turns out that they are
both caused by a common cause: drinking the night before. This kind of variables are called \"confounder\" or lurking variable. We will call this kind of association confounding association since the association is facilitated by a confounder.
![\"Headache](\"https://user-images.githubusercontent.com/33357428/210763817-267c4af8-6ac6-4392-8dcb-3196a94ed19a.jpg\")
The main problem motivating causal inference is that association is not causation.
If the two were the same, then causal inference would be easy. Traditional statistics and machine learning would already have causal inference solved, as measuring causation would be as simple as just looking at measures such as correlation and predictive performance in data.
Lets look at another example. Lets attempt to determine the causal effect of vitamin C intake on resistance to sickness. Let X be defined as a binary indicator representing if this subject intakes vitamin C and let Y be a binary indicator of being healthy (not getting sick). X is also referred to as the ‘treatment’ in a more general setting. Now, let C1 be the value of Y if X=1 (vitamin C is taken) and C0 be the value of Y if X=0 (vitamin C is not taken). We call C0 and C1 the potential outcomes of this experiment.
![\"Association](\"https://user-images.githubusercontent.com/33357428/210771012-3809777d-83cb-4027-a73c-495690ec5876.jpg\")
\nFor a single person, the causal effect of taking vitamin C in this context would be the difference between the expected outcome of taking vitamin C and the expected outcome of not taking vitamin C.
Causal Effect = E(C1) – E(C0)\nUnfortunately, we can only ever observe one of the possible outcomes C0 or C1. We cannot perfectly reset all conditions to see the result of the opposite treatment. Instead, we can use multiple samples and calculate the association between Vitamin C and being healthy.
Association = E(Y|X=1) – E(Y|X=0)\nAssociation as being (1+1+1+1)/4 – (0+0+0+0)/4 = 1
Causal effect, using the unobserved outcomes*, as being (4*0 + 4*1)/4 – (4*0 + 4*1)/4 = 0
We just calculated that, in this case, association does not equal causation. Observationally, there seems to be a perfect association between taking Vitamin C intake and being healthy. However, we can see there is no causal effect because we are privileged with the values of the unobserved outcomes. This inequality could be explained by considering that the people that stayed healthy practiced healthy habits which included taking Vitamin C.
Okay, one more motivating example :
In response to a large study that studied the relationship between income and life expectancy, Vox published an article titled “Want to live longer, even if you’re poor? Then move to a big city in California” (Klein, 2016). However, as is implied by the title of the study “The Association Between Income and Life Expectancy in the United States, 2001-2014”, the study did not presume to make this recommendation and in fact the closest statement made to the Vox recommendation was “… the strongest pattern in the data was that low-income individuals tend to live longest (and have more healthful behaviors) in cities with highly educated populations, high incomes, and high levels of government expenditures, such as New York, New York, and San Francisco, California.” (Chetty et al., 2016).
Similar to the example regarding vitamin C and health, this study only found associative effects. However, just like it is incorrect to say that vitamin C causes a person to be healthy, it is also incorrect to say that moving to California will cause you to live longer.
That's all for now ! , Hope you enjoyed reading so far !
In the next blog post, we will further investigate the differences between association and causation, by starting with Pearl’s three-level causal hierarchy. That will be much interesting to watch out for !
This post is the second post of the series on Causal Machine Learning. In the last blog post i have given a brief introduction on Causal Machine Learning. In this we will uncover the WHY !. This blog post is based on the work of Judea Pearl. I will discuss how naive statistics can fail (Spurious Correlations , Simpson Paradox & Asymmetry In Causal Inference). As always i will try to keep the things as simple as possible. So stay with me , Enjoy reading !
\n\nCorrellation Is Not Causation !!!
\n
I know you are bored of this like every other, hearing this oft-repeated saying “correlation does not imply causation”. I will formally illustrate why this is in this post.
In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of statistical correlations (that are due to chance) with no causal implications. One of them is listed below.
\nWe clearly know No of people who drowned while in a swimming pool has nothing to do with the power generated by US nuclear power plants !!. You can find more interesting “Spurious Correlations” from Tyler Vigen website.
Spurious correlations are well-known in statistics, so it’s easy (Somewhat !) to be on the lookout for it. Lets see a little known paradox in statistics.
Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. Lets look at an example of Simpson's paradox from Wikipedia itself.
\nIn both 1995 and 1996, Justice had a higher batting average (in bold type) than Jeter did. However, when the two baseball seasons are combined, Jeter shows a higher batting average than Justice.
\n\nSame data give contradictory conclusions depending on how you look at them! .
\n
Simpson’s paradox highlights that how you look at your data matters. So the question becomes, how do we partition data? Although there is no standard method in statistics for this, causal inference provides a formalism to handle this problem. It all boils down to causal effects, which quantify the impact a variable has on another variable after adjusting for the appropriate confounders.
Let us look at an example from “The Book of Why: The New Science of Cause and Effect” by Judea Pearl , and a post the legend himself posted in his twitter handle.
Consider the below study that measures weekly exercise and cholesterol in various age groups. When we plot exercise on the X-axis and cholesterol on the Y-axis and segregate by age, as in left side of Fig , we see that there is a general trend downward in each group; the more young people exercise, the lower their cholesterol is, and the same applies for middle-aged people and the elderly. If, however, we use the same scatter plot, but we don’t segregate by gender (as in right side of Fig), we see a general trend upward; the more a person exercises, the higher their cholesterol is.
\n\n\nExcercise appears to be beneficial in each age group but harmful in the population as a whole !!!
\n
To resolve this problem, we once again turn to the story behind the data. If we know that older people, who are more likely to exercise are also more likely to have high cholesterol regardless of exercise, then the reversal is easily explained, and easily resolved. Age is a common cause of both treatment (exercise) and outcome (cholesterol). So we should look at the age-segregated data in order to compare same-age people and thereby eliminate the possibility that the high exercisers in each group we examine are more likely to have high cholesterol due to their age, and not due to exercising.
However, please do not get confused , segregated data does not always give the correct answer.
Lets look at another example from the Causal Inference In Statistics book by Pearl.
In the classical example used by Simpson (1951), a group of sick patients are given the option to try a new drug. Among those who took the drug, a lower percentage recovered than among those who did not. However, when we partition by gender, we see that more men taking the drug recover than do men are not taking the drug, and more women taking the drug recover than do women are not taking the drug!
\nIn other words, the drug appears to help men and women, but hurt the general population. It seems nonsensical, or even impossible—which is why, of course, it is considered a paradox. Some people find it hard to believe that numbers could even be combined in such a way.
\n\nThe data seem to say that if we know the patient’s gender male or female we can prescribe the drug, but if the gender is unknown we should not! Obviously, that conclusion is ridiculous. If the drug helps men and women, it must help anyone; our lack of knowledge of the patient’s gender cannot make the drug harmful.
\n
Given the results of this study, then, should a doctor prescribe the drug for a woman? A man? A patient of unknown gender? Or consider a policy maker who is evaluating the drug’s overall effectiveness on the population. Should he/she use the recovery rate for the general population? Or should he/she use the recovery rates for the gendered sub-populations?
The answer is nowhere to be found in simple statistics.
In order to decide whether the drug will harm or help a patient, we first have to understand the story behind the data , the causal mechanism that led to, or generated, the results we see. For instance, suppose we knew an additional fact: Estrogen has a negative effect on recovery, so women are less likely to recover than men, regardless of the drug. In addition, as we can see from the data, women are significantly more likely to take the drug than men are. So, the reason the drug appears to be harmful overall is that, if we select a drug user at random, that person is more likely to be a woman and hence less likely to recover than a random person who does not take the drug. Put differently, being a woman is a common cause of both drug taking and failure to recover. Therefore, to assess the effectiveness, we need to compare subjects of the same gender, thereby ensuring that any difference in recovery rates between those who take the drug and those who do not is not ascribable to estrogen.
In fact, as statistics textbooks have traditionally (and correctly) warned students, correlation is not causation, so there is no statistical method that can determine the causal story from the data alone. Consequently, there is no statistical method that can aid in our decision.
The problems with traditional statistics when thinking about causality stems from a fundamental property of algebra, symmetry . The left-hand side of an equation equals the right-hand side (that’s the point of algebra). The equal sign implies symmetry. However, causality is fundamentally asymmetric i.e. causes lead to effects and not the other way around.
This distinction further implies that causal relations cannot be expressed in the language of probability and, hence, that any mathematical approach to causal analysis must acquire new notation – probability calculus is insufficient. To illustrate, the syntax of probability calculus does not permit us to express the simple fact that “symptoms do not cause diseases,” let alone draw mathematical conclusions from such facts. All we can say is that two events are dependent—meaning that if we find one, we can expect to encounter the other, but we cannot distinguish statistical dependence, quantified by the conditional probability P(disease|symptom) from causal dependence, for which we have no expression in standard probability calculus.
Let’s look at a simple example taken from Judea Pearl book itself . Suppose we model the relationship between a disease and the symptoms it produces, with the expression below. Y represents the severity of the symptoms, X the severity of the disease, m is the connection between the two, and b represents all other factors.
Using the rules of algebra we can invert the equation above to get the following expression.
Here’s the problem, if we interpret the first equation as diseases cause symptoms, then we have to interpret the second equation as symptoms cause diseases! Which is of course not true.
Note : Linear relations are used here for illustration purposes only; they do not represent typical disease-symptom relations but illustrate the historical development of path analysis.
Before moving to the next set of blog posts , I should precisely define what a correlation is. I know you all are bored with listening this oft-repeated saying \"Correlation is not causation\" , so am i. So lets sort this out before moving to anything else!
Before moving ahead lets clarify one more thing : “Correlation” is often colloquially used as a synonym for statistical dependence. However, “correlation” is technically only a measure of linear statistical dependence. We will largely be using the term association to refer to statistical dependence from now on.
Lets take an example from Brady Neal's Causal Course book.
Say you happen upon some data that relates wearing shoes to bed and
waking up with a headache, as one does. It turns out that most times
that someone wears shoes to bed, that person wakes up with a headache.
And most times someone doesn’t wear shoes to bed, that person doesn’t
wake up with a headache. It is not uncommon for people to interpret
data like this (with associations) as meaning that wearing shoes to bed
causes people to wake up with headaches, especially if they are looking
for a reason to justify not wearing shoes to bed.
We can explain how wearing shoes to bed and headaches are associated
without either being a cause of the other. It turns out that they are
both caused by a common cause: drinking the night before. This kind of variables are called \"confounder\" or lurking variable. We will call this kind of association confounding association since the association is facilitated by a confounder.
The main problem motivating causal inference is that association is not causation.
If the two were the same, then causal inference would be easy. Traditional statistics and machine learning would already have causal inference solved, as measuring causation would be as simple as just looking at measures such as correlation and predictive performance in data.
Lets look at another example. Lets attempt to determine the causal effect of vitamin C intake on resistance to sickness. Let X be defined as a binary indicator representing if this subject intakes vitamin C and let Y be a binary indicator of being healthy (not getting sick). X is also referred to as the ‘treatment’ in a more general setting. Now, let C1 be the value of Y if X=1 (vitamin C is taken) and C0 be the value of Y if X=0 (vitamin C is not taken). We call C0 and C1 the potential outcomes of this experiment.
\nFor a single person, the causal effect of taking vitamin C in this context would be the difference between the expected outcome of taking vitamin C and the expected outcome of not taking vitamin C.
Causal Effect = E(C1) – E(C0)\nUnfortunately, we can only ever observe one of the possible outcomes C0 or C1. We cannot perfectly reset all conditions to see the result of the opposite treatment. Instead, we can use multiple samples and calculate the association between Vitamin C and being healthy.
Association = E(Y|X=1) – E(Y|X=0)\nAssociation as being (1+1+1+1)/4 – (0+0+0+0)/4 = 1
Causal effect, using the unobserved outcomes*, as being (4*0 + 4*1)/4 – (4*0 + 4*1)/4 = 0
We just calculated that, in this case, association does not equal causation. Observationally, there seems to be a perfect association between taking Vitamin C intake and being healthy. However, we can see there is no causal effect because we are privileged with the values of the unobserved outcomes. This inequality could be explained by considering that the people that stayed healthy practiced healthy habits which included taking Vitamin C.
Okay, one more motivating example :
In response to a large study that studied the relationship between income and life expectancy, Vox published an article titled “Want to live longer, even if you’re poor? Then move to a big city in California” (Klein, 2016). However, as is implied by the title of the study “The Association Between Income and Life Expectancy in the United States, 2001-2014”, the study did not presume to make this recommendation and in fact the closest statement made to the Vox recommendation was “… the strongest pattern in the data was that low-income individuals tend to live longest (and have more healthful behaviors) in cities with highly educated populations, high incomes, and high levels of government expenditures, such as New York, New York, and San Francisco, California.” (Chetty et al., 2016).
Similar to the example regarding vitamin C and health, this study only found associative effects. However, just like it is incorrect to say that vitamin C causes a person to be healthy, it is also incorrect to say that moving to California will cause you to live longer.
That's all for now ! , Hope you enjoyed reading so far !
In the next blog post, we will further investigate the differences between association and causation, by starting with Pearl’s three-level causal hierarchy. That will be much interesting to watch out for !
\n\nTo Build Truly Intelligent Machines, Teach Them Cause and Effect.
\n
\n\nCorrelation is not Causation!.
\n
This post is the first of the series on Causal Machine Learning. I will start with the very basics of causal inference , Provide some basic background in Bayesian networks/graphical models , Show how graphical models can be used in causal inference and Describe application scenarios , case studies and the practical difficulties in Causal Inference. I will follow step by step approach and keep the things as simple as possible with lot of practical tutorials. In each of the session i will provide additional learning materials and references , which should help you to keep the flow. In this blog post , i will present Why we need causality in Machine Learning! . Enjoy the reading!
Deep learning techniques do a good job at building models by correlating data points. But many AI researchers believe that more work needs to be done to understand causation and not just correlation. The field causal deep learning -- useful in determining why something happened -- is still in its infancy, and it is much more difficult to automate than neural networks.
\n\nEven though we can observe correlation, it does not prove causation.
\n
Correlation: measures the relationship between two things.\nCausation: means that one thing will cause the other to happen.\nI know that when I see X, I will see Y (Association/Correlation)\nI know that X causes Y (Causality)\nThe distinctions between the two can have important implications. In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of correlations that are due to chance. One of the interesting spurious correlation is listed below.
![](\"https://user-images.githubusercontent.com/33357428/209831536-c820c96b-3347-4a30-b0f0-dca9e409e6f0.jpeg\")
While these two variables have a correlation of 95.23%, it is highly unrealistic to think that a degree in mathematics caused an increase in the amount of Uranium stored in the United States!
![](\"https://user-images.githubusercontent.com/33357428/208087796-ad8b5727-8db3-4d71-9881-8a4f56169964.png\")
\n\n“If Correlation Doesn’t Imply Causation, Then What Does?”
\n
We will uncover this in the upcoming blog posts.
The excitement in the field has been kindled by Judea Pearl, a professor at UCLA, who did some of the formative work on implementing Bayesian networks for statistical analysis. More recently he has been developing a framework for diagramming causation and teasing apart the factors that contribute to observed events in a computable framework.
![](\"https://user-images.githubusercontent.com/33357428/208084630-fd654f0e-0bc2-4576-844a-5d398c75d5a8.jpg\")
In his latest book, “The Book of Why: The New Science of Cause and Effect,” he argues that artificial intelligence has been handicapped by an incomplete understanding of what intelligence really is. In his new book, Pearl, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. Instead of the mere ability to correlate fever and malaria, machines need the capacity to reason that malaria causes fever. Pearl expects that causal reasoning could provide machines with human-level intelligence.
![\"The](\"https://user-images.githubusercontent.com/33357428/208090292-28654bde-487b-4f4e-b1ea-dd957c8ae7f0.jpg\")
Understanding why requires understanding of the whats, the wheres, and the whens. The hows, however, seem to be an implementation of the whys !!
Machine learning is a type of artificial intelligence that involves training algorithms to recognise patterns in data and make predictions or decisions based on those patterns. The goal of machine learning is to build models that can generalise to new data and make accurate predictions or decisions.
The main difference between machine learning and causal machine learning is the focus of the analysis. While traditional machine learning aims to predict an outcome based on patterns in the data, causal machine learning aims to identify the specific variables that are causing an outcome and how they contribute to it. This involves identifying and modelling the causal relationships between variables, rather than just the statistical relationships.
Causal machine learning is a type of machine learning that focuses on identifying the cause-and-effect relationships between variables. In other words, it aims to identify the factors that influence a particular outcome and how they influence it.
Causal machine learning can be used to improve the accuracy of predictive models by taking into account the underlying causes of the outcomes being predicted. It is also useful in a variety of applications, such as evaluating the effectiveness of interventions, predicting the impact of policy changes, and identifying the factors that contribute to certain outcomes.
Explainable AI (XAI) is a type of artificial intelligence (AI) that is designed to be transparent and interpretable, meaning that it can provide explanations for its decisions and actions. The goal of XAI is to make AI systems more transparent and understandable to humans, so that they can be trusted and used more effectively.
Causal AI, on the other hand, is a type of AI that focuses on identifying the cause-and-effect relationships between variables. It is a sub field of machine learning that aims to identify the specific factors that influence a particular outcome and how they contribute to it. One key difference between explainable AI and causal AI is their focus. Explainable AI is concerned with making AI systems more transparent and interpretable, while causal AI is focused on identifying the causes and effects of various phenomena.
Another difference is the techniques used. Explainable AI often employs techniques such as feature importance, sensitivity analysis, and model interpretation methods to provide insights into the decision-making process of AI systems. Causal AI, on the other hand, typically uses techniques such as structural equation modelling, instrumental variables, and counterfactual analysis to identify the causal relationships between variables.
While XAI and causal AI are related in that they both aim to increase our understanding of how AI systems work, they are distinct concepts. XAI focuses on providing explanations for the decisions and predictions made by AI systems, while causal AI focuses on identifying the underlying causes of outcomes.
One way in which XAI and causal AI can be related is that XAI methods, such as counterfactual analysis, can be used to identify the causes of particular outcomes or decisions. However, it is important to note that not all XAI methods are causal in nature, and not all causal AI systems are necessarily explainable. Overall, both XAI and causal AI are important tools for improving our understanding of how AI systems work and for increasing the transparency and trustworthiness of AI systems.
Causal machine learning can be applied in many different fields and industries to identify and understand the causal relationships between variables. Here are a few examples of the application of causal machine learning:
These are just a few examples of the many ways in which causal machine learning can be applied. There are many more applications in various fields and industries.
In this post I have provided an introduction to Causal machine learning. I have tried to be as compact as possible. Below you can find some additional resources if you want to know more about Causal machine learning. Hope you enjoyed reading so far!.
Course : Machine Learning & Causal Inference: A Short Course
\nTexbook : Causal inference in statistics:An overview. Judea Pearl
\nIndustry : Causality and Machine Learning: Microsoft Research
\n\n\nTo Build Truly Intelligent Machines, Teach Them Cause and Effect.
\n
\n\nCorrelation is not Causation!.
\n
This post is the first of the series on Causal Machine Learning. I will start with the very basics of causal inference , Provide some basic background in Bayesian networks/graphical models , Show how graphical models can be used in causal inference and Describe application scenarios , case studies and the practical difficulties in Causal Inference. I will follow step by step approach and keep the things as simple as possible with lot of practical tutorials. In each of the session i will provide additional learning materials and references , which should help you to keep the flow. In this blog post , i will present Why we need causality in Machine Learning! . Enjoy the reading!
Deep learning techniques do a good job at building models by correlating data points. But many AI researchers believe that more work needs to be done to understand causation and not just correlation. The field causal deep learning -- useful in determining why something happened -- is still in its infancy, and it is much more difficult to automate than neural networks.
\n\nEven though we can observe correlation, it does not prove causation.
\n
Correlation: measures the relationship between two things.\nCausation: means that one thing will cause the other to happen.\nI know that when I see X, I will see Y (Association/Correlation)\nI know that X causes Y (Causality)\nThe distinctions between the two can have important implications. In the website, “Spurious Correlations” by Tyler Vigen, we can explore a wide variety of correlations that are due to chance. One of the interesting spurious correlation is listed below.
While these two variables have a correlation of 95.23%, it is highly unrealistic to think that a degree in mathematics caused an increase in the amount of Uranium stored in the United States!
\n\n“If Correlation Doesn’t Imply Causation, Then What Does?”
\n
We will uncover this in the upcoming blog posts.
The excitement in the field has been kindled by Judea Pearl, a professor at UCLA, who did some of the formative work on implementing Bayesian networks for statistical analysis. More recently he has been developing a framework for diagramming causation and teasing apart the factors that contribute to observed events in a computable framework.
In his latest book, “The Book of Why: The New Science of Cause and Effect,” he argues that artificial intelligence has been handicapped by an incomplete understanding of what intelligence really is. In his new book, Pearl, elaborates a vision for how truly intelligent machines would think. The key, he argues, is to replace reasoning by association with causal reasoning. Instead of the mere ability to correlate fever and malaria, machines need the capacity to reason that malaria causes fever. Pearl expects that causal reasoning could provide machines with human-level intelligence.
Understanding why requires understanding of the whats, the wheres, and the whens. The hows, however, seem to be an implementation of the whys !!
Machine learning is a type of artificial intelligence that involves training algorithms to recognise patterns in data and make predictions or decisions based on those patterns. The goal of machine learning is to build models that can generalise to new data and make accurate predictions or decisions.
The main difference between machine learning and causal machine learning is the focus of the analysis. While traditional machine learning aims to predict an outcome based on patterns in the data, causal machine learning aims to identify the specific variables that are causing an outcome and how they contribute to it. This involves identifying and modelling the causal relationships between variables, rather than just the statistical relationships.
Causal machine learning is a type of machine learning that focuses on identifying the cause-and-effect relationships between variables. In other words, it aims to identify the factors that influence a particular outcome and how they influence it.
Causal machine learning can be used to improve the accuracy of predictive models by taking into account the underlying causes of the outcomes being predicted. It is also useful in a variety of applications, such as evaluating the effectiveness of interventions, predicting the impact of policy changes, and identifying the factors that contribute to certain outcomes.
Explainable AI (XAI) is a type of artificial intelligence (AI) that is designed to be transparent and interpretable, meaning that it can provide explanations for its decisions and actions. The goal of XAI is to make AI systems more transparent and understandable to humans, so that they can be trusted and used more effectively.
Causal AI, on the other hand, is a type of AI that focuses on identifying the cause-and-effect relationships between variables. It is a sub field of machine learning that aims to identify the specific factors that influence a particular outcome and how they contribute to it. One key difference between explainable AI and causal AI is their focus. Explainable AI is concerned with making AI systems more transparent and interpretable, while causal AI is focused on identifying the causes and effects of various phenomena.
Another difference is the techniques used. Explainable AI often employs techniques such as feature importance, sensitivity analysis, and model interpretation methods to provide insights into the decision-making process of AI systems. Causal AI, on the other hand, typically uses techniques such as structural equation modelling, instrumental variables, and counterfactual analysis to identify the causal relationships between variables.
While XAI and causal AI are related in that they both aim to increase our understanding of how AI systems work, they are distinct concepts. XAI focuses on providing explanations for the decisions and predictions made by AI systems, while causal AI focuses on identifying the underlying causes of outcomes.
One way in which XAI and causal AI can be related is that XAI methods, such as counterfactual analysis, can be used to identify the causes of particular outcomes or decisions. However, it is important to note that not all XAI methods are causal in nature, and not all causal AI systems are necessarily explainable. Overall, both XAI and causal AI are important tools for improving our understanding of how AI systems work and for increasing the transparency and trustworthiness of AI systems.
Causal machine learning can be applied in many different fields and industries to identify and understand the causal relationships between variables. Here are a few examples of the application of causal machine learning:
These are just a few examples of the many ways in which causal machine learning can be applied. There are many more applications in various fields and industries.
In this post I have provided an introduction to Causal machine learning. I have tried to be as compact as possible. Below you can find some additional resources if you want to know more about Causal machine learning. Hope you enjoyed reading so far!.
Course : Machine Learning & Causal Inference: A Short Course
\nTexbook : Causal inference in statistics:An overview. Judea Pearl
\nIndustry : Causality and Machine Learning: Microsoft Research
\nIn this article , we will talk about SHAP in detail. We will discuss What are Shapley Values ? , Mathematical foundation behind the Shapley Values and How does SHAP (Shapley Additive Explanations) reframes the Shapey Value problem in detail. Also we will discuss What is Local Accuracy, Missingness, and Consistency in the context of explainable models , What is the Shapley Kernel etc.
SHAP values were introduced by Lundberg and Lee (2016) as a way to provide local explanations for machine learning models. They built upon Shapley's work in cooperative game theory, using the Shapley value to explain how each feature in a model contributes to the prediction for a particular instance.
Shapley values were invented by Lloyd Shapley as a way of providing a fair solution to the following question: if we have a coalition C that collaborates to produce a value V, how much did each individual member contribute to that final value ?
So what does this mean ? We have a coalition C, a group of cooperating members that work together to produce some value V, called the coalition value. This could be something like, a corporation of employees that together generate a certain profit, or a dinner group running up a restaurant bill. We want to know exactly how much each member contributed to that final coalition value; what share of the profit does each employee deserve, how much each person in the dinner party owes to settle the bill.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222118086-efdfcda4-01e6-4d4d-bc6d-30ce019c464b.png\")
\nHowever, answering this gets tricky when there are interacting effects between members, when certain permutations cause members to contribute more than the sum of their parts. To find a fair answer to this question that takes into account these interaction effects, we can compute the Shapley value for each member of the coalition.
So let’s how we can compute the Shapley value for member 1 of our example coalition. The way this is done is by sampling a coalition that contains member 1, and then looking at the coalition formed by removing that member.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222119980-a7c2c357-eecf-4ea9-8de6-7eefacf4e362.png\")
\nWe then look at the respective values of these two coalitions, and compare the difference between the two.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222120650-7a9cd97e-3dfd-488c-9a24-45638a75847a.png\")
\nThis difference is the marginal contribution of member 1 to the coalition consisting of members 2, 3, and 4; how much member 1 contributed to that specific group.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222120708-c27b48f7-7712-4e7d-ae44-c998a859215c.png\")
\nSo we then enumerate all such pairs of coalitions, that is, all pairs of coalitions that only differ based on whether or not member 1 is included, and then look at all the marginal contributions for each.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222121573-e1c9b15d-d883-44f0-b6e3-41fff235ff06.png\")
\nThe mean marginal contribution is the Shapley value of that member.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222121676-1e0e6852-0b66-44b7-bbd4-11fd8e25b39f.png\")
\nWe can do this same process for each member of the coalition, and we’ve found a fair solution to our original question.!
Mathematically, the whole process looks like this, but all we need is to know that the Shapley value is the average amount of contribution that a particular member makes to the coalition value.
Now, translating this concept to model explainability is relatively straightforward, and that’s exactly what Scott Lundberg and Su-In Lee did in 2017 with their paper “A Unified Approach to Interpreting Model Predictions,” where they introduced SHAP.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222123143-27853042-9e12-4526-91dd-466e04f8a68f.png\")
\nSHAP reframes the Shapley value problem from one where we look at how members of a coalition contribute to a coalition value to one where we look at how individual features contribute to a model’s outputs.
![\"Shapley](\"https://user-images.githubusercontent.com/33357428/222123344-4acf9530-e696-4ee7-80f0-4835cd9cc775.png\")
They do this in a very specific way, one that we can get a clue to by looking at the name of their algorithm; Shapley Additive Explanations. We know what Shapley values are, we know what Explanations are, but what do they mean by Additive?
\nLundberg and Lee define an additive feature attribution as follows: if we have a set a of inputs x, and a model f(x), we can define a set of simplified local inputs x’ (which usually means that we turn a feature vector into a discrete binary vector, where features are either included or excluded) and we can also define an explanatory model g.
\nWhat we need to ensure is that One: if x’ is roughly equal to x then g(x’) should be roughly equal to f(x)
![\"SHAP\"](\"https://user-images.githubusercontent.com/33357428/222126320-96c62ae5-2dcd-46eb-8a64-38727338c7dd.png\")
two: g must take below form, where phi_0 is the null output of the model, that is, the average output of the model, and phi_i is the explained effect of feature_i; how much that feature changes the output of the model. This is called it’s attribution.
![\"SHAP\"](\"https://user-images.githubusercontent.com/33357428/222126643-5e4a7d98-53a6-4389-86cd-ac6d37041150.png\")
If we have these two, we have an explanatory model that has additive feature attribution. The advantage of this form of explanation is really easy to interpret; we can see the exact contribution and importance of each feature just by looking at the phi values.
Now Lundberg and Lee go on to describe a set of three desirable properties of such an additive feature method; local accuracy, missingness, and consistency.
\nLocal accuracy : it simply says if the input and the simplified input are roughly the same, then the actual model and the explanatory model should produce roughly the same output.
Missingness states that if a feature is excluded from the model, it’s attribution must be zero; that is, the only thing that can affect the output of the explanation model is the inclusion of features, not the exclusion.
Finally, we have consistency (and this one’s a little hard to represent mathematically), but it states that if the original model changes so that the a particular feature’s contribution changes, the attribution in the explanatory model cannot change in the opposite direction;
So for example, if we have a new model where a specific feature has a more positive contribution than in the original; the attribution in our new explanatory model cannot decrease.
![\"SHAP\"](\"https://user-images.githubusercontent.com/33357428/222395551-04403b51-1e8f-4bbe-84d7-1bb2f9d0b966.png\")
Now while a bunch of different explanation methods satisfy some of these properties, Lundberg and Lee argue that only SHAP satisfies all three; if the feature attributions in our additive explanatory model are specifically chosen to be the shapley values of those features, then all three properties are upheld.
The problem with SHAP, however, is that computing Shapley values means you have to sample the coalition values for each possible feature permutation, which in a model explainability setting means we have to evaluate our model that number of times.
For a model that operates over 4 features, it’s easy enough, it’s just 16 coalitions to sample to get all the Shapley values. For 32 features, that’s over 17 billion samples, which is entirely untenable. To get around this, Lundberg and Lee devise the Shapley Kernel, a means of approximating shapley values through much fewer samples.
So what we do is we pass samples through the model, of various feature permutations of the particular datapoint that we’re trying to explain.
![\"Shapley-Permu\"](\"https://user-images.githubusercontent.com/33357428/222397887-4451072f-be0b-4e27-accc-e401b7ac35bf.png\")
Of course, most ML models won’t just let you omit a feature !
![\"Shapley-Omit-Feature\"](\"https://user-images.githubusercontent.com/33357428/222398010-2e845aae-b21e-48ea-b16e-5da41af83b86.png\")
\nSo what we do is define a background dataset B, one that contains a set of representative datapoints that the model was trained over.
![\"BackgroundDataset\"](\"https://user-images.githubusercontent.com/33357428/222398330-d64ddccf-b8f2-4f5e-880a-dfe05d99eb77.png\")
\nWe then fill in our omitted feature or features with values from the background dataset, while holding the features that are included in the permutation fixed to their original values.
![\"IncludedFeaturesFromDatasetB\"](\"https://user-images.githubusercontent.com/33357428/222398831-31f4f53d-2610-47d6-9b45-89a7ff663fe4.png\")
\nWe then take the average of the model output over all of these new synthetic datapoints as our model output for that feature permutation, which we’ll call that y bar.
![\"Shapley-Avg\"](\"https://user-images.githubusercontent.com/33357428/222399307-fc75df99-27a3-4d11-8fff-d785616fbee9.png\")
\n![\"Shapley-ybar\"](\"https://user-images.githubusercontent.com/33357428/222399733-737c4ba1-23f2-4f97-996e-f3ace97fb3cf.png\")
\nSo once we have a number of samples computed in this way,
![\"Shapley-NoOfYbar\"](\"https://user-images.githubusercontent.com/33357428/222399945-9a2510d2-5595-42eb-887c-5b5762c9aa1e.png\")
\nWe can formulate this as a weighted linear regression, with each feature assigned a coefficient.
![\"SShapley-LinearReg\"](\"https://user-images.githubusercontent.com/33357428/222400421-c3964125-f4d1-44e1-ba12-1231cb76671a.png\")
With a very specific choice of weighting for each sample, based on a combination of the total number of features in the model, the number of coalitions with the same number of features as this particular sample, and the number of features included and excluded in this permutation, we ensure that the solution to this weighted linear regression is such that the returned coefficients are equivalent to the Shapley values.
![\"Shapley-Weighting\"](\"https://user-images.githubusercontent.com/33357428/222400526-11e50df4-880d-44e8-bd15-812560a8e005.png\")
This weighting scheme is the basis of the Shapley Kernel, and the weighted linear regression process as a whole is Kernel SHAP.
![\"Shapley-Kernel-Weights\"](\"https://user-images.githubusercontent.com/33357428/222400590-c55c9b16-5ce4-4022-a6aa-73c294f2d1a8.png\")
Now, there are a lot of other forms of SHAP that are presented in the paper (Deep SHAP, Low Order SHAP , Max SHAP , Linear SHAP , Tree SHAP etc) , ones that make use of model-specific assumptions and optimizations to speed up the algorithm and the sampling process, but Kernel SHAP is the one among them that is universal and can be applied to any type of machine learning model. This general applicability is why we chose Kernel SHAP as the first form of SHAP to implement for TrustyAI.
I’ll run through an example of the Python SHAP implementation provided by Lundberg and Lee. So first I’ll grab a dataset to run our example over, and I’ve picked the Boston housing price dataset, which is a dataset consisting of various attributes about Boston neighborhoods and the corresponding house prices within that neighborhood.
Incomplete. Coming soon
","url":"http://localhost:2368/shapley-additive-explanations-shap/","canonical_url":null,"uuid":"2239e072-7466-4d35-ba08-3fd48b607f1a","codeinjection_foot":null,"codeinjection_head":null,"codeinjection_styles":null,"comment_id":"63ff26ec72a3c427182edd36","reading_time":7,"send_email_when_published":null,"email_subject":null,"childHtmlRehype":{"html":"In this article , we will talk about SHAP in detail. We will discuss What are Shapley Values ? , Mathematical foundation behind the Shapley Values and How does SHAP (Shapley Additive Explanations) reframes the Shapey Value problem in detail. Also we will discuss What is Local Accuracy, Missingness, and Consistency in the context of explainable models , What is the Shapley Kernel etc.
SHAP values were introduced by Lundberg and Lee (2016) as a way to provide local explanations for machine learning models. They built upon Shapley's work in cooperative game theory, using the Shapley value to explain how each feature in a model contributes to the prediction for a particular instance.
Shapley values were invented by Lloyd Shapley as a way of providing a fair solution to the following question: if we have a coalition C that collaborates to produce a value V, how much did each individual member contribute to that final value ?
So what does this mean ? We have a coalition C, a group of cooperating members that work together to produce some value V, called the coalition value. This could be something like, a corporation of employees that together generate a certain profit, or a dinner group running up a restaurant bill. We want to know exactly how much each member contributed to that final coalition value; what share of the profit does each employee deserve, how much each person in the dinner party owes to settle the bill.
\nHowever, answering this gets tricky when there are interacting effects between members, when certain permutations cause members to contribute more than the sum of their parts. To find a fair answer to this question that takes into account these interaction effects, we can compute the Shapley value for each member of the coalition.
So let’s how we can compute the Shapley value for member 1 of our example coalition. The way this is done is by sampling a coalition that contains member 1, and then looking at the coalition formed by removing that member.
\nWe then look at the respective values of these two coalitions, and compare the difference between the two.
\nThis difference is the marginal contribution of member 1 to the coalition consisting of members 2, 3, and 4; how much member 1 contributed to that specific group.
\nSo we then enumerate all such pairs of coalitions, that is, all pairs of coalitions that only differ based on whether or not member 1 is included, and then look at all the marginal contributions for each.
\nThe mean marginal contribution is the Shapley value of that member.
\nWe can do this same process for each member of the coalition, and we’ve found a fair solution to our original question.!
Mathematically, the whole process looks like this, but all we need is to know that the Shapley value is the average amount of contribution that a particular member makes to the coalition value.
Now, translating this concept to model explainability is relatively straightforward, and that’s exactly what Scott Lundberg and Su-In Lee did in 2017 with their paper “A Unified Approach to Interpreting Model Predictions,” where they introduced SHAP.
\nSHAP reframes the Shapley value problem from one where we look at how members of a coalition contribute to a coalition value to one where we look at how individual features contribute to a model’s outputs.
They do this in a very specific way, one that we can get a clue to by looking at the name of their algorithm; Shapley Additive Explanations. We know what Shapley values are, we know what Explanations are, but what do they mean by Additive?
\nLundberg and Lee define an additive feature attribution as follows: if we have a set a of inputs x, and a model f(x), we can define a set of simplified local inputs x’ (which usually means that we turn a feature vector into a discrete binary vector, where features are either included or excluded) and we can also define an explanatory model g.
\nWhat we need to ensure is that One: if x’ is roughly equal to x then g(x’) should be roughly equal to f(x)
two: g must take below form, where phi_0 is the null output of the model, that is, the average output of the model, and phi_i is the explained effect of feature_i; how much that feature changes the output of the model. This is called it’s attribution.
If we have these two, we have an explanatory model that has additive feature attribution. The advantage of this form of explanation is really easy to interpret; we can see the exact contribution and importance of each feature just by looking at the phi values.
Now Lundberg and Lee go on to describe a set of three desirable properties of such an additive feature method; local accuracy, missingness, and consistency.
\nLocal accuracy : it simply says if the input and the simplified input are roughly the same, then the actual model and the explanatory model should produce roughly the same output.
Missingness states that if a feature is excluded from the model, it’s attribution must be zero; that is, the only thing that can affect the output of the explanation model is the inclusion of features, not the exclusion.
Finally, we have consistency (and this one’s a little hard to represent mathematically), but it states that if the original model changes so that the a particular feature’s contribution changes, the attribution in the explanatory model cannot change in the opposite direction;
So for example, if we have a new model where a specific feature has a more positive contribution than in the original; the attribution in our new explanatory model cannot decrease.
Now while a bunch of different explanation methods satisfy some of these properties, Lundberg and Lee argue that only SHAP satisfies all three; if the feature attributions in our additive explanatory model are specifically chosen to be the shapley values of those features, then all three properties are upheld.
The problem with SHAP, however, is that computing Shapley values means you have to sample the coalition values for each possible feature permutation, which in a model explainability setting means we have to evaluate our model that number of times.
For a model that operates over 4 features, it’s easy enough, it’s just 16 coalitions to sample to get all the Shapley values. For 32 features, that’s over 17 billion samples, which is entirely untenable. To get around this, Lundberg and Lee devise the Shapley Kernel, a means of approximating shapley values through much fewer samples.
So what we do is we pass samples through the model, of various feature permutations of the particular datapoint that we’re trying to explain.
Of course, most ML models won’t just let you omit a feature !
\nSo what we do is define a background dataset B, one that contains a set of representative datapoints that the model was trained over.
\nWe then fill in our omitted feature or features with values from the background dataset, while holding the features that are included in the permutation fixed to their original values.
\nWe then take the average of the model output over all of these new synthetic datapoints as our model output for that feature permutation, which we’ll call that y bar.
\n\nSo once we have a number of samples computed in this way,
\nWe can formulate this as a weighted linear regression, with each feature assigned a coefficient.
With a very specific choice of weighting for each sample, based on a combination of the total number of features in the model, the number of coalitions with the same number of features as this particular sample, and the number of features included and excluded in this permutation, we ensure that the solution to this weighted linear regression is such that the returned coefficients are equivalent to the Shapley values.
This weighting scheme is the basis of the Shapley Kernel, and the weighted linear regression process as a whole is Kernel SHAP.
Now, there are a lot of other forms of SHAP that are presented in the paper (Deep SHAP, Low Order SHAP , Max SHAP , Linear SHAP , Tree SHAP etc) , ones that make use of model-specific assumptions and optimizations to speed up the algorithm and the sampling process, but Kernel SHAP is the one among them that is universal and can be applied to any type of machine learning model. This general applicability is why we chose Kernel SHAP as the first form of SHAP to implement for TrustyAI.
I’ll run through an example of the Python SHAP implementation provided by Lundberg and Lee. So first I’ll grab a dataset to run our example over, and I’ve picked the Boston housing price dataset, which is a dataset consisting of various attributes about Boston neighborhoods and the corresponding house prices within that neighborhood.
Incomplete. Coming soon
","htmlAst":{"type":"root","children":[{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"In this article , we will talk about SHAP in detail. We will discuss What are Shapley Values ? , Mathematical foundation behind the Shapley Values and How does SHAP (Shapley Additive Explanations) reframes the Shapey Value problem in detail. Also we will discuss What is Local Accuracy, Missingness, and Consistency in the context of explainable models , What is the Shapley Kernel etc. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h3","properties":{"id":"what-is-shapley-values"},"children":[{"type":"text","value":"What is Shapley Values ?"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"SHAP values were introduced by "},{"type":"element","tagName":"a","properties":{"href":"https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf","rel":["noopener","ugc","nofollow"]},"children":[{"type":"text","value":"Lundberg and Lee (2016)"}]},{"type":"text","value":" as a way to provide local explanations for machine learning models. They built upon Shapley's work in cooperative game theory, using the Shapley value to explain how each feature in a model contributes to the prediction for a particular instance."}]},{"type":"element","tagName":"blockquote","properties":{"className":["kg-blockquote-alt"]},"children":[{"type":"text","value":"Shapley values were invented by Lloyd Shapley as a way of providing a fair solution to the following question: "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"if we have a coalition "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"C"}]},{"type":"text","value":" that collaborates to produce a value "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"text","value":"V"}]},{"type":"text","value":", how much did each individual member contribute to that final value ?"}]},{"type":"text","value":" "}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So what does this mean ? We have a coalition C, a group of cooperating members that work together to produce some value V, called the coalition value. This could be something like, a corporation of employees that together generate a certain profit, or a dinner group running up a restaurant bill. We want to know exactly how much each member contributed to that final coalition value; what share of the profit does each employee deserve, how much each person in the dinner party owes to settle the bill."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222118086-efdfcda4-01e6-4d4d-bc6d-30ce019c464b.png","align":"center","alt":"Shapley Values","width":720,"height":434},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"However, answering this gets tricky when there are interacting effects between members, when certain permutations cause members to contribute more than the sum of their parts. "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"To find a fair answer to this question that takes into account these interaction effects, we can compute the Shapley value for each member of the coalition."}]}]}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So let’s how we can compute the Shapley value for member 1 of our example coalition. The way this is done is by "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"sampling a coalition that contains member 1, and then looking at the coalition formed by removing that member."}]},{"type":"text","value":" "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222119980-a7c2c357-eecf-4ea9-8de6-7eefacf4e362.png","align":"center","alt":"Shapley Values","width":720,"height":404},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We then look at the respective values of these two coalitions, and compare the difference between the two."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222120650-7a9cd97e-3dfd-488c-9a24-45638a75847a.png","align":"center","alt":"Shapley Values","width":720,"height":512},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"This difference is the marginal contribution of member 1 to the coalition consisting of members 2, 3, and 4; how much member 1 contributed to that specific group."}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222120708-c27b48f7-7712-4e7d-ae44-c998a859215c.png","align":"center","alt":"Shapley Values","width":720,"height":136},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So we then enumerate all such pairs of coalitions, that is, all pairs of coalitions that only differ based on whether or not member 1 is included, and then look at all the marginal contributions for each. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222121573-e1c9b15d-d883-44f0-b6e3-41fff235ff06.png","align":"center","alt":"Shapley Values","width":720,"height":333},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"The mean marginal contribution is the Shapley value of that member."}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222121676-1e0e6852-0b66-44b7-bbd4-11fd8e25b39f.png","align":"center","alt":"Shapley Values","width":720,"height":143},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We can do this same process for each member of the coalition, and we’ve found a fair solution to our original question.!"}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Mathematically, the whole process looks like this, but all we need is to know that the "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Shapley value is the average amount of contribution that a particular member makes to the coalition value."}]}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h3","properties":{"id":"shapley-values-to-shap"},"children":[{"type":"text","value":"Shapley Values To SHAP"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Now, translating this concept to model explainability is relatively straightforward, and that’s exactly what Scott Lundberg and Su-In Lee did in 2017 with their paper "},{"type":"element","tagName":"a","properties":{"href":"https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf"},"children":[{"type":"text","value":"“A Unified Approach to Interpreting Model Predictions,”"}]},{"type":"text","value":" where they introduced SHAP. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222123143-27853042-9e12-4526-91dd-466e04f8a68f.png","align":"center","alt":"Shapley Paper","width":720,"height":666},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"SHAP reframes the Shapley value problem from one where we look at "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"how members of a coalition contribute to a coalition value"}]},{"type":"text","value":" to one where we look at "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"how individual features contribute to a model’s outputs."}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222123344-4acf9530-e696-4ee7-80f0-4835cd9cc775.png","align":"center","alt":"Shapley Paper","width":580,"height":352},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"They do this in a very specific way, one that we can get a clue to by looking at the name of their algorithm; "},{"type":"element","tagName":"span","properties":{"style":"color:red"},"children":[{"type":"text","value":"Shapley"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"style":"color:blue"},"children":[{"type":"text","value":"Additive"}]},{"type":"text","value":" "},{"type":"element","tagName":"span","properties":{"style":"color:green"},"children":[{"type":"text","value":"Explanations"}]},{"type":"text","value":". We know what "},{"type":"element","tagName":"span","properties":{"style":"color:red"},"children":[{"type":"text","value":"Shapley"}]},{"type":"text","value":" values are, we know what "},{"type":"element","tagName":"span","properties":{"style":"color:green"},"children":[{"type":"text","value":"Explanations"}]},{"type":"text","value":" are, but what do they mean by "},{"type":"element","tagName":"span","properties":{"style":"color:blue"},"children":[{"type":"text","value":"Additive"}]},{"type":"text","value":"?"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Lundberg and Lee define an additive feature attribution as follows: if we have a set a of "},{"type":"element","tagName":"span","properties":{"style":"color:blue"},"children":[{"type":"text","value":"inputs x"}]},{"type":"text","value":", and "},{"type":"element","tagName":"span","properties":{"style":"color:blue"},"children":[{"type":"text","value":"a model f(x)"}]},{"type":"text","value":", we can define a set of simplified "},{"type":"element","tagName":"span","properties":{"style":"color:green"},"children":[{"type":"text","value":"local inputs x’"}]},{"type":"text","value":" (which usually means that we turn a feature vector into a discrete binary vector, where features are either included or excluded) and we can also define an "},{"type":"element","tagName":"span","properties":{"style":"color:green"},"children":[{"type":"text","value":"explanatory model g"}]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"What we need to ensure is that One: if x’ is roughly equal to x then g(x’) should be roughly equal to f(x)"}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222126320-96c62ae5-2dcd-46eb-8a64-38727338c7dd.png","align":"center","alt":"SHAP","width":404,"height":222},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"two: g must take below form, where "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"phi_0 is the null output of the model"}]},{"type":"text","value":", that is, the average output of the model, and "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"phi_i is the explained effect of feature_i"}]},{"type":"text","value":"; how much that feature changes the output of the model. "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"This is called it’s attribution."}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222126643-5e4a7d98-53a6-4389-86cd-ac6d37041150.png","align":"center","alt":"SHAP","width":625,"height":261},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"If we have these two, we have an explanatory model that has additive feature attribution. The advantage of this form of explanation is really easy to interpret; we can see the exact contribution and importance of each feature just by looking at the phi values."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h3","properties":{"id":"local-accuracy-missingness-and-consistency"},"children":[{"type":"text","value":"Local Accuracy, Missingness and Consistency"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Now Lundberg and Lee go on to describe a set of three desirable properties of such an additive feature method; "},{"type":"element","tagName":"span","properties":{"style":"color:red"},"children":[{"type":"text","value":"local accuracy"}]},{"type":"text","value":", "},{"type":"element","tagName":"span","properties":{"style":"color:blue"},"children":[{"type":"text","value":"missingness"}]},{"type":"text","value":", and "},{"type":"element","tagName":"span","properties":{"style":"color:green"},"children":[{"type":"text","value":"consistency"}]},{"type":"text","value":"."}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h4","properties":{"id":"local-accuracy"},"children":[{"type":"text","value":"Local Accuracy"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Local accuracy : it simply says if the input and the simplified input are roughly the same, then the actual model and the explanatory model should produce roughly the same output. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h4","properties":{"id":"missingness"},"children":[{"type":"text","value":"Missingness"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Missingness states that if a feature is excluded from the model, it’s attribution must be zero; that is, the only thing that can affect the output of the explanation model is the inclusion of features, not the exclusion. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h4","properties":{"id":"consistency"},"children":[{"type":"text","value":"Consistency"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Finally, we have consistency (and this one’s a little hard to represent mathematically), but it states that if the original model changes so that the a particular feature’s contribution changes, the attribution in the explanatory model cannot change in the opposite direction; "}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So for example, if we have a new model where a specific feature has a more positive contribution than in the original; the attribution in our new explanatory model cannot decrease. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222395551-04403b51-1e8f-4bbe-84d7-1bb2f9d0b966.png","align":"center","alt":"SHAP","width":720,"height":299},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Now while a bunch of different explanation methods satisfy some of these properties, "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"Lundberg and Lee argue that only SHAP satisfies all three"}]},{"type":"text","value":"; "},{"type":"element","tagName":"strong","properties":{},"children":[{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"if the feature attributions in our additive explanatory model are specifically chosen to be the shapley values of those features, then all three properties are upheld."}]}]}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h3","properties":{"id":"shapley-kernel"},"children":[{"type":"text","value":"Shapley Kernel"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"The problem with SHAP, however, is that computing Shapley values means you have to sample the coalition values for each possible feature permutation, which in a model explainability setting means we have to evaluate our model that number of times. "}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"For a model that operates over 4 features, it’s easy enough, it’s just 16 coalitions to sample to get all the Shapley values. For 32 features, that’s over 17 billion samples, which is entirely untenable. "},{"type":"element","tagName":"em","properties":{},"children":[{"type":"text","value":"To get around this, Lundberg and Lee devise the Shapley Kernel, a means of approximating shapley values through much fewer samples."}]}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So what we do is we pass samples through the model, of various feature permutations of the particular datapoint that we’re trying to explain."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222397887-4451072f-be0b-4e27-accc-e401b7ac35bf.png","align":"center","alt":"Shapley-Permu","width":633,"height":407},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Of course, most ML models won’t just let you omit a feature !"}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222398010-2e845aae-b21e-48ea-b16e-5da41af83b86.png","align":"center","alt":"Shapley-Omit-Feature","width":588,"height":112},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So what we do is define a background dataset B, one that contains a set of representative datapoints that the model was trained over. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222398330-d64ddccf-b8f2-4f5e-880a-dfe05d99eb77.png","align":"center","alt":"BackgroundDataset","width":636,"height":239},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We then fill in our omitted feature or features with values from the background dataset, while holding the features that are included in the permutation fixed to their original values."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222398831-31f4f53d-2610-47d6-9b45-89a7ff663fe4.png","align":"center","alt":"IncludedFeaturesFromDatasetB","width":621,"height":331},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We then take the average of the model output over all of these new synthetic datapoints as our model output for that feature permutation, which we’ll call that y bar."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222399307-fc75df99-27a3-4d11-8fff-d785616fbee9.png","align":"center","alt":"Shapley-Avg","width":720,"height":291},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222399733-737c4ba1-23f2-4f97-996e-f3ace97fb3cf.png","align":"center","alt":"Shapley-ybar","width":683,"height":340},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"So once we have a number of samples computed in this way, "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222399945-9a2510d2-5595-42eb-887c-5b5762c9aa1e.png","align":"center","alt":"Shapley-NoOfYbar","width":672,"height":400},"children":[]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"We can formulate this as a weighted linear regression, with each feature assigned a coefficient."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222400421-c3964125-f4d1-44e1-ba12-1231cb76671a.png","align":"center","alt":"SShapley-LinearReg","width":672,"height":400},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"With a very specific choice of weighting for each sample, based on a combination of the total number of features in the model, the number of coalitions with the same number of features as this particular sample, and the number of features included and excluded in this permutation, we ensure that the solution to this weighted linear regression is such that the returned coefficients are equivalent to the Shapley values. "}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222400526-11e50df4-880d-44e8-bd15-812560a8e005.png","align":"center","alt":"Shapley-Weighting","width":720,"height":148},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"This weighting scheme is the basis of the Shapley Kernel, and the weighted linear regression process as a whole is Kernel SHAP."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"img","properties":{"src":"https://user-images.githubusercontent.com/33357428/222400590-c55c9b16-5ce4-4022-a6aa-73c294f2d1a8.png","align":"center","alt":"Shapley-Kernel-Weights","width":720,"height":359},"children":[]},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Now, there are a lot of other forms of SHAP that are presented in the paper (Deep SHAP, Low Order SHAP , Max SHAP , Linear SHAP , Tree SHAP etc) , ones that make use of model-specific assumptions and optimizations to speed up the algorithm and the sampling process, but Kernel SHAP is the one among them that is universal and can be applied to any type of machine learning model. This general applicability is why we chose Kernel SHAP as the first form of SHAP to implement for TrustyAI."}]},{"type":"comment","value":"kg-card-begin: markdown"},{"type":"element","tagName":"h3","properties":{"id":"kernel-shap-example"},"children":[{"type":"text","value":"Kernel SHAP Example"}]},{"type":"text","value":"\n"},{"type":"comment","value":"kg-card-end: markdown"},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"I’ll run through an example of the Python SHAP implementation provided by Lundberg and Lee. So first I’ll grab a dataset to run our example over, and I’ve picked the Boston housing price dataset, which is a dataset consisting of various attributes about Boston neighborhoods and the corresponding house prices within that neighborhood."}]},{"type":"element","tagName":"p","properties":{},"children":[{"type":"text","value":"Incomplete. Coming soon"}]}],"data":{"quirksMode":false}},"tableOfContents":[{"id":"what-is-shapley-values","heading":"What is Shapley Values ?"},{"id":"shapley-values-to-shap","heading":"Shapley Values To SHAP"},{"id":"local-accuracy-missingness-and-consistency","heading":"Local Accuracy, Missingness and Consistency","items":[{"id":"local-accuracy","heading":"Local Accuracy"},{"id":"missingness","heading":"Missingness"},{"id":"consistency","heading":"Consistency"}]},{"id":"shapley-kernel","heading":"Shapley Kernel"},{"id":"kernel-shap-example","heading":"Kernel SHAP Example"}]},"featureImageSharp":{"base":"photo-1522069213448-443a614da9b6.jpg","publicURL":"/static/c604d810168bc7c01df3faa4bf1a4f00/photo-1522069213448-443a614da9b6.jpg","imageMeta":{"width":2000,"height":1330},"childImageSharp":{"fluid":{"base64":"data:image/jpeg;base64,/9j/2wBDABALDA4MChAODQ4SERATGCgaGBYWGDEjJR0oOjM9PDkzODdASFxOQERXRTc4UG1RV19iZ2hnPk1xeXBkeFxlZ2P/2wBDARESEhgVGC8aGi9jQjhCY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2P/wgARCAANABQDASIAAhEBAxEB/8QAFwABAQEBAAAAAAAAAAAAAAAAAAMBBf/EABQBAQAAAAAAAAAAAAAAAAAAAAH/2gAMAwEAAhADEAAAAeVsLDEJ/8QAGxAAAgEFAAAAAAAAAAAAAAAAARIDAAIQERP/2gAIAQEAAQUCjXZjK46XJX//xAAWEQADAAAAAAAAAAAAAAAAAAAQESH/2gAIAQMBAT8BiH//xAAUEQEAAAAAAAAAAAAAAAAAAAAQ/9oACAECAQE/AT//xAAYEAADAQEAAAAAAAAAAAAAAAAQEkEAIv/aAAgBAQAGPwLrNCsH/8QAGRABAQADAQAAAAAAAAAAAAAAAREAEIFx/9oACAEBAAE/IQlF7MG/Tm1Lun//2gAMAwEAAgADAAAAEAwP/8QAFhEBAQEAAAAAAAAAAAAAAAAAARAh/9oACAEDAQE/EBwSf//EABYRAQEBAAAAAAAAAAAAAAAAABEBEP/aAAgBAgEBPxArn//EABwQAAICAgMAAAAAAAAAAAAAAAERITEQYQBBwf/aAAgBAQABPxAarAAhquZ4ENRbaNF+Zr1qAFA6x//Z","aspectRatio":1.5086206896551724,"src":"/static/c604d810168bc7c01df3faa4bf1a4f00/ea4ab/photo-1522069213448-443a614da9b6.jpg","srcSet":"/static/c604d810168bc7c01df3faa4bf1a4f00/477ba/photo-1522069213448-443a614da9b6.jpg 175w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/06776/photo-1522069213448-443a614da9b6.jpg 350w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/ea4ab/photo-1522069213448-443a614da9b6.jpg 700w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/3055e/photo-1522069213448-443a614da9b6.jpg 1050w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/eff08/photo-1522069213448-443a614da9b6.jpg 1400w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/4e5f3/photo-1522069213448-443a614da9b6.jpg 2000w","srcWebp":"/static/c604d810168bc7c01df3faa4bf1a4f00/89afa/photo-1522069213448-443a614da9b6.webp","srcSetWebp":"/static/c604d810168bc7c01df3faa4bf1a4f00/9fca7/photo-1522069213448-443a614da9b6.webp 175w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/37a4e/photo-1522069213448-443a614da9b6.webp 350w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/89afa/photo-1522069213448-443a614da9b6.webp 700w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/78e7a/photo-1522069213448-443a614da9b6.webp 1050w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/03d34/photo-1522069213448-443a614da9b6.webp 1400w,\n/static/c604d810168bc7c01df3faa4bf1a4f00/49d6b/photo-1522069213448-443a614da9b6.webp 2000w","sizes":"(max-width: 700px) 100vw, 700px"}}}}}]}},"pageContext":{"slug":"causal-machine-learning-part-5","prev":"explaining-bayesian-neural-networks","next":"causal-machine-learning-part-4","tag":"machine-learning","limit":3,"skip":0,"primaryTagCount":7,"collectionPaths":{}}},"staticQueryHashes":["1272700106","1676991999","2138873178","2546165603","2681841279","2938721187","293880488","3052966952","4156497161"]}