Model Explainability That Goes Beyond the Notebook for Data Science in Microsoft Fabric
A machine learning model can be remarkably accurate and still leave people completely unconvinced. That is one of the strange realities of working in data science. You can spend weeks cleaning data, testing algorithms, tuning parameters, and validating performance, only to reach the moment when somebody looks at a prediction and asks a very reasonable question: “Why?” Suddenly, accuracy alone does not feel like enough. A probability score might tell you what the model believes, but it does not necessarily help the person receiving that prediction understand how the model arrived there. That gap between prediction and understanding is exactly where model explainability becomes important. What you will learn: In this edition, we’re exploring model explainability in production and how SHAP, FastAPI, and Plotly can work together to make predictions easier to understand. By the time you’re done with this, you’ll have a clear view of how SHAP can explain individual model decisions, how those explanations can be served dynamically through an API, and how interactive visualizations can turn technical attribution values into something far more approachable. Source: Sahir Maharaj (https://sahirmaharaj.com) It is tempting to think of explainability as a chart you generate after training a model, but production explainability is really an ongoing capability. Every new observation can produce a different prediction, and every prediction may have a different explanation. A customer predicted to leave a service might receive a high risk score because of declining engagement, while another customer receives the same score because of repeated service issues. The prediction may look identical at the surface, but the reasoning behind it can be completely different. That difference matters when somebody needs to decide what to do next. This is where SHAP becomes particularly valuable because it gives you a structured way to describe feature contributions. Imagine a model predicting whether a machine is likely to fail. The prediction itself might tell you that failure risk is 82 percent. Useful, certainly, but incomplete. SHAP can help reveal that rising operating temperature, vibration intensity, and unusually long operating hours are pushing the prediction upward, while another factor such as recent maintenance is reducing the predicted risk. Suddenly the prediction becomes much easier to reason about. You still have a probabilistic model, but you also have something resembling an explanation of its behavior. As a data scientist, I find that this distinction between global and local understanding is especially important. During model development, I often want to understand the model globally. Which variables matter most overall? Is the model relying heavily on variables I expected it to use? Are there surprising relationships hiding in the data? In production, however, people often care about something much more specific. They want to know why this customer, this transaction, this machine, or this observation received the prediction it did. That is a local explanation, and SHAP is particularly useful for helping bridge that gap. Source: Sahir Maharaj (https://sahirmaharaj.com) You can think about the difference through a credit-risk example. Globally, a model might rely heavily on payment history, debt ratios, and income stability. That tells you something useful about the overall model. But suppose one applicant receives a higher risk prediction than expected. A general feature importance chart does not really answer the person's question. You need to understand what happened for that particular prediction. Perhaps the debt ratio increased the risk score significantly, while long-term employment reduced it. That level of explanation provides much more context than simply showing which variables matter across thousands of predictions. Explainability can also become an important debugging tool for you as the person building the model. Suppose a model performs well according to the metrics you are monitoring, but its explanations repeatedly show that one unexpected variable dominates many predictions. That should make you curious. import numpy as np import pandas as pd import shap import matplotlib.pyplot as plt from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import train_test_split np.random.seed(42) n = 800 X = pd.DataFrame({ "usage_drop": np.random.normal(20, 10, n), "support_calls": np.random.poisson(3, n), "login_days": np.random.randint(1, 30, n), "tenure_months": np.random.randint(1, 72, n), "monthly_cost": np.random.normal(80, 20, n) }) score = ( 0.08 * X["usage_drop"] + 0.45 * X["support_calls"] - 0.05 * X["login_days"] - 0.02 * X["tenure_months"] + 0.015 * X["monthly_cost"] ) y = (score + np.random.normal(0, 1, n) > np.median(score)).astype(int) X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.25, random_state=42 ) model = RandomForestClassifier(n_estimators=150, random_state=42) model.fit(X_train, y_train) explainer = shap.TreeExplainer(model) values = explainer(X_test) class_values = values[:, :, 1] if values.values.ndim == 3 else values shap.plots.beeswarm(class_values, max_display=5) plt.show() shap.plots.waterfall(class_values[0], max_display=5) plt.show() Imagine that somebody submits new information to a prediction service. The model receives those features, processes them, and generates a result. If explainability is part of the design, the same interaction can also produce SHAP information describing the important factors behind that specific prediction. Instead of storing explanations for every imaginable case beforehand, you generate them when they are needed. FastAPI fits naturally into this kind of workflow because it gives the prediction and explanation process a consistent interface. Rather than another application needing to understand your model directly, it can send the required information to an API. The API can coordinate the prediction process, calculate the explanation, and return the relevant output. The application consuming the result does not need to know the mathematical details of SHAP or how the model itself works internally. It simply receives information in an agreed structure. That separation can make a production system much easier to manage. From what I observe when building data science workflows, this separation becomes valuable surprisingly quickly. During early experimentation, everything often exists together. Data preparation, prediction, explanation, and visualization may all happen inside one environment. That is perfectly reasonable when you are exploring an idea. Production introduces more responsibilities. The application requesting a prediction should not need access to the entire modeling environment, and your model should not depend on somebody manually generating explanations. Creating a clear service boundary helps each part of the system do its job without becoming unnecessarily tangled with everything else. Dynamic explanations also give you flexibility over how much information you return. A technical monitoring application may want detailed SHAP values for every available feature. A decision-support interface may only need the five strongest contributors. Another application may need a short summary explaining the strongest positive and negative influences. The explanation engine can be the same while the presentation changes according to the audience. That matters because explainability is not only a modeling problem. It is also a communication problem. Source: Sahir Maharaj (https://sahirmaharaj.com) Performance deserves careful attention here as well. Calculating an explanation adds work beyond producing the prediction itself, and depending on the model, the amount of additional work may not be trivial. That does not mean explanations should be avoided. It means you should think about when they are required. Some systems may need an explanation for every single prediction. Others may generate explanations only when a user requests additional detail. You might also decide that certain high-impact decisions always receive explanations while low-impact automated predictions do not. Production design is often about finding that balance between completeness and responsiveness. There is also an important consistency issue. An explanation is only useful if it corresponds to the exact model and transformation process that generated the prediction. Imagine updating your model but accidentally continuing to generate explanations using an older version. The results could look perfectly reasonable while describing behavior that no longer matches the prediction being shown. import numpy as np import pandas as pd import shap import plotly.graph_objects as go from sklearn.ensemble import GradientBoostingClassifier np.random.seed(7) n = 700 X = pd.DataFrame({ "temperature": np.random.normal(70, 12, n), "vibration": np.random.normal(4, 1.5, n), "operating_hours": np.random.normal(900, 250, n), "maintenance_days": np.random.randint(1, 120, n) }) risk = ( 0.06 * X["temperature"] + 0.7 * X["vibration"] + 0.003 * X["operating_hours"] - 0.025 * X["maintenance_days"] ) y = (risk + np.random.normal(0, 1, n) > np.median(risk)).astype(int) model = GradientBoostingClassifier(random_state=7) model.fit(X, y) explainer = shap.Explainer(model, X) row = pd.DataFrame([{ "temperature": 92, "vibration": 6.8, "operating_hours": 1250, "maintenance_days": 75 }]) prediction = model.predict_proba(row)[0, 1] explanation = explainer(row) names = row.columns.tolist() impacts = explanation.values[0] waterfall = go.Figure(go.Waterfall( orientation="h", y=names, x=impacts, measure=["relative"] * len(names), text=[f"{x:+.2f}" for x in impacts], textposition="outside" )) waterfall.update_layout( title="SHAP Explanation: What Drove This Prediction?", xaxis_title="SHAP Impact on Model Output", yaxis_title="Feature", height=450 ) waterfall.show() gauge = go.Figure(go.Indicator( mode="gauge+number+delta", value=prediction * 100, number={"suffix": "%"}, delta={"reference": 50}, title={"text": "Predicted Machine Failure Risk"}, gauge={ "axis": {"range": [0, 100]}, "steps": [ {"range": [0, 40]}, {"range": [40, 70]}, {"range": [70, 100]} ], "threshold": { "line": {"width": 4}, "thickness": 0.8, "value": prediction * 100 } } )) gauge.update_layout(height=400) gauge.show() pd.DataFrame({ "Feature": names, "Value": row.iloc[0].values, "SHAP Impact": impacts }).sort_values("SHAP Impact", key=abs, ascending=False) The third objective is turning those explanation values into something people can interpret without needing to understand every detail of SHAP. This is where visualization becomes more than decoration. Raw values can tell you exactly how features influenced a prediction, but they require effort to interpret. A Plotly visualization can organise those contributions so the strongest influences immediately stand out. Features pushing a prediction in one direction can be separated visually from those pushing it in the opposite direction, allowing somebody to understand the basic story of the prediction within a few moments. Imagine a customer churn model that predicts an unusually high probability of cancellation. A dynamic explanation might show that a recent drop in product usage is the strongest contributor, followed by repeated support contacts and a reduction in login frequency. At the same time, a long customer relationship might slightly reduce the churn prediction. That is much more meaningful than presenting a table containing feature names and decimal values. The visualization helps the reader understand not only which features mattered but also how they worked together. Source: Sahir Maharaj (https://sahirmaharaj.com) When I look at explanations intended for other people, I try not to assume that more information automatically creates more transparency. It is very easy to generate a detailed visualization containing dozens of variables simply because the values are available. The result may technically be comprehensive while being practically overwhelming. If somebody has to study a chart for several minutes before understanding the main message, the explanation probably needs refinement. In many situations, highlighting the strongest contributors and allowing additional detail to be explored when needed creates a much better experience. Interactivity can make this even more useful. Plotly allows explanations to become something people can investigate instead of merely observe. A user might hover over a feature to inspect the contribution more closely or compare several predictions to understand why similar observations received different results. Imagine comparing two loan applications that received different risk scores. An interactive explanation can make it easier to see that one prediction was strongly influenced by debt levels while another was affected primarily by inconsistent payment history. That comparison helps reveal how the model responds to different combinations of information. At the same time, visual polish should never become a substitute for clarity. A sophisticated interactive chart can create an impression of authority, and that means you have a responsibility to provide enough context around what the visualization represents. The user should understand that the chart explains the model's behavior, not necessarily the underlying truth of the real-world situation. If a feature makes a large contribution, that means the model relied heavily on that feature for the prediction. It does not automatically prove that the feature caused the outcome. Keeping that distinction clear is a major part of responsible model communication. Source: Sahir Maharaj (https://sahirmaharaj.com) So give it a try. Do not wait until you have a huge production system or a complicated machine learning platform before thinking about transparency. Start with one prediction, one explanation, and one clear visualization. Build from there as your understanding grows. The more comfortable you become explaining what your model is doing, the easier it becomes to recognise when its behaviour deserves a closer look. And that ability is valuable whether you are just entering data science or have been building models for years. If you ever need anything while exploring it, just reach out to me. I’ll be more than happy to help. Thanks for taking the time to read my post! I’d love to hear what you think and connect with you! 🙂 LinkedIn Kaggle Topmate (Free Power BI / Data Science Sessions and Resources) Website The Tech Journal (Blog) About the author Sahir Maharaj is a Lead Data Scientist who leads the design and deployment of end-to-end AI solutions that drive strategic decisions at scale. As a Microsoft MVP, he has been featured internationally, including in The Indian Express and on New York's Times Square billboards, and is a prolific content creator on LinkedIn.Mastering Advanced Regression for Data Science in Microsoft Fabric
In this edition, we’re exploring two regression techniques that every data professional eventually bumps into when the simple models stop telling the full story. You’ll get a clear sense of what quantile regression actually solves, especially when your data behaves in unpredictable or uneven ways. By the time you’re done, you’ll feel more confident choosing the regression approach that truly fits the question you’re trying to answer, instead of defaulting to whatever is familiar.Microsoft Fabric, RAG, and the Conversations That Follow You Home
In this edition, we will look at why RAG is quickly becoming a much bigger conversation than simply retrieving a few documents before asking an AI model a question. From there, I want to explore something I think deserves far more attention: what happens when retrieval becomes messy, contextual, relational, and dependent on human feedback. And finally, I want to look at what all of this means for data professionals working around Microsoft Fabric and trying to understand where they fit as AI systems become more capable.Bayesian Thinking for Data Science in Microsoft Fabric
In this edition, we’re exploring Bayesian modeling and how to think about uncertainty in a more realistic, practical way using PyMC3. By the time you’re done, you’ll have a clear intuition for what Bayesian thinking really means, why it’s so useful in day-to-day data work, and how it changes the way you interpret results. We’ll also explore how PyMC3 supports this mindset in a structured but approachable way.Causal Inference for Data Science in Microsoft Fabric
In this edition, we’re exploring into causal inference and why it matters once you move beyond basic reporting and prediction. By the time you’re done, you’ll have a clear understanding of how causal thinking differs from traditional analytics and how to reframe everyday business questions around cause and effect instead of simple correlation. And because insights only matter if they’re understood, we’ll look at how to communicate causal findings clearly and responsibly so decision-makers know what they can trust and act on.Ground Truth Before Go-Live: Building Better Fabric Data Agents with Automated Evaluation Datasets
How do you know if your Fabric data agent is actually ready for production? A few successful test questions aren't enough. In this post, I'll share why ground truth datasets are essential for evaluating Fabric data agents and how a reusable notebook can help you generate them automatically from any Fabric or Power BI semantic model.Level Up Your Forecasting with Temporal Fusion Transformers for Data Science in Microsoft Fabric
In this edition, we’re exploring Temporal Fusion Transformers in a way that actually makes sense in the real world. You’ll also get a clear walkthrough of the key ideas inside the architecture, like variable selection, gating, and attention, and how they work together to make sense of messy, real-life data. And more importantly, you’ll walk away understanding how TFTs can support you with complexity every day, giving you both clarity and confidence in your forecasting work.Exploring Text Intelligence through TF-IDF for Data Science in Microsoft Fabric
In this edition, we’re exploring how TF-IDF helps you discover meaning from language. You’ll see how this technique balances frequency and rarity to spotlight the words that truly matter, instead of the ones that just appear most often. By the time you’re done, you’ll have a solid understanding of how TF-IDF bridges the gap between unstructured text and structured analytics and why they are still relevant in the rise of Large Language Models (LLMs).Mastering Advanced Regex Techniques for Data Science in Microsoft Fabric
In this edition, we’re exploring how advanced regex can help you make sense of unpredictable text fields that show up in real projects. By the time you get through it, you’ll have a clearer way of spotting patterns that other people miss, expressing those patterns in a structured way, and shaping unstructured data into something that finally behaves. You’ll also get a feel for how this kind of thinking changes the way you approach cleaning work overall, because once regex clicks, you start seeing text differently.Semantic Intelligence using Word2Vec and GloVe for Data Science in Microsoft Fabric
In this edition, we’re exploring the world of word embeddings and finally making sense of why they’ve become the backbone of modern NLP. You’ll get a clear feel for what embeddings actually represent, explore how Word2Vec learns meaning through prediction and why that tiny training task uncovers so much structure. And to bring it all together, you’ll learn how to think like an embedding model itself, giving you the intuition you need before stepping into the world of transformer-based NLP.
