Data-driven decision making: the reason to use data science to enable business value
Data is the lifeblood for digitalised businesses, and making decisions based on that data is crucial to success. In today’s complex, fast-paced and ever-changing business landscape, relying on intuition and good feelings it's not enough. In order to stay ahead of the competition and make the most informed decisions, businesses must adopt a data-driven approach. We’ll explore the benefits of data-driven decision making, its key characteristics, how it is related to data science as well as real-life examples of how data-driven decision making can be applied in different areas of the business.
Decision making (DM) in general is a way to evaluate the consequences of a decision so that an optimal decision can be chosen. It is about how to make the best decision (normative decision making). The criteria are technical and economic, such as cost-benefit-analysis or risk assessment.
Objectives for decision making in industry can be very diverse. Based on individual characteristics, DM can be structured into three types:
Strategic: general goals and visions, e.g. development of a new product.
Time horizon: mid-term, long-term
Tactical (Planning): Present issues, e.g. production planning and scheduling.
Time horizon: hours, days
Operational: Present issues, e.g. fault detection or process control.
Time horizon: seconds, minutes
Data-driven decision making (DDDM)
Data-driven decision making (DDDM) is one way how systems can come up with a decision. As the name suggests, the decisions made are based on data as well as on methods that extract the patterns and information given in the data. Different types of methods can be used for modelling the decision-making process. For well-known processes, rule-based or model-based methods are very suitable. Risk systems can be used, if the underlying process is not fully understood but the uncertainty of the unknown aspects can be modelled explicitly. Those types of systems are rather data-inspired instead of data-driven since data is used as an input for the system. Still, the rules are defined in an explicit manner and therefore they are not stochastic. But in many cases, the uncertainty cannot be defined explicitly. This is when we talk about data-driven stochastic decision making. Data Science comes into play to handle those use cases.
Figure 1: Deterministic Systems vs. Risk Systems vs. Machine Learning Systems.
Benefits of using DDDM
Objectivity: Decisions are fact-based and therefore reduce the influence of personal bias.
Transparency & accountability: Transparency refers to the quality of knowing for what reasons a decision was made, while accountability means being willing to accept responsibility for the actions, and being willing to explain and justify those actions.
Improved accuracy: Data-driven decisions are based on facts and evidence, which can lead to more accurate decisions.
Improved efficiency: Data-driven decision making can help organisations make decisions faster and with less effort through automatisation.
Continuous improvement: Data-driven decision making enables organisations to track the effectiveness of their decisions and make adjustments as needed, leading to continuous improvement.
Scalability: Data-driven decision making allows organisations to process large amounts of data and make decisions at scale.
Figure 2: Benefits of using DDDM.
Data science is the process of extracting insights from data by using statistical methods to inform decision making. Those methods include machine learning algorithms to extract the statistical regularities from data, which they represent in the form of models. The underlying pattern should be recognised as accurately as possible to be able to apply a generalisable statement to new or unknown data afterwards.
In simple words, rule-based approaches have hard limits: either something applies to the rules or not. In most cases, they are a simplification of reality. As an example, we want to classify whether a given image shows a tree or something else. We define concrete rules like the size of leaves or the colour of the trunk. We apply those rules to our picture and either it fits the rules or not. Since trees can be very diverse, this is a complex task. It is complicated to come up with a set of rules that can be easily applied to a large variety of trees. ML-based systems have grey areas and can handle individual cases flexibly if there is a general abstraction that can be captured by the algorithm.
The figure 3 shows a simple illustration of the difference between these two methods. Imagine you use both methods to build a classifier for the images at the bottom of the figure. I could not come up with a set of rules to satisfy all images with a tree. On the other hand, an abstract representation does easily meet the goal. ML approaches are also a simplification of reality, which is why ML-based solutions must be evaluated to what extent they represent reality. The image on the very right might be classified as a tree but it shows a bunch of flowers.
Figure 3: Tree simplification.
As we have just learned, data-driven decision making and data science are closely related. For both methods, there is a well-defined process that supports their implementation. Data-driven decision making is the process of using data and statistical analysis to inform decisions, rather than relying on intuition or experience. It involves identifying the key decision to be made, gathering relevant data, analysing the data using statistical techniques and machine learning models, and using the insights gained from the analysis to inform the decision.
Figure 4: Process that supports their implementation.
Use Cases, for which DDDM is commonly used:
Demand & sales forecasting: Optimising processes in sales operations and supply chains through data-driven forecasting.
Predictive maintenance: Machine learning can be used to analyse sensor data from equipment and predict when maintenance is needed. This can help companies avoid costly equipment failures and improve overall efficiency.
Marketing: Targeting the right customers with the right message at the right time through data-driven segmentation and personalisation.
Finance: DDDM can be used to analyse financial transactions and identify patterns that may indicate fraudulent activity.
Image recognition: Analysis of images, such as photos or videos, for object detection, facial recognition, and scene understanding. This can be used in industries such as security, retail, and healthcare.
Natural language processing: DDDM can be used in text analysis and natural language processing tasks, such as sentiment analysis, language translation, and text summarisation. This can be used in customer service, marketing, and finance.
This is the first of a series of articles on Data-driven Decision Making, Data Science, AI and Machine Learning. In the following ones, we will have a closer look at how to approach projects within the Data Science Life Cycle.
We would be pleased to discuss your interest with you in person as well. Contact the Munich office or write an email to firstname.lastname@example.org.