What is Data Discovery?
Data discovery is the process of obtaining actionable information by finding patterns in data from multiple sources with interactive visual analysis. The term is used to express a mode of analysis in which users attempt to get a holistic view of all their data sources. The two main ingredients are the ability to join data sources and the interactive visual analysis component which allows you to explore the data to find patterns.
The ability to join data enables you to analyze data from all your sources for that big picture view. With distributed joins, you can mashup datasets from different data sources.
Interactive visual analysis enables the discovery part of the definition. In short, interactivity allows you to view the data from many angles. Specifically, you can drill down, slice and dice, pivot, parametrize, etc. This empowers you to start looking at the data and discover actionable patterns to captivate on.
Data Discovery vs. Traditional BI
More data was created in 2017 than in the previous 5,000 years of human history, and that trend is continuing to grow. It may not come as a surprise then, to say that business’s analytics needs too have grown. In today’s data-driven world, all types of businesses are using data to inform and improve day to day workflows and decisions. And more and more companies are looking to data to drive a business forward and stay ahead in the competitive markets of today. Traditional BI has its place in this process, but end users want more agility and quicker data insights. That’s where data discovery comes in.
So what is traditional BI? How does it differ from data discovery? And what role does data discovery play in modern BI systems?
Let’s start with traditional BI. Traditional BI encompasses a variety of tools, methodologies, and techniques used to turn data into actionable business intelligence. When business intelligence started becoming more widely used by businesses, it was mainly used by technical analysts, data scientists, and/or developers. All of whom would spend many man hours and technical resources to develop reports, dashboards, and other data visualizations that would be mainly used by executives. This process often required tedious implementations and specific expertise for even day to day use. This meant it was cumbersome and expensive for businesses to fully utilize.
As the true value of these types of tools started to be realized, a demand for more intuitive, simpler tools that could be used a variety of users technical or non-technical started to grow. And the market met this demand in two different ways: with more self-service analytics and with data discovery tools. Each of which, helped meet this demand by giving end users the ability to manipulate data into visualizations for use in daily workflow and decision support. Self-service did so through the creation of structured reports and dashboards, and data discovery through intuitive and quick visual analysis.
Both of these capabilities provided organizations with greater agility to find and react to business problems and empowered even non-technical end users to dig through their data. However, technical developers and BI specialist are still a necessity in the data processing system of modern BI. But instead of actively participating in developing reports and dashboards on a day to day basis, the majority of organizations have these technical teams managing the data resources used by end users. In this way they are one step removed from the physical process of utilizing BI in day-to-day business, meaning they have more time to manage other essential IT functions. In the case of Logi Report, this is done via a semantic data layer system we call a business view. A business view is a data resource system which is created and managed by a developer or report developer, and which can contain a variety of different data resources and queries. A business view allows an end user to better understand the data resources available to them, without having to understand the underlying data structure of your system.
Modern BI systems often include the features of traditional BI as well as self-service and data discovery, and as the BI market has developed, these capabilities have been expanded upon and refined to be easier to use and easier to implement. And as these solutions become easier to use, the more they’ve been embedded and used in business applications already used by different departments of businesses. This has allowed end users to be more efficient in their use of analytics by not only putting analytical capabilities at their fingertips in the systems they use every day, but also helping to provide an additional layer of context so in-context analysis can be done quickly. Meaning users can identify potential places for improvement and act on those systems without having to leave a single software suite.
For example, a warehouse manager using an inventory and logistics management system which includes data discovery tools could quickly pull information on the current inventory status of a type of product, see how many of that product is left on the shelf, and when the next shipment is due to arrive. If a new order larger than the current inventory of that product has come in and is to be fulfilled in a week, and the next shipment isn’t due to arrive for 8 days, that’s an issue. Using a traditional BI system, gathering all of this data might take searching through several reports all of which might use disparate data sources. But using a data discovery tool, it’s possible to gather all of this data quickly, allowing the warehouse manager to take action quickly and enabling them to expedite the incoming order so the outgoing order can be fulfilled in time. This is a simple example, but it shows how the ability to empower non-technical end users with pre-gathered data resources can have a marked effect on day to day business.
Governed Embeddable Data Discovery
Logi Report’s visual analysis and self-service analytics capabilities allow you to embed white-labeled data discovery into your application. Empowering your users with the type of in-context analysis shown in the above example can be done with ease.
From a developer’s perspective, enabling has these capabilities in multi-tenant applications, or applications with specific permissions and user rights can be challenging because you have to manage both the different data sources, but also the access rights of each for every user. Logi Report fully supports multi-tenant architecture and gives you full security control over your data down to the row and column level, and can match your current application’s access rights. Meaning exposing different data layers to your end users can be done quickly and easily.
- Data discovery enables you to find meaningful patterns in your data through joining multiple data sources and interactive analysis.
- Joining multiple data sources allows you to gain a big picture view of your data to answer any business question.
- Interactive analysis enables you to view your data from many angles to discover actionable patterns.
- Logi Report fully supports embeddable, governed, data discovery.