Many people in their daily jobs lack the information they need to make well informed decisions, take decisive action, or solve problems. This lack of “the right information at the right time” results in unhappy customers, loss of sales, or the inability to recover quickly from broken processes and systems. Tragically, the data that is needed to solve this problem already exists somewhere in your systems, but it can’t be organized and served up when needed. While the company drowns in a sea of data every individual goes thirsty on their own desert island. The lack of data integration and the island (aka silo) people’s unwillingness to share represents one of the great data management challenges of the 21st century.
Sherlock Holmes – the world’s greatest detective – solved mysteries by linking together shreds of information in ways that were not immediately obvious to the casual observer. This is the same problem many people face at work, they understand the data made immediately available to them, but have only a vague awareness of what additional data is available within the company. Faced with a crisis that requires resolution (or senior managers demanding “answers”), they need a Sherlock Holmes to help navigate these islands of data. The result is predictable, everyone dumping what they have into spreadsheets, a hero effort to organize and analyze the data, and a one-off analysis to save the day, but no lasting value or solution that can be applied to the next crisis.
With hundreds of application systems and a spider web of data warehouses, operational data stores, MDM/CDM repositories, Data Lakes, and BI tools – end users still can’t gain the situational awareness they need to deal with day to day issues.
- Question: Why didn’t “data architecture” solve this problem for us?
- Answer: There is no metadata defining the data artifacts – therefore there is no way to navigate the data.
Organizations struggle to provide adequate management and governance of their ever increasing data assets. Traditional Metadata Management approaches combined with silo’ed Data Governance tools are inadequate to meet today’s demands. Similar issues exist in the IT Service Management (ITSM) space with traditional ITIL / CMDB tools failing to bridge the gap between business and technical architectures.
An innovative approach to solving this age-old problem is emerging – The “Knowledge Graph”. Knowledge Graphs combine best practices from semantics with graph database technologies to build a meaning based map over the existing data stores. This series of blog posts will introduce the concepts and solutions needed to get started on this exciting adventure.
This article originally appeared on my DATAVERSITY Blog on