In today’s technology-oriented world, mountains of data are created at an unprecedented rate, both outside and inside of an organization. The challenge now is in how to organize and harmonize these piles of data to unleash its true value, and that is where knowledge graphs come in.
Transforming tables of data and unstructured content into a rich, interconnected web of information, a knowledge graph enables organizations to synthesize data and turn it into actionable insights for smarter, better business decisions.
To gain an in-depth understanding about knowledge graph technology, how it works, and the benefits it provides to businesses, read our exclusive interview with Craig Danton, Vice President of Product at Enigma. Danton also shares his thoughts on the future of knowledge graphs and what lies ahead for Enigma this 2019.
Q: Please tell our readers more about Enigma. When was the company established and what problems does it seek to solve?
Craig Danton, Vice President of Product at Enigma
A: Enigma was founded in 2011 by Hicham Oudghiri and Marc DaCosta. During the 2008 financial crisis, they were both perplexed that in a world that was increasingly data-centric, people were still relying on the data directly in front of them rather than looking to data from outside of their four walls to understand the world. They felt that the existing models of the world were weak and saw the need for greater context and connection in data. Solving for this required both a vast amount of data on the real world as well as a way to connect all of this data together with proprietary enterprise data to inform key decisions.
Today, some of the world’s largest organizations rely on Enigma’s vast repository of real-world data and machine learning technology to connect datasets and streamline operations. Enigma’s knowledge bank of people, places, and companies, grounded in public data, creates a unique map of the real world for individuals and enterprises. When combined with internal or third party data, enterprises gain access to a new picture of their world to make smarter, more informed decisions.
Q: How do you define the concept of a knowledge graph?
A: Knowledge graphs transform tables of data and unstructured content into a rich, interconnected web of information about entities (such as people, places, and companies) and the relationships between them. This enables us to ask advanced questions of our data by placing the data into its semantic context and allowing us to model the ambiguity inherent with real-world use cases.
Q: What separates knowledge graphs from data lakes or data warehouses?
A: Knowledge graphs are more of a data structure than a storage technology. Data lakes and warehouses focus on the co-location of data to expedite analysis, they do not inherently connect these sources together. This co-location can be a useful — though not essential — first step towards a knowledge graph, but it does little to help create context around the data. A knowledge graph, on the other hand, can connect data between multiple data lakes or warehouses and can even transcend the types of data that would present in most data warehouses (like unstructured content or application data).
Additionally, data lakes either impose extremely strict schema requirements (in MDM) or very few at all — the former makes it difficult to adapt the variability in different subject domains while the latter places the burden on the end user to comprehend the data at the time of use. Knowledge graphs allow for an adaptable schema and for the embedding of the key context within the data model itself, which makes the data easier to query and use.
For Enigma, which operates in domains where there are few linking keys or unique identifiers, knowledge graphs allow us to capture non-deterministic links between objects that traditional models can not.
Q: Tell us more about Enigma’s knowledge graph capabilities. How are you using public data sources to build a knowledge graph?
A: Enigma began by collecting thousands of disparate sources of public data. In doing so, we quickly realized that the problem of using these sources in the enterprise was not just one of access, but also of unification. Most of our users began asking for all of our data on a particular company, place or person — in order to provide this we needed to think differently, which is where knowledge graphs come in.
Using both automated and manual processes, we map each of our sources of data to an ontological model (or sometimes multiple models). This identifies the key entities within the data sources and associates the desired attributes to them (for example, a name or age for a person or street address for a location). We can then iteratively compare all like-entities (people to people, companies to companies, etc.) and notice which ones are the same or perhaps related across all our different sources — this process is a version of entity resolution.
At the end of this process, we have produced an interconnected graph of the people, places, and companies in public data, which we can then expose to our users. Users help us notice where we made the correct or incorrect connections, so the model is continuously improving.
Once we have connected all the external data sources together, the final step of connecting it with internal/proprietary data becomes much easier by using a very similar process to the one I outlined above.
Q: What types of businesses do you think can benefit the most from utilizing knowledge graphs?
A: I think most any company can find value in levering knowledge graphs, but in particular, a few things come to mind:
- Companies that are trying to better understand what can be known about their customers, suppliers or counterparties across both internal and external data sources. Think businesses like credit lenders, insurance companies, direct marketing groups, and compliance teams.
- Companies looking to keep track of complex relationships in their business and track how one event may impact or be connected to another. Specifically fraud, or supply chain.
- Companies that have not yet seen the lift they wanted from their data lake efforts.
Q: What are the best applications of knowledge graphs in the enterprise?
A: A few examples from our own experiences include:
- Delivering hyper-personalized, dynamic marketing based on a rich view of customers or marketing segments enabled by connecting heterogeneous data sources.
- Connecting discrete customer information to conduct smarter compliance processes in financial services.
- Linking diverse data on drug and physician experiences to better inform fraud or drug safety processes.
- Connecting and reconciling data on small and medium-sized businesses to streamline customer lifecycle in insurance and credit underwriting and broaden revenue opportunities.
Q: Why are knowledge graphs crucial to enterprise decision-making?
A: In most cases, decisions are made with the data that’s accessible, rather than all the relevant information, which may be hard to collect. Knowledge graphs make data more accessible and thus enable better-informed decisions.
Knowledge graphs also enable an ever-increasing amount of data to be on-boarded at a greater velocity to continuously inform decisions with more — and more accurate — data.
Lastly, knowledge graphs present data in a way that more closely resembles human decision making. Humans think in terms of entities — people, places, and companies — and the relationships between them, not in terms of tables of data.
Q: Gartner included knowledge graphs in the 2018 Hype Cycle for Emerging Technologies. Where do you think this technology is headed in the future?
A: While certainly not the first to use the term, Google has popularized “Knowledge Graph” in association with its attempt to make search more entity or “things”-centric. This product is closely associated with “semantic web” efforts, which attempted to create a more structured version of the web. Considering the influence the Google search experience has had on the expectations of users of even non-google search products, I expect there will be a lot of interest in creating knowledge graph search products for consumers and enterprise. However, I don’t think this will be the most exciting application of this technology.
At Enigma, we believe that the technology will gain traction not just from data on the web, but also from real-world sources like supply chain sensors, payment transactions, and public records. Most knowledge graph products focus on unified entity views (think profile pages for entities), not on the relations between these entities (think Facebook or LinkedIn connections explorers). I believe that considering graph storage and graph processing technology is finally reaching a scale where larger sets of data can be integrated, we will see even more AI/ML algorithms running on top of these structures, which will help deliver much deeper insights or “reasoning” than knowledge bases did before. Some of the most significant, but also slightly unnerving, examples include “inferring relationships” — essentially determining there is a high likelihood things are connected to each other even when there is no data that certainly proves it.
Q: Enigma recently raised $95 million in funding. What is in store for Enigma? Are there any new offerings or developments that customers can look forward to?
2018 was a big year for Enigma, and we’re confident that 2019 will be even bigger. As you mentioned, we recently announced $95m in additional funding, which will allow us to:
- Expand our core offering by bringing to market our knowledge graphs and expanding the number of integrated solutions powered by our knowledge graph platform. Additionally, we’ll be doubling down on the value of public data by scaling our data acquisition to continue building on the world’s broadest collection of public data.
- Invest in product R&D and building innovative, industry-leading data delivery, and integration capabilities. The next year will also see a focus on continuing to build out our machine learning capabilities to break the model that good data has to be human curated data.
About Enigma Technologies
Enigma Technologies is a premier Data-as-a-Service company headquartered in New York. Established in 2011, Enigma empowers businesses and people to interpret and improve the world around them by transforming how data is seen and used across their enterprise. Enigma specializes in data acquisition, data management, knowledge graphs, machine learning, cloud platform, enterprise data, and more.