Data visualization and querying tools are all the rage, as businesses rely more on data analytics to compete more effectively. Meanwhile, the ability to query data is becoming more important in light of the need to adhere to data privacy regulations and eDiscovery requests. The problem is that business’ data is no longer confined to a singular database or data warehouse. It is spread out, often being generated and stored through a variety of means across the world, and it is being put to work by a variety of tools. New data streams and querying tools pop up frequently and unexpectedly, as do new data-driven business requirements. Getting a handle on this data is a laborious process that can take months – a luxury that modern enterprises cannot afford.
Introducing Promethium
Promethium was founded with the mission of helping enterprises to quickly get to the right data for a particular business need. According to Promethium, its Data Navigation System can cut the time to locate, ingest and prepare data for analysis from months to minutes, vastly reducing the time it takes to complete queries. This frees up valuable data scientists’ and engineers’ time so they can support a larger number of queries and enables them to focus on higher-value tasks. Less time is spent locating and making sense of files, and more time can be spent gaining insights. This also enables business analysts to service more of their own requests by reducing the time and complexity to obtain the relevant data they need to answer desired questions.
Promethium calls its Data Navigation System a “data context solution” for data analytics and governance. The platform uses an application programming interface (API) to connect natively to all of the organization’s data sources, or the user may upload data directly into the Data Navigation System. From there, the platform aggregates metadata that enables data to be synced to the Data Navigation System without the need to move the data. Data sources and visualization tools do not need to be changed, which helps when it comes to adhering to data privacy regulations and to avoiding the need for costly and complex data migration.
Business users input their query, in the form of a natural language question, into the Data Navigation System. For example, a user might ask for an aggregation of all files that contain personally identifiable information (PII). The Data Navigation System then automatically identifies data that is relevant to the question, from across the total available pool of information. It provides a “Match Score” to the query, based on the data’s quality (including its relevance to the query at hand), its completeness, and whether or not there is a sufficient volume of data available to address the query. Users can search via a variety of criteria, including data sources, file name and key terms. The Data Navigation System is intelligent enough to understand if the file in question can answer the query even if the specific term is not in the file. Queries may be pre-generated or they may be prescribed by the user.
The Data Navigation System can understand the relationship between data (table to table, table to file, and file to file, both within and across all data sources), and provide a diagram or “data map”, to help the user to visually identify significant relationships – typically a task that a database administrator or a data engineer might need to set up. This mapping is dynamically updated as data sources are added or removed. It is also automatically updated as metadata changes occur in any of the data sources. To enhance accuracy, Promethium can detect multiple instances of a table or file, and then suggest which to use and why. It also tracks data’s lineage, for example what specific changes may have been made to a file, when they were made, and by whom so that users can trust where the data comes from. This can add up to months of work that Promethium is handling automatically.
Once Match Scores are identified, the user selects the data that they want to export to support their query, or this is handled automatically. The Data Navigation System then creates a logical map of specifically what data is needed and how to assemble it. These are step-by-step, Google-like instructions for technical teams on how to assemble the data so that queries can be executed. It will even provide course correction if the user gets off track, and if data needs to be cleansed, the Data Navigation System details specifically where the data that needs to be cleansed is located – for example, down to a specific row in a table.
StorageSwiss Take
The quality of a data analytics module is only as good as the data that it is fed. The Data Navigation System offers both a comprehensive and a lightweight means through which the business user can quickly identify the most relevant data to their query. In a world in which data growth and utilization as well as requirements for data governance are not going to slow down, being able to more quickly lay a stronger data foundation for business intelligence and data governance (including eDiscovery requests and compliance requirements) is quickly becoming crucial. Users need to be able to simply run queries and validate results, without a weeks or months long process of harvesting and cleansing data.
Promethium can add up various attributes of files across data sources to provide better quality analytics, in a way that a piecemeal approach of using specific vendors or tools for specific components of the data analytics process cannot. It also can help users to better understand what questions their data can answer, to unlock new business value.
Senior Analyst, Krista Macomber produces analyst commentary and contributes to a range of client deliverables including white papers, webinars and videos for Storage Switzerland. She has a decade of experience covering all things storage, data center and cloud infrastructure, including: technology and vendor portfolio developments; customer buying behavior trends; and vendor ecosystems, go-to-market positioning, and business models. Her previous experience includes leading the IT infrastructure practice of analyst firm Technology Business Research, and leading market intelligence initiatives for media company TechTarget.
StorageSwiss.com - The Home of Storage Switzerland 