Annalect’s Director of Data Services, Scott Slavens, talks about data quality issues and solutions
Question: How does Annalect define data quality? What are some of the biggest issues with data quality?
Scott Slavens: At Annalect, we see data quality as not so much being an issue of right versus wrong, but instead as a measurement of confidence in our data. We use the lens of data quality dimensions to look at everything we capture and store. We use this approach to measure the quality of the data products that our vendors provide, data sets and analytics produced for clients, including individual metric values.
Data quality has historically centered on Master Data, but our industry is by definition Master-less Data. With much of the analytics we produce being the result of gathering vendor-supplied information, we are at the mercy of these vendors to supply accurate data.
The biggest issues with data quality are:
- Big Data: The variety, velocity and volume of data are presenting new challenges in data quality. The past paradigms for analytics are being superseded by predictive analytics approaches. In the past, history was measured in days, weeks, months and years. Today data is moving through our platforms at such velocity that history is now measured in minutes or even seconds. How do you perform data quality analysis and remediation quickly enough so as to not stop the immediacy for data of high quality?
- Software: The data quality software marketplace has not kept up with the needs of big data. The majority of third-party data quality software platforms cannot handle the volume of data at the velocity at which it is moving through our platforms.
- Master-less Data: There is often no master data source against which to validate data. This requires our organization to approach data quality from a more holistic perspective rather than from the approaches that simply qualify data as being right or wrong.
Q: What are Annalect’s solutions for data governance? What sets them apart from the industry?
SS: Annalect’s Data Governance Office (DGO) is structured similar to other highly federated organizations. Being global requires that processes and supporting technologies that support the requirement for such a federated structure be established. Setting up a centralized framework for data standards, metadata, data quality and reference data, all supported by federated data stewardship, has allowed us to remain flexible in serving the regional needs of the organization while maintaining a clear set of guidelines for process.
As far as setting us apart, our data governance processes have entered a mature state in many disciplines, and through strong definition of key performance indicators and our approach to continual service improvement, we are positioned for the advancements in data management that will be necessitated in the near future.
Q: How has the usage of data in marketing changed over the past several years?
SS: The biggest change in the usage of data in marketing is that in the past, the industry lagged other sectors in the ability to bring together data and accurately present insights in the social and competitive spaces. Our ability to ingest and analyze social data for brand trends through Facebook, Twitter, Google, YouTube and other social sites, and then compare that to various competitive spend and placement analysis has provided for the ability to develop whole new analytics for our clients.
Q: What types of decisions are being made based on data and do you expect that to change as quality improves?
SS: As our data quality methods continually improve, we believe that our approaches to media buying for our clients will become more precise, as well our ability to bring more detailed analytics and other “decision-able” data to our clients through our products and services. We foresee our expertise in data management moving to the forefront in the year to come, with large corporations reaching out to ad industry data partners for this expertise supplanting the more traditional management consulting and large technology platform vendors.
Q: How can companies ensure they’re making the best decisions possible given that some data will be false/inaccurate?
Seek to better understand and document what data quality means to you and the decisions you make. A certain level of crudeness or uncertainty in data quality may be acceptable for making low impact decisions, while high impact decisions likely require a higher standard of precision and accuracy. What level of quality do you hold yourself to, and for what types of decisions?
Develop strong relationships with your data and analytic providers, working toward understanding the more critical risk areas for data quality. Strive to develop a data risk framework that will help you understand the potential risks involved in making decisions based on various categories of data and analytics yielded from that data.
Since analytics providers in the ad industry are often dependent upon the ingestion of key client data that will be used to drive integrations with vendor data, and subsequently data analysis in the build of key market insights, companies should assure that they themselves have implemented strong data quality processes and technologies so that they are confident in the data that they are giving to their analytics providers.
Companies should always question their ad industry partners, requiring them to provide an in-depth explanation of their processes and technologies supporting data quality.
Q: What is the importance of data certification and how should companies think about that as big data expands?
SS: The concept of data certification has gained great traction in many industries such as banking, finance, insurance, healthcare and pharmaceuticals. These industries have achieved high levels of maturity in the development of data standards and the anointing and funding of industry specific organizations tasked with developing standards for data and metadata.
The ad industry has traditionally lagged other industries in these disciplines. I believe that it will be critical for the industry to recognize this type of framework over the coming years.