A very simple answer to that question if everyone. From the small shop on the corner, to the medium sized Accounting firm, to the large corporate. Everyone needs to worry about their data!
Data quality is not something most companies worry about much, since they have what they think is manageable data, so a person can maintain it, and it is their job to ensure it is accurate.
As that company grows, they keep in believing that it is being maintained and the person they have is doing their job. Before they know it, the data volumes have increase and that person it taking strain ensuring that the data is maintained on a daily basis. This could be that one large customer that you bring on board as you grow your business, that one that is going to make you grow, the one we are all looking for. Your excited about the growth, and the last thing on your mind is that one person capturing the data and ensuring it is accurate. You soon find yourself adding more and more staff to the data capturing team to ensure that the data is updated and maintained. By the time you stop and look around, you have a team of people being paid to ensure that the main thing making your business possible is safe, and correct.
Unfortunately, there will be a time where people cannot maintain data any more, it needs to be done by a system to double check what has been input is correct. That one safeguard to remove the risk from your business to ensure you have one less thing to worry about. The truth is simply that smaller companies, and your SME, are staring to gather and store data at an incredible rate, and now I am not even talking about the corporate giants that generate and consume data in a scale we sometimes cannot imagine. The business world is becoming driven by data, if we like to hear it or not. And if you do not need a data quality solution now, you will need it very soon.
Maybe the data was late? How do you currently manage your data suppliers, do you know how many times they send bad data, how many times the data arrives later than expected? Can you report historically how many times a specific data provider did not follow the data policy you supplied? Can you report to see what data type is always an issue for most partners? The solution needs to be able to answer all these questions.What more should it do?
Taking it a bit further, the solution needs to assist you in managing your data providers, if bad data is received it needs to inform them of the issue and allow them to repair the data and resubmit. The solution needs to gracefully handle this resubmitted data and provide the corrected data to your internal systems.
Lastly the solution needs to store all this information, making it possible to report on historical data changes, and even load it to a Staging area for your Enterprise data warehouse or other internal systems. A good data quality solution will allow the business to use the data that is stored for many more applications than they previously thought possible, since it will form a holistic view of all data entering into the business over all time.
Well we all know that small companies are fighting the cash-flow problem every day, they do not have the money to spend on buying software that manages this for them. As for SMEs they have strict budget constraints and will not get approval for a new investment for data quality that management thinks can be done by a data capture, or some expensive Dev-Ops resources.
With Multi-National corporations they will likely invest in a large-scale data quality program, to ensure they are on track, and based on estimates it looks like big numbers. It is estimated that the industry will spent $14.7 million per year on correcting data quality issues, and that the average corporation has 18 full time employees to just focus on Data quality.
So, the question needs to be asked, how can I get Data Quality that is right for my business, be it small, medium or large. You have three options available to you, buy a on premise Data Quality solution, develop a in house solution with your own Development teams to achieve data quality and lastly get a Data Quality as a Service solution.On Premise Data Quality Solution
With a on premise Data Quality solution you will need to invest money in the following:
If you were thinking of going the route of implementing an in-house developed Data Quality solution the sky is the limit, you can spend as much as you like, or rather have available the following needs to be considered in such an approach:
If you are a smaller company and do not have the development resources or the skills to implement it yourself, or you do not have the budgets required for buying a new product, you are left with Data Quality as a Service.Data Quality as a Service
With Data Quality as a Service you can benefit in the following way:
As a business owner, CTO, CEO or CDO you need to decide what best fits your requirements, based on the insight I have provided the choice should now be easier to make. There are multiple solutions available in the market, and many consultancy firms that can assist in making sure you reduce your data quality risk. Keep in mind the new GDPR requirements, as good data quality and data governance processes are key and need to be reported on. It is estimated by 2020 that more than 80% of organisations will have a newly appointed CDO (Chief Data Officer) and they will be solely responsible for the data governance of the organisation. We as business owners need to understand that the ownership of data is becoming the biggest focus of the international market and will keep on increasing over the next 5 years, we need to be ready and stay ahead of the requirements, once we fall behind it is extremely difficult to play catch up.