Susan Walsh
We all work with data but we’re not all data people. We must recognize that everybody who interacts with data plays an important part in cleaning and maintaining it so that it is reliable and can be fully exploited by all stakeholders in a business. Of course, we’re not all professionals in that area and it can be quite intimidating to some people.
The Very Real Problems Caused By Data Errors
One common problem is not getting documents to match up because the dates are in different formats (UK vs USA, for example), causing someone to spend ages manually reconciling the different formats.
I see people waste hours or even days on something basic that could have easily been avoided. When dealing with clients, I might see specifications labeled with incorrect measurements like centimeters instead of meters, and a whole host of other errors that are easy to make but can cause the wrong things to be ordered or in the wrong quantity or in the wrong size which can cause huge disruption to the business.
Rogue zeroes are a classic case, “I only meant to order ten, not a thousand!”
Data errors can mean excess stock in your warehouse or fines from your customer for not delivering on time. All kinds of problems can result from one small little data error. Rogue zeroes are a classic case, “I only meant to order ten, not a thousand!”
Let’s say we sell sticky notes online and in stores and we have 3 suppliers. Supplier A calls them ‘Post-It notes’, Supplier B calls them ‘Sticky Notes’ and Supplier C calls them ‘notepads’. They’re all the same thing so we need to categorize them as the same thing if we want an accurate picture of how much you’re selling, buying, or forecasting.
When you have bad data or missing data, your forecasts will be compromised.
Now we see this problem especially with forecasting. When you have bad data or missing data, your forecasts will be compromised. I’ve seen seven ways to format ‘United States’. If you’re forecasting products sold within the US, you need all the data points labeled in the same way which means getting everybody to agree a set standard about how to input data.
Maintaining Data Discipline
You can facilitate this by controlling what people can put into certain columns; some might be mandatory or maybe sometimes you can do drop-down lists (although those have problems because people are lazy and naturally we’ll just pick the first thing). Expectation setting and getting people to understand the importance of data classification is key – letting your colleagues know that just by putting a little bit more information in a spreadsheet means somebody else isn’t to waste two hours doing unnecessary work down the line.
Check Your Data Regularly
I’m a real strong believer in data maintenance because you cannot keep your data clean if you don’t maintain it. You have to check it regularly and make sure that it’s still the way it’s supposed to be because people can delete things and people can cut and paste over things.
The most important thing is to look at your data on a daily or weekly basis because you’ll know if it doesn’t look right. For example, if somebody accidentally inputs a thousand units instead of a hundred, and you know that every week you’re ordering 100 units, you’re going to spot that difference and be able to fix it.
Driving Consistency In Data Management
I’ve categorized retail data, I’ve categorized food data, procurement data – I’ve categorized everything. The one thing that is true across all datasets is the importance of maintain standards and consistency. I came up with something to help clients remember that which is making sure your data has its “C.O.A.T” on. Your data should always be:
Consistent: Your data should always be consistent so everybody’s using the same terminology, the same units of measure, the same formats, and the same processes for data input.
Organized: Categorize data in such a way that if you need it you can pull out that information really quickly. If you need to look at it by country or division or region or by buyer or by department, categorize it like that and then you can pull off a report quickly. How many people within companies trying to cut and paste different spreadsheets together to get what they want when all you need to do is a quick VLOOKUP?
Accurate: Make sure it’s as accurate as possible. I would never claim that you could get your data 100% accurate – if you do it’s not going to stay like that for very long because too many people involved. But striving for 100% accuracy means it’ll be accurate enough to have utility in the business.
Trustworthy: This is where the magic happens. Trustworthy data means you know you can go to your senior decision makers and say these are the right numbers – this is exactly what we’re doing, this is what we’re buying, this is what we’re selling, this is what we’re forecasting etc. Data facilitates decisions, after all, and we need to have faith in the numbers.
Data Classification Is Key To Business Efficiency
All too often I see businesses taking weeks to prepare reports for end of month and in every case it’s easy to streamline that process and get that reporting process done in a few days rather than weeks. Rather than spending hours resolving queries, we can create lookups and formulas in Excel and get the data we need so much quicker.
It’s only when you fix these data issues that you realize how long these things were taking – so often people just sit in a time vacuum when you’re working through these things and nobody raises an eyebrow because in the absence of a better system, it just has to get done. I’ve seen that many, many times.
Cleaning your dirty data means greater productivity, whatever your role or functional area
And that’s the value driver of proper data management – speeding up processes to make your business more efficient. It means your people can work on more value-added activities instead of manual, repetitive tasks. Cleaning your dirty data means greater productivity, whatever your role or functional area. – Send comments to the Editor at andrews@ibf.org