Data quality: the bottleneck holding your AI project back

80% of an AI project is spent preparing data. If your data is incomplete, inconsistent or out of date, no machine learning model will produce reliable results. We explain the six quality criteria, how to diagnose your situation and how to build a data quality programme that works.

There is a phrase every data professional knows: garbage in, garbage out. If you feed an artificial intelligence model dirty data, the model will produce dirty results. With great confidence, attractive charts and the appearance of technical precision, but dirty all the same.

The problem is that most companies underestimate the true state of their data. A Gartner study estimates that the average cost of poor data quality for an organisation is $12.9 million a year. And this is not only a problem for large corporations: an ecommerce business with incomplete product pages, out-of-date prices and duplicate customer data has the same problem at its own scale.

The worst part is that the problem remains invisible until you try to do something intelligent with your data. An ERP works reasonably well with mediocre data because people manually compensate for errors. But a machine learning model does not compensate: it learns the errors, amplifies them and reproduces them at scale.

In this article, we explain what “data quality” really means in the context of AI, the six criteria you should assess, how to diagnose your current situation and how to build a data quality programme that works without needing a team of 20 data engineers.

The six data quality criteria for AI

Data quality is not an abstract concept. It is measured using specific criteria. These are the six fundamental criteria that directly affect the performance of an AI model:

1. Completeness

Are all the necessary fields completed? A customer dataset where 30% of records have no email, 15% have no postcode and 40% have no last-purchase date is an incomplete dataset that will severely limit any segmentation or prediction model.

Traffic light: Green: >95% completeness in critical fields. Amber: 80–95%. Red: <80%.

2. Accuracy

Do the data reflect reality? A price of €0.01 on a product page, a birth date of 1900 in a customer profile or a shipping address that does not exist. Inaccurate data do not only distort the model: they can lead to incorrect operational decisions.

3. Consistency

Do the same data mean the same thing in every system? If your CRM stores countries using ISO codes (ES, FR, DE), your ecommerce platform uses full names (Spain, France, Germany) and your ERP uses numeric codes (034, 033, 049), you have a consistency problem that will make it impossible to cross-reference data without a transformation layer.

4. Timeliness

Is the data up to date? A demand-forecasting model trained on data from three years ago that does not include the post-COVID impact will produce outdated forecasts. The update frequency must be consistent with the speed at which the business changes.

5. Uniqueness

Are there duplicate records? A customer who appears three times with variations of their name (Juan Pérez, J. Pérez, Juan Pérez López) distorts purchase-frequency, lifetime-value and segmentation analyses. In SME databases, typical duplicate rates are between 5% and 15%.

6. Validity

Does the data comply with the expected format and domain rules? An email field without an @, a five-digit telephone number or an alphanumeric postcode where it should be numeric. Validity can be checked automatically using business rules.

Without quality data, no AI will work. Audit before you model.

How to diagnose your data health in one week

You do not need a six-month project to find out whether your data is ready for AI. You can perform a quick diagnosis in one week by following these steps:

1. Days 1–2: Identify the critical sources. Which databases will feed your AI project? Usually: CRM (customers), ERP (transactions), ecommerce platform (products, orders) and marketing tools (campaigns, leads). Make a list.
2. Days 2–3: Extract representative samples. You do not need to analyse millions of records. A sample of 1,000–5,000 records from each source is enough for a diagnosis. Export them to CSV and load them into a spreadsheet or profiling tool.
3. Days 3–4: Apply the six criteria. For each source, measure: the percentage of empty fields (completeness), the percentage of out-of-range values (validity), the number of duplicates detected (uniqueness), and the date of the oldest and most recent record (timeliness). Free tools such as OpenRefine or pandas (Python) let you do this in hours.
4. Days 4–5: Classify using a traffic light. For each source and each criterion, assign green, amber or red. The result is a data-health matrix that tells you exactly where the problems are and which ones are critical.

Recommended tools: For SMEs without a data team: OpenRefine (free and excellent for cleaning and deduplication), Google Sheets plus validation formulas, or a basic Python script using pandas. For medium-sized companies: Great Expectations (open source) or Talend Data Quality.

The real cost of bad data: concrete examples

To put the impact into perspective, here are real examples of how poor data quality affects specific AI projects:

• Chatbot with incomplete product data: A RAG chatbot that answers product questions needs complete, up-to-date product pages. If 20% of your products do not have a detailed description, the chatbot will answer “I do not have information about that product” in one out of every five queries. The first-contact resolution rate (FCR) falls by 20% and CSAT plummets.
• Recommendation engine with inconsistent categories: If your products are categorised differently on your website, in your ERP and on the marketplace, the recommendation engine cannot identify cross-purchase patterns. The AOV that should rise by 15% barely moves by 3%.
• Demand forecasting with contaminated historical data: If you do not distinguish between real sales and returns, or if stockout periods are not marked, the model learns false patterns. The MAPE that should be 15–20% remains at 35–40%, worse than a simple moving average.
• Recruitment with biased historical data: If your hiring dataset contains 10 years of decisions in which 90% of people selected for technical roles were men, the model will encode that bias. The four-fifths rule will fail in the first audit.

In all these cases, the solution is not a more powerful model. It is to improve the data before modelling. And that starts with an honest diagnosis of the current situation.

The five most common data quality mistakes in AI

1. Clean once and forget about it. Data quality is not a one-off project. It is an ongoing process. If you clean your customer database today but do not establish validation rules for incoming data, you will be in the same situation in six months. Cleaning without prevention is an endless cycle.
2. Do not involve the business. Only the business team knows what “correct data” means in its context. A data engineer can detect that a field has 20% null values, but only the sales team knows whether those nulls are a real problem or an expected situation.
3. Underestimate the cost of preparation. 80% of the time in a typical AI project is spent preparing data. If you budget only for the model and the tool but ignore data cleaning, transformation and validation, your project will be delayed and cost more than expected.
4. Do not define a Data Owner. Every critical data source needs a person responsible for it (a Data Owner) who is accountable for its quality. If nobody is responsible, nobody improves quality. The Data Owner does not have to clean the data personally: they have to ensure that standards are met.
5. Ignore bias in the data. Historical data reflect past decisions, including biased ones. If your recruitment model is trained on hiring data in which 85% of candidates were men, the model will learn to prioritise male profiles. Data quality is not only technical: it is also ethical.

A minimum viable data quality programme

You do not need a data governance department to improve the quality of your data. A minimum viable programme includes:

1. Designated Data Owners. Assign a responsible person to each critical data source. In an SME, this is usually the head of the area that generates the data (sales for the CRM, operations for the ERP and marketing for the CMS).
2. Validation rules at input. Configure automatic validation in the forms and systems where data is entered: mandatory fields, standard formats (email, telephone, postcode) and valid ranges (price >0, positive integer quantity). This is the measure with the highest ROI: preventing errors costs 10 times less than correcting them.
3. Periodic deduplication. Schedule a monthly or quarterly review of duplicates in your main databases. Tools such as OpenRefine can identify fuzzy duplicates (variations of the same name or address) in minutes.
4. Data-health dashboard. Create a simple dashboard with the key indicators for each source: percentage completeness, percentage duplicates and date of last update. Review it monthly. If an indicator crosses the amber threshold, act before it reaches red.
5. Data dictionary documentation. A simple document defining what each field means, which format it uses, which values are valid and who the Data Owner is. Without a data dictionary, each person interprets fields in their own way and inconsistency grows.

Data quality and the AI Act: a regulatory obligation

The AI Act does not only regulate AI models. It also establishes requirements for the data that feeds them. For high-risk systems, the regulation requires training datasets to be representative, free from errors (as far as possible) and appropriate for the system’s intended purpose.

This means that data quality stops being a good practice and becomes a documentable obligation. If an auditor asks you which data you used to train your recruitment model, you need to be able to demonstrate that you assessed its representativeness, detected bias and corrected quality errors.

Companies that already have a data quality programme are better positioned to comply with the AI Act. Those that do not will need to invest in data governance before they can deploy high-risk systems with confidence.

Advice: Make data quality assessment a mandatory step in Gate 0 of every AI project. If the data does not pass the traffic light (green for critical criteria), the project does not move on to the pilot. It is better to discover this before rather than after.

Data is the real fuel of AI

You can have the best model, the most advanced platform and the most talented team. If your data is mediocre, your results will be mediocre. Data quality is not sexy, it does not generate headlines and it is rarely included in strategy presentations. But it is the variable that correlates most strongly with the success of an AI project.

The good news is that you do not need perfect data to get started. You need data that is good enough for your specific use case, a process of continuous improvement and the discipline to measure and act on quality before every project.

If you want to audit your data quality, define a minimum viable data quality programme or prepare your data for an AI project, Impulsa3 can audit your data quality and support you with the preparation.

impulsa3.com · Digital Transformation and AI for SMEs and ecommerce · servicios@impulsa3.com

This data is also the foundation of GDPR and artificial intelligence, generative AI and predictive AI, and AI agents.