Data Annotation & Its Role in Machine Learning

Data annotation plays an essential role in the world of machine learning. It is a core ingredient to the success of any AI model because the only way for an image detection AI to detect a face in a photo is if many photos already labelled as “face” exist.

If there is no annotated data, there is no machine learning model.

What is data annotation?

The core function of annotating data is to label data. Labeling data is among the first steps in any data pipeline. Plus, the act of labeling data often results in cleaner data and additional areas of opportunity.

Labeling data

It is necessary to have two things when annotating data:

1. Data
2. A consistent naming convention

As labeling projects grow more mature, the labeling conventions likely increase in complexity.

Sometimes, too, after training a model on the data, you might discover that the naming convention was not sufficient to create the kind of predictions or ML model you intended. Now you need to get back to the drawing board and redesign the tags for the dataset.

Clean data

Clean data builds more reliable ML models. To measure if the data is clean:

1. Test the data for outliers.
2. Test data for missing values or null values.
3. Ensure labels are consistent with conventions.

Annotation can help make a dataset cleaner. It can fill in gaps where there are some. When exploring the dataset, it might be possible to find bad data and data outliers. Data annotation can both:

• Salvage poorly tagged data or data with missing labels
• Create new data for the ML model to use

Automated vs human annotation

Data annotation can be costly, depending on the method.

Some data can be automatically annotated, or, at least annotated through automated means with a degree of accuracy. For example, here are simple forms of annotation:

• Googling an image of a horse and downloading the top 1,000 photos into a horse file.
• Scraping a media site for all its sports content and labeling all the articles as sports articles.

You’ve automatically collected horse and sports data, but the degree of accuracy of that data is unknown until investigated. It’s possible some horse photos downloaded are not actual photos of horses, after all.

Automation saves costs but risks accuracy. In contrast, human annotation can be much more costly, but it’s more accurate.

Data annotators can annotate data to the specificity of their collected knowledge. If it is a horse photo, the human can confirm it. If the person is an expert in horse breeds, the data can be further annotated to the specific breed of the horse. It’s even possible for the person to draw a polygon around the horse in the picture to annotate exactly which pixels are the horse.

For the sports articles, the article could be broken down to which sport, a game report, player analysis, or game predictions. If the data is tagged only as sports, the annotation has less specificity.

In the end, data is annotated to both:

• A degree of specificity
• A degree of accuracy

Which is more necessary, however, always depends on how the machine learning problem is defined.