The challenge

In practice, an experienced worker can quickly recognise the type of monitor, and separate it correctly. This is thanks to subtle differences such as monitor thickness or colour. As a result, these workers can recognise some monitor types from a distance, even without having seen a label.

However, this process requires the training of several employees who, in many cases, are only employed by a waste processor for a short time. Therefore, the current process is time-consuming, and error-prone with inexperienced workers. And so there was a need for a digital solution in the form of a mercury scanner.

To make the use of the application as effective and user-friendly as possible, it was necessary to come up with a tailor-made solution.

Using AI to recognise a label

All monitors contain mandatory labels on the back that contain various characteristics to identify mercury. These features are expressed as logos or codes. We deployed 'optical character recognition' (OCR) to train our model to recognise specific words or letter combinations. In addition to OCR, we also developed a model that uses object recognition to identify various icons. By combining these two techniques, the app can recognise the type of screen the monitor contains from the label, and then display this information to the user.

This information then allows the user to correctly place the monitor in the correct waste stream.

A fast scanner

To minimise the waiting time between scanning and displaying a result, we carefully considered the choice of appropriate software and hardware, and how we would train the machine learning model. In some cases, this process took several seconds longer, leading to additional waiting time for the worker. This is not desirable, given that up to hundreds of monitors are processed daily per waste processor.

User-friendly interface that matches the working environment

When designing the interface, our goal was to minimise the number of actions users have to perform to run a scan. This was important to ensure that using the application takes little time and does not interfere with the employee's work. We created an interface with only two pages, which can be operated with a single physical button to activate the scanner. After scanning, the result is displayed; this page closes automatically after a set time, allowing the user to start a new scan immediately.

Users

When developing our software, we always consider the end user. For this application, there were several interesting requirements that we had to pay specific attention to.

Navigation and gloves

As waste handling takes place in an environment with many sharp objects, workers wear thick gloves. These gloves hamper the use of digital buttons on the screen, as the finger is not recognised due to the thickness of the glove. To circumvent this problem, we considered two possible solutions.

The first solution involved a buttonless scanner, where the photo camera continuously tries to recognise labels without user action. We did not choose this option because the technique consumes too much data.

The second solution, which was our preferred one, uses the physical (volume) buttons on the side of the smartphone. These buttons can be pressed by workers wearing gloves, though, which means we do not have to resort to data-intensive techniques.

By activating the scanner via the volume buttons, users can operate the application even with gloves on, without having to take off their gloves for each scan.

Command of the Dutch language

Employees often come from different backgrounds; for example, there are immigrants who do not speak the Dutch language. To take this into account, it was essential for the application to be understandable without textual support, and to use recognisable words such as 'LED' and 'LCD' as much as possible.

Working visit

To gain these insights, we visited several waste processors to observe how workers currently do their work and assess how the application can complement their working methods. During the visit, we paid extra attention to different working conditions, such as: who is working, is the use of gloves mandatory, can we use audio or is there a lot of loud noise, is there enough light available to perform a scan and how likely is the smartphone to be damaged on the shop floor.