Research for more fairness in biometrics

Currently, a room at h_da’s Faculty of Computer Science is being converted into a photo studio on a regular basis. Here, researchers take portrait photos for a database that is unique worldwide. The purpose? To assess the fairness of biometric facial recognition algorithms. The aim, on the one hand, is to produce photographs that exhibit a consistently high quality. On the other hand, the database should pool as wide a variety of skin tones as possible. The team is looking for further test subjects.

By Alexandra Welsch, 22.6.2026

Please sit up straight, look directly ahead, don’t smile, keep a neutral expression. A young man sits on a chair without a backrest in front of a dazzling white screen and looks into a camera in front of him, with flash umbrellas to his left and right. The photographer takes six photos one after the other, each with a different exposure and accompanied by a brief flash. Then another camera comes into play and the procedure is repeated. Please sit up straight, keep a neutral expression, five photos, five different exposure levels.

At regular intervals over the last year, researchers at h_da’s Faculty of Computer Science have converted a group room into a photo studio, where they are taking pictures of people with different skin tones to create a database that is unique worldwide. This aim is to use this database to make the assessment of biometric images by means of algorithms – in the context of facial recognition – fairer. This “Darmstadt Skin Tone” (DAST) database is part of the “Next Generation Biometric Systems” research project led by Christoph Busch, Professor of Computer Science and biometrics expert at h_da, in the frame of the National Research Centre for Applied Cybersecurity (ATHENE).

The background to this is a problem with existing facial recognition systems, such as ones used for travel purposes or when crossing borders. Their effectiveness greatly depends on the quality of the passport photos analysed using biometric software. The result is that international standards –regarding sharpness or exposure, for example – are not always complied with. “Whether or not passport photos meet the requirements has been left until now to the discretion of staff at registration offices, on the basis of a visual inspection,” explains Christoph Busch. “That is not an objective assessment.”

That is why a study published by the European Commission in 2019 identified the need for a computer-assisted, objective assessment of facial image quality. As part of the project “Open Source Face Image Quality” (OFIQ), scientists have developed software that is freely available online; it automates and assesses the quality of facial images for biometric applications – and aims to show distortions and reduce them in the long term. The tool was developed at the initiative of Germany’s Federal Office for Information Security (BSI), and Professor Busch was also involved. h_da contributed to OFIQ as one of seven international research groups.

Back in the h_da photo studio. “What we are endeavouring to do is facilitate objective image assessment,” explains Fabian Stockhardt, a researcher working at the Faculty of Computer Science on the DAST database. That is why all photographs are taken under the same conditions. For example, strips of adhesive tape on the floor ensure that all the equipment is always positioned at the same distances and in the same direction. In addition, six photos are taken in each case because, in addition to the best exposure level, pictures of each person are also always taken with strong or very strong underexposure and overexposure. Why? Because this can also affect automated facial recognition and lead to distortions at the assessment stage – particularly when different skin types are involved.

It is here that another key problem associated with the use of facial recognition software lies: “Existing data is extremely biased,” points out Stockhardt. The OFIQ software’s algorithm systematically assesses the quality of photos of people with darker skin tones differently from those with lighter skin tones and assigns them a lower quality score, as Professor Busch explains. This is due above all to the primary data: the reference photos were mainly sourced from the internet, he says, and consist disproportionately of celebrities, such as actors or politicians, who are mostly white and of a certain age. Furthermore, contrasts appear differently on darker skin than on lighter skin, which can also lead to distortions when assessing image quality. The consequence of such racial bias in biometrics can be that automated facial recognition systems incorrectly reject or misidentify people more often, for example at airports.

“Our data tries to compensate for this,” Fabian Stockhardt goes on to explain. With the help of the database being compiled in the frame of their photography project, they want to find out how different skin types affect the performance of facial recognition systems. This calls for as broad a spectrum of skin tones as possible. When test subjects come to their photo studio, the first thing the researchers do is determine their skin tone. For this they use a colorimeter – a dermatological measuring device – to measure light reflection, and thus the colour, of the skin on the person’s forehead, cheekbones and the backs of their hands, and to classify it on a ten-point scale.

To date, about 160 volunteers have had their photos taken for the project. However, the researchers are looking for even more. The target is around 30 participants per skin type category. So far, it has only been met for the lighter skin types 2 and 3; the researchers still especially need people with very dark or very light skin. To attract more people for their pool of test subjects, the team is now promoting the project beyond the university, too. For example, they have distributed flyers in English on Luisenplatz, the main square in Darmstadt city centre, with the invitation: “Do your bit for fairer biometrics!” Apart from the “good deed” of contributing to a more objective assessment of facial images, participants are also tempted with €30 for their trouble.

“We now have a third of the data we want,” reports Professor Busch. But work is nevertheless already underway with the dataset available so far. A Master’s student has just dealt with it in his thesis. He fed the existing skin type data into various machine learning models and then used additional data to test how accurately the algorithms of these digital computing tools can determine skin types. The preliminary result: “The tool assessed the correct skin type class 98% of the time,” Professor Busch, his supervisor, is pleased to say, “which is fantastic, of course.”

At the end of the day, the team is primarily interested in one thing: “We want to minimise the impact of racial bias,” says Christoph Busch. “And we want all demographic groups to be treated equally fairly.” In his view, biometrics offers the fundamental advantage of objectively assessing a person on the basis of their individual physical characteristics and can therefore also help prevent misuse.

“I’m also a big fan of biometrics,” says computer scientist Fabian Stockhardt in the photo studio. Automated facial recognition relieves humans of unnecessary work and can compare a photo much more quickly and objectively. And what about facial recognition’s potential for discriminating against people? “You can use any technology for negative purposes,” says Stockhardt. But their research project is clearly aimed in the opposite direction: “The main purpose of our database is to ensure fairness.”