With the help of artificial intelligence, state-of-the-art space probes ought to be able to identify the best place to land on the moon, but at present the technology often fails to perform. Especially the moon’s rugged surface causes problems. The aim now is to train the computers with many gigabytes of image data. Here, gamers are playing a key role: they are charting lunar craters “by the by” in a riveting game – and in this way feeding the AI with essential information.

By Kilian Kirchgeßner, 26.11.2023

When Stephan Jacob opened the email that was ultimately to make him a kind of astronaut, two lunar missions had just failed miserably. The Russian and Japanese unmanned space probes missed landing on the moon by just a few metres. Artificial intelligence was supposed to navigate them to touchdown, but in the very last moments the technology failed and the probes, which cost millions and millions of euros, crashed shortly before reaching their destination. Together with NASA, the European Space Agency (ESA) will now spend the next few years planning its own moon base, and the aim is for several automated landing modules to deliver the materials – a far larger project than the lost space probes. Stephan Jacob and his students are helping to accomplish this.

“The problem with the moon is its craters,” says Jacob, Professor for Creative Technologies in Digital Art at h_da. “The entire surface is full of them, and they are very close together. The smallest ones are only a few centimetres in diameter and were created by impacts from small meteorites.” The modules supposed to land there usually have four legs for touchdown – and if just one of these legs lands in a crater and has no stable hold as a result, the whole module tips onto its side and is lost. Although the artificial intelligence that should recognise the craters automatically already exists, it is simply not accurate enough. The probability of it spotting a crater is around 80 percent – “but I wouldn’t board an airplane that only lands safely 80 percent of the time,” says Stephan Jacob.

When the people from ESA contacted him, a whole volcano of ideas erupted in his head. He specialises in gamification, the art of transforming complex contexts into riveting computer games. They asked him in their email whether he could think of a way to chart all the craters on real images of the moon’s surface so that this information could then be used to train the artificial intelligence. “Humans,” says Stephan Jacob, “are namely incredibly good at spotting the craters. We are able to distinguish between light and dark, we can recognise shapes, and we are capable of doing all this extremely well. For the computer to learn this too, however, we need to pre-sort the data for it.” This involves 80 million datasets from the moon’s surface. Charting the craters on it manually would be a punishing task.

Unless you get gamers to do the work “by the by” in a riveting game, that is! Stephan Jacob is the right person for the job. “In fact, I’ve always had a predilection for space travel,” he says with a grin: “I even thought about studying astrophysics before I chose computer science.” Now, his computer science background is sending him back to the stars – and he can pursue an old passion that won’t let him go: prior to being appointed as professor, he was a start-up entrepreneur. Together with two fellow students, he set up a company that designs landscapes on the computer. There is a crazy story behind this: while they were at university, they in fact wanted to programme a game where a planet explodes. The aim was for the gamer to fly seamlessly from space to Earth during this spectacle, the graphics would be breathtaking. The problem? It wasn’t technically possible. He and his friends therefore set to work on new innovations. Their company became a kind of “stage designer” for other game programmers, who can use the World Creator software – nomen est omen – to model exactly the right surfaces for their applications, such as gently rolling hills, rugged cliffs – or craters.

Because Stephan Jacobs knows the gaming scene down to the smallest detail, the many results from scientific studies immediately sprang to mind when he received ESA’s enquiry. The fact, for example, that “serious games” are scorched earth and have a bad reputation among gamers. Indeed, ESA had previously launched an attempt to get volunteers to chart the moon’s craters in an app. “At the peak, there were 9,000 players, but they soon threw in the towel when they realised how much work it was,” says Stephan Jacob. “75 percent charted less than ten craters because gamers in the first instance just want to have fun.” In the gaming industry, this is mostly achieved via action: Stephan Jacob estimates that 90 percent of games contain competitive elements aimed at increasing excitement. However, ESA ruled that out for charting craters right from the outset, meaning that Stephan Jacob was not allowed to programme a new Star Wars. “We then went for settlement building,” he says: “Not all gamers are fans, but it does appeal to an enthusiastic part of the community.”

This is where Professor Jacob brought his students on board: as part of a compulsory elective seminar, a group of 15 space enthusiasts came together to ponder how to build a computer game around the craters. In technical jargon, this is called “game mechanics”, i.e. what do players have to do, what is the setting, which tasks must they perform, which rewards beckon? Everything that theorists know about computer games has to be incorporated. The awareness, for example, that to maintain players’ attention they need to make progress. And the realisation that most games are meanwhile played on mobile phones. Ultimately, three of the ideas that effervesced from the seminar seemed viable. These are now being fused together to form one big game idea for smartphones, which the team is currently programming over months of painstaking work.

The concept? The players slip into the role of astronauts whose goal is to hover over the moon and safely touch down in a landing module. To do this, they must spot and chart the tricky craters on the moon’s surface, which is composed of real images. As soon as they have landed, they start to colonise it by looking for a suitable place for a moon base. The ideal place is in a large crater, where the future settlement is sheltered and largely protected. To identify the spot, the players chart further craters. Then things start to get more and more exciting: ion storms are brewing, meteorites are hailing down on the moon, and the gamers must make sure that they and their settlement are out of harm’s way. Fortunately, they can earn additional features for their rocket as a reward for charting as many craters as they can: protective shields, for example, or a more powerful engine. If they need supplies for their growing settlement, they must land transporters with new solar modules or other components between all the craters – and to ensure that they land safely, they chart further craters. And the better the players perform, the more areas of the moon’s surface are unlocked for them so that their settlement can grow bigger and bigger.