Last year, former Feinstein Institutes neuroengineer Viktor Tóth set a simple goal: He wanted to teach mice to play the 1994 classic video game "Doom II."
No, this is not just a weird hobby. No, he didn't want to open a Twitch channel. The Hungarian software engineer and neuroscientist wanted to learn more about brain-computer interfaces, and at the same time show the potential insights that researchers can gather through similar experiments.
He wrote in a viral Medium post that his rat training setup is bootstrapped and fragmented, consisting of a large polystyrene ball that can be rolled in any direction through a ball bearing. A mouse is suspended from a safety belt on the top. It can move the ball with its feet. Sensors convert it into motion in the game world and mirror it on the curved computer monitor in front of the game rodent.
Therefore, when the mouse moves on the ball, it will also move in the famous corridor of the video game. Tóth uses a small tube to strengthen the mouse's movement. Whenever it does the "right" thing, such as walking down the corridor, it will treat it with sweetened water as a treat. All in all, Tóth said, the setup cost is less than $2,000.
This is a bold, ambitious, and downright silly scientific project, so naturally, we have to learn more. We approached Tóth and asked him what it's like to train a mouse to become a gamer — and whether he plans to open a Twitch channel.
For clarity and length, this interview has been edited.
Futurism: Where did ideas like training mice to play video games come from?
Victor Tóth: This idea was basically born out of thin air. I just thought, "Why not?"
I think it was also when Neuralink implanted brain implants into pigs to analyze nose detection. Then I thought of this idea. I see some use cases for something similar.
It is very related to brain-computer interface (BCI)-in the long run, I am trying to enter this field. Several companies in this field, such as Neuralink, BlackRock and Paradromics, are testing their devices on monkeys and then pushing them to humans. But the problem is that many of these cognitive abilities already exist in mice-so why can't we just use mice?
Part of the reason is that you can't fully train mice to play things like "ping pong," right? Research design is difficult, but once you bridge this by training mice in a virtual environment, it is huge. You can record various brain signals from mice, such as simple visual cortex information, or you can make decisions and plans more deeply.
This looks complicated and has not really been done yet-but you can do it in the same setup I used. The only thing that changes is the software, or the game the mouse is playing.
Other than that, I did it because it's cool.
Why did you choose Doom II? Why not "Duke of Destruction", "Wolfenstein" or "The Legend of Zelda"?
VT: The plan has always been to train them on the first map of Doom II. I think the first map is perfect because it has various environments. It also has a very easy start, with clear corridors and simple turns. It is also a larger space, so you can test many things there.
Doom II also has a map that is very easy to edit. I was able to delete the first corridor from the map and open the door when the mouse approached. Therefore, they do not need to press a button, nor do they need to show the act of actually opening the door.
I designed different iterations of the same map in different locations with little ghosts—that is, bad guys. For shooting training, I randomly selected one of these maps, so every time the mouse starts from scratch, the kid is in a different position. In this way, the mouse will not learn to shoot at a specific point, but the kid itself needs to be shot.
How far can the mouse go through the maze?
They cannot complete the complete maze. Romero, one of the three mice I have used, performed best. I was actually able to start training him to complete the shooting part on a separate device. The idea is that if they push their front legs up and back, then the seat belt they connect can press a button with a separate arm attached to it. That button can be used for shooting.
When Romero meets a kid, I pull my arm up and start shooting. This is the same operational conditioning logic for running and sugary water-but Romero is just confused. He was like, "What happened? Why did this thing appear in front of me? Why would I be rewarded?" At that time, he still couldn't see what was going on.
At that time, I set myself a strict deadline at the end of July this year-I just couldn't complete the training for Romero in time.
Neural engineer Viktor Tóth trained three mice to play "Doom II". This is a video in which Romero, a mouse, was taken aback by a kid during shooting training! 🐀 pic.twitter.com/whrTftl4mr
— T🎅ny H🤶 Tr🎄n (@TonyHoWasHere) December 13, 2021
I can't imagine that it would be easy to train a mouse to do anything, let alone play a first-person shooter game.
VT: At first I thought naively, "I want to raise a few mice. They need a week to adapt to me. Then they need a week to run." This is far from what actually happened.
This is my first animal experiment. This is also a huge learning process for me. So instead, it took them two weeks to get used to me. At the same time, I watched a lot of mouse training videos on YouTube, showing me how to touch them, approach them, play with them, and so on. I mean, it's fun. It just took a lot of time.
Then I started putting them on the ball-but I couldn't let them run. First of all, I must teach them to eat with the ball where the sugary water is. Then I started training them to run-but as long as they heard the linear actuator continue to run or the motor started to run, they would be afraid and jump off the ball. After a while, they got used to it.
Compared to most behavioral experiments, it did not take that long, but it still does. There are many things I haven't considered.
Are there any plans to start a Twitch live broadcast?
Yes, I think so. I think this is a very effective way to profit from such projects. The only question is how long the mouse can run.
Once I let Romero run for 15 minutes, which was great. That was crazy, because he did it for so long and didn't feel tired and didn't want to get out of the car. Therefore, if you can really make the mouse really express curiosity in the game, then it will become very interesting.
If you can do this, the mouse will really "play" for 10 to 20 minutes in a row, then yes. Twitch streaming will be a very effective way to let people see this.
Do you have plans to build this project? Maybe teach the mouse to play a new game?
VT: Yes. When I experimented and moved back to Hungary, I was in the United States. I brought all the electronic equipment in the experiment, but did not bring the ball and aluminum frame. However, I have talked with some of my electrical engineer friends, and I want to make the next iteration more professional.
The original settings are a bit hacky. If you watch the video, you can see wires everywhere. There are some things that I will change next.
In the first experiment, people suggested to me different ideas about games that mice can play. One of my favorites is 3D Pac-Man, because it makes sense in terms of behavior. Rats really don't like to attack things. They fled from things in the wild. This is the dynamics of 3D Pac-Man. You have to look back to see if there are any ghosts behind you.
Of course, this setting is not realistic yet, because the mouse must turn around.
You need multiple screens.
VT: Yes, it is very difficult for rats. This is why these Doom-like first-person shooters make more sense. It's easy and fun to make mice run and shoot.
More information about mice: Our efforts to kill mice have allowed them to evolve at a super speed
Are you concerned about supporting the adoption of clean energy? Find out how much you can save (and the planet!) by switching to solar power on Learn Solar.com. Register through this link, Futurism.com may receive a small commission.