All it takes is a bit of Googling to know that artificial intelligence that is self-taught has a lot of problems when it comes to dealing with the physical (real) world.
But why is that?
There is no doubt about the fact that modern artificial intelligence systems are wonderful things.
They can start from ground zero and then learn enough about a given game to grow and become world-beaters inside a few hours.
On the other hand, artificial intelligence researchers till have not gotten over their struggle to apply these latest artificial intelligence systems in the real world or at least beyond the overly-simplistic arcade world.
Machines weren’t this good from the beginning though.
Even when they could easily trounce world champions, they gave their human counterpart some respect by first starting to learn the game from their experience.
But that was the past.
Now, the game has changed.
Engineers at IBM came up with the Deep Blue computer back in 1997 in order to defeat Garry Kasparov (the famous Chessmaster).
They used centuries worth of quality chess wisdom in order to train their machine.
Fast forward to 2016 and we saw DeepMind AlphaGo from Google thrashing Lee Sedlo, the then champion at Go.
Go, just like chess, is an ancient board game.
How did Google DeepMind do it?
Well, it pored over probably millions and millions of various positions from close to hundreds and thousands of human Go matches.
That apparently is not enough for artificial intelligence researchers.
They are rethinking the methods with which they enable their bots to incorporate human knowledge in its totality.
As of now, the trend is clear enough:
Artificial intelligence researchers simply do not bother.
Back in October of 2017, Google DeepMind researchers published some details regarding a modern Go artificial intelligent playing System.
They called the new system as AlphaGo Zero.
It learned to play Go without studying any human Go games.
What did it do instead?
It started right at zero.
The new AlphaGo Zero started to learn the game’s official rules first.
Then it started to play the game against no other than itself.
Researchers found that the first few moves that the new system made were actually utterly random.
AlphaGo Zero, after each Go Game, would fold in all the new knowledge that it had gained from the game.
It studied which moves led the system to a win.
And then it studied which movies did NOT lead to a victory.
After some time had passed and AlphaGo Zero ended its series of scrimmages with itself, it went ahead to a Go match.
More specifically, the superhuman edition of AlphaGo.
The same version that badly beat Lee Sedol.
So well did AlphaGo Zero match up against AlphaGo?
It turns out, AlphaGo Zero won a total of 100 games while AlphaGo won zero.
Moreover, the same Google DeepMind team didn’t just stop their work with AlphaGo Zero.
And created another system which would eventually evolve into another superhuman master player in the already prestigious AlphaGo family.
Google DeepMind team called the latest artificial intelligence system as simply AlphaZero.
The team also posted a paper on the subject to arxiv.org (a scientific preprint website) in December of last year.
In the paper, Google DeepMind artificial intelligence researchers revealed some interesting information.
The paper stated that the team actually had built its new system from scratch.
They found that the newly trained-up system they called AlphaZero managed to outperform even AlphaGo Zero.
To put it another way, the latest version of AlphaZero, a bot, beat the bot that managed to beat the older bot that actually beat Lee Sedol, the greatest Go player of the last 20 years.
AlphaZero didn’t just have the skills to learn and beat everybody at Go.
It also had applications in other similar games such as chess.
Chess, just like Go, relies on a set of rules.
And AlphaZero eats up such games with ease.
Researchers gave AlphaZero rules for Shogi (a variant of chess in Japan).
What did they find?
They found that AlphaZero managed to learn chess and then defeat made-to-order world class level algorithms for Shogi.
And it did that in record time.
Experts in the field had to marvel at AlphaZergo’s ability to play in an unfamiliar and aggressive style.
They were so impressed that Peter Heine Nielsen, a Danish grandmaster, told an interviewer from BBC that he had always wondered about what he would see if a superior alien species coincidently landed on the planet that we call earth and then showed humans how they went about playing chess.
By seeing AlphaZero in action, he said, now he knew what that would feel like.
But that’s isn’t it.
The last 12 months allowed us to see more otherworldly bots which were completely self-taught and great other tasks.
Bots easily emerged in other similar rules-based settings such as Dota 2 and the no-limit edition of poker.
Needless to say, AlphaZero and other bots showed that they could perform better than humans in a diverse range of rules-based activities.
Readers should keep in mind that beating the best players in the world at Dota 2 is no easy feat.
Unlike Go, Dota 2 is an internationally renowned multiplayer online competitive video game.
It is probably the most popular video game in the world right now.
In this game, fantasy-themes superheroes use their weapons and some magic to battle other players in order to control the world (an alien one).
As one would expect, the technology companies (and otherwise) who are pouring in their resourcing in these bots and other similar artificial intelligence systems don’t want their bots to waste their potential on games.
In other words, they have much grander and important ambitions.
They don’t just want their bots dominating the virtual tournaments where people play video games.
They actually want to improve their bots so that one day they may have the ability to perform similar feats in the real world.
That’s what DeepMind researchers want to do.
Of course, they would have to use slightly different methods in order to enable bots to perform well when trying to solve real-world problems such as building various types of superconductors that work at room temperature.
Or understand the whole origami that one needs in order to fold proteins directly into effective and powerful drug molecules.
One also can’t ignore the fact that many practitioners in the field of artificial intelligence have this hope of eventually building a general artificial intelligence.
Some experts say that the term itself is pretty ill-defined.
But that shouldn’t take us away from the fact that it is a captivating goal.
A general artificial intelligence would have the ability to think just like a real person.
Moreover, it would have the versatility and the knowledge to attack and then solve several problems of varying kinds and in different fields.
That is probably the reason technology companies are investing so much of their resources in such artificial intelligence systems.
With that said, no one has a clear idea how far researchers can go beyond the arcade game board with current AI techniques.
A computer scientist working at the University of Washington, Pedro Domingos, recently said that he did really know if the ideas that researchers had used in AlphaZero would generalize readily.
He also said that games (including video games) represented something very unusual.
Imperfect World And Perfect Goals
There is one small little detail or characteristic that all games including the likes of Go and Chess share with each other.
That characteristic is that players have no problems in seeing all the relevant pieces on each side and that too at all times.
Let’s put in that in simpler terms.
Each given player playing these games always has unrestricted access to what researchers term as perfect information.
Well, about the current state of any given game.
There is no doubt about the fact that sometimes, these games (even with their rules and the fact that players have knowledge of all pieces on both sides) can get devilishly complex.
However, all that a player has to do is take the current situation of the game into account and think forwards.
No one needs to present any proof that there are a ton of real-life situations that aren’t like these games.
Let’s take an example to make things easier to understand.
Assume for a second that a researcher asks a machine to conduct a specific type of business deal via some negotiation.
Or, look at a given patient and diagnose the patient’s illness.
A doctoral student of computer science at the Carnegie Mellon University, Noam Brown, recently said that the majority of the real-world problems and strategic interactions involved information that was hidden.
He also said that he felt the major portion of the artificial intelligence community had neglected the above-mentioned fact.
Noam Brown has specialized in Poker.
And according to him, this presents artificial intelligence systems with a completely different challenge.
Poker is different from Chess and Go in the sense that a given player does not have any means to see the opponent’s cards.
On the other hand, researchers now know that machines which have the ability to play against themselves can reach superhuman mastery levels.
Back in January of last year, Noam Brown along with Tuomas Sandholm (Noam’s adviser) created Libratus (a program).
With Libratus, they managed to outplay a total of four professional players of poker at a no-limit and head up Texas Hold’em.
The program managed to finish a full $1.7 million ahead of all other players by the time the competition reached its end after 20 days.
Apart from Poker, there is another video game by the name of StarCraft II.
It is perhaps the more daunting game when compared to Poker and Chess because it involves even more imperfect information.
StarCraft II, just like Dota 2, is an online multiplayer video game that has managed to attract a vast following of players.
In StarCraft II, players first have to pick a single team from a list of given teams.
Then they have to build an entire army from scratch.
The final step, as usual, is to wage a war against the other player/players across a wide sci-fi virtual landscape.
There is one twist though.
The game shrouds that virtual landscape in a fog of war.
Because of that, a given player can only see those areas where the player has buildings or soldiers.
Even simple looking decisions such as scouting one’s opponent comes fraught with many uncertainties.
StarCraft II represents just one game that artificial intelligence still can’t seem to beat.
What are the barriers to artificial intelligence system’s success?
Well, these include,
- The speed at which the AI system has to play the game.
- The number of moves that each player has to make in a given game.
Sometimes, that number can reach astronomical values in the thousands.
Moreover, one has to make all those moves very quickly.
Each player, irrespective if it is a machine or a human, has to deal with worries that inevitably come when managing a huge set of possible different futures with each and every single click of their mouse buttons.
Artificial intelligence is great at many things.
StarCraft II isn’t one of them.
Currently, artificial intelligence systems have a hard time fighting toe-to-toe against some of the top human players in the StarCraft II arena.
It is simply not within the current AI systems’ reach according to some.
Back in August of 2017, researchers at Google DeepMind managed to strike up a partnership with Blizzard Entertainment.
Blizzard Entertainment is the sole company behind the development of StarCraft II.
In other words, it made the game.
What was the purpose of the partnership?
To release all the various tools which the researchers at Google DeepMind and people at Blizzard Entertainment believe would assist artificial intelligence researchers in opening up the game to further study it.
That partnership comes with its due set of challenges.
But despite all the complexity, the core and straightforward enunciated objective of Starcraft II is to simply annihilate the enemy.
Moreover, this is something that StarCraft II probably shares with all other related video games including,
- Dota 2
As mentioned just now, the idea of winning isn’t just central to StarCraft.
All video games want the player to do one thing:
Let’s take a look at that from the perspective of a given algorithm.
As far as an algorithm is concerned, it must have problems which have a defined objective function.
In other words, it has to seek out a goal.
So in a scenario where an artificial intelligence system like AlphaZero is playing chess, that isn’t a hard problem to solve.
For the artificial intelligence system, a win in chess would mean an increase in total score by one.
A loss would mean minus one.
And a draw would mean zero effect on the overall score.
The algorithm behind AlphaZero had one main objective function.
And that was to simply do everything to maximize its total score.
Poker is similar in the sense that the objective function of the game is equally as simple as Chess’.
The player must win the maximum amount of money.
Needless to say, situations that humans face in real life don’t have such simple objectives.
To take an example, an autonomous self-driving vehicle has this need of an objective function that is much more nuanced.
The objective function in such a case is akin to something that one would take time to phrase carefully as in the case of explaining one’s wish to Aladdin’s genie.
Another example is that of a prompt delivery of the passenger who is sitting in the car from the current location to the final (and correct) location.
In doing so, the autonomous vehicle has to obey all the traffic laws and then have the capacity to appropriately weigh the human life value in uncertain and dangerous situations.
But how do artificial intelligence researchers manage to craft such complex objective functions?
According to Domingos, crafting the appropriate objective function is just one of the many things that actually distinguished average machine-learning researchers from a great machine-learning researcher.
Let’s also consider the example of Tay.
What is Tay?
Tay is a Twitter chatbot.
Microsoft released the Tay chatbot back in March of 2016.
Tay had the simple objective of engaging with the people.
And Tay managed to do that with ease.
Domingos then revealed that Tay discovered something very unfortunate while trying to engage with the people on the platform.
It discovered that the most efficient way of maximizing engagement with the people was to spew out various racial slurs.
After less than 24 hours, the company behind Tay snatched it back offline.
What people really need to think about is the ultimate goal of artificial intelligence.
Do they just want machines to have the ability to do close to as much as a human can do?
If that is the case then a generalist and self-taught board-game beater-of-world-champions AI system like that of AlphaZero may have several ways to go.
According to a cognitive scientist at MIT, Josh Tenenbaum, people had to observe that there was a huge gulf between what researchers could currently see in artificial intelligence and real activities related to thinking and creative exploration to come up with different ideas.
There is little doubt, according to Josh, that such kind of intelligence is present.
But, for the most part, only the best artificial intelligence researchers have that problem in their heads.
Of course, there are many artificial intelligence researchers who are pretty conscious of the huge amount of hype that has recently surrounded the field of artificial intelligence.
Because of that, they offer qualifiers of their own.
Another deep learning researcher working at Google, Francois Chollet recently mentioned that he would exercise great caution to not overestimate the significance of different AI systems playing such games even for artificial intelligence itself or job in a general view.
The fact is, humans aren’t terribly good at playing games.
He also said that the community should keep a specific lesson in mind:
Specialized, but simple, tools have the ability to achieve a lot.