For the first part of this post click here.
The professor of informatics working at the University of Sussex, Thomas Nowotny, has recently managed to uncover parallels between a class of models and the olfactory system that the community calls the support vector machines.
However, Thomas Nowotny has not stopped since then and has actually continued to work harder in order to develop a more deeper and better understanding of systems such as olfactory and how these systems work.
Thomas Nowotny also tries to always keep in mind the potential artificial intelligence applications of his research.
As mentioned before as well, research in the field of olfactory systems did not really begin in a meaningful way until the 1990s.
By the time the early 2000s came around, many researchers started to study the olfaction mechanism.
Through that, researchers managed to develop computer algorithms in order to determine how they could help computational efficiency with concepts such as sparsity in higher dimensions and random embedding.
A particular pair of computer scientists, Ramon Huerta who works at the University of California and Thomas Nowotny who works in England at the University of Sussex even managed to draw connections from their research to a different type of model for machine learning.
They called that new machine learning model as a support vector machine.
Both the researchers made arguments about all the ways both artificial and natural systems managed to process information were, in a formal context, equivalent.
In other words, both made use of dimensionality expansion along with random organization to represent any given complex data efficiently.
According to some, evolution and artificial intelligence had converged rather independently but on the same kind of solution.
That observation and connection intrigued Nowotny along with his colleagues to keep at it and continue to not only understand but also explore the actual interface between machine learning and olfaction.
Researchers wanted to look to a much deeper link between machine learning and olfaction.
By the time 2009 came around, researchers successfully showed that a given olfactory model which was based on flies and insects and initially created in order to recognize different odors, could actually also come in handy when recognizing digits which were handwritten.
Additionally, researchers also found out that by removing a large number of the system’s neurons (in other words, the majority portion of the related neurons so that researchers could mimic the way cells in the brain died and did not get replaced) actually did not manage to affect the system’s performance to any significant level.
Not only that, Nowotny also mentioned that some parts of the olfactory system might go down because of the removal of neurons, but as a whole, the system would continue to work properly.
He also said that he envisioned implementing such kind of hardware in a machine like the Mars rover so that it has an easier time when operating under the harsh environmental conditions of the planet.
However, even though the potential of the olfactory system became evident, researchers did not do much work on it for a good while.
Hence, they also failed to do any proper follow up on all those olfactory findings.
That is not true now, however.
Very recently, some researchers got their heads together and actually began to revisit the biological structure of a given olfaction system in order to gain more insights of how they could use more information in order to improve slightly more particular problems in the field of machine learning and artificial intelligence.
Fast Learning with hard-wired knowledge
Delahunt along with another group of his colleagues have, in fact, managed to repeat a pretty similar kind of experiment which Nowotny previously conducted.
In other words, Delahunt’s team made use the moth olfactory system.
They used it as the foundation of their experiment.
Not only that, they also compared it to all the traditional and existing machine learning models.
What researchers found was that even with as few as 20 samples, the model based on the moth olfactory system recognized a set of handwritten digits much better.
However, when researchers provided the same olfactory system a greater number of training data, some of the other available machine learning models proved themselves as the stronger ones and also the more accurate ones.
Delahunt also mentioned that he considered machine learning methods as very good methods for providing researchers with extremely precise classifiers when there is the availability of tons of training data.
However, the insect model (the olfactory system) model had shown good promise at doing a relatively rough classification but do so at a rapid pace.
According to some experts, the olfactory system tends to work better and more efficiently when it comes to the actual speed of learning.
Why is that?
That is because in such a case, the “learning” phase departs from being totally about seeking out representations and features which are optimal and suited for the specific task that is at hand.
The olfactory system, instead of doing that, reduces its work to just recognizing those features among the slew of random features which it finds useful and useless.
This is what it does.
According to a biologist working at the Southern Medical University in China, Fei Peng, if one could train a given system with the help of a single click, then that would certainly provide more beauty.
If we’re only talking about the effects of using the olfactory system, this strategy almost becomes like successfully baking a few primitive and basic concepts right into the given model.
This is much like the human brain which seemingly has a hard-wired general understanding of the physical world around it.
Some say that this structure itself is pretty much capable of carrying out some innate and simple tasks without the help of much instructions.
Nathan Kutz along with Charles Delahunt working at the University of Washington have actually managed to successfully port the definite structure of the olfactory network in a given month into the context of machine learning.
By doing so, they have also managed to create what they have called insect cyborgs.
It is also true that last year, Navlakha lab gave the scientific community one of best and most striking examples of how an olfactory strategy could work.
Navlakha along with his colleagues Sanjoy Dasgupta (who works as a computer scientist at the prestigious University of California in San Diego) and Stevens, had this desire to discover an olfaction-inspired method of performing searches which were based on the model of similarity.
They conjectured that just as the hugely popular streaming platform that is YouTube has the ability to generate a new list of videos in the user’s sidebar by just looking at the video that the user is currently streaming, biological organisms must also have such an ability to make accurate and quick comparisons when they wanted to identify odors.
Some believe that a fly may actually learn fairly early one that it has to avoid the smell which is related to vinegar and should try to approach the smell which is associated with a ripe banana.
However, that is not always the case.
The fly has to deal with a complex environment.
Not only that, it has to deal with an environment which offers plenty of noise sources.
The fly may actually never get to experience the precise same odor ever again.
Researchers say that whenever a fly detects a new kind of smell, it has to somehow figure out a way in which it can associate that new smell with something it has experienced before.
The solution usually lies in an odor that the fly thinks resembles the new scent most closely.
This is the way the fly is able to recall and apply the most appropriate behavioral response to a given new odor.
Apart from doing that research, Navlakha also spent some time in creating a search algorithm based on the olfactory-based similarity concept.
He then applied the search algorithm to some data sets of photos.
Navlakha along with his team of researchers also found out that their search algorithm actually performed better when compared to traditional and current non-biological methods which involved only dimensionality reduction.
In fact, researchers found that sometimes the olfactory-based search algorithm for solving similarity problems outperformed traditional non-biological methods about two to three times.
It is also true that in the more standard model techniques, researchers compared objects by focusing on a very few number of basic dimensions and/or features.
Navlakha mentioned that the fly-based olfactory system approach also made use of less computation process in order to reach similar levels of result accuracy.
In fact, it utilized computation which was an order of magnitude less than traditional non-biological models.
According to Navlakha, the olfactory-based system either won in performance or in cost.
Then there is the fact that Adam Marblestone has gone on record saying that the olfactory system and the visual system presented researchers with an interesting nexus point.
He thought of this point as an entry point for researchers to actually think about neural networks of the next generation.
Delahunt, Navlakha and Nowotny also showed that as far as classification computations and other similar kinds of tasks were concerned, even a fundamentally untrained network could provide some usefulness.
Moreover, it is also true that building something in such a given encoding scheme left the given system poised to go ahead and make all subsequent learning phases shorter and easier.
Researchers could use such a system in tasks which involved memory and/or navigation to take just one instance.
It could also prove itself useful in situations where there are a lot of changing conditions.
Such situations include obstructed paths and others.
Readers should know that any situation where there are a lot of changing conditions, always has a proclivity to leave the given system with very little time to learn.
Moreover, it also reduces the number of examples that the system has available to learn from.
Fei Peng along with his research group has also managed to start a research project by simply studying such systems.
They have also created a model based on the ant olfactory system in order to enable it to make decisions on how to go about navigating a familiar route.
The current system has to do that from a reasonable series of overlapping images.
Of course, currently, the actual work is under review.
That has not stopped Navlakha from applying a similar method based on the olfactory system for tasks such as novelty detection.
What does novelty detection mean?
It means the recognition of a given thing as new even though the system has exposed itself to thousands and thousands of similar looking objects in an earlier phase of its learning.
With that said, it is also true that Nowotny and his research team are trying to examine how a given olfactory system is able to process mixtures.
Because of his hard work, Nowotny has already started to see some possibilities for potential applications to other challenges involving machine learning.
To take an example, some organisms have to perceive some kind of odors as a mix and other kinds as a single scent.
In order to further understand that, a person may just take in, probably, dozens of different chemicals and only know that he/she actually smelled nothing but a rose.
On the other hand, it is also possible that he/she might manage to sense an equal number of different chemical coming from a bakery nearby and only differentiate between croissants and coffee.
Nowotny along with his team of researchers have managed to find that odors which are separable are not really perceived at the exact same time.
Instead of that, someone might actually process croissant and coffee odors extremely rapidly but in alternation.
There is no doubt about the fact that such an insight could prove very beneficial for further work in the field of artificial intelligence as well.
For example, let’s talk about the cocktail party artificial intelligence problem.
This problem refers to the level of difficulty that a given system has to manage in order to separate various and numerous conversations when all are happening in a setting that is noisy.
Assuming that there are several speakers in the given room, an artificial intelligence system might go about solving this problem by first cutting all the sound signals into extremely tiny time windows.
Again, assuming that the artificial intelligence system could recognize the sound that comes from a given single speaker, the system could then make attempts to suppress all input sounds coming from other speakers.
If a given artificial intelligence system could alternate like that, the actual network could manage to take those conversations and disentangle them.
What about those insect cyborgs?
Well, arxiv.org (a scientific preprint site) showed a paper which researchers posted in recent months, Delahunt and Nathan Kutz (his colleague at the University of Washington) took the kind of research that we have described so far, a step further.
How did they do that?
They did that by creating insect cyborgs.
At least, that is what they call them.
Researchers actually utilized the outputs coming from the moth-based model they had as the inputs of a given computer machine learning algorithm.
Subsequently, they managed to see vast improvements in the given system’s competence at classifying images.
Delahunt later mentioned that the moth-based model gave the existing machine learning algorithms material (which was much stronger) to work with.
He also mentioned that there were some different types of structure which the moth’s brain had managed to pull out.
According to Delahunt, the moth-based model’s ability to have that different types of structure helped their machine learning algorithm.
Gaining confidence from the research, other computer scientists now have hope that they will also be able to make use of studies in the olfactory system in order to figure out how they could use different and multiple forms of learning to coordinate with each other in deeper networks.
However, according to Peng, right now, the research community had only managed to cover a tiny bit of the new system.
He said that he was not sure how he would go about improving existing deep learning systems at this moment in time.
A neurobiologist working at the Salk Institute, Tatyana Sharpee, recently managed to find that it was possible to map odors directly onto a hyperbolic space.
However, she still wonders whether such an insight could actually inform researchers how to best structure all the input data that they have to feed to a given deep learning system.
Of course, one of the places where researchers could start their quest lies in coming up with ways on how to define a given system’s inputs.
Researchers need to pay more attention to figure that out in addition to implementing the actual olfaction-based architecture.
Tatyana Sharpee, published a recent paper in Science Advances, where her team sought out a way to appropriately describe smells.
According to this research team, images represented something more or less the same.
Of course, it also depended on the exact distances between the images’ pixels in a sort of visual space.
However, when it comes to olfaction, that type of distance did not apply.
That also had true for the structural correlations as they could not provide any kind of reliable bearing.
It is true that there are a variety of different ways to perceive odors that have similar chemical structures.
On the other hand, it is also true that sometimes there are some very similar ways to perceive odors that have very different chemical structures.