Indigo.ai is on a mission to enhance communication between people and businesses. We developed an artificial intelligence platform that uses machine learning to automate conversations between businesses and customers via chat.
In the Ai team we are the R&D soul of the company. We are engineers and mathematicians with experience in Machine Learning and a deep understanding of Natural Language Processing. We try to investigate every problem starting from the theoretical and modeling base, reaching the development and production deployment in a lean and effective way.
In the last 4 years NLP faced a revolution represented by the introduction of the Transformer architecture, a network designed to process sequential inputs using attention mechanisms. Among NLP models adopting the Transformer architecture, BERT affirmed as a groundbreaking and flexible alternative. Thanks to its two-steps training, this network demonstrated being able to reach high quality performances in any NLP task.
A limit that affects Language Models consists in the fact that they learn using textual information exclusively. If on one hand humans gather information from the external world using vision, audio signals and so on, on the other NLP models are only able to gather information by “reading”. An attempt to break this paradigm is represented by the Vokenization approach, published last year by two researchers of Chapel Hill university. The idea of Vokenization is to pre-train a BERT model to predict both correct words in a sentence and correct images associated with words. For this reason, this visually-supervised BERT network can be considered a multimodal model gathering information both from text and images.
With this project we want to assess if the visually-supervised training proposed with Vokenization is a valid approach to bridge the gap between human and artificial intelligence. In particular we want to explore the sphere of commonsense reasoning tasks, a field in which classical Language Models have trouble and that we believe can obtain sensible improvements thanks to visual supervision.
Here is a set of possible experiment in the field of commonsense reasoning that are in target for the objective of this project:
We underline that the experiments listed above are just a proposal, and the optimal set of experiments and tests to conduct will be defined during the internship period as part of the project.
The skills we believe a successful candidate should have are: