On Natural Language Processing

Although I am not a parent, I could easily imagine one of the most wholesome and beautiful moments in the life of one is watching your child say its first word. It’s something you’d want to record on a camera and watch over again.

Language is very important to humans. It’s how we communicate with each other, bring our opinions to life, and also preserve our culture and knowledge. When we learn languages spoken by people from other cultures, we learn a great deal about the way they live and what they know about. We even teach it to our pets, who we hope are able to match the sounds we make to a specific action.

I can remember the very first thing I did on the internet — a google search. Somehow, the search engine was able to take the words I provide to it and find relevant information for me. Nowadays, it is significantly better at doing this task. I can ask it indirect questions like, ‘do I need an umbrella tomorrow?’, and it understands that I need to know the weather condition of the next day.

It seems machines like Google understand human language so well they could even reserve a restaurant for you by talking to a human on the other end. My favorite demo of this level of apparent understanding of human language was at Google I/O 2021 with the Language Model for Dialogue Applications (LaMDA). Imagine a chatbot with the ability to hold a conversation on any topic, maintaining context and achieving almost human-level of understanding. You can watch a demo of it here. Trust me, you’d be amazed.

LaMDA, Source: Google

Natural Language Processing

I’ve been drawn to the way computers work with words since my first encounter with the internet. And the discipline that deals with this is aptly named natural language processing.

You see computers are quite good at understanding mathematical rules which we code in programming languages. These rules and programming languages allow it to quickly perform some tasks that are very laborious for people to carry out. Imagine having to perform some logarithmic scaling on millions of rows of excel data by hand!

However, the challenge with computers is with things like language that are intuitive to us and are quite hard to describe formally. How would you explain the concept of sarcasm to a computer? It turns out the answer is in the field of machine learning/artificial intelligence. Let’s define some terms before we move on.

Natural language processing is an area of research in computer science and artificial intelligence (AI) concerned with processing natural languages such as English or Mandarin. This processing generally involves translating natural language into data (numbers) that a computer can use to learn about the world. And this understanding of the world is sometimes used to generate natural language text that reflects that understanding. — Natural Language Processing in Action, Hobson Lane

NLP, for short, is closely related to another field of research, Natural Language Understanding. Just like the name implies, Natural Language Understanding (NLU), deals with making artificial intelligence understand natural language without formalized syntax of computer languages.

The pipeline of an NLP engineer would involve trying to convert the words into numbers that computers can process and then building machine learning architectures that can use these numbers for the various tasks we require. In concrete terms, it should involve steps like:

1. Data Collection: For every task in mind, the first step would be to collect data closely related to the task at hand. The field of Machine Learning relies on this concept. We feed several algorithms with large amounts of data which could be expensive to collect. In the case of NLP, this step might involve collecting tweets or even reviews on different e-commerce sites like amazon or yelp and may include cleaning and labeling them.
2. Tokenization: This involves breaking each text into chunks of words in preparation for the next step. This step could also involve removing stop words, lemmatization, and stemming.
3. Vectorization: In this step, the tokens obtained from the tokenization step are turned into vectors that the ML algorithms can process. We understand here that the models we create do not actually see the words and understand them in the way we humans do (or we think we do) but actually work on the vector representations of these words.
4. Model Creation and Evaluation: This involves building ML models and architectures like transformers that can chew on the word vectors provided for the several tasks required. These tasks could involve translation, semantic analysis, named entity recognition, and so on. The NLP engineer needs to constantly evaluate the models and architectures against already defined goals and metrics.

What are the tasks we can perform with the progress made on NLP/NLU?

1. Semantic Analysis: This application is very important in e-commerce. We can already build machine learning models that can classify a text based on its emotions. For an e-commerce site, this could be applied with the ML model classifying the review given to know if a product is bad or good.
2. Question Answering
3. Named-entity recognition
4. Translate
5. Chat-bots
6. Knowledge extraction
7. Behavioral prediction
8. Summarization
9. Creative Writing

10. We could also combine NLP with the progress made in the field of Computer Vision to automatically caption images and scenes from videos.

This is the first article in a series on Natural Language Processing with Deep Learning. The next article will cover the first step in the NLP pipeline which is tokenization and vectorization.