chatbots

HOW TO MAKE A CHAT BOT

chatbots

All 7 billion people on earth would have the capability of learning anything much faster. The web democratize information and this next evolution will democratize something just as important, guidance. The ideal chat boot can talk intelligently about any domain.

That's the holy grail, but domain specific chat are definitely possible. The technical term for this Isa question answering system. Surprisingly, we've been able to do this since way back in the '70s.

Lunar was one of the first. It was, as you might have guessed, rule based, so it allowed geologists task questions about moon rocks from the Apollo missions.

A later improvement to rule based Q&A systems, allowed programmers to encode patterns into their boot called artificial intelligence markup language, or AIML. That meant less code for the same results. But yeah, don't use AIML. It's so old it makes Nuka look new. Now with deep learning, we can do this without hard coded responses and have much better results.

how to make chatbots in python

The generic case is that you give it some DAX as input, and then asking a question. It'll give you the right answer after logically reasoning about it. The input could also be that everybody is happy. And then the question could be what's the sentiment? The answer would be positive.

Other possible questions are what's the entity? What are the part of speech tags? What's the translation to French? We need a common model for all of these questions. when they released a paper introducing this really cool idea called a memory network. LSTM networks proved to be useful tool in tasks like tech summarization,

Their memory, encoded by hidden states and weights, is too small for very, very long sequences of data. Be that a book or a movie. A way around this for language translation, for example, what's to store multiple LSTM states, and use an attention mechanism to choose between them.

But they developed another strategy that outperformed LSTM's for Q&A systems. The idea was to allow a neural network to use an external data structure as memory storage. It learns where to retrieve the required memory from the memory bank in a supervised way.

When it came to answering questions from POI data that was generated, that info was pretty easy to come by. But in real world data, it is not that easy. Most recently, there was a foul month long Cagle contest that a startup called Meta Mind placed in the top 5% for.

To do this they built a new state of the art model called a dynamic memory network that built on Face book’s initial idea. That's the one we'll focus on, so let's build it programmatically using Keas. (saran (voiceover)) This dataset is pretty well organized. It was created by Face book AI research for the specific goal of improving textual reasoning. It's grouped into20 different tasks. Each task test a different aspect of reasoning.

how to make chatbots in java

So, overall it provides good overview of all the different capabilities of your learning model. There are 1,000questions for training, and 1,000 for testing per task. Each question is paired with a statement, or series of statements, as well as an answer. The goal is to have one model that can succeed in all tasks easily. We'll use pre-trained Glove vectors to help create sequence of word vectors from our input sentences. And these vectors will acts inputs to the model.

The daemon architecture defines two types of memory. Semantic, and episodic. These input factors are considered the semantic memory, whereas, episodic memory might contain other knowledge as well. And we'll talk about that in a second.

We can fetch our Babel data set from the web, and split them into training and testing data. Glove will help convertor words to vectors, so they're ready tube fed into our model. The first module, the input module, is a GRU, or gated recurrent unit,

chatbot

that runs on a sequence of word vectors. A GRU cell is kind of like an LSTM cell, but it's more computationally efficient since it only has two gates, and it doesn’t use a memory unit. The two gates control when its content is updated, and when it's erased. Update. Reset. Update. Reset. Update. Reset.

And the hidden state of the input module represents the input process, so far in a vector. It outputs hidden states after every sentence, and these outputs are called facts in the paper, because they represent the essence of what is fed. Given a word vector and the previous time step vector, we'll compute the current time step vector. The update gate is a single layer neural network. We sum up the matrix multiplications, and add a biased term. And then the sigmoid squashes it to a list of values between 0 and1, the output vector.

We do this twice with different sets of weights, then we use a reset gate that will learn to ignore the past time steps when necessary. For example, if the next sentence has nothing to do with those that came before it.

The update gate is similar in that it can learn to ignore the current time step entirely. Maybe the current sentence has nothing to do with the answer. Whereas, previous ones did. (saran (voiceover)) Then, there’s the question module. It processes the question word by word, and outputs a vector using the same GRU as the input module, and the same weights. We can encode both of them by creating embedding layers for both.

Then we'll create an episodic memory representation for both. The motivation for this in the paper, came from the hippocampus function in our brain. It's able to retrieve temporal states that are triggered by some response, like a sight or a sound. (saran (voiceover)) Both the fact and question vectors that are extracted from the input enter the episodic memory module. It's composed of two nested GRU's. The inner GRU generates what are called episodes. It does this by passing over the facts from the input module.

When updating its interstate, takes into account the output of attention function on the current fact. The attention function gives score between 0 and 1 to each fact.

chatbot

And so, the GRU ignores facts with low scores. After each full passion all the facts, the inner GRU outputs an episode which is then fed to the outer GRU. The reason we need multiple episodes, is so our model can learn what part of a sentence it should pay attention to after realizing after one pass, that something else is important. With multiple passes,

techguide

how to make a chatbot

chatbots

HOW TO MAKE A CHAT BOT

how to make chatbots in python

how to make chatbots in java

techguide

0 Comments: