Tuesday, June 26, 2018

Recurrent Neural Network RNN Basics

Recurrent Neural Network (RNN) is useful for processing sequence data like sound, words, and time series data . RNN remembers a bit of the state from before. It can predict what will come next. Time series is good for heart disease over time, hormone level, blood sugar. 

RNN weakness: sometimes gradients too close to 0 or too computationally large. It can also be bad at tracking long term memories - need to use LSTM instead, which has a forget gate, input gate, update layer, output layer

Real world usage:
Transform sequences like text, music, time series data,
Build a RNN generate new text character by character
Natural language processing, Word embedding, Word2Vec model, Semantic relationship between words, 
Combine embedding and RNN to predict sentiment of movie reviews

No comments:

Post a Comment

Sample .gitignore file

Some good files to ignore .DS_Store a hidden file generated by Mac, .ipynb_checkpoints, hidden files generated by Python Notebooks aka Ju...