- Gradient Descent may have issues when the scale of the data is large
- If the number of training samples m is large
- Gradient Descent algorithm requires summing over all of m
- e.g. US population census population of 300MM
- Stochastic Gradient Descent is a modification of gradient descent
- In other words, the cost functions are different
- Stochastic every iteration is faster
- Steps: randomly shuffle dataset, optimize one training data at a time, improve parameters early one at a time, instead of looking at the examples together as a batch
- Weakness: generally moves towards global minimum, but doesn't always go there, can reach the general vicinity of the global minimum. Does not converge as nicely as gradient descent.
- In reality, practical data science, once it gets close to the global minimum its parameters are good enough. In real life, it works out.
Your byte size news and commentary from Silicon Valley the land of startup vanities, coding, learn-to-code and unicorn billionaire stories.
Saturday, March 25, 2017
Difference between Batch Gradient Descent and Stochastic Gradient Descent - Udacity Machine Learning Nanodegree Coursera
Recommend this great 13 minutes crystal clear video by Andrew Ng on Coursera explaining the differences between batch gradient descent aka gradient descent aka normal gradient descent versus Stochastic Gradient Descent. https://www.coursera.org/learn/machine-learning/lecture/DoRHJ/stochastic-gradient-descent It's clear simple and easy to understand without prerequisite. Andrew Ng shows you how the formula differs, how the step by step train strategy differs and a visualization of the trajectory to find global minimum (the center of all the ellipse in his graph).
Subscribe to: Post Comments (Atom)
React UI, UI UX, Reactstrap React Bootstrap
React UI MATERIAL Install yarn add @material-ui/icons Reactstrap FORMS. Controlled Forms. Uncontrolled Forms. Columns, grid
This review is updated continuously throughout the program. Yay I just joined the Udacity Nanodegree for Digital Marketing! I am such an Uda...
All you need to know about Snap IPO. Tech startup news explained for Youtubers in minutes.
This is the photo collection, Youtube video construction in progress. Fisherman's Wharf Lure
Post a Comment