- Uses L1-norm
- Uses L2-norm
Quiz: which formula is Lasso? Which one is ridge?
- Regularization penalizes overly complex models
- Large weights usually make penalty term higher, so smaller effective weights are preferred
- Larger weights cost more
- Regularization = regular_loss_function + extra_penalty_term(lambda, weights)
- The extra penalty term also depends on the weights parameter and the lambda rate parameter