Ad

Sunday, January 26, 2020

Handling Imbalanced Dataset in Machine Learning


  • Imbalanced dataset can affect machine learning decision boundaries.
  • Distribution of target class matters!
  • Why is imbalanced class bad - source 1
  • Sometimes imbalanced dataset is "good" such as in fraud detection. It is important to find those anomalies. 
  • "imbalancec class puts accuracy out of business". It is important to not choose accuracy as metric because the model can cheat by just guessing the majority class and achieve high accuracy. The high accuracy in this case is an illusion. It is about using a trade-off of precision and recall, confusion matrix.

Related Concepts:
  • Imbalanced datasets
  • Confusion matrix
  • Resampling
  • Random under-sampling
  • Random over-sampling
  • Python imbalanced-learn module
  • Random under-sampling and over-sampling with imbalanced-learn
  • Under-sampling: Tomek links
  • Under-sampling: Cluster Centroids
  • Over-sampling: SMOTE
Source 4


1 comment:

  1. Amazingly written blog. Great Post!!! Thanks for the data update and waiting for your new updates.

    Wordpress Development Company
    Mobile App Ideas
    Web Development

    ReplyDelete

My Little Green Book of Machine Learning and Deep Learning, Artificial Intelligence

Data pre-processing Turn Complex Data into Numbers Turn data into features. Turn data into feature vectors. Machine Learning models can...