Sunday, January 26, 2020

Handling Imbalanced Dataset in Machine Learning

  • Imbalanced dataset can affect machine learning decision boundaries.
  • Distribution of target class matters!
  • Why is imbalanced class bad - source 1
  • Sometimes imbalanced dataset is "good" such as in fraud detection. It is important to find those anomalies. 
  • "imbalancec class puts accuracy out of business". It is important to not choose accuracy as metric because the model can cheat by just guessing the majority class and achieve high accuracy. The high accuracy in this case is an illusion. It is about using a trade-off of precision and recall, confusion matrix.

Related Concepts:
  • Imbalanced datasets
  • Confusion matrix
  • Resampling
  • Random under-sampling
  • Random over-sampling
  • Python imbalanced-learn module
  • Random under-sampling and over-sampling with imbalanced-learn
  • Under-sampling: Tomek links
  • Under-sampling: Cluster Centroids
  • Over-sampling: SMOTE
Source 4

1 comment:

  1. Amazingly written blog. Great Post!!! Thanks for the data update and waiting for your new updates.

    Wordpress Development Company
    Mobile App Ideas
    Web Development


Algolia Search API Basics Tutorial

I write full time now for write me to say hi, request content or be notified of new tutorials like this. Unqitech writes abou...