Skip to content
Related Articles

Related Articles

Winsorization

View Discussion
Improve Article
Save Article
  • Last Updated : 30 May, 2021
View Discussion
Improve Article
Save Article

Winsorization is the process of replacing the extreme values of statistical data in order to limit the effect of the outliers on the calculations or the results obtained by using that data. The mean value calculated after such replacement of the extreme values is called winsorized mean.

 For example, 90% winsorization means the replacement of the top 5% and bottom 5% of the data. The top 5% of the data is replaced by the value of the data at the 95th percentile and the value of the bottom 5% of the data is replaced by the value of the data at the 5th percentile. 

Input:

  • A numeric array whose values at the upper end and the lower end are to be winsorized.
  • The first argument of the tuple is the percentage of values at the lower end which are to be winsorized.
  • The second argument of the tuple is the percentage of values at the upper end which are to be winsorized.

Output:

A numeric array whose values at the upper end and at the lower end are winsorized as defined by the user.

Example #1:

Python3




# Libraries to be imported
import numpy as np
import matplotlib.pyplot as plt
import random
from scipy.stats.mstats import winsorize


Let us see an example where outliers are present on both the upper end and the lower end of the data.

Python3




# Creating an array with 100 random values
array = [np.random.randint(100) for i in range(100)]
  
# Creating outliers
# Here, the values which are selected for creating outliers 
# are appended so that same outliers are not created again.
AlreadySelected = []
i = 0
  
# Creating 5 outliers on the lower end
while (i < 5):
    x = np.random.choice(array)  # Randomly selecting a value from the array
    y = x - mean*3
    array = np.append(array, y)
    if (x not in already_selected):
        AlreadySelected.append(y)
  
        i += 1
  
    else:
        continue
  
# Creating 5 outliers on the upper end
i = 0
while (i < 5):
    x = np.random.choice(array)  # Randomly selecting a value from the array
    y = x + mean*4
    array = np.append(array, y)
    if (x not in already_selected):
        AlreadySelected.append(y)
  
        i += 1
  
    else:
        continue
  
std = np.std(array)  # Storing the standard deviation of the array
mean = np.mean(array)  # Storing the mean of the array
  
plt.boxplot(array)
plt.title('Array with Outliers')
plt.show()


Output:



Python3




print(mean) # mean of the numeric array with outliers


Output:

Now, we winsorize the array by 10% i.e. we winsorize 5% of the highest values and 5% of the lowest value of the array:

Python3




WinsorizedArray = winsorize(array,(0.05,0.05))
  
plt.boxplot(WinsorizedArray)
plt.title('Winsorized array')
plt.show()


Output:


Python3




WinsorizedMean = np.mean(WinsorizedArray)
print(WinsorizedMean)


Output:



In this case, there is only a slight change in the mean value of the data.

Now, let us see an example where outliers are present only at one end of the data.

Python3




# Creating another array with 100 random values
array2 = [np.random.randint(100) for i in range(100)] 
std = np.std(array2)
mean = np.mean(array2)
AlreadySelected = []
# Creating outliers on the upper end
i = 0 
while (i<5):
    x = np.random.choice(array2) # Randomly selecting a value from the array
    y = x + mean*4
    array2 = np.append(array2,y)
    if (x not in AlreadySelected):
        AlreadySelected.append(y)
  
        i+=1
          
    else:
        continue
          
plt.boxplot(array2)
plt.title('Array with outliers')
plt.show()


Output:



Python3




OutlierArray2Mean = np.mean(array2)
print(OutlierArray2Mean)


Output:



Python3




WinsorizedArray2 = winsorize(array2,(0.1,0.1))
# In this case, the lower 10% values of 
# the data will have their values set equal to the value of the data point at 
#the 10th percentile.
  
plt.boxplot(WinsorizedArray2)
plt.show()
  
WinsorizedArray2Mean = np.mean(WinsorizedArray2)


Output:




Python3




WinsorizedArray2Mean = np.mean(WinsorizedArray2)
print(WinsorizedArray2Mean)


Output:



In this case, there is a significant difference in the mean value.


My Personal Notes arrow_drop_up
Recommended Articles
Page :

Start Your Coding Journey Now!