Remove Duplicates

You often encounter duplicates in your data when you get it from different sources. Although there are inbuilt functions in the Pandas library to remove duplicates which you will learn in upcoming modules, this raises the question of how do you really remove duplicates. 

Let's take a smaller version of the same problem. You will be given a list of integers, you have to remove all duplicate values from the list. How would you do it?

Let's see how will this look like in code on the console.

It is your turn to try it out now!

You can also store lists or strings as keys and values in dictionary. The condition is that no two keys are the same in a dictionary, we use this and solve the problem. There however is one more way using sets. Why do you think we did not use sets?