Python | Remove all duplicates words from a given sentence

Given a sentence containing n words/strings. Remove all duplicates words/strings which are similar to each others.

Examples:

Input : Geeks for Geeks
Output : Geeks for

Input : Python is great and Java is also great
Output : is also Java Python and great

We can solve this problem quickly using python Counter() method. Approach is very simple.

1) Split input sentence separated by space into words. 
2) So to get all those strings together first we will join each string in given list of strings. 
3) Now create a dictionary using Counter method having strings as keys and their frequencies as values. 
4) Join each words are unique to form single string. 

Python

from collections import Counter

def remov_duplicates(input):

    # split input string separated by space
    input = input.split(" ")

    # now create dictionary using counter method
    # which will have strings as key and their
    # frequencies as value
    UniqW = Counter(input)

    # joins two adjacent elements in iterable way
    s = " ".join(UniqW.keys())
    print (s)

# Driver program
if __name__ == "__main__":
    input = 'Python is great and Java is also great'
    remov_duplicates(input)

Output

and great Java Python is also

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 2: 

Python

# Program without using any external library
s = "Python is great and Java is also great"
l = s.split()
k = []
for i in l:

    # If condition is used to store unique string
    # in another list 'k'
    if (s.count(i)>=1 and (i not in k)):
        k.append(i)
print(' '.join(k))

Output

Python is great and Java also

Time Complexity: O(N*N)
Auxiliary Space: O(N)

Method 3: Another shorter implementation:

Python3

# Python3 program

string = 'Python is great and Java is also great'

print(' '.join(dict.fromkeys(string.split())))

Output

Python is great and Java also

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 4: Using set() 

Python3

string = 'Python is great and Java is also great'
print(' '.join(set(string.split())))

Output

Java also great and Python is

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 5:  using operator.countOf()

Python3

# Program using operator.countOf()
import operator as op
s = "Python is great and Java is also great"
l = s.split()
k = []
for i in l:
  # If condition is used to store unique string
  # in another list 'k'
  if (op.countOf(l,i)>=1 and (i not in k)):
    k.append(i)
print(' '.join(k))

Output

Python is great and Java also

Time Complexity: O(N)
Auxiliary Space: O(N)

Method 6:  

It uses a loop to traverse through each word of the sentence, and stores the unique words in a separate list using an if condition to check if the word is already present in the list.

Follow the steps below to implement the above idea:

• Split the given sentence into words/strings and store it in a list.
• Create an empty set to store the distinct words/strings.
• Iterate over the list of words/strings, and for each word, check if it is already in the set.
• If the word is not in the set, add it to the set.
• If the word is already in the set, skip it.
• Finally, join the words in the set using a space and return it as the output.

Below is the implementation of the above approach:

Python3

def remove_duplicates(sentence):
    words = sentence.split(" ")
    result = []
    for word in words:
        if word not in result:
            result.append(word)
    return " ".join(result)

sentence = "Python is great and Java is also great"
print(remove_duplicates(sentence))

Output

Python is great and Java also

Time complexity: O(n^2) because of the list result that stores unique words, which is searched for every word in the input sentence. 
Auxiliary space: O(n) because we are storing unique words in the result list.