Extract Author’s information from Geeksforgeeks article using Python

In this article, we are going to write a python script to extract author information from GeeksforGeeks article.

Module needed

• bs4: Beautiful Soup(bs4) is a Python library for pulling data out of HTML and XML files. This module does not come built-in with Python. To install this type the below command in the terminal.

pip install bs4

• requests: Requests allows you to send HTTP/1.1 requests extremely easily. This module also does not comes built-in with Python. To install this type the below command in the terminal.

pip install requests

Approach:

• Import module
• Make requests instance and pass into URL
• Initialize the article Title
• Pass URL into a getdata()
• Scrape the data with the help of requests and Beautiful Soup
• Find the required details and filter them.

Stepwise execution of scripts:

Step 1: Import all dependence

Python

# import module
import requests
from bs4 import BeautifulSoup

Step 2: Create a URL get function 

Python3

# link for extract html data
# Making a GET request

def getdata(url):
    r=requests.get(url)
    return r.text

Step 3: Now merge the Article name into URL and pass the URL into the getdata() function and Convert that data into HTML code 

Python3

# input article by geek
article = "optparse-module-in-python"

# url
url = "https://www.geeksforgeeks.org/"+article

# pass the url
# into getdata function
htmldata=getdata(url)
soup = BeautifulSoup(htmldata, 'html.parser')

# display html code
print(soup)

Output:

Step 4: Traverse the author’s name from the HTML document. 

Python

# traverse author name
for i in soup.find('div', class_="author_handle"):
    Author = i.get_text()
print(Author)

Output:

kumar_satyam

Step 5: Now create a URL with author-name and get HTML code. 

Python3

# now get author information
# with author name
profile ='https://auth.geeksforgeeks.org/user/'+Author+'/profile'

# pass the url
# into getdata function
htmldata=getdata(profile)
soup = BeautifulSoup(htmldata, 'html.parser')

Step 6: Traverse the author’s information.

Python3

# traverse information of author
name = soup.find(
    'div', class_='mdl-cell mdl-cell--9-col mdl-cell--12-col-phone textBold medText').get_text()


author_info = []
for item in soup.find_all('div', class_='mdl-cell mdl-cell--9-col mdl-cell--12-col-phone textBold'):
    author_info.append(item.get_text())

print("Author name :")
print(name)
print("Author information  :")
print(author_info)

Output:

Author name : Satyam Kumar 
Author information  : 
[‘LNMI patna’, ‘\nhttps://www.linkedin.com/in/satyam-kumar-174273101/’] 

Complete code:

Python3

# import module
import requests
from bs4 import BeautifulSoup

# link for extract html data
# Making a GET request


def getdata(url):
    r = requests.get(url)
    return r.text


# input article by geek
article = "optparse-module-in-python"

# url
url = "https://www.geeksforgeeks.org/"+article


# pass the url
# into getdata function
htmldata = getdata(url)
soup = BeautifulSoup(htmldata, 'html.parser')

# traverse author name
for i in soup.find('div', class_="author_handle"):
    Author = i.get_text()

# now get author information
# with author name
profile = 'https://auth.geeksforgeeks.org/user/'+Author+'/profile'

# pass the url
# into getdata function
htmldata = getdata(profile)
soup = BeautifulSoup(htmldata, 'html.parser')

# traverse information of author
name = soup.find(
    'div', class_='mdl-cell mdl-cell--9-col mdl-cell--12-col-phone textBold medText').get_text()


author_info = []
for item in soup.find_all('div', class_='mdl-cell mdl-cell--9-col mdl-cell--12-col-phone textBold'):
    author_info.append(item.get_text())

print("Author name :", name)
print("Author information  :")
print(author_info)

Output:

Author name : Satyam Kumar 
Author information  : 
[‘LNMI patna’, ‘\nhttps://www.linkedin.com/in/satyam-kumar-174273101/’]