How to Install Scrapy on MacOS?

In this article, we will learn how to install Scrapy in Python on MacOS. 

Scrapy is a fast high-level web crawling and web scraping framework used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.

Installation:

Method 1: Using pip to install Scrapy

Follow the below steps to install the SCrapy package on macOS using pip:

Step 1: Install the latest Python3 in MacOS

Step 2: Check if pip3 and python3 are correctly installed.

python3 --version
pip3 --version

Step 3: Upgrade your pip to avoid errors during installation.

pip3 install --upgrade pip

Step 4: Enter the following command to install Scrapy using pip3.

pip3 install Scrapy

Method 2: Using setup.py to install Scrapy 

Follow the below steps to install the Scrapy package on macOS using the setup.py file:

Step 1: Download the latest source package of Scrapy for python3 from here.

curl https://files.pythonhosted.org/packages/e9/dc/b2f41d802f548fba35c2d7c443ab0ab3df358eebd261f4d30d19ad39658a/Scrapy-2.5.0.tar.gz > scrapy.tar.gz

Step 2: Extract the downloaded package using the following command.

tar -xzvf scrapy.tar.gz

Step 3: Go inside the folder and Enter the following command to install the package.

Note: You must have developer tools for XCode MacOS installed in your system

cd Scrapy-2.5.0
python3 setup.py install

Verifying Scrapy installation on macOS:

Make the following import in your python terminal to verify if the installation has been done properly:

import scrapy

If there is any error while importing the module then is not installed properly.

My Personal Notes arrow_drop_up

Save