Scrape Craigslist Product data [A Complete Tutorial]

6 min readAug 21, 2020

Introduction:

What if you could magically turn on a switch and get hold of Craigslist product data?

Well, it would be wonderful but unfortunately there’s no magical switch for it.

Instead, what you can have is an amazingly customized Craigslist product scraper that can help you extract product related data such as title, price, image, condition etc, almost magically!

Since there’s no ready-made solution available of this kind, we have put together this tutorial for you regarding how to build a Craigslist product Scraper to scrape the different items mentioned above. We’ll build this web scraper using ProWebScraper.

The Craigslist product scraper will extract the following fields, from the Craigslist product page:

Title
Price
Image
ID
Address
Features

Take a look at the screenshot given below which shows the data which we will be scrapping.

Well, let’s get started.

To make it remarkably easy for you, we have converted it into a 3-step process and easy steps as given below:

Create a Free Account on ProWebScraper
Create and Configure a Scraper
Download your Data

Step 1: Create a Free Account on ProWebScraper

Yes, you can create an absolutely free account on ProWebScraper! All you need to do is go to ProWebScraper.com and carry out the simple process of creating a free account.

As it’s a free account, you will not need to pay anything for scraping data up to 1000 pages. Moreover, you will not need to enter any of your payment related information such as Credit or Debit card numbers etc.

After you create your free account and log in to ProWebScraper, you’ll be taken to your ProWebScraper scraper tab. Here’s what it’ll look like:

Step 2: Create & Configure a Scraper

Now we come to the part where you actually start setting up the scraper.
To configure the customized Craigslist scraper, all you need to do is enter the URL of the Craigslist product page.
As soon as the specific page is loaded on ProWebScraper, we will make use of ProWebScraper’s point-and-click selector feature and easily select data.
Let’s take a look at it how it’s done step by step:

(2.1) Enter Craigslist Product URL and click on the Go to load website.

Once this is done, it’s easy to get started.
On ProWebScraper Scraper tab, you can simply copy and paste the URL of the Craigslist product page, such as https://newyork.craigslist.org/mnh/apa/d/new-york-w48th-10thbright-rent/7152145269.html, and click on “Go” to load the page.

(2.2) Click on the title you want to scrape.

As soon as the page is loaded, you can start configuring the scraper in the following way:
To select the data you need, just click on it.
You simply need to click on the title of the product to scrape it.

NOTE: If you want to rename data column either double click on the column name or select rename from the list of data columns settings.

(2.3) To select more data points.

To continue to scrape more data points like price, images, address, you can click on “Add column”, and then click that data point on page to select that data point.

(2.4) To scrape an image URLs and get it in full size by using Regex.

Once you select images using point and click, you will get all images in a small size.
However, if you want all the images in full size, you need to apply regex to resize the image.
It’s not as complicated as you think. Here’s a sample of how you can get images in full size.
To apply regex, Go to column settings > Set Regular Expressions
Then match the size number like: 50x50c
And replace it with: 600x450
You can get to see the preview of the images in small sizes that we got using point and click functionality, and after applying regex how the same images look.
Image that we get by selection:
https://images.craigslist.org/01010_357R012aKoa_0dI098_50x50c.jpg
After applying regex, images look like this:
https://images.craigslist.org/01010_357R012aKoa_0dI098_600x450.jpg

Note: Using this scraper, you will only get to scrape URLs of images. If you want to download images, ProWebScraper has another functionality. In this functionality, you simply need to purchase image credit to download high-quality images.

(2.5) Save and Run the scraper.

Once you are done with the selection of all the data points, click the “Save” button to save your scraper.
For the moment, you can disable the option “is this listing page?”, name your Scraper, and then click Save and run.

Once the scraper is saved, it will take you to the scraper Dashboard. From there, you can download the results of the run once it is over.

Step 3: Download your Data in CSV or JSON

Once the scraper has successfully completed its run, you will be able to see that the data will be available for download within the Scraper Dashboard.
All you need to do is take a look at the Run History tab. You can get to see the previous runs including the latest one with the results at the top.
As you would be able to see in the screenshot given below, the data will be available for download (1) in JSON or CSV format.
You can see the “Data Preview” of scraped data (2).

Sample Data File (CSV Format) :

https://drive.google.com/file/d/123JF_uQTjPyVdzdhdyV-TYV6wZLvfF9b/view?usp=sharing

Sample Data Preview:

Why Scraping Data From Craigslist?

You might have come across the Craigslist product data and wondered how to use it.
Well, there are various uses of Craigslist product data.
If you are running a jobs board, you would be amazed by the job listings that you can find on Craigslist. With the scope to scrape it at will, you would be thrilled the way you can get the job listings data and benefit from it.
Likewise, if you like housing related data, you can simply feast on Craigslist as it abounds with housing data.
For sale data on Craigslist is yet another area to explore. From bicycles to computers, you can find a wide variety of products and its data on Craigslist.
It depends on your business idea and creativity how you can leverage Craigslist data. These are only sample examples of how you can turn Craigslist data into a business opportunity!

Conclusion:

In the world of business, data is the differentiator today. If that be true, Craigslist is a juicy source of such data. Whether it’s used cars, apartments, jobs, you can get an unlimited supply of such data on Craigslist.
As you can see in this tutorial, you can easily and efficiently scrape Craigslist data using ProWebScraper in a hassle-free manner.
What are you waiting for then?
Go right ahead and leverage this amazingly simple and yet truly efficient Craigslist scraper to scale up your business!
Don’t forget to share your valuable inputs and comments and help us enrich our content!