Skip to main
All claybooks
/
RevOps
/
Extract data from paginated databases in seconds

Extract data from paginated databases in seconds

Don’t let pagination stop you from extracting the custom data you need to fuel your marketing and sales. Extract it with Clay instead.

Start building now with
ZenRows
Download the template
Get started

Before we begin

Manually scraping data from large, paginated databases takes forever. And many web scrapers simply can’t navigate pagination easily to do it automatically. 

Good news: this Claybook uses Zenrow and Clay’s custom formulas to scrape paginated databases in seconds. 

How it works: 

  • Create thousands of paginated database URLs using Clay formulas, automatically 
  • Use Zenrows (natively integrated in Clay) to scrape each page, one by one
  • Use scraped data for enrichment, personalization, outbound, and more
Information
Follow the step by steps here in Clay
Download the free template
1

First, copy this template in Clay. 

If you’re already a user, you’ll be directed to your workspace. If not, you’ll be prompted to create a free Clay account.

The interactive demos on the right will walk you through each step of this Claybook. 

2
Step 2:

Add paginated page numbers

How many pages exist in the paginated database you want to scrape? Add them to the first column, giving each page its own row. We’ll use these page numbers in the next step to create dynamic URLs. 

3
Step 3:

Create pagination URLs using Clay formulas 

Next, so you don’t have to manually paste URLs into Clay, we’ll use a formula to add the page number to the database URL string automatically. Simply paste the URL of the database in the formula and insert the page number cell in the proper spot. 

Once finished, run the Zenrow data scrape column. 

4
Step 4:

Edit AI data cleaning prompt to fit your needs

Next, to ensure we’re pulling usable data from often complex databases, use ChatGPT to clean the dataset. 

5
Step 5:

Write data to another table for further enrichment 

Last, write your newly scraped database data to a new table. Why? We want to get all the datapoints into their own columns. The only way to do that is to write to a new table. In the new table, you can use this data for anything, like further enrichment or for outbound messaging. 

Other popular resources

Experts

Find a Clay Expert

Explore our network of Clay experts and agencies.

View experts
Community

Join our slack community

Find help in our slack community, and support channels.

Go to slack
Cohorts

Join a cohort, learn Clay fast!

The faster way to master Clay. Sign in if you're enrolled in a cohort (current or past) or apply to an upcoming one!

Learn more about cohorts