NewUnlimited residential proxies: unmetered traffic on a fixed price. From $33.25/day

Use case · Web scraping

Web scraping proxies that keep your crawlers running

Collect public web data at any scale through 55M+ residential IPs and 40,000 datacenter IPs. Rotate on every request, hold a sticky session when you need one and target any country, city or ASN.

  • 55M+Residential IPs
  • 195+Countries
  • UnlimitedConcurrent sessions

The challenge

What gets in the way.

The problems every team in web scraping runs into sooner or later.

  1. 01

    IP bans and rate limits

    Sites count requests per IP. Push a few hundred pages through one address and you get throttled, served 429 errors or blocked outright, often halfway through a job that had to finish tonight.

  2. 02

    CAPTCHAs and bot walls

    Anti-bot systems score every visitor. Datacenter ranges, mismatched locations and identical request patterns raise that score until each page returns a challenge instead of the data you came for.

  3. 03

    Content that changes by location

    Prices, stock, search results and even page layouts depend on where the visitor is. Scrape from the wrong country and your dataset quietly describes a market your customers never see.

The solution

How Proxonym solves it.

Settings you control from the username, with no extra tooling or separate endpoints.

A new IP on every request

Leave out the session parameter and each request exits from a different residential IP. Load spreads across a pool of 55M+ addresses, so per-IP limits stop being your bottleneck.

Sticky sessions on demand

Add -session-{id} to your username to keep one IP for up to 120 minutes on residential. Logins, paginated listings and multi-step flows stay consistent from the first request to the last.

Country, city and ASN targeting

Choose the location in the username, for example -country-de-city-berlin, and collect the exact version of each page that local visitors see. No separate ports or lists to manage.

Concurrency without caps

Every plan allows unlimited concurrent sessions, with a burst limit around 10,000 requests per second per account. Scale threads, workers and headless browsers as far as your own infrastructure allows.

Get started

Up and running in three steps.

  1. 1

    Pick the right pool

    Start on residential for protected targets or datacenter for easy ones. Plans activate within minutes of payment, and you can switch or combine products whenever a target changes.

  2. 2

    Point your scraper at the gateway

    Set gw.proxonym.com:8000 as the proxy with your dashboard credentials. Anything that speaks HTTP or SOCKS5 works, from requests and Scrapy to Playwright, Puppeteer and plain cURL.

  3. 3

    Tune targeting and retries

    Add country or city parameters, keep sessions only where state matters and retry 502 responses on a fresh IP. Track usage per sub-user in the dashboard or through the API.

Best practices for web scraping

  • Send a realistic User-Agent and an Accept-Language header that match the country you target.
  • Use sticky sessions only for multi-step flows and let everything else rotate.
  • Retry a 502 right away on a new IP, but back off on 429 instead of hammering the target.
  • Cache what you already have and fetch only pages that changed, which cuts both GB usage and load on the site.
  • Read robots.txt and the site terms before each new project and keep request rates polite.
Concurrent scraping with a new IP per request and retries
# pip install requests
import random
import time
from concurrent.futures import ThreadPoolExecutor

import requests

# No -session parameter: every request exits from a new US residential IP
PROXY = "http://USERNAME-country-us:PASSWORD@gw.proxonym.com:8000"
PROXIES = {"http": PROXY, "https": PROXY}
RETRY_ON = {429, 502, 504}


def fetch(url, attempts=4):
    for attempt in range(attempts):
        try:
            r = requests.get(url, proxies=PROXIES, timeout=30)
            if r.status_code not in RETRY_ON:
                return url, r.status_code, len(r.content)
        except requests.RequestException:
            pass  # tunnel or network error: the next attempt uses a new IP
        time.sleep(2 ** attempt + random.random())  # exponential backoff
    return url, None, 0


urls = [f"https://books.toscrape.com/catalogue/page-{n}.html" for n in range(1, 51)]

with ThreadPoolExecutor(max_workers=20) as pool:
    for url, status, size in pool.map(fetch, urls):
        print(status, f"{size:>7} bytes", url)

More code examplesIntegration guides

FAQ

Web scraping questions.

Still unsure? The help center answers billing, setup and account questions.

Open the help center
Should I use residential or datacenter proxies for scraping?

Start with datacenter proxies on sites without serious bot protection: they are fast and cost as little as $0.013 per IP on the 40,000 IP plan. Move to residential when you see blocks, CAPTCHAs or empty pages. Residential IPs belong to real consumer connections, so anti-bot systems trust them more. Many teams run both and route each domain to the cheapest pool that still works.

How much traffic will my scraping project use?

It depends on page weight. A plain HTML page is often 50 to 300 KB, while a full browser render with images and scripts can pass 2 MB. Block images, fonts and media in headless browsers, call JSON endpoints instead of full pages where possible and accept compressed responses. The dashboard shows usage per sub-user, so you can measure each job before you scale it.

Can I keep the same IP for several requests?

Yes. Add -session- followed by 1 to 32 letters or digits to your username, for example USERNAME-session-a7k2. The IP then stays the same for 10 minutes by default, and you can set -lifetime- from 1 to 120 minutes on residential. Use a new session ID whenever you want a different IP.

Does it work with Scrapy, Playwright and Puppeteer?

Yes. Proxonym behaves as a standard HTTP, HTTPS or SOCKS5 proxy, so every library or framework that accepts a proxy URL works with it. Our integration guides cover cURL, Python requests, Scrapy, Node.js, Puppeteer, Playwright, Selenium, Go, PHP and Java, each with copy-ready configuration.

Is web scraping legal?

Collecting publicly available data is lawful in many jurisdictions, but the rules vary by country and by type of data. Personal data falls under laws such as the GDPR, copyrighted content has its own limits and a site's terms may restrict automated access. Collect only what you need, stay away from content behind logins, keep the load reasonable and get legal advice for your specific project.

Platforms

Proxies for the sites you collect from.

Platform guides with targeting tips, ready-to-run code and the proxy types that work best on each site.

Related

Related use cases.

Teams often combine these with web scraping on the same account.

Market research

Gather localized content, reviews and trends in any market.

One gateway. Millions of identities.

Pick a plan, pay with crypto and send your first request through Proxonym in minutes.

  • Pay as you go
  • Instant activation
  • Crypto accepted
  • No auto-renewal