Home / Guides / Python Requests

How to use a proxy with Python requests

Q: Does requests support SOCKS proxies?

Yes, with pip install requests[socks] , but you don’t need it here: Storm gateways are HTTP(S) proxies, not SOCKS, and work with plain requests.

Pass a proxies dict to every call (or to a Session) with your gateway as http://IP:PORT for both http and https. That’s the whole setup. The rest of this guide is about doing it at scale without errors.

Updated October 2026Code & scraping frameworks

Python’s requests library is still the default way to fetch pages, call APIs and feed scrapers. It has proxy support built in: no extra package, no plugin. But most proxy problems people hit with it aren’t about the syntax. They come from three things: the proxy URL scheme for HTTPS, connection reuse quietly pinning you to one IP, and running more parallel requests than your plan allows.

This guide covers the basic setup first, then sessions, rotation, concurrency, environment variables and the errors you’ll actually see, with the exact fix for each. Every example works with Storm Proxies gateways, and with most other HTTP proxies too.

Which Storm plan fits

For scraping with requests, start with rotating proxies: a new IP on every connection through the Main gateway, and you pay per thread, not per GB, so large crawls cost the same as small ones. Need the same IP for a login session? Use the 3- or 15-minute gateway, a residential port (one IP for 5 minutes) or a private dedicated proxy (one IP for as long as you keep it).

Rotating proxies (from $14/mo): 700,000+ IPs behind fixed gateway IP:PORTs. New IP on every request, or every 3 or 15 minutes. USA, EU, USA+EU or Worldwide. Unlimited bandwidth on every plan.

Get 40 threads for $39/mo See all rotating proxies plans

Before you start

Log in to the member area and copy your gateway IP:PORTs. They never change; the rotation happens on our side.
Add the public IP of the computer or server that will run your tool under Authorized IPs, click Save, and allow up to 15 minutes before testing. Rotating and residential proxies use IP authentication, so there is no username or password.
Dedicated proxies work with either IP authentication or a username and password. Use user:pass if your IP changes or the tool runs on several machines.
Count your threads: the tool’s total open connections must stay within your plan (for example 40 threads on the 40-thread plan).

On this page

Set up a proxy in requests, step by step
Copy-paste examples
Why your IP doesn’t change on every request
How many threads to run
Timeouts, retries and backoff
Headers matter as much as the IP
Where to run the script
Common errors and fixes
FAQ

Set up a proxy in requests, step by step

Install requests
Run pip install requests. Proxy support for HTTP and HTTPS is part of the core library. You only need an extra package (requests[socks]) for SOCKS proxies, and Storm gateways are HTTP(S), so you don’t.
Copy a gateway from the member area
Pick a gateway IP:PORT. For rotating plans, the Main gateway gives a new IP per connection, the 3-minute and 15-minute gateways keep an IP for that long. For residential, each port is one IP that changes every 5 minutes.
Build the proxies dict
Map both schemes to the same gateway, and start the proxy URL with http:// even for the https key. The scheme in the URL is how requests talks to the proxy, not to the website. HTTPS sites still stay encrypted end to end through a CONNECT tunnel.
Make a test request
Call https://httpbin.org/ip (or any “what is my IP” endpoint) with proxies=proxies and a timeout. The IP in the response should be a proxy IP, not yours. If it’s your own IP, the proxies dict isn’t being passed.
Move to a Session for real work
A requests.Session() keeps cookies and headers and lets you set session.proxies once. It also reuses connections, which is faster but matters for rotation (see below).
Add concurrency within your thread limit
Use a ThreadPoolExecutor with max_workers at or below your plan’s threads. A 40-thread plan means at most 40 requests open at the same moment, across every script and machine using that plan.

Copy-paste examples

Replace GATEWAY_IP:PORT with a gateway from your member area. Rotating and residential gateways need no username or password once your IP is authorized.

Basic request through a rotating gateway

import requests

PROXY = "http://GATEWAY_IP:PORT"          # http:// even for https sites
proxies = {"http": PROXY, "https": PROXY}

r = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=20)
print(r.json())                            # shows the proxy IP, not yours

Dedicated proxy with username and password

import requests
from urllib.parse import quote

user, pwd = "USERNAME", quote("PASSWORD", safe="")   # quote() handles @ : / in passwords
PROXY = f"http://{user}:{pwd}@PROXY_IP:PORT"
proxies = {"http": PROXY, "https": PROXY}

print(requests.get("https://httpbin.org/ip", proxies=proxies, timeout=20).json())

Session + 40 parallel requests with retries

import requests
from concurrent.futures import ThreadPoolExecutor
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

PROXY = "http://GATEWAY_IP:PORT"
THREADS = 40                                # never above your plan's thread count

def make_session():
    s = requests.Session()
    s.proxies = {"http": PROXY, "https": PROXY}
    s.headers["User-Agent"] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64)"
    retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504])
    s.mount("http://", HTTPAdapter(max_retries=retry, pool_maxsize=THREADS))
    s.mount("https://", HTTPAdapter(max_retries=retry, pool_maxsize=THREADS))
    return s

session = make_session()

def fetch(url):
    try:
        r = session.get(url, timeout=(10, 30))   # (connect, read) seconds
        return url, r.status_code
    except requests.RequestException as e:
        return url, repr(e)

urls = [f"https://httpbin.org/anything/{i}" for i in range(200)]
with ThreadPoolExecutor(max_workers=THREADS) as pool:
    for url, result in pool.map(fetch, urls):
        print(url, result)

Force a fresh connection (new IP) on every request

import requests

PROXY = "http://GATEWAY_IP:PORT"            # Main gateway: new IP per connection
proxies = {"http": PROXY, "https": PROXY}

for _ in range(5):
    # no Session = no connection reuse, so each call opens a new tunnel
    r = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=20,
                     headers={"Connection": "close"})
    print(r.json()["origin"])

Environment variables instead of code

# macOS / Linux
export HTTP_PROXY="http://GATEWAY_IP:PORT"
export HTTPS_PROXY="http://GATEWAY_IP:PORT"
export NO_PROXY="localhost,127.0.0.1"
python my_scraper.py        # requests picks these up automatically

# Windows PowerShell
$env:HTTP_PROXY="http://GATEWAY_IP:PORT"
$env:HTTPS_PROXY="http://GATEWAY_IP:PORT"

Why your IP doesn’t change on every request

This is the most common surprise. You connect to a rotating gateway, send ten requests in a loop, and see the same IP ten times. Nothing is broken: requests reuses connections.

A Session (and the connection pool behind it) keeps the TCP connection to the proxy open between requests. For HTTPS sites, requests opens one CONNECT tunnel and sends request after request through it. A rotating gateway picks the exit IP when a connection or tunnel is opened, so every request inside that tunnel leaves from the same IP.

That’s good for logins and multi-step flows, where a changing IP would look suspicious. When you want a new IP per request, open a new connection each time: call requests.get() without a session, send Connection: close, or create a fresh Session per task. If you need the same IP on purpose for several minutes, use the 3- or 15-minute gateway instead of fighting the Main one.

How many threads to run

Your plan is sold in threads (rotating), ports (residential) or proxies (dedicated), and the thread limit counts open connections at the same moment, not requests per day. Some rules of thumb that keep scrapers stable:

Rotating plans: set max_workers to your thread count or a little below it. If two scripts share the plan, split the threads between them.
Search engines: keep scraping of Google and other search engines to at most 25% of your threads, through the Main gateway. On a 40-thread plan that’s 10 workers.
Residential ports: each port allows up to 50 threads, and all of them share that port’s current IP. For 10 different IPs at the same time, use 10 ports.
Dedicated proxies: up to 100 threads, always from the same static IP. Spread work across several proxies when one site starts to rate-limit you.

Going over the limit doesn’t speed anything up. Extra connections fail, and you’ll see connection errors that look like a proxy outage.

Timeouts, retries and backoff

Always pass a timeout. Without one, a single stalled connection can hang a worker forever, and with 40 workers that adds up to a scraper that slowly stops. A (connect, read) tuple such as (10, 30) fails fast when the proxy can’t be reached but gives slow pages time to load.

Retries belong at the adapter level (urllib3.Retry) so they happen with backoff: wait 1, 2, then 4 seconds. Retrying instantly on a 429 just asks the site to block you faster. On the Main rotating gateway, a retry that opens a new connection also goes out from a new IP, which is often all a blocked request needs.

Headers matter as much as the IP

A clean IP with the default python-requests/2.x User-Agent still looks like a script. Send a real browser User-Agent, an Accept-Language header, and keep cookies within a session for sites that set them. For pages that build their content with JavaScript, requests only sees the empty shell. Switch those to a headless browser such as Playwright with the same gateway.

Where to run the script

Rotating and residential plans are authorized by IP: the gateway accepts connections from the IPs saved in your member area. That works perfectly on your own computer or on a VPS with a fixed IP. It doesn’t work from places where the outgoing IP changes on every run, such as serverless functions, some CI runners or notebooks hosted in the cloud.

For those, run the scraper on a small server with a static IP and authorize that IP, or use a private dedicated proxy with username and password. If your home IP changes, you can authorize a free dynamic DNS hostname (No-IP or similar) instead of a raw IP.

Common errors and fixes

ProxyError with ConnectionResetError or RemoteDisconnected (“Connection aborted”)On rotating and residential plans, your current IP isn’t authorized yet, or it was saved less than 15 minutes ago. The gateway accepts the connection and resets it. Check your public IP, compare it with Authorized IPs in the member area, and wait up to 15 minutes after saving.

ProxyError: Cannot connect to proxy / Connection refusedNothing is listening at that address: the gateway IP or port has a typo, or a firewall blocks the port. Copy the gateway again from the member area. This isn’t an authorization problem.

407 Proxy Authentication RequiredThe proxy didn’t accept your credentials. On dedicated proxies with user:pass, check the username and password and URL-encode special characters with quote(). Storm rotating and residential gateways don’t answer an unauthorized IP with a 407 (you get a reset instead), so a 407 there comes from another proxy set in your environment or system.

SSLError or wrong version numberThe proxy URL starts with https://. Change it to http://GATEWAY_IP:PORT for both keys. The site itself still loads over HTTPS.

ConnectTimeout / ReadTimeoutToo many parallel connections for your plan, a slow target site, or no timeout tuning. Lower max_workers below your thread limit and use a (10, 30) timeout.

Same IP on every requestConnection reuse. Use the Main gateway, drop the Session or send Connection: close (see “Why your IP doesn’t change”).

429 Too Many Requests or CAPTCHAsThe site is rate-limiting. Slow down per domain, add backoff, rotate on retry, and send realistic headers. For search engines stay under 25% of your threads.

Your real IP shows upThe proxies dict isn’t reaching the call, or only the http key is set and the URL is https. Set both keys, or set them on the Session.

FAQ

Do I need a username and password for Storm rotating proxies in requests?

No. Rotating and residential plans authorize your IP in the member area, so the proxy URL is just http://GATEWAY_IP:PORT. Private dedicated proxies can use either IP authorization or http://user:pass@IP:PORT.

Why is the proxy URL http:// when I’m scraping https sites?

The scheme describes the connection to the proxy. requests opens a CONNECT tunnel through the HTTP proxy and then runs TLS to the website inside it, so the page is still encrypted. Using https:// for the proxy URL usually causes SSL errors.

How do I get a different IP for every request?

Use the Main rotating gateway and open a new connection per request: no shared Session, or a Connection: close header. Requests that reuse one connection or tunnel keep the same exit IP.

How many requests per second can I send?

The limit is concurrent connections, not requests per second. With a 40-thread plan you can keep 40 requests in flight at once; how many finish per second depends on the sites you call. Bandwidth is unlimited, so large pages don’t cost extra.

Can I use the same proxies in requests and in a browser at the same time?

Yes, as long as the total open connections stay within your plan. A browser tab can open around 10 connections on its own, so leave room for it.

Does requests support SOCKS proxies?

Yes, with pip install requests[socks], but you don’t need it here: Storm gateways are HTTP(S) proxies, not SOCKS, and work with plain requests.

Still have questions? Contact us here. A real person answers.

Tool facts checked against the official documentation (October 2026): Requests docs: Proxies · urllib3 Retry. Storm Proxies facts: our plans page and refund policy.

Unlimited bandwidth. One flat monthly price.

Access is live the moment you pay, and the smallest package of each proxy type has a 24-hour money-back guarantee on your first order.

Get 40 threads for $39/mo Compare all plans