How to Use SOCKS5 Proxies with BeautifulSoup

BeautifulSoup is the simplest way to parse HTML in Python. It doesn't handle network requests itself — that's what requests or urllib3 is for. But once you know how to configure SOCKS5 on the transport layer, BeautifulSoup scrapers become just as powerful as any framework.

Basic Setup

First, install requests with SOCKS support:

pip install requests[socks]

Then configure the proxy:

import requests
from bs4 import BeautifulSoup

proxies = {
    'http': 'socks5h://your_username:your_password@gw.snowpad.io:9999',
    'https': 'socks5h://your_username:your_password@gw.snowpad.io:9999'
}

response = requests.get('https://httpbin.org/ip', proxies=proxies)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.text)

Notice I use socks5h:// not socks5://. The h is critical — it tells PySocks to resolve DNS through the proxy, not your local network. Without it, your ISP still sees which domains you're visiting.

Rotating Proxies

For larger scraping jobs, rotate through multiple Snowpad accounts:

import itertools
import random

proxy_list = [
    'socks5h://user1:pass1@gw.snowpad.io:9999',
    'socks5h://user2:pass2@gw.snowpad.io:9999',
    'socks5h://user3:pass3@gw.snowpad.io:9999',
]

proxy_cycle = itertools.cycle(proxy_list)

def fetch_with_proxy(url):
    proxy = next(proxy_cycle)
    return requests.get(url, proxies={
        'http': proxy,
        'https': proxy
    })

Adding User-Agent Headers

Snowpad's mobile IPs already look like real devices, but adding a mobile User-Agent makes the fingerprint even more convincing:

headers = {
    'User-Agent': 'Mozilla/5.0 (Linux; Android 14; SM-S928B) AppleWebKit/537.36'
}

response = requests.get(url, proxies=proxies, headers=headers)

When to Use BeautifulSoup + Snowpad

This combo is perfect for small to medium scraping tasks — price monitoring, content aggregation, lead generation. You don't need Scrapy's complexity or Playwright's browser overhead. Just fetch, parse, and extract.

For larger projects, check out the Scrapy proxy setup guide. For JavaScript-heavy sites, see the Playwright and Puppeteer guide.

For more on why proxy choice matters, read the SOCKS5 vs HTTP proxy comparison and what is a rotating proxy.