How to scrape a website for emails — one page, or a list of 200

"Scrape a website for emails" hides three different jobs, and most of the frustration around this comes from picking a tool built for one of them and pointing it at another.

• One page. You are looking at a contact page and want what is on it. Seconds.
• One whole site. You want everything a company has published anywhere on its domain. Usually the wrong job — see below.
• A list of sites. You have a directory, an exhibitor list, a map export, a CRM column — dozens or hundreds of companies. This is the job that eats afternoons, and most of this guide is about it.

Job 1: a single page

Nothing clever required. Any page-reading extension does this, including ours, and our other guide covers where the address usually hides and how to search for it when it does not appear at all — how to find a company's email address. Come back here when you have more than one company to deal with.

Job 2: an entire site (and why you probably shouldn't)

Crawling a whole domain sounds thorough and is mostly waste. A 400-page company site publishes contact details on two or three of those pages. The other 397 are blog posts, product pages and legal boilerplate, and you pay for every one of them — in time, in requests against someone else's server, and in the odds of being rate-limited or blocked partway through.

The version of this that actually works is not a crawl. It is checking the handful of pages that carry contact details by convention: /contact, /contact-us, /about, /team, and /impressum for anything German, Austrian or Swiss. Four or five requests instead of four hundred, and a higher hit rate. Keep that shape in mind — it is exactly what a good bulk tool should be doing per site, and section 5 comes back to it.

Job 3: a list of sites — where the list comes from

Nobody starts with a clean list of URLs. In practice they come from one of these, and each has its own quirk:

Whatever the source, the shape is the same: a page full of links to other companies. Getting those links out of the page and into a queue is the first step, and it is worth doing mechanically rather than by copying them one at a time.

The three kinds of bulk tool

Online extractors. You paste a list of URLs into a web form and a server fetches each one. Fast, usually has a free tier, and it does not tie up your machine. The limitation is structural: the server receives raw HTML. Anything a page assembles in the browser — an address rendered by a script, a contact block that loads after the rest of the page — is not in that HTML, so the tool reports nothing and you have no way to tell that from a site that genuinely has no address. These tools also arrive as an unknown datacentre IP with no session, which is what bot protection is built to stop.

Desktop crawlers. The oldest category, and it shows — most of the products ranking for "bulk email extractor" today are Windows applications with a licence key. They do work. They also tend to default to crawling entire sites (see job 2), and they run outside your browser, so anything that needs a logged-in session is out of reach.

Browser queues. An extension opens each URL in your own browser, lets the page render completely, reads it, and moves on. You get the page as a human would see it, from your own IP, in your own session. The trade-off is honest: it is slower than a server farm and it occupies one machine. For lists in the tens-to-hundreds — which is what most B2B prospecting actually looks like — that trade is usually worth it. Mail Extractor is this kind.

What actually goes wrong at scale

Everything below is fine on one site and becomes the whole problem on two hundred. This is the part worth reading before you pick anything.

1. A third of homepages have no address on them

This is the single biggest source of silent loss. The company has an address; it is on the contact page or the imprint, one click from where your list pointed. A tool that reads only the URL it was given records a blank, and a blank is indistinguishable from "no address exists". You do not find out until you look at the results and wonder why the yield is so low.

The fix is for the tool to notice the blank and follow the contact link itself — the /contact, /about, /impressum shape from job 2, bounded to a few candidates per site so it stays cheap. The sample run further down measures exactly what this is worth on six companies: 9 addresses without it, 14 with it, and two of the six companies go from nothing at all to five addresses between them.

2. Dead URLs quietly eat the budget

Old lists are full of domains that no longer resolve, and sites that return a 404 page which looks like a normal page. If the tool cannot tell a real page from a 404 that renders the site header, it burns its per-site allowance on nothing — and on a site where the real contact page was the fifth thing it would have tried, it never gets there. Checking the status code before opening the page is unglamorous and it is the difference between finding the imprint and not.

3. Speed is not free

Two hundred requests as fast as your connection allows looks like an attack, because from the other side it is indistinguishable from one. You get rate-limited, then challenged, then blocked — and the block lands mid-run, so you cannot tell which sites genuinely had nothing. A few seconds between pages costs you an hour you were going to spend elsewhere anyway. Remember that with contact-page discovery on, one start URL can be several requests.

4. The same address, forty times

A group with regional sites publishes the same info@ on every one of them. Without deduplication across the whole run — not just within a page — a 200-site list produces a spreadsheet that looks impressive and contains sixty distinct addresses. Deduplicate on the address, and keep a note of which site each one came from, because that is the context you need when you write.

5. The list itself is dirty

Before starting a run, spend five minutes on the URLs: strip tracking parameters (?utm_source=…), collapse http/https and www/non-www variants of the same domain, drop anything pointing at a social profile rather than a website, and deduplicate on the registrable domain rather than the full URL. Ten duplicate rows removed here is ten sites you do not visit and ten seconds of someone else's server you do not use.

6. Long runs get interrupted

A run of two hundred sites takes long enough that something will happen to it: the laptop sleeps, a page hangs and never fires its load event, the browser discards a background tab under memory pressure. What matters is not that this is prevented — it cannot be — but that the run notices, records where it stopped, and can be resumed without redoing the first hundred and eighty. Ask about that before you trust a tool with a long list.

Try it on a sample directory

We built a fictional trade directory to make all of the above concrete. Six member companies, each with its own page, and the whole thing behaves like the real situation — including two companies that publish nothing on their landing page.

What yield to expect

A demo with six curated companies gives a flattering ratio. On a real list, expect a meaningful share of sites to give you nothing at all: some publish only a web form, some hide the address in an image, some genuinely have no email anywhere. A list of two hundred small-business websites returning usable addresses for somewhere around half of them is an ordinary result, not a failure — and it is still two hundred contact pages you did not open by hand. Judge a tool on how many of the remainder it explains rather than silently drops.

One thing before you send

Collecting is not the same as being allowed to email. In the EEA and the UK a work address that identifies a person is personal data under the GDPR and needs a lawful basis; in the US, CAN-SPAM requires accurate headers, a postal address and a working opt-out in every commercial message; Canada's CASL wants consent up front. The first guide has a longer note on this. Scale makes it matter more, not less — a bad list sent to two hundred people damages the deliverability of everything you send afterwards.