To extract emails from a list of websites, clean the list to one root domain per business, crawl each site's homepage plus its contact, about, footer-linked and policy pages, read visible text, mailto: links and structured data, filter out junk, and export one row per business with each address, the page it came from and what the inbox is for. Most published addresses sit on those few pages, so a shallow, polite crawl beats a deep one. Verification comes after extraction, not instead of it.
This is the bulk version of the methods in how to find email addresses from a domain, the pillar for this cluster.
Step 1: prepare the domain list
Input quality decides output quality. A fictional raw list:
https://www.Oakline-Joinery.example/contact?utm_source=maps
oakline-joinery.example
http://brightside-dental.example/
www.facebook.com/brightsidedental.example
riverside-bakery.example/menu.pdf
kestrel-plumbing.example
kestrel-plumbing.example/
After cleaning:
oakline-joinery.example
brightside-dental.example
riverside-bakery.example
kestrel-plumbing.example
The rules applied:
| Rule | Why |
|---|---|
Lower-case and strip http://, https://, www. | So the same site is not crawled twice |
| Drop paths, query strings and fragments | Start from the homepage; tracking parameters add nothing |
| Remove duplicates after normalising | Two variants of one domain look like two businesses |
| Drop social, directory and marketplace URLs | The Facebook line is a profile, not the business's site; handle it separately |
| Keep subdomains only when they are the business's site | shop.brand.example may be a separate storefront; blog.platform.example/user is not a business domain |
| Keep a business name or ID next to each domain | So the output joins back to the rest of your record |
Keep the original URL in a column. When a crawl fails, it helps to see what was supplied.
Step 2: choose a crawl depth
| Depth | Pages per site | Finds | Costs |
|---|---|---|---|
| Homepage only | 1 | The footer address on most small sites | Misses contact-page-only addresses |
| Key pages | About 5 to 15: contact, about, footer links, policies, services | Nearly all published business addresses | Moderate time |
| Deep | 20 to 50, following links two levels | Staff, careers, press, blog author and PDF addresses | Slow, noisy, more load on the site |
A practical approach is two passes: key pages for everyone, then a deep crawl only for sites where the first pass found nothing. Most of the time, a site with no address on its contact, about or legal pages is a forms-only site, and a deeper crawl will not change that.
Some sites render their content with JavaScript, so the raw HTML is an empty shell. A crawler needs to detect this and render the page in a browser, otherwise it reports "no emails" for a site whose contact page plainly shows one. The same applies to Cloudflare's email address obfuscation, which replaces addresses in the HTML with [email protected] and decodes them in the browser. See Cloudflare email protection and scraping for the detail.
Step 3: crawl politely
Bulk extraction means many requests to many small business servers, often on cheap shared hosting. Be a good guest:
- Read
robots.txtfirst. RFC 9309 defines how crawlers read the rules a site publishes at/robots.txt. It is not an access control, but it is the owner's statement of what automated clients should fetch. - Respect the site's terms of use where they address automated access.
- Keep concurrency per site low, a few pages at a time, and spread work across many sites rather than hammering one.
- Honour
Retry-Afteron429and503responses, and back off. - Record blocks, don't fight them. A CAPTCHA, a
403or a challenge page is the site saying no. Record the outcome and move on; do not try to get around it. - Record dead and parked domains rather than retrying them in every run.
Step 4: extract, then filter the junk
Extraction reads three places on each page: visible text (including the common name [at] domain [dot] example style), mailto: link targets, and structured data such as a Schema.org email property.
The raw matches will contain junk. Filter before you verify:
| Junk pattern | Example | Filter rule |
|---|---|---|
| Image and asset file names | [email protected], [email protected] | Reject if the "domain" ends in a file extension |
| Placeholder text | [email protected], [email protected], [email protected] | Reject known placeholder local parts and example domains |
| Developer and agency credits | [email protected] in the footer of many unrelated sites | Flag domains that appear on many different businesses' sites |
| Third-party script addresses | Error-reporting or analytics addresses in inline scripts | Ignore addresses found only inside scripts |
| URL-encoded fragments | %[email protected] | Decode, trim and re-test syntax |
| Disposable and adult domains | Throwaway mail services | Reject from reference lists |
| Addresses with no mail host | Typos such as .co.uk domains that do not exist | Reject after an MX lookup |
The free email list cleaner removes most of these patterns from a pasted list.
Step 5: label, deduplicate and keep one row per business
A business site often publishes the same address on every page, plus two or three others. Merge them so each business is one row, and give each address a purpose:
| Purpose | Typical local parts | Outreach use |
|---|---|---|
| General enquiries | hello, info, enquiries, office | Usually suitable |
| Sales and quotes | sales, quotes, orders, trade | Often the best fit |
| Bookings | bookings, reservations, appointments | Depends on your offer |
| Support | support, help, service | Rarely for sales |
| Named person | sam.reid, s.reid | Personal data; needs a lawful basis and privacy information |
| Excluded | privacy, dpo, abuse, noreply, unsubscribe, postmaster, jobs | Do not use for outreach |
Step 6: output columns
A useful export for a fictional run:
| business_name | domain | email_1 | email_1_source | email_1_purpose | email_2 | email_2_source | email_2_purpose | crawl_status |
|---|---|---|---|---|---|---|---|---|
| Oakline Joinery | oakline-joinery.example | [email protected] | /contact/ | General | [email protected] | /contact/ | Sales | ok |
| Brightside Dental | brightside-dental.example | [email protected] | / (footer) | Bookings | [email protected] | /privacy/ | Excluded | ok |
| Riverside Bakery | riverside-bakery.example | forms only | ||||||
| Kestrel Plumbing | kestrel-plumbing.example | parked |
Keep the crawl status. "Forms only" and "parked" are useful facts: the first tells you the right channel is the contact form, the second tells you the business may have closed. Add verification columns after this step; see email verification results explained.
Before you send anything
The list is a research output, not permission. In the UK, PECR's email marketing rule does not apply to corporate subscribers, but you must identify yourself and offer an opt-out; sole traders and some partnerships need consent or the soft opt-in, per the ICO's B2B guidance. Named addresses are personal data under UK GDPR, so you need a lawful basis such as legitimate interests and must give privacy information. Other countries differ: Germany generally needs prior consent for advertising email, including B2B, and the US has CAN-SPAM.
How Map Extract helps
In Map Extract, the website data extractor takes a list of URLs or domains, pasted or imported from a file, with a duplicate policy (skip or reprocess), subdomain filtering and email and domain blacklists. It canonicalises each domain, checks DNS and reachability, then crawls at one of three levels: Fast (the homepage), Balanced (menus, footer and key pages, up to 15) or Deep (two link levels, 20 pages by default, up to 50). Contact, about, services, booking, pricing and policy pages are fetched first, sitemaps are read, and a rendered browser fallback handles script-only sites. Blocked, CAPTCHA, parked and dead sites are recorded and quarantined rather than treated as data.
The email extractor works on that crawl and on the business's public social profiles. It decodes common obfuscation, applies bundled filters for disposable, adult and junk addresses plus your own blocklists, and labels each address by inbox purpose, with privacy, abuse, unsubscribe and no-reply inboxes excluded from outreach. It keeps up to ten emails per business, each with its source, and exports to CSV, XLSX or JSONL.
On etiquette, be aware of how it behaves: it reads robots.txt and skips disallowed pages, although pages matching its legal-page pattern (such as privacy and terms) are still fetched; it keeps page concurrency at 5 by default with a 12 second timeout, honours Retry-After cooldowns, and sends a standard desktop browser user agent rather than a named bot token. Check a site's terms and robots rules yourself if they matter for your use. It does not find addresses a site does not publish, and it does not decide whether you may email them.
Public business and social results the app collects are uploaded automatically to Map Extract's servers while it is licensed and running; the privacy notice explains this.
Related guides
- How to find email addresses from a domain
- Cloudflare email protection and why scrapers miss emails
- Clean a B2B lead list, for the rest of the record.
Key takeaways
- Clean the input to one root domain per business; bad input doubles work and output.
- Key pages find most addresses; go deep only where the first pass found nothing.
- Render script-only pages and decode common obfuscation, or you will miss published addresses.
- Filter file names, placeholders and developer credits before verification.
- One row per business, each address with a source and a purpose.
- Crawl politely: robots.txt, low concurrency, Retry-After, and stop on blocks.
Frequently asked questions
How do I extract emails from a list of websites?
Clean the list into one root domain per line, then crawl each site's homepage, contact, about, footer-linked and policy pages. Read the visible text, mailto links and structured data, decode common obfuscation, filter out junk such as image file names and placeholder addresses, then keep one row per business with each address, the page it came from and what the inbox is for.
How many pages per website should an email extractor crawl?
For most small business sites, the homepage plus the contact, about and policy pages find nearly every published address, which is about five to fifteen pages. Crawling deeper finds staff and blog addresses but costs time, adds noise and puts more load on the site. Start shallow, then rerun only the sites where nothing was found with a deeper crawl.
Why does my bulk email extractor return junk like [email protected]?
Simple extractors match any text shaped like an address, which includes retina image file names, placeholder text such as [email protected], version strings and addresses inside third-party scripts. A good filter rejects strings ending in file extensions, known placeholder and example domains, and addresses whose domain has no mail server, before anything is verified.
Do I need to respect robots.txt when extracting emails?
robots.txt is not an access control, but it is the site owner's published statement of what automated clients should fetch, and RFC 9309 expects crawlers to honour it. Respecting it, along with the site's terms, low request rates and Retry-After headers, is part of collecting data responsibly. Data protection law applies to what you collect whatever robots.txt says.
Sources
Written by the Map Extract editorial team. Found an error? Email [email protected] and we will check and correct it. How we research guides: editorial policy.