datagrit › Guides › Website Tech Stack Lookup - Technology Detector

How to detect the tech stack and email provider of a list of websites

Detect CMS, ecommerce platform, analytics, CDN and hosting of many domains, plus MX mail provider, SPF and DMARC, from Python or cURL.

Published 2026-10-01 · uses the Website Tech Stack Lookup - Technology Detector Actor

"Find me every Shopify store in this list of 5,000 domains" is a typical request in B2B sales. So are its cousins: which prospects already use a competitor's product, which ones run Magento 2 and might need a migration, which domains still have no DMARC policy, which companies receive email through Microsoft 365 rather than Google Workspace.

Agencies use the answer to qualify leads, SaaS companies to size a market, deliverability consultants to audit a portfolio, and investors to understand a target's stack before a call. The information is public: a website's HTML, headers and DNS records reveal most of it. The work is in reading it reliably at scale.

This post shows what a technology detector actually looks at, where homemade versions go wrong, and how to run one over a list of domains through the Apify API.

Where the data comes from

There is no registry of "which site uses what". Detection is inference from two public sources:

1. The web response. Headers (Server, X-Powered-By), cookie names, meta generator tags, script URLs and HTML snippets. A /wp-content/plugins/woocommerce/ path or a wc_add_to_cart_params variable means WooCommerce.
2. Public DNS. MX records show who receives mail for the domain, the SPF record lists the services allowed to send on its behalf (SendGrid, Mailgun, Amazon SES, HubSpot and others), the _dmarc record holds the DMARC policy, NS records give the DNS provider, CNAME, A and PTR records hint at hosting and CDN, and TXT verification records show services the domain was verified with.

Writing this yourself sounds like a few regexes. In practice:

• Bare domains do not always answer at the apex. The Website Tech Stack Lookup Actor on Apify tries https://domain/, then https://www.domain/, then http://domain/, and follows up to 8 redirects and one meta refresh.
• Bot challenge pages look like websites. Cloudflare, Akamai, Fastly or DataDome may return a challenge page to data center addresses. Fingerprinting that page describes the challenge, not the site. The Actor recognises challenge pages and returns found: false with the reason blocked-by-bot-protection instead.
• Fingerprints produce false positives. In a hand check of 175 page and header detections on 23 well-known sites on 1 October 2026, the first pass found 2 false positives (a Jetpack statistics script read as WordPress.com hosting, and a Vue.js signal that also appears in Inertia/React pages). Both were fixed, after which all 175 were correct. The same sample was used to tune fingerprints, so it is a sanity check, not a benchmark.
• Implied technologies. WordPress implies PHP; Next.js implies React. These are marked as implied, with a lower confidence.
• Registrable domains. DNS checks must run on bbc.co.uk, not www.bbc.co.uk and not co.uk. The Actor uses a public-suffix-aware domain for DNS.

The Actor matches 847 technology fingerprints in 62 categories, each detection with a version when exposed, a confidence from 0 to 100 and up to three pieces of evidence.

Step by step

1. Define the input

A list of prospects, keeping only those that run one of three ecommerce platforms:

{
  "domains": [
    "porterandyork.com",
    "coxandcox.co.uk",
    "allbirds.com",
    "https://example.com/pricing"
  ],
  "categories": [],
  "requireTechnologies": ["Shopify", "WooCommerce", "Magento"],
  "followRedirects": true,
  "includeDns": true,
  "maxConcurrency": 10,
  "requestTimeoutSecs": 20
}

domains accepts bare domains, hostnames or full URLs; a full URL is fetched exactly as given. Duplicates (with or without www, http or https) are merged. requireTechnologies is the lead filter: sites that do not use any of the listed technologies are skipped and not charged. categories only trims the technologies list; the summary columns are always complete.

2. Run it from Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

with open("domains.txt") as f:
    domains = [line.strip() for line in f if line.strip()]

run = client.actor("datagrit/website-tech-stack-lookup").call(run_input={
    "domains": domains,
    "requireTechnologies": ["Shopify", "WooCommerce"],
    "maxConcurrency": 20,
})

for site in client.dataset(run["defaultDatasetId"]).iterate_items():
    if site.get("found"):
        print(site["domain"], site["ecommerce"], site["mailProvider"], site["dmarcPolicy"])
    else:
        print(site.get("input"), "skipped:", site.get("errorReason"))

3. Or with cURL

curl -X POST \
  "https://api.apify.com/v2/acts/datagrit~website-tech-stack-lookup/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"domains": ["shopify.com", "wordpress.org", "bbc.co.uk"]}'

Each site costs one page request plus a few DNS lookups. In a local test on 1 October 2026, 12 well-known sites took 5.9 seconds including DNS. The Actor analyses 10 sites at a time by default (up to 50) and never more than 2 URLs of the same host at once.

What you get

One row per site, with ready-made summary columns plus the full evidence list:

Other columns: analytics, tagManager, webServer, jsFramework, programmingLanguage, marketingAutomation, liveChat, payment, dnsProvider, emailServices (sending services from SPF), mxRecords, hasSpf, hasDmarc, title, language, finalUrl and statusCode. Failed sites carry errorReason, for example dns-not-found, timeout or blocked-by-bot-protection, and are not charged.

Recipes

Lead list for a Shopify app

Run your prospect domains monthly with requireTechnologies: ["Shopify"]. Only Shopify stores are returned and charged; push them into your CRM with marketingAutomation and payment as segmentation fields. Stacks change slowly, so a monthly run is usually enough for a lead list.

Email deliverability audit

import pandas as pd

rows = [s for s in client.dataset(run["defaultDatasetId"]).iterate_items() if s.get("found")]
df = pd.DataFrame(rows)

audit = df[["domain", "mailProvider", "hasSpf", "hasDmarc", "dmarcPolicy", "dnsChecked"]]
at_risk = audit[(audit["dnsChecked"] == True) & ((audit["hasDmarc"] == False) | (audit["dmarcPolicy"] == "none"))]
print(at_risk.sort_values("domain").to_string(index=False))

Filtering on dnsChecked matters: when a DNS lookup failed, a missing value means "unknown", not "absent".

Market share in a segment

Run a list of competitors' customers or a category of sites, then count cms, ecommerce or cdn values with df["ecommerce"].value_counts(). For per-technology analysis, explode the technologies column (or use the Actor's "Technologies with evidence" dataset view, which already has one row per technology).

Limitations

• One page per site. Only the home page or the URL you give is analysed (first 2 MB of HTML). Tools loaded by JavaScript after rendering, or only on inner pages, are not visible.
• Bot protection. Strictly protected sites may return a challenge page to data center addresses; they get a free found: false row. A residential proxy in proxyConfiguration often gets through. DNS lookups do not use the proxy.
• TXT verifications are historical signals. A verification record shows the domain was verified with a service at some point, not that it still uses it.
• Fingerprints cover 847 technologies, not every tool in existence; mailProvider: "Other" means MX records exist but point to a provider the Actor does not recognise.
• Inference, not certainty. Each detection carries a confidence and evidence so you can judge borderline cases yourself.

The Actor reads only what a browser or DNS client receives from public websites and public DNS. It does not log in, use cookies or solve captchas. If you use the results for outreach, anti-spam rules still apply.

Wrapping up

Website stack plus mail and DNS data in one row is usually what lead qualification needs; pricing is pay per analysed website, the Actor is on Apify, and the full list of fields and categories is in the documentation.

More guides