DEV Community: Data Drip

How to scrape Greenhouse, Lever and Ashby job postings to CSV/JSON

Data Drip — Mon, 02 Nov 2026 13:04:57 +0000

Disclosure: I maintain a hosted version of this as an Apify actor (linked at the end). Everything before that section is free and needs only Python.

If you're building a job board, tracking competitors' hiring, or feeding "hiring signals" into a sales pipeline, the best data source isn't LinkedIn. It's the applicant tracking system (ATS) behind each company's careers page. Three ATSs power a huge share of startup and scale-up careers pages, and all three publish public, unauthenticated JSON:

ATS	Careers URL looks like	Public API
Greenhouse	`boards.greenhouse.io/stripe` or `job-boards.greenhouse.io/stripe`	`GET https://boards-api.greenhouse.io/v1/boards/{slug}/jobs?content=true`
Lever	`jobs.lever.co/palantir`	`GET https://api.lever.co/v0/postings/{slug}?mode=json` (EU: `api.eu.lever.co`)
Ashby	`jobs.ashbyhq.com/ramp`	`GET https://api.ashbyhq.com/posting-api/job-board/{slug}?includeCompensation=true`

No HTML parsing, no headless browser, no proxies. One request returns every open job for that company.

1. Find the slug

The slug is the path segment after the ATS host in the careers URL: stripe, palantir, ramp. If a company's careers page is on its own domain, open any job and look at the "Apply" link or the page source for greenhouse.io, lever.co or ashbyhq.com.

2. What each API returns

Greenhouse: {"jobs": [...], "meta": {"total": N}}. Each job has:

title, location.name, absolute_url
updated_at, first_published
departments
with content=true, the description as HTML-escaped HTML in content

Lever: a plain list. Each posting has:

text (the title)
categories.location / team / commitment
hostedUrl, applyUrl
createdAt (epoch milliseconds)
workplaceType
descriptionPlain
sometimes salaryRange

Ashby: {"jobs": [...]}. Each job has:

title, department, team, location
isRemote, workplaceType
publishedAt, jobUrl
descriptionPlain
with includeCompensation=true, a compensation object with a human-readable compensationTierSummary (e.g. "$211.4K – $290.6K • Offers Equity")

3. A complete script: three ATSs → one CSV

import csv, html, re, requests

def strip_html(s):
    return re.sub(r"\s+", " ", re.sub(r"<[^>]+>", " ", html.unescape(s or ""))).strip()

def greenhouse(slug):
    r = requests.get(f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs",
                     params={"content": "true"}, timeout=30)
    if r.status_code == 404:
        return None
    r.raise_for_status()
    return [{
        "company": slug, "ats": "greenhouse", "title": j["title"],
        "location": (j.get("location") or {}).get("name"),
        "department": ", ".join(d["name"] for d in j.get("departments", [])) or None,
        "posted": j.get("first_published") or j.get("updated_at"),
        "url": j["absolute_url"], "salary": None,
        "description": strip_html(j.get("content")),
    } for j in r.json()["jobs"]]

def lever(slug):
    for host in ("api.lever.co", "api.eu.lever.co"):     # EU-hosted boards live on api.eu.lever.co
        r = requests.get(f"https://{host}/v0/postings/{slug}", params={"mode": "json"}, timeout=30)
        if r.status_code == 404:
            continue
        r.raise_for_status()
        out = []
        for j in r.json():
            sal = j.get("salaryRange") or {}
            out.append({
                "company": slug, "ats": "lever", "title": j["text"],
                "location": (j.get("categories") or {}).get("location"),
                "department": (j.get("categories") or {}).get("team"),
                "posted": j.get("createdAt"),               # epoch milliseconds
                "url": j["hostedUrl"],
                "salary": f'{sal.get("min")}-{sal.get("max")} {sal.get("currency")}' if sal else None,
                "description": j.get("descriptionPlain"),
            })
        return out
    return None

def ashby(slug):
    r = requests.get(f"https://api.ashbyhq.com/posting-api/job-board/{slug}",
                     params={"includeCompensation": "true"}, timeout=30)
    if r.status_code == 404:
        return None
    r.raise_for_status()
    return [{
        "company": slug, "ats": "ashby", "title": j["title"],
        "location": j.get("location"), "department": j.get("department"),
        "posted": j.get("publishedAt"), "url": j["jobUrl"],
        "salary": (j.get("compensation") or {}).get("compensationTierSummary"),
        "description": j.get("descriptionPlain"),
    } for j in r.json()["jobs"]]

FETCHERS = {"greenhouse": greenhouse, "lever": lever, "ashby": ashby}

def jobs_for(company):
    """'ashby:ramp' -> that ATS only; plain 'notion' -> try each ATS in turn."""
    ats, _, slug = company.rpartition(":")
    if ats:
        return FETCHERS[ats](slug) or []
    for fetch in FETCHERS.values():
        try:
            jobs = fetch(company)
        except requests.HTTPError:
            jobs = None
        if jobs:                      # empty boards can exist on several ATSs, keep looking
            return jobs
    return []

rows = []
for company in ["stripe", "lever:palantir", "ashby:ramp", "notion"]:
    rows += [j for j in jobs_for(company) if "engineer" in j["title"].lower()]

with open("jobs.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=list(rows[0]))
    w.writeheader()
    w.writerows(rows)
print(f"{len(rows)} jobs written to jobs.csv")

pip install requests
python ats_jobs.py
# e.g. "433 jobs written to jobs.csv"

On 2 November 2026 this returned 433 open engineering roles across Stripe (Greenhouse), Palantir (Lever), Ramp and Notion (Ashby).

4. Gotchas worth knowing

Slug collisions. The same slug can exist on more than one ATS. Plain ramp returns a 1-job Greenhouse board, while Ramp's real board is on Ashby with 100+ jobs. When you know the ATS, prefix it (ashby:ramp). When you don't, prefer the board with the most jobs, or check that the company name matches.
Lever EU. Some companies are hosted on api.eu.lever.co. A 404 on the US host doesn't mean "no board".
Salary. Ashby and Lever expose it as structured fields when the company publishes it. On Greenhouse it's usually only in the description text, so you need a regex like [$€£]\s?\d{2,3}[kK]\s?[-–]\s?[$€£]?\d{2,3}[kK] or an LLM.
Dates. Greenhouse uses ISO strings, Lever uses epoch ms, Ashby uses ISO with offset. Normalise before filtering by "posted in the last N days".
Descriptions contain personal data sometimes (recruiter emails and phone numbers). Strip them if you store or redistribute.
Only open roles. None of these APIs return closed postings, so if you need history, snapshot daily and diff.

5. Going further: enrichment

Raw postings answer "what's open". Most use cases need more:

Is this senior?
Which skills?
Remote or hybrid?
Visa sponsorship?

The pragmatic approach is a cheap LLM (e.g. gpt-4o-mini) with a strict JSON schema per job.

6. Hosted option: ATS Jobs Scraper API + AI (what I built)

If you don't want to maintain the glue, I packaged all of the above as an Apify actor: ATS Jobs Scraper API: Greenhouse, Lever & Ashby + AI.

companies accepts:
- careers URLs
- prefixed slugs (greenhouse:stripe, lever:palantir, ashby:ramp)
- plain names, which are auto-detected across all three ATSs
Filters:
- keywords
- locations
- remoteOnly
- postedWithinDays
- maxJobsPerCompany / maxJobsTotal
One normalised schema for all three platforms. Salary comes from Ashby/Lever fields, with a fallback to parsing the description text. Emails and phone numbers are stripped from descriptions.
Optional AI enrichment with your own OpenAI or OpenRouter key (default gpt-4o-mini):
- seniority, jobFunction, remotePolicy
- skills[], yearsExperienceMin
- salaryEstimate (only when stated)
- visaSponsorship, languages[]
- a short summary

Example input:

{"companies":["https://boards.greenhouse.io/stripe","lever:palantir","ashby:ramp","notion"],"keywords":["engineer"],"maxJobsPerCompany":200}

From code (replace <APIFY_TOKEN> with your own token):

curl -X POST "https://api.apify.com/v2/acts/data-drip~ats-jobs-scraper/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"companies":["https://boards.greenhouse.io/stripe","lever:palantir","ashby:ramp","notion"],"keywords":["engineer"],"maxJobsPerCompany":200}'

Pricing is pay per result:

$0.0015 per job ($1.50 per 1,000; 10,000 jobs = $15)
$0.002 per AI-enriched job, plus your own LLM tokens

So 1,000 jobs with AI cost $3.50 plus tokens. Apify's free plan includes $5 of monthly usage, which covers ~3,333 jobs without AI.

It runs on a schedule (Apify Schedules), exports JSON/CSV/Excel, and plugs into Google Sheets, Zapier, Make, n8n, Clay ("Run Apify Actor") and AI agents via the Apify MCP server.

Limits:

Greenhouse, Lever and Ashby only: no Workday, LinkedIn or Indeed.
Open postings only.

Wrap-up

Before you reach for a browser scraper, check whether the careers page runs on Greenhouse, Lever or Ashby. If it does, there's a free JSON endpoint.
Normalise early (title, location, department, posted date, URL, salary) and key on the job URL.
Watch for slug collisions and Lever's EU host.

Questions or a company whose board you can't find? Drop it in the comments.

How to monitor EU public tenders with the TED API (Python, free, no API key)

Data Drip — Mon, 02 Nov 2026 13:03:24 +0000

Disclosure: I build data tools on Apify, including the hosted TED actor mentioned at the end. Everything up to that section works with nothing but Python and the free official API.

Every public contract above the EU thresholds has to be published in TED (Tenders Electronic Daily), the EU's official procurement journal. That means thousands of new notices every publishing day, from ministries, hospitals, universities and city councils across 30+ countries, in 24 languages.

If your company sells to the public sector, TED is a free sales pipeline. The problem is reading it. This tutorial shows how to:

Query the official TED Search API v3: anonymous, free, no key.
Filter to open calls for tender in your sector (CPV codes) and countries.
Export a clean CSV with buyer, deadline, estimated value and link.
Turn it into a daily alert.

1. The TED Search API in one request

The endpoint is POST https://api.ted.europa.eu/v3/notices/search. It takes a JSON body with:

query: TED's expert search syntax (the same as the "Expert search" box on ted.europa.eu)
fields: which fields to return
limit: page size (up to 100 per page here)
paginationMode: "ITERATION" lets you walk large result sets with a iterationNextToken

Try it with curl:

curl -s -X POST https://api.ted.europa.eu/v3/notices/search \
  -H "Content-Type: application/json" \
  -d '{
    "query": "classification-cpv=72* AND buyer-country IN (ESP PRT) AND form-type=competition",
    "fields": ["publication-number", "notice-title", "buyer-name", "deadline-receipt-tender-date-lot"],
    "limit": 5,
    "paginationMode": "ITERATION"
  }'

You get back notices, totalNoticeCount and an iterationNextToken for the next page.

The query syntax, briefly

What you want	Expert query
IT services and every sub-code	`classification-cpv=72*`
Several sectors	`(classification-cpv=72* OR classification-cpv=48*)`
Buyers in Spain or Portugal	`buyer-country IN (ESP PRT)`. Alpha-3 codes, not `ES`/`PT`
Calls for tender only (not awards)	`form-type=competition` (awards are `form-type=result`)
Published since a date	`publication-date>=20261026` (YYYYMMDD)
Still open	`deadline-receipt-tender-date-lot>=20261102`
Full-text keyword	`FT~(cybersecurity)`
Newest first	append `SORT BY publication-number DESC`

CPV codes are the EU's product and service taxonomy. A few useful parents:

72000000 IT services
48000000 software
45000000 construction
79000000 business services
33000000 medical equipment

Because trailing zeros mean "everything below", 72* covers all IT-service sub-codes.

Gotchas I hit

Multilingual fields. notice-title comes back as an object keyed by language ({"eng": "...", "spa": "..."}), and buyer-name often only in the original language, as a list. Write a small helper that picks your language and falls back to the first value.
Lot-level fields are arrays. A tender with 5 lots can have 5 deadlines. For alerts, the first (or earliest) is usually what you want.
Values are strings, and many notices don't publish an estimated value at all.
Use ITERATION pagination for anything bigger than a page or two. Plain page numbers are capped for deep result sets.

2. A complete script: open IT tenders in Spain and Portugal → CSV

import csv, datetime, requests

API = "https://api.ted.europa.eu/v3/notices/search"
FIELDS = [
    "publication-number", "notice-title", "buyer-name", "buyer-country",
    "classification-cpv", "publication-date", "deadline-receipt-tender-date-lot",
    "estimated-value-proc", "estimated-value-cur-proc",
]

def first(value, lang="eng"):
    """TED returns multilingual fields as {"eng": "..."} or {"spa": ["..."]}."""
    if isinstance(value, dict):
        v = value.get(lang) or next(iter(value.values()), None)
    else:
        v = value
    if isinstance(v, list):
        v = v[0] if v else None
    return v

def search(query, max_results=500):
    body = {"query": query, "fields": FIELDS, "limit": 100, "paginationMode": "ITERATION"}
    out = []
    while len(out) < max_results:
        r = requests.post(API, json=body, timeout=60)
        r.raise_for_status()
        data = r.json()
        out += data["notices"]
        token = data.get("iterationNextToken")
        if not token or not data["notices"]:
            break
        body["iterationNextToken"] = token
    return out[:max_results]

today = datetime.date.today()
since = (today - datetime.timedelta(days=7)).strftime("%Y%m%d")
query = (
    "classification-cpv=72* "                       # IT services and all child codes
    "AND buyer-country IN (ESP PRT) "                # alpha-3 country codes
    "AND form-type=competition "                     # calls for tender, not awards
    f"AND publication-date>={since} "
    f"AND deadline-receipt-tender-date-lot>={today:%Y%m%d} "  # still open
    "SORT BY publication-number DESC"
)

rows = []
for n in search(query):
    pub = n["publication-number"]
    rows.append({
        "id": pub,
        "title": first(n.get("notice-title")),
        "buyer": first(n.get("buyer-name")),
        "country": first(n.get("buyer-country")),
        "cpv": ";".join(sorted(set(n.get("classification-cpv") or []))),
        "published": (first(n.get("publication-date")) or "")[:10],
        "deadline": (first(n.get("deadline-receipt-tender-date-lot")) or "")[:10],
        "value": first(n.get("estimated-value-proc")),
        "currency": first(n.get("estimated-value-cur-proc")),
        "url": f"https://ted.europa.eu/en/notice/-/detail/{pub}",
    })

with open("tenders.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=list(rows[0]) if rows else ["id"])
    w.writeheader()
    w.writerows(rows)
print(f"{len(rows)} open tenders written to tenders.csv")

Run it:

pip install requests
python ted_tenders.py
# e.g. "83 open tenders written to tenders.csv"

When I ran it on 2 November 2026 it returned 83 open IT-services tenders from Spanish and Portuguese buyers published in the previous 7 days. They included an IT outsourcing contract for a Spanish social-security mutual (Asepeyo) with an estimated value of about EUR 49.4M. Your numbers will differ day to day.

3. Make it a daily alert

The simplest robust setup:

Change since to yesterday (timedelta(days=1)) so each run only pulls new notices.
Schedule it with cron (0 8 * * 1-5 python ted_tenders.py), GitHub Actions on a schedule: trigger, or n8n/Make.
Send the CSV rows to Slack, email or a Google Sheet. Keep the id column as the dedupe key.

4. The hard part isn't fetching, it's reading

Once alerts work, you'll get 20-80 notices a day for a broad sector. Most are irrelevant, many are in languages you don't read, and the details that decide bid/no-bid are buried in the description. That's the part I wanted to automate.

5. Hosted option: TED scraper with AI bid summaries (what I built)

If you'd rather not maintain the script, I packaged it as an Apify actor: EU Public Tenders Scraper (TED) + AI Bid Summaries. It uses the same official API and adds:

Simple inputs:
- keywords
- countries (2- or 3-letter codes)
- cpvCodes (parent codes include children)
- noticeType (competition, result, planning, all)
- onlyOpen (default true)
- publishedFrom/publishedTo (default: last 30 days)
- expertQuery for raw TED syntax
A flat schema:
- title and translated title
- buyer name, country and city
- CPV codes
- publication date and submission deadline
- estimated value and currency
- framework flag, duration
- documents, submission, notice and PDF URLs
Optional AI bid triage with your own OpenAI or OpenRouter key:
- English title and short summary
- sector, key requirements
- red flags
- a 0-100 relevance score with a one-line reason, against a companyProfile you describe in plain words

Example input:

{ "keywords": "software", "countries": ["ES", "PT"], "noticeType": "competition", "onlyOpen": true, "maxResults": 100 }

Or call it from code (replace <APIFY_TOKEN> with your own token):

curl -X POST "https://api.apify.com/v2/acts/data-drip~eu-tenders-ted-ai/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"keywords":"software","countries":["ES","PT"],"noticeType":"competition","onlyOpen":true,"maxResults":100}'

Pricing is pay per result:

$0.003 per notice ($3 per 1,000)
$0.006 per AI tender summary, plus your own LLM tokens

So 1,000 notices with AI cost $9 plus tokens. Apify's free plan includes $5 of monthly usage, enough for ~1,666 notices without AI. You can schedule it with Apify Schedules and push results to Google Sheets, Slack, Zapier, Make or n8n. It also works from AI assistants via the Apify MCP server.

Limitations (same as TED itself):

Only above-threshold notices, so smaller national tenders (e.g. many on Spain's PLACSP) aren't in TED.
No contact persons and no award-winner details.

Wrap-up

The TED API is one of the best free B2B data sources in Europe: official, anonymous and structured.
classification-cpv=XX* + buyer-country IN (...) + form-type=competition + a deadline filter gets you 90% of the way to a useful alert.
Fetching is easy. Triage is where the time goes, so automate that next.

Questions about the query syntax? Ask in the comments. Happy to help debug queries.