Monitor Google News for your brand, competitors or any keyword. Give it a list of keywords, and each run writes the matching Google News articles — title, source, date, snippet and link — to a CSV and a JSON Feed (the JSON equivalent of an RSS feed). It remembers what it has already seen, so a scheduled run with --only-new gives you just the new coverage. Python with one dependency (requests).
Uses the CrawlerData API — you need an API key (free trial credits available at https://app.crawlerdata.com/register). Each keyword costs 1 credit per results page.
- Google News results per keyword:
title, source, date, snippet, link, position published_estimate: Google's relative dates ("3 hours ago", "2 days ago") converted to ISO timestamps (approximate), absolute dates parsed- New-article detection across runs (
seen.json) withis_newandfirst_seencolumns;--only-newwrites only new articles - JSON Feed 1.1 output (
feed.json) for feed readers, Zapier/Make/n8n, or your own pipeline --include/--excludetitle filters to cut noise (e.g. a brand name that is also a common word)- Exact-phrase searches with quotes, 31 countries, up to 10 pages per keyword
- Articles deduplicated across keywords within a run
git clone https://github.com/CrawlerData/google-news-scraper.git
cd google-news-scraper
pip install -r requirements.txt
export CRAWLERDATA_API_KEY="your_api_key" # PowerShell: $env:CRAWLERDATA_API_KEY="your_api_key"
python news.py nvidia
python news.py --input examples/keywords.txt --only-new # run this on a schedule
python news.py "acme corp" --country gb --pages 2 --exclude "acme anvils"Real runs on 13 October 2026 with examples/keywords.txt (nvidia and "electric vehicles" battery):
$ python news.py --input examples/keywords.txt
'nvidia' p1 (us): 22 articles, 22 new
'"electric vehicles" battery' p1 (us): 10 articles, 10 new
Wrote 32 articles (32 new) to news.csv and feed.json. Credits used: 2
$ python news.py --input examples/keywords.txt --only-new # a few minutes later
'nvidia' p1 (us): 22 articles, 0 new
'"electric vehicles" battery' p1 (us): 10 articles, 0 new
Wrote 0 articles (0 new) to news.csv and feed.json. Credits used: 2
news.csv (sample), newest first:
| keyword | title | source | date | published_estimate |
|---|---|---|---|---|
| nvidia | Dow Jones Futures: Micron, Nvidia Lead AI Trade Lower; Apple, Meta Rally Near Buy Points | Investor's Business Daily | 2 hours ago | 2026-10-13T03:30:49Z |
| nvidia | China faces uphill battle to wean tech firms off Nvidia chips | Financial Times | 3 hours ago | 2026-10-13T02:30:49Z |
| nvidia | Nvidia's China AI-chip share falls as Huawei closes gap, report finds | Reuters | 3 hours ago | 2026-10-13T02:30:49Z |
| "electric vehicles" battery | 2 hospitalized after EV battery fire at Wallingford car dealership, officials say | Yahoo | 7 hours ago | 2026-10-12T22:30:49Z |
feed.json (sample):
{
"version": "https://jsonfeed.org/version/1.1",
"title": "Google News: nvidia, \"electric vehicles\" battery",
"home_page_url": "https://news.google.com/",
"description": "Google News results (us) collected 2026-10-13T05:30:49Z",
"items": [
{
"id": "https://www.ft.com/content/9dbd99db-6b0c-43bd-b7be-849f129df098?syn-25a6b1a6=1",
"url": "https://www.ft.com/content/9dbd99db-6b0c-43bd-b7be-849f129df098?syn-25a6b1a6=1",
"title": "China faces uphill battle to wean tech firms off Nvidia chips",
"content_text": "Chinese technology companies have bought more than 1mn Nvidia gaming and workstation chips this year, turning to products never designed to...",
"date_published": "2026-10-13T02:30:49Z",
"authors": [{ "name": "Financial Times" }],
"tags": ["nvidia"]
}
]
}Article counts per page vary by keyword (22 and 10 above). snippet is empty for some articles.
1 credit per keyword per page. 20 brand keywords checked every hour, 1 page each = 20 × 24 × 30 = 14,400 credits a month: $14.40 at the smallest pack's $1.00 per 1,000 credits, less on larger packs (down to $0.30, see pricing). Checking every 6 hours instead is 2,400 credits. Failed jobs are free.
| Option | Default | Description |
|---|---|---|
keywords |
Brands or keywords; use quotes inside for exact phrases: '"acme corp"' |
|
-i, --input FILE |
One keyword per line | |
-c, --country |
us |
31 supported country codes (us, gb, ca, au, de, fr, in, ...) |
-l, --language |
en |
2-letter language code |
-p, --pages N |
1 |
Pages per keyword, 1-10 |
--include a,b |
Keep articles whose title contains any of these words | |
--exclude a,b |
Drop articles whose title contains any of these words | |
--state FILE |
seen.json |
Remembers seen article links between runs |
--only-new |
off | Output only articles not seen before |
-o, --output |
news.csv |
CSV output |
--feed |
feed.json |
JSON Feed output |
Run it from cron or Windows Task Scheduler (see the price tracker README for both). To get alerts, point an automation tool at feed.json, or add a few lines that post new rows of news.csv to Slack (the competitor price monitor has a send_slack() you can copy).
published_estimatefor relative dates is computed from the time of the run, so "3 hours ago" is accurate to about an hour, "2 days ago" to about a day.- Results are what Google News shows for that country and language at that moment; they are not a complete archive of all coverage.
| Problem | Fix |
|---|---|
HTTP 401 |
The API key is missing or wrong. Check echo $CRAWLERDATA_API_KEY (or $env:CRAWLERDATA_API_KEY in PowerShell) and create a key under API Access in the dashboard. |
HTTP 402 |
You are out of credits. Buy a credit pack in the dashboard (from $1.00 down to $0.30 per 1,000 credits, see pricing). |
HTTP 429 |
Rate limit: 100 requests per minute and 20 per second per key. The client waits until X-RateLimit-Reset and retries automatically; for big batches, spread requests out. |
| Timeouts | Live requests return when Google has been fetched: a few seconds for search, 30 to 120 seconds for Google Shopping sellers. The client uses a 200 second timeout. |
job ... failed |
The HTTP call worked but the job did not (for example Google blocked that attempt). Failed jobs cost 0 credits and the client retries them twice before giving up. |
- CrawlerData
- Google News API: parameters and a full example response
- News and brand monitoring with CrawlerData
- API docs
| Repo | What it does |
|---|---|
| google-shopping-scraper | Search Google Shopping, export products to CSV/JSON (Python + Node.js) |
| google-shopping-sellers-scraper | Every seller and price for one product, as a comparison table |
| google-shopping-price-tracker | Scheduled price history in SQLite, CSV export and an HTML chart |
| competitor-price-monitoring | Your catalog vs Google Shopping sellers, with undercut alerts |
| map-violation-monitor | Flag sellers below MAP with timestamped evidence |
| google-search-scraper | Google SERP to JSON/CSV: organic, ads, People Also Ask, AI Overview (Python + Node.js) |
| google-rank-tracker | Daily keyword positions by country and device, with a change report |
| google-local-business-leads | Google Local results to a lead list CSV |
| google-news-scraper | Brand and keyword news monitoring to CSV and JSON Feed |
| google-search-mcp-server | Connect Claude, Cursor or n8n to the CrawlerData MCP server |
| crawlerdata-examples | Python, Node.js and curl snippets for every endpoint |
MIT. See LICENSE. Google and Google Shopping are trademarks of Google LLC; this project is not affiliated with Google.