The official Node.js/TypeScript client library for the ScrapeBadger API — Twitter, Google, Vinted, and general web scraping.
- Full TypeScript Support - Complete type definitions for all API endpoints
- Modern ESM & CommonJS - Works with both module systems
- Async Iterators - Automatic pagination with
for await...ofsyntax - Smart Rate Limiting - Reads API headers and throttles pagination automatically
- Resilient Retries - Exponential backoff with colored console warnings
- Typed Exceptions - Distinct error classes for every failure scenario
- 37+ Twitter endpoints - Tweets, users, lists, communities, trends, geo, real-time streams
- 20 Google product APIs - Search (with optional deferred AI Overview follow-up), Maps, News, Hotels, Trends (incl. topic autocomplete), Jobs, Shopping (+ merchant URL enrichment, barcode offers), Patents, Scholar (search + profiles + author + author citation + cite formats), Images, Videos, Finance, AI Mode, Lens, Local Pack, Shorts, Flights, Products, Ads Transparency Center
- Vinted scraping - Search items, item details, user profiles, brands, colors, markets
- Web scraping - Anti-bot bypass, JS rendering, screenshots, and selector/AI data extraction
npm install scrapebadgeryarn add scrapebadgerpnpm add scrapebadgerimport { ScrapeBadger } from "scrapebadger";
const client = new ScrapeBadger({ apiKey: "your-api-key" });
// Get a tweet
const tweet = await client.twitter.tweets.getById("1234567890");
console.log(`@${tweet.username}: ${tweet.text}`);
// Scrape a website
const result = await client.web.scrape("https://scrapebadger.com", { format: "markdown" });
console.log(result.content);
// Get a user profile
const user = await client.twitter.users.getByUsername("elonmusk");
console.log(`${user.name} has ${user.followers_count.toLocaleString()} followers`);// Pass API key directly
const client = new ScrapeBadger({ apiKey: "sb_live_xxxxxxxxxxxxx" });
// Or use environment variable SCRAPEBADGER_API_KEY
const client = new ScrapeBadger();The etsy, aliexpress, target, asos, and bestbuy clients expose 44 dedicated ecommerce endpoints. These generated methods return response objects; consult the endpoint documentation for response fields, supported filters, pagination and current pricing.
const client = new ScrapeBadger({ apiKey: "your-api-key" });
const etsy = await client.etsy.searchListings({ query: "ceramic mug" });
const aliexpress = await client.aliexpress.searchProducts({ query: "wireless mouse" });
const target = await client.target.searchProducts({ query: "coffee" });
const asos = await client.asos.searchProducts({ query: "shoes", market: "uk" });
const bestbuy = await client.bestbuy.searchProducts({ query: "laptop" });| Client | Endpoints | Documentation |
|---|---|---|
client.etsy |
8 — listing search/detail/reviews, shop profile/listings/reviews, categories, autocomplete | Etsy |
client.aliexpress |
8 — product search/detail/reviews, store profile/products/categories, category navigation, autocomplete | AliExpress |
client.target |
11 — search, category browse/navigation, autocomplete, product detail/reviews/questions/availability, nearby stores/store detail, markets | Target |
client.asos |
7 — product search/detail, stock/prices, category navigation, search filters, autocomplete, markets | ASOS |
client.bestbuy |
10 — search, category browse/navigation, deals, autocomplete, product detail/reviews/questions/availability, markets | Best Buy |
Etsy and AliExpress category navigation is separate from search: their search methods do not offer a category filter. ASOS search follows its upstream semantic matching. Target and Best Buy serve the US market. The linked endpoint references describe further measured limits.
| API | Description | Documentation |
|---|---|---|
| Web Scraping | Scrape any website with JS rendering, anti-bot bypass, screenshots, and selector/AI extraction | Web Scraping Guide |
| 37+ endpoints for tweets, users, lists, communities, trends, and real-time streams | Twitter Guide | |
| 20 products — Search, Maps, News, Hotels, Trends, Jobs, Shopping, Patents, Scholar, Images, Videos, Finance, AI Mode, Lens, Autocomplete, Local, Shorts, Flights, Products, Ads Transparency | Google Guide | |
| Vinted | Search items, get details, user profiles, and reference data across all Vinted markets | Vinted Guide |
| Wallapop | 10 endpoints across 3 markets (es, it, pt) — listing search with Wallapop's own filters (incl. cars: year, km, gearbox, horsepower), listing detail, seller profile/listings/reviews, autocomplete, categories, search filters and filter options, markets | Wallapop Docs |
| Search posts/subreddits/users, subreddit details, post comments, user profiles, wiki pages, trophies | Reddit Guide | |
| User profile/about/related/posts/videos/reels/tagged/pinned/followers/following/stories/highlights, media detail/comments/replies/likers/oEmbed, search (users/hashtags/places/top/reels/music/autocomplete), hashtag/location/audio feeds | Instagram Guide | |
| Amazon | 14 endpoints — search, autocomplete, product detail, offers, reviews, bestsellers, new releases, deals, category browse, seller profile/products/feedback, markets, categories | Amazon Guide |
| Shopee | 6 endpoints across 11 markets (sg, my, ph, id, vn, th, tw, br, co, cl, mx) — product search, category items, product detail, reviews, category tree, markets | Shopee Guide |
| TikTok | 26 endpoints — user profile/videos/followers/following/liked/reposts, video detail/comments/replies/related/transcript, oEmbed, hashtag detail/videos, music detail/videos, search (general/videos/users/hashtags), trending (videos/hashtags/songs), ad library, regions | TikTok Guide |
| eBay | 12 endpoints across 18 markets — search, search by image, completed/sold listings, item detail, item reviews, seller profile/items/feedback, category browse, categories, autocomplete, markets | eBay Guide |
| Walmart | 11 endpoints (walmart.com, US-only) — search, category browse, deals feed, autocomplete, product detail, reviews, seller profile/products, store detail with nearby stores, markets | |
| Gumtree | 9 endpoints across gumtree.com, gumtree.com.au and gumtree.co.za (market: "uk" default, "au", "za") — search (category, location/postcode radius, price, seller type, urgent/photos, category-specific filters), search filters with live counts, listing detail (full vehicle spec, seller rating/verification, masked phone), seller listings, keyword and location autocomplete, location and category trees, markets. E.g. client.gumtree.search({ market: "au", category: "18320", location: "3003435", distance: 10 }) or client.gumtree.search({ market: "za", category: "p6dnh", location: "dkp8y" }) (za ids are refs) |
Gumtree Docs |
| Bing | 6 endpoints — web search (with ads + related searches), image search, video search, news vertical, autocomplete, markets | |
| DuckDuckGo | 7 endpoints — web search (with instant-answer abstract), image search, news vertical, video search, autocomplete, instant answers, regions | |
| Yahoo | 6 endpoints across 35 markets — web search (with ads + related searches), image search, video search, news vertical, autocomplete, markets | Yahoo Docs |
| Yandex | 4 endpoints across 6 markets (tr/com/ru/by/kz/uz) — web search (organic + ads + sitelinks + inline media), image search, reverse-image (CBIR) search, markets | Yandex Docs |
| YouTube | 39 endpoints — search, autocomplete, video detail/related/comments/replies/transcript/captions/streams/live chat/batch, channel detail/videos/shorts/streams/playlists/community/about/subscriber count/search/resolve, playlist detail/items, mixes, trending/shorts, hashtags, home, shorts detail/by sound, community posts/comments, music search, oEmbed, categories/languages/regions/markets | YouTube Guide |
| Immobiliare | 8 endpoints across 4 markets (it, es, gr, lu) — autocomplete, search, listing detail, agency profile/listings, price stats, markets, reference | Immobiliare Guide |
| LoopNet | 5 endpoints across loopnet.com/.ca/.co.uk/.fr/.es — commercial-real-estate search (for-lease/for-sale/auctions), listing detail, broker profile, markets, property types | LoopNet Guide |
| Apartments.com | US rental listings with UNIT-LEVEL pricing — search by location with bed/price filters, plus per-unit rent, beds, baths, sqft and availability for every rentable unit | |
| Baidu | 4 endpoints (baidu.com, China's #1 search engine) — web search with language and date-window filters, news vertical, image search, autocomplete. Results carry the real target URL, not just Baidu's tracking redirect | Baidu Docs |
| ChatGPT | Prompt the real chatgpt.com anonymously — structured answer with citations anchored to character offsets, the full retrieved search trail, and AEO/GEO brand-visibility analysis | ChatGPT Guide |
| Gemini | Prompt the real gemini.google.com anonymously — structured answer with cited web sources, the full retrieved search trail, and AEO/GEO brand-visibility analysis | Gemini Docs |
import {
ScrapeBadger,
AuthenticationError,
RateLimitError,
NotFoundError,
InsufficientCreditsError,
} from "scrapebadger";
const client = new ScrapeBadger({ apiKey: "your-api-key" });
try {
const tweet = await client.twitter.tweets.getById("1234567890");
} catch (error) {
if (error instanceof AuthenticationError) {
console.error("Invalid API key");
} else if (error instanceof RateLimitError) {
console.error(`Rate limited. Retry after: ${error.retryAfter}`);
} else if (error instanceof NotFoundError) {
console.error("Tweet not found");
} else if (error instanceof InsufficientCreditsError) {
console.error("Out of credits");
} else {
throw error;
}
}const client = new ScrapeBadger({
// Required: Your API key (or use SCRAPEBADGER_API_KEY env var)
apiKey: "your-api-key",
// Optional: Custom base URL (default: https://scrapebadger.com)
baseUrl: "https://scrapebadger.com",
// Optional: Request timeout in milliseconds (default: 30000)
timeout: 30000,
// Optional: Maximum retry attempts (default: 10)
maxRetries: 10,
// Optional: Initial retry delay in milliseconds (default: 1000)
retryDelay: 1000,
});The SDK automatically retries requests that fail with server errors (5xx) or rate limits (429) using exponential backoff (1s, 2s, 4s, 8s, ...). Each retry prints a colored warning:
⚠ ScrapeBadger: 503 Service Unavailable — retrying in 4s (attempt 3/10)
When using *All pagination methods (e.g. searchAll, getFollowersAll), the SDK
reads X-RateLimit-Remaining and X-RateLimit-Reset headers from each response.
When remaining requests drop below 20% of your tier's limit, pagination automatically
slows down to spread requests across the remaining window — preventing 429 errors:
⚠ ScrapeBadger: Rate limit: 25/300 remaining (resets in 42s), throttling pagination
This works transparently with all tier levels (Free: 60/min, Basic: 300/min, Pro: 1000/min, Enterprise: 5000/min).
ScrapeBadgerError- Base exception classAuthenticationError- Invalid or missing API keyRateLimitError- Rate limit exceededNotFoundError- Resource not foundValidationError- Invalid requestServerError- Server errorTimeoutError- Request timeoutInsufficientCreditsError- Out of creditsAccountRestrictedError- Account restrictedWebSocketStreamError- WebSocket stream failure (auth, limit, or network)
- Node.js 18+ (for native
fetchsupport) - TypeScript 5.0+ (for best type inference)
MIT License - see LICENSE for details.