Sometimes the data you need is already waiting for you. No scraping required.
Before you build a web scraper for stock prices, weather forecasts, or country metadata, check if someone has already done the hard work. Public APIs provide structured data in clean formats like JSON, eliminating the fragility of parsing HTML and the cat-and-mouse game of anti-bot measures.
This guide covers the best free public APIs organized by category, explains when APIs beat scraping (and when they don't), and shows you how to combine both approaches when projects demand it.
Why Consider APIs Before Scraping?
Web scraping extracts data from websites designed for humans. Public APIs deliver data through endpoints designed for machines. The difference matters:
Structured output. APIs return JSON or XML with consistent field names and data types. Scraped HTML requires parsing, cleaning, and constant maintenance when layouts change.
Reliability. Well-maintained APIs come with documentation, versioning, and uptime guarantees. Scraping breaks when a site's CSS class names change from product-title to productTitle without warning.
Legal clarity. Most public APIs have explicit terms of service allowing data access. Scraping often exists in legal gray areas, especially for commercial use.
Rate limiting transparency. APIs tell you upfront how many requests you can make. Scraping means guessing at thresholds and risking IP blocks.
That said, APIs have limits. Many don't exist for the data you need. Others restrict access behind paywalls. And some return only a subset of what's visible on the website. Knowing when to use each approach, or both together, is the skill that separates capable data practitioners from frustrated ones.
Government and Open Data APIs
Governments worldwide now publish data through APIs as part of transparency initiatives. These are among the most reliable free data sources available.
Data.gov (United States)
The US federal government's open data portal provides access to over 300,000 datasets from agencies like NASA, NOAA, and the Census Bureau. While Data.gov primarily catalogs datasets (many as downloadable files), it includes links to agency-specific APIs.
Notable APIs accessible through Data.gov:
- NASA APIs — Astronomy Picture of the Day, Mars rover photos, asteroid tracking
- NOAA Climate Data — Historical weather, ocean temperatures, storm data
- Census Bureau — Population statistics, economic indicators, geographic data
Best for: Academic research, civic applications, climate and environmental data.
API Setu (India)
India's API Setu platform offers thousands of government APIs covering everything from vehicle registration to ration cards to education certificates. It's one of the most comprehensive government API ecosystems globally.
Best for: Applications serving Indian citizens, government service integrations.
UK Government API Catalogue
The UK API catalogue lists APIs published by public sector organizations, including planning data, company registrations, and public health statistics.
Best for: UK-focused applications, business research involving UK entities.
Public API Marketplaces
These platforms aggregate APIs from multiple providers, making discovery and integration easier.
RapidAPI
RapidAPI hosts over 40,000 APIs with a unified authentication and billing system. Many APIs offer free tiers. Categories span sports scores, stock data, SMS services, and AI models.
How it works: Create a RapidAPI account, subscribe to APIs (many are free), and use a single API key for all services. The platform handles rate limiting and usage tracking.
Best for: Developers prototyping applications who need multiple data sources quickly.
Public APIs GitHub Repository
The public-apis repository on GitHub maintains a curated list of free APIs across 50+ categories. Each entry includes authentication requirements, HTTPS support, and CORS compatibility.
This isn't an API marketplace but a community-maintained directory. It's often the first place experienced developers check when looking for free data sources.
Best for: Finding niche or unusual APIs that aren't on commercial marketplaces.
Weather and Environmental APIs
Weather data is one of the most common API use cases, and several providers offer generous free tiers.
OpenWeatherMap API
OpenWeatherMap provides current weather, forecasts, historical data, and air quality information for locations worldwide.
Free tier: 60 API calls per minute, current weather and 5-day forecasts.
Data available:
- Temperature, humidity, wind speed, precipitation
- 16-day forecasts (paid tier)
- Historical weather data (paid tier)
- Air pollution indexes
Best for: Travel apps, farming applications, smart home integrations.
Open-Meteo
Open-Meteo offers completely free weather forecasts with no API key required for non-commercial use.
Data available:
- 7-day forecasts with hourly granularity
- Historical weather since 1940
- Climate models and marine weather
Best for: Personal projects, educational applications, research.
Financial and Market APIs
Stock prices and financial data traditionally required expensive subscriptions. Several APIs now offer free alternatives.
CoinGecko API
CoinGecko provides cryptocurrency prices, market caps, trading volumes, and historical data for thousands of coins and tokens.
Free tier: 10-30 calls per minute depending on endpoint.
Data available:
- Real-time prices in multiple currencies
- Historical OHLC (Open, High, Low, Close) data
- Exchange-specific trading pairs
- Token metadata and categories
Best for: Crypto portfolio trackers, DeFi applications, market research.
Alpha Vantage
Alpha Vantage offers free APIs for stocks, forex, and cryptocurrencies with technical indicators.
Free tier: 25 requests per day.
Data available:
- Real-time and historical stock prices
- Technical indicators (SMA, RSI, MACD, etc.)
- Fundamental data (earnings, balance sheets)
- Forex and crypto rates
Best for: Personal finance apps, algorithmic trading prototypes.
Content and Media APIs
NewsAPI
NewsAPI aggregates headlines and articles from thousands of news sources globally, allowing filtering by language, country, and topic.
Free tier: 100 requests per day, developer use only. Commercial use requires paid plans.
Data available:
- Top headlines by country and category
- Full-text article search
- Source metadata and logos
Best for: News aggregators, media monitoring, content research.
The Movie Database (TMDb)
TMDb provides comprehensive data about movies and TV shows, including cast, crew, ratings, and images.
Free tier: Rate-limited but generous for personal projects.
Data available:
- Movie and TV show metadata
- Cast and crew information
- Poster and backdrop images
- User ratings and reviews
Best for: Entertainment apps, media libraries, recommendation engines.
Unsplash API
Unsplash offers access to millions of high-resolution stock photos with permissive licensing.
Free tier: 50 requests per hour.
Data available:
- Photo search by keyword, color, or orientation
- Curated collections
- Photographer information
Best for: Design applications, content generation, placeholder images.
Geographic and Location APIs
REST Countries
REST Countries provides detailed information about every country without requiring an API key.
Data available:
- Country names, capitals, populations
- Languages, currencies, time zones
- Borders, flags, and regional groupings
Best for: Educational apps, travel tools, internationalization features.
OpenStreetMap APIs
OpenStreetMap offers mapping data through several APIs, including the Nominatim geocoding service and the Overpass API for geographic features.
Data available:
- Forward and reverse geocoding
- Points of interest
- Road networks and building footprints
Best for: Mapping applications, location-based services (with appropriate usage policies).
Developer and Tech APIs
GitHub API
The GitHub REST API provides access to repositories, users, issues, and commits without requiring authentication for public data.
Data available:
- Repository metadata and statistics
- User profiles and activity
- Issues, pull requests, and discussions
- Code search
Best for: Developer tools, open-source analytics, portfolio showcases.
Stack Exchange API
The Stack Exchange API covers Stack Overflow and related sites, providing access to questions, answers, and user data.
Data available:
- Questions with tags, scores, and answers
- User reputation and badges
- Comments and discussions
Best for: Developer documentation, code assistance tools, community analytics.
When You Still Need to Scrape
APIs solve many data collection problems, but not all. Here's when scraping remains the right approach:
No API exists. Many websites, especially smaller businesses, directories, and niche platforms, simply don't offer APIs. If the data only exists on web pages, scraping is your only option.
APIs hide data. Some companies deliberately limit API access to preserve competitive advantages. A product page might show 50 data points while the API returns 5.
Real-time isn't available. APIs often delay data compared to what's visible on the website. For time-sensitive monitoring (price changes, stock availability), scraping the live page may be faster.
Cost prohibitive. Some APIs charge per request at rates that make large-scale collection impractical. A scraper running on your infrastructure might be cheaper for high-volume needs.
Format transformation. Scraping lets you capture exactly the data representation users see, including formatted text, images, and layout that APIs may not expose.
Combining APIs and Scraping
Many real-world projects benefit from both approaches. Here's how tools like Lection fit into hybrid workflows:
Start with APIs for structured data. Use CoinGecko for crypto prices, TMDb for movie info, or GitHub for repository stats. This gives you reliable, clean data for the foundation.
Scrape for what APIs miss. When you need additional context, use Lection's Chrome extension to extract supplementary data from web pages. A product research project might pull specifications from an e-commerce API but scrape customer reviews that aren't exposed.
Validate and enrich. Cross-reference API data with scraped data to verify accuracy. APIs can have stale or incomplete information. Scraping the source confirms what's current.
Example workflow:
- Query the GitHub API for repository metadata (stars, forks, description)
- Scrape the README.md rendered page for formatted content and images
- Use web scraping with Zapier to automate the data flow into your database
How to Evaluate Free APIs
Not all free APIs are equally useful. Before building on one, check:
Documentation quality. Good APIs have clear documentation with examples, error codes, and SDKs. Poor documentation signals an API that may not be maintained.
Rate limits. Understand the limits before you start. Building around 25 requests per day is very different from 1,000 requests per minute.
Uptime and reliability. Check if the provider publishes status pages. Search for community discussions about outages. A free API that's down 10% of the time costs more in lost development time than a reliable paid alternative.
Terms of service. Verify commercial use is allowed if you're building a product. Many free tiers restrict to personal or educational use.
Data freshness. Ask how often data updates. Some free APIs provide delayed data (15-minute stock prices, yesterday's weather) while paid tiers offer real-time.
Authentication requirements. Some APIs need no key (REST Countries). Others require email registration. A few need credit card information even for free tiers.
API Discovery Resources
Finding the right API can be half the challenge. These resources help:
- Public APIs GitHub repo — Curated list with 1,400+ entries
- RapidAPI Hub — Searchable marketplace with free tier filtering
- ProgrammableWeb — API directory and news (though less active than before)
- API Layer — Curated APIs focused on reliability
- MashApe (via RapidAPI) — Legacy marketplace now part of RapidAPI
When searching, use terms like "[topic] API free tier" or "[topic] public API" to find options quickly.
Quick Reference: Top Free APIs by Category
| Category | API | Free Tier Limits | No Auth |
|---|---|---|---|
| Weather | OpenWeatherMap | 60 calls/min | No |
| Weather | Open-Meteo | Unlimited (non-commercial) | Yes |
| Crypto | CoinGecko | 10-30 calls/min | Yes |
| News | NewsAPI | 100 calls/day | No |
| Movies | TMDb | Rate limited | No |
| Countries | REST Countries | Unlimited | Yes |
| Stocks | Alpha Vantage | 25 calls/day | No |
| Maps | OpenStreetMap | Fair use | Varies |
| Images | Unsplash | 50 calls/hour | No |
| GitHub | GitHub REST API | 60 calls/hour (unauth) | Yes |
| NASA | NASA APIs | Generous | No |
When to Upgrade to Paid APIs
Free tiers work well for prototypes, personal projects, and low-volume applications. Consider paid plans when:
Hitting rate limits regularly. If you're constantly working around request caps, you're losing more in development time than a subscription costs.
Need real-time data. Most free tiers include delays. Paid plans often provide live data.
Production reliability matters. Free tiers may have lower SLA guarantees. For business-critical applications, paid APIs offer better uptime commitments.
Support is necessary. Free users typically get community-only support. Paid plans include direct support channels.
Data depth increases. Premium tiers often unlock additional endpoints, historical data, or enhanced detail not available for free.
Conclusion
Free public APIs eliminate scraping complexity for many common data needs. Weather, financial markets, geographic information, media metadata, and developer resources are all well-covered by reliable, documented APIs.
But APIs can't replace scraping entirely. When the data you need isn't exposed through an API, or when you need exactly what's visible on a webpage, tools like Lection bridge the gap.
The best approach is pragmatic: use APIs where they exist and scrape where they don't. Start your next project by checking if an API already provides what you need. If not, install Lection and extract the data directly.