The Problem That Started It All

Every AI developer has seen it: you ask an LLM a factual question, and it confidently returns something that's just... wrong. GDP figures from nowhere. Statistics that don't exist. Citations to papers that were never written.

The root cause isn't the model — it's the lack of structured access to primary sources. Government databases, international organizations, research institutions — they all publish incredible data, but discovering and accessing them is a nightmare.

So we started building a structured, verified catalog of authoritative data sources from around the world.

What We Built

FirstData is an open-source knowledge base that currently covers 230+ verified data sources across 50+ domains, spanning government agencies, international organizations, research institutions, and industry authorities.

Every entry includes structured metadata: what data is available, how to access it (including APIs), authority level, update frequency, and bilingual descriptions (English & Chinese).

It also ships as an MCP server, so AI agents can query it directly to find the right data source for any question.

3 Things That Surprised Me

1. China's government data is massively underrepresented globally

When we started mapping data sources, we noticed a huge gap: most international data directories barely cover Chinese government data. Yet China publishes extensive public data through agencies like the National Bureau of Statistics, the People's Bank of China, and dozens of domain-specific authorities.

We ended up building what might be the most comprehensive English-language index of Chinese government data sources. This turned out to be one of our most unique contributions.

2. "Open data" doesn't mean "easy to find"

Many governments publish fantastic datasets, but discoverability is terrible. Some agencies bury their API documentation three clicks deep. Others have data portals that look like they were designed in 2005 (and probably were).

We spent more time verifying URLs and finding API endpoints than actually writing code. Every single link in our catalog has been manually tested — and we regularly find that 5-10% of URLs break within any given month.

3. Authority level matters more than you think

Not all data sources are created equal. A GDP figure from the World Bank carries different weight than one from a commercial aggregator. We introduced an authority classification system (government, international organization, research, market, commercial) and it fundamentally changed how we think about data quality.

For AI applications, this distinction is critical. When your LLM cites a source, knowing whether it's a government primary source or a third-party estimate makes all the difference.

What's Next

We're at 230+ sources today, targeting 1,000+. The biggest challenge isn't adding more entries — it's maintaining quality. Every source needs regular URL verification, API testing, and metadata updates.

If you work with public data and want to contribute, check out the project on GitHub. We especially need help with data sources from regions we haven't covered well yet — Africa, South America, and Southeast Asia.

Building in public, learning in public. That's the idea.

FirstData is maintained by the team at Mininglamp Technology. It's MIT licensed and open for contributions.