Been thinking about a second project before I need to head back to Python report-building land.
I recently wrote a Python-based vulnerability reporting application that cached API lookups from the NIST NVD service.
API Constraints
The most severe NVD API constraint is the rate limiting:
- 5 lookups per 30 seconds without an API key.
- 50 lookups per 30 seconds with an API key.
I’ve also seen API requests take over 30 seconds to respond.
A typical report requires hundreds of CVE lookups which makes caching mandatory.
Web Service
A locally-run caching web service seems a reasonable approach. It provides a language-neutral, centralized cache.
Initial Design Thoughts
- Single CVE lookup using the same URL path as the NVD API.
https://services.nvd.nist.gov/rest/json/cves/2.0?cveId={cve_id}
- Rate limit for with and without an API key.
- Implement in Elixir.
- Write a Bandit plug.
- Use the req HTTP client.
- Store data in Mnesia.
Why Elixir?
- I’m looking for a second Elixir project in order to learn the language.
- With an Elixir app you can either embed the server in your own application or run it standalone.
- Elixir is fun!
Why Bandit vs. Phoenix?
- An opportunity to get a bit further “under the hood”.
- The web service is designed for internal usage so many of the Phoenix bells and whistles are less important.
Why req?
- req appears to be a popular modern http client.
- Very simple HTTP client requirements, so simple is best.
Why Mnesia?
- Need (slightly) more than a simple key/value store.
- PostgreSQL (my favorite database) is overkill.
- SQLite3 is always fun to use, but I keep running into the issue of being unable to dynamically update the schema.
- Mnesia will let me learn about distributed databases.
- Mnesia lets a standalone implementation share the cache with an embedded server.
Possible Future Features
- Implement batch updates.
Terms
- CVE: Common Vulnerabilities and Exposures
- NIST: National Institute of Standards and Technology
- NVD: National Vulnerability Database