Implemented a stream directly from the source CSV file into the Mnesia data store.
Was surprised to see that the Mnesia-stored data is smaller than the original CSV data:
- CSV: 835 Mbytes
- Mnesia: 757 MBytes
Loading a CSV File
NimbleCSV
Importing a .csv file using NimbleCSV
An article by Jacob Swanner NimbleCSV: Parsing into Elixir Maps filled in some gaps in the official documentation.
[Read More]Sidetrip Through the National Vulnerability Database
I tossed together some Python scripts that download and massage the Common Vulnerabilities and Exposure (CVE) data from the NIST/NVD website.
[Read More]Bandit Routing
and Req
Built a simple playground consisting of:
- Bandit
- Plug.Router, Plug.Debugger, and Plug.ErrorHandler
- Req
CacheNVD
Project #2
Been thinking about a second project before I need to head back to Python report-building land.
I recently wrote a Python-based vulnerability reporting application that cached API lookups from the NIST NVD service.
API Constraints
The most severe NVD API constraint is the rate limiting:
- 5 lookups per 30 seconds without an API key.
- 50 lookups per 30 seconds with an API key.
I’ve also seen API requests take over 30 seconds to respond.
A typical report requires hundreds of CVE lookups which makes caching mandatory.
[Read More]