Decided to switch from (A)Mnesia to Ecto with SQLite3.

1. The historic NVD data is over 1GB which means I’m already starting to run into limitations.
2. The planned database is extremely simple (two tables?).
3. Experience with Ecto is more important in the long term for planned projects.

Useful and interesting links:

Can’t believe it has been almost two months since I’ve written any Elixir. I’m sufficiently ahead on my client’s second Python project that I’m taking a break and getting back to Elixir for the rest of the month. My goal is to make significant progress on the NVD Cache project.

Implemented a stream directly from the source CSV file into the Mnesia data store.

Was surprised to see that the Mnesia-stored data is smaller than the original CSV data:

• CSV: 835 Mbytes
• Mnesia: 757 MBytes

Importing a .csv file using NimbleCSV

An article by Jacob Swanner NimbleCSV: Parsing into Elixir Maps filled in some gaps in the official documentation.

I tossed together some Python scripts that download and massage the Common Vulnerabilities and Exposure (CVE) data from the NIST/NVD website.

Built a simple playground consisting of:

• Bandit
• Plug.Router, Plug.Debugger, and Plug.ErrorHandler
• Req

Been thinking about a second project before I need to head back to Python report-building land.

I recently wrote a Python-based vulnerability reporting application that cached API lookups from the NIST NVD service.

API Constraints

The most severe NVD API constraint is the rate limiting:

• 5 lookups per 30 seconds without an API key.
• 50 lookups per 30 seconds with an API key.

I’ve also seen API requests take over 30 seconds to respond.

A typical report requires hundreds of CVE lookups which makes caching mandatory.

SimpleCardBrand version 1.0.0

I found a gentle and an in-depth article about OptionParser. Neither quite matched my specific use case of pattern matching options at the function level.

The key is converting the opts keyword list into a map.

SimpleCardBrand version 0.3.0 changes the :error reason from a String.t to a tuple: {:atom, String.t}. This allows simple programmatic decision making based off the atom along with detailed error messages. Example:

{:error, {:pan_too_short,"Minimum PAN length is 12, found 10."}}
{:error, {:pan_unknown,"Unknown card brand."}}
{:error, {:pan_too_long, "Maximum PAN length is 19, found 20."}}

Added a (NOT PCI COMPLIANT) command-line interface. Use only with test credit card account numbers.

$ ./simplecardbrand 4111111111111111
PAN: 4111111111111111 -> Brand: visa