It's been a while since NIST changed the API for their NVD (National Vulnerability Database), so I (finally) got around to writing some code against that API.  This API gives you a way for your code to query CVE's (Common Vulnerabilities and Exposures) against a broad range of products (or against specific products).  What this immediately brought to my mind was what I always ask my clients to put in someone's job description "monitor vendor announcements and industry news for vulnerabilities in the products in use by the organization".  This can be a tough row to hoe, especially if we're not talking the Microsoft / Cisco / Oracle and other "enterprise flagship" products - the ones that might immediately come to mind - if you monitor the cve list you'll see dozens or hundreds of CVEs scroll by in a day.   Also, subscribing to all the vendor security newsgroups and feeds can also quickly turn into a full-time proposition.  I think using the NIST API can be a viable alternative to just plain "keeping up".  CVE's often are a few days behind vendor announcements and patches, but on the other hand the CVE database is a one-stop-shop, (theoretically) everything is posted here.

In most cases I'd expect a hybrid approach - monitor industry news and vendor sources for your "big gun" applications, and then query this database for everything (to catch smaller apps and anything missed on your list of major apps).

First of all, in this API products are indexed by "CPE" (Common Platform Enumeration), a structured naming convention.  Let's see how this works - first, let's query the CPE database to get the NVD representation of the products that you have in your organization.  You can do this iteratively.

For instance, to get PeopleSoft versions, starty by querying Oracle products:

https://services.nvd.nist.gov/rest/json/cpes/1.0?cpeMatchString=cpe:2.3:*:oracle

Then narrow it down with the info you find in that search to get PeopleSoft.  

https://services.nvd.nist.gov/rest/json/cpes/1.0?cpeMatchString=cpe:2.3:*:oracle:peoplesoft_enterprise

As you can see, you can narrow this down further by version:

https://services.nvd.nist.gov/rest/json/cpes/1.0?cpeMatchString=cpe:2.3:a:oracle:peoplesoft_enterprise:8.22.14

.. however you might also want to leave that a bit open - you don't want to be in the position where your Operations team has updated the application but maybe didn't tell the security team or the security developer team (that'd be you).
(Or the dev-sec-dev-devops or whatever we're calling that these days :-))

Repeat this process for everything in your organization's flock of applications and Operating Systems.  As with all things of this type, of course test with one or three applications/CPEs, then grow your list over time.  You can use our previous diary on collecting hardware and software inventories (CIS Critical Controls 1 and 2) - code for these is here:
https://github.com/robvandenbrink/Critical-Controls/tree/master/CC01
https://github.com/robvandenbrink/Critical-Controls/tree/master/CC02

Full docs for the CPE API are here:
https://csrc.nist.gov/publications/detail/nistir/7696/final
https://csrc.nist.gov/CSRC/media/Projects/National-Vulnerability-Database/documents/web%20service%20documentation/Automation%20Support%20for%20CPE%20Retrieval.pdf

So how do we translate from a "real" inventory as reported by Windows WMI or PowerShell to what the CPE API expects?  The short answer is "the hard way" - we hit each application, one by one, and do a "it looks sorta like this" query.  The NVD gives us a handy page to do this in - for a manual thing like this I find the website to be quicker than the API.
First, navigate to https://nvd.nist.gov/products/cpe/search/

Then, put in your application name, vendor name, or any substring of those.

For instance, putting in "AutoCad" gives you 4 pages of various versions and application "bolt-ons" (or "toolsets" in autocad-speak).  It's interesting that the versioning convention is not standard, not even in this single first product example.
Some versions are named as "autocad:2010", and some are named as "autodesk:autocad_architecture_2009"
What you'll also find in this exploration is that "AutoCad version 2020" is actually version "24.0" as reported by the Windows interfaces - some of this oddness creeps into the CPE database as well.

Looking at a few apps that are more commonly used, let's look at the Adobe and FoxIT PDF readers.

To get all current-ish versions of FoxIT reader, we'll look for 10.0 and 10.0.1, these boil down to:

cpe:2.3:a:foxitsoftware:foxit_reader:10.*

This query gives us a short list:

cpe:2.3:a:foxitsoftware:foxit_reader:10.0.0:*:*:*:*:*:*:* cpe:2.3:a:foxitsoftware:foxit_reader:10.0.0.35798:*:*:*:*:*:*:* cpe:2.3:a:foxitsoftware:foxit_reader:10.0.1.35811:*:*:*:*:*:*:*

For our final list, we'll modify that last one to catch future sub-versions of 10.0.1, as:

cpe:2.3:a:foxitsoftware:foxit_reader:10.0.1.*

Looking for Adobe Reader DC version 19 and 20, we end up with:

cpe:2.3:a:adobe:acrobat_reader_dc:19.* cpe:2.3:a:adobe:acrobat_reader_dc:20.*

This all looks straightforward, except that when you look at a few versions of 20, it looks like:

cpe:2.3:a:adobe:acrobat_reader_dc:20.009.20074:*:*:*:continuous:*:*:* cpe:2.3:a:adobe:acrobat_reader_dc:20.001.30002:*:*:*:classic:*:*:*

So you'd think a properly formed query for version 20/classic would look like:
cpe:2.3:a:adobe:acrobat_reader_dc:20.*.*:*:*:*:classic:*:*:* - but no, that doesn't work

So you see, there's a bit of back-and-forth for each one - for each of my clients I tend to put an afternoon aside to turn their software inventory into a "CPE / CVE friendly" inventory.
You can speed this process up by downloading the entire CPE dictionary and "grep" through it for what you need: https://nvd.nist.gov/products/cpe
Just using grep, you now likely relate your software inventory back to the CPE dictionary much easier - for instance:

>type official-cpe-dictionary_v2.3.xml | grep -i title | grep -i microsoft | grep -i office     <title xml:lang="en-US">Avery Wizard For Microsoft Office Word 2003 2.1</title>     <title xml:lang="en-US">Microsoft BackOffice</title>     <title xml:lang="en-US">Microsoft BackOffice 4.0</title>     <title xml:lang="en-US">Microsoft BackOffice 4.5</title>     <title xml:lang="en-US">Microsoft backoffice_resource_kit</title>     <title xml:lang="en-US">Microsoft backoffice_resource_kit 2.0</title>     <title xml:lang="en-US">Microsoft Office Excel 2002 Service Pack 3</title>     <title xml:lang="en-US">Microsoft Office Excel 2003 Service Pack 3</title>     <title xml:lang="en-US">Microsoft Office Excel 2007 Service Pack 2</title>     <title xml:lang="en-US">Microsoft Office Excel 2007 Service Pack 3</title>     <title xml:lang="en-US">Microsoft Office Excel 2010</title>     <title xml:lang="en-US">Microsoft Office Excel 2010 Service Pack 1</title>     <title xml:lang="en-US">Microsoft Office Excel 2010 Service Pack 2 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office Excel 2010 Service Pack 2 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office Excel 2010 Service Pack 2 x86 (32-bit)</title>     <title xml:lang="en-US">Microsoft Internet Explorer 5 (Office 2000)</title>     <title xml:lang="en-US">Microsoft Internet Explorer 5.01 (Office 2000 SR-1)</title>     <title xml:lang="en-US">Microsoft Office</title>     <title xml:lang="en-US">Microsoft Office 3.0</title>     <title xml:lang="en-US">Microsoft Office 4.0</title>     <title xml:lang="en-US">Microsoft Office 4.3</title>     <title xml:lang="en-US">Microsoft Office 95</title>     <title xml:lang="en-US">Microsoft Office 97</title>     < ... more older versions .. >     <title xml:lang="en-US">Microsoft Office 2013</title>     <title xml:lang="en-US">Microsoft Office 2013 RT</title>     <title xml:lang="en-US">Microsoft Office 2013 Click-to-Run (C2R)</title>     <title xml:lang="en-US">Microsoft Office 2013 RT Edition</title>     <title xml:lang="en-US">Microsoft Office 2013 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office 2013 x86 (32-bit)</title>     <title xml:lang="en-US">Microsoft Office 2013 SP1</title>     <title xml:lang="en-US">Microsoft Office 2013 Service Pack 1 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office 2013 Service Pack 1 on X86</title>     <title xml:lang="en-US">Microsoft Office 2013 RT SP1</title>     <title xml:lang="en-US">Microsoft Office 2013 Service Pack 1 RT Edition</title>     <title xml:lang="en-US">Microsoft Office 2016</title>     <title xml:lang="en-US">Microsoft Office 2016 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office 2016</title>     <title xml:lang="en-US">Microsoft Office 2016 for MacOS</title>     <title xml:lang="en-US">Microsoft Office 2016 Click-to-Run (C2R)</title>     <title xml:lang="en-US">Microsoft Office 2016 Click-to-Run (C2R) x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Office 2016 MacOS Edition</title>     <title xml:lang="en-US">Microsoft Office 2016</title>     <title xml:lang="en-US">Microsoft Office 2016 Mac OS Edition</title>     <title xml:lang="en-US">Microsoft Office 2016</title>     <title xml:lang="en-US">Microsoft Office 2019</title>     <title xml:lang="en-US">Microsoft Office 2019 on x64</title>     <title xml:lang="en-US">Microsoft Office 2019 on x86</title>     <title xml:lang="en-US">Microsoft Office 2019 for Mac OS</title>     <title xml:lang="en-US">Microsoft Office 2019 for Mac OS X</title>     <title xml:lang="en-US">Microsoft Office 2019 for macOS</title>     <title xml:lang="en-US">Microsoft Office 2019 Mac OS Edition</title>     <title xml:lang="en-US">Microsoft Office 2013 RT</title>     < and so on ... >

You can see this list relates much better to your "actual" inventory as reported by more "traditional" means (WMI / Powershell).  
Now refine this list, relating the results back to your real inventory.  Sorry, since the "match" on the title isn't always exact, this again usually involves some manual components.  Even mainstream products don't always have a match.

For instance, looking for MS Project 2013:
From our cross-domain software inventory (in PowerShell) we have:

> $domainapps.name | grep "Project" Microsoft Project MUI (English) 2013 Microsoft Project Professional 2013

And from the CPE Dictionary:

>type official-cpe-dictionary_v2.3.xml | grep -i title | grep -i microsoft | grep -i project | grep 2013     <title xml:lang="en-US">Microsoft Project 2013 Service Pack 1</title>     <title xml:lang="en-US">Microsoft Project Server 2013</title>     <title xml:lang="en-US">Microsoft Project Server 2013 Service Pack 1</title>     <title xml:lang="en-US">Microsoft Project Server 2013 Service Pack 1 (x64) 64-bit</title>

So a match, but not exact.

Another illustration - even MS Word there isn't an exact match, in almost every case we're either picking a few CPE's or guesstimating for the closest one:

> $domainapps.name | grep "Word" Microsoft Word MUI (English) 2013

>type official-cpe-dictionary_v2.3.xml | grep -i title | grep -i microsoft | grep -i word | grep 2013     <title xml:lang="en-US">Microsoft Word 16.0.11425.20132 for Android</title>     <title xml:lang="en-US">Microsoft Word 2013</title>     <title xml:lang="en-US">Microsoft Word 2013 RT Edition</title>     <title xml:lang="en-US">Microsoft Word 2013 for Microsoft Windows RT</title>     <title xml:lang="en-US">Microsoft Word 2013 Service Pack 1</title>     <title xml:lang="en-US">Microsoft Word 2013 Service Pack 1 x64 (64-bit)</title>     <title xml:lang="en-US">Microsoft Word 2013 Service Pack 1 on x86</title>     <title xml:lang="en-US">Microsoft Word 2013 SP1 RT</title>     <title xml:lang="en-US">Microsoft Word RT 2013 Service Pack 1</title>

 

Where this plays well for me is if I exclude Windows and Office - just using Windows Update (or  your WSUS, SCCM or whatever patch manager you use), then auditing the "last updated" date for Windows tends to catch those really well.  It's the MS server apps - Exchange, SQL and so on, and everything that isn't Microsoft where this method really helps turn the flood of CVE information into a once a week summary that you can use.

 

As you'd expect, if you are in a niche industry, you may not find all of your applications - for instance ICS clients and Banking clients may not find their user-facing main business applications in the CPE list, and sadly will never see CVEs to keep their vendors honest.

In a welcome recent development, as I was writing this article I noticed that if your application has a listening port, nmap will make a reasonably good estimate of what that application is with the "sV" option (Probe open ports to determine service/version info).   In the example below nmap very nicely inventories my ESXI 7.0 server, and gives me the exact CPE for it:

C:\>nmap -sV -p 443 --open 192.168.122.51 Starting Nmap 7.80 ( https://nmap.org ) at 2021-01-06 18:45 Eastern Standard Time Nmap scan report for 192.168.122.51 Host is up (0.0011s latency). PORT    STATE SERVICE   VERSION 443/tcp open  ssl/https VMware ESXi SOAP API 7.0.0 MAC Address: 00:25:90:CB:00:18 (Super Micro Computer) Service Info: CPE: cpe:/o:vmware:ESXi:7.0.0 Service detection performed. Please report any incorrect results at https://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 145.70 seconds

Now, with your software inventory complete and your CPE strings in hand, you can start querying for CVE's!

Sticking with our Acrobat and FoxIT examples, we can search for all CVE's -  to do this we'll change the API URI called slightly, all the URI conventions stay consistent:

https://services.nvd.nist.gov/rest/json/cves/1.0?cpeMatchString=cpe:2.3:a:adobe:acrobat_reader_dc:20.*

Now things start to look interesting! Of course, given our search this gets us a boatload of CVE's.  Let's look for the last 60-0ish days - we'll querying "modified since a Nov 1, 2020":

https://services.nvd.nist.gov/rest/json/cves/1.0?modStartDate=2020-11-01T00:00:00:000%20UTC-05:00&cpeMatchString=cpe:2.3:a:adobe:acrobat_reader_dc:20.*

Keep in mind to that there are a few dates involved- the publishedDate and lastModifiedDate.  While the NVD examples typically go after "last modified", I've seen CVE's from the mid 2000's with a "last modified" date in 2019 - so be careful.  Normally I'm querying using "publishedDate".

Again, rinse and repeat for all of your CPE's collected.  We're still playing in the browser, but you can see it's a simple query, and the return values are in JSON, so things look simple.  Except that as things nest and nest, what looks simple on the screen isn't necessarily simple in code.  For instance, to get the CVE numbers for the Acrobat Reader DC 20 query above, the PowerShell code looks like:

$request = "https://services.nvd.nist.gov/rest/json/cves/1.0?modStartDate=2020-11-01T00:00:00:000%20UTC-05:00&cpeMatchString=cpe:2.3:a:adobe:acrobat_reader_dc:20.*" $CVES = (invoke-webrequest $request | ConvertFrom-Json).result.CVE_items.cve.CVE_data_meta.id $CVES CVE-2020-24439 CVE-2020-24438 CVE-2020-24436 CVE-2020-24434 CVE-2020-24426 CVE-2020-24437 CVE-2020-24435 CVE-2020-24433 CVE-2020-24432 CVE-2020-24431 CVE-2020-24430 CVE-2020-24429 CVE-2020-24428 CVE-2020-24427

While this is starting to get useful, keep in mind that it's only as good as the database.  For instance, looking at any particular product, writing it will involve the use of a language and likely some libraries, maybe a framework and some other components.  There isn't a team of elves drawing up that heirarchy for public comsumption anywhere.  We can only hope that the dev team who wrote the thing is keeping that list for their own use! (hint - lots of them are not).

You can deconstruct any particular application to some degree ( https://isc.sans.edu/forums/diary/Libraries+and+Dependencies+It+Really+is+Turtles+All+The+Way+Down/20533/ ), but if you go that deep, you'll find that your list will likely see components get added or removed from version to version - it'll turn into a whack-a-mole-science-project in no time!
For a point-in-time application pentest this can be useful, but to just keep tabs on how your security posture is from week to week, you'll likely find yourself spending more time on application archeology than it's worth.

All is not lost though - as we've seen above we can get useful information just at the application level - and keep in mind the CVE list is what your scanner uses as it's primary source, also your red team, your malware, and any "hands on keyboard" attackers that your malware escorts onto your network.  So while the CVE list will never be complete, it's the data all of your malicious actors go to first.

Let's dig deeper into this - with our inventory of CPE's now solid, tomorrow we'll play more with the API details the CVE level, and Monday there'll be a full working app (I promise!) that you can use in your organization.

==== of note ====

In the course of me playing with this API, I ended up opening a case with the folks at the NIST, they were very responsive (2 day turnaround from initial question to final answer), which for me is very responsive, especially for a free product!

===============
References:

Full docs for CVE API are here:
https://csrc.nist.gov/publications/detail/nistir/7696/final
https://csrc.nist.gov/CSRC/media/Projects/National-Vulnerability-Database/documents/web%20service%20documentation/Automation%20Support%20for%20CPE%20Retrieval.pdf

===============
Rob VandenBrink
rob@coherentsecurity.com