Calculating CVSS Scores with ChatGPT

Published: 2023-04-25
Last Updated: 2023-04-25 13:58:35 UTC
by Johannes Ullrich (Version: 1)
5 comment(s)

Everybody appears to be set to use ChatGPT for evil. After all, what is the fun in making the world a better place if, instead, you can make fun of a poor large large-scale language model whose developers only hinted at what it could mean to be good?

Having not given up on machines finally taking over to beat the "humane" into "humanity," I recently looked at some ways to use ChatGPT more defensively.

An issue I have been struggling with is vendors like Apple providing very terse and unstructured vulnerability summaries. You may have seen my attempt to create a more structured version of them and to assign severities to these vulnerabilities. Given that there are often dozens of vulnerabilities and limitations of my human form, the severity I assign is more of a "best guess." So I figured I would try to automate this with ChatGPT, and the initial results are not bad. 

For example, let's take the last Apple vulnerability, CVE-2023-28206. This was an already exploited ("0-Day") privilege escalation vulnerability. 

Chat GPT delivers the following analysis:

Given the limited information, I think a score of 8.8, and the analysis, isn't bad. Personally, I would have rated it probably a bit lower. There is no network access here (I think). But it is "close enough".

I will probably add this to my Apple vulnerability parser and use this the next time Apple releases an update :)

 

 

---
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
 

 

Keywords:
5 comment(s)

Comments

Thanks for this interesting use case for CHATGPT. As a new vulnerability is published would a CHATGPT CVSS score converge to a stable consensus score as additional formation or analysis is published? How quickly would the CHATGPT score “adapt” to new information? Did you need to “train” your model?
Thanks for this interesting use case for CHATGPT. As a new vulnerability is published would a CHATGPT CVSS score converge to a stable consensus score as additional formation or analysis is published? How quickly would the CHATGPT score “adapt” to new information? Did you need to “train” your model?
Thanks for this interesting use case for CHATGPT. As a new vulnerability is published would a CHATGPT CVSS score converge to a stable consensus score as additional formation or analysis is published? How quickly would the CHATGPT score “adapt” to new information? Did you need to “train” your model?
I shared this story with Chat GPT 4 and it addressed Kevin's question. Here is it's response:
Thank you for bringing this to my attention! It's great to see that my capabilities can be used to help with tasks like calculating CVSS scores for vulnerabilities. While I'm not perfect, I strive to provide useful and relevant information based on the data available.

As for your questions, the ChatGPT's ability to adapt to new information would depend on the availability of that information and the context provided during the conversation. If there is new or updated information regarding a vulnerability, you can mention it, and I'll try to incorporate that into my analysis.

Regarding training, I'm based on OpenAI's GPT-4 model, which has been trained on a vast amount of text from the internet. However, I haven't been specifically trained on CVSS scores or vulnerability analysis. My responses are based on the patterns and knowledge I've learned during my training, and I try to provide the best possible answer given the context and information provided.

It's important to note that, as an AI language model, my knowledge is limited to the text I've been trained on, and I'm not always aware of the latest updates or developments. My knowledge is up to date as of September 2021. It's always a good idea to verify the information and scores with other sources, particularly when dealing with critical security-related issues.

Chat GPT 4 on 11:06 AM 4/26/2023 PST
Please take a look at this: https://www.linkedin.com/events/25yearsofapisecurity-acomprehen7047575727730286592/comments/

we did this a few weeks ago at 25 years scale

Diary Archives