Large language models in security
In the last couple weeks, a few interesting products have been launched applying large language models to security workflows. Socket AI uses GPT4 to analyze and summarize potential vulnerabilities it in open source repositories, and Microsoft Security CoPilot uses a chat like interface to aid in incident response workflows (e.g. you can ask the AI about a newly published vulnerability and it’s potential impact on your systems).
I suspect there are a wide range of other applications of LLMs in security, primarily because so much of security is either a search problem (e.g. logs analysis) or a synthesis/parsing problem (e.g. data loss prevention). Language models are very good at both of these things, especially on the types of semistructured datasets that are quite common in security.
Below is an initial list of some areas where I imagine LLMs could be applied or integrated within security:
SIEM
Products like Panther, Splunk, and Exabeam assist with logs analysis for security operations centers. There are two possible applications of LLMs in this context:
Detection - Some companies may attempt to augment rule-based detections with semantic detections. For example, rather than write a rule that checks whether the userIdentity field in a cloudtrail event = “root”, one may be able to prompt an LLM to find all events where root is used to access an AWS resource. Realistically, I think that cost, latency, and determinism/accuracy will be barriers to this becoming widespread any time soon, but it may be possible. Products like Hunters which have traditionally relied on more automated detection techniques may be more likely to explore things like this.
Incident Response - Once an alert is identified, LLMs might be used to do things like explain or enrich an alert. What other events were connected? What was the lineage of access? What is the potential implication of this breach? Language models are likely to be very good at systematizing this type of analysis, which is often very manual today, ultimately lowering the time to remediation. This use case feels extremely compelling to me.
Static Analysis
Traditionally, tools in cloud security and application security have used a range of parsing and linting techniques to identify possible vulnerabilities. Abstract syntax tree analysis is common in SAST tools, and products like Clair apply static analysis to infrastructure configuration.
It is likely that language model based analysis of code and config will greatly alter how much of this space works, and may very well render obsolete most traditional techniques. SAST may become much more commoditized as a result.
Email Security & Phishing
A lot of the email security market is essentially intelligent static analysis of email content. For the same reason LLMs may be useful in code analysis, they are likely to be very powerful in email analysis.
Multimodal models may be particularly interesting to consider in email security given that image analysis is a very important technique in email security. Many email security products look for disconnects between the domain an email is coming from and the images in the email - e.g. if the image looks like the Microsoft logo but the email is clearly not from Microsoft.
SOAR
Products like Tines help automate remediation as incidents & vulnerabilities are discovered, and are essentially vertical specific low-code automation tools analogous to Retool in terms of workflow.
LLMs are an interesting primitive building block to be offered in systems like this because they allow a much broader range of automations to be done, especially on unstructured or semistructured data. I suspect most tools in this category will include LLM building blocks, similar to how Zapier and Retool are pushing in this direction.
Data loss prevention & Data Security
Many product categories in security essentially involve using various heuristics and NLP techniques to identify private data in unstructured systems. For example, products like Nightfall help identify private data in Dropbox files.
While I don’t think simple parsing techniques will go away, LLMs will likely allow these tools to generalize much better, especially in situations that go beyond a social security number or other common form of PII being directly written down somewhere.
Remediation
LLMs are very good at writing code, and most of security remediation ultimately comes down to writing code to patch or update a system.
I suspect it will become very common for security products to no longer end at the ticket creation stage, but to in fact go one step further and produce an initial pull request based on the vulnerability at hand and the code base in question via LLMs. Many of the most successful security companies, like Snyk, took off largely due to their ability to automate remediation via pull requests. I believe LLMs have likely increased the number of situations where this can be done by 10x.
A corollary of this is that LLMs may enable less technical security teams to become closer to “security engineers”, the predominant security persona seen in more tech-forward companies.
Translation between security and everyone else
A classic issue that applies to essentially every single product in security is translating security speak to something the rest of the company can understand. A lot of the conflict between engineering and security organizations stems from engineering often not understanding why a vulnerability that has been assigned to them matters. The output of many security tools is unintelligible to the average engineer.
Language models are extremely good at translation tasks, and I suspect could be immensely useful to take vulnerability information and make it legible or understandable to the typical person.
Secure defaults and engineer-oriented security workflows
It may now be much easier to build “CoPilot” like functionality for engineers which nudges them towards writing secure-by-default code as they develop. This is in many respects a combination of a lot of the above ideas. If analysis is easier, security findings can be more easily explained & contextualized, and security policy can be defined in a more generalized way thanks to language models, perhaps in combination these trends allow security products to target engineers in a much more direct way than before.
Vendor Security Questionnaires & Compliance
It is likely now possible to almost fully automate the vendor security questionnaire process. Many other areas of compliance can likely be re-imagined as well.
_
I am sure I am missing a lot. Let me know if you have other ideas of how LLMs might be applied in security.
Thanks to Dhruv Iyer, Feross Aboukhadijeh, and Travis McPeak for thoughts/feedback on this