Uploaded April 2025 | Updated September 2026, 1 week ago
Check out our recording from our hack::soho meetup that took place 30 January. This talk, 'RAGs to Reqs,' was presented by guest speaker Irene Michlin, Application Security Lead at Neo4j. The abstract of the talk can be found below!
Learn more about IOActive by visiting our website: ioactive.com
ABSTRACT
ASVS is awesome! At the same time it contains 200+ requirements. Even after localising it for your context, it’s likely to have 100+ relevant requirements. Can we - the security team - ask the developers to go through this list for every feature?
We can, but how likely it is to happen in a modern DevSecOps environment, and what will be the quality of the engagement with ASVS?
(Also, spreadsheets are boring and everyone hates them). Can we do better? Yes!
Retrieval-Augmented Generation (RAG) is the process of optimising the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response. Luckily, ASVS is a very “graphy” data that lends itself well to being stored in a graph format.
Using this graph as the authoritative knowledge base, we use semantic similarity search on the feature description to look up relevant ASVS requirements, which is already very useful on its own. But passing these requirements as context to OpenAI or another LLM gives further useful results, reducing hallucinations and giving the developers specific security requirements for their feature that are based on ASVS.
Bottom line: Don’t let ASVS become a chore, let the developers have an easy initial engagement with it. They can always go deeper if needed.
The workflow we’ve achieved in our engineering org: Let the tool run on internal portal. Developers provide a feature description (a paragraph or though), the tool gives them top-10 most relevant ASVS requirements and some further recommendations based on these requirements. Lots of ideas for refining and expanding, we’ll discuss some of these.
We released as open source the prototype tool and the underlying graph database including pre-calculated vector embeddings for semantic similarity search: github.com/neo4j-examples/appsec-asvs-bot
The ideas in this talk will help AppSec engineers with their scaling and culture building efforts, and will help all the developers/builders to get some security impact in their features fast.
Check out our recording from our hack::soho meetup that took place 30 January. This talk, 'RAGs to Reqs,' was presented by guest speaker Irene Michlin, Application Security Lead at Neo4j. The abstract of the talk can be found below!
Learn more about IOActive by visiting our website: ioactive.com
ABSTRACT
ASVS is awesome! At the same time it contains 200+ requirements. Even after localising it for your context, it’s likely to have 100+ relevant requirements. Can we - the security team - ask the developers to go through this list for every feature?
We can, but how likely it is to happen in a modern DevSecOps environment, and what will be the quality of the engagement with ASVS?
(Also, spreadsheets are boring and everyone hates them). Can we do better? Yes!
Retrieval-Augmented Generation (RAG) is the process of optimising the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response. Luckily, ASVS is a very “graphy” data that lends itself well to being stored in a graph format.
Using this graph as the authoritative knowledge base, we use semantic similarity search on the feature description to look up relevant ASVS requirements, which is already very useful on its own. But passing these requirements as context to OpenAI or another LLM gives further useful results, reducing hallucinations and giving the developers specific security requirements for their feature that are based on ASVS.
Bottom line: Don’t let ASVS become a chore, let the developers have an easy initial engagement with it. They can always go deeper if needed.
The workflow we’ve achieved in our engineering org: Let the tool run on internal portal. Developers provide a feature description (a paragraph or though), the tool gives them top-10 most relevant ASVS requirements and some further recommendations based on these requirements. Lots of ideas for refining and expanding, we’ll discuss some of these.
We released as open source the prototype tool and the underlying graph database including pre-calculated vector embeddings for semantic similarity search: github.com/neo4j-examples/appsec-asvs-bot
The ideas in this talk will help AppSec engineers with their scaling and culture building efforts, and will help all the developers/builders to get some security impact in their features fast.
