Getting the Most Out of Takumi
The English user guide is currently in beta preview. Most of the documents have been automatically translated from the Japanese version. Should you find any inaccuracies, please reach out to Flatt Security.
The most important point to keep in mind when interacting with Takumi is to reduce the scope it needs to investigate. However, this is much easier said than done, even for security experts. In this document, we explain how anybody can get the most out of Takumi.
Good Example Prompts
First, let's look at a prompt that is not very effective.
Investigate for vulnerablities.
Takumi will still manage to output a report, but it may have trouble picking up critical issues and the quantity of minor findings will be fairly high.
Let's reduce Takumi's scope of investigation by specifying the feature in question.
Investigate for vulnerabilities in the user purchase history feature. The feature is implemented in path/to/directory.
With the feature specified and even a file path included, Takumi becomes much more focused and the output quality will be dramatically better.
We’d like you to investigate vulnerabilities related to authentication. If such a vulnerability exists, it may be possible for a malicious user to take over another user’s account, resulting in a severe security incident. In particular, please be very attentive to the sign up, login in, and password reset features.
By specifying the type of vulnerability, Takumi likewise becomes much more focused and can produce a much higher quality output.
How to Actually Write Good Prompts
So for we have showed you what a good prompt looks like. From here, we will discuss strategies so that anybody can write good prompts.
Starting Abstractly
Although we explained the importance of being specific with your prompts, it can be very difficult to immediately be able to write such prompts.
One strategy, is to start with a high overview prompt, such as asking Takumi to outline the features of the application, and then based on the results dig into the specifics.
Let's look at an example that starts with outlining the features of an application.
We’d like to conduct a security review on the application. First, please list up the features offered by this application and any common security concerns associated with each of them.
Let's suppose a file upload feature which manages user data was included in the report. Becasue user data may include confidential information, it may be a good idea to focus here. Now it is much easier to write a prompt with higher specificity.
Investigate for vulnerabilities in the file upload feature. Because this feature manages sensitive user data, be extra careful. In particular, look out for cases where personal data such as names and addresses can be leaked.
You could also go in a different direction and ask about the vulnerabilities related to file uploads.
It seems the file upload feature has several common vulnerabilities. Please explain each of them in detail, and provide examples of potential risks associated with our application.
These examples specifically dealt with starting with an abstract prompt. However, this method is also useful as follow up questions even if the original prompt was fairly specific.
In this example, we start with the feature and file path specified.
Regarding the review posting feature, investigate for vulnerabilities. The implementation is in path/to/file.
Let's say there were multiple vulnerabilities reported in the output. By asking a followup question, we can dig deeper into the vulnerability.
It seems the review postintg feature contains an XSS vulnerability. Explain the risks associated with this. The implementation is in path/to/file.
By repeatedly asking follow up questions, you will be able to ask questions that allow Takumi to output high quality responses.
Identifying the Objective
By making the objective clear, it becomes easier to write good prompts with high specificity.
For example, let's suppose there is a release coming up. In this case, the objective is to have a release without critical issues. By also considering the content of the release, the objective becomes even more precise.
When the objective is clear, the details become easy to write.
The pull request, __url__, adds new logic to the authentication functionality. Investigate this pull request for new vulnerabilities regarding authentication.
The pull request, __url__, modifies data base queries. Investigate this pull request for new vulnerabilities regarding data base access.
Scheduling Tasks
Scheduled tasks can be set for Takumi. Here we introduce how you can check the previous day's code changes everyday at midnight.
Cron settings：0 0 * * *
Prompt：
Conduct a security review based on the commits from the last 24 hours.
- Before you start the review, ensure the default branch (main) is checked out. Make sure the branch is in the newest state. If it cannot be updated, state the reason in the report and continue.
- For the commits from the last 24 hours, investigate for new vulnerabilities or concerns regarding the specification. There is no need to report fixed vulnerabilities.
Other uses
Here we introduce other uses for Takumi.
To investigate the risks and severity of a discovered vulnerability:
Regarding the review posting feature, it turns out there are flaws in the CSRF token logic. Investigate the risks and severity. The implementation is in path/to/file.
To investigate whether a vulnerability discovered in a particular feature is present in other features as well:
Regarding the review posting feature, it turns out an SQL injection is possible. Investigate whether the same vulnerability exists in other features as well.
To ask about a technology you are considering using:
We are considering using GraphQL. What should we be concerned about regarding the transfer of confidential information?
To confirm about the safety of a library:
This application uses JWTs. The JWT processing uses the xxxxx library. The implementation is in path/to/file. Check for vulnerablities.
To understand how to reproduce a potential vulnerability:
An account takeover vulnerability was discovered in the authentication functionality. Investigate the steps required to reproduce this vulnerability.