Blackbox Assessment
The English user guide is currently in beta preview. Most of the documents have been automatically translated from the Japanese version. Should you find any inaccuracies, please reach out to Flatt Security.
Overview
Takumi's blackbox assessment feature receives an application's URL and authentication information, performs simulated attacks on the application, and outputs assessment results as a report on the web.
It can be used from the Shisho Cloud byGMO web interface.
Currently, this feature is not available from Slack. This is to prevent unintended attack traffic from being suddenly generated from chat instructions.
A feature that allows starting assessments and receiving reports with pre-execution content confirmation and user consent is planned for the future.
Regarding Credit Consumption
Credits are required to use this feature. Credit consumption varies depending on the characteristics and size of the assessment target.
Compared to other features, this tends to have relatively higher credit consumption. Depending on the scale of the application, it may cost around 100-200 credits or more.
If you are concerned about credit consumption, please also utilize the "Partial Assessment" mode described below.
How to Start an Assessment
Assessments can be started from the "New Assessment" button in the upper right of the screen within the "Assessment" tab.
The settings during assessment are roughly just the application URL and authentication information.
When starting an assessment, you can choose one of two modes:
- In "Full Assessment" mode, the entire application is assessed together. Crawling and assessment are performed in one seamless process.
- In "Partial Assessment" mode, Takumi stops at the end of crawling. After that, you select the features and perspectives you want to assess, then resume the assessment. This is recommended for those who want to control the assessment scope and minimize credit consumption.
Assessments take several hours to about 2 days. Please wait patiently.
Checking Assessment Results
Assessment reports can be viewed on the web, roughly as shown in the following screen.
The format is roughly similar to the output you would receive when requesting an assessment from a security vendor.
Each item in the assessment results explains which feature was assessed from what perspective, and what severity and risk vulnerabilities were found.
Re-assessment
The re-assessment feature is a function to assess only some features or some perspectives. Credit consumption varies depending on the number of features and perspectives being re-assessed.
Assessment Perspectives
Broadly speaking, Takumi reviews from the following perspectives:
- Logic flaws
- Authorization control flaws
- General injections
- SSRF
- XSS
- Clickjacking
- Open redirects
- File operation flaws
- Session management flaws
- CORS configuration flaws
- CSRF
Each major category contains various detailed assessment perspectives. The distinctive feature is that business logic vulnerabilities are particularly included in the assessment scope. Our technology utilizing LLM agents enables assessments based on application specifications.
If you want to know more details, meetings with developers are also available. Please feel free to contact us.
Accuracy Evaluation
As a result of performing assessment using only the blackbox assessment feature on a demo application with vulnerabilities embedded independently by GMO Flatt Security, through approximately 20 hours of scanning, the detection rate was 48% and the false positive rate was 33.3%.
Note that the demo application also contains vulnerabilities that are inherently difficult to discover without referring to the source code, and when excluding such vulnerabilities, the detection rate was 70%.
Source IP Addresses for Assessment
The source IP addresses for this feature are as follows.
These source IP addresses are subject to change due to provider circumstances.
34.97.228.101
13.113.43.245
18.178.91.176
54.150.4.178
54.168.46.6
54.248.85.10
143.189.213.53