Concepts

info

The English user guide is currently in beta preview. Most of the documents have been automatically translated from the Japanese version. Should you find any inaccuracies, please reach out to Flatt Security.

This document introduces the concepts behind Shisho Cloud's web application assessment feature. It specifically details the following:

• What you can achieve with the web application assessment feature
• Features of the web application assessment feature
• Threat types and vulnerability severity levels in web application assessments
• Specific inspection items and industry standard coverage
• When to use web application assessments versus manual penetration testing services

What you can achieve with the web application assessment feature

By utilizing the web application assessment feature, you can achieve the following three things:

1. Understand the risks that can negatively impact web application providers and users.
2. Understand the approach to realizing secure web applications.
3. Perform risk identification for web applications at low cost.

The details of each are as follows.

Understand the risks that can negatively impact web application providers and users

For example, if a database is accessed without authorization or a server running a web application is hijacked by an attacker, there is a risk of attacks with a significant impact on business continuity, such as the leakage of sensitive information such as personal information managed by the web application or the alteration of payment information. Additionally, if vulnerabilities exist that allow payment processing to be executed against the user's intentions or important data to be deleted, users will be disadvantaged through the application.

Shisho Cloud's web application assessment feature contributes to hardening web applications by prioritizing the detection of such high-impact security risks and security risks that could harm users.

Understand the approach to realizing secure web applications

Simply running scans and listing vulnerabilities is not enough to achieve secure web applications. It is important to implement initiatives efficiently and systematically, such as which of the detected vulnerabilities could actually lead to business risks, how much impact they are expected to have, and how to fix them.

Shisho Cloud's web application assessment feature not only focuses on vulnerabilities that lead to business risks but also strongly supports the process of investigating and fixing detected vulnerabilities through easy-to-understand reports and triage functions.

Perform risk identification for web applications at low cost

As mentioned above, the web application assessment feature strongly supports business growth and web application hardening, but services that provide such features tend to be expensive and complex.

We promise to provide this feature at a low cost. This is achieved through efforts such as in-house development of the assessment engine and detection engine and reduction of assessment costs by using appropriate assessment methods (dynamic assessment and static assessment using source code) according to the inspection items. We never provide low prices by lowering the quality of our assessments.

In addition, we provide an environment where assessments can be performed with reduced workload costs by providing a user-friendly and easy-to-understand user interface and triage functions.

To realize Flatt Security's mission of "supporting engineers," it is essential not only to provide high-quality services, but also to provide services that are easy to use by organizations of all environments. Based on this philosophy, we will continue to create a mechanism that enables high-quality assessments to be performed at low cost, both economically and in terms of workload.

Features of the web application assessment feature

With Shisho Cloud's web application assessment feature, you can focus on truly important alerts without being bothered by a large number of low-certainty alerts.

Listing many vulnerabilities and potential issues is important in web application security assessments. However, it is also true that some of the vulnerabilities and potential problems that are commonly detected are of low severity. In an environment where a large number of such low-severity alerts are reported, there is a concern that high-severity alerts that should essentially be addressed will be overlooked or addressed late.

Therefore, we believe that "simply listing vulnerabilities and potential problems and alerting all of them does not lead to an appropriate security assessment."

Therefore, Shisho Cloud's web application assessment feature prioritizes and reports vulnerabilities and potential problems that could have a significant impact on business continuity. Therefore, this feature intentionally does not report in the following cases:

• When the target web application exhibits unintended behavior when certain data is sent, but it cannot be determined that a vulnerability exists.
• When a mechanism to improve security is not used, but it is determined that the cost of implementing it is not worth the benefits.

Threat types and vulnerability severity levels

We define threat types to classify and identify vulnerabilities to be detected, and then use vulnerability severity to determine what should be addressed first.

First, threats are classified into "Information Disclosure," "Data Tampering," and "Service Disruption," which threaten each of the CIA triad ¹ of confidentiality, integrity, and availability. Note that STRIDE ², a representative threat modeling methodology, also defines "Spoofing," "Elevation of Privilege," and "Repudiation" in addition to these, but all of these eventually result in one of the above threats (e.g., user information is leaked by spoofing, unintended data is written by elevation of privilege, etc.), so they are not included in the classification here.

Threat type	Description
Information Disclosure	Information managed by the web application is unintentionally disclosed.
Data Tampering	Information managed by the web application is unintentionally tampered with.
Service Disruption	The service provided by the web application is stopped. Alternatively, the response is significantly delayed.

The severity of a vulnerability is classified into the following three levels. These classifications are based on the evaluation metrics defined in Flatt Security's security assessment service, and refer to the framework of Damage Potential and Affected Users in the DREAD model.

Severity	Description
High	Vulnerabilities that could have a significant impact on business continuity.
Medium	Vulnerabilities that could have an impact on business continuity.
Low	Vulnerabilities that could have a minor impact on business continuity. Or, misconfigurations and other issues that could facilitate the occurrence of a vulnerability or its impact.

Then, we identify specific vulnerabilities and issues that correspond to the combination of these elements.

Below are some examples.

• Information Disclosure x High: Information in the database used by the web application is leaked due to SQL injection.
• Information Disclosure x Medium: Information such as version control files is unintentionally disclosed in a specific path.
• Information Disclosure x Low: Introspection queries are allowed in GraphQL API.
• Data Tampering x High: Arbitrary files on the server used by the web application are tampered with by OS command injection.
• Data Tampering x Medium: CSRF allows a legitimate user to request unintended changes.
• Service Disruption x High: Requests using regular expressions that take a huge amount of time to match cause the entire web application to stop, making it impossible to provide the service.
• Service Disruption x Medium: GraphQL queries that perform circular references increase the load on the web application and delay the provision of services.

The web application assessment feature provides dynamic and source code inspection based on inspection items that Flatt Security has determined to be necessary. When deciding on the inspection items to be provided, we utilize our know-how and experience as a security company that provides professional services (manual penetration testing) in addition to the above process.

In summary, we provide the optimal inspection to be able to explain that the target web application is secure.

Enhancing assessment capabilities by leveraging our expertise as a security firm

Flatt Security's manual penetration testing goes beyond simply investigating typical and known vulnerabilities; it may also employ recently discovered or unique attack techniques.

By defining such attack techniques as inspection items, we are leveraging the know-how gained from our professional services to enhance our web application assessment capabilities.

Inspection items and industry standard coverage

The inspection items are determined based on the above premise, as well as the inspection items defined by the security assessment service provided by Flatt Security and the following industry standards:

• OWASP Application Security Verification Standard 4.0.3
• OWASP Top Ten (2021)
• IPA How to Create a Secure Website, 7th Edition - Security Implementation Checklist

Specific inspection items are determined based on our evaluation criteria, and their content does not necessarily cover all items defined by industry standards such as those listed above. This is because the scope they cover includes items that are difficult to mechanically judge because they depend on the context of the application and the development organization, and items that do not necessarily pose a business risk.

In other words, the inspection items provided by the web application assessment feature include items that we have determined to be important among the inspection items defined by various industry standards.

See Managed Inspections for Web Applications to check the inspection items currently available.

When to use web application assessments versus manual penetration testing services

In conclusion, for web applications that handle sensitive information such as personal information and web applications that provide services that are important to the business, we strongly recommend that manual penetration testing be conducted in addition to using the web application assessment feature, as the residual risk is often considered unacceptable if only the web application assessment feature is used.

First, the types of web application assessments can be broadly classified as follows.

Automated security assessments

• Refers to methods that use only application scanners, source code analysis tools, etc.
• While it is inexpensive and can be performed continuously, there are disadvantages such as false positives and aspects that cannot be inspected. In addition, the impact and reproducibility of the existence of vulnerabilities are not evaluated.
• As a result, vulnerabilities existing in web applications are identified.
• Shisho Cloud's web application assessment feature also falls into this category.

Manual security assessments

• Refers to a method in which an assessor reviews the results obtained by running the above tools and evaluates in more detail whether or not vulnerabilities exist, and if so, their impact and reproducibility.
• While false positives can be suppressed, the detection rate and inspectable aspects depend on the performance of the tools.
• As a result, vulnerabilities existing in web applications and business risks caused by the vulnerabilities are identified.

Advanced manual security assessments

• Refers to a method in which an assessor conducts an inspection at the source code level based on an understanding of the application's specifications and business logic, in addition to running the tools and reviewing the results.
• It is possible to identify vulnerabilities unique to the web application, such as access control deficiencies and deviations from specifications, that are difficult to detect with tools.
• As a result, vulnerabilities existing in web applications and business risks caused by the vulnerabilities are identified.
• The security assessment service provided by Flatt Security falls into this category.

Summary

As mentioned above, the web application assessment feature cannot identify and assess all vulnerabilities that could negatively impact the business or users. This is because some vulnerabilities cannot be detected without scrutinizing the application's source code and architecture, or can only be assessed by someone who understands the application's specifications. Here are some examples.

• For example, suppose there is an API accessible via the Internet that allows retrieval of the names of all users registered with the application.
  • If the API is part of an application where the names of all users are published as a specification, such as an SNS, it is intentional and not a vulnerability.
  • On the other hand, if it exists in an application where it is not desirable to publish the names of all users as a matter of specification, such as a multi-tenant application, it is appropriate to judge it as a vulnerability.
• For example, suppose a SQL injection vulnerability exists in a function in the source code, but a validation mechanism in the code path to that function blocks malicious input values.
  • In an advanced manual assessment, the source code, dependent libraries, etc. are scrutinized to investigate whether there is a way to bypass the validation mechanism and attack.
  • On the other hand, the web application assessment feature sends requests containing attack data to the target web application and determines that a vulnerability exists if certain behavior is confirmed. In this case, such requests would be blocked by the validation mechanism, so the feature would determine that no vulnerability was found.

Which assessment type should be adopted cannot be determined in general, and it is necessary to make a judgment each time in light of the confidentiality level of information managed by the web application, the impact on the business if an attack occurs, etc. However, especially for web applications that handle sensitive information such as personal information and web applications that provide services that are important to the business, it is preferable to conduct advanced manual penetration testing periodically while continuously conducting automated assessments.

Please note that Flatt Security also offers plans that flexibly combine advanced manual penetration testing by security engineers with Shisho Cloud's web application assessment feature. For details, please contact support.

Footnotes

1. NIST - NIST SP 1800-26A (https://www.nccoe.nist.gov/publication/1800-26/VolA/index.html) ↩

2. OWASP - Threat Modeling Process (https://owasp.org/www-community/Threat_Modeling_Process#stride) ↩