Meta DPA

As a part of Meta's Data Protection Assessment, you may need to collect evidence on your cloud security posture management implementation with Shisho Cloud. This document guides how to collect evidence on cloud security audits for Meta's Data Protection Assessment.

note

This document is not officially authorized by Meta. Please use this document as a reference when you prepare for Meta's Data Protection Assessment. You may find this useful for explaining how Shisho Cloud works for your cloud data protection to your assessor.

Overview

By following the steps (A) - (F) below, you'll see basic information on your cloud security posture management implementation with Shisho Cloud:

$ EXPORT ORG_ID=flatt-security

# (A) Show all reviews are enabled
####################
$ shishoctl workflow list --org $ORG_ID | jq -r

# (B) Demonstrate that workflows are running properly
####################
$ shishoctl run list --org $ORG_ID | jq -r

# (C) Demonstrate that integrations between Shisho Cloud and cloud accounts to process Platform Data are available now
####################
$ shishoctl integration aws list --org $ORG_ID | jq -r
$ shishoctl integration googlecloud list --org $ORG_ID | jq -r

# (D) Demonstrate that cloud accounts to process Platform Data are covered
####################
$ shishoctl cloud-account list --org $ORG_ID | jq -r

# (E) Demonstrate the number of resources observed so far and their security posture
####################
$ shishoctl finding aggregate per-resource-kind --org $ORG_ID | wc -l
# Note: If you want to see your compliance report only for a specific framework/benchmark, one against CIS benchmarks for example, you can run the following instead:
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:aws/cis-benchmark/v1.5.0" --org $ORG_ID | wc -l
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:googlecloud/cis-benchmark/v1.3.0" --org $ORG_ID | wc -l

# (F) Demonstrate that there are no active critical severity findings; One can run the similar command to collect high severity findings (SeverityHigh) as well.
####################
$ shishoctl finding aggregate per-resource-kind --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'
# Note: If you want to see your compliance report only for a specific framework/benchmark, one against CIS benchmarks for example, you can run the following instead:
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:aws/cis-benchmark/v1.5.0" --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:googlecloud/cis-benchmark/v1.3.0" --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'

Details

(A) List of enabled workflows

The following command will show the list of enabled workflows:

# (A) Show all reviews are enabled
####################
$ shishoctl workflow list --org $ORG_ID | jq -r

Here's an example output:

[
  {
    "id": "notification-security",
    "snapshotID": "WS01H1SGHWKRKCWNPRNMXAQX16BP",
    "manifest": {
      "id": "notification-security",
      "version": "0.1.0",
      "name": "Prebundle: Notify important security events"
    }
  },
  // snipped...

flatt-security/shisho-cloud-managed-workflows includes a set of workflows that are enabled by default. It includes ones for CIS benchmarks, AWS Foundational Security Best Practices, and Google Cloud Security Command Center (SCC) findings, etc.

DPA assessors may find this useful to understand the scope of the assessment. Each workflow in the GitHub repository above includes prebundle workflows, which are a set of workflows that most organizations require for their security posture. If all manifest IDs in the repository like "prebundle-aws-config" exist in the command output, it means the evidence shows that all prebundle workflows are enabled.

(B) Recent workflow runs

The following command will show the recent workflow runs:

# (B) Demonstrate that workflows are running properly
####################
$ shishoctl run list --org $ORG_ID | jq -r

Here's an example output:

[
  {
    "id": "WH01HCTCWPCPNW318JG0RSJ62PTD",
    "workflowId": "new-workflow-2",
    "workflowSnapshotId": "WS01HARP70FMPK8DWV32Y1NZ84A1",
    "startedAt": "2023-10-15T19:27:01Z",
    "runningState": "SCompleted",
    "cause": "IScheduled"
  },
  // snipped....

DPA assessors may find this useful to confirm the workflows are running properly. If the command output reports workflows are running properly, and other integration status is also okay, it means the evidence shows that enough data is collected and reviewed actively to assess your cloud security posture.

(C) Integration Health

The following command will show the status of integrations between Shisho Cloud and cloud accounts to process Platform Data:

# (C) Demonstrate that integrations between Shisho Cloud and cloud accounts to process Platform Data are available now
####################
$ shishoctl integration aws list --org $ORG_ID | jq -r
$ shishoctl integration googlecloud list --org $ORG_ID | jq -r

Here's an example output:

[
  {
    "federationId": "AWSOIDC01GXBAY25BGGK741PTHH66HA9S",
    "availability": {
      "available": true,
      "unavailableReason": null
    },
    "body": {
      "__typename": "AWSRoleOIDCFederationBody",
      "roleArn": "arn:aws:iam::065248681517:role/shisho-cloud-integration",
      "type": "SINGLE_ACCOUNT"
    }
  },
  // snipped ...

DPA assessors may find availability field useful to understand integration between Shisho Cloud and your cloud accounts to process Platform Data are working properly.

(D) Integrated Scopes

The following command will show cloud accounts (e.g. Google Cloud projects, AWS accounts, etc.) observed so far:

# (D) Demonstrate that cloud accounts to process Platform Data are covered
####################
$ shishoctl cloud-account list --org $ORG_ID | jq -r

Here's an example output:

{
  "pageInfo": {
    "totalEntries": 240
  },
  "entries": [
    {
      "id": "aws-account|779392188153",
      "displayName": "779392188153",
      "highestSeverity": "SeverityCritical",
      "actionNeededDecisions": 124,
      "__typename": "AWSAccountResource"
    },
  // snipped...

DPA assessors may find this useful to understand the scope of the assessment. The list of accounts should include ones to process Platform Data.

(E) Findings per resource type or per compliance item

The following command will show the number of resources observed so far and their security posture:

# (E) Demonstrate the number of resources observed so far and their security posture
####################
$ shishoctl finding aggregate per-resource-kind --org $ORG_ID | wc -l
# Note: If you want to see your compliance report only for a specific framework/benchmark, one against CIS benchmarks for example, you can run the following instead:
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:aws/cis-benchmark/v1.5.0" --org $ORG_ID | wc -l
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:googlecloud/cis-benchmark/v1.3.0" --org $ORG_ID | wc -l

The first command, shishoctl finding aggregate per-resource-kind --org $ORG_ID, will show the number of resources observed so far and their security posture. For instance:

{
  "entries": [
    {
      "kind": "aws-account",
      "count": 4,
      "decisions": 123,
      "decisionGroups": 31,
      "allowedDecisions": 24,
      "deniedDecisions": 99,
      "neededActions": 85,
      "highestSeverity": "SeverityHigh"
    },
    {
      "kind": "aws-cloudfront-distribution",
      "count": 6,
      "decisions": 36,
      "decisionGroups": 6,
      "allowedDecisions": 15,
      "deniedDecisions": 21,
      "neededActions": 20,
      "highestSeverity": "SeverityCritical"
    },
    // snipped ...

Each field gives the following information:

kind: the resource type
count: the number of scanned resources with that resource type
decisions: the number of decisions for that resource type in Shisho Cloud.
- A decision here is what represents whether this resource is compliant with a specific compliance item or not.
- For simplicity, you can think of a decision as a review result of one resource with one viewpoint. If an S3 resource is reviewed with two CIS benchmark items, for example, there will be two decisions for that S3 resource.
decisionGroups: the number of decision groups for that resource type.
- A decision group here is what aggregates decisions with their type.
- For simplicity, you can think of a decision group as a point of review. If some S3 resources are reviewed with two CIS benchmark items, for example, there will be two decision groups counted for resource type S3.
allowedDecisions: the number of allowed decisions for that resource type
- An allowed decision is a decision that represents that the resource is compliant with a specific compliance item.
deniedDecisions: to the number of denied decisions for that resource type
- A denied decision is a decision that represents that the resource is not compliant with a specific compliance item.
- If this value is greater than 0, it means there are some resources that are not compliant with some compliance items. However, it does not necessarily mean that there are some findings that require your attention. For more information, see neededActions and highestSeverity.
neededActions: the number of needed actions for that resource type
- Shisho Cloud users can acknowledge findings that are not relevant to their cloud security posture. For example, if an S3 bucket is set as public intentionally, users can legitimately acknowledge a finding that reports S3 bucket is public.
- The number of needed actions is the number of decisions that are denied and not acknowledged.
- If this value is greater than 0, it means there are some findings that require your attention.
highestSeverity: the highest severity among needed actions for that resource type

(F) Needed Actions per resource type or per compliance item

If the following command returns any output, it means that there are some findings that require your attention. Otherwise, it means that there are no findings that require your attention.

# (F) Demonstrate that there are no active critical severity findings; One can run the similar command to collect high severity findings (SeverityHigh) as well.
####################
$ shishoctl finding aggregate per-resource-kind --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'
# Note: If you want to see your compliance report only for a specific framework/benchmark, one against CIS benchmarks for example, you can run the following instead:
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:aws/cis-benchmark/v1.5.0" --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'
$ shishoctl finding aggregate per-annotation-value "decision.api.shisho.dev:googlecloud/cis-benchmark/v1.3.0" --org $ORG_ID | jq -r '.entries[] | select(.highestSeverity == "SeverityCritical")'

Comparison with other tools

SecurityHub evidence

Meta DPA provides the following example steps to collect evidence of cloud security monitoring implementation with SecurityHub:

# (1) Show that AWS Foundational Security Best Practices are enabled
$ aws securityhub get-enabled-standards
{
    "StandardsSubscriptions": [
        {
            "StandardsSubscriptionArn": "arn:aws:securityhub:us-west-1:043954759379:subscription/aws-foundational-security-best-practices/v/1.0.0",
            "StandardsArn": "arn:aws:securityhub:us-west-1::standards/aws-foundational-security-best-practices/v/1.0.0",
            "StandardsStatus": "READY"
        }
    ]
}

# (2) Show that aggregator is configured for a representative region used to process Platform Data
$ aws securityhub list-finding-aggregators

$ aws securityhub get-finding-aggregator --finding-aggregator-arn '{REPLACE-WITH-FINDING-AGGREGATOR-ARN}'

# (3) Demonstrate that the ruleset is running by fetching active findings and counting the number of lines of output
$ aws securityhub get-findings --query 'Findings[?RecordState==`ACTIVE`]' --filters '{"GeneratorId":[{"Value": "aws-foundational-security","Comparison":"PREFIX"}]}' --output text | wc -l

# (4) Demonstrate that there are no active critical severity findings
$ aws securityhub get-findings --query 'Findings[?Severity.Label==`CRITICAL`] | [?RecordState==`ACTIVE`] | [*][Title, GeneratorId]' --filters '{"GeneratorId":[{"Value": "aws-foundational-security","Comparison":"PREFIX"}]}'

Each step corresponds to the steps mentioned above. In detail:

(1) corresponds to (A).
(2) corresponds to (B), (C), and (D).
(3) corresponds to (E).
(4) corresponds to (F).

ScoutSuite evidence

Meta DPA provides the following example ouput as evidence of cloud security monitoring implementation with ScoutSuite:

{
  "last_run": {
    "ruleset_about": "This ruleset consists of numerous rules that are considered standard by NCC Group. The rules enabled range from violations of well-known security best practices to gaps resulting from less-known security implications of provider-specific mechanisms. Additional rules exist, some of them requiring extra-parameters to be configured, and some of them being applicable to a limited number of users.",
    "ruleset_name": "default",
    "run_parameters": {
      "excluded_regions": [],
      "regions": [],
      "services": [],
      "skipped_services": []
    },
    "summary": {
      "acm": {
        "checked_items": 4,
        "flagged_items": 2,
        "max_level": "warning",
        "resources_count": 2,
        "rules_count": 2
      },
      // ... snipped ...
    }
    "time": "2022-08-22 11:42:25-0400",
    "version": "5.11.0"
  }
}

Each field corresponds to the steps mentioned above. In detail:

ruleset_name corresponds to (A), and ruleset_about corresponds to the Markdown documentation in flatt-security/shisho-cloud-managed-workflows.
run_parameters corresponds to the output of shishoctl workflow describe <workflow name> if Shisho Cloud users have customized the workflows. Otherwise, it corresponds to the content of flatt-security/shisho-cloud-managed-workflows.
summary.<service_name>.checked_items corresponds to decisions in (E).
summary.<service_name>.flagged_items corresponds to allowedDecisions, deniedDecisions, and neededActions in (E). The closest one is neededActions in (E), because it represents the number of findings that require your attention.
summary.<service_name>.resources_count corresponds to count in (E).
summary.<service_name>.rules_count corresponds to decisionGroups in (E).
summary.<service_name>.max_level corresponds to highestSeverity in (E).
time corresponds to (B).

Meta DPA

Overview​

Details​

(A) List of enabled workflows​

(B) Recent workflow runs​

(C) Integration Health​

(D) Integrated Scopes​

(E) Findings per resource type or per compliance item​

(F) Needed Actions per resource type or per compliance item​

Comparison with other tools​

SecurityHub evidence​

ScoutSuite evidence​

Overview

Details

(A) List of enabled workflows

(B) Recent workflow runs

(C) Integration Health

(D) Integrated Scopes

(E) Findings per resource type or per compliance item

(F) Needed Actions per resource type or per compliance item

Comparison with other tools

SecurityHub evidence

ScoutSuite evidence