# Workflow Manifest

Workflows are defined by a **Manifest**, written in YAML format:

![](/docs/_md-assets/0e83c08b19-workflow.png)

By defining this manifest and creating workflows on Shisho Cloud, you can codify auditing, inspections, and subsequent actions.

## Structure

A workflow manifest consists of the following five elements:

- `version`: A constant (`0.1.0`)
- `id`: A unique ID for the workflow
- `name`: A human-readable display name for the workflow
- `triggers`: Configuration for when to execute the workflow
- `jobs`: Configuration for audits, inspections, and subsequent actions to be performed within the workflow

In other words, a workflow manifest is a YAML file that follows the format below:

```yaml
version: 0.1.0
id: "example-workflow"
name: "Example"

triggers:
  # ...

jobs:
  # ....
```

## Triggers

A workflow defines multiple **triggers** as a `triggers` block:

```yaml
version: 0.1.0
id: "example-workflow"

# Definition of multiple triggers
triggers:
  # ...
```

Each trigger defines **when the workflow should be executed**. Definitions based on time, such as "execute the trigger at regular intervals," or definitions based on events, such as "execute when a webhook message is received from another service," can be used.

More specifically, the following triggers are available:

- `schedule` trigger
- `github` trigger

### `schedule` Trigger

The `schedule` trigger uses the `cron` format to define the timing of workflow execution.
The `schedule` trigger can include one or more execution timing settings in `cron` format, and the workflow will be executed **at the time that matches at least one cron setting**.

For example, a workflow with the following trigger will be executed **every 10 minutes**:

```yaml
version: 0.1.0
id: "example-workflow"
triggers:
  schedule:
    # Run every 10 minutes
    - cron: "*/10 * * * *"
# ...
```

The following is an example of a `schedule` trigger that includes multiple `cron` settings:

```yaml
version: 0.1.0
id: "example-workflow"
triggers:
  schedule:
    - cron: "*/10 * * * *"
    - cron: "*/30 * * * *"
# ...
```

:::note
Even if multiple `cron` settings instruct the workflow to execute at the same time, the workflow execution is **only guaranteed at least once**.
:::

### `github` Trigger

The `github` trigger is used to define the timing of workflow execution using webhook messages delivered from GitHub to Shisho Cloud.
You can configure one or more workflow execution conditions based on webhook messages for the `github` trigger.
With such a configuration, the workflow will be executed **if the webhook message received from GitHub matches at least one of the settings**.

For example, a workflow with the following trigger will be executed when a **push event to the default branch of each repository** occurs:

```yaml
version: 0.1.0
id: "example-workflow"
triggers:
  github:
    - push:
        branches:
          - :default_branch
```

Here is an example of a `github` trigger that includes multiple conditions:

```yaml
version: 0.1.0
id: "example-workflow"
triggers:
  github:
    - push:
        branches:
          - :default_branch
    - push:
        branches:
          - staging
```

Of the event information that GitHub delivers through webhook messages, Shisho Cloud currently supports the following events:

- `push` events ([GitHub documentation](https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#push))
- `pull_request` events ([GitHub documentation](https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#pull_request))

#### Handling `push` Events

When controlling workflow execution based on push events, the following filters are available:

- `organization`: The GitHub Organization name to which the push destination repository belongs matches the specified string
- `name`: The push destination repository name matches the specified string
- `push.branches`: The push destination branch name matches at least one of the specified **patterns**
  - When comparing patterns and push destination branch names, the following characters/strings in the pattern have special behavior:
    - `*`: Matches one or more characters excluding `/`
    - `**`: Matches one or more characters
    - Three or more consecutive `*`:
      - Odd number of `*`: One or more characters not ending with `/`
      - Even number of `*`: Matches one or more characters
  - If the pattern exactly matches any of the following, the match verification will behave specially:
    - `:default_branch`: Instead of comparing the pattern and the push destination branch name, compare the default branch name of the repository with the push destination branch name
- `push.branches_ignore`: The push destination branch name does not match any of the specified **patterns**
  - Special characters and identifiers equivalent to `branches` can be used in the pattern
- `push.tags`: The pushed tag name matches at least one of the specified **patterns**
  - When comparing patterns and pushed tag names, the following characters/strings in the pattern have special behavior:
    - `*`: Matches one or more characters excluding `/`
    - `**`: Matches one or more characters
    - Three or more consecutive `*`:
      - Odd number of `*`: One or more characters not ending with `/`
      - Even number of `*`: Matches one or more characters
- `push.tags_ignore`: The pushed tag name does not match any of the specified **patterns**
  - Special characters equivalent to `tags` can be used in the pattern

If multiple filters are specified, they are treated as AND conditions. That is, the workflow will be executed only if all filters match.

Here are some examples of how to use the above filters:

```yaml
triggers:
  github:
    # Execute the workflow based on push events from GitHub. However:
    # - Only react to push events for octcat/Hello-World
    # - Only react to the staging branch or branches starting with feat/
    - organization: "octcat"
      name: "Hello-World"
      push:
        branches:
          - staging
          - feat/**
```

#### Handling `pull_request` Events

When controlling workflow execution based on `pull_request` events, the following filters are available:

- `organization`: The GitHub Organization name to which the push destination repository belongs matches the specified string
- `name`: The push destination repository name matches the specified string
- `pull_request.types`: The `pull_request` event type is included in the specified list
- `pull_request.branches`: The **base branch name** of the `pull_request` matches at least one of the specified **patterns**
  - When comparing patterns and the base branch name of the `pull_request`, the following characters/strings in the pattern have special behavior:
    - `*`: Matches one or more characters excluding `/`
    - `**`: Matches one or more characters
  - If the pattern exactly matches any of the following, the match verification will behave specially:
    - `:default_branch`: Instead of comparing the pattern and the **base branch name** of the `pull_request`, compare the default branch name of the repository with the **base branch name** of the `pull_request`
- `PULL_REQUEST.branches_ignore`: The **base branch name** of the `pull_request` does not match any of the specified **patterns**
  - Special characters and identifiers equivalent to `branches` can be used in the pattern

If multiple filters are specified, they are treated as AND conditions. That is, the workflow will be executed only if all filters match.

Here are some examples of how to use the above filters:

```yaml
triggers:
  github:
    # Execute the workflow based on pull_request events from GitHub. However:
    # - Only react to pull_request events for octcat/Hello-World
    # - Only react to synchronize, opened, and reopened events
    - organization: "octcat"
      name: "Hello-World"
      pull_request:
        types:
          - synchronize
          - opened
          - reopened
```

## Jobs

A workflow defines multiple **jobs**:

```yaml
version: 0.1.0
id: "example-workflow"

# Definition of multiple jobs
jobs:
  # ....
```

Each job includes:

- `id`: Job's unique ID within the workflow
- `name`: A human-readable display name for the job
- `decide`: What data to inspect/audit and how to do it
- `notify` (optional): How to notify at each phase of job execution

### `decide` Block

To define what data to inspect/audit and how to do it, a job must include a **`decide` block**:

```yaml
version: 0.1.0
id: "example-workflow"

jobs:
  - id: "..."
    name: "..."

    # decide block
    decide:
      # ...
```

#### Structure

To define **"what data"** to inspect/audit, the `decide` block must include:

- `input` block

To define **"how to inspect/audit"**, at least one of the following must be included:

- `rego` block
- `uses` block

As shown in the figure, these two elements define the target of inspection/audit (left side of the image) and the method of inspection/audit (center of the image):

![](/docs/_md-assets/2aa0a73b37-decide-block.png)

#### `input` Block

The `input` block defines the data to be inspected/audited in the form of a GraphQL query.
Shisho Cloud retrieves data for each job according to the definition of such a query and inputs the retrieved data into the policy code.

```yaml
jobs:
  - decide:
      input:
        # GraphQL query
        schema: |
          query {
            ...
          }
```

A [simple playground](https://cloud.shisho.dev/*/playground/query) is available for checking and writing GraphQL query specifications.

Here is an example of a GraphQL query to retrieve **"connected GitHub Organizations and repositories"**:

```graphql
query {
  github {
    organizations {
      login
      metadata {
        id
      }

      repositories {
        metadata {
          id
        }
        name
      }
    }
  }
}
```

#### `rego` Block

The `rego` block uses the policy language [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) to define **"how to inspect/audit"**.
A policy receives data retrieved by Shisho Cloud as defined by the GraphQL query in the job.
Then, by outputting data called multiple **Decisions**, it communicates the inspection/audit results to Shisho Cloud.
Shisho Cloud uses this communicated data to support the visualization and improvement of the security posture.

More specifically, Shisho Cloud and policies written in Rego exchange values as follows:

- Input to the policy: Given via the `input` variable on Rego
- Output from the policy: Done via the `decisions` variable on Rego

##### Inputting Data According to GraphQL Query into the Policy

Shisho Cloud uses the GraphQL query described in the `input` block in the job to retrieve data and inputs it into the policy. The retrieved data is accessible from the `input` **variable**.

For example, suppose you wrote the following query in the `input` block in a job to retrieve a list of GitHub organizations and inspect/audit them:

```graphql
query {
  github {
    organizations {
      login
      requiresTwoFactorAuthentication
    }
  }
}
```

In this case, an object like the following is stored in a variable called `input` that can be accessed from within the Rego policy:

```json
{
  "github": {
    "organizations": [
      {
        "login": "octcat",
        "requiresTwoFactorAuthentication": true
      },
      {
        "login": "your-org-name",
        "requiresTwoFactorAuthentication": false
      }
    ]
  }
}
```

Therefore, by writing the following in the Rego policy, you can get the list of GitHub organizations retrieved using the GraphQL query:

```rego
input.github.organizations
```

If you want to verify the value of `requiresTwoFactorAuthentication` (a bool value indicating whether to enforce the use of two-factor authentication for members in the organization) for each GitHub organization, you can write as follows:

```rego
org := input.github.organizations[_],
allowed := org.requiresTwoFactorAuthentication == true
```

:::info
By using the [`shishoctl`](/docs/c/accessing-via-shishoctl-cli/index.md) command, you can create test input for the policy based on the GraphQL query you wrote.
This command can be used when testing Rego policies with the [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) command.

The following is an example of a command that generates test data in JSON format that can be used as input for the `jobs[].decide.rego` policy from a GraphQL query:

```sh
shishoctl testcase generate \
  --type decision \
  --org (Shisho Cloud organization ID) \
  --query-path (path to the file where the GraphQL query is stored)
```

Even if you are using something other than Rego to describe the policy, or if the policy is used in `jobs[].notify` or later, you can handle it by changing the options of the above command.
For details, check the output of the `shishoctl testcase generate --help` command.
:::

##### Outputting Results from the Policy

By setting the value of the `decisions` variable in Rego, you can record problems found as a result of inspection/audit or the fact that no problems were found. In other words, Shisho Cloud processes and records the value of the `decisions` variable in the Rego policy.

```rego
decisions := [
  ...
]

decisions[n] {
  n := ...
}
```

Here, when setting a value to the `decisions` variable from the policy, the value must be a [**Decision object**](/docs/g/api/objects/decision.md). A **Decision object** is a data object that represents **what kind of target, what kind of result, and what kind of result**, and is usually generated using the [Rego SDK](https://github.com/flatt-security/shisho-cloud-rego-libraries/tree/main/decision). Here are some examples:

```rego
decisions[d] {

  allowed := org.requiresTwoFactorAuthentication == true

  d := shisho.decision.github.org_2fa_status({
    # whether the target is allowed by this policy or not
    "allowed": allowed,
    # the target of the decision (e.g. a GitHub repository, etc.)
    "subject": org.metadata.id,
    # clues on this decision
    "payload": shisho.decision.github.org_2fa_status_payload({"enabled": org.requiresTwoFactorAuthentication}),
  })
}
```

#### `decide` Block Example

The following job definition example defines an `input` block that defines the data to be inspected/audited and a `rego` block that defines how to inspect/audit it:

```yaml
jobs:
  - name: check-with-foobar
    id: check-with-foobar
    decide:
      rego: |
        import data.shisho

        decisions[d] {
          org := input.github.organizations[_],
          allowed := org.requiresTwoFactorAuthentication == true

          d := shisho.decision.github.org_2fa_status({
            "allowed": allowed,
            "subject": org.metadata.id,
            "payload": shisho.decision.github.org_2fa_status_payload({"enabled": org.requiresTwoFactorAuthentication}),
          })
        }
      input:
        schema: |
          query {
            github {
              organizations {
                requiresTwoFactorAuthentication
                metadata {
                  id
                }
              }
            }
          }
```

#### `with` Block

The `with` block is for defining parameters to be passed to policy code. Using this feature makes it possible to create more flexible and efficient policy code.

The defined parameters can be used in both **Rego / TypeScript** policy code.

The following `with` block definition example defines the symbol name, parameter type, description, and value to be used in the policy code.

```yaml
jobs:
  - decide:
      decide: ...
      input: ...
      with:
        allowName:
          type: string
          description: skip specific instances
          value: "test-sample"
```

The following shows an example of using parameters in **Rego / TypeScript** policy code.

```rego
# example: Rego policy code
...
allow_specific_instance(instance) {

  instance.machineType == "f1-micro"
} else = contains(instance.name, input.params.allowName)
...
```

```typescript
// example: TypeScript policy code
...
if (instance.name.includes(params.allowName)) {
    return {
      ...
    }
}
...
```

When a `with` block is defined, the parameter setting UI is displayed in the Shisho Cloud workflow editor.

![](/docs/_md-assets/fd6fc3f1e0-with-block-example.png)

:::info
There are four `parameter types`, and the display of the parameter setting UI changes depending on the type.

- **string**
- **notification_target**
- **slack_channel**
- **resource_exception**

Select `string` to pass parameters to the policy code.
For other types, please refer to [**this page**](/docs/g/getting-started/optimize-your-workflows/parameter.md).
:::

### `notify` Block

To define how to notify at each phase of job execution, a job may include a **`notify` block**:

```yaml
version: 0.1.0
id: "example-workflow"

jobs:
  - id: "..."
    name: "..."

    # notify block
    notify:
      # Input definition
      input: # ...

      # Notification method definition
      rego: # ....
      uses: # ....
```

#### Structure

The `notify` block must include the following block, which defines **"the data needed to determine whether to notify or to construct the notification content"**:

- `input` block

It must also include the following block, which defines **how to determine whether to notify and how to construct the notification content**:

- `rego` block
- `uses` block

As shown in the figure, these two elements define the auxiliary data required to determine what notification to send (left side of the image) and the notification logic (center of the image):

![](/docs/_md-assets/b7175a123e-notify-block.png)

#### `input` Block

The `input` block in the `notify` block behaves the same as the `input` block in the `decide` block.
See the description of the `input` block in the `decide` block for details.

:::info
By using the [`shishoctl`](/docs/c/accessing-via-shishoctl-cli/index.md) command, you can create test input for the policy based on the GraphQL query you wrote.
This command can be used when testing Rego policies with the [Open Policy Agent (OPA)](https://www.openpolicyagent.org/) command.

The following is an example of a command that generates test data in JSON format that can be used as input for the `jobs[].notify.rego` policy from a GraphQL query:

```sh
shishoctl testcase generate \
  --type notification \
  --org (Shisho Cloud organization ID) \
  --query-path (path to the file where the GraphQL query is stored)
```

Even if you are using something other than Rego to describe the policy, or if the policy is used in `jobs[].decide` or later, you can handle it by changing the options of the above command.
For details, check the output of the `shishoctl testcase generate --help` command.
:::

#### `rego` Block

The `rego` block uses the policy language [Rego](https://www.openpolicyagent.org/docs/latest/policy-language/) to define "how to determine whether to notify and how to construct the notification content."

##### Inputting Data According to GraphQL Query into the Policy

Shisho Cloud retrieves data using the GraphQL query described in the `input` block in the job and inputs it into the policy. The retrieved data is accessible from the `input.query` **variable**.

:::info
Various values related to workflow functions, such as job execution status, are available from the `input` variable. See the [Rego Inline Policy API documentation](/docs/g/api/rego.md) for details.
:::

##### Outputting Results from the Policy

You can instruct Shisho Cloud to send notifications by setting a value to the `notifications` variable in the Rego policy.

```rego
notifications := [
  ...
]

notifications[n] {
  n := ...
}
```

Here, when setting a value to the `notifications` variable from the policy, the value must be an array of [Notification objects](/docs/g/api/objects/notification.md).
A **Notification object** is a data object that represents **what kind of target, what kind of result, and what kind of result**, and is usually generated using the [Rego SDK](https://github.com/flatt-security/shisho-cloud-rego-libraries/tree/main/notification). Here are some examples:

```rego
notifications[n] {
  target := shisho.notification.to_notification_group("id-of-notification-group")
  msg := "test"

  n := shisho.notification.new(target, msg)
}

notifications[n] {
  target := shisho.notification.to_github_issue(
      #...
  )
  msg := "test"

  n := shisho.notification.new(target, msg)
}
```

#### `notify` Block Example

The following job definition example defines an `input` block that defines the data to be inspected/audited and a `rego` block that defines how to inspect/audit it:

```yaml
jobs:
  - name: check-with-foobar
    id: check-with-foobar
    decide:
      # ...
    notify:
      input:
        schema: |
          query {
            __typename
          }
      rego: |
        package policy.github.config.notify

        import data.shisho

        notification_groups := []

        notifications[n] {
          input.running_state == shisho.job.running_state_in_queue

          ng := notification_groups[_],
          target := shisho.notification.to_notification_group(ng)

          msg := concat("", ["Shisho Cloud has started to audit the GitHub configurations! See https://cloud.shisho.dev for further information."])

          n := shisho.notification.new(target, msg)
        }
```