Skip to main content

Managing Audit Rules on GitHub

info

The English user guide is currently in beta preview. Most of the documents have been automatically translated from the Japanese version. Should you find any inaccuracies, please reach out to Flatt Security.

In the tutorial on triage, we described security issues and recommendations detected by Shisho Cloud. These detections are made using managed audit rules provided by Flatt Security.

You can edit workflows through the web console, but by versioning workflows on GitHub, you can track changes to audit rules and collaborate on them just like regular development processes.

info

From now on, we use both "workflows on Shisho Cloud" and "GitHub Actions workflows." When simply mentioning "workflows," it refers to the workflows on Shisho Cloud.

In this tutorial, we will import workflows from Shisho Cloud to a GitHub repository and then set up a mechanism using GitHub Actions to synchronize workflows in the repository with those on Shisho Cloud.

You can set up a mechanism to manage workflows with GitHub in four steps:

  1. Create a GitHub repository to store workflows on Shisho Cloud
  2. On Shisho Cloud, allow access from GitHub Actions of a specific repository
  3. Create a GitHub Actions workflow to deploy workflows to Shisho Cloud
  4. Create a GitHub Actions workflow to check differences between the workflow in the GitHub repository and on Shisho Cloud

As a result of step 3., changes to workflows in the repository will be immediately reflected in Shisho Cloud. Additionally, as a result of step 4., when workflows are edited directly from the Shisho Cloud web interface, corresponding pull requests will be created to accommodate those changes. With this setup, workflows between the GitHub repository and Shisho Cloud will always remain synchronized.

info

You are not required to issue persistent secrets in GitHub for Shisho Cloud. Shisho Cloud will use short-lived credentials that will be assigned to each GitHub Action job. For similar examples, see this article by GitHub.

Creating a GitHub Repository to Store Workflows on Shisho Cloud​

To manage all audit rules on Shisho Cloud using Git, first export the existing workflows on Shisho Cloud and incorporate them into your repository.

First, create a repository on GitHub and clone it. To export workflows from Shisho Cloud to the cloned directory, execute the following command. Please replace $SHISHO_CLOUD_ORG_ID in the command with your Shisho Cloud organization's ID.

shishoctl workflow export --structured --org $SHISHO_CLOUD_ORG_ID --path .

Once the export is complete, commit the created files and push them to GitHub.

Authorizing Access to Shisho Cloud organization from Specific Repository's GitHub Actions​

Next, we will configure settings on the Shisho Cloud side to enable logging into Shisho Cloud via GitHub Actions. First, we will create a "bot" on Shisho Cloud. A bot is an entity with access permissions to Shisho Cloud organizations, and GitHub Actions jobs will log into Shisho Cloud as this bot. Let's create the bot from the following URL.

https://cloud.shisho.dev/[oid]/settings/bots
note

Please appropriately replace [oid] with your Shisho Cloud organization ID.

Creating a bot

Once the bot is created, clicking on the bot's name will take you to the trust condition configuration panel.

Trust condition settings panel

Trust conditions are conditions that GitHub Actions jobs need to fulfill to log into Shisho Cloud as the bot in question. By providing the organization and repository name of your GitHub repository where the workflows are stored, you enable the GitHub Actions jobs associated with that repository to log into Shisho Cloud as the bot you created. After finishing the entries, click the "Save" button.

Create a GitHub Actions Workflow to Deploy Workflows on Shisho Cloud​

The next step is to create a GitHub Actions workflow to deploy the workflows on Shisho Cloud. Create a file as shown below in the repository.

.github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]
  workflow_dispatch:

permissions:
  contents: read
  id-token: write # required permission to log into Shisho Cloud

jobs:
  main:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        SHISHO_CLOUD_ORG_ID: ["flatt-security"] # FIXME: replace with your Shisho Cloud organization ID
    steps:
      - uses: actions/checkout@v3

      - name: Install shishoctl
        run: |
          SHISHOCTL_URL="https://shisho.dev/releases/shishoctl-0.8.0-x86_64-unknown-linux-gnu"
          sudo curl -L $SHISHOCTL_URL -o /usr/local/bin/shishoctl
          sudo chmod +x /usr/local/bin/shishoctl

      - name: Sign in
        uses: flatt-security/shisho-cloud-action@v1
        with:
          # FIXME: the bot-id value to be input is listed on the trust condition settings panel
          bot-id: "BTXXXXXXXXXXXXXXXXXXXXXXXXXX"

      - name: Deploy workflows
        run: shishoctl workflow apply --org "$SHISHO_CLOUD_ORG_ID" --path .
        env:
          SHISHO_CLOUD_ORG_ID: ${{ matrix.SHISHO_CLOUD_ORG_ID }}

Please replace the sections marked as FIXME appropriately. Note that the bot-id value to be input is listed at the bottom of the trust condition settings panel you created earlier.

The bot-id value is listed at the bottom of the trust condition settings panel

After creating the file .github/workflows/deploy.yml as above, commit and push it to GitHub.

With these settings, every time you push to the main branch, the workflow in the repository will be deployed on Shisho Cloud. Open the Actions tab of your GitHub repository and confirm that the Deploy workflow is running correctly.

Log of a successfully run GitHub Actions job

Creating a GitHub Actions Workflow to Check Differences between Workflows in the Repository and in Shisho Cloud​

So far, we have set up so that every time there is a push to the default branch, the workflow will be deployed on Shisho Cloud. However, if you directly edit the workflows from the Shisho Cloud console, there will be a disparity between the workflows in the repository and the one on Shisho Cloud. In order to keep the workflows managed in the GitHub repository and ones actually running on Shisho Cloud the same, you need to regularly check the differences between the two.

The shishoctl workflow pull command is to pull the workflows from Shisho Cloud to local. Using this command, let's create a GitHub Actions workflow that automatically creates a pull request if there is a difference between the workflows on Shisho Cloud and the ones in the repository.

For the GitHub Actions workflow to create pull requests automatically, you need to allow GitHub Actions to create pull requests. To check or change this setting, open the repository settings from the following URL.

https://github.com/[organization]/[repository]/settings/actions

If the "Allow GitHub Actions to create and approve pull requests" check box in the "Workflow permissions" section is not checked, check it and click the Save button.

Workflow permissions settings

If you can't check the checkbox even after clicking it, GitHub Actions to create pull requests may be prohibited at the GitHub Organization level. In that case, please open the Organization settings from the following URL and check "Allow GitHub Actions to create and approve pull requests" in the "Workflow permissions" section.

https://github.com/organizations/[organization]/settings/actions

Then, create a GitHub Actions workflow file as shown below.

.github/workflows/check-workflow-drift.yml
name: Check for drift from the workflows code on Shisho Cloud

on:
  schedule:
    - cron: "0 0 * * *" # Every day at 0:00 UTC
  workflow_dispatch:

permissions:
  contents: write # required permission to push auto-generated commits
  id-token: write # required permission to log into Shisho Cloud
  pull-requests: write # required permission to auto-create pull requests

jobs:
  main:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        SHISHO_CLOUD_ORG_ID: ["flatt-security"] # FIXME: replace with your Shisho Cloud organization ID
    steps:
      - uses: actions/checkout@v3

      - name: Install shishoctl
        run: |
          SHISHOCTL_URL="https://shisho.dev/releases/shishoctl-0.8.0-x86_64-unknown-linux-gnu"
          sudo curl -L $SHISHOCTL_URL -o /usr/local/bin/shishoctl
          sudo chmod +x /usr/local/bin/shishoctl

      - name: Sign in
        uses: flatt-security/shisho-cloud-action@v1
        with:
          # FIXME: the bot-id value to be input is listed on the trust condition settings panel
          bot-id: "BTXXXXXXXXXXXXXXXXXXXXXXXXXX"

      - name: Pull workflows from Shisho Cloud
        run: shishoctl workflow pull --org "$SHISHO_CLOUD_ORG_ID" --path . --structured | tee "$OUTPUT_PATH"
        env:
          SHISHO_CLOUD_ORG_ID: ${{ matrix.SHISHO_CLOUD_ORG_ID }}
          OUTPUT_PATH: ${{ runner.temp }}/output.txt

      - name: Generate PR description
        run: |
          echo 'The workflows on Shisho Cloud has changed from those in this repository.

          > [!NOTE]  
          > This PR was automatically created by `.github/workflows/check-workflow-drift.yml`.
          ' > "$PR_DESCRIPTION_PATH"

          if [ -s "$OUTPUT_PATH" ]; then
            echo 'Output of command:
          ```' >> "$PR_DESCRIPTION_PATH"
            cat "$OUTPUT_PATH" >> "$PR_DESCRIPTION_PATH"
            echo '```' >> "$PR_DESCRIPTION_PATH"
          fi
        env:
          OUTPUT_PATH: ${{ runner.temp }}/output.txt
          PR_DESCRIPTION_PATH: ${{ runner.temp }}/pr-description.txt

      - name: Create pull request
        uses: peter-evans/create-pull-request@eebb6ccce1e609378f84426acf60c49144cf2d3a
        id: create_pull_request
        with:
          title: "Workflows on Shisho Cloud has changed"
          body-path: ${{ runner.temp }}/pr-description.txt
          branch: shisho-cloud
          commit-message: "pulled changes from Shisho Cloud"
note

The above GitHub Actions workflow creates a pull request when a difference occurs in the workflows. Alternatively, you could send a notification to Slack, for example. In that case, it would be good to add the following steps instead of the 'Create pull request' step. For detailed usage of slackapi/slack-github-action, such as how to obtain 'SLACK_WEBHOOK_URL', please refer to the official repository.

- name: Check for diff
  id: diff
  run: |
    git add -N .
    git diff --quiet || echo "changed=1" >> $GITHUB_OUTPUT

- name: Notify Slack
  uses: slackapi/slack-github-action@v1
  if: steps.diff.outputs.changed == '1'
  env:
    SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Once you create this file, commit and push it to GitHub.

Now, whether the workflows in the repository and on Shisho Cloud match or not is being regularly checked. From here, let's directly edit the workflows from the Shisho Cloud console and confirm that GitHub Actions correctly detects this change.

First, through the following URL, open the edit page for the demo workflow demo-notification.

https://cloud.shisho.dev/[oid]/workflows/edit?wfid=demo-notification

Edit the manifest of this workflow as shown below, for instance, and click the "Save" button.

version: 0.1.0
id: "demo-notification"
name: "DEMO: Send notifications to various channels (edited from console)"
triggers: {}
jobs:
- id: group
  name: Send to notification groups
  notify:
    rego: |
      package demo.notification.group

      import data.shisho

      notifications[n] {
        input.running_state == shisho.job.running_state_preprocessing

        data.params.group != ""
        n := shisho.notification.to_group(
          data.params.group,
          "hello! (edited from console)",
        )
      }
# omitted the rest

Next, open the GitHub Actions workflow page from the following URL, and manually trigger the workflow_dispatch trigger by clicking the 'Run workflow' button.

https://github.com/[organization]/[repository]/actions/workflows/check-workflow-drift.yml

Manually triggering the workflow from the GitHub Actions workflow page

A pull request to incorporate the changes made on the console should be created by the GitHub Actions workflow created in this tutorial. Let's make sure the changes made on the console are correctly reflected.

Automatically created pull request

In this way, even if workflows are changed on the Shisho Cloud console, the changes can be detected and incorporated into the repository with the GitHub Actions workflow created. Here, we confirmed the execution result when an existing workflow was changed, but even if you create or delete a workflow on the console, you can incorporate that change into the GitHub repository using this GitHub Actions workflow.

Last updated on