Implement Your Checks
The English user guide is currently in beta preview. Most of the documents have been automatically translated from the Japanese version. Should you find any inaccuracies, please reach out to Flatt Security.
Shisho Cloud's native inspection capabilities cover a wide range of inspection items for many cloud services. However, you may want to implement inspection items specific to your company. In such cases, you can extend the inspection capabilities provided by Shisho Cloud.
This tutorial explains how to extend the inspection capabilities provided by Shisho Cloud. Specifically, you will practice the following:
- Registering inspection specifications (Specification) with Shisho Cloud
- Writing and deploying Shisho Cloud policies
Preparation
The steps differ depending on the language in which the policy is written.
- Rego
- TypeScript
After cloning the Shisho Cloud Rego SDK as follows, create a your-policy
directory as a place to store the policy code you will create in this tutorial:
git clone https://github.com/flatt-security/shisho-cloud-rego-libraries.git
mkdir ./your-policy
cd ./your-policy
At this point, you should have the following directory structure:
.
├── your-policy
└── shisho-cloud-rego-libraries
The final directory structure will be as follows:
.
├── your-policy
│ ├── implementation
│ │ ├── decide.rego
│ │ ├── decide_test.rego
│ │ ├── decide.graphql
│ │ └── manifest.yaml
| ├── specification
│ └── dgspec.yaml
└── shisho-cloud-rego-libraries
The inspection rule implementation uses Deno as the JavaScript/TypeScript runtime, so please refer to the Deno official documentation to install Deno. It is also more convenient to install the Deno plugin for your editor.
The library used to write policy code for Shisho Cloud is published as shisho_cloud_policy_helpers. In Deno, you can directly reference the library in an import statement as shown below, so there is no need to install libraries, etc.
import { DecisionPolicy } from "https://deno.land/x/shisho_cloud_policy_helpers/decision/mod.ts";
Define an inspection specification
First, let's define an inspection specification. An inspection specification is a definition of an inspection item in Shisho Cloud. The inspection specification is displayed in a location such as "A test security check" or "Top-level heading 1" in the screen example below, and is used to explain the background of the risk added on Shisho Cloud:
YAML is used to define the inspection specification.
Create a file named ./your-policy/specification/dgspec.yaml
and try to explain the outline of the inspection while following the instructions in the comments:
version: "1.0"
# Each specification defines metadata for a single security check, identified with the pair (apiVersion, kind).
specifications:
- # A custom value to group your "kind"s; it should satisfy ^[0-9a-zA-Z./\\-_]{1,128}$
# You can change this value as you like.
apiVersion: "user.decision.api.shisho.dev/v1"
# A value to represent what this specification is for with the combination of apiVersion; it should satisfy ^[0-9a-zA-Z.\\-_]{1,128}$
# You can change this value as you like.
kind: "my-security-review"
# Any non-empty string to summarize the security check
# You can change this value as you like.
title: "A test security check"
# Any non-empty string, describing background of the security check; potential risk, how to mitigate, etc.
# You can change this value as you like in Markdown format.
explanation: |
This is a test explanation. You can use **Markdown** here.
When you're done, run the following command to register the inspection specification with Shisho Cloud:
shishoctl dgspec apply -o $SHISHO_ORG_ID -f "your-policy/specification/dgspec.yaml"
Once the registration is complete, run the following command to verify that the inspection specification has been registered correctly:
$ shishoctl dgspec describe "user.decision.api.shisho.dev/v1" "my-security-review" -o $SHISHO_ORG_ID | jq -r
{
"apiVersion": "user.decision.api.shisho.dev/v1",
"kind": "my-security-review",
"title": "A test security check",
"explanationMarkdown": "This is a test explanation. You can use **Markdown** here.\n",
"createdAt": "2023-11-13T19:05:41Z",
"updatedAt": "2023-11-13T19:05:41Z"
}
This completes the definition of the inspection specification in Shisho Cloud.
Note that you can use Markdown in explanation
and specify explanation
with a file path in dgspec.yaml
.
Here are some examples of definitions:
version: "1.0"
specifications:
- apiVersion: "user.decision.api.shisho.dev/v1"
kind: "my-security-review"
title: "A test security check"
explanation: !include "./explanation.md"
This is a test explanation. You can use **Markdown** here.
Define the inspection target data
Now, let's actually implement the inspection using the inspection specification we just created.
As an example, here we will implement an inspection that reports a problem if the machine type of an instance of Google Compute Engine is f1-micro
.
Replace [oid]
in the following URL with your Shisho Cloud organization ID and access the Shisho Cloud GraphQL Playground:
https://cloud.shisho.dev/[oid]/playground/query
Using the displayed Playground, you can see that in order to implement the inspection as described above, you can obtain the inspection target data in the following form:
query {
googleCloud {
projects {
number
computeEngine {
instances {
metadata {
id
}
name
machineType
}
}
}
}
}
Implement the inspection
- Rego
- TypeScript
Once you can get the inspection target data, the next step is to implement the inspection.
If you want to report a problem if the instance machine type of Google Compute Engine is f1-micro
, you can implement it in the following way, for example:
package custom
import data.shisho
decisions[d] {
project := input.googleCloud.projects[_]
instance := project.computeEngine.instances[_]
allowed := instance.machineType != "f1-micro"
d := shisho.decision.new({
# The API version and kind are used to identify the decision.
# In this case you should specify what you defined in your dgspec.yaml file.
"api_version": "user.decision.api.shisho.dev/v1",
"kind": "my-security-review",
# The subject is the resource that the decision is about.
# In this case this policy reports posture review on each instance, so the subject is the instance.
# You should use `metadata` to identify the resource.
"subject": instance.metadata.id,
# The locator is used to identify the location of the detected issue within `subject`.
# It should be empty for most cases.
"locator": "",
# The severity is used to indicate the importance of the decision.
"severity": shisho.decision.severity_info,
# The allowed field is used to indicate whether the resource is allowed or not.
"allowed": allowed,
# Any additional information can be added to the payload.
# This will be useful when you want to display more information about the decision in the UI.
"payload": json.marshal({"machine_type": instance.machineType}),
})
}
Let's make the above implementation ./your-policy/implementation/decide.rego
and this time write test code as well.
Create a file named ./your-policy/implementation/decide_test.rego
and refer to the following to write test code for the policy code you have written:
package custom
import data.shisho
import future.keywords
test_my_policy_works if {
count([d |
decisions[d]
shisho.decision.is_allowed(d)
]) == 1 with input as {"googleCloud": {"projects": [{"computeEngine": {"instances": [
{
"metadata": {"id": "google-cloud-ce-instance|354711641168|asia-northeast2-a|7816137240199088770"},
"name": "pod",
"machineType": "f1-micro",
},
{
"metadata": {"id": "google-cloud-ce-instance|354711641168|asia-northeast2-a|7816137240199088771"},
"name": "pod",
"machineType": "g1-small",
},
]}}]}}
}
Running the test code shows that the implemented inspection seems to work correctly:
$ opa test ./your-policy ./shisho-cloud-rego-libraries
Once the inspection target data can be retrieved, the type definition of the data is generated, and the inspection logic is implemented based on the type definition.
The TypeScript type definition corresponding to the GraphQL query earlier can be generated using the following command:
shishoctl codegen typescript-input --query-path=./your-policy/implementation/decide.graphql --output=./your-policy/implementation/input.gen.ts
When you run this command, the type Input
representing the inspection target data will be defined as follows. This file contains not only the type itself but also documentation for each field, so it may be easier to refer to this input.gen.ts
than the original GraphQL query when implementing inspection logic.
/** The type corresponding to the result of the GraphQL query */
export type Input = {
/** All data from Google Cloud integration */
googleCloud: {
/** Projects in Google Cloud */
projects: Array<{
/** The number uniquely identifying the project like `415104041262` */
number_: Int;
/** Data on Google Cloud Compute Engine */
computeEngine: {
/** All Google Cloud Compute Engine instances */
instances: Array<{
/** The metadata to identify this resource in Shisho Cloud */
metadata: {
/** The ID to identify the report in Shisho Cloud */
id: ResourceID;
};
/** The user-provided instance name */
name: string;
/** The machine type to use for instances that are created from this machine image */
machineType: string;
}>;
};
}>;
};
};
Now that we have the type definition for the inspection target data, let's implement the function decide
, which represents the inspection logic, using this type definition.
Let's name the file in which to implement this function ./your-policy/implementation/decide.ts
.
First, let's explicitly specify the type of the function decide
as DecisionPolicy<Input>
, as shown below. This type represents a function that takes Input
as input 1 and returns an array of inspection results Decision[]
.
import { Input } from "./input.gen.ts";
import { DecisionPolicy } from "https://deno.land/x/shisho_cloud_policy_helpers@v0.0.1/decision/mod.ts";
// Basically the same as `const decide = (input: Input): Decision[] => { ... }`
const decide: DecisionPolicy<Input> = (input) => {
/* TODO */
};
For this function to be executed on Shisho Cloud, you need to default export the wrapper function decision_policy_adapter
with decide
wrapped in it, as shown below. The wrapper function decision_policy_adapter
should be given convert_input
defined in input.gen.ts
generated earlier as an argument.
import { convert_input, Input } from "./input.gen.ts";
import {
DecisionPolicy,
decision_policy_adapter,
} from "https://deno.land/x/shisho_cloud_policy_helpers@v0.0.1/decision/mod.ts";
const decide: DecisionPolicy<Input> = (input) => {
/* TODO */
};
export default decision_policy_adapter(convert_input)(decide);
Now let's implement the body of the function decide
. If you want to report a problem if the instance machine type of Google Compute Engine is f1-micro
, you can implement it in the following way, for example:
import {
type Decision,
type DecisionPolicy,
SEVERITY_INFO,
TYPE_ALLOW,
TYPE_DENY,
decision_policy_adapter,
} from "https://deno.land/x/shisho_cloud_policy_helpers@v0.0.1/decision/mod.ts";
import { type Input, convert_input } from "./input.gen.ts";
const decide: DecisionPolicy<Input> = (input) =>
input.googleCloud.projects.flatMap((project) =>
project.computeEngine.instances.map((instance): Decision => {
return {
header: {
// The API version and kind are used to identify the decision.
// In this case you should specify what you defined in your dgspec.yaml file.
api_version: "user.decision.api.shisho.dev/v1",
kind: "my-security-review",
// The subject is the resource that the decision is about.
// In this case this policy reports posture review on each instance, so the subject is the instance.
// You should use `metadata` to identify the resource.
subject: instance.metadata.id,
// The severity is used to indicate the importance of the decision.
severity: SEVERITY_INFO,
// The type field is used to indicate whether the resource is allowed or not.
type: instance_is_allowed(instance) ? TYPE_ALLOW : TYPE_DENY,
},
// Any additional information can be added to the payload.
// This will be useful when you want to display more information about the decision in the UI.
payload: { machine_type: instance.machineType },
};
}),
);
// Tip: You can obtain a part of the type `Input` using indexed access types.
type Instance =
Input["googleCloud"]["projects"][number]["computeEngine"]["instances"][number];
const instance_is_allowed = (instance: Instance) =>
instance.machineType !== "f1-micro";
export default decision_policy_adapter(convert_input)(decide);
Now that we have finished implementing the policy code, let's write the test code.
The testing strategy is to give some input to the decide
function that represents the inspection logic and see if the output inspection result is as expected. Export decide
in the previous decide.ts
so that the test code can import the decide
function.
-const decide: DecisionPolicy<Input> = (input) =>
+export const decide: DecisionPolicy<Input> = (input) =>
Now create a file named ./your-policy/implementation/decide_test.ts
and refer to the following to write test code for the policy code you have written:
import { assertEquals } from "https://deno.land/std@0.202.0/assert/assert_equals.ts";
import {
TYPE_ALLOW,
TYPE_DENY,
} from "https://deno.land/x/shisho_cloud_policy_helpers@v0.0.1/decision/mod.ts";
import { decide } from "./decide.ts";
import type { Input } from "./input.gen.ts";
Deno.test("my policy works", () => {
const input: Input = {
googleCloud: {
projects: [
{
number_: 111111111111,
computeEngine: {
instances: [
{
name: "instance-1",
metadata: {
id: "googlecloud-ce-instance|654126103884|asia-northeast2|5358538573343679550",
},
machineType: "t2-micro",
},
{
name: "instance-2",
metadata: {
id: "googlecloud-ce-instance|654126103884|asia-northeast1|4792791042783564850",
},
machineType: "t1-large",
},
],
},
},
{
number_: 222222222222,
computeEngine: {
instances: [
{
name: "instance-3",
metadata: {
id: "googlecloud-ce-instance|471807136351|asia-northeast2|1620402794530716984",
},
machineType: "f1-micro",
},
{
name: "instance-4",
metadata: {
id: "googlecloud-ce-instance|471807136351|asia-northeast1|15166759912670900731",
},
machineType: "t2-large",
},
],
},
},
],
},
};
const decisions = decide(input, undefined);
assertEquals(decisions.filter((d) => d.header.type === TYPE_ALLOW).length, 3);
assertEquals(decisions.filter((d) => d.header.type === TYPE_DENY).length, 1);
});
Running the test code shows that the implemented inspection seems to work correctly:
$ deno test ./your-policy/implementation/decide_test.ts
Check file:///path/to/your-policy/implementation/decide_test.ts
running 1 test from ./your-policy/implementation/decide_test.ts
my policy works ... ok (0ms)
ok | 1 passed | 0 failed (1ms)
Define a workflow
As a final step, we define a workflow to execute the inspection logic we just implemented. A workflow is a unit that groups inspections in Shisho Cloud, and contains a trigger that defines when the policy should be executed.
YAML is used to define the workflow.
Create a file named ./your-policy/implementation/manifest.yaml
and try to define a workflow by following the instructions in the comments:
- Rego
- TypeScript
version: 0.1.0
id: "my-review-workflow"
name: "My first workflow"
triggers:
schedule:
# Runs every hour
- cron: "0 */1 * * *"
jobs:
- id: review-googlecloud-instance-type
name: "Review instance types of Google Cloud instances"
decide:
rego: !include ./decide.rego
input:
schema: !include ./decide.graphql
version: 0.1.0
id: "my-review-workflow"
name: "My first workflow"
triggers:
schedule:
# Runs every hour
- cron: "0 */1 * * *"
jobs:
- id: review-googlecloud-instance-type
name: "Review instance types of Google Cloud instances"
decide:
typescript: !bundle_typescript ./decide.ts
input:
schema: !include ./decide.graphql
As a preparation for deploying this workflow, you need to download the libraries used by the policy code with the following command:
deno vendor $(find . -name '*.[jt]s' -not -path './vendor/*')
Deploy the workflow
Now, finally, let's deploy the completed workflow to Shisho Cloud. Run the following command:
- Rego
- TypeScript
shishoctl workflow apply -o $SHISHO_ORG_ID -f ./your-policy/implementation/manifest.yaml
shishoctl workflow apply -o $SHISHO_ORG_ID -f ./your-policy/implementation/manifest.yaml --js-import-map=./vendor/import_map.json --workspace-root=.
Once the deployment is complete, you can run the workflow without waiting for the schedule
trigger to fire by running a command like this:
shishoctl workflow run -o $SHISHO_ORG_ID my-review-workflow
After a while, the workflow execution will complete:
When you check the details of the execution result and the inspection result, you can see that the instance with f1-micro
is surely rejected:
In this way, we were finally able to extend the inspection in Shisho Cloud!
Summary
This tutorial explained how to extend the inspection capabilities provided by Shisho Cloud. Specifically, you practiced the following:
- Registering inspection specifications (Specification) with Shisho Cloud
- Writing and deploying Shisho Cloud policies
Footnotes
-
As an additional argument, you can pass parameters passed from the workflow manifest. ↩