# Performance {#performance}

For the vast majority of use cases, Takumi Runner delivers performance on par with or better than GitHub-hosted runners. This is a very important point for us, because we do not want users to endure a slower CI/CD execution time to use Takumi Runner.

## Benchmark Results {#benchmark-results}

The table below shows execution times for several CI workflows. It compares Takumi Runner to the GitHub-hosted runner.

| Workflow    | GitHub-hosted runner | Takumi Runner | Difference |
| ----------- | -------------------- | ------------- | ---------- |
| Frontend CI | 13m 26s              | 12m 58s       | -28s       |
| Rust CI     | 39m 37s              | 29m 16s       | -10m 21s   |
| Proto CI    | 51s                  | 1m 56s        | +1m 5s     |

The workflows used above contain typical CI flows such as linting, building, and running tests, with multiple jobs executed both in series and in parallel. Larger jobs, especially CPU-intensive ones, tend to show noticeable improvements, while small jobs sometimes end up slightly slower.

On Takumi Runner, job queue times tends to be slightly slower, usually by about 5-10s. This means a workflow with 3 jobs in series will slow the total execution time by about 15-30s. In most jobs though, the faster processing time will bring the total execution time to a faster time than the GitHub-hosted runner's execution time.

<details>
<summary>Frontend CI content</summary>

The Frontend CI checks for formatting, typescript typing, and runs jest tests, all in parallel.

```
jobs:
    check-formatting:
        name: Check formatting
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <checkout repo, install neccesary packages, check lint for js, markdown, css, etc>

    check-typing:
        name: Check typing
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <checkout repo, install neccesary packages, run pnpm typescript checks>

    jest:
        runs-on: <ubuntu-latest | takumi-runner>
        name: Jest test (${{ matrix.segment }} - ${{ matrix.chunk }})

        strategy:
            fail-fast: false
            matrix:
                segment: ['FOSS', 'EE']
                chunk: [1, 2, 3]

        steps: <checkout repo, install neccesary packages, test with jest>

    all-checks-pass:
        needs: [jest, check-formatting, check-typing]
        name: All checks pass
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <check if all other jobs passed>
```

</details>

<details>
<summary>Rust CI content</summary>

The Rust CI builds, lints, and tests the application.

```
jobs:
    check-changes:
        runs-on: <ubuntu-latest | takumi-runner>
        name: Check for changes
        steps: <checkout repo, check for changes>

    build:
        name: Build services
        needs: check-changes
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <checkout repo, install rust, build rust services>

    check-linting:
        name: Run linting
        needs: check-changes
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <checkout repo, install neccesary packages, run linting>

    test:
        name: Run tests
        needs: check-changes
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <checkout repo, install dependencies and packages, run tests>

    all-checks-pass:
        needs: [build, test, check-linting]
        name: All checks pass
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <check if all other jobs passed>
```

</details>

<details>
<summary>Proto CI content</summary>

The Proto CI checks linting and for breaking changes.

```
jobs:
    check-changes:
        runs-on: <ubuntu-latest | takumi-runner>
        name: Check for changes
        outputs:
            proto: ${{ steps.filter.outputs.proto || 'true' }}
        steps: <checkout repo, check for changes>

    check-linting:
        name: Check linting
        needs: check-changes
        runs-on: <ubuntu-latest | takumi-runner>

        steps: <checkout repo, install buf, lint proto files>

    check-breaking-changes:
        name: Check for breaking changes
        needs: check-changes
        runs-on: <ubuntu-latest | takumi-runner>

        steps: <checkout repo, install buf, check for breaking changes>

    all-checks-pass:
        needs: [check-changes, check-linting, check-breaking-changes]
        name: All checks pass
        runs-on: <ubuntu-latest | takumi-runner>
        steps: <check if all other jobs passed>
```

</details>

## Hardware Benchmarks {#hardware-benchmarks}

The following benchmarks compare the CPU, memory, and disk I/O performance of Takumi Runner against the GitHub-hosted runner. Both runners use 2 vCPUs and 7.8 GiB of RAM.

| Metric           | GitHub-hosted runner | Takumi Runner  |
| ---------------- | -------------------- | -------------- |
| CPU throughput   | 4,098 events/s       | 5,457 events/s |
| Memory bandwidth | 8,493 MiB/s          | 3,433 MiB/s    |
| Disk write       | 1,559 MiB/s          | 416 MiB/s      |
| Disk read        | 117 MiB/s            | 1,588 MiB/s    |

`sysbench` and `fio` are used for these measurements. CPU performance has the largest impact on most CI/CD runtimes, because linting, building, and unit testing are primarily CPU-bound. Our top priority is therefore to ensure Takumi Runner CPUs outperform those on GitHub-hosted runners.
