Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
3dac016
Add custom providers documentation for the plugin system
frostebite Mar 5, 2026
d706850
Comprehensive orchestrator documentation overhaul
frostebite Mar 5, 2026
172ae89
Remove old standalone GitLab pages from versioned docs
frostebite Mar 5, 2026
c95128d
Restructure Orchestrator docs: promote Providers to top-level, improv…
frostebite Mar 5, 2026
07229a6
Merge Configuration Override into Secrets page, rename to Pull Secrets
frostebite Mar 5, 2026
6f9667f
Fix broken link: hooks directory has no index page
frostebite Mar 5, 2026
843fe49
Add comprehensive GitHub Actions examples page
frostebite Mar 5, 2026
e9abe77
Fix ASCII diagram alignment in Game-CI vs Orchestrator
frostebite Mar 5, 2026
d128415
Add ASCII diagrams to custom providers, GitHub integration, and retai…
frostebite Mar 5, 2026
759a16f
Add ASCII diagrams to container hooks, garbage collection, AWS, and p…
frostebite Mar 5, 2026
db90cbc
Replace provider decision tree with simple 4-across comparison
frostebite Mar 5, 2026
07da84a
Fix sidebar ordering: Secrets before Advanced Topics
frostebite Mar 5, 2026
557659d
Rename Premade Container Hooks to Built-In Hooks
frostebite Mar 5, 2026
9df05bc
Rename Built-In Hooks file, move Custom Job out of Hooks, fix alignment
frostebite Mar 5, 2026
b70b662
Fix alignment of all ASCII diagrams across orchestrator docs
frostebite Mar 5, 2026
c7ac032
Add Load Balancing documentation page
frostebite Mar 5, 2026
cf9039b
Add Storage, Architecture pages and Build Caching section
frostebite Mar 5, 2026
a997186
Fix introduction diagram to list all supported CI platforms
frostebite Mar 5, 2026
7b370f1
Fix ASCII diagram alignment in load balancing page
frostebite Mar 5, 2026
1bc57cb
docs(orchestrator): build services — submodule profiles, caching, LFS…
frostebite Mar 5, 2026
2321c1e
docs(providers): add GCP Cloud Run and Azure ACI provider documentation
frostebite Mar 5, 2026
90917a2
docs(orchestrator): CLI provider protocol documentation
frostebite Mar 5, 2026
fe34ffd
docs(load-balancing): add built-in automatic fallback API section
frostebite Mar 5, 2026
ca51beb
docs(load-balancing): rewrite as comprehensive load balancing guide
frostebite Mar 5, 2026
265a194
docs(load-balancing): add workflow dispatch and reusable workflow rou…
frostebite Mar 5, 2026
975faab
docs(secrets): comprehensive secret sources documentation
frostebite Mar 5, 2026
c90b7b6
docs(secrets): add HashiCorp Vault as premade secret source
frostebite Mar 5, 2026
93c86da
docs: add Orchestrator Jobs page and Custom LFS Agents page
frostebite Mar 5, 2026
73f5dc4
docs: test workflow engine and hot runner protocol
frostebite Mar 5, 2026
ccb4a27
docs: add structured build output system page
frostebite Mar 5, 2026
591f549
docs: separate incremental sync protocol, update hot runner focus
frostebite Mar 5, 2026
768106a
docs: add massive projects and monorepo support advanced topics
frostebite Mar 5, 2026
0f40a4a
docs: add build reliability advanced topics page
frostebite Mar 5, 2026
9fed5d5
docs: add CI dispatch and infrastructure automation provider pages
frostebite Mar 5, 2026
15d7e36
fix(docs): add missing sidebar_position frontmatter to advanced topic…
frostebite Mar 5, 2026
ea1c6fb
docs(orchestrator): Fix typos and formatting in documentation
GabLeRoux Mar 5, 2026
a70b831
Apply suggestion from @GabLeRoux
GabLeRoux Mar 5, 2026
caa81b8
docs(cli): add game-ci CLI documentation
frostebite Mar 5, 2026
641fade
fix(cli): correct flag names, defaults, and coverage to match impleme…
frostebite Mar 5, 2026
b8ff9c9
Merge remote-tracking branch 'origin/docs/cloud-providers' into relea…
frostebite Mar 5, 2026
eea4df0
Merge remote-tracking branch 'origin/docs/build-services' into releas…
frostebite Mar 5, 2026
4a70791
Merge remote-tracking branch 'origin/docs/load-balancing-api' into re…
frostebite Mar 5, 2026
549b4d3
Merge branch 'docs/secret-sources' into release/lts-2.0.0 (resolve co…
frostebite Mar 5, 2026
8d3889c
Merge branch 'docs/improve-orchestrator-docs' into release/lts-2.0.0 …
frostebite Mar 5, 2026
ca4ac9c
Merge branch 'docs/test-workflow-hot-runners' into release/lts-2.0.0 …
frostebite Mar 5, 2026
66b8450
Merge remote-tracking branch 'origin/docs/cli-support' into release/l…
frostebite Mar 5, 2026
9671491
chore: add integration branch update scripts
frostebite Mar 5, 2026
dcebd3d
docs: update for standalone @game-ci/orchestrator package
frostebite Mar 10, 2026
cf646dc
Merge remote-tracking branch 'origin/main' into release/lts-2.0.0
frostebite Mar 10, 2026
776342d
docs: add dedicated AWS and Kubernetes example pages
frostebite Mar 10, 2026
6d818da
fix: restore versioned_docs that were accidentally deleted
frostebite Mar 10, 2026
16b850b
fix: resolve broken Docusaurus links
frostebite Mar 10, 2026
a113d55
docs: replace npm install with GitHub Releases install scripts
frostebite Mar 10, 2026
0854736
docs: update install URLs to point to orchestrator repo
frostebite Mar 10, 2026
32d6d0f
style: format orchestrator docs with prettier
frostebite Mar 10, 2026
229a739
docs: remove em-dashes and convert ASCII diagram to mermaid
frostebite Mar 10, 2026
c05c4ed
style: run prettier on documentation files
frostebite Mar 10, 2026
6f04c03
docs: update CLI examples to use standalone orchestrator binary
frostebite Mar 10, 2026
64a8769
docs: add mermaid support and convert ASCII diagrams across orchestra…
frostebite Mar 10, 2026
7888efb
docs: refine introduction and comparison pages for orchestrator
frostebite Mar 10, 2026
cb9f0f5
fix: resolve cytoscape webpack export error and broken doc link
frostebite Mar 10, 2026
e02299e
fix: remove npm install step, add missing providers to overview
frostebite Mar 11, 2026
0b722a6
docs: improve test workflow engine taxonomy framing
frostebite Mar 12, 2026
c3c866c
docs: document engine plugin system for game-engine agnostic orchestr…
frostebite Mar 14, 2026
3f50e66
docs: reframe orchestrator as hardware-agnostic, not cloud-first
frostebite Mar 18, 2026
de78f6b
docs: replace "hardware" with "machines" for beginner friendliness
frostebite Mar 18, 2026
39dc031
docs: replace em dashes with hyphens and fix remaining "hardware"
frostebite Mar 18, 2026
7ba5f93
docs: revert circleci em dash changes, fix remaining hardware refs
frostebite Mar 18, 2026
2178c4c
docs: document AWS ECS 8192-byte containerOverrides limit and secret …
frostebite Mar 25, 2026
e354106
docs: rename AWS section from Limitations to Troubleshooting
frostebite Mar 25, 2026
525f8c8
feat: add reset button for maxed-out failed builds on versions page
frostebite Mar 26, 2026
d34729e
fix: use destructured meta variable to satisfy lint rule
frostebite Mar 26, 2026
bc838bf
revert: move troubleshooting entry and reset button UI to dedicated P…
frostebite Mar 26, 2026
299c33c
docs: restore 8192 troubleshooting entry for orchestrator LTS 2.0.0
frostebite Mar 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
194 changes: 102 additions & 92 deletions docs/03-github-orchestrator/01-introduction.mdx
Original file line number Diff line number Diff line change
@@ -1,100 +1,110 @@
# Introduction

## Concept - What Does Orchestrator Do

**Orchestrator enables you to run, build and test (Unity) projects in the cloud. You can start jobs
from the command line, the "Workbench" GUI in the Unity Editor or from GitHub Actions.**

**Orchestrator will automatically provision an environment at a Cloud Provider such as GCP and AWS.
It will then send the project to be built and/or tested depending on your workflow configuration.**

**Orchestrator is especially useful for game development because it supports large projects.
Orchestrator provides first-class support for the Unity game engine.**

Orchestrator uses git to track and syncronize your projects and uses native cloud services such as
AWS Fargate and Kubernetes to run your jobs. Other version control systems are not actively
supported.
## What Does Orchestrator Do?

**Orchestrator is an advanced build layer on top of
[Game CI unity-builder](https://github.com/game-ci/unity-builder).** It takes whatever machines you
give it and provides the flexibility, control, and tools to manage all your build workflows across
it. Instead of running builds directly on your CI runner, Orchestrator dispatches them to any
infrastructure you choose -from a local machine to a Kubernetes cluster. While Unity is the built-in
default, Orchestrator is engine agnostic -Godot, Unreal, and custom engines can plug in via the
[engine plugin system](advanced-topics/engine-plugins). Start jobs from GitHub Actions, the command
line, or any CI system.

```mermaid
flowchart LR
subgraph trigger["Trigger"]
A["GitHub Actions\nGitLab CI\nCLI"]
end
subgraph orchestrator["Orchestrator"]
B["Provision\nSync\nCache\nBuild\nCleanup"]
end
subgraph target["Build Target"]
C["AWS Fargate\nKubernetes\nDocker\nYour Hardware"]
end
A --> B --> C
C -- "artifacts + logs" --> A
```

:::info Built-in and Standalone

The orchestrator is built into [`game-ci/unity-builder`](https://github.com/game-ci/unity-builder)
and activates automatically when you set `providerStrategy`. It is also available as a
[standalone CLI](https://github.com/game-ci/orchestrator) for use outside GitHub Actions.

:::

## Why Orchestrator?

1. **Orchestrator is flexible and elastic**
1. _You can balance your use of high-performance and cost-saving modes._ Configurable cost/speed
effeciency
2. _Works great for projects of almost any size, from AAA projects and assets to micro projects_
3. _Extended configuration options._ More control over disk size, memory and CPU
4. _Easily scale all job resources as your project grows_ e.g. storage, CPU and memory
2. **Scale fully on demand from zero (not in use) to many concurrent jobs.** Benefits from
"pay-for-what-you-use" cloud billing models. We have made an effort to make sure that it costs
you nothing (aside from artifact and cache storage) while there are no builds running (no
guarantees)
3. **Easy setup and configuration**
4. **Run custom jobs** and extend the system for any workload

## Why not orchestrator?

1. Your project is small in size. Below 5GB Orchestrator should not be needed.
2. You already have dedicated servers running you can use.

Although the speed of a CI pipelines is an important metric to consider, there are real challenges
for game development pipelines.

This solution prefers convenience, ease of use, scalability, throughput and flexibility.

Faster solutions exist, but would all involve self-hosted hardware with an immediate local cache of
the large project files and working directory and a dedicated server.

# Orchestrator Release Status

Orchestrator is in "active development" ⚠️🔨

Orchestrator overall release status: `preview` This means some APIs may change, features are still
being added but the minimum feature set works and is stable.

Release Stages: `experimental` ➡️ `preview` ➡️ `full release`

You must use a provider with Orchestrator, each provider's release status is described below. This
indicates the stability and support for orchestrator features and workflows.

### Supported Orchestrator Platforms

| Cloud Provider Platform | Release Status |
| ----------------------- | ------------------ |
| Kubernetes | ✔️ preview release |
| AWS | ✔️ full release |
| GCP | ⚠ Considered |
| Azure | ⚠ Considered |

_Note for Kubernetes support:_ _Usually the cluster needs to be up and running at all times, as
starting up a cluster is slow._ _Use Google Cloud's Kubernetes Autopilot you can scale down to the
free tier automatically while not in use._ _Kubernetes support currently requires cloud storage,
only S3 support is built-in._

| Git Platform | Release Status |
| --------------------- | ------------------ |
| GitHub | ✔️ full release |
| GitLab | ✔️ preview release |
| Command Line | ✔️ preview release |
| Any Git repository | ✔️ preview release |
| Any Git automation/Ci | ✔️ preview release |
Orchestrator benefits projects of any size. Even small projects gain access to configurable
resources, caching, and cost-efficient scaling. Larger projects get retained workspaces, automatic
failover, and multi-provider load balancing.

| Benefit | What it means |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| **Configurable resources** | Set CPU, memory, and disk per build instead of accepting CI runner defaults |
| **Scale from zero** | No idle servers. Cloud providers provision on demand and tear down when done |
| **Retained workspaces** | Cache the entire project folder across builds for faster rebuilds on large projects |
| **Automatic caching** | Unity Library, LFS objects, and build output cached to S3 or 70+ backends via rclone |
| **Provider failover** | Automatically route to a fallback provider when the primary is unavailable or overloaded |
| **Extensible** | Run [custom hooks](advanced-topics/hooks/container-hooks), middleware, or your own [provider plugin](providers/custom-providers) |
| **Engine agnostic** | Built-in Unity support with a plugin system for [other engines](advanced-topics/engine-plugins) (Godot, Unreal, custom) |
| **Self-hosted friendly** | Complements self-hosted runners with automatic fallback, load balancing, and runner availability checks |

## When You Might Not Need It

- Your project fits comfortably on standard GitHub runners and you don't need caching, hooks, or
custom resources
- You already have a fully managed build pipeline that meets your needs

See [Standard Game-CI vs Orchestrator](game-ci-vs-orchestrator) for a detailed comparison.

## What Orchestrator Handles

Orchestrator manages the full build lifecycle so you don't have to script it yourself:

- **Provisioning** - creates cloud resources (CloudFormation stacks, Kubernetes jobs, Docker
containers) and tears them down after the build
- **Git sync** - clones your repo with configurable depth, pulls LFS objects, initializes
submodules, and handles SSH/HTTP auth
- **Caching** - persists the Unity Library folder, LFS objects, and build output across builds using
S3 or rclone
- **Hooks** - inject custom containers or shell commands at any point in the build lifecycle with
phase, provider, and platform filtering
- **Secrets** - pulls secrets from AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or
HashiCorp Vault and injects them as environment variables
- **Logging** - streams structured build logs in real-time via Kinesis (AWS), kubectl (K8s), or
stdout (local)
- **Cleanup** - removes cloud resources, temporary files, and expired caches automatically

## Supported Providers

| Provider | Description |
| -------------------------------------- | -------------------------------------------------------- |
| [AWS Fargate](providers/aws) | Fully managed containers on AWS. No servers to maintain. |
| [Kubernetes](providers/kubernetes) | Run on any Kubernetes cluster. |
| [Local Docker](providers/local-docker) | Docker containers on the local machine. |
| [Local](providers/local) | Direct execution on the host machine. |

See [Providers](providers/overview) for the full list including
[GCP Cloud Run](providers/gcp-cloud-run), [Azure ACI](providers/azure-aci),
[custom](providers/custom-providers), and [community](providers/community-providers) providers.

## Supported Platforms

| Platform | Description |
| ---------------------------------------------- | ------------------------------------- |
| [GitHub Actions](providers/github-integration) | First-class support with Checks API. |
| [GitLab CI](providers/gitlab-integration) | Via the command line mode. |
| [Command Line](examples/command-line) | Run from any terminal or script. |
| Any CI system | Anything that can run shell commands. |

## External Links

### Orchestrator Releases

[Game CI Releases - GitHub](https://github.com/game-ci/unity-builder/releases) _Packaged and
released with game-ci's unity-builder module._

### Open Incoming Pull Requests

[Orchestrator PRs - GitHub](https://github.com/game-ci/unity-builder/pulls?q=is%3Apr+orchestrator)

### 💬Suggestions and 🐛Bugs (GitHub Issues):

[Game CI Issues - GitHub](https://github.com/game-ci/unity-builder/labels/orchestrator)

### Community

**Share your feedback with us!**

- [**Discord Channel**](https://discord.com/channels/710946343828455455/789631903157583923)
- [**Feedback Form**](https://forms.gle/3Wg1gGf9FnZ72RiJ9)
- [Orchestrator Repository](https://github.com/game-ci/orchestrator) - standalone orchestrator
package
- [Releases](https://github.com/game-ci/orchestrator/releases) - orchestrator releases
- [Pull Requests](https://github.com/game-ci/orchestrator/pulls) - open orchestrator PRs
- [Issues](https://github.com/game-ci/orchestrator/issues) - bugs and feature requests
- [Discord](https://discord.com/channels/710946343828455455/789631903157583923) - community chat
- [Feedback Form](https://forms.gle/3Wg1gGf9FnZ72RiJ9) - share your experience
101 changes: 59 additions & 42 deletions docs/03-github-orchestrator/02-game-ci-vs-orchestrator.mdx
Original file line number Diff line number Diff line change
@@ -1,42 +1,59 @@
# Game-CI vs Orchestrator

# Standard Game-CI (Use Cases)

The Game CI core is a maintained set of docker images that can be used to run workloads in many
scenarios.

Game CI also provides specific GitHub actions for running workflows on GitHub. And a similar
workflow for running Game CI on GitLab and Circle CI. _All of these options use the build server
resources provided by those systems, this can be a constraint or very convenient depending on the
size of your project and the workloads you need to run._

# Orchestrator (Use Cases)

## Sending Builds to the cloud

You may want to take advantage of cloud resources for lots of reasons (scale, speed, cost,
flexibility) or may want to start remote builds from the command line without slowing down your
development machine. Orchestrator can help you do this.

This may be a preference, more efficient, or you may want to use systems that struggle to handle
large game development projects (GitHub being a good example).

### Large GitHub Projects

GitHub Actions by default run on
[build machines provided by GitHub](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners).
For Unity projects the available disk size is quite small. You may experience an error related to
running out of disk space. You may also want to run the build on a server with more memory or
processing resources.

### GitHub Self-Hosted Runners vs Game CI Orchestrator

_GitHub users can consider:
[GitHub self-hosted runners](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners)
and Orchestrator. Both can enable you to build larger projects._

_Orchestrator is better if you don't have a server setup or don't want to manage or maintain your
own build server._

_Self-hosted runners are best used when you already have a server available, running 24/7 that you
can setup as a runner. And you're happy to maintain and keep that server available and running._
---
sidebar_label: Game-CI vs Orchestrator
---

# Standard Game-CI vs Orchestrator Mode

## Standard Game-CI

Game CI provides Docker images and GitHub Actions for running Unity workflows on the build server
resources provided by your CI platform (GitHub, GitLab, Circle CI).

**Best for:** Projects that fit within your CI runner's resource limits and don't need advanced
caching, hooks, or multi-provider routing.

## Orchestrator Mode

Orchestrator is an advanced layer on top of Game CI. It takes whatever machines you give it and
manages the full build lifecycle across it: provisioning, git sync, caching, hooks, and cleanup.
Whether you're running on a cloud provider, a local machine, or your own servers, Orchestrator gives
you the tools and flexibility to manage your workflows.

Projects of any size can benefit from orchestrator features like configurable resources, automatic
caching, and extensible hooks. Larger projects additionally benefit from retained workspaces,
provider failover, and load balancing.

```mermaid
flowchart LR
subgraph Standard Game-CI
GR["GitHub Runner\n(builds locally)\n\n~14 GB disk\nFixed resources"]
end
subgraph Orchestrator Mode
GA["GitHub Action\nCLI / Any CI\n(dispatches only)\n\nConfigurable CPU, memory, disk\nScales to zero when idle"]
CC["Build Target\n(any machine)"]
GA <--> CC
end
```

## Self-Hosted Runners + Orchestrator

Self-hosted runners and orchestrator are not mutually exclusive. Orchestrator **complements**
self-hosted runners by adding automatic failover, load balancing, and runner availability checks.

| | Self-Hosted Runners Alone | Self-Hosted + Orchestrator |
| ------------------ | ----------------------------------- | ------------------------------------------------------ |
| **Failover** | Manual intervention if server fails | Automatic fallback to cloud when runner is unavailable |
| **Load balancing** | Fixed capacity | Overflow to cloud during peak demand |
| **Caching** | Local disk only | S3/rclone-backed caching with retained workspaces |
| **Hooks** | Custom scripting | Built-in middleware pipeline with lifecycle hooks |
| **Maintenance** | You manage everything | Orchestrator handles provisioning, sync, and cleanup |

## Choosing Your Setup

| Scenario | Recommendation |
| ---------------------------------------------- | ----------------------------------------------------------------- |
| Small project, standard runners work fine | Standard Game-CI |
| Need configurable resources or caching | Orchestrator with any provider |
| Large project, no existing servers | Orchestrator with AWS Fargate or Kubernetes |
| Existing self-hosted runners, want reliability | Orchestrator with self-hosted primary + cloud fallback |
| Want to test orchestrator locally before cloud | Orchestrator with [Local Docker](providers/local-docker) provider |
Loading
Loading