Skip to content
Cadence OneFive° Internal Handbook

CI/CD Principles

CI/CD Principles

Section titled “CI/CD Principles”

This document covers our general approach to continuous integration and testing across all repositories.

For repository-specific deployment procedures, see:

Troubleshooting

Section titled “Troubleshooting”

Stalled Fly deployment coordinator (fly_app_release_command)

Section titled “Stalled Fly deployment coordinator (fly_app_release_command)”

On rare occasions, deployment failures occur at a stage involving a special ephemeral fly machine used to coordinate the deployment (that’s the machine on which e.g. migrations are run, fly_app_release_command). This machine is normally spun up just for the deployment, then automatically destroyed. When an issue occurs at deploy, fly sometimes does not destroy the machine, which then gets in the way of future deployments. This might be to let us access the machine logs for investigation.

You can detect this by reviewing the list of machines (e.g. for production). If this happens, take a look at the logs on the fly_app_release_command machine, and if there’s nothing concerning, you can destroy that machine before trying your deployment again.