Continuous integration
Logging in from CI, frozen installs, publishing without prompts
A CI job is a person who cannot answer questions. It needs Gritpack installed, a login it can use without typing a code, exactly the packages the lock file names, and a way to publish a release without being asked "are you sure?". This chapter builds that job one step at a time. The examples use GitHub Actions syntax because it is common, but every step is a plain shell command, so the same recipe works in any CI system.
What you will be able to do
- Install Gritpack in a fresh CI machine.
- Give the job a login from your CI system's secrets, safely.
- Install exactly the locked packages, and fail when the lock is stale.
- Keep the download cache between runs.
- Publish a release from CI and keep its signed receipt.
What changes when nobody is watching
Three things behave differently without a person at a terminal:
- Gritpack never prompts. When standard input is not a terminal, or the
CIvariable is set, nothing waits for an answer. A command that would ask needs--yes, and fails with a usage error (exit 2) without it. logincannot ask for a code. A one-time code is for a person. It works once and expires within a day, so it cannot live in a CI secret.- Output is read by programs. Ask for JSON results and JSON errors (see Scripting with JSON), so later steps read named fields rather than sentences.
Step 1: install Gritpack
On a Linux runner, the one-line installer from Install Gritpack
works without a person if you pass --yes. Arguments after sh -s -- reach
the installer:
curl -fsSL https://get.gritpack.org/install/gritpack | sh -s -- --yes --no-modify-path
echo "$HOME/.frogfish/gritpack/bin" >> "$GITHUB_PATH"
gritpack version
--no-modify-path stops the installer offering to edit a shell profile that
the job will never read. The second line is how GitHub Actions adds a
directory to PATH for the following steps; other CI systems have their own
way. On a Windows runner, PowerShell's irm cannot pass arguments, so use
the form that can:
& ([scriptblock]::Create((irm https://get.gritpack.org/install.ps1/gritpack))) --yes
For automation.
gritpack version --jsonprints the version, platform and the path of the executable. Log it at the start of every job, so a failure can be matched to the release that produced it.
Step 2: a login for the job
A CI job should have its own account, not borrow a person's. Ask your hub's
administrator to add one, for example ci-releases@example.com, and to give
it only the scopes the job publishes to (see
Administering a hub).
Then make the credential once, on a trusted machine, with the invitation code the administrator sends:
gritpack login ci-releases@example.com --hub packages.example.com \
--code <code> --store file --profile ci
--store file keeps the credential as three files under
~/.frogfish/gritpack/credentials/<credential-id>/:
| File | What it is |
|---|---|
client.pem | the credential's certificate; not secret |
client-key.pem | its private key; secret |
server-ca.pem | the certificate authority that vouches for the hub; not secret |
Put the contents of those files into three CI secrets. In GitHub that is
Settings, Secrets and variables, Actions, New repository secret; call
them GRITPACK_CLIENT_CERT, GRITPACK_CLIENT_KEY and GRITPACK_HUB_CA.
Then delete the copies on the trusted machine, or log that profile out once
the secrets are saved.
In the job, write the secrets to files and save them as a profile with
auth import:
- name: Gritpack login
env:
CLIENT_CERT: ${{ secrets.GRITPACK_CLIENT_CERT }}
CLIENT_KEY: ${{ secrets.GRITPACK_CLIENT_KEY }}
HUB_CA: ${{ secrets.GRITPACK_HUB_CA }}
run: |
umask 077
mkdir -p "$RUNNER_TEMP/gritpack"
printf '%s\n' "$CLIENT_CERT" > "$RUNNER_TEMP/gritpack/client.pem"
printf '%s\n' "$CLIENT_KEY" > "$RUNNER_TEMP/gritpack/client-key.pem"
printf '%s\n' "$HUB_CA" > "$RUNNER_TEMP/gritpack/server-ca.pem"
gritpack auth import ci --hub packages.example.com \
--cert "$RUNNER_TEMP/gritpack/client.pem" \
--key "$RUNNER_TEMP/gritpack/client-key.pem" \
--ca "$RUNNER_TEMP/gritpack/server-ca.pem" --default
gritpack auth status
auth import saves a profile that names the files; it does not copy the key
anywhere else. On a test run it looks like this:
$ gritpack auth import ci --hub packages.example.com --cert client.pem --key client-key.pem --ca server-ca.pem --default
saved profile ci in ~/.frogfish/gritpack/config.toml
$ gritpack auth list
ci (default) server=packages.example.com provider=file
$ gritpack auth status
account=01a1e865-dd2b-7f10-9340-1658dbafa174 email=ci-releases@example.com status=Active
...
Without a profile: environment variables
If you would rather not save a profile at all, every connection flag has an environment variable. With these four set, every command uses them:
env:
GRITPACK_SERVER: packages.example.com
GRITPACK_CERT: ${{ runner.temp }}/gritpack/client.pem
GRITPACK_KEY: ${{ runner.temp }}/gritpack/client-key.pem
GRITPACK_CA: ${{ runner.temp }}/gritpack/server-ca.pem
A flag on the command line still wins over the variable. GRITPACK_PROFILE
chooses a profile the same way, but it cannot be combined with the other
four.
When the credential runs out
A login lasts 30 days. From a week before it expires, every command that
uses it prints a warning on standard error saying when it expires and that it
should be renewed with gritpack auth renew. Make that warning visible:
let the job's log show standard error, or check gritpack auth status --json
and fail early when expires_at is close.
Renewing needs a new invitation code from the administrator, so it is a person's job. On the trusted machine:
gritpack auth renew --profile ci --hub packages.example.com --code <new code>
then copy the new files into the secrets. An expired login cannot renew
itself; Gritpack says so and points to gritpack login.
Careful. The private key in
GRITPACK_CLIENT_KEYcan publish to every scope the CI account owns. Give the CI account no more scopes than it needs, keep the secret out of pull requests from forks, and ask the administrator to revoke it (admin user viewshows its credentials) if it leaks.
Step 3: install exactly what the lock file says
In CI, use sync --frozen:
gritpack sync --frozen
--frozen installs the exact versions in gritpack.lock and fails if they
would change. That catches the commit where someone edited gritpack.toml
and forgot to commit the new lock:
$ gritpack sync --frozen
locked package @ada/textkit is unavailable or no longer matches its requirements
(Here gritpack.toml had been changed to require ^1.0.2 while the lock
still named 1.0.0.)
The command exits with status 5 and changes nothing. The fix belongs on the
developer's machine: run gritpack sync (or add, update) there and commit
gritpack.lock. See The lock file and sync
for what --frozen and --offline promise.
A locked package keeps its source: the lock records the hub each package came from. A CI machine must be able to reach those hubs, and its login must be for the same private hub the developers use.
Step 4: keep the download cache
Every downloaded package file is kept in a cache addressed by its SHA-256,
under ~/.frogfish/gritpack/cache (or GRITPACK_CACHE_DIR). Every read from
the cache checks the digest again, so restoring a cache from an earlier run is
safe: a damaged file is fetched again, never used.
- uses: actions/cache@v4
with:
path: ~/.frogfish/gritpack/cache
key: gritpack-${{ runner.os }}-${{ hashFiles('**/gritpack.lock') }}
restore-keys: gritpack-${{ runner.os }}-
With a warm cache you can go further and install with no network at all:
gritpack sync --offline
--offline implies --frozen and reads only the lock file and the cache. If
the cache lacks a file, it fails with "offline cache does not contain ..."
(exit 5) instead of reaching for the network. That is useful in build steps
that must not depend on the hub, and in air-gapped builders fed by a bundle
(see Offline bundles and slices).
For automation.
gritpack cache prune --max-bytes 2GiB --applykeeps a long-lived self-hosted runner's cache from growing without bound. Without--applyit only reports what it would remove.
Step 5: check, and publish a release
On every pull request, run the same checks a publication would, without publishing:
gritpack publish --dry-run
A dry run builds the package files, checks the README and licence rules, and asks the hub whether it would accept the versions. It never changes the hub. A failure here (a missing README, a licence the hub's rules refuse, a version someone already published) is the same failure the release would hit, found earlier.
On a release tag, publish for real. --yes answers the confirmation, and
JSON gives the next step something to read:
- name: Publish
run: |
gritpack publish --yes --json > publish.json
echo "transaction=$(jq -r .receipt.transaction_id publish.json)" >> "$GITHUB_OUTPUT"
The result's top-level fields are mutated, plan_path, receipt, schema
(gritpack/publication-result@1) and verification. receipt.transaction_id
names the publication; it is also the ID of its signed receipt. Keep that
receipt with the build:
gritpack evidence receipt "$TRANSACTION" --output gritpack-receipt.json
Evidence and receipts explains what the receipt proves and how to check it later without the hub.
Retries are safe
Publication is a transaction: every package in the plan becomes visible
together, or none does. If a job is cancelled or the network drops, run the
same job again. Running publish again with the same plan finishes the
publication that is already under way instead of starting a second one; when
the transaction had already committed, Gritpack says so:
$ gritpack publish --yes --jq '.receipt.transaction_id'
publication transaction 01a1e867-3b33-7eb0-9d9d-ea97e5fe85db is already Committed; finishing its commit
...
What is never safe is changing the files and publishing the same version
again. A version's content cannot change; the hub refuses with
PUBLICATION_IDENTITY_CONFLICT and exit 6. Raise the version instead.
For a release that must be paused between planning and publishing (an approval step, a signature from another machine), use the step-by-step commands in Publishing step by step.
A complete workflow
Putting the steps together:
name: build
on:
push:
tags: ["v*"]
pull_request:
jobs:
build:
runs-on: ubuntu-latest
env:
GRITPACK_ERROR_FORMAT: json
steps:
- uses: actions/checkout@v4
- name: Install Gritpack
run: |
curl -fsSL https://get.gritpack.org/install/gritpack | sh -s -- --yes --no-modify-path
echo "$HOME/.frogfish/gritpack/bin" >> "$GITHUB_PATH"
- name: Gritpack login
env:
CLIENT_CERT: ${{ secrets.GRITPACK_CLIENT_CERT }}
CLIENT_KEY: ${{ secrets.GRITPACK_CLIENT_KEY }}
HUB_CA: ${{ secrets.GRITPACK_HUB_CA }}
run: |
umask 077
mkdir -p "$RUNNER_TEMP/gritpack"
printf '%s\n' "$CLIENT_CERT" > "$RUNNER_TEMP/gritpack/client.pem"
printf '%s\n' "$CLIENT_KEY" > "$RUNNER_TEMP/gritpack/client-key.pem"
printf '%s\n' "$HUB_CA" > "$RUNNER_TEMP/gritpack/server-ca.pem"
gritpack auth import ci --hub packages.example.com \
--cert "$RUNNER_TEMP/gritpack/client.pem" \
--key "$RUNNER_TEMP/gritpack/client-key.pem" \
--ca "$RUNNER_TEMP/gritpack/server-ca.pem" --default
- uses: actions/cache@v4
with:
path: ~/.frogfish/gritpack/cache
key: gritpack-${{ runner.os }}-${{ hashFiles('**/gritpack.lock') }}
restore-keys: gritpack-${{ runner.os }}-
- name: Install dependencies
run: gritpack sync --frozen
- name: Build and test
run: make test
- name: Check the release
if: github.event_name == 'pull_request'
run: gritpack publish --dry-run
- name: Publish
if: startsWith(github.ref, 'refs/tags/v')
run: |
gritpack publish --yes --json > publish.json
gritpack evidence receipt "$(jq -r .receipt.transaction_id publish.json)" \
--output gritpack-receipt.json
For Cargo, npm and Go projects the same job usually adds one more step after login, the setup command from that ecosystem's chapter, so the ecosystem's own tool can reach the hub. See Rust and Cargo, npm and Go modules.
The variables a CI job uses
| Variable | What it does |
|---|---|
CI | When set, Gritpack never prompts |
GRITPACK_ERROR_FORMAT | json makes every failure one JSON object on standard error |
GRITPACK_SERVER, GRITPACK_CERT, GRITPACK_KEY, GRITPACK_CA | The hub and the credential, instead of a profile |
GRITPACK_PROFILE | Which saved profile to use |
GRITPACK_HOME | Moves the whole Gritpack directory, for example into the job's workspace |
GRITPACK_CACHE_DIR | Moves only the download cache |
GRITPACK_JOBS | How many downloads run at once, 1 to 64 (default 4) |
GRITPACK_PACKAGES_DIR | Where a project's packages are installed |
gritpack help environment lists them all.
What just happened
You installed Gritpack on a machine with no person at it, gave it a login of its own from secrets, installed exactly what the lock file names, kept the cache between runs, checked releases on every pull request, and published on tags with a receipt you can keep. Nothing in the job waits for an answer, and every failure has a stable exit code.
You can stop here
Teams that build and release from CI have the whole recipe. If your CI only installs dependencies, steps 1 to 4 are all you need.
Next
Make the interactive shell more pleasant in Shell comforts. For releases of several packages at once, read Releasing a workspace.