The lock file and sync
gritpack.lock, sync, --frozen, --offline, and the download cache
You added dependencies on your machine. Now a colleague clones the
repository, a CI job builds it, and a release is rebuilt next year. All of
them need the same packages you had, not whatever is newest on the day. That
is the lock file's job, and gritpack sync is the command that honours it.
What you will be able to do
- read
gritpack.lockand say what each entry promises; - rebuild a project's packages exactly from its lock;
- make a build fail rather than change versions, with
--frozen; - install with no network at all, with
--offline; - look after the shared download cache.
What the lock file records
After gritpack add @ada/greet, the project has a gritpack.lock next to
its gritpack.toml. Here it is, with one line per package trimmed (the
package file's digest, explained below):
lock_version = 4
roots = ["@ada/greet"]
[[packages]]
name = "@ada/greet"
version = "1.0.0"
dialect = "c/c23"
storage_id = "01M7M6E1F5EHG8NX5TXKG44YDG"
...
source = "registry"
source_server = "https://packages.example.com:443"
[[packages.dependencies]]
name = "@ada/strings"
storage_id = "01M7M6E17HEF1R2PZ6CWMKB10Y"
[[packages]]
name = "@ada/strings"
version = "1.0.0"
dialect = "c/c23"
storage_id = "01M7M6E17HEF1R2PZ6CWMKB10Y"
...
source = "registry"
source_server = "https://packages.example.com:443"
dependencies = []
Read it from the top:
rootslists your direct dependencies, the ones ingritpack.toml. With them, the lock can be checked against the manifest on its own.- Each
[[packages]]entry is one package in the graph, direct or not, with its exact version and variant. storage_idnames that exact published version on the hub. The trimmed line holds the SHA-256 digest of its package file, so a download that does not match is refused.source_serveris the hub it came from. A locked package always comes back from the same hub (chapter 9).dependenciesrecords which packages each one needs, by their exactstorage_id.
You never edit the lock by hand. Commands that may change it say so: add,
update and upgrade resolve and rewrite it; sync only rewrites it when
gritpack.toml asks for something it does not describe.
Rebuilding from the lock
gritpack sync makes packages/ match the lock, and nothing else:
$ gritpack sync
synchronized 2 packages into ./packages status=unchanged
$ rm -rf packages
$ gritpack sync
synchronized 2 packages into ./packages status=changed
$ gritpack sync
synchronized 2 packages into ./packages status=unchanged
status=unchanged is a promise, not a shrug: the folder already matched the
lock, and Gritpack touched nothing. status=changed means it put files back,
still without choosing any versions again. Run sync after every clone, every
git pull and every branch switch, and whenever something in packages/
looks wrong.
sync also repairs. If a file under packages/ has been edited, verify
reports it and sync replaces the whole folder with the published files:
$ gritpack verify
installed file is stale or missing: ./packages/@ada/greet/src/greet.c
$ gritpack sync
synchronized 2 packages into ./packages status=changed
$ gritpack verify
verified 2 packages and 8 files universe=8efb3ff135ddaf86907acaa20815fcf8a31be1f95aef343cd66629f600d37b4b
The new folder is built next to the old one and swapped in only when it is
complete, so an interrupted sync never leaves you half-installed.
When the manifest and the lock disagree
If you edit gritpack.toml by hand, plain sync notices and resolves what it
must: a new dependency is added to the lock, a deleted one is removed. Versions
that are already locked and still allowed are kept.
That is convenient on your machine and wrong in CI, where a build should use
exactly what was reviewed. --frozen turns the convenience off: Gritpack uses
the versions in the lock and fails if it would have to change them.
$ gritpack sync --frozen
locked package @ada/greet is unavailable or no longer matches its requirements
Here someone added a requirement to gritpack.toml without updating the lock.
The command stopped with exit code 5 and changed nothing. The fix is to run
gritpack sync (or add) on a developer machine and commit the new lock.
Use gritpack sync --frozen in CI and in release builds.
With no network: --offline
--offline installs from the lock and the local cache only. It makes no
network requests and does not even read your login, so it works on a plane, on
a build machine with no route to the hub, or when your login has expired. It
implies --frozen.
$ gritpack sync --offline
synchronized 2 packages into ./packages status=changed
If a package file is not in the cache, it fails with exit code 5 (not found),
naming the package and the digest it was looking for, and leaves your
existing installation as it was. Any online
sync, add or update fills the cache, so run one while you still have a
connection. To carry packages to a machine that never has a connection, use a
signed bundle (chapter 26).
The download cache
Every package file Gritpack downloads is kept in one cache, shared by all your projects and applications:
~/.frogfish/gritpack/cache
The cache is keyed by each file's SHA-256 digest, so the same version is downloaded once however many projects use it. Every read from the cache checks the digest again; a damaged file is deleted and downloaded afresh (or, offline, reported as an error).
Three commands look after it, none of which need a hub:
$ gritpack cache status
cache root=/home/ada/.frogfish/gritpack/cache objects=2 bytes=2442 corrupt=0 locks=0 temporary=0
$ gritpack cache verify
cache root=/home/ada/.frogfish/gritpack/cache objects=2 bytes=2442 corrupt=0 locks=0 temporary=0
status counts what is there without reading the files. verify reads every
file and checks its digest and structure; corrupt counts the ones that
failed.
prune makes room. It only reports what it would remove unless you add
--apply:
$ gritpack cache prune --max-bytes 1KiB
cache objects=2 bytes=2442 selected=2 selected_bytes=2442 removed=0 removed_bytes=0 dry_run=true
$ gritpack cache prune --max-age-days 30 --max-bytes 5GiB --apply
--max-bytes keeps the cache under a size, removing the oldest files first.
--max-age-days removes files older than that many days. Files still being
downloaded are never removed. Pruning is always safe for projects: anything a
lock needs is downloaded again on the next online sync. Only --offline
depends on what is still in the cache.
Move the cache with --cache-dir or GRITPACK_CACHE_DIR.
Faster downloads
Gritpack downloads up to four package files at once. Choose between 1 and 64
with --jobs or GRITPACK_JOBS. An interrupted download keeps what it has,
and the next online command continues from where it stopped, then checks the
whole file before using it.
What just happened
The lock file turns "whatever was newest" into "exactly this". It names every
package in the graph, its exact version, the digest of its package file and
the hub it came from. sync rebuilds packages/ from it, --frozen makes
any change an error, and --offline does it all from the local cache. The
cache is shared, checked on every read, and safe to prune.
You can stop here
If your colleagues and your CI now get the same packages you do, you are done with locks.
Next
- Moving forward on purpose: Keeping up to date.
- Optional: locks in CI pipelines, in Continuous integration.