Workspaces
Several packages in one repository: workspace init, status, graph, affected, verify
Sooner or later one repository holds several packages: a core library, a formatting library built on it, an application that uses both. You edit them together, and you want a change to the core to be seen by the others at once, without publishing it first. A Gritpack workspace gives you that. This chapter builds one with three packages, shows how Gritpack decides which dependencies come from your working tree and which from the hub, and answers the questions you ask every day: what is in here, what depends on what, and what does my change affect.
What you will be able to do
- turn a directory of packages into a workspace with one command;
- read the workspace manifest and change it with confidence;
- see which dependencies resolve to your own editable members and which to published versions on the hub;
- find every member a change affects;
- install and check the whole workspace's dependencies with one lock file.
A workspace is a map, not a warehouse
A workspace does not move your files or copy your packages anywhere. It is one
small file, gritpack-workspace.toml, at the root of your repository, that
lists where the packages are. Each package keeps its own manifest in its own
directory: gritpack.toml for a native package, or Cargo.toml,
package.json or go.mod for the ecosystems in chapters 10 to 12. How the
files got there, from Git, a generator or a shared drive, is not Gritpack's
business.
Here is the repository this chapter uses:
ws/
libs/core/ gritpack.toml README.md @bea/core
libs/fmt/ gritpack.toml README.md @bea/fmt, needs @bea/core
apps/report/ gritpack.toml README.md @bea/report, needs @bea/fmt, @bea/core and @ada/greet
Each member is an ordinary package. libs/fmt/gritpack.toml:
[package]
name = "@bea/fmt"
version = "0.1.0"
dialect = "c/c23"
description = "Formatting helpers built on the core types"
readme = "README.md"
license = "MIT"
[dependencies]
"@bea/core" = "^0.1.0"
and apps/report/gritpack.toml asks for two members and one package that is
already published on the hub:
[dependencies]
"@bea/fmt" = "^0.1.0"
"@bea/core" = "^0.1.0"
"@ada/greet" = "^0.2.0"
Notice that nothing says "this one is local". The dependencies are written exactly as they would be if every package came from the hub. The workspace decides.
Making the workspace
From the repository root, see what Gritpack finds before it writes anything:
$ gritpack workspace init --dry-run
workspace . members=3 status=preview
member=apps/report adapter=Native role=Publishable
member=libs/core adapter=Native role=Publishable
member=libs/fmt adapter=Native role=Publishable
It walked the tree, found three package manifests, and recognised each as a native package. If that is right, write it:
$ gritpack workspace init
workspace . members=3 status=created
member=apps/report adapter=Native role=Publishable
member=libs/core adapter=Native role=Publishable
member=libs/fmt adapter=Native role=Publishable
The result is gritpack-workspace.toml:
schema = 2
[resolution]
workspace = "prefer-compatible"
[[members]]
path = "apps/report"
adapter = "native"
role = "publishable"
[[members]]
path = "libs/core"
adapter = "native"
role = "publishable"
[[members]]
path = "libs/fmt"
adapter = "native"
role = "publishable"
workspace init never overwrites an existing workspace file. Run it again and
it stops with a local file error (exit code 9); edit the file by hand from then
on, or delete it and start again.
Each member has:
path, relative to the root;adapter, the kind of package:native,cargo,npmorgo;role, what the member is for (below);- optionally
release_group, for members that must always be released together (Releasing a workspace); - for Go modules,
versionandpublished_at, because a Go module records those outside its own files (Go modules).
Roles
| Role | Means |
|---|---|
publishable | an ordinary package, released to the hub (the default) |
internal-only | used inside the repository, never published |
tooling | a build or developer tool for this repository |
generated | produced by a generator, not edited by hand |
Only publishable members can be released. A publishable member that depends
on a non-publishable one cannot be released either, and Gritpack says so
rather than publishing something its users could not install.
Asking the workspace questions
What is in it?
$ gritpack workspace status
workspace_snapshot=0daecd09b35ab5763988c0069ae392966d5703b8ea92a4825cb29adeac0c35b9 policy=PreferCompatible members=3 edges=4
member=apps/report package=@bea/report version=0.1.0 adapter=Native role=Publishable snapshot=ae919d8112fd4e7273868295bdcf07a3b79447beb8d080aecdfd4d13b50439cd files=2
member=libs/core package=@bea/core version=0.1.0 adapter=Native role=Publishable snapshot=9a437fa20c86da943a9b5e9ee19a2777c215453fff701215c50bd2be56c7a514 files=2
member=libs/fmt package=@bea/fmt version=0.1.0 adapter=Native role=Publishable snapshot=dcaacdf744d6fe1da9d669047a161e368d6f9f79bbcbe9edd2b60a1af4ff658b files=2
Each member's snapshot is a digest of its files as they are right now. Change
a file and its snapshot changes, and so does the workspace snapshot on the first
line. Two people who see the same workspace snapshot are looking at exactly the
same sources.
Where does each dependency come from?
$ gritpack workspace graph
@bea/fmt -> @bea/core ^0.1.0 source=Editable member=libs/core
@bea/report -> @ada/greet ^0.2.0 source=Immutable
@bea/report -> @bea/core ^0.1.0 source=Editable member=libs/core
@bea/report -> @bea/fmt ^0.1.0 source=Editable member=libs/fmt
This is the heart of a workspace. A dependency is Editable when a member of
the workspace provides it: your working copy is used, edits and all. It is
Immutable when no member provides it, so it comes from the hub as a
published, unchangeable version. @ada/greet is somebody else's package, so it
comes from the hub; the others are yours and come from your tree.
What does my change affect?
$ gritpack workspace affected @bea/core
analysis=declared_package_conservative changed=@bea/core affected=@bea/core,@bea/fmt,@bea/report
$ gritpack workspace affected libs/fmt
analysis=declared_package_conservative changed=@bea/fmt affected=@bea/fmt,@bea/report
Name the changed member by package name or by path. The answer is every member that depends on it, directly or through others, plus the member itself. That is the set to rebuild and test. The analysis is "conservative": it follows the dependencies the manifests declare, so it may include a member your change does not really touch, but it never leaves one out.
For automation. Every workspace command takes
--json. A CI job can rungritpack workspace affected "$CHANGED" --jq '.affected[]'and test only those members.
status and graph take --member (repeatable) to show only some members and
what they depend on.
One lock for the whole workspace
Install the workspace's dependencies from the root, the same way as for a single project:
$ gritpack sync
synchronized 4 packages into ./packages status=changed
$ gritpack list
@ada/greet 0.2.0 dialect=c/c23 transitive source=https://packages.example.com:443 dependencies=-
@bea/core 0.1.0 dialect=c/c23 direct source=<local> dependencies=-
@bea/fmt 0.1.0 dialect=c/c23 direct source=<local> dependencies=@bea/core
@bea/report 0.1.0 dialect=c/c23 direct source=<local> dependencies=@ada/greet,@bea/core,@bea/fmt
There is one gritpack.lock at the root. It records the published package with
its hub and exact digest, and each member as source = "workspace" with the
member's path and the snapshot of its files. Only the published package is
unpacked into packages/; the members stay where they are, since you are
editing them.
Check that everything matches:
$ gritpack workspace verify
hybrid universe verified id=b631fb23d128498aed61a512c6d2f316c1a2514f2502676620cf3efc9a3a7972 packages=4 files=11
Before the first sync, workspace verify checks only the workspace itself
(members, manifests and edges); afterwards it also checks every installed file
against the lock.
When a member does not fit
Suppose you start work on version 0.2.0 of the core and change its manifest,
while @bea/fmt still asks for ^0.1.0. The editable core no longer satisfies
that requirement. Gritpack does not force it in:
$ gritpack workspace graph
@bea/fmt -> @bea/core ^0.1.0 source=IncompatibleLocal member=libs/core
@bea/report -> @ada/greet ^0.2.0 source=Immutable
@bea/report -> @bea/core ^0.1.0 source=IncompatibleLocal member=libs/core
@bea/report -> @bea/fmt ^0.1.0 source=Editable member=libs/fmt
$ gritpack sync
synchronized 4 packages into ./packages status=changed
$ gritpack list
...
@bea/core 0.1.0 dialect=c/c23 transitive source=https://packages.example.com:443 dependencies=-
...
IncompatibleLocal means "there is a member with this name, but not a version
that fits". With the default policy, Gritpack uses the published 0.1.0 from the
hub instead, so the formatting library keeps building against the core it was
written for. When you are ready, raise the requirement in libs/fmt to
^0.2.0 and the edge becomes Editable again.
Choosing how members are used
The [resolution] table's workspace setting says how members stand in for
published packages:
| Setting | Behaviour |
|---|---|
prefer-compatible | use a member when its version fits the requirement, else the hub's (the default) |
require | use a member whenever one has the name; a version that does not fit is an error |
selected | use members only for the members you name with --member, and what they need |
disabled | never substitute members; each --member is resolved on its own against the hub |
Most teams never change the default. require suits a repository where every
package must always build against its neighbours' working copies; with it, the
0.2.0 example above stops instead of quietly using the hub:
$ gritpack workspace graph
invalid hybrid workspace: @bea/report requires @bea/core ^0.1.0, but the editable member has version 0.2.0
What just happened
gritpack-workspace.tomllists the packages in a repository; each member keeps its own manifest.- Dependencies between members are editable: your working copy is used. Everything else comes from the hub as an immutable version.
workspace status,graphandaffectedanswer what is here, where each dependency comes from and what a change touches.- One lock at the root records both kinds, and
workspace verifychecks them.
You can stop here
If you only develop the packages together and publish them one by one, you have what you need. If they must ship together, read on.
Next
- Releasing a workspace: publishing several members as one all-or-nothing release.
- Optional: The lock file and sync, for
--frozenand--offline, which work for workspaces too.