Workspaces

Several packages in one repository: workspace init, status, graph, affected, verify

Chapter 23 of 34Current for Gritpack 1.2.63Source: docs/guide/23-workspaces.md

Sooner or later one repository holds several packages: a core library, a formatting library built on it, an application that uses both. You edit them together, and you want a change to the core to be seen by the others at once, without publishing it first. A Gritpack workspace gives you that. This chapter builds one with three packages, shows how Gritpack decides which dependencies come from your working tree and which from the hub, and answers the questions you ask every day: what is in here, what depends on what, and what does my change affect.

What you will be able to do

  • turn a directory of packages into a workspace with one command;
  • read the workspace manifest and change it with confidence;
  • see which dependencies resolve to your own editable members and which to published versions on the hub;
  • find every member a change affects;
  • install and check the whole workspace's dependencies with one lock file.

A workspace is a map, not a warehouse

A workspace does not move your files or copy your packages anywhere. It is one small file, gritpack-workspace.toml, at the root of your repository, that lists where the packages are. Each package keeps its own manifest in its own directory: gritpack.toml for a native package, or Cargo.toml, package.json or go.mod for the ecosystems in chapters 10 to 12. How the files got there, from Git, a generator or a shared drive, is not Gritpack's business.

Here is the repository this chapter uses:

ws/
  libs/core/      gritpack.toml  README.md     @bea/core
  libs/fmt/       gritpack.toml  README.md     @bea/fmt, needs @bea/core
  apps/report/    gritpack.toml  README.md     @bea/report, needs @bea/fmt, @bea/core and @ada/greet

Each member is an ordinary package. libs/fmt/gritpack.toml:

[package]
name = "@bea/fmt"
version = "0.1.0"
dialect = "c/c23"
description = "Formatting helpers built on the core types"
readme = "README.md"
license = "MIT"

[dependencies]
"@bea/core" = "^0.1.0"

and apps/report/gritpack.toml asks for two members and one package that is already published on the hub:

[dependencies]
"@bea/fmt" = "^0.1.0"
"@bea/core" = "^0.1.0"
"@ada/greet" = "^0.2.0"

Notice that nothing says "this one is local". The dependencies are written exactly as they would be if every package came from the hub. The workspace decides.

Making the workspace

From the repository root, see what Gritpack finds before it writes anything:

$ gritpack workspace init --dry-run
workspace . members=3 status=preview
member=apps/report adapter=Native role=Publishable
member=libs/core adapter=Native role=Publishable
member=libs/fmt adapter=Native role=Publishable

It walked the tree, found three package manifests, and recognised each as a native package. If that is right, write it:

$ gritpack workspace init
workspace . members=3 status=created
member=apps/report adapter=Native role=Publishable
member=libs/core adapter=Native role=Publishable
member=libs/fmt adapter=Native role=Publishable

The result is gritpack-workspace.toml:

schema = 2

[resolution]
workspace = "prefer-compatible"

[[members]]
path = "apps/report"
adapter = "native"
role = "publishable"

[[members]]
path = "libs/core"
adapter = "native"
role = "publishable"

[[members]]
path = "libs/fmt"
adapter = "native"
role = "publishable"

workspace init never overwrites an existing workspace file. Run it again and it stops with a local file error (exit code 9); edit the file by hand from then on, or delete it and start again.

Each member has:

  • path, relative to the root;
  • adapter, the kind of package: native, cargo, npm or go;
  • role, what the member is for (below);
  • optionally release_group, for members that must always be released together (Releasing a workspace);
  • for Go modules, version and published_at, because a Go module records those outside its own files (Go modules).

Roles

RoleMeans
publishablean ordinary package, released to the hub (the default)
internal-onlyused inside the repository, never published
toolinga build or developer tool for this repository
generatedproduced by a generator, not edited by hand

Only publishable members can be released. A publishable member that depends on a non-publishable one cannot be released either, and Gritpack says so rather than publishing something its users could not install.

Asking the workspace questions

What is in it?

$ gritpack workspace status
workspace_snapshot=0daecd09b35ab5763988c0069ae392966d5703b8ea92a4825cb29adeac0c35b9 policy=PreferCompatible members=3 edges=4
member=apps/report package=@bea/report version=0.1.0 adapter=Native role=Publishable snapshot=ae919d8112fd4e7273868295bdcf07a3b79447beb8d080aecdfd4d13b50439cd files=2
member=libs/core package=@bea/core version=0.1.0 adapter=Native role=Publishable snapshot=9a437fa20c86da943a9b5e9ee19a2777c215453fff701215c50bd2be56c7a514 files=2
member=libs/fmt package=@bea/fmt version=0.1.0 adapter=Native role=Publishable snapshot=dcaacdf744d6fe1da9d669047a161e368d6f9f79bbcbe9edd2b60a1af4ff658b files=2

Each member's snapshot is a digest of its files as they are right now. Change a file and its snapshot changes, and so does the workspace snapshot on the first line. Two people who see the same workspace snapshot are looking at exactly the same sources.

Where does each dependency come from?

$ gritpack workspace graph
@bea/fmt -> @bea/core ^0.1.0 source=Editable member=libs/core
@bea/report -> @ada/greet ^0.2.0 source=Immutable
@bea/report -> @bea/core ^0.1.0 source=Editable member=libs/core
@bea/report -> @bea/fmt ^0.1.0 source=Editable member=libs/fmt

This is the heart of a workspace. A dependency is Editable when a member of the workspace provides it: your working copy is used, edits and all. It is Immutable when no member provides it, so it comes from the hub as a published, unchangeable version. @ada/greet is somebody else's package, so it comes from the hub; the others are yours and come from your tree.

What does my change affect?

$ gritpack workspace affected @bea/core
analysis=declared_package_conservative changed=@bea/core affected=@bea/core,@bea/fmt,@bea/report
$ gritpack workspace affected libs/fmt
analysis=declared_package_conservative changed=@bea/fmt affected=@bea/fmt,@bea/report

Name the changed member by package name or by path. The answer is every member that depends on it, directly or through others, plus the member itself. That is the set to rebuild and test. The analysis is "conservative": it follows the dependencies the manifests declare, so it may include a member your change does not really touch, but it never leaves one out.

For automation. Every workspace command takes --json. A CI job can run gritpack workspace affected "$CHANGED" --jq '.affected[]' and test only those members.

status and graph take --member (repeatable) to show only some members and what they depend on.

One lock for the whole workspace

Install the workspace's dependencies from the root, the same way as for a single project:

$ gritpack sync
synchronized 4 packages into ./packages status=changed
$ gritpack list
@ada/greet 0.2.0 dialect=c/c23 transitive source=https://packages.example.com:443 dependencies=-
@bea/core 0.1.0 dialect=c/c23 direct source=<local> dependencies=-
@bea/fmt 0.1.0 dialect=c/c23 direct source=<local> dependencies=@bea/core
@bea/report 0.1.0 dialect=c/c23 direct source=<local> dependencies=@ada/greet,@bea/core,@bea/fmt

There is one gritpack.lock at the root. It records the published package with its hub and exact digest, and each member as source = "workspace" with the member's path and the snapshot of its files. Only the published package is unpacked into packages/; the members stay where they are, since you are editing them.

Check that everything matches:

$ gritpack workspace verify
hybrid universe verified id=b631fb23d128498aed61a512c6d2f316c1a2514f2502676620cf3efc9a3a7972 packages=4 files=11

Before the first sync, workspace verify checks only the workspace itself (members, manifests and edges); afterwards it also checks every installed file against the lock.

When a member does not fit

Suppose you start work on version 0.2.0 of the core and change its manifest, while @bea/fmt still asks for ^0.1.0. The editable core no longer satisfies that requirement. Gritpack does not force it in:

$ gritpack workspace graph
@bea/fmt -> @bea/core ^0.1.0 source=IncompatibleLocal member=libs/core
@bea/report -> @ada/greet ^0.2.0 source=Immutable
@bea/report -> @bea/core ^0.1.0 source=IncompatibleLocal member=libs/core
@bea/report -> @bea/fmt ^0.1.0 source=Editable member=libs/fmt
$ gritpack sync
synchronized 4 packages into ./packages status=changed
$ gritpack list
...
@bea/core 0.1.0 dialect=c/c23 transitive source=https://packages.example.com:443 dependencies=-
...

IncompatibleLocal means "there is a member with this name, but not a version that fits". With the default policy, Gritpack uses the published 0.1.0 from the hub instead, so the formatting library keeps building against the core it was written for. When you are ready, raise the requirement in libs/fmt to ^0.2.0 and the edge becomes Editable again.

Choosing how members are used

The [resolution] table's workspace setting says how members stand in for published packages:

SettingBehaviour
prefer-compatibleuse a member when its version fits the requirement, else the hub's (the default)
requireuse a member whenever one has the name; a version that does not fit is an error
selecteduse members only for the members you name with --member, and what they need
disablednever substitute members; each --member is resolved on its own against the hub

Most teams never change the default. require suits a repository where every package must always build against its neighbours' working copies; with it, the 0.2.0 example above stops instead of quietly using the hub:

$ gritpack workspace graph
invalid hybrid workspace: @bea/report requires @bea/core ^0.1.0, but the editable member has version 0.2.0

What just happened

  • gritpack-workspace.toml lists the packages in a repository; each member keeps its own manifest.
  • Dependencies between members are editable: your working copy is used. Everything else comes from the hub as an immutable version.
  • workspace status, graph and affected answer what is here, where each dependency comes from and what a change touches.
  • One lock at the root records both kinds, and workspace verify checks them.

You can stop here

If you only develop the packages together and publish them one by one, you have what you need. If they must ship together, read on.

Next