Scripting with JSON
--json, --json=fields, --jq, NDJSON, JSON errors and exit codes
Sooner or later you want a script to read what Gritpack says: the newest version of a package, the transaction a release committed, whether a sync changed anything. Human output is written for people and may change between releases. JSON output keeps its schema. This chapter shows how to ask for it, how to cut it down to the fields you need, and how to tell one kind of failure from another without reading the error message.
What you will be able to do
- Ask any command that prints a result for JSON, or for only some fields.
- Run a jq expression over the result without installing jq.
- Read streaming NDJSON from
searchandpublish. - Get failures as one JSON object on standard error, and branch on the exit code.
- Know which output is a stable contract and which is not.
Two streams: results and messages
Gritpack writes the result of a command to standard output and everything
else (progress, notes, warnings, errors) to standard error. A pipe therefore
receives only the result. You saw this already with publish: the line
planning publication; ... is a message, so it never reaches the next program
in a pipe.
That rule holds in human and JSON mode alike. It is why the commands in this chapter can be piped into other tools without filtering out chatter first.
Asking for JSON
Every command that prints a result has --format. Where JSON is offered,
--json is a short way to write --format json:
$ gritpack search textkit
@ada/textkit 1.0.0 ecosystem=native score=900 dialect=c/c23 variants=1 — Small text helpers for C programs
$ gritpack search textkit --json
{"index_generation":"37db61c5...","next_cursor":null,"ranking_profile":"gritpack-lexical-v1","results":[{"deprecation_message":null,"description":"Small text helpers for C programs","ecosystem":"native","keywords":["example","text"],"latest_version":"1.0.0","matched":[{"field":"local_name","term":"textkit","weight":900}],"name":"@ada/textkit","primary_dialect":"c/c23","publisher":{"scope":"@ada"},"score":900,"status":"available","variant_count":1,"versions":["1.0.0"]}],"schema":"gritpack-catalogue-search/1"}
Most JSON documents carry a schema (or schema_version) field that names
their shape and version, such as gritpack-catalogue-search/1 above or
gritpack-project-change/1 for the project commands. A new field may appear
in a later release; a field your script reads keeps its meaning while the
schema name stays the same. Read fields by name and ignore the ones you do not
know.
The project commands add, remove, sync, update and upgrade all print
the same kind of report, so one script can read any of them:
$ gritpack sync --json
{
"changed": false,
"command": "sync",
"packages": [
{
"dialect": "c/c23",
"direct": true,
"name": "@ada/greet",
"source": "https://packages.example.com:443",
"version": "0.2.0"
}
],
"packages_dir": "./packages",
"schema": "gritpack-project-change/1"
}
changed tells you whether the packages directory was rewritten. A CI job can
use it to decide whether a later step has anything to do.
For automation.
gritpack help formattingprints a short summary of everything in this chapter. It is the same text in every installed copy of Gritpack, so it is a handy reminder on a machine you do not know.
Keeping only the fields you need
--json=a,b keeps only those fields of each record. When a command returns a
list inside an envelope, as search does with results, the fields are taken
from each record in that list:
$ gritpack search textkit --json=name,latest_version
[{"latest_version":"1.0.0","name":"@ada/textkit"}]
The field names are the names in the full JSON. If none of the names you give exists, Gritpack lists the ones that do and exits with status 2:
$ gritpack search textkit --json=nope
Error: unknown field nope; this command's fields are: deprecation_message, description, ecosystem, keywords, latest_version, matched, name, primary_dialect, publisher, score, status, variant_count, versions
That error is the quickest way to discover a command's fields. Be careful with
a list that mixes known and unknown names: in 1.2.63 --json=name,nope keeps
name and prints nope as null instead of failing. Check the spelling of
each field once against the full --json output.
Note the =: --json name (with a space) means plain --json followed by an
argument called name. That is how --json behaved before field selection
existed, and it still does.
Running jq without jq
--jq EXPRESSION runs a jq expression over the JSON and prints the result.
Gritpack carries its own jq engine, so nothing else needs to be installed.
Strings are printed bare, one per line, which is what you usually want in a
shell:
$ gritpack search --jq '.results[].name'
@ada/greet
@ada/textkit
$ gritpack sync --jq '.packages[] | .name + "@" + .version'
@ada/textkit@1.0.0
A few expressions that come up again and again:
# The versions of a package the hub offers, newest first
gritpack info @ada/textkit --jq '.families[0].family.publications[].version'
# Every direct dependency in the lock file
gritpack list --jq '.packages[] | select(.direct) | .name'
# Dependencies that have a newer version within their requirement
gritpack outdated --jq '.packages[] | select(.update_available) | .name'
# The transaction ID of a publication
gritpack publish --yes --jq '.receipt.transaction_id'
--jq implies JSON output, so you do not need --json as well. If the
expression is not valid jq, the command stops with a usage error (exit 2):
$ gritpack search --jq '.results[[['
error: --jq: ".results[[[" is not a jq expression
Under the hood.
inforeturns one entry infamiliesper ecosystem that has a package with that name, so.families[0]is the first of them. Add--ecosystem native(orcargo) when a name exists in more than one ecosystem and you need a particular one.
Streaming with NDJSON
Two commands can stream: search and publish (and publication plan)
take --format ndjson. Each record is one JSON object on its own line, so a
program can act on each as it arrives:
$ gritpack search textkit --format ndjson
{"index_generation":"37db61c5...","ranking_profile":"gritpack-lexical-v1","schema":"gritpack-catalogue-search/1","type":"search_metadata"}
{"result":{"deprecation_message":null,"description":"Small text helpers for C programs","ecosystem":"native","keywords":["example","text"],"latest_version":"1.0.0","name":"@ada/textkit",...},"type":"package"}
Every line has a type, so a reader can switch on it. For publish, the
plan records ("type":"plan", then one "type":"node" per package, then
observations) stream as single lines.
Careful. In 1.2.63,
publish --format ndjsonprints the later records (the hub's verification report and the receipt) as indented JSON spread over several lines, not one per line. A strict NDJSON reader stops there. Until that is fixed, use--format json(one document at the end) when you need the receipt.
JSON errors
By default, a failure is a sentence on standard error. With
--error-format json, or GRITPACK_ERROR_FORMAT=json in the environment, it
is one JSON object instead, followed by a newline:
$ gritpack info no-such-package --error-format json
{"schema":"gritpack/error@1","code":"GRITPACK_NOT_FOUND","category":"not_found","exit_code":5,"message":"package no-such-package was not found","causes":[]}
--error-format affects only failures; a successful command prints exactly
what it printed before. It works anywhere on the command line, and it covers
mistakes in the arguments themselves:
$ gritpack search --frobnicate --error-format json
{"schema":"gritpack/error@1","code":"GRITPACK_USAGE","category":"usage","exit_code":2,"message":"error: unexpected argument '--frobnicate' found\n\n tip: a similar argument exists: '--from-ca'\n\n...","causes":[]}
The fields schema, code, category and exit_code are the stable part.
message and causes are for people: they explain, and their wording may
change. causes is often the most useful part when you are debugging,
because the human error shows only the first line. A hub whose certificate
Gritpack does not trust, for example, reports only that the request failed in
human mode, while the JSON names the reason:
$ gritpack login ada@example.com --hub packages.example.com --code AAAA-BBBB-CCCC-DDDD-EEEE-FFFF --store file --error-format json
{"schema":"gritpack/error@1","code":"GRITPACK_UNAVAILABLE","category":"unavailable","exit_code":8,"message":"enrollment request failed: error sending request for url (https://packages.example.com/v1/enrollments)","causes":["error sending request for url (https://packages.example.com/v1/enrollments)","client error (Connect)","invalid peer certificate: UnknownIssuer"]}
Exit codes
The exit code says what kind of failure happened. The list is short on purpose, and it is the same in every command:
| Exit | Code | What it means | Example that produced it |
|---|---|---|---|
| 0 | Success | ||
| 2 | GRITPACK_USAGE | Wrong arguments, configuration or request | an unknown flag; --profile naming a profile that does not exist; publish without --yes and no terminal |
| 3 | GRITPACK_AUTHENTICATION | The hub did not accept who you are | login with a used, revoked or wrong code |
| 4 | GRITPACK_AUTHORIZATION | You are known, but may not do this | yanking a version in someone else's scope; any command from a suspended account |
| 5 | GRITPACK_NOT_FOUND | No such package, version, login or installed item | info no-such-package; add of a yanked exact version |
| 6 | GRITPACK_CONFLICT | Something already there disagrees | claiming a scope someone owns; publishing a version again with different content; a publication blocked by the README rule or a licence rule |
| 7 | GRITPACK_INTEGRITY | A package, lock file or answer failed its checks | a damaged file in the download cache; verify or doctor finding a missing installed file |
| 8 | GRITPACK_UNAVAILABLE | The hub or network could not be reached; try again | a hub that is down; a hub whose TLS certificate is not trusted |
| 9 | GRITPACK_LOCAL_IO | A local file or directory could not be read or written | pack --output into a directory you cannot write |
| 70 | GRITPACK_INTERNAL | Gritpack could not finish safely | report it |
gritpack help exit-codes prints the same table. Scripts should branch on the
exit code or on code, keep message for the log, and treat a code they do
not know as a failure.
A small example in shell:
gritpack sync --frozen --error-format json 2> sync-error.json
case $? in
0) echo "installed" ;;
8) echo "hub unreachable; retrying later"; exit 75 ;;
*) jq -r .message sync-error.json; exit 1 ;;
esac
For automation. Exit 8 is the only code that means "try again later" by itself. Everything else needs a change: different arguments, a different login, a fixed manifest or a repaired cache.
Prompts, and how scripts avoid them
Gritpack never prompts when standard input is not a terminal or when the CI
environment variable is set. A command that would ask takes --yes instead:
$ gritpack publish
...
publication requires --yes when input is non-interactive or structured output is selected
publish asks for --yes whenever it cannot ask, and also whenever you ask
for structured output, so a JSON-reading script never sits waiting on a
question. The administrator commands that remove or hide something (admin user suspend and delete, admin scope unassign, admin package quarantine) and auth logout take --yes (or -y) too. In 1.2.63 they go
ahead with a warning when there is nobody to ask; the warning says that from
1.4.0 they will need --yes. Pass it now and your scripts will not break.
Naming things the way people do
Scripts used to need storage IDs and account UUIDs. Since 1.2.62 you can name
a published version as name@version and an account by its email wherever an
ID was asked for:
gritpack package yank @ada/textkit@1.0.0
gritpack evidence attestations @ada/greet@0.1.0
gritpack admin user view ada@example.com
The IDs still work, and JSON output still includes them, so a script can store the ID it was given and use it later.
What just happened
You asked Gritpack for JSON, cut it down to the fields you needed, ran jq expressions without installing jq, read the streaming form, and turned failures into one machine-readable object with a stable exit code. Results went to standard output and everything else to standard error, so each pipe carried only what it should.
You can stop here
If you write shell scripts or small tools around Gritpack, you have what you need: ask for JSON, read named fields, and branch on the exit code.
Next
Put this to work in a pipeline in Continuous integration. If you would rather make the interactive shell faster, see Shell comforts.