Introducing CLI Builders

In this blog post, we’re going to look at a new API in Angular CLI, which allows you to add CLI features and augment existing ones. We’ll discuss how to interact with this API and what are the extension points which allow you to add additional features to the CLI.

You can find the code from the examples below in this GitHub repository.

History

About a year ago, we introduced the workspace file (angular.json) in the Angular CLI and reworked many core concepts of how its commands were implemented. We ended up putting commands into boxes:

1. Schematic commands. By now, you’ve probably heard of Schematics, the library the CLI uses to generate and modify your code. It was introduced in version 5 and is now used in most commands that touch your code, such as new, generate, add and update.
2. Miscellaneous commands. These are commands that are specially coded and are not specific to a project; help, version, config, doc, our newly added analytics, and our easter eggs (ssshhh! don’t tell anyone!).
3. Task commands. This category is essentially “running a process on people’s code”. Build is a good example, but so is linting and testing.

We started designing this last one a long time ago; this was originally developed to allow people to replace their webpack configuration or switch to a different underlying build implementation. We ended up drafting an initial task running system that was simple and we could keep as experimental for the time being. We named this API “Architect”.

Even though it wasn’t officially supported, Architect was a success amongst people who wanted to use a custom build, or third party libraries that wanted ways to customize their workflow. Nx used it to execute Bazel commands, Ionic used it to run unit tests with Jest, and users could extend their webpack configuration with tools such as ngx-build-plus. And this was just the start.

In Angular CLI version 8, an improved version of this API is now stable and officially supported.

Conceptual Overview

The Architect API has tools to schedule and coordinate tasks, used by the CLI for its command implementations. It uses functions called builders as the implementation of a task (which can schedule other builders), and the workspace’s angular.json to resolve projects and targets to their builder implementation.

It’s a very generic system built to be malleable and forward looking. It contains APIs for progress reporting, logging and testing, and can be extended for new features.

Builders

Builders are functions that implement the logic and behaviour for a task that can replace a command, for example running the linter.

A builder receives two arguments; an input (or options), and a context which provides communication between the CLI and the builder. The separation of concerns here is the same as with Schematics; options are given by the CLI user, context is provided by the API, and you provide the behaviour. It can be either synchronous, asynchronous, or watching and outputting multiple values. The output should always be of type BuilderOutput, which contains a success boolean field and an optional error field which can contain an error message.

Workspace File and Targets

Architect is reliant on the angular.json workspace file to resolve targets and their options.

The angular.json separates the workspace into projects, and each projects have a number of targets. An example of a project is your default application, created when running ng new. One target of this project is build, which is run automatically when using ng build. That target has (by default) three keys:

1. builder. The name of the builder to use when running this target, which is of the form packageName:builderName.
2. options. A default set of options that is used when running this target.
3. configurations. A map of name to options to apply when running this target with a specific configuration.

The way the options are resolved when executing a target is by taking the default options object, then overwriting values from the configuration used (if any), then overwriting values from the overrides object passed to scheduleTarget(). For the Angular CLI, the overrides object is built from command line arguments. This is then validated against the schema of the builder, and only then, if valid, the context will be created and the builder itself will execute.

For more information about the workspace, see https://angular.io/guide/workspace-config.

Creating a Custom Builder

As an example, let’s create a Builder that executes a shell command. To create a Builder, use the createBuilder factory, and return a BuilderOutput object:

Now let’s add some logic to it; we want to use the user options to get the command and arguments, spawn the new process, wait for the process to finish, and if the process is successful (returns a code of 0), we will indicate that we were successful to Architect:

Handling Output

Right now the spawn method outputs everything to the process standard output and error. We probably want to forward those to the logger. We do this for two reasons; one, it makes it easier to debug when testing, and two, Architect itself might execute our builder itself in a separate process or deactivate the standard output and error (e.g. in an Electron app).

For this purpose, we can use the Logger instance available in the context object, which allows us to forward our process’s output:

Progress and Status

The last piece of API that is relevant when implementing your own builder is progress and status reporting.

In our case, the shell command either finishes, or is still executing, so there’s no point is adding progress. But we can still report status so that a parent builder calling us would know what’s going on.

To report progression, use the reportProgress method, which takes a current and (optional) total values as arguments. The total can be any number; for example, if you know how many files you have to process, the total could be the number of files, and current should be the number processed so far. This is how the tslint builder reports progress.

Validating Inputs

The options object that the builder receives is also validated by using a JSON Schema. If you’ve worked with Schematics, this is the same process. That file should live and be published with your code, and we will see how to link to it below.

In our example builder, we expect our options to be an object that receives two keys: a command that is a string, and an args array of string. So we create our schema with that validation:

Schemas are really powerful and can apply a large amount of validation. For more information about JSON Schemas, you can refer to the official JSON Schema website.

Creating a Builder Package

There is one final file to create for our custom builder package and make it compatible with the Angular CLI; the builder.json file, which links our builder implementation with its schema and name. The file looks like this:

And then in the package.json file we add a builders key pointing to that file:

This will tell Architect where to find our builder definition file.

The official name of our builder is then "@example/command-runner:command". The first part before the : is the package name (resolved using node resolution), and the second part is the builder name (resolved using the builder.json).

Testing Your Builder

The recommended way to test your builder itself is through integration testing. That is because you cannot create a context easily, you need to go through the architect scheduler.

To reduce the boilerplates, we created an easy way to instantiate Architect; you first create a JsonSchemaRegistry (for schema validation), then a TestingArchitectHost, and finally you create an Architect instance. You can then add your builders.json file.

Here’s an example of running the command builder which run ls, then validating that it ran successfully and listed the proper files. Remember that we forwarded the STDOUT of the command the logger, so we will use this:

To run the snippet above, you should use a ts-node package. If you prefer to run the test with Node, rename 'index_spec.ts' to 'index_spec.js'.

Adding the builder to a project

So let’s create a simple angular.json that shows everything we learned so far. Assuming we published our builder to @example/command-runner, and that we created a new application with ng new builder-test, our angular.json could look like this (most parts were removed for brievity):

If we were to add a new target for using (e.g.) the touch shell command on a file (that updates its modified date) using our new builder, we would npm install @example/command-runner, then update the angular.json file to look like this:

The Angular CLI has a command named run, which is the generic command to run Architect builders. It takes as its first argument a target string of the form project:target[:configuration]. To run our target, we would use the following command:

ng run builder-test:touch

Now we might want to override some arguments. Unfortunately we cannot override arrays from the command line yet, but for demonstration we can change the command itself:

ng run builder-test:touch --command=ls

This will list the src/main.ts file.

Watch Mode

Builders are expected to run once and return by default, but they can also return an Observable to implement their own watch mode (like the webpack builder). The builder handler function should return an Observable. Architect will subscribe to it until it completes or stops, and can reuse it if the builder is scheduled again with the same arguments (not guaranteed though).

1. A builder should always return a BuilderOutput object after each completion. Once it’s been completed, it can enter a watch mode that will be triggered by an external event, and if it restarts it should execute the context.reportRunning() function to tell Architect that the builder is running again. This will prevent Architect from stopping the builder if there’s a new scheduling.
2. Also, Architect will unsubscribe from the Observable when the builder is stopped (using run.stop() for example), and its teardown logic will be called. This allows you to clean up and stop a build if there’s one currently running.

In general, if your builder is watching external events, there will be 3 phases to it:

1. running. For example, webpack compiles. This ends when webpack finishes and your builder posts a BuilderOutput object to the Observable.
2. watching. Between two runs, watch the external event. For example, webpack watches the file system for any changes. This ends when webpack restarts building, and context.reportRunning() is called. This goes back to step 1.
3. completes. Either the task is fully completed (for example, webpack was supposed to run a number of times), or the builder run was stopped (using run.stop()). Your teardown logic is executed, and the Observable is freed.

Conclusion

Here’s a summary of what we learned in this post:

1. We’re providing a new API to let developers change the behaviour of Angular CLI command, or adding new ones, using builders to execute custom logic.
2. Builders can be synchronous, asynchronous, or watch for external events and run multiple times, and can schedule other builders or targets.
3. Options received by the builder when running a target are first read from the angular.json file, then overwritten by the configuration (if any), then overwritten by command line flags (if any).
4. The recommended way for testing Architect builders is using integration tests. Keep in mind that you can unit test separately the logic that the builder executes.
5. If your builder returns an Observable, it should clean up in the teardown logic of that Observable.

There will be more usage of these APIscoming up. For example, the Bazel implementation is heavily dependent on them to change the build and serve commands.

We’ve already seen the community implement other builders which allows the CLI to use jest, cypress for testing, for example. The sky is really the limit and the CLI is there to extend and adapt for your project.

Thanks for reading!