HTTP Queries Coming To The Internet Computer SDK

There is a big breaking change coming to the way developers build frontends on the Internet Computer. This will not affect new projects and documentation as it is a one time event.

A Recap

Currently, if you’re developing a frontend on the Internet Computer, your canister provides a retrieve() function that takes a path and returns a blob. Then that function has an index.js file that contains the root JavaScript.

Going to CANISTER_ID.ic0.app loads a root static HTML and JavaScript (what we call the bootstrap code) that does 3 things;

• Create a secure worker that contains your private key, then
• Polyfill the window.ic to provide some mechanics to the application running to talk to the Internet Computer, then
• call the canister’s retrieve() method with index.js as the path, and evals it, giving up control on the DOM and the page to the canister’s JavaScript code.

This is a big departure on how people are build web applications currently; there was no HTML, downloading assets (like PNG) was either done by loading them from another domain (e.g. an AWS bucket) or loading in JavaScript, transforming them into data URI, and setting the src attribute accordingly. This in turn had the problem that browsers were not waiting for relayout, deferring loading or executing JavaScript out of bands. And this was before looking at things like E-Tags, SRI, base tags, caching, and other features that browser excelled at.

In short, it was not the great experience that frontends developers and users were used to. There were improvements in term of security and decentralization, but those were negated by the poor user experience.

Over the last year we’ve been constantly re-evaluating and gathering feedback, and decided that we should enable developers more, while working on the potential for an equivalent security model.

Replacing the Bootstrap Server

Thus a proposal that was in the making for while came back to the surface; directly having canisters answer HTTP Requests. This can be done easily;

• Have an HTTP Server that takes a request, boxes it (its method, uri, headers and body) into a Candid structure, and resolves the Canister ID in a similar way that the bootstrap used to.
• This HTTP Server (lets call it the HTTP Middleware) would then use an Agent to talk to the canister and call a http_request() query method.
• If the canister implements this method, this HTTP Server would decode the response, take the headers and body and construct an HTTP Response with this.
• As one last step of backward compatibility, if the canister does not implement this method, we will return a specially made bootstrap server that points out the deprecation to users.

For any errors in the process, a 400 Bad Request response will be given for any invalid requests (e.g. could not find or decode a canister ID), a 500 Internal Server Error for errors from the HTTP middleware itself (e.g. could not connect to a replica), and a 502 Bad Gateway if an error is coming from the replica itself (including canister trapping).

dfx itself will follow these errors but also include details in the body itself.

Revisiting the Asset Storage Canister

To make this transition easier, new and existing projects built with the newer version of dfx will include an improved Asset Canister that supports http_request() out of the box. This means that assets you upload (including binary assets like images) will be available directly from your browser using their URL.

For example, in a new project, the sample-asset.txt file would be uploaded and available after publishing to the IC at https://CANISTER_ID.ic0.app/sample-asset.txt.

We are also going to be providing additional functionalities for managing cache, default asset (for handling 404s), and more HTTP specific features to the asset canister in the future.

Routing

Both the /api (for replicas) and /_ (for tool purposes) routes are reserved by the HTTP Request specification. Any other route are now usable by your application. This means you don’t have to rely on a hash router anymore.

Candid UI

Because of this routing limitation, the Candid UI had to be moved to a different URI, on localhost. Now Candid will be available at /_/candid. Be sure to update your bookmarks!

The Structure of a new DFX Project

Before going into how to migrate a new project, let’s take a small look at a new project. The frontend changes include:

• dfx.json. The DFX configuration still includes a frontend key for the asset canister, but it now points to an index.html file instead of the javascript entry point.
• package.json. Now has webpack 5 installed by default. This was a bit overdue.
• webpack.config.js. This webpack config is very similar to the one previously used as it is separated in two parts; generate the list of canister imports for your JavaScript, and generate the configuration for each canister which has a frontend key.
• src/<project_name>_assets/src/index.html. This is new. You now have an index.html file that is the HTML file of your frontend. It will (currently) be served by default by the asset canister when a file isn’t found.

Finally, there’s the old index.js, but its content changed significantly. Let’s take a look.

Agent and Actor Creation

The import section is slightly different. Where before there was only 1 import from files generated by dfx, now there are two, and we don’t import a canister Actor instance directly from generated files.

import { Actor, HttpAgent } from '@dfinity/agent';
import { idlFactory as example_idl, canisterId as example_id } from 'dfx-generated/example';

const agent = new HttpAgent();
const example = Actor.createActor(example_idl, { agent, canisterId: example_id });

With this new paradigm, without Bootstrap to help us create an Actor, we have to explicitly create an Agent instance, and then create the Actor which we’ll use for our canister as we did before (it’s the same type as before).

This is better for a couple reasons; first, the Agent itself is entirely configurable by the application, and so is the Actor. For example, authentication can only be set when the Agent is constructed, so if you want to manage a user identity, you’ll need to do it before creating the Agent (there are ways that we can help with this, see <<INSERT AUTHENTICATION DOCUMENTATION HERE>>).

Second,

Migrating an Existing Project

If you have an existing project, chances are it will not work seamlessly once you update DFX. Unfortunately, a direct migration path isn’t possible in this case. There are two suggested ways to upgrade your current frontend;

Create a New Project

Deprecation

A period of deprecation will be given