Illust WebAR Discovery App

This is a collection of Aframe/8th Wall pages to viewing Illust Space art pieces in AR, often geolocated in real space.

It also holds the Illustudio admin app in webar/src/admin, which is used for configuring art pieces, and the larger application ecosystem.

• Staging: ar.staging.illust.space

• Prod: ar.illust.space

Code Organization

Map Discovery App

Url Path: /

The main map-based AR app is set up in ./src/xr8/index.html, and uses ./src/xr8/index.ts file to set up the JavaScript code to go with it.

The basic flow of this application is: ./src/aframe/components/ui/onboarding.ts: Shows an onboarding modal, checks for some permissions, and then starts the main application by adding the geofenced-model. This uses WatchPieces to get a Firestore realtime connection for all art pieces, and through the mini-map get the art piece the user wants to navigate towards. When the user navigates to a new art piece, geofenced-modal updates target-art-piece.component to render that piece. target-art-piece then starts preloading the GLB file, and watches the user's location. When the user is close enough to see the piece, target-art-piece uses the ArtPieceRenderer to actuall configure and render the piece in AR.

Most configuration of a single Art Piece should be called in ArtPieceRenderer.configureArtPiece

Instant AR Preview

Url Path: /web/:artist-slug/:piece-slug

This renders a single ArtPiece, without checking for geolocation. It's good for previews for the owner of a piece, but should generally not be shown to other users, as the owner should be able to control who can see their art in AR.

The flow of this application is similar to the discovery app - but for single art pieces, instead of using onboarding and geofenced-model to choose a piece, it directly uses the single-art-piece component to find the art piece from the URL path, and then uses the same ArtPieceRenderer to configure and render it.

Face Tracking

Url Path: /face/:artist-slug/:piece-slug

This renders a face-tracking experience for an ArtPiece, which can be one or both of a mesh tracked on a face, or a texture overlayed on the user's face.

The flow is ./src/face/index.html loads face-ar-object, which uses the FacePieceRenderer to configure and load the pice. The face piece renderer is quite similar to the ArtPieceRender, and should probably be combined in a refactor.

Marker Tracking

Url Path: /marker/:artistSlug/:pieceSlug

This page renders an ArtPiece over an 8th Wall image marker.

Flow: ./src/marker/index.html also uses single-art-piece to load up an art piece from the URL, and users ArtPieceRenderer to configure and load it. But in marker mode, the ArtPieceRenderer also uses specify-image-targets-general to load image targets (pre-configured in 8th Wall), and multi-marker-target-general.component to keep the mesh attached to the marker (when visible).

RSA

The Max Cooper PVNP RSA experience is configured in packages/webar/src/rsa/index.html. It uses the rsa-tower component to load a set of Three.js scripts to generate an RSA tower, and shows some UI to configure it. Users can click the "copy link" button to put that configuration in the URL to make it easy to play with. In production, we should instead use RsaAddon in ArtPiece to store that configuration. When it's stored and configured through the studio, the same RSA entity will be displayed in the normal AR pages.

Webpack Config

The ./webpack directory holds the build configuration. As this is not a server-rendered application, webpack.common.js sets up each HTML page as a HtmlWebpackPlugin, with configures the root HTML file for the page, the javascript entrypoint to include (through chunks), and the final filename to serve the file as.o

Then, the configuration for target: "webar" in firebase.json sets up rewrites to allow urls like /web/:artistSlug/:artPieceSlug to load index.html files like /web/demo/index.html. Then the rewritten page looks at the URL structure to decide with piece to render.

Marker Generation for AR.js

This only applies to the old Centipede experience.

To generate the files to make an image into a marker for Ar.js, you can use the web app or NodeJS app. Note that the NodeJS app seems to have problems with JPGs right now, so the web app is probably the way to go.

See this article for tips on picking good marker images.

Using the files

However they are generated, place the generated files into /src/public/assets/markers/<marker-name> or Google Cloud Storage, and reference the filename (without an extension) in functions/src/data/ar-objects.json. It's also helpful to include the original image in that directory as demo.jpg, to make it easier to test.