Multi Web Worker Example

Introduction

This example is not intended for beginners. I will assume that you have understood the Single Web Worker Example. If you are unsure how IFC works, then check out: What is IFC?

Background

A single Web Worker will not be enough for a large file, say > 20MB. Yes, the Web Worker runs in the background but its synchronous code.

We can use multiple Web Workers build asynchronous code.

Check out: Asynchronous Vs Synchronous Programming

One might call this approach multithreading or parallelizing. I really don't want to go into technicalities right now... but for the sake of simplicity.

Let's call this approach:

Multithreading

This might be a new concept if you only worked with web development. If you have a background with C, Java, Python, etc. than this will be easier to understand.

Anyways, let's continue!

Prerequisites

  • A modern Web Browser

  • An understanding what multithreading is

  • Patience you must have, my young Padawan

Work with modern browsers

In order to do multithreading we need to do one thing:

We need to spawn additional, descendant dedicated Web Workers

Most modern browser should be able to do this... but not Safari.

Babel

Let's recall that we use Babel to build our bundles. In more detail, we use the preset-env.

This is where the fun begins: We can use the Browserslist integration. Simply look at the package.json file.

{
    ...
    "browserslist":"last 2 Chrome versions"
    ...
}

What and How?

What SHOULD we multithread?

First, we need to identify which parts are blocking our code. The original code has three major parts:

  1. loadIfcFileItems

  2. constructProject

  3. buildGeometry

src/IFC.js
function loadIfc(ifcData) {
    const loaded = loadIfcFileItems(ifcData);
    const structured = constructProject(loaded);
    return buildGeometry(structured);
}

As we discussed earlier, we cannot build the geometry inside a web worker. That leaves loadIfcFileItems and constructProject

Let's take a look at this picture:

Your eyes can deceive you, don't trust them. Instead, trust the Chrome DevTools Performance panel.

We can see that loadIfcFileItems and constructProject takes roughly the same amount of time to compute.

What CAN we multithread?

We must consider one thing.

What can we parse independently of other IFC entities?

If you know your A, B, and (IF)C, then you realize that we cannot do that much. The IFC schema realizes heavily on references to other entities.

Therefore, the only thing we can do is loadIfcFileItems independently.

Using multiple Web Workers

How do we load IFC file items independently?

The first thing we need to do is to split up the IFC into different parts. One approach is to split the file itself into several parts. My approach is to split each IfcType into groups.

In layman's terms:

  • Let's load all properties using one Web Worker

  • Let's load all materials using one Web Worker

  • Let's load all building elements using one Web Worker

  • Let's load all relationships using one Web Worker

  • etc. etc.

See examples/web-worker/worker/utils/custom-ifc-types.js for more info.

How do we construct multiple Web Workers?

Now that we have figured out how to divide our IFC file, let's look at how we spawn our nested Web Workers. Take a look at multi-worker.js, on line 32:

examples/web-worker/worker/multi-worker.js
for (const [ifcTypesGroupName, ifcTypesGroup] of Object.entries(ifcTypesGroups)) {
    constructWebWorker(dataSection, ifcTypesGroupName, ifcTypesGroup);
}

A simple for loop will do the trick. Each Web Worker can work on its own. Once its done, it will call the function workerDone.

examples/web-worker/worker/multi-worker.js
function workerDone(e) {
    const _loaded = e.data.loaded;
    Object.assign(loaded, _loaded);
    --running;
    if (running === 0) {
        // DO STUFF WHEN DONE
    }
}

The global variable loaded will store ALL the loaded data. In other words: loaded will store the result from each web workers.

Since the Web Works parse different IfcTypes, the result will not conflict.

We can do a dirty and quick check to see if multiple Web Workers are running.

Merging the result

We now have all the loaded data. But, remember that we have used specific Web Workers to parse the loaded data.

We are missing the references to other entities!

We can simply fix this by running this command AT THE END (!).

examples/web-worker/worker/multi-worker.js
IFCjs.referenceEntities(loaded);

Then its a matter of constructing the project like before.

examples/web-worker/worker/multi-worker.js
const structured = IFCjs.constructProject(loaded);
postMessage(structured);

Final notes

This is a very dirty and simple approach. Take a look at these commits for a general understanding of what I did.

PreviousSingle Web Worker ExampleNextResults

Last updated