Skip to content

Commit

Permalink
Prepare 1.0.0 release:
Browse files Browse the repository at this point in the history
- README is fully updated
- Added a Hello World example showing standard and module worker using three
  • Loading branch information
kaisalmen committed Jun 18, 2021
1 parent 5d109b7 commit 0ba9232
Show file tree
Hide file tree
Showing 11 changed files with 382 additions and 57 deletions.
93 changes: 70 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,26 @@
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/kaisalmen/three-wtm/blob/main/LICENSE)
[![Gitpod Ready-to-Code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/kaisalmen/three-wtm)

***This README is being updated for 1.0.0 release...***

# Quick Overview
# Overview
The `WorkerTaskManager` allows to register tasks expressed by an initialization, and an execution function with an optional comRounting function to be run in a web worker. It creates one to a maximum number of workers that can be used for execution. Multiple execution requests can be handled in parallel of the main task, If all workers are currently occupied the requested are enqueued, and the returned promise is fulfilled once a worker becomes available again.

## Features

- `WorkerTaskManager` supports standard workers:
- Code for standard workers can be created from module code on main or just provided as file obeying the standard worker.
- exec and init functions can be declared in module and then be packaged in standard worker if needed. I used this to define the worker code once and support both code paths
- Loading of dependencies for standard workers is available
- Main-Execution fallback is available
- `WorkerTaskManager` supports module workers:
- Dependencies are declared regularly and therefore no extra functionality is needed
- Code is written in ES6+ notation and uses ES6+ features and class notation is used.
- `WorkerTaskManager` has an execution queue allowing to queue more task than can be currently handled
OBJLoader2
- `WorkerTaskManager` is used by [wwobjloader2](https://github.com/kaisalmen/WWOBJLoader) and it is potentially added to three.js itself.
- Example shows [webgl_loader_workertaskmanager.html](public/examples/webgl_loader_workertaskmanager.html) how an existing `OBJLoader` can be wrapped in a worker.
- Additional utility functions are available allowing to transfer BufferGeometry, Material (meta-information) and Mesh bi-driectionally between Main and workers. All "transferables" shall be treated as such.


# Getting Started

There exist three possibilities:
Expand All @@ -18,27 +33,59 @@ There exist three possibilities:

Whatever environment you choose to start [snowpack](https://www.snowpack.dev/) is used to serve the code and the examples using it. With this setup you are able to change the code and examples without invoking an additional bundler. Snowpack ensures all imported npm modules are available if previously installed in local environment (see `npm install`).

**TODO: code example**
If you run snowpack locally you require a `nodejs` and `npm`. The Gitpod and local docker environment ensure all prerequisites are fulfilled.

## Examples

This gives you an idead how you can define a standard worker inline and the register it with `WorkerTaskManager`, init and execute it:
```javascript
const init = function ( context, id, config ) {
/* init code ... */
context.postMessage( { cmd: "init", id: id } );
};

const execute = function ( context, id, config ) {
/* execution code ... */
context.postMessage( { cmd: "execComplete", data: {} } );
}

const buildDependencies = function () {
return [
{ url: '/node_modules/three/build/three.min.js' },
{ code: 'const hello = "Hello World!";' }
]
}

const taskName = 'InlineWorker';
this.workerTaskManager.registerTaskType(taskName, init, execute, null, false, buildDependencies());
await this.workerTaskManager.initTaskType(taskName, { name: taskName })
.then( () => this.workerTaskManager.enqueueForExecution( taskName, {}, null))
.then( data => processData(data) )
.catch( e => console.error( e ) );
```

This is the same block required for a module Worker. The worker code resides in an extra file an contains all imports and exports required:
```javascript
const taskNameModule = 'ModuleWorker';
this.workerTaskManager.registerTaskTypeModule('myModuleTask', '/examples/worker/helloWorldWorker.js');
await this.workerTaskManager.initTaskType(taskNameModule, {})
.then( () => this.workerTaskManager.enqueueForExecution( taskName, {}, null))
.then( data => processData(data) )
.catch( e => console.error( e ) );

```
The examples above are fully specified and used in [wtm_helloworld.html](public/examples/wtm_helloworld.html").

# Bundling

Rollup is used to create the libraries that are contained in the deployed npm package.
Just run `npm run build`

## Local library test environment

Rollup is also used to create local test environments where the library and uglified library are used by the examples.
Use `npm run prerelease` to build the libraries and the test environment. Inside both `build/verify` or `build/verifymin` you find a npm package.json and snowpack configuration that allows to serve everything. Run `npm run dev`. The example then use the packages library.

**TODO: rollup is used for building the bundle**

**TODO: Describe local bundle (regular&minified) test env**

# Features

- `WorkerTaskManager` supports standard workers:
- Code for standard workers can be created from module code on main or just provided as file obeying the standard worker.
- exec and init functions can be declared in module and then be packaged in standard worker if needed. I used this to define the worker code once and support both code paths
- Loading of dependencies for standard workers is available
- Main-Execution fallback is available
- `WorkerTaskManager` supports module workers:
- Dependencies are declared regularly and therefore no extra functionality is needed
- Code is written in ES6+ notation and uses ES6+ features and class notation is used.
- `WorkerTaskManager` has an execution queue allowing to queue more task than can be currently handled
OBJLoader2
- `WorkerTaskManager` is used by [wwobjloader2](https://github.com/kaisalmen/WWOBJLoader) and it is potentially added to three.js itself.
- Example shows [webgl_loader_workertaskmanager.html](public/examples/webgl_loader_workertaskmanager.html) how an existing `OBJLoader` can be wrapped in a worker.
- Additional utility functions are available allowing to transfer BufferGeometry, Material (meta-information) and Mesh bi-driectionally between Main and workers. All "transferables" shall be treated as such.

Happy coding!

Expand Down
2 changes: 1 addition & 1 deletion dev/verify/min/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@
"author": "Kai Salmen",
"license": "MIT",
"dependencies": {
"three-wtm": "file:../../../three-wtm/three-wtm-1.0.0-beta.6.tgz"
"three-wtm": "file:../../../three-wtm/three-wtm-1.0.0.tgz"
}
}
2 changes: 1 addition & 1 deletion dev/verify/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@
"author": "Kai Salmen",
"license": "MIT",
"dependencies": {
"three-wtm": "file:../../../three-wtm/three-wtm-1.0.0-beta.6.tgz"
"three-wtm": "file:../../../three-wtm/three-wtm-1.0.0.tgz"
}
}
50 changes: 25 additions & 25 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "three-wtm",
"version": "1.0.0-rc.0",
"version": "1.0.0",
"description": "WorkerTaskManager for three.js",
"author": "kaisalmen",
"license": "MIT",
Expand Down Expand Up @@ -46,10 +46,10 @@
"@rollup/plugin-node-resolve": "^13.0.0",
"jsdoc": "^3.6.7",
"rimraf": "^3.0.2",
"rollup": "^2.51.2",
"rollup": "^2.52.1",
"rollup-plugin-copy": "^3.4.0",
"rollup-plugin-terser": "^7.0.2",
"snowpack": "^3.5.7",
"typescript": "^4.3.2"
"snowpack": "^3.5.9",
"typescript": "^4.3.4"
}
}
4 changes: 2 additions & 2 deletions public/examples/webgl_loader_workertaskmanager.html
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@
name: 'InlineWorker',
funcInit: InlineWorker.init,
funcExec: InlineWorker.execute,
dependencies: InlineWorker.buildStandardWorkerDependencies( '/libs/three/build/three.min.js' )
dependencies: InlineWorker.buildStandardWorkerDependencies( '/node_modules/three/build/three.min.js' )
};
this.tasksToUse.push( taskDescr );
this.workerTaskManager.registerTaskType( taskDescr.name, taskDescr.funcInit, taskDescr.funcExec, null, false, taskDescr.dependencies );
Expand All @@ -167,7 +167,7 @@
filenameObj: './models/female02_vertex_colors.obj',
funcInit: OBJLoaderWorker.init,
funcExec: OBJLoaderWorker.execute,
dependencies: OBJLoaderWorker.buildStandardWorkerDependencies( '/libs/three/build/three.min.js', '/libs/three/examples/js/loaders/OBJLoader.js' )
dependencies: OBJLoaderWorker.buildStandardWorkerDependencies( '/node_modules/three/build/three.min.js', '/node_modules/three/examples/js/loaders/OBJLoader.js' )
};
this.tasksToUse.push( taskDescrObj );
this.workerTaskManager.registerTaskType( taskDescrObj.name, taskDescrObj.funcInit, taskDescrObj.funcExec, null, false, taskDescrObj.dependencies );
Expand Down
20 changes: 20 additions & 0 deletions public/examples/worker/helloWorldWorker.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { SphereBufferGeometry } from 'three';
import { WorkerTaskManagerDefaultRouting } from '/src/loaders/workerTaskManager/worker/defaultRouting.js';
import { MeshTransport } from '/src/loaders/utils/TransportUtils.js';

const init = function ( context, id, config ) {
context.postMessage( { cmd: "init", id: id } );
};

const execute = function ( context, id, config ) {
let bufferGeometry = new SphereBufferGeometry( 40, 64, 64 );
bufferGeometry.name = config.name + config.id;
let vertexArray = bufferGeometry.getAttribute( 'position' ).array;
for ( let i = 0; i < vertexArray.length; i ++ ) vertexArray[ i ] = vertexArray[ i ] * Math.random() * 0.48;
new MeshTransport( 'execComplete', config.id )
.setGeometry( bufferGeometry, 0 )
.package( false )
.postMessage( context );
}

self.addEventListener( 'message', message => WorkerTaskManagerDefaultRouting.comRouting( self, message, null, init, execute ), false );
Loading

0 comments on commit 0ba9232

Please sign in to comment.