1. Introduction:

In traditional Web development, due to the DOM tree and event capture bubbling mechanisms, we can easily register events on a DOM node and perform a series of operations such as the parent element event broker. However, in the 3D world of WebGL, users use mouse or touch events, and the event receiver is Canvas container. How to map such clicking behavior to the 3D world needs to make use of the ability of the 3D world, and build a bridge from the Canvas plane container to the 3D world for so-called object picking

2. Basic knowledge: DOM and NDC coordinate conversion, camera ray, observer mode. For those familiar with this section, you can skip to the following code implementation.

  • A, DOM coordinate and NDC coordinate conversion: here, we know that the DOM coordinate point (0,0) is in the upper left corner of the container, and the NDC coordinate point (0,0) is in the center of the container, which requires coordinate conversion. See the two links below for more details

3 d coordinates transform screen coordinates definition

  • B. Camera rays: According to the official definition of Three.js, rays are the mechanism used for object pickup. Compared with the traditional color pick, ray pick can recognize multiple objects and get the sequence, which is more convenient and intuitive to use. Let’s look at an official code example

“`javascript

const raycaster = new THREE.Raycaster(); const mouse = new THREE.Vector2(); Function onMouseMove(event) {mouse. X = (event. ClientX/window.innerWidth) * 2-1; mouse.y = - ( event.clientY / window.innerHeight ) * 2 + 1; } function render() {// From the camera position, emit a ray raycaster. SetFromCamera (mouse, camera); / / inspection camera ray intersection of obj list const intersects. = raycaster intersectObjects (scene. Children); for ( let i = 0; i < intersects.length; I + +) {/ / set the material of intersecting objects color to red intersects [I] object, material, color, set (0 xff0000); } renderer.render( scene, camera ); } window.addEventListener( 'mousemove', onMouseMove, false ); window.requestAnimationFrame(render);Copy the code

` ` `

  • C. Observer mode: Similar to the DOM event mechanism, observer mode is very suitable for this register-trigger mechanism.

3. Concrete Implementation:

Considering that it is very inconvenient to traverse the Intersects array every time, especially when there are hundreds of Object3D in the scene. So here we define a global object store registration event, and then modify the prototype chain of Object3D to add on and on and on and off methods to implement event registration and destruction similar to DOM elements.

const globalEvent = {click: {}} Object.assign(Object3D.prototype, { $on(eventType, cb) { if(globalEvent.hasOwnProperty(eventType)) { globalEvent[eventType][this.id] = { object3d: this, callback: cb }; } else { // error warn} } $off(eventType) { if (! eventType) throw new Error('') if(globalEvent.hasOwnProperty(eventType)) { delete globalEvent[eventType][this.id] } else  { throw new Error('') } } }) init(camera) function init(camera, container) { let intersectPoint, obj, mouseX, mouseY, clicked; const targetObj = globalEvent.click const rayCaster = new Raycaster(); function down(e) { obj = null; e.preventDefault(); mouseX = event.clientX; mouseY = event.clientY; if (! globalEvent.click) return; rayCaster.setFromCamera( new Vector2( (mouseX / window.innerWidth) * 2 - 1, -(mouseY / window.innerHeight) * 2 + 1 ), camera ); let intersects = rayCaster.intersectsObjects(getVisibleList(targetObj)); if (intersects.length > 0) { if (clicked) { obj = null; return; } clicked = true; obj = intersects[0].object; intersectPoint = intersects[0].point; } else { clicked = false; } } function move(e) { event.preventDefault(); Function up(e) {event.preventdefault (); if (clicked && !! obj && obj.callback) { obj.callback(obj.object3d, intersectPoint); } clicked = false } const eventOption = { passive: false }; container.addEventListener('mousedown', down, {passive: false}); container.addEventListener('mousemove', move, {passive: false}); container.addEventListener('mouseup', up, {passive: false}); container.addEventListener('touchstart', down, {passive: false}); container.addEventListener('touchmove', move, {passive: false}); container.addEventListener('touchend', up, {passive: false}); } function getVisibleList(targetObj) { const list = [] for (const key in targetObj) { const target = targetObj[key].object3d; if (target.visible) list.push(target); $on('click', (target, point)) {} $on('click', (target, point)) {}Copy the code

4, summarize

In this paper, by modifying the prototype chain of Object3D in Three. js, click event binding is added based on the observer mode. Users can click the Canvas container to pick up objects in the scene. In the specific production environment, we also need to consider how to achieve event bubbling, rendering hierarchy, mouse drag and long press and other operations, which are not described in the main process, you can gradually improve the above code in the debug process.

If you have any questions, please contact the author at [email protected]

Bytedance — Product Development and Engineering Architecture department — VR Lab is looking for software and hardware engineers, including algorithms, embedded development, PCB engineers, front-end WebGL engineers, etc. Please send your resume to the email above.

Requirements for front-end WebGL Engineers:

1. Proficient in JavaScript and WebGL; 2. Familiar with computer graphics, rendering pipeline/linear algebra; 3. Familiar with common Shader principles and programming; 4. Familiar with at least one H5 rendering engine, such as ThreeJS, Babylon, etc. 5. Love to study new technology, have strong curiosity and thirst for knowledge, and have good coding standards; Plus: 1, familiar with VR house viewing related business 2, familiar with ThreeJS, Babylon, Unity3D related 3D works or DEMO. 3. Active in major front-end technology communities and own open source projects;Copy the code

I wrote this article in the interval of work, but it is inevitable that there are some loose concepts or code errors. I hope you can correct me more, and I will update and improve the article in time