WebXR Body Tracking Module

Status of this document

This WebXR Augmented Reality Module is designed as a module to be implemented in addition to WebXR Device API, and is originally included in WebXR Device API which was divided into core and modules.

1. Introduction

On some XR devices it is possible to get fully articulated information about the user’s body.

This API exposes the poses of each of the users' body joints. This can be used to render an avatar in VR scenarios.

2. Initialization

If an application wants to get access to body information during a session, the session MUST be requested with an appropriate feature descriptor. The string "body-tracking" is introduced by this module as a new valid feature descriptor for articulated body tracking.

The "body-tracking" feature descriptor should only be granted for an XRSession when its XR device has support for body tracking.

2.1. XRFrame

partial interface XRFrame {
   [SameObject] readonly attribute XRBody? body;
};

The body attribute on an XR Device that has support for body tracking will be an XRBody object giving access to the underlying body-tracking capabilities. body will be set to this.

If the XRFrame belongs to an XRSession that has not been requested with the "body-tracking" feature descriptor, body MUST be null.

2.2. Body Joints

A body is made up of many body joints.

A body joint for a given body can be uniquely identified by a body joint name, which is an enum of type XRBodyJoint.

A body joint may have an associated bone that it is named after and used to orient its -Z axis. The associated bone of a body joint is the bone that comes after the joint when moving towards the fingertips. The tip and wrist joints have no associated bones.

2.3. XRBody

enum XRBodyJoint {
  "hips",
  "spine-lower",
  "spine-middle",
  "spine-upper",
  "chest",
  "neck",
  "head",
  "left-shoulder",
  "left-scapula",
  "left-arm-upper",
  "left-arm-lower",
  "left-hand-wrist-twist",
  "right-shoulder",
  "right-scapula",
  "right-arm-upper",
  "right-arm-lower",
  "right-hand-wrist-twist",
  "left-hand-palm",
  "left-hand-wrist",
  "left-hand-thumb-metacarpal",
  "left-hand-thumb-phalanx-proximal",
  "left-hand-thumb-phalanx-distal",
  "left-hand-thumb-tip",
  "left-hand-index-metacarpal",
  "left-hand-index-phalanx-proximal",
  "left-hand-index-phalanx-intermediate",
  "left-hand-index-phalanx-distal",
  "left-hand-index-tip",
  "left-hand-middle-phalanx-metacarpal",
  "left-hand-middle-phalanx-proximal",
  "left-hand-middle-phalanx-intermediate",
  "left-hand-middle-phalanx-distal",
  "left-hand-middle-tip",
  "left-hand-ring-metacarpal",
  "left-hand-ring-phalanx-proximal",
  "left-hand-ring-phalanx-intermediate",
  "left-hand-ring-phalanx-distal",
  "left-hand-ring-tip",
  "left-hand-little-metacarpal",
  "left-hand-little-phalanx-proximal",
  "left-hand-little-phalanx-intermediate",
  "left-hand-little-phalanx-distal",
  "left-hand-little-tip",
  "right-hand-palm",
  "right-hand-wrist",
  "right-hand-thumb-metacarpal",
  "right-hand-thumb-phalanx-proximal",
  "right-hand-thumb-phalanx-distal",
  "right-hand-thumb-tip",
  "right-hand-index-metacarpal",
  "right-hand-index-phalanx-proximal",
  "right-hand-index-phalanx-intermediate",
  "right-hand-index-phalanx-distal",
  "right-hand-index-tip",
  "right-hand-middle-metacarpal",
  "right-hand-middle-phalanx-proximal",
  "right-hand-middle-phalanx-intermediate",
  "right-hand-middle-phalanx-distal",
  "right-hand-middle-tip",
  "right-hand-ring-metacarpal",
  "right-hand-ring-phalanx-proximal",
  "right-hand-ring-phalanx-intermediate",
  "right-hand-ring-phalanx-distal",
  "right-hand-ring-tip",
  "right-hand-little-metacarpal",
  "right-hand-little-phalanx-proximal",
  "right-hand-little-phalanx-intermediate",
  "right-hand-little-phalanx-distal",
  "right-hand-little-tip",
  "left-upper-leg",
  "left-lower-leg",
  "left-foot-ankle-twist",
  "left-foot-ankle",
  "left-foot-subtalar",
  "left-foot-transverse",
  "left-foot-ball",
  "right-upper-leg",
  "right-lower-leg",
  "right-foot-ankle-twist",
  "right-foot-ankle",
  "right-foot-subtalar",
  "right-foot-transverse",
  "right-foot-ball"
};

[Exposed=Window]
interface XRBody {
    iterable<XRBodyJoint, XRBodySpace>;

    readonly attribute unsigned long size;
    XRBodySpace get(XRBodyJoint key);
};

The XRBodyJoint enum defines the various joints that each XRBody MUST contain.

Each XRBody object has a [[joints]] internal slot, which is an ordered map of pairs with the key of type XRBodyJoint and the value of type XRBodySpace.

The ordering of the [[joints]] internal slot is given by the list of joints under body joints.

[[joints]] MUST NOT change over the course of a session.

The value pairs to iterate over for an XRBody object are the list of value pairs with the key being the XRBodyJoint and the value being the XRBodySpace corresponding to that XRBodyJoint, ordered by list of joints} under body joints.

If an individual device does not support a joint defined in this specification, it MUST emulate it instead.

The size attribute MUST return the number 83.

The get(jointName) method when invoked on an XRBody this MUST run the following steps:

Let joints be the value of this’s [[joints]] internal slot.
Return joints[jointName]. (This implies returning undefined for unknown jointName.)

2.4. XRBodySpace

[Exposed=Window]
interface XRBodySpace: XRSpace {
  readonly attribute XRBodyJoint jointName;
};

The native origin of an XRBodySpace is the position and orientation of the underlying joint.

The native origin of the XRBodySpace may only be reported when native origins of all other XRBodySpaces on the same body are being reported. When a body is partially obscured the user agent MUST either emulate the obscured joints, or report null poses for all of the joints.

Note: This means that when fetching poses you will either get an entire body or none of it.

The native origin has its -Y direction pointing perpendicular to the skin, outwards from the palm, and -Z direction pointing along their associated bone, away from the wrist.

For tip body joints where there is no associated bone, the -Z direction is the same as that for the associated distal joint, i.e. the direction is along that of the previous bone.

The left-hand-wrist and right-hand-wrist body joints are located at the pivot point of the wrist, which is location invariant when twisting the hand without moving the forearm. The backward (+Z) direction is parallel to the line from wrist joint to middle finger metacarpal joint, and points away from the finger tips. The up (+Y) direction points out towards back of the hand and perpendicular to the skin at wrist. The X direction is perpendicular to the Y and Z directions and follows the right hand rule.

The left-hand-palm and right-hand-palm body joints are located at the center of the middle finger’s metacarpal bone. The backward (+Z) direction is parallel to the middle finger’s metacarpal bone, and points away from the finger tips. The up (+Y) direction is perpendicular to palm surface and pointing towards the back of the hand. The X direction is perpendicular to the Y and Z directions and follows the right hand rule.

Every XRBodySpace has an associated body, which is the XRBody that created it.

jointName returns the joint name of the joint it tracks.

Every XRBodySpace has an associated joint, which is the body joint corresponding to the jointName.

3. Privacy & Security Considerations

The WebXR Body Tracking API is a powerful feature that carries significant privacy risks.

Since this feature MAY return new sensor data, the User Agent MUST ask for explicit consent from the user at session creation time.

Data returned from this API, MUST NOT be so specific that one can detect individual users. If the underlying hardware returns data that is too precise, the User Agent MUST anonymize this data before revealing it through the WebXR Body Tracking API.

This API MUST only be supported in XRSessions created with XRSessionMode of "immersive-vr" or "immersive-ar". "inline" sessions MUST not support this API.

When anonymizing the body data, the UA can follow these guidelines:

Noising is discouraged in favour of rounding.
If the UA uses rounding, each joint must not be rounded independently. Instead the correct way to round is to map each body to a static body-model.
If noising, the noised data must not reveal any information over time:
- Each new WebXR session in the same browsing context must use the same noise to make sure that the data cannot be de-noised by creating multiple sessions.
- Each new browsing context must use a different noise vector.
- Any seed used to initialize the noise must not be predictable.
Anonymization must be done in a trusted environment.

Changes

Conformance

Document conventions

Conformance requirements are expressed with a combination of descriptive assertions and RFC 2119 terminology. The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in the normative parts of this document are to be interpreted as described in RFC 2119. However, for readability, these words do not appear in all uppercase letters in this specification.

All of the text of this specification is normative except sections explicitly marked as non-normative, examples, and notes. [RFC2119]

Examples in this specification are introduced with the words “for example” or are set apart from the normative text with class="example", like this:

This is an example of an informative example.

Informative notes begin with the word “Note” and are set apart from the normative text with class="note", like this:

Note, this is an informative note.