WebXR Lighting Estimation API Level 1

Editor’s Draft,

Previous Versions:
Issue Tracking:
GitHub
Editors:
(Google)
(Mozilla)

Abstract

This specification describes support for exposing estimates of environmental lighting conditions to WebXR sessions.

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.

This document was published by the Immersive Web Working Group as an Editors' Draft. This document is intended to become a W3C Recommendation. Feedback and comments on this specification are welcome. Please use Github issues. Discussions may also be found in the public-immersive-web@w3.org archives.

Publication as an Editors' Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 15 September 2020 W3C Process Document.

1. Introduction

The WebXR Lighting Estimation module expands the WebXR Device API, the WebXR Augmented Reality Module, and the WebXR Layers module with the ability to expose estimates of the lighting conditions of the user’s environment.

2. Light Primitives

2.1. XRLightProbe

An XRLightProbe collects estimated lighting information at a given point in the user’s environment.

[SecureContext, Exposed=Window]
interface XRLightProbe : EventTarget {
  readonly attribute XRSpace probeSpace;
  attribute EventHandler onreflectionchange;
};

The probeSpace attribute is an XRSpace that has a native origin tracking the position and orientation that the XRLightProbe's lighting estimations are being generated relative to.

The onreflectionchange attribute is an Event handler IDL attribute for the reflectionchange event type.

2.2. XRReflectionFormat

enum XRReflectionFormat {
  "srgba8",
  "rgba16f",
};

Reflection cube maps have an internal reflection format that indicates how the texture data is represented, and may change how applications choose to use the texture. Cube maps MAY be requested with the "srgba8" format or the preferredReflectionFormat of the light probe.

XRReflectionFormat WebGL Format WebGL Internal Format WebGPU Format HDR
"srgba8" RGBA SRGB8_ALPHA8 "rgba8unorm-srgb"
"rgba16f" RGBA RGBA16F "rgba16float"

2.3. XRLightEstimate

An XRLightEstimate provides the estimated lighting values for an XRLightProbe at the time represented by an XRFrame. XRLightEstimates are queried by passing an XRLightProbe to the getLightEstimate() method of an XRFrame.

[SecureContext, Exposed=Window]
interface XRLightEstimate {
  readonly attribute Float32Array sphericalHarmonicsCoefficients;
  readonly attribute DOMPointReadOnly primaryLightDirection;
  readonly attribute DOMPointReadOnly primaryLightIntensity;
};

The sphericalHarmonicsCoefficients attribute returns a Float32Array containing 9 spherical harmonics coefficients. The array MUST be 27 elements in length, with every 3 elements defining the red, green, and blue components respectively of a single coefficient. The first term of the sphericalHarmonicsCoefficients, meaning the first 3 elements of the array, MUST be representative of a valid lighting estimate. All other terms are optional, and MAY be 0 if a corresponding lighting estimate is not available due to either user privacy settings or the capabilities of the platform.

The order of coefficients in sphericalHarmonicsCoefficients, is [C00, C1-1, C10, C11, C2-2, C2-1, C20, C21, C22], where Clm is the coefficient of spherical harmonic Ylm.

The primaryLightDirection represents the direction to the primary light source from the native origin of the probeSpace of the XRLightProbe that produced the XRLightEstimate. The value MUST be a unit length 3D vector and the w value MUST be 0.0. If estimated values from the users’s environment are not available the primaryLightDirection MUST be { x: 0.0, y: 1.0, z: 0.0, w: 0.0 }, representing a light shining straight down from above.

The primaryLightIntensity represents the direction to the primary light source from the origin of the probeSpace of the XRLightProbe that produced the XRLightEstimate. The value MUST represent an RGB value mapped to the x, y, and z values respectively where each component is greater than or equal to 0.0 and the w value MUST be 1.0. If estimated values from the users’s environment are not available the primaryLightIntensity MUST be {x: 0.0, y: 0.0, z: 0.0, w: 1.0}, representing no illumination.

3. WebXR Device API Integration

Both the XRSession and XRFrame interfaces from the WebXR Device API are expanded by this module.

3.1. Session Initialization

The string "light-estimation" is introduced by this module as a new valid feature descriptor. Applications that wish to use light estimation features MUST be requested with an the "light-estimation" feature descriptor.

3.2. XRSession

The XRSession interface is extended with the ability to create new XRLightProbe instances.

dictionary XRLightProbeInit {
  XRReflectionFormat reflectionFormat = "srgba8";
};

partial interface XRSession {
  Promise<XRLightProbe> requestLightProbe(optional XRLightProbeInit options = {});
  readonly attribute XRReflectionFormat preferredReflectionFormat;
};

3.3. XRFrame

The XRFrame interface is extended with the ability to query the XRLightEstimate for a given XRLightProbe.

partial interface XRFrame {
  XRLightEstimate? getLightEstimate(XRLightProbe lightProbe);
};

4. WebXR Layers Integration

The XRWebGLBinding interface from the WebXR Layers module is expanded by this module.

4.1. XRWebGLBinding

The XRWebGLBinding interface is extended with the ability to query a reflection cube map for a given XRLightProbe.

partial interface XRWebGLBinding {
  WebGLTexture? getReflectionCubeMap(XRLightProbe lightProbe);
};

5. Events

The task source for all queue a task in this specification is the XR task source, unless otherwise specified.

5.1. Event Types

The user agent MUST fire a reflectionchange event on an XRLightProbe object each time the contents of the cube map returned by calling getReflectionCubeMap() have changed. The event MUST be of type Event.

6. Privacy & Security Considerations

The lighting estimation API shares many potential privacy and security risks with the Ambient Light Sensor API [AMBIENT-LIGHT], including:

In addition to these, there are a few vectors unique to lighting estimation to consider.

Lighting estimation must be declared when creating an XR Session as a feature descriptor, which will allow the user agent to notify the user of the potential privacy implications of allowing the lighting estimation API to be used by the website. The user agent is encouraged to NOT provide real-time updates to any portion of the lighting estimation API, especially the reflection cube map. By default, only low spaital frequency and low temporal frequency information should be returned by the WebXR API. Reflection cube maps should be kept low resolution, unless the user has also consented to camera permissions for a particular origin.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[AMBIENT-LIGHT]
Anssi Kostiainen. Ambient Light Sensor. 7 March 2019. WD. URL: https://www.w3.org/TR/ambient-light/
[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[GEOMETRY-1]
Simon Pieters; Chris Harrelson. Geometry Interfaces Module Level 1. 4 December 2018. CR. URL: https://www.w3.org/TR/geometry-1/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[WEBGL-2]
Dean Jackson; Jeff Gilbert. WebGL 2.0 Specification. 12 August 2017. URL: https://www.khronos.org/registry/webgl/specs/latest/2.0/
[WebIDL]
Boris Zbarsky. Web IDL. 15 December 2016. ED. URL: https://heycam.github.io/webidl/
[WEBXR]
Brandon Jones; Manish Goregaokar; Nell Waliczek. WebXR Device API. 24 July 2020. WD. URL: https://www.w3.org/TR/webxr/
[WEBXR-LAYERS-1]
WebXR Layers API Level 1 URL: https://immersive-web.github.io/layers/

IDL Index

[SecureContext, Exposed=Window]
interface XRLightProbe : EventTarget {
  readonly attribute XRSpace probeSpace;
  attribute EventHandler onreflectionchange;
};

enum XRReflectionFormat {
  "srgba8",
  "rgba16f",
};

[SecureContext, Exposed=Window]
interface XRLightEstimate {
  readonly attribute Float32Array sphericalHarmonicsCoefficients;
  readonly attribute DOMPointReadOnly primaryLightDirection;
  readonly attribute DOMPointReadOnly primaryLightIntensity;
};

dictionary XRLightProbeInit {
  XRReflectionFormat reflectionFormat = "srgba8";
};

partial interface XRSession {
  Promise<XRLightProbe> requestLightProbe(optional XRLightProbeInit options = {});
  readonly attribute XRReflectionFormat preferredReflectionFormat;
};

partial interface XRFrame {
  XRLightEstimate? getLightEstimate(XRLightProbe lightProbe);
};

partial interface XRWebGLBinding {
  WebGLTexture? getReflectionCubeMap(XRLightProbe lightProbe);
};