Name QCOM_texture_foveated_subsampled_layout Name Strings GL_QCOM_texture_foveated_subsampled_layout Contributors Tate Hornbeck Jonathan Wicks Robert VanReenen Jeff Leger Contact Jeff Leger - jleger 'at' qti.qualcomm.com Status Complete Version Last Modified Date: Revision: #3 Number OpenGL ES Extension #306 Dependencies OpenGL ES 2.0 is required. This extension is written against OpenGL ES 3.2. QCOM_texture_foveated is required. This extension interacts with OES_EGL_image_external and OES_EGL_image_external_essl3. Overview This extension builds on QCOM_texture_foveated by introducing a new foveation method bit that aims to reduce memory bandwidth by avoiding the upscaling that occurred as part of the original extension. With the original FOVEATION_SCALED_BIN_METHOD_BIT_QCOM foveation method, the render target in system memory is entirely populated. The lower resolution framebuffer data is upscaled to fill the entire render target. The subsampled layout method introduced in this extension leaves the framebuffer data at the calculated lower density and instead samples directly from the the lower resolution texels. The primary usecase this is targeting is traditional VR pipeline. The application eye buffers would be rendered as textures with a subsampled layout and then sampled by the warp process. Sampling from a texture with a subsampled layout requires a new sampler layout qualifier. New Tokens Accepted as a value to for the TexParameter{if} and to for the TexParameter{if}v commands with a of TEXTURE_FOVEATED_FEATURE_BITS_QCOM returned as possible values for when GetTexParameter{if}v is queried with a of TEXTURE_FOVEATED_FEATURE_BITS_QCOM: FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM 0x4 Accepted by the parameter of GetBooleanv, GetIntegerv, GetInteger64v, and GetFloatv: MAX_SHADER_SUBSAMPLED_IMAGE_UNITS_QCOM 0x8FA1 Additions to the OpenGL ES 3.2 Specification Modify section 8.1 "Texture Objects" Modify rows in Table 8.19 "Texture parameters and their values" Name | Type | Legal Values ------------------------------------------------------------ TEXTURE_FOVEATED_FEATURE_BITS_QCOM | uint | 0, FOVEATION_ENABLE_BIT_QCOM, (FOVEATION_ENABLE_BIT_QCOM | FOVEATION_SCALED_BIN_METHOD_BIT_QCOM), (FOVEATION_ENABLE_BIT_QCOM | FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM) TEXTURE_FOVEATED_FEATURE_QUERY_QCOM | uint | 0, FOVEATION_ENABLE_BIT_QCOM, (FOVEATION_ENABLE_BIT_QCOM | FOVEATION_SCALED_BIN_METHOD_BIT_QCOM), (FOVEATION_ENABLE_BIT_QCOM | FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM) Additions to the end of section 8.19 of the OpenGL ES 3.2 Specification after the description of FOVEATION_SCALED_BIN_METHOD_QCOM: FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM: Requests that the implementation perform foveated rendering by dividing the texture render target into a grid of subregions. Each subregions will be greater than or equal to one pixel and less than or equal to the full size of the texture. Then rendering the geometry to each of these regions with a different projection or scale. No upscale is done when writing out to system memory, instead, to sample from a texture with a subsampled layout, the application must declare the sampler with a "subsampled" layout qualifier. Any attempt to read/write this subsampled memory with the CPU will result in a reconstruction pass. glGetTexParameteriv(GL_TEXTURE_2D, GL_TEXTURE_FOVEATED_FEATURE_QUERY_QCOM, &query); if ((query & GL_FOVEATION_ENABLE_BIT_QCOM == GL_FOVEATION_ENABLE_BIT_QCOM) && (query & GL_FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM == GL_FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM)) { // Implementation supports subsampled layout scaled bin method of foveation } glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM, GL_FOVEATION_ENABLE_BIT_QCOM | GL_FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM); This will set a texture as having a subsampled layout once it has been rendered to. If any shader attempts to use more than MAX_SHADER_SUBSAMPLED_IMAGE_UNITS_QCOM a compile time error will occur. Add a new row in Table 21.52 "Implementation Dependent Aggregate Shader Limits" Get Value Type Get Command Minimum Value Description Sec --------- ---- ----------- ------------- ----------- ------ MAX_SHADER_SUBSAMPLED_IMAGE_UNITS_QCOM Z+ GetIntegerv 1 No. of subsampled texture 8.19 images allowed in any shader stage. Errors INVALID_ENUM is generated by TexParameter{if} or TexParameter{if}v if is TEXTURE_FOVEATED_FEATURE_BITS_QCOM and has both FOVEATION_SCALED_BIN_METHOD_BIT_QCOM and FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM set. INVALID_VALUE is generated by TexParameter{if} or TexParameter{if}v if is TEXTURE_MAX_ANISOTROPY_EXT and is a value > 1.0f and the texture at target has a subsampled layout. INVALID_OPERATION is generated by TexParameter{if} or TexParameter{if}v if is TEXTURE_WRAP_S, TEXTURE_WRAP_T, or TEXTURE_WRAP_R and is not CLAMP_TO_EDGE or CLAMP_TO_BORDER and the texture at target has a subsampled layout. INVALID_OPERATION is generated by GenerateMipmap if the texture at target has a subsampled layout. Modifications to the OpenGL ES Shading Language Specification, Version 1.0.17 #extension GL_QCOM_texture_foveated_subsampled_layout: The above line is needed to control the GLSL features described in this section. A new preprocessor #define is added to the OpenGL ES Shading Language: #define GL_QCOM_texture_foveated_subsampled_layout 1 [[ The following applies if GL_QCOM_texture_foveated_subsampled_layout is supported. ]] Add a new Section 4.x (Layout Qualifiers) as follows: 4.x Layout Qualifiers Layout qualifiers can appear with an individual variable declaration: ; : layout( ) : comma separated list of Declarations of layouts can only be made at global scope, and only where indicated in the following subsection; their details are specific to what the declaration interface is, and are discussed individually. The tokens in any are identifiers, not keywords. Generally they can be listed in any order. Order-dependent meanings exist only if explicitly called out below. Similarly, these identifiers are not case-sensitive, unless explicitly noted otherwise. 4.x.1 Sampler Layout Qualifiers Shaders may specify the following layout qualifier only for samplers of type: sampler2D if OES_EGL_image_external is supported: samplerExternalOES The allowed layout qualifier identifiers for these samplers are: : subsampled Specifying subsampled layout qualifier for any other sampler types will result in a compile time error. Additionally, dynamically indexing an array of subsampled samplers will result in a compile time error. For samplers specified with subsampled layout qualifier only texture2D() lookup function may be used. Any other texel lookup function will produce a compile time error. This identifier specifies that the sampler is reading from a texture with a subsampled layout. Attempting to sample a texture previously rendered with FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM without this layout qualifier will result in undefined behaviour. Declarations are done as follows: layout(subsampled) mediump uniform sampler2D u_sampler2D; Modifications to the OpenGL ES Shading Language Specification, Version 3.10 #extension GL_QCOM_texture_foveated_subsampled_layout: The above line is needed to control the GLSL features described in this section. A new preprocessor #define is added to the OpenGL ES Shading Language: #define GL_QCOM_texture_foveated_subsampled_layout 1 Modify section 8.9 "Texture Functions" Add paragraph at end: For samplers specified with subsampled layout qualifier only texture() lookup function may be used. Any other texel lookup function will produce a compile time error. Add a new section 4.4.8 "Sampler Layout Qualifiers" Shaders may specify the following layout qualifier only for samplers of type: sampler2D sampler2DArray isampler2D isampler2DArray usampler2D usampler2DArray if OES_EGL_image_external_essl3 is supported: samplerExternalOES The allowed layout qualifier identifiers for these samplers are: : subsampled Specifying subsampled layout qualifier for any other sampler types will result in a compile time error. Additionally, dynamically indexing an array of subsampled samplers will result in a compile time error. This identifier specifies that the sampler is reading from a texture with a subsampled layout. Attempting to sample a texture previously rendered with FOVEATION_SUBSAMPLED_LAYOUT_METHOD_BIT_QCOM without this layout qualifier will result in undefined behaviour. Declarations are done as follows: layout(subsampled) mediump uniform sampler2D u_sampler2D; Issues 1. Mipmap support RESOLVED: Mipmaps are not supported for textures that have a subsampled layout. 2. How does ReadPixels / CPU access work? RESOLVED: A reconstruction pass will occur to fill in the subsampled texture before attempting to access. For the sake of completeness, CPU access is supported for textures with a subsampled layout. The implementation guarantees that no uninitialized data in the texture will be returned to the CPU. Accessing a texture with a subsampled layout in this manner removes any bandwidth benefits from this method of foveated rendering and should be avoided when possible. 3. How does this extension interact with BlitFramebuffer? RESOLVED: Similar to ReadPixels, BlitFramebuffer will trigger a reconstruction pass that will be followed by the normal BlitFramebuffer. 4. TexImage2D/TexSubImage2D/CopyTexImage2D RESOLVED: Similar to ReadPixels, TexImage2D type calls will trigger a reconstruction pass and then the data will be uploaded to texture memory. 5. Wrap Modes Resolved: Only CLAMP_TO_EDGE and CLAMP_TO_BORDER are allowed for textures that have a subsampled layout 6. Aniso Resolved: Aniso > 1.0f is not supported for textures that have a subsampled layout. 7. Dynamically indexing subsampled sampler arrays Resolved: Do not allow dynamically indexed subsampled sampler arrays. This is to ease the GLSL->SPIRV translation that glslang will perform. Revision History Rev. Date Author Changes ---- -------- -------- ---------------------------------------------- 1 08/22/17 tateh Initial spec 2 07/27/18 tateh Update to layout qualifiers and update wrap mode limitations 3 08/24/18 tateh Added MAX_SHADER_SUBSAMPLED_IMAGE_UNITS_QCOM query and dynamically indexed issue