Name IMG_framebuffer_downsample Name Strings GL_IMG_framebuffer_downsample Contributors Tobias Hector, Imagination Technologies (tobias.hector 'at' imgtec.com) Contact Tobias Hector (tobias.hector 'at' imgtec.com) Status Complete Version Last Modified Date: August 20, 2015 Revision: 11 Number OpenGL ES Extension #255 Dependencies OpenGL ES 2.0 or OES_framebuffer_object are required. This extension is written against the OpenGL ES 3.0.4 Specification (August 27, 2014). This extension has interactions with GL_EXT_multisampled_render_to_texture. This extension has interactions with OpenGL ES 3.1. This extension has interactions with GL_EXT_color_buffer_float. This extension has interactions with GL_EXT_color_buffer_half_float. Overview This extension introduces the ability to attach color buffers to a framebuffer that are at a lower resolution than the framebuffer itself, with the GPU automatically downsampling the color attachment to fit. This can be useful for various post-process rendering techniques where it is desirable to generate downsampled images in an efficient manner, or for a lower resolution post-process technique. This extension exposes at least a 2 x 2 downscale. Other downsampling modes may be exposed on the system and this can be queried. IP Status No known IP claims. New Procedures and Functions void FramebufferTexture2DDownsampleIMG( enum target, enum attachment, enum textarget, uint texture, int level, int xscale, int yscale); void FramebufferTextureLayerDownsampleIMG( enum target, enum attachment, uint texture, int level, int layer, int xscale, int yscale); New Tokens Returned by CheckFramebufferStatus: FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_AND_DOWNSAMPLE_IMG 0x913C Accepted by the parameter of GetBooleanv, GetIntegerv, GetFloatv, GetInteger64v, and GetInternalFormativ: NUM_DOWNSAMPLE_SCALES_IMG 0x913D Accepted by the parameter of GetIntegerv, GetInteger64v, GetIntegeri_v, GetInteger64i_v and GetInternalFormativ: DOWNSAMPLE_SCALES_IMG 0x913E Accepted by the parameter of GetFramebufferAttachmentParameteriv: FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG 0x913F Additions to Chapter 4 of the OpenGL ES 3.0 Specification: Modify figure 4.1, "Per-Fragment Operations.", to add an additional box "Downscaling" after "Additional Multisample Fragment Operations". Add a new section 4.1.11, "Downscaling": If no multisampling was performed, and downscaling is enabled, fragment outputs may be optionally downscaled in a similar way to how multiple samples are resolved. If the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_- SCALE_IMG is not {1,1}, fragment values are written to an intermediate buffer. After all other fragment operations have completed, they are then combined to a produce a single color value, and that value is written into the corresponding color buffer selected by DrawBuffers. An implementation may defer the writing of color buffers until a later time, but the state of the framebuffer must behave as if the color buffers were updated as each fragment is processed. The method of combination is not specified. If the framebuffer contains sRGB values, then it is recommended that an average of samples is computed in a linearized space, as for blending (see section 4.1.7). Otherwise, a simple average computed independently for each color component is recommended. Add the following to Section 4.4.2 "Attaching Images to Framebuffer Objects" after the paragraph describing FramebufferTexture2D: The command void FramebufferTexture2DDownsampleIMG( enum target, enum attachment, enum textarget, uint texture, int level, uint xscale, uint yscale); allows a rendering into the image of a texture object that has a lower resolution than the framebuffer. target, textarget, texture, and level correspond to the same parameters for FramebufferTexture2D and have the same restrictions. attachment corresponds to the same parameter for FramebufferTexture2D, but must be COLOR_ATTACHMENTn. xscale and yscale are multiplied by texture's width and height, respectively, to produce the effective size of the attachment when rendering. For example, a texture width of 128 with an xscale of 2 would produce a color attachment with the effective width of 256. xscale and yscale must be one of the value pairs in DOWNSAMPLE_SCALES_IMG. If the xscale and yscale value pair is not available on the implementation, then the error INVALID_VALUE is generated. The implementation allocates an implicit color buffer for the same internalformat as the specified texture, and widths and heights from the specified texture level, multiplied by xscale and yscale. This buffer is used as the target for rendering instead of the specified texture level. The buffer is associated with the attachment and gets deleted after the attachment is broken. When the texture level is used as a source or destination for any operation, the attachment is flushed, or when the attachment is broken, an implicit downsample of the color data from the color buffer to the texture level may be performed. After such a downsample, the contents of the color buffer become undefined. The operations which may cause a resolve include: * Drawing with the texture bound to an active texture unit * ReadPixels or CopyTex[Sub]Image* while the texture is attached to the framebuffer * CopyTex[Sub]Image*, Tex[Sub]Image*, CompressedTex[Sub]Image* with the specified level as destination * GenerateMipmap * Flush or Finish while the texture is attached to the framebuffer * BindFramebuffer while the texture is attached to the currently bound framebuffer. Whether each of the above cause a resolve or not is implementation- dependent. Add the following to the sub-section "Attaching Texture Images to a Framebuffer" after the paragraph describing FramebufferTextureLayer: The command void FramebufferTextureLayerDownsampleIMG( enum target, enum attachment, uint texture, int level, int layer, uint xscale, uint yscale); allows a rendering into a single layer of a texture object that has a lower resolution than the framebuffer. It operates like a combination of FramebufferTexture2DDownsampleIMG and FramebufferTextureLayer; it allows the developer to set scaling values and attaches a single layer of a three-dimensional or two-dimensional array texture level. target, attachment, level, xscale and yscale correspond to the same parameters for FramebufferTexture2DDownsampleIMG and have the same restrictions. texture can only be a two-dimensional array texture, but otherwise has the same restrictions as it does for FramebufferTextureLayer. layer corresponds to the same parameter for FramebufferTextureLayer and has the same restrictions. In the sub-section "Effects of Attaching a Texture Image", change the bullet list to the following: * The value of FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE is set to TEXTURE. * The value of FRAMEBUFFER_ATTACHMENT_OBJECT_NAME is set to texture. * The value of FRAMEBUFFER_ATTACHMENT_TEXTURE_LEVEL is set to level. * If FramebufferTexture2D is called and texture is a cube map texture, then the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_CUBE_MAP_FACE is set to textarget; otherwise it is set to the default (NONE). * If FramebufferTextureLayer is called, then the value of FRAMEBUFFER_- ATTACHMENT_TEXTURE_LAYER is set to layer; otherwise it is set to zero. * If FramebufferTexture2DDownsampleIMG or FramebufferTextureLayerDownsampleIMG is called, then the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG is set to {xscale, yscale}; otherwise it is set to {1, 1}. In section 4.4.4 "Framebuffer Completeness", add the following bullet to the end of the list in subsection "Framebuffer Attachment Completeness": * If the value of FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE is TEXTURE and the value of FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG is supported by the internal format of the attachment (see GetInternalFormativ in section 6.1.15). In section 4.4.4 "Framebuffer Completeness", add the following bullet to the end of the list in subsection "Whole Framebuffer Completeness": * The value of FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG for all attachments is {1,1}, or if not, the value of TEXTURE_SAMPLES_EXT or RENDERBUFFER_SAMPLES for all attachments is zero. FRAMEBUFFER_INCOMPLETE_MULTISAMPLE_AND_DOWNSAMPLE_IMG Additions to Chapter 6 of the OpenGL ES 3.0 Specification: Add the following bullet point to the list in Section 6.1.13 "Framebuffer Object Queries" describing valid pname values when FRAMEBUFFER_ATTACHMENT_- OBJECT_TYPE is TEXTURE: * If pname is FRAMEBUFFER_ATTACHMENT_TEXTURE_SCALE_IMG, then params will contain two integer values - the downsample scale pair for that attachment. Change the paragraph in Section 6.1.15 "Internal Format Queries" describing valid target values to: target indicates the usage of the internalformat, and must be one of RENDERBUFFER, TEXTURE_2D, TEXTURE_CUBE_MAP or TEXTURE_2D_ARRAY. Add the following paragraphs to Section 6.1.15 "Internal Format Queries" to the paragraphs describing valid pname values: If pname is NUM_DOWNSAMPLE_SCALES_IMG, the number of downscales that would be returned by querying DOWNSAMPLE_SCALES_IMG is returned in params. If pname is DOWNSAMPLE_SCALES_IMG, the available downscale pairs for the format are written into params. Formats that don't support downsampling will still return one valid downsample scale pair - {1,1}. A value of one for NUM_DOWNSAMPLE_SCALES_IMG will always mean no downscaling available, as {1,1} must be supported by every format. Targets that don't support downscaling (e.g. RENDERBUFFER) will return no downsample scale pairs. Interactions with OpenGL ES 2.0 In section 4.4.5 of the OpenGL ES 2.0 Specification "Framebuffer Completeness", subsection "Framebuffer Attachment Completeness", replace: * All attached images have the same width and height. FRAMEBUFFER_INCOMPLETE_DIMENSIONS with: * All attached images have the same value of width * xscale and height * yscale. FRAMEBUFFER_INCOMPLETE_DIMENSIONS Interactions with OpenGL ES 3.1 If OpenGL ES 3.1 is supported, replace TEXTURE_SAMPLES_EXT with TEXTURE_- SAMPLES, and add TEXTURE_2D_MULTISAMPLE to the list of valid targets for GetInternalFormativ. Interactions with EXT_multisampled_render_to_texture If EXT_multisampled_render_to_texture is not supported: - ignore references to TEXTURE_SAMPLES_EXT - the sample counts returned by GetInternalFormativ with a target of TEXTURE* will be the sample values available to be used with FramebufferTexture2DMultisampleEXT Dependencies on OpenGL ES 3.0 If OpenGL ES 3.0 or higher is not supported, ignore references to glFramebufferTextureLayerDownsample and glGetIntegeri_v. Interactions with EXT_color_buffer_float and EXT_color_buffer_half_float If either of these extensions are supported, it is not guaranteed that downscale of these formats is supported, but it may be - users will have to check with the GetInternalFormat query. This equally applies to any other additional render formats provided by extension. Errors The error INVALID_VALUE is generated if FramebufferTextureLayerDownsampleIMG or FramebufferTexture2DDownsampleIMG are are called with an and value pair that isn't reported by DOWNSAMPLE_SCALES_IMG. The error INVALID_ENUM is generated if FramebufferTextureLayerDownsampleIMG or FramebufferTexture2DDownsampleIMG are called with an that is not COLOR_ATTACHMENTn. New State Add to Table 6.14 "Framebuffer (state per attachment point)" Initial Get Value Type Get Command Value Description Sec. ----------------- ----------- --------------------- ------- --------------- ---- FRAMEBUFFER_- 2 x Z+ GetFramebuffer- {1,1} Framebuffer 4.4.2 ATTACHMENT_- AttachmentParameteriv texture scale TEXTURE_SCALE_IMG New Implementation Dependent State Add to Table 6.35 "Framebuffer Dependent Values" Minimum Get Value Type Get Command Value Description Sec. ----------------- ----------- ------------------ ------- --------------- ---- NUM_DOWNSAMPLE_- 2 x Z+ GetIntegerv 2 Number of 4.4.2 SCALES_IMG scale value pairs available DOWNSAMPLE_- 1* x 2 x Z+ GetIntegeri_v ** Scale value 4.4.2 SCALES_IMG pairs available ** At least {1,1} and {2,2} must be supported as a minimum to support this extension. Example GLint xDownscale = 1; GLint yDownscale = 1; // Select a downscale amount if possible if (extension_is_supported("GL_IMG_framebuffer_downsample") { // Query the number of available scales GLint numScales; glGetIntegerv(GL_NUM_DOWNSAMPLE_SCALES_IMG, &numScales); // 2 scale modes are supported as minimum, so only need to check for // better than 2x2 if more modes are exposed. if (numScales > 2) { // Try to select most aggressive scaling. GLint bestScale = 1; GLint tempScale[2]; GLint i; for (i = 0; i < numScales; ++i) { glGetIntegeri_v(GL_DOWNSAMPLE_SCALES_IMG, i, tempScale); // If the scaling is more aggressive, update our x/y scale values. if (tempScale[0] * tempScale[1] > bestScale) { xDownscale = tempScale[0]; yDownscale = tempScale[1]; } } } else { xDownscale = 2; yDownscale = 2; } } // Create depth texture. Depth and stencil buffers must be full size GLuint depthTexture; glGenTextures(1, &depthTexture); glBindTexture(GL_TEXTURE_2D, depthTexture); glTexStorage2D(GL_TEXTURE_2D, 1, GL_DEPTH_COMPONENT16, width, height); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glBindTexture(GL_TEXTURE_2D, 0); // Create a full size RGBA texture with single mipmap level GLuint texture; glGenTextures(1, &texture); glBindTexture(GL_TEXTURE_2D, texture); glTexStorage2D(GL_TEXTURE_2D, 0, GL_RGBA4, width, height); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glBindTexture(GL_TEXTURE_2D, 0); // Scale the width and height appropriately. GLint scaledWidth = width / xDownscale; GLint scaledHeight = height / yDownscale; // Create a reduced size RGBA texture with single mipmap level GLuint scaledTexture; glGenTextures(1, &scaledTexture); glBindTexture(GL_TEXTURE_2D, scaledTexture); glTexStorage2D(GL_TEXTURE_2D, 0, GL_RGBA4, scaledWidth, scaledHeight); glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR); glBindTexture(GL_TEXTURE_2D, 0); // Create framebuffer object, attach textures GLuint framebuffer; glGenFramebuffers(1, &framebuffer); glBindFramebuffer(GL_FRAMEBUFFER, framebuffer); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, depthTexture); glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0); glFramebufferTexture2DDownsampleIMG(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, scaledTexture, 0, xDownscale, yDownscale); // Handle unsupported cases if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE) { ... } // Draw to the texture glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT); ... // Discard the depth renderbuffer contents if possible/available if (extension_supported("GL_EXT_discard_framebuffer")) { GLenum discard_attachments[] = { GL_DEPTH_ATTACHMENT }; glDiscardFramebufferEXT(GL_FRAMEBUFFER, 1, discard_attachments); } /* Draw to the default framebuffer using the textures with various post processing effects. */ glBindFramebuffer(GL_FRAMEBUFFER, 0); glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT); glActiveTexture(GL_TEXTURE0); glBindTexture(GL_TEXTURE_2D, texture); glActiveTexture(GL_TEXTURE1); glBindTexture(GL_TEXTURE_2D, scaledTexture); ... Issues 1) Should renderbuffers be resolvable in this way too? RESOLVED No, renderbuffers are considered somewhat legacy and thus will not be supported by this extension. 2) Should any scale values other than {1,1} be mandated as minimum? RESOLVED Yes, {2,2} will also be required. Implementations may support additional values though, so a query is also added for other values. 3) What formats support downscaling? RESOLVED Formats that are guaranteed color-renderable by the core ES 3.1 specification, excluding integer and signed integer formats, support all available downscale modes. Other formats only support {1,1} (no downscaling). 4) What should happen if an application calls GetInternalFormativ with a target of TEXTURE* (not TEXTURE_2D_MULTISAMPLE)? RESOLVED For standard OpenGL ES, NUM_SAMPLE_COUNTS should be zero. However, if EXT_multisampled_render_to_texture is supported, valid configurations for FramebufferTexture2DMultisampleEXT should be returned here. Revision History Revision 1, 2014/08/27 - First draft Revision 2, 2015/03/16 - Mandated {2,2} as a required downsample scale. - Coupled x and y scale values into pairs Revision 3, 2015/03/19 - Moved framebuffer completeness information to correct (whole framebuffer completeness) section, and corrected wording. - Added note about minimum support in the overview. Revision 4, 2015/03/19 - Added a specific revision of the OpenGL ES 3.0 specification - Added an error that only COLOR_ATTACHMENTn can be used as an attachment point Revision 5, 2015/06/02 - Added internalformat query capability, so that formats can opt into downscaling support - Added section on downscaling to per-fragment operations. - Added issue about what formats support downscaling. - Restricted layer downscaling to 2D array textures. Revision 6, 2015/06/03 - Fixed typo in incomplete framebuffer condition - Added a bullet point to describe the FRAMEBUFFER_ATTACHMENT_TEXTURE_- SCALE_IMG parameter to GetFramebufferAttachmentiv - Clarified targets to GetInternalFormativ Revision 7, 2015/06/17 - Clarified interactions with EXT_color_buffer_float and EXT_color_buffer_half_float Revision 8, 2015/06/22 - Added NUM_DOWNSAMPLE_SCALES_IMG as a parameter to GetInternalFormativ - Added framebuffer attachment incomplete message and removed error when scale pair isn't supported by textures' internalformat, as internalformat is not necessarily known at attachment time. Revision 9, 2015/08/19 - Assigned enum values Revision 10, 2015/08/20 - Allowed DOWNSAMPLE_SCALES_IMG to be used with GetIntegerv/GetInteger64v Revision 11, 2015/12/18 - Fixed example - "tempScale" is an array so doesn't need to be dereferenced.