Name NV_stream_remote NV_stream_cross_object NV_stream_cross_display NV_stream_cross_process NV_stream_cross_partition NV_stream_cross_system Name Strings EGL_NV_stream_remote Contributors Daniel Kartch Bogdan Naodovic James Jones Zander Clucas Contacts Daniel Kartch, NVIDIA (dkartch 'at' nvidia.com) Status Draft Version Version 4 - June 01, 2016 Number EGL Extension #114 Extension Type EGL display extension Dependencies Requires EGL_KHR_stream Interacts with EGL_KHR_stream_cross_process_fd All EGL_NV_stream_cross_* extensions require EGL_NV_stream_remote, but are independent of each other. Overview This extension provides a generic framework for establishing stream connections when the producer and consumer endpoints are not associated with the same EGLStream object. The cross-process streams allowed by the EGL_KHR_stream_cross_process_fd extension are one example of such a connection. Other potential examples include streams between objects associated with different EGLDisplays in a single process, streams between processes residing in separate virtual machine partitions on a single system, or streams between entirely separate systems connected via the internet. The approach used by EGL_KHR_stream_cross_process_fd has several drawbacks: - It creates a new pair of stream creation and query functions which are specific to both the type of stream and the method used for communicating the stream's identity. Extending this methodology would require new pairs of functions for every new type of stream connection. - It allows creation of the connected stream objects without any indication of which side will be the consumer and which will be the producer. It even allows, although this was probably not intended, a cross-process stream to be established, but have both producer and consumer exist in the same process, with the other process acting as a voyeur that can observe state and perhaps modify attributes, but not operate on frames. - The use of file descriptors generated by EGL in one process and accessed by EGL in another to establish communication has potential security flaws, and may not be available at all on some real-time operating systems. Rather than implement new connection types using this model, we propose a more generic approach that does not require a proliferation of new interfaces and avoids any ambiguities in how the stream will be used. These extensions can coexist with EGL_KHR_stream_cross_process_fd, but allow for more control of stream communication by the applications. These extensions provide the framework with which arbitrary stream object pairs can be established, but by themselves are insufficient to create such pairs. In addition, a separate extension must be used to determine the protocol by which the two objects will communicate. See for example the EGL_NV_stream_socket extension. Several optional types of separate stream objects are defined, with successive levels of remoteness. It should be assumed that a more remote type will be less efficient in transfering images, and therefore applications should choose the least remote type available that is compatible with their use cases. New Types None New Functions None New Tokens Returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is EGL_STREAM_STATE_KHR: EGL_STREAM_STATE_INITIALIZING_NV 0x3240 Accepted as attribute names in eglCreateStreamKHR, eglCreateStreamAttribKHR, eglQueryStreamKHR, and eglQueryStreamAttribKHR: EGL_STREAM_TYPE_NV 0x3241 EGL_STREAM_PROTOCOL_NV 0x3242 EGL_STREAM_ENDPOINT_NV 0x3243 Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR, as value when attribute is EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV or EGL_STREAM_ENDPOINT_NV: EGL_STREAM_LOCAL_NV 0x3244 When EGL_NV_stream_cross_object is present, accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR as value for attribute EGL_STREAM_TYPE_NV: EGL_STREAM_CROSS_OBJECT_NV 0x334D When EGL_NV_stream_cross_display is present, accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR as value for attribute EGL_STREAM_TYPE_NV: EGL_STREAM_CROSS_DISPLAY_NV 0x334E When EGL_NV_stream_cross_process is present, accepted as value for attribute EGL_STREAM_TYPE_NV by eglCreateStreamKHR and eglCreateStreamAttribKHR. When either EGL_NV_stream_cross_process or EGL_KHR_stream_cross_process_fd is present, returned by eglQueryStreamKHR and eglQueryStreamAttribKHR for attribute EGL_STREAM_TYPE_NV: EGL_STREAM_CROSS_PROCESS_NV 0x3245 When EGL_NV_stream_cross_partition is present, accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR as value for attribute EGL_STREAM_TYPE_NV: EGL_STREAM_CROSS_PARTITION_NV 0x323F When EGL_NV_stream_cross_system is present, accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR as value for attribute EGL_STREAM_TYPE_NV: EGL_STREAM_CROSS_SYSTEM_NV 0x334F If EGL_KHR_stream_cross_process_fd is present, returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is EGL_STREAM_PROTOCOL_NV: EGL_STREAM_PROTOCOL_FD_NV 0x3246 Accepted by eglCreateStreamKHR and eglCreateStreamAttribKHR, and returned by eglQueryStreamKHR and eglQueryStreamAttribKHR when attribute is EGL_STREAM_ENDPOINT_NV: EGL_STREAM_PRODUCER_NV 0x3247 EGL_STREAM_CONSUMER_NV 0x3248 Add to "Glossary" of EGL_KHR_stream: Local stream A stream implemented with both consumer and producer attached to a single EGLStream object. Remote stream A stream implemented with two EGLStream objects that communicate with each other, with the consumer attached to one object and the producer attached to the other. Add to section "3.10.1 Creating an EGLStream" in EGL_KHR_stream: If a remote EGLStream is created, as described in section "3.10.5.x Remote streams", and is paired with an EGLStream object which has conflicting values for some attributes, creation will succeed, but the stream will enter state EGL_STREAM_STATE_DISCONNECTED_KHR once communication is established. Add to list of failures in section "3.10.1 Creating an EGLStream" in EGL_KHR stream: - EGL_BAD_MATCH is generated if the value of any one of the EGL_STREAM_TYPE_NV, EGL_STREAM_PROTOCOL_NV, or EGL_STREAM_ENDPOINT_NV attributes is EGL_STREAM_LOCAL_NV and any other of these attributes is neither EGL_STREAM_LOCAL_NV nor EGL_DONT_CARE. Add to section "3.10.2 Connecting an EGLStream to a consumer" in EGL_KHR_stream: Any function which connects a consumer to an EGLStream will fail and generate an EGL_BAD_ACCESS error if the value of the EGLStream's EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_PRODUCER_NV. Add to section "3.10.3 Connecting an EGLStream to a producer" in EGL_KHR_stream: Any function which connects a producer to an EGLStream will fail and generate an EGL_BAD_ACCESS error if the value of the EGLStream's EGL_STREAM_ENDPOINT_NV attribute is EGL_STREAM_CONSUMER_NV. Add to "Table 3.10.4.4 EGLStream Attributes" in EGL_KHR_stream: Attribute Read/Write Type Section -------------------------- ---------- ------ ---------- EGL_STREAM_TYPE_NV io EGLint 3.10.4.x EGL_STREAM_PROTOCOL_NV io EGLint 3.10.4.x+1 EGL_STREAM_ENDPOINT_NV io EGLint 3.10.4.x+2 Add to beginning of state list in section "3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in EGL_KHR_stream: - EGL_STREAM_STATE_INITIALIZING_NV - The EGLStream object represents one endpoint of a remote stream and has not yet established communication with the opposite endpoint. Replace the first entry in the state transition table in section "3.10.4.3 EGL_STREAM_STATE_KHR Attribute" in EGL_KHR_stream: -> EGL_STREAM_STATE_INITIALIZING_NV A new remote EGLStream is created in this state. EGL_STREAM_STATE_INITIALIZING_NV -> EGL_STREAM_STATE_CREATED_KHR Occurs when a remote EGLStream object establishes communication with the opposite endpoint. -> EGL_STREAM_STATE_CREATED_KHR A new local EGLStream or EGLStream of undetermined type is created in this state. Add new subsections to the end of section "3.10.4 EGLStream Attributes" in EGL_KHR_stream: 3.10.4.x EGL_STREAM_TYPE_NV Attribute The EGL_STREAM_TYPE_NV attribute may be set when the stream is created, and indicates the type of connection between the EGLStream objects to which the consumer and producer are attached. Legal values are EGL_DONT_CARE and EGL_STREAM_LOCAL_NV. The default value is EGL_DONT_CARE. A value of EGL_STREAM_LOCAL_NV indicates that this is a local stream with both consumer and producer connected to a single EGLStream object. If EGL_DONT_CARE is initially specified, the value of the attribute will automatically be changed to the appropriate value once both consumer and producer are attached, depending on the functions used. Other possible values for EGL_STREAM_TYPE_NV may be defined by additional extensions to indicate a remote stream. Where used, the EGL_STREAM_PROTOCOL_NV attribute must also be set to a communication protocol compatible with the stream type. 3.10.4.x+1 EGL_STREAM_PROTOCOL_NV Attribute The EGL_STREAM_PROTOCOL_NV attribute may be set when the stream is created, and indicates the manner in which communication is established between the EGLStream objects to which the consumer and producer are attached. Legal values are EGL_DONT_CARE and EGL_STREAM_LOCAL_NV. The default value is EGL_DONT_CARE. A value of EGL_STREAM_LOCAL_NV indicates that this is a local stream with both consumer and producer connected to a single EGLStream object, so no communication protocol is required. If EGL_DONT_CARE is initially specified, the value of the attribute will automatically be changed to the appropriate value once both consumer and producer are attached, depending on the functions used. Other possible values for EGL_STREAM_PROTOCOL_NV may be defined by additional extensions to indicate the communication protocol to be used for a remote stream. Not all communication protocols are compatible with all stream types, and vice versa. If incompatible types and protocols are specified, an EGL_BAD_MATCH error will be generated. 3.10.4.x+2 EGL_STREAM_ENDPOINT_NV Attribute The EGL_STREAM_ENDPOINT_NV attribute may be set when the stream is created, and indicates the endpoints which will be attached to the EGLStream object. Legal values are EGL_DONT_CARE, EGL_STREAM_LOCAL_NV, EGL_STREAM_CONSUMER_NV, and EGL_STREAM_PRODUCER_NV. The default value is EGL_DONT_CARE. A value of EGL_STREAM_LOCAL_NV indicates that this is a local stream with both consumer and producer connected to a single EGLStream object. A value of EGL_STREAM_CONSUMER_NV indicates that the EGLStream object represents the consumer side of a remote stream. A value of EGL_STREAM_PRODUCER_NV indicates that the EGLStream object represents the producer side of a remote stream. If EGL_DONT_CARE is initially specified, the value of the attribute will automatically be changed to the appropriate value once both consumer and producer are attached, depending on the functions used. If EGL_NV_stream_cross_object is present, in section 3.10.4.x above, add EGL_STREAM_CROSS_OBJECT_NV to the list of legal values, and insert A value of EGL_STREAM_CROSS_OBJECT_NV indicates that the stream object represents one endpoint of a remote stream whose other endpoint is obtained from the same EGLDisplay. If EGL_NV_stream_cross_display is present, in section 3.10.4.x above, add EGL_STREAM_CROSS_DISPLAY_NV to the list of legal values, and insert A value of EGL_STREAM_CROSS_DISPLAY_NV indicates that the stream object represents one endpoint of a remote stream whose other endpoint may be obtained from a different EGLDisplay in the same process. If EGL_NV_stream_cross_process or EGL_NV_stream_cross_process_fd is present, in section "3.10.4.x" above, add EGL_STREAM_CROSS_PROCESS_NV to the list of legal values, and insert A value of EGL_STREAM_CROSS_PROCESS_NV indicates that the stream object represents one endpoint of a remote stream whose other endpoint may reside in a separate process. If EGL_NV_stream_cross_partition is present, in section 3.10.4.x above, add EGL_STREAM_CROSS_PARTITION_NV to the list of legal values, and insert A value of EGL_STREAM_CROSS_PARTITION_NV indicates that the stream object represents one endpoint of a remote stream whose other endpoint may reside in a separate virtual machine partition on the same system. The partitions are not required to be using the same operating systems, but must support compatible communication protocols. If EGL_NV_stream_cross_system is present, in section 3.10.4.x above, add EGL_STREAM_CROSS_SYSTEM_NV to the list of legal values, and insert A value of EGL_STREAM_CROSS_SYSTEM_NV indicates that the stream object represents one endpoint of a remote stream whose other endpoint may reside on an independent hardware system with no directly sharable memory resources. If EGL_KHR_stream_cross_process_fd is present, in section 3.10.4.x+1 above, add EGL_STREAM_PROTOCOL_FD_NV to the list of legal values, and insert A value of EGL_STREAM_PROTOCOL_FD_NV indicates that the stream is a remote stream whose communication is established using a file descriptor. The details of what this file descriptor represents are implementation dependent. If the EGL_STREAM_PROTOCOL_NV attribute is initialized with this value, the EGL_STREAM_TYPE_NV attribute must specify a value of EGL_STREAM_CROSS_PROCESS_NV or an EGL_BAD_MATCH failure is generated. If an initial value of EGL_DONT_CARE is specified and a file descriptor is subsequently obtained with eglGetStreamFileDescriptorKHR, the value will be automatically changed to EGL_STREAM_PROTOCOL_FD_NV. Add a new subsection to the end of "3.10.5 EGLStream operation": 3.10.5.x Remote streams An EGLStream object may be created as the endpoint of a remote stream by specifying EGL_STREAM_PRODUCER_NV or EGL_STREAM_CONSUMER_NV as the value for its EGL_STREAM_ENDPOINT_NV attribute. Valid and compatible EGL_STREAM_TYPE_NV and EGL_STREAM_PROTOCOL_NV values other than EGL_DONT_CARE or EGL_STREAM_LOCAL_NV must also be specified. If a value for EGL_STREAM_ENDPOINT_NV is not specified, the stream is assumed to be local, but other extensions (see for example EGL_KHR_stream_cross_process_fd) may allow it to be converted to a remote stream before the producer has been attached. When an EGLStream object is created as remote, any unspecified attribute will be initially set to a value of EGL_DONT_CARE. Pairs of EGLStream objects representing opposite endpoints of a stream are not required to specify the same attribute lists, but their attributes must be compatible. When communication is established between the endpoints, they will exchange attribute settings: - If both endpoints have a value of EGL_DONT_CARE for an attribute, the default value will be assigned. - If one endpoint has a value of EGL_DONT_CARE for an attribute, it will be set to the other endpoint's value. - If neither endpoint has a value of EGL_DONT_CARE for an attribute, their values must agree. In most cases, this means the values must be identical, but this may not be true for all attributes. In particular, one endpoint must specify an EGL_STREAM_ENDPOINT_NV value of EGL_STREAM_CONSUMER_NV, and the other must specify a value of EGL_STREAM_PRODUCER_NV. If the values for any attribute are not compatible, the stream will transition to the EGL_STREAM_STATE_DISCONNECTED_KHR state. Additionally, if the two EGLStream objects representing a remote stream are created for EGLDisplays which cannot share resources, the stream will transition to the EGL_STREAM_STATE_DISCONNECTED_KHR state. When using remote streams, there may be latency in communicating state changes between the EGLStream objects representing the two endpoints. For instance, when a new frame is inserted into the stream by the producer, the consumer endpoint may not immediately transition to EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR. Similarly, when a frame is acquired by the consumer, the producer endpoint may not immediately transition to EGL_STREAM_STATE_OLD_FRAME_AVAILABLE_KHR. Applications should rely only on the state of the endpoint on which they are operating to determine the expected results of that operation. In section "3.10.1.1 Duplicating an EGLStream from a file descriptor" of the EGL_KHR_stream_cross_process_fd extension, add to the failure list for eglGetStreamFileDescriptor - EGL_BAD_ACCESS is generated if the EGL_STREAM_TYPE_NV attribute of is anything other than EGL_DONT_CARE or EGL_STREAM_CROSS_PROCESS_NV. - EGL_BAD_ACCESS is generated if the EGL_STREAM_PROTOCOL_NV attribute of is anything other than EGL_DONT_CARE or EGL_STREAM_PROTOCOL_FD_NV. - EGL_BAD_ACCESS is generated if the EGL_STREAM_ENDPOINT_NV attribute of is EGL_STREAM_LOCAL_NV. Issues None Revision History #4 (June 01, 2016) - Removed sync object definition. It will be provided by a separate extension. - Folded in cross-partition extension. - Added types for cross-object, cross-display, and cross-system streams. - General cleanup in preparation for publication. #3 (September 16, 2015) Zander Clucas - Removed dependence of the CROSS_PROCESS type on the EGL_NV_cross_process_fd extension - Added CROSS_PROCESS to the list of STREAM_TYPE legal values - Added CROSS_PROCESS requirement for cross_process_fd protocol #2 (December 11, 2014) Daniel Kartch - Rewrote as NV draft for earlier release. - Added section for creation of sync object. - Reserved enum values. #1 (October 10, 2014) Daniel Kartch - Initial EXT draft