Yiwei Zhang zhangyiwei
- Last Modified Date
- IP Status
No known IP claims.
Yiwei Zhang, Google
Jesse Hall, Google
This device extension allows registration of device memory event callbacks upon device creation, so that applications or middleware can obtain detailed information about memory usage and how memory is associated with Vulkan objects. This extension exposes the actual underlying device memory usage, including allocations that are not normally visible to the application, such as memory consumed by vkCreateGraphicsPipelines. It is intended primarily for use by debug tooling rather than for production applications.
1) Should this be better expressed as an extension to VK_EXT_debug_utils and its general-purpose messenger construct?
The intended lifecycle is quite different.
We want to make this extension tied to the device’s lifecycle.
Each ICD just handles its own implementation of this extension, and this
extension will only be directly exposed from the ICD.
So we can avoid the extra implementation complexity used to accommodate the
2) Can we extend and use the existing internal allocation callbacks instead of adding the new callback structure in this extension?
Our memory reporting layer that combines this information with other memory
information it collects directly (e.g. bindings of resources to
VkDeviceMemory) would have to intercept all entry points that take a
VkAllocationCallbacks parameter and inject its own
That may be doable for the extensions we know about, but not for ones we do
The proposal would work fine in the face of most unknown extensions.
But even for ones we know about, since apps can provide a different set of
callbacks and userdata and those can be retained by the driver and used
for pool object, but not just those), we would have to dynamically allocate
the interception trampoline every time.
That is getting to be an unreasonably large amount of complexity and
We are interested in both alloc/free and import/unimport.
The latter is fairly important for tracking (and avoiding double-counting)
of swapchain images (still true with “native swapchains” based on external
memory) and media/camera interop.
Though we might be able to handle this with additional
VkInternalAllocationType values, for import/export we do want to be
able to tie this to the external resource, which is one thing that the
memoryObjectId is for.
The internal alloc/free callbacks are not extensible except via new VkInternalAllocationType values. The VkDeviceMemoryReportCallbackDataEXT in this extension is extensible. That was deliberate: there is a real possibility we will want to get extra information in the future. As one example, currently this reports only physical allocations, but we believe there are interesting cases for tracking how populated that VA region is.
The callbacks are clearly specified as only callable within the context of a call from the app into Vulkan. We believe there are some cases where drivers can allocate device memory asynchronously. This was one of the sticky issues that derailed the internal device memory allocation reporting design (which is essentially what this extension is trying to do) leading up to 1.0.
VkAllocationCallbacks is described in a section called “Host memory” and the intro to it is very explicitly about host memory. The other callbacks are all inherently about host memory. But this extension is very focused on device memory.
3) Should the callback be reporting which heap is used?
It is important for non-UMA systems to have all the device memory
allocations attributed to the corresponding device memory heaps.
For internally-allocated device memory,
heapIndex will always
correspond to an advertised heap, rather than having a magic value
indicating a non-advertised heap.
Drivers can advertise heaps that do not have any corresponding memory types
if they need to.
The pointer to the VkDeviceDeviceMemoryReportCreateInfoEXT structure
itself is const and you cannot just cast it away.
Thus we cannot update the callback array inside the structure.
In addition, we cannot drop this
pNext chain either, so making a copy
of this whole structure does not work either.
5) Should we track bulk allocations shared among multiple objects?
Take the shader heap as an example.
Some implementations will let multiple VkPipeline objects share the
same shader heap.
We are not asking the implementation to report
along with a VK_NULL_HANDLE for this bulk allocation.
Instead, this bulk allocation is considered as a layer below what this
extension is interested in.
Later, when the actual VkPipeline objects are created by suballocating
from the bulk allocation, we ask the implementation to report the valid
handles of the VkPipeline objects along with the actual suballocated
sizes and different
6) Can we require the callbacks to be always called in the same thread with the Vulkan commands?
RESOLVED No. Some implementations might choose to multiplex work from multiple application threads into a single backend thread and perform JIT allocations as a part of that flow. Since this behavior is theoretically legit, we cannot require the callbacks to be always called in the same thread with the Vulkan commands, and the note is to remind the applications to handle this case properly.
7) Should we add an additional “allocation failed” event type with things like size and heap index reported?
This fits in well with the callback infrastructure added in this extension,
and implementation touches the same code and has the same overheads as the
rest of the extension.
It could help debugging things like getting an
VK_ERROR_OUT_OF_HOST_MEMORY error when ending a command buffer.
Right now the allocation failure could have happened anywhere during
recording, and a callback would be really useful to understand where and
Revision 1, 2020-08-26 (Yiwei Zhang)
Revision 2, 2021-01-06 (Yiwei Zhang)
Minor description update
For more information, see the Vulkan Specification
This page is a generated document. Fixes and changes should be made to the generator scripts, not directly.