1	//
2	// Copyright (c) 2017-2022 Advanced Micro Devices, Inc. All rights reserved.
3	//
4	// Permission is hereby granted, free of charge, to any person obtaining a copy
5	// of this software and associated documentation files (the "Software"), to deal
6	// in the Software without restriction, including without limitation the rights
7	// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
8	// copies of the Software, and to permit persons to whom the Software is
9	// furnished to do so, subject to the following conditions:
10	//
11	// The above copyright notice and this permission notice shall be included in
12	// all copies or substantial portions of the Software.
13	//
14	// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
15	// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
16	// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
17	// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
18	// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
19	// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
20	// THE SOFTWARE.
21	//
22
23	#ifndef AMD_VULKAN_MEMORY_ALLOCATOR_H
24	#define AMD_VULKAN_MEMORY_ALLOCATOR_H
25
26	/** \mainpage Vulkan Memory Allocator
27
28	<b>Version 3.0.1 (2022-05-26)</b>
29
30	Copyright (c) 2017-2022 Advanced Micro Devices, Inc. All rights reserved. \n
31	License: MIT
32
33	<b>API documentation divided into groups:</b> [Modules](modules.html)
34
35	\section main_table_of_contents Table of contents
36
37	- <b>User guide</b>
38	- \subpage quick_start
39	- [Project setup](@ref quick_start_project_setup)
40	- [Initialization](@ref quick_start_initialization)
41	- [Resource allocation](@ref quick_start_resource_allocation)
42	- \subpage choosing_memory_type
43	- [Usage](@ref choosing_memory_type_usage)
44	- [Required and preferred flags](@ref choosing_memory_type_required_preferred_flags)
45	- [Explicit memory types](@ref choosing_memory_type_explicit_memory_types)
46	- [Custom memory pools](@ref choosing_memory_type_custom_memory_pools)
47	- [Dedicated allocations](@ref choosing_memory_type_dedicated_allocations)
48	- \subpage memory_mapping
49	- [Mapping functions](@ref memory_mapping_mapping_functions)
50	- [Persistently mapped memory](@ref memory_mapping_persistently_mapped_memory)
51	- [Cache flush and invalidate](@ref memory_mapping_cache_control)
52	- \subpage staying_within_budget
53	- [Querying for budget](@ref staying_within_budget_querying_for_budget)
54	- [Controlling memory usage](@ref staying_within_budget_controlling_memory_usage)
55	- \subpage resource_aliasing
56	- \subpage custom_memory_pools
57	- [Choosing memory type index](@ref custom_memory_pools_MemTypeIndex)
58	- [Linear allocation algorithm](@ref linear_algorithm)
59	- [Free-at-once](@ref linear_algorithm_free_at_once)
60	- [Stack](@ref linear_algorithm_stack)
61	- [Double stack](@ref linear_algorithm_double_stack)
62	- [Ring buffer](@ref linear_algorithm_ring_buffer)
63	- \subpage defragmentation
64	- \subpage statistics
65	- [Numeric statistics](@ref statistics_numeric_statistics)
66	- [JSON dump](@ref statistics_json_dump)
67	- \subpage allocation_annotation
68	- [Allocation user data](@ref allocation_user_data)
69	- [Allocation names](@ref allocation_names)
70	- \subpage virtual_allocator
71	- \subpage debugging_memory_usage
72	- [Memory initialization](@ref debugging_memory_usage_initialization)
73	- [Margins](@ref debugging_memory_usage_margins)
74	- [Corruption detection](@ref debugging_memory_usage_corruption_detection)
75	- \subpage opengl_interop
76	- \subpage usage_patterns
77	- [GPU-only resource](@ref usage_patterns_gpu_only)
78	- [Staging copy for upload](@ref usage_patterns_staging_copy_upload)
79	- [Readback](@ref usage_patterns_readback)
80	- [Advanced data uploading](@ref usage_patterns_advanced_data_uploading)
81	- [Other use cases](@ref usage_patterns_other_use_cases)
82	- \subpage configuration
83	- [Pointers to Vulkan functions](@ref config_Vulkan_functions)
84	- [Custom host memory allocator](@ref custom_memory_allocator)
85	- [Device memory allocation callbacks](@ref allocation_callbacks)
86	- [Device heap memory limit](@ref heap_memory_limit)
87	- <b>Extension support</b>
88	- \subpage vk_khr_dedicated_allocation
89	- \subpage enabling_buffer_device_address
90	- \subpage vk_ext_memory_priority
91	- \subpage vk_amd_device_coherent_memory
92	- \subpage general_considerations
93	- [Thread safety](@ref general_considerations_thread_safety)
94	- [Versioning and compatibility](@ref general_considerations_versioning_and_compatibility)
95	- [Validation layer warnings](@ref general_considerations_validation_layer_warnings)
96	- [Allocation algorithm](@ref general_considerations_allocation_algorithm)
97	- [Features not supported](@ref general_considerations_features_not_supported)
98
99	\section main_see_also See also
100
101	- [**Product page on GPUOpen**](https://gpuopen.com/gaming-product/vulkan-memory-allocator/)
102	- [**Source repository on GitHub**](https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator)
103
104	\defgroup group_init Library initialization
105
106	\brief API elements related to the initialization and management of the entire library, especially #VmaAllocator object.
107
108	\defgroup group_alloc Memory allocation
109
110	\brief API elements related to the allocation, deallocation, and management of Vulkan memory, buffers, images.
111	Most basic ones being: vmaCreateBuffer(), vmaCreateImage().
112
113	\defgroup group_virtual Virtual allocator
114
115	\brief API elements related to the mechanism of \ref virtual_allocator - using the core allocation algorithm
116	for user-defined purpose without allocating any real GPU memory.
117
118	\defgroup group_stats Statistics
119
120	\brief API elements that query current status of the allocator, from memory usage, budget, to full dump of the internal state in JSON format.
121	See documentation chapter: \ref statistics.
122	*/
123
124
125	#ifdef __cplusplus
126	extern "C" {
127	#endif
128
129	#ifndef VULKAN_H_
130	#include <vulkan/vulkan.h>
131	#endif
132
133	// Define this macro to declare maximum supported Vulkan version in format AAABBBCCC,
134	// where AAA = major, BBB = minor, CCC = patch.
135	// If you want to use version > 1.0, it still needs to be enabled via VmaAllocatorCreateInfo::vulkanApiVersion.
136	#if !defined(VMA_VULKAN_VERSION)
137	#if defined(VK_VERSION_1_3)
138	#define VMA_VULKAN_VERSION 1003000
139	#elif defined(VK_VERSION_1_2)
140	#define VMA_VULKAN_VERSION 1002000
141	#elif defined(VK_VERSION_1_1)
142	#define VMA_VULKAN_VERSION 1001000
143	#else
144	#define VMA_VULKAN_VERSION 1000000
145	#endif
146	#endif
147
148	#if defined(__ANDROID__) && defined(VK_NO_PROTOTYPES) && VMA_STATIC_VULKAN_FUNCTIONS
149	extern PFN_vkGetInstanceProcAddr vkGetInstanceProcAddr;
150	extern PFN_vkGetDeviceProcAddr vkGetDeviceProcAddr;
151	extern PFN_vkGetPhysicalDeviceProperties vkGetPhysicalDeviceProperties;
152	extern PFN_vkGetPhysicalDeviceMemoryProperties vkGetPhysicalDeviceMemoryProperties;
153	extern PFN_vkAllocateMemory vkAllocateMemory;
154	extern PFN_vkFreeMemory vkFreeMemory;
155	extern PFN_vkMapMemory vkMapMemory;
156	extern PFN_vkUnmapMemory vkUnmapMemory;
157	extern PFN_vkFlushMappedMemoryRanges vkFlushMappedMemoryRanges;
158	extern PFN_vkInvalidateMappedMemoryRanges vkInvalidateMappedMemoryRanges;
159	extern PFN_vkBindBufferMemory vkBindBufferMemory;
160	extern PFN_vkBindImageMemory vkBindImageMemory;
161	extern PFN_vkGetBufferMemoryRequirements vkGetBufferMemoryRequirements;
162	extern PFN_vkGetImageMemoryRequirements vkGetImageMemoryRequirements;
163	extern PFN_vkCreateBuffer vkCreateBuffer;
164	extern PFN_vkDestroyBuffer vkDestroyBuffer;
165	extern PFN_vkCreateImage vkCreateImage;
166	extern PFN_vkDestroyImage vkDestroyImage;
167	extern PFN_vkCmdCopyBuffer vkCmdCopyBuffer;
168	#if VMA_VULKAN_VERSION >= 1001000
169	extern PFN_vkGetBufferMemoryRequirements2 vkGetBufferMemoryRequirements2;
170	extern PFN_vkGetImageMemoryRequirements2 vkGetImageMemoryRequirements2;
171	extern PFN_vkBindBufferMemory2 vkBindBufferMemory2;
172	extern PFN_vkBindImageMemory2 vkBindImageMemory2;
173	extern PFN_vkGetPhysicalDeviceMemoryProperties2 vkGetPhysicalDeviceMemoryProperties2;
174	#endif // #if VMA_VULKAN_VERSION >= 1001000
175	#endif // #if defined(__ANDROID__) && VMA_STATIC_VULKAN_FUNCTIONS && VK_NO_PROTOTYPES
176
177	#if !defined(VMA_DEDICATED_ALLOCATION)
178	#if VK_KHR_get_memory_requirements2 && VK_KHR_dedicated_allocation
179	#define VMA_DEDICATED_ALLOCATION 1
180	#else
181	#define VMA_DEDICATED_ALLOCATION 0
182	#endif
183	#endif
184
185	#if !defined(VMA_BIND_MEMORY2)
186	#if VK_KHR_bind_memory2
187	#define VMA_BIND_MEMORY2 1
188	#else
189	#define VMA_BIND_MEMORY2 0
190	#endif
191	#endif
192
193	#if !defined(VMA_MEMORY_BUDGET)
194	#if VK_EXT_memory_budget && (VK_KHR_get_physical_device_properties2 || VMA_VULKAN_VERSION >= 1001000)
195	#define VMA_MEMORY_BUDGET 1
196	#else
197	#define VMA_MEMORY_BUDGET 0
198	#endif
199	#endif
200
201	// Defined to 1 when VK_KHR_buffer_device_address device extension or equivalent core Vulkan 1.2 feature is defined in its headers.
202	#if !defined(VMA_BUFFER_DEVICE_ADDRESS)
203	#if VK_KHR_buffer_device_address || VMA_VULKAN_VERSION >= 1002000
204	#define VMA_BUFFER_DEVICE_ADDRESS 1
205	#else
206	#define VMA_BUFFER_DEVICE_ADDRESS 0
207	#endif
208	#endif
209
210	// Defined to 1 when VK_EXT_memory_priority device extension is defined in Vulkan headers.
211	#if !defined(VMA_MEMORY_PRIORITY)
212	#if VK_EXT_memory_priority
213	#define VMA_MEMORY_PRIORITY 1
214	#else
215	#define VMA_MEMORY_PRIORITY 0
216	#endif
217	#endif
218
219	// Defined to 1 when VK_KHR_external_memory device extension is defined in Vulkan headers.
220	#if !defined(VMA_EXTERNAL_MEMORY)
221	#if VK_KHR_external_memory
222	#define VMA_EXTERNAL_MEMORY 1
223	#else
224	#define VMA_EXTERNAL_MEMORY 0
225	#endif
226	#endif
227
228	// Define these macros to decorate all public functions with additional code,
229	// before and after returned type, appropriately. This may be useful for
230	// exporting the functions when compiling VMA as a separate library. Example:
231	// #define VMA_CALL_PRE __declspec(dllexport)
232	// #define VMA_CALL_POST __cdecl
233	#ifndef VMA_CALL_PRE
234	#define VMA_CALL_PRE
235	#endif
236	#ifndef VMA_CALL_POST
237	#define VMA_CALL_POST
238	#endif
239
240	// Define this macro to decorate pointers with an attribute specifying the
241	// length of the array they point to if they are not null.
242	//
243	// The length may be one of
244	// - The name of another parameter in the argument list where the pointer is declared
245	// - The name of another member in the struct where the pointer is declared
246	// - The name of a member of a struct type, meaning the value of that member in
247	// the context of the call. For example
248	// VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount"),
249	// this means the number of memory heaps available in the device associated
250	// with the VmaAllocator being dealt with.
251	#ifndef VMA_LEN_IF_NOT_NULL
252	#define VMA_LEN_IF_NOT_NULL(len)
253	#endif
254
255	// The VMA_NULLABLE macro is defined to be _Nullable when compiling with Clang.
256	// see: https://clang.llvm.org/docs/AttributeReference.html#nullable
257	#ifndef VMA_NULLABLE
258	#ifdef __clang__
259	#define VMA_NULLABLE _Nullable
260	#else
261	#define VMA_NULLABLE
262	#endif
263	#endif
264
265	// The VMA_NOT_NULL macro is defined to be _Nonnull when compiling with Clang.
266	// see: https://clang.llvm.org/docs/AttributeReference.html#nonnull
267	#ifndef VMA_NOT_NULL
268	#ifdef __clang__
269	#define VMA_NOT_NULL _Nonnull
270	#else
271	#define VMA_NOT_NULL
272	#endif
273	#endif
274
275	// If non-dispatchable handles are represented as pointers then we can give
276	// then nullability annotations
277	#ifndef VMA_NOT_NULL_NON_DISPATCHABLE
278	#if defined(__LP64__) || defined(_WIN64) || (defined(__x86_64__) && !defined(__ILP32__) ) || defined(_M_X64) || defined(__ia64) || defined (_M_IA64) || defined(__aarch64__) || defined(__powerpc64__)
279	#define VMA_NOT_NULL_NON_DISPATCHABLE VMA_NOT_NULL
280	#else
281	#define VMA_NOT_NULL_NON_DISPATCHABLE
282	#endif
283	#endif
284
285	#ifndef VMA_NULLABLE_NON_DISPATCHABLE
286	#if defined(__LP64__) || defined(_WIN64) || (defined(__x86_64__) && !defined(__ILP32__) ) || defined(_M_X64) || defined(__ia64) || defined (_M_IA64) || defined(__aarch64__) || defined(__powerpc64__)
287	#define VMA_NULLABLE_NON_DISPATCHABLE VMA_NULLABLE
288	#else
289	#define VMA_NULLABLE_NON_DISPATCHABLE
290	#endif
291	#endif
292
293	#ifndef VMA_STATS_STRING_ENABLED
294	#define VMA_STATS_STRING_ENABLED 1
295	#endif
296
297	////////////////////////////////////////////////////////////////////////////////
298	////////////////////////////////////////////////////////////////////////////////
299	//
300	// INTERFACE
301	//
302	////////////////////////////////////////////////////////////////////////////////
303	////////////////////////////////////////////////////////////////////////////////
304
305	// Sections for managing code placement in file, only for development purposes e.g. for convenient folding inside an IDE.
306	#ifndef _VMA_ENUM_DECLARATIONS
307
308	/**
309	\addtogroup group_init
310	@{
311	*/
312
313	/// Flags for created #VmaAllocator.
314	typedef enum VmaAllocatorCreateFlagBits
315	{
316	/** \brief Allocator and all objects created from it will not be synchronized internally, so you must guarantee they are used from only one thread at a time or synchronized externally by you.
317
318	Using this flag may increase performance because internal mutexes are not used.
319	*/
320	VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT = 0x00000001,
321	/** \brief Enables usage of VK_KHR_dedicated_allocation extension.
322
323	The flag works only if VmaAllocatorCreateInfo::vulkanApiVersion `== VK_API_VERSION_1_0`.
324	When it is `VK_API_VERSION_1_1`, the flag is ignored because the extension has been promoted to Vulkan 1.1.
325
326	Using this extension will automatically allocate dedicated blocks of memory for
327	some buffers and images instead of suballocating place for them out of bigger
328	memory blocks (as if you explicitly used #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT
329	flag) when it is recommended by the driver. It may improve performance on some
330	GPUs.
331
332	You may set this flag only if you found out that following device extensions are
333	supported, you enabled them while creating Vulkan device passed as
334	VmaAllocatorCreateInfo::device, and you want them to be used internally by this
335	library:
336
337	- VK_KHR_get_memory_requirements2 (device extension)
338	- VK_KHR_dedicated_allocation (device extension)
339
340	When this flag is set, you can experience following warnings reported by Vulkan
341	validation layer. You can ignore them.
342
343	> vkBindBufferMemory(): Binding memory to buffer 0x2d but vkGetBufferMemoryRequirements() has not been called on that buffer.
344	*/
345	VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT = 0x00000002,
346	/**
347	Enables usage of VK_KHR_bind_memory2 extension.
348
349	The flag works only if VmaAllocatorCreateInfo::vulkanApiVersion `== VK_API_VERSION_1_0`.
350	When it is `VK_API_VERSION_1_1`, the flag is ignored because the extension has been promoted to Vulkan 1.1.
351
352	You may set this flag only if you found out that this device extension is supported,
353	you enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
354	and you want it to be used internally by this library.
355
356	The extension provides functions `vkBindBufferMemory2KHR` and `vkBindImageMemory2KHR`,
357	which allow to pass a chain of `pNext` structures while binding.
358	This flag is required if you use `pNext` parameter in vmaBindBufferMemory2() or vmaBindImageMemory2().
359	*/
360	VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT = 0x00000004,
361	/**
362	Enables usage of VK_EXT_memory_budget extension.
363
364	You may set this flag only if you found out that this device extension is supported,
365	you enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
366	and you want it to be used internally by this library, along with another instance extension
367	VK_KHR_get_physical_device_properties2, which is required by it (or Vulkan 1.1, where this extension is promoted).
368
369	The extension provides query for current memory usage and budget, which will probably
370	be more accurate than an estimation used by the library otherwise.
371	*/
372	VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT = 0x00000008,
373	/**
374	Enables usage of VK_AMD_device_coherent_memory extension.
375
376	You may set this flag only if you:
377
378	- found out that this device extension is supported and enabled it while creating Vulkan device passed as VmaAllocatorCreateInfo::device,
379	- checked that `VkPhysicalDeviceCoherentMemoryFeaturesAMD::deviceCoherentMemory` is true and set it while creating the Vulkan device,
380	- want it to be used internally by this library.
381
382	The extension and accompanying device feature provide access to memory types with
383	`VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD` and `VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` flags.
384	They are useful mostly for writing breadcrumb markers - a common method for debugging GPU crash/hang/TDR.
385
386	When the extension is not enabled, such memory types are still enumerated, but their usage is illegal.
387	To protect from this error, if you don't create the allocator with this flag, it will refuse to allocate any memory or create a custom pool in such memory type,
388	returning `VK_ERROR_FEATURE_NOT_PRESENT`.
389	*/
390	VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT = 0x00000010,
391	/**
392	Enables usage of "buffer device address" feature, which allows you to use function
393	`vkGetBufferDeviceAddress*` to get raw GPU pointer to a buffer and pass it for usage inside a shader.
394
395	You may set this flag only if you:
396
397	1. (For Vulkan version < 1.2) Found as available and enabled device extension
398	VK_KHR_buffer_device_address.
399	This extension is promoted to core Vulkan 1.2.
400	2. Found as available and enabled device feature `VkPhysicalDeviceBufferDeviceAddressFeatures::bufferDeviceAddress`.
401
402	When this flag is set, you can create buffers with `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT` using VMA.
403	The library automatically adds `VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT` to
404	allocated memory blocks wherever it might be needed.
405
406	For more information, see documentation chapter \ref enabling_buffer_device_address.
407	*/
408	VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT = 0x00000020,
409	/**
410	Enables usage of VK_EXT_memory_priority extension in the library.
411
412	You may set this flag only if you found available and enabled this device extension,
413	along with `VkPhysicalDeviceMemoryPriorityFeaturesEXT::memoryPriority == VK_TRUE`,
414	while creating Vulkan device passed as VmaAllocatorCreateInfo::device.
415
416	When this flag is used, VmaAllocationCreateInfo::priority and VmaPoolCreateInfo::priority
417	are used to set priorities of allocated Vulkan memory. Without it, these variables are ignored.
418
419	A priority must be a floating-point value between 0 and 1, indicating the priority of the allocation relative to other memory allocations.
420	Larger values are higher priority. The granularity of the priorities is implementation-dependent.
421	It is automatically passed to every call to `vkAllocateMemory` done by the library using structure `VkMemoryPriorityAllocateInfoEXT`.
422	The value to be used for default priority is 0.5.
423	For more details, see the documentation of the VK_EXT_memory_priority extension.
424	*/
425	VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT = 0x00000040,
426
427	VMA_ALLOCATOR_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
428	} VmaAllocatorCreateFlagBits;
429	/// See #VmaAllocatorCreateFlagBits.
430	typedef VkFlags VmaAllocatorCreateFlags;
431
432	/** @} */
433
434	/**
435	\addtogroup group_alloc
436	@{
437	*/
438
439	/// \brief Intended usage of the allocated memory.
440	typedef enum VmaMemoryUsage
441	{
442	/** No intended memory usage specified.
443	Use other members of VmaAllocationCreateInfo to specify your requirements.
444	*/
445	VMA_MEMORY_USAGE_UNKNOWN = 0,
446	/**
447	\deprecated Obsolete, preserved for backward compatibility.
448	Prefers `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
449	*/
450	VMA_MEMORY_USAGE_GPU_ONLY = 1,
451	/**
452	\deprecated Obsolete, preserved for backward compatibility.
453	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` and `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT`.
454	*/
455	VMA_MEMORY_USAGE_CPU_ONLY = 2,
456	/**
457	\deprecated Obsolete, preserved for backward compatibility.
458	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`, prefers `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
459	*/
460	VMA_MEMORY_USAGE_CPU_TO_GPU = 3,
461	/**
462	\deprecated Obsolete, preserved for backward compatibility.
463	Guarantees `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`, prefers `VK_MEMORY_PROPERTY_HOST_CACHED_BIT`.
464	*/
465	VMA_MEMORY_USAGE_GPU_TO_CPU = 4,
466	/**
467	\deprecated Obsolete, preserved for backward compatibility.
468	Prefers not `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
469	*/
470	VMA_MEMORY_USAGE_CPU_COPY = 5,
471	/**
472	Lazily allocated GPU memory having `VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT`.
473	Exists mostly on mobile platforms. Using it on desktop PC or other GPUs with no such memory type present will fail the allocation.
474
475	Usage: Memory for transient attachment images (color attachments, depth attachments etc.), created with `VK_IMAGE_USAGE_TRANSIENT_ATTACHMENT_BIT`.
476
477	Allocations with this usage are always created as dedicated - it implies #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
478	*/
479	VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED = 6,
480	/**
481	Selects best memory type automatically.
482	This flag is recommended for most common use cases.
483
484	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
485	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
486	in VmaAllocationCreateInfo::flags.
487
488	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
489	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
490	and not with generic memory allocation functions.
491	*/
492	VMA_MEMORY_USAGE_AUTO = 7,
493	/**
494	Selects best memory type automatically with preference for GPU (device) memory.
495
496	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
497	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
498	in VmaAllocationCreateInfo::flags.
499
500	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
501	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
502	and not with generic memory allocation functions.
503	*/
504	VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE = 8,
505	/**
506	Selects best memory type automatically with preference for CPU (host) memory.
507
508	When using this flag, if you want to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT),
509	you must pass one of the flags: #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
510	in VmaAllocationCreateInfo::flags.
511
512	It can be used only with functions that let the library know `VkBufferCreateInfo` or `VkImageCreateInfo`, e.g.
513	vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo()
514	and not with generic memory allocation functions.
515	*/
516	VMA_MEMORY_USAGE_AUTO_PREFER_HOST = 9,
517
518	VMA_MEMORY_USAGE_MAX_ENUM = 0x7FFFFFFF
519	} VmaMemoryUsage;
520
521	/// Flags to be passed as VmaAllocationCreateInfo::flags.
522	typedef enum VmaAllocationCreateFlagBits
523	{
524	/** \brief Set this flag if the allocation should have its own memory block.
525
526	Use it for special, big resources, like fullscreen images used as attachments.
527	*/
528	VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT = 0x00000001,
529
530	/** \brief Set this flag to only try to allocate from existing `VkDeviceMemory` blocks and never create new such block.
531
532	If new allocation cannot be placed in any of the existing blocks, allocation
533	fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY` error.
534
535	You should not use #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT and
536	#VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT at the same time. It makes no sense.
537	*/
538	VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT = 0x00000002,
539	/** \brief Set this flag to use a memory that will be persistently mapped and retrieve pointer to it.
540
541	Pointer to mapped memory will be returned through VmaAllocationInfo::pMappedData.
542
543	It is valid to use this flag for allocation made from memory type that is not
544	`HOST_VISIBLE`. This flag is then ignored and memory is not mapped. This is
545	useful if you need an allocation that is efficient to use on GPU
546	(`DEVICE_LOCAL`) and still want to map it directly if possible on platforms that
547	support it (e.g. Intel GPU).
548	*/
549	VMA_ALLOCATION_CREATE_MAPPED_BIT = 0x00000004,
550	/** \deprecated Preserved for backward compatibility. Consider using vmaSetAllocationName() instead.
551
552	Set this flag to treat VmaAllocationCreateInfo::pUserData as pointer to a
553	null-terminated string. Instead of copying pointer value, a local copy of the
554	string is made and stored in allocation's `pName`. The string is automatically
555	freed together with the allocation. It is also used in vmaBuildStatsString().
556	*/
557	VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT = 0x00000020,
558	/** Allocation will be created from upper stack in a double stack pool.
559
560	This flag is only allowed for custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT flag.
561	*/
562	VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT = 0x00000040,
563	/** Create both buffer/image and allocation, but don't bind them together.
564	It is useful when you want to bind yourself to do some more advanced binding, e.g. using some extensions.
565	The flag is meaningful only with functions that bind by default: vmaCreateBuffer(), vmaCreateImage().
566	Otherwise it is ignored.
567
568	If you want to make sure the new buffer/image is not tied to the new memory allocation
569	through `VkMemoryDedicatedAllocateInfoKHR` structure in case the allocation ends up in its own memory block,
570	use also flag #VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT.
571	*/
572	VMA_ALLOCATION_CREATE_DONT_BIND_BIT = 0x00000080,
573	/** Create allocation only if additional device memory required for it, if any, won't exceed
574	memory budget. Otherwise return `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
575	*/
576	VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT = 0x00000100,
577	/** \brief Set this flag if the allocated memory will have aliasing resources.
578
579	Usage of this flag prevents supplying `VkMemoryDedicatedAllocateInfoKHR` when #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT is specified.
580	Otherwise created dedicated memory will not be suitable for aliasing resources, resulting in Vulkan Validation Layer errors.
581	*/
582	VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT = 0x00000200,
583	/**
584	Requests possibility to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT).
585
586	- If you use #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO*` value,
587	you must use this flag to be able to map the allocation. Otherwise, mapping is incorrect.
588	- If you use other value of #VmaMemoryUsage, this flag is ignored and mapping is always possible in memory types that are `HOST_VISIBLE`.
589	This includes allocations created in \ref custom_memory_pools.
590
591	Declares that mapped memory will only be written sequentially, e.g. using `memcpy()` or a loop writing number-by-number,
592	never read or accessed randomly, so a memory type can be selected that is uncached and write-combined.
593
594	\warning Violating this declaration may work correctly, but will likely be very slow.
595	Watch out for implicit reads introduced by doing e.g. `pMappedData[i] += x;`
596	Better prepare your data in a local variable and `memcpy()` it to the mapped pointer all at once.
597	*/
598	VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT = 0x00000400,
599	/**
600	Requests possibility to map the allocation (using vmaMapMemory() or #VMA_ALLOCATION_CREATE_MAPPED_BIT).
601
602	- If you use #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO*` value,
603	you must use this flag to be able to map the allocation. Otherwise, mapping is incorrect.
604	- If you use other value of #VmaMemoryUsage, this flag is ignored and mapping is always possible in memory types that are `HOST_VISIBLE`.
605	This includes allocations created in \ref custom_memory_pools.
606
607	Declares that mapped memory can be read, written, and accessed in random order,
608	so a `HOST_CACHED` memory type is required.
609	*/
610	VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT = 0x00000800,
611	/**
612	Together with #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT,
613	it says that despite request for host access, a not-`HOST_VISIBLE` memory type can be selected
614	if it may improve performance.
615
616	By using this flag, you declare that you will check if the allocation ended up in a `HOST_VISIBLE` memory type
617	(e.g. using vmaGetAllocationMemoryProperties()) and if not, you will create some "staging" buffer and
618	issue an explicit transfer to write/read your data.
619	To prepare for this possibility, don't forget to add appropriate flags like
620	`VK_BUFFER_USAGE_TRANSFER_DST_BIT`, `VK_BUFFER_USAGE_TRANSFER_SRC_BIT` to the parameters of created buffer or image.
621	*/
622	VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT = 0x00001000,
623	/** Allocation strategy that chooses smallest possible free range for the allocation
624	to minimize memory usage and fragmentation, possibly at the expense of allocation time.
625	*/
626	VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT = 0x00010000,
627	/** Allocation strategy that chooses first suitable free range for the allocation -
628	not necessarily in terms of the smallest offset but the one that is easiest and fastest to find
629	to minimize allocation time, possibly at the expense of allocation quality.
630	*/
631	VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT = 0x00020000,
632	/** Allocation strategy that chooses always the lowest offset in available space.
633	This is not the most efficient strategy but achieves highly packed data.
634	Used internally by defragmentation, not recomended in typical usage.
635	*/
636	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT = 0x00040000,
637	/** Alias to #VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT.
638	*/
639	VMA_ALLOCATION_CREATE_STRATEGY_BEST_FIT_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT,
640	/** Alias to #VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT.
641	*/
642	VMA_ALLOCATION_CREATE_STRATEGY_FIRST_FIT_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT,
643	/** A bit mask to extract only `STRATEGY` bits from entire set of flags.
644	*/
645	VMA_ALLOCATION_CREATE_STRATEGY_MASK =
646	VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT |
647	VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT |
648	VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
649
650	VMA_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
651	} VmaAllocationCreateFlagBits;
652	/// See #VmaAllocationCreateFlagBits.
653	typedef VkFlags VmaAllocationCreateFlags;
654
655	/// Flags to be passed as VmaPoolCreateInfo::flags.
656	typedef enum VmaPoolCreateFlagBits
657	{
658	/** \brief Use this flag if you always allocate only buffers and linear images or only optimal images out of this pool and so Buffer-Image Granularity can be ignored.
659
660	This is an optional optimization flag.
661
662	If you always allocate using vmaCreateBuffer(), vmaCreateImage(),
663	vmaAllocateMemoryForBuffer(), then you don't need to use it because allocator
664	knows exact type of your allocations so it can handle Buffer-Image Granularity
665	in the optimal way.
666
667	If you also allocate using vmaAllocateMemoryForImage() or vmaAllocateMemory(),
668	exact type of such allocations is not known, so allocator must be conservative
669	in handling Buffer-Image Granularity, which can lead to suboptimal allocation
670	(wasted memory). In that case, if you can make sure you always allocate only
671	buffers and linear images or only optimal images out of this pool, use this flag
672	to make allocator disregard Buffer-Image Granularity and so make allocations
673	faster and more optimal.
674	*/
675	VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT = 0x00000002,
676
677	/** \brief Enables alternative, linear allocation algorithm in this pool.
678
679	Specify this flag to enable linear allocation algorithm, which always creates
680	new allocations after last one and doesn't reuse space from allocations freed in
681	between. It trades memory consumption for simplified algorithm and data
682	structure, which has better performance and uses less memory for metadata.
683
684	By using this flag, you can achieve behavior of free-at-once, stack,
685	ring buffer, and double stack.
686	For details, see documentation chapter \ref linear_algorithm.
687	*/
688	VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT = 0x00000004,
689
690	/** Bit mask to extract only `ALGORITHM` bits from entire set of flags.
691	*/
692	VMA_POOL_CREATE_ALGORITHM_MASK =
693	VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT,
694
695	VMA_POOL_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
696	} VmaPoolCreateFlagBits;
697	/// Flags to be passed as VmaPoolCreateInfo::flags. See #VmaPoolCreateFlagBits.
698	typedef VkFlags VmaPoolCreateFlags;
699
700	/// Flags to be passed as VmaDefragmentationInfo::flags.
701	typedef enum VmaDefragmentationFlagBits
702	{
703	/* \brief Use simple but fast algorithm for defragmentation.
704	May not achieve best results but will require least time to compute and least allocations to copy.
705	*/
706	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT = 0x1,
707	/* \brief Default defragmentation algorithm, applied also when no `ALGORITHM` flag is specified.
708	Offers a balance between defragmentation quality and the amount of allocations and bytes that need to be moved.
709	*/
710	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT = 0x2,
711	/* \brief Perform full defragmentation of memory.
712	Can result in notably more time to compute and allocations to copy, but will achieve best memory packing.
713	*/
714	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT = 0x4,
715	/** \brief Use the most roboust algorithm at the cost of time to compute and number of copies to make.
716	Only available when bufferImageGranularity is greater than 1, since it aims to reduce
717	alignment issues between different types of resources.
718	Otherwise falls back to same behavior as #VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT.
719	*/
720	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT = 0x8,
721
722	/// A bit mask to extract only `ALGORITHM` bits from entire set of flags.
723	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_MASK =
724	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT |
725	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT |
726	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT |
727	VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT,
728
729	VMA_DEFRAGMENTATION_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
730	} VmaDefragmentationFlagBits;
731	/// See #VmaDefragmentationFlagBits.
732	typedef VkFlags VmaDefragmentationFlags;
733
734	/// Operation performed on single defragmentation move. See structure #VmaDefragmentationMove.
735	typedef enum VmaDefragmentationMoveOperation
736	{
737	/// Buffer/image has been recreated at `dstTmpAllocation`, data has been copied, old buffer/image has been destroyed. `srcAllocation` should be changed to point to the new place. This is the default value set by vmaBeginDefragmentationPass().
738	VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY = 0,
739	/// Set this value if you cannot move the allocation. New place reserved at `dstTmpAllocation` will be freed. `srcAllocation` will remain unchanged.
740	VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE = 1,
741	/// Set this value if you decide to abandon the allocation and you destroyed the buffer/image. New place reserved at `dstTmpAllocation` will be freed, along with `srcAllocation`, which will be destroyed.
742	VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY = 2,
743	} VmaDefragmentationMoveOperation;
744
745	/** @} */
746
747	/**
748	\addtogroup group_virtual
749	@{
750	*/
751
752	/// Flags to be passed as VmaVirtualBlockCreateInfo::flags.
753	typedef enum VmaVirtualBlockCreateFlagBits
754	{
755	/** \brief Enables alternative, linear allocation algorithm in this virtual block.
756
757	Specify this flag to enable linear allocation algorithm, which always creates
758	new allocations after last one and doesn't reuse space from allocations freed in
759	between. It trades memory consumption for simplified algorithm and data
760	structure, which has better performance and uses less memory for metadata.
761
762	By using this flag, you can achieve behavior of free-at-once, stack,
763	ring buffer, and double stack.
764	For details, see documentation chapter \ref linear_algorithm.
765	*/
766	VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT = 0x00000001,
767
768	/** \brief Bit mask to extract only `ALGORITHM` bits from entire set of flags.
769	*/
770	VMA_VIRTUAL_BLOCK_CREATE_ALGORITHM_MASK =
771	VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT,
772
773	VMA_VIRTUAL_BLOCK_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
774	} VmaVirtualBlockCreateFlagBits;
775	/// Flags to be passed as VmaVirtualBlockCreateInfo::flags. See #VmaVirtualBlockCreateFlagBits.
776	typedef VkFlags VmaVirtualBlockCreateFlags;
777
778	/// Flags to be passed as VmaVirtualAllocationCreateInfo::flags.
779	typedef enum VmaVirtualAllocationCreateFlagBits
780	{
781	/** \brief Allocation will be created from upper stack in a double stack pool.
782
783	This flag is only allowed for virtual blocks created with #VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT flag.
784	*/
785	VMA_VIRTUAL_ALLOCATION_CREATE_UPPER_ADDRESS_BIT = VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT,
786	/** \brief Allocation strategy that tries to minimize memory usage.
787	*/
788	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT,
789	/** \brief Allocation strategy that tries to minimize allocation time.
790	*/
791	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT,
792	/** Allocation strategy that chooses always the lowest offset in available space.
793	This is not the most efficient strategy but achieves highly packed data.
794	*/
795	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT = VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
796	/** \brief A bit mask to extract only `STRATEGY` bits from entire set of flags.
797
798	These strategy flags are binary compatible with equivalent flags in #VmaAllocationCreateFlagBits.
799	*/
800	VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MASK = VMA_ALLOCATION_CREATE_STRATEGY_MASK,
801
802	VMA_VIRTUAL_ALLOCATION_CREATE_FLAG_BITS_MAX_ENUM = 0x7FFFFFFF
803	} VmaVirtualAllocationCreateFlagBits;
804	/// Flags to be passed as VmaVirtualAllocationCreateInfo::flags. See #VmaVirtualAllocationCreateFlagBits.
805	typedef VkFlags VmaVirtualAllocationCreateFlags;
806
807	/** @} */
808
809	#endif // _VMA_ENUM_DECLARATIONS
810
811	#ifndef _VMA_DATA_TYPES_DECLARATIONS
812
813	/**
814	\addtogroup group_init
815	@{ */
816
817	/** \struct VmaAllocator
818	\brief Represents main object of this library initialized.
819
820	Fill structure #VmaAllocatorCreateInfo and call function vmaCreateAllocator() to create it.
821	Call function vmaDestroyAllocator() to destroy it.
822
823	It is recommended to create just one object of this type per `VkDevice` object,
824	right after Vulkan is initialized and keep it alive until before Vulkan device is destroyed.
825	*/
826	VK_DEFINE_HANDLE(VmaAllocator)
827
828	/** @} */
829
830	/**
831	\addtogroup group_alloc
832	@{
833	*/
834
835	/** \struct VmaPool
836	\brief Represents custom memory pool
837
838	Fill structure VmaPoolCreateInfo and call function vmaCreatePool() to create it.
839	Call function vmaDestroyPool() to destroy it.
840
841	For more information see [Custom memory pools](@ref choosing_memory_type_custom_memory_pools).
842	*/
843	VK_DEFINE_HANDLE(VmaPool)
844
845	/** \struct VmaAllocation
846	\brief Represents single memory allocation.
847
848	It may be either dedicated block of `VkDeviceMemory` or a specific region of a bigger block of this type
849	plus unique offset.
850
851	There are multiple ways to create such object.
852	You need to fill structure VmaAllocationCreateInfo.
853	For more information see [Choosing memory type](@ref choosing_memory_type).
854
855	Although the library provides convenience functions that create Vulkan buffer or image,
856	allocate memory for it and bind them together,
857	binding of the allocation to a buffer or an image is out of scope of the allocation itself.
858	Allocation object can exist without buffer/image bound,
859	binding can be done manually by the user, and destruction of it can be done
860	independently of destruction of the allocation.
861
862	The object also remembers its size and some other information.
863	To retrieve this information, use function vmaGetAllocationInfo() and inspect
864	returned structure VmaAllocationInfo.
865	*/
866	VK_DEFINE_HANDLE(VmaAllocation)
867
868	/** \struct VmaDefragmentationContext
869	\brief An opaque object that represents started defragmentation process.
870
871	Fill structure #VmaDefragmentationInfo and call function vmaBeginDefragmentation() to create it.
872	Call function vmaEndDefragmentation() to destroy it.
873	*/
874	VK_DEFINE_HANDLE(VmaDefragmentationContext)
875
876	/** @} */
877
878	/**
879	\addtogroup group_virtual
880	@{
881	*/
882
883	/** \struct VmaVirtualAllocation
884	\brief Represents single memory allocation done inside VmaVirtualBlock.
885
886	Use it as a unique identifier to virtual allocation within the single block.
887
888	Use value `VK_NULL_HANDLE` to represent a null/invalid allocation.
889	*/
890	VK_DEFINE_NON_DISPATCHABLE_HANDLE(VmaVirtualAllocation);
891
892	/** @} */
893
894	/**
895	\addtogroup group_virtual
896	@{
897	*/
898
899	/** \struct VmaVirtualBlock
900	\brief Handle to a virtual block object that allows to use core allocation algorithm without allocating any real GPU memory.
901
902	Fill in #VmaVirtualBlockCreateInfo structure and use vmaCreateVirtualBlock() to create it. Use vmaDestroyVirtualBlock() to destroy it.
903	For more information, see documentation chapter \ref virtual_allocator.
904
905	This object is not thread-safe - should not be used from multiple threads simultaneously, must be synchronized externally.
906	*/
907	VK_DEFINE_HANDLE(VmaVirtualBlock)
908
909	/** @} */
910
911	/**
912	\addtogroup group_init
913	@{
914	*/
915
916	/// Callback function called after successful vkAllocateMemory.
917	typedef void (VKAPI_PTR* PFN_vmaAllocateDeviceMemoryFunction)(
918	VmaAllocator VMA_NOT_NULL allocator,
919	uint32_t memoryType,
920	VkDeviceMemory VMA_NOT_NULL_NON_DISPATCHABLE memory,
921	VkDeviceSize size,
922	void* VMA_NULLABLE pUserData);
923
924	/// Callback function called before vkFreeMemory.
925	typedef void (VKAPI_PTR* PFN_vmaFreeDeviceMemoryFunction)(
926	VmaAllocator VMA_NOT_NULL allocator,
927	uint32_t memoryType,
928	VkDeviceMemory VMA_NOT_NULL_NON_DISPATCHABLE memory,
929	VkDeviceSize size,
930	void* VMA_NULLABLE pUserData);
931
932	/** \brief Set of callbacks that the library will call for `vkAllocateMemory` and `vkFreeMemory`.
933
934	Provided for informative purpose, e.g. to gather statistics about number of
935	allocations or total amount of memory allocated in Vulkan.
936
937	Used in VmaAllocatorCreateInfo::pDeviceMemoryCallbacks.
938	*/
939	typedef struct VmaDeviceMemoryCallbacks
940	{
941	/// Optional, can be null.
942	PFN_vmaAllocateDeviceMemoryFunction VMA_NULLABLE pfnAllocate;
943	/// Optional, can be null.
944	PFN_vmaFreeDeviceMemoryFunction VMA_NULLABLE pfnFree;
945	/// Optional, can be null.
946	void* VMA_NULLABLE pUserData;
947	} VmaDeviceMemoryCallbacks;
948
949	/** \brief Pointers to some Vulkan functions - a subset used by the library.
950
951	Used in VmaAllocatorCreateInfo::pVulkanFunctions.
952	*/
953	typedef struct VmaVulkanFunctions
954	{
955	/// Required when using VMA_DYNAMIC_VULKAN_FUNCTIONS.
956	PFN_vkGetInstanceProcAddr VMA_NULLABLE vkGetInstanceProcAddr;
957	/// Required when using VMA_DYNAMIC_VULKAN_FUNCTIONS.
958	PFN_vkGetDeviceProcAddr VMA_NULLABLE vkGetDeviceProcAddr;
959	PFN_vkGetPhysicalDeviceProperties VMA_NULLABLE vkGetPhysicalDeviceProperties;
960	PFN_vkGetPhysicalDeviceMemoryProperties VMA_NULLABLE vkGetPhysicalDeviceMemoryProperties;
961	PFN_vkAllocateMemory VMA_NULLABLE vkAllocateMemory;
962	PFN_vkFreeMemory VMA_NULLABLE vkFreeMemory;
963	PFN_vkMapMemory VMA_NULLABLE vkMapMemory;
964	PFN_vkUnmapMemory VMA_NULLABLE vkUnmapMemory;
965	PFN_vkFlushMappedMemoryRanges VMA_NULLABLE vkFlushMappedMemoryRanges;
966	PFN_vkInvalidateMappedMemoryRanges VMA_NULLABLE vkInvalidateMappedMemoryRanges;
967	PFN_vkBindBufferMemory VMA_NULLABLE vkBindBufferMemory;
968	PFN_vkBindImageMemory VMA_NULLABLE vkBindImageMemory;
969	PFN_vkGetBufferMemoryRequirements VMA_NULLABLE vkGetBufferMemoryRequirements;
970	PFN_vkGetImageMemoryRequirements VMA_NULLABLE vkGetImageMemoryRequirements;
971	PFN_vkCreateBuffer VMA_NULLABLE vkCreateBuffer;
972	PFN_vkDestroyBuffer VMA_NULLABLE vkDestroyBuffer;
973	PFN_vkCreateImage VMA_NULLABLE vkCreateImage;
974	PFN_vkDestroyImage VMA_NULLABLE vkDestroyImage;
975	PFN_vkCmdCopyBuffer VMA_NULLABLE vkCmdCopyBuffer;
976	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
977	/// Fetch "vkGetBufferMemoryRequirements2" on Vulkan >= 1.1, fetch "vkGetBufferMemoryRequirements2KHR" when using VK_KHR_dedicated_allocation extension.
978	PFN_vkGetBufferMemoryRequirements2KHR VMA_NULLABLE vkGetBufferMemoryRequirements2KHR;
979	/// Fetch "vkGetImageMemoryRequirements2" on Vulkan >= 1.1, fetch "vkGetImageMemoryRequirements2KHR" when using VK_KHR_dedicated_allocation extension.
980	PFN_vkGetImageMemoryRequirements2KHR VMA_NULLABLE vkGetImageMemoryRequirements2KHR;
981	#endif
982	#if VMA_BIND_MEMORY2 || VMA_VULKAN_VERSION >= 1001000
983	/// Fetch "vkBindBufferMemory2" on Vulkan >= 1.1, fetch "vkBindBufferMemory2KHR" when using VK_KHR_bind_memory2 extension.
984	PFN_vkBindBufferMemory2KHR VMA_NULLABLE vkBindBufferMemory2KHR;
985	/// Fetch "vkBindImageMemory2" on Vulkan >= 1.1, fetch "vkBindImageMemory2KHR" when using VK_KHR_bind_memory2 extension.
986	PFN_vkBindImageMemory2KHR VMA_NULLABLE vkBindImageMemory2KHR;
987	#endif
988	#if VMA_MEMORY_BUDGET || VMA_VULKAN_VERSION >= 1001000
989	PFN_vkGetPhysicalDeviceMemoryProperties2KHR VMA_NULLABLE vkGetPhysicalDeviceMemoryProperties2KHR;
990	#endif
991	#if VMA_VULKAN_VERSION >= 1003000
992	/// Fetch from "vkGetDeviceBufferMemoryRequirements" on Vulkan >= 1.3, but you can also fetch it from "vkGetDeviceBufferMemoryRequirementsKHR" if you enabled extension VK_KHR_maintenance4.
993	PFN_vkGetDeviceBufferMemoryRequirements VMA_NULLABLE vkGetDeviceBufferMemoryRequirements;
994	/// Fetch from "vkGetDeviceImageMemoryRequirements" on Vulkan >= 1.3, but you can also fetch it from "vkGetDeviceImageMemoryRequirementsKHR" if you enabled extension VK_KHR_maintenance4.
995	PFN_vkGetDeviceImageMemoryRequirements VMA_NULLABLE vkGetDeviceImageMemoryRequirements;
996	#endif
997	} VmaVulkanFunctions;
998
999	/// Description of a Allocator to be created.
1000	typedef struct VmaAllocatorCreateInfo
1001	{
1002	/// Flags for created allocator. Use #VmaAllocatorCreateFlagBits enum.
1003	VmaAllocatorCreateFlags flags;
1004	/// Vulkan physical device.
1005	/** It must be valid throughout whole lifetime of created allocator. */
1006	VkPhysicalDevice VMA_NOT_NULL physicalDevice;
1007	/// Vulkan device.
1008	/** It must be valid throughout whole lifetime of created allocator. */
1009	VkDevice VMA_NOT_NULL device;
1010	/// Preferred size of a single `VkDeviceMemory` block to be allocated from large heaps > 1 GiB. Optional.
1011	/** Set to 0 to use default, which is currently 256 MiB. */
1012	VkDeviceSize preferredLargeHeapBlockSize;
1013	/// Custom CPU memory allocation callbacks. Optional.
1014	/** Optional, can be null. When specified, will also be used for all CPU-side memory allocations. */
1015	const VkAllocationCallbacks* VMA_NULLABLE pAllocationCallbacks;
1016	/// Informative callbacks for `vkAllocateMemory`, `vkFreeMemory`. Optional.
1017	/** Optional, can be null. */
1018	const VmaDeviceMemoryCallbacks* VMA_NULLABLE pDeviceMemoryCallbacks;
1019	/** \brief Either null or a pointer to an array of limits on maximum number of bytes that can be allocated out of particular Vulkan memory heap.
1020
1021	If not NULL, it must be a pointer to an array of
1022	`VkPhysicalDeviceMemoryProperties::memoryHeapCount` elements, defining limit on
1023	maximum number of bytes that can be allocated out of particular Vulkan memory
1024	heap.
1025
1026	Any of the elements may be equal to `VK_WHOLE_SIZE`, which means no limit on that
1027	heap. This is also the default in case of `pHeapSizeLimit` = NULL.
1028
1029	If there is a limit defined for a heap:
1030
1031	- If user tries to allocate more memory from that heap using this allocator,
1032	the allocation fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
1033	- If the limit is smaller than heap size reported in `VkMemoryHeap::size`, the
1034	value of this limit will be reported instead when using vmaGetMemoryProperties().
1035
1036	Warning! Using this feature may not be equivalent to installing a GPU with
1037	smaller amount of memory, because graphics driver doesn't necessary fail new
1038	allocations with `VK_ERROR_OUT_OF_DEVICE_MEMORY` result when memory capacity is
1039	exceeded. It may return success and just silently migrate some device memory
1040	blocks to system RAM. This driver behavior can also be controlled using
1041	VK_AMD_memory_overallocation_behavior extension.
1042	*/
1043	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount") pHeapSizeLimit;
1044
1045	/** \brief Pointers to Vulkan functions. Can be null.
1046
1047	For details see [Pointers to Vulkan functions](@ref config_Vulkan_functions).
1048	*/
1049	const VmaVulkanFunctions* VMA_NULLABLE pVulkanFunctions;
1050	/** \brief Handle to Vulkan instance object.
1051
1052	Starting from version 3.0.0 this member is no longer optional, it must be set!
1053	*/
1054	VkInstance VMA_NOT_NULL instance;
1055	/** \brief Optional. The highest version of Vulkan that the application is designed to use.
1056
1057	It must be a value in the format as created by macro `VK_MAKE_VERSION` or a constant like: `VK_API_VERSION_1_1`, `VK_API_VERSION_1_0`.
1058	The patch version number specified is ignored. Only the major and minor versions are considered.
1059	It must be less or equal (preferably equal) to value as passed to `vkCreateInstance` as `VkApplicationInfo::apiVersion`.
1060	Only versions 1.0, 1.1, 1.2, 1.3 are supported by the current implementation.
1061	Leaving it initialized to zero is equivalent to `VK_API_VERSION_1_0`.
1062	*/
1063	uint32_t vulkanApiVersion;
1064	#if VMA_EXTERNAL_MEMORY
1065	/** \brief Either null or a pointer to an array of external memory handle types for each Vulkan memory type.
1066
1067	If not NULL, it must be a pointer to an array of `VkPhysicalDeviceMemoryProperties::memoryTypeCount`
1068	elements, defining external memory handle types of particular Vulkan memory type,
1069	to be passed using `VkExportMemoryAllocateInfoKHR`.
1070
1071	Any of the elements may be equal to 0, which means not to use `VkExportMemoryAllocateInfoKHR` on this memory type.
1072	This is also the default in case of `pTypeExternalMemoryHandleTypes` = NULL.
1073	*/
1074	const VkExternalMemoryHandleTypeFlagsKHR* VMA_NULLABLE VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryTypeCount") pTypeExternalMemoryHandleTypes;
1075	#endif // #if VMA_EXTERNAL_MEMORY
1076	} VmaAllocatorCreateInfo;
1077
1078	/// Information about existing #VmaAllocator object.
1079	typedef struct VmaAllocatorInfo
1080	{
1081	/** \brief Handle to Vulkan instance object.
1082
1083	This is the same value as has been passed through VmaAllocatorCreateInfo::instance.
1084	*/
1085	VkInstance VMA_NOT_NULL instance;
1086	/** \brief Handle to Vulkan physical device object.
1087
1088	This is the same value as has been passed through VmaAllocatorCreateInfo::physicalDevice.
1089	*/
1090	VkPhysicalDevice VMA_NOT_NULL physicalDevice;
1091	/** \brief Handle to Vulkan device object.
1092
1093	This is the same value as has been passed through VmaAllocatorCreateInfo::device.
1094	*/
1095	VkDevice VMA_NOT_NULL device;
1096	} VmaAllocatorInfo;
1097
1098	/** @} */
1099
1100	/**
1101	\addtogroup group_stats
1102	@{
1103	*/
1104
1105	/** \brief Calculated statistics of memory usage e.g. in a specific memory type, heap, custom pool, or total.
1106
1107	These are fast to calculate.
1108	See functions: vmaGetHeapBudgets(), vmaGetPoolStatistics().
1109	*/
1110	typedef struct VmaStatistics
1111	{
1112	/** \brief Number of `VkDeviceMemory` objects - Vulkan memory blocks allocated.
1113	*/
1114	uint32_t blockCount;
1115	/** \brief Number of #VmaAllocation objects allocated.
1116
1117	Dedicated allocations have their own blocks, so each one adds 1 to `allocationCount` as well as `blockCount`.
1118	*/
1119	uint32_t allocationCount;
1120	/** \brief Number of bytes allocated in `VkDeviceMemory` blocks.
1121
1122	\note To avoid confusion, please be aware that what Vulkan calls an "allocation" - a whole `VkDeviceMemory` object
1123	(e.g. as in `VkPhysicalDeviceLimits::maxMemoryAllocationCount`) is called a "block" in VMA, while VMA calls
1124	"allocation" a #VmaAllocation object that represents a memory region sub-allocated from such block, usually for a single buffer or image.
1125	*/
1126	VkDeviceSize blockBytes;
1127	/** \brief Total number of bytes occupied by all #VmaAllocation objects.
1128
1129	Always less or equal than `blockBytes`.
1130	Difference `(blockBytes - allocationBytes)` is the amount of memory allocated from Vulkan
1131	but unused by any #VmaAllocation.
1132	*/
1133	VkDeviceSize allocationBytes;
1134	} VmaStatistics;
1135
1136	/** \brief More detailed statistics than #VmaStatistics.
1137
1138	These are slower to calculate. Use for debugging purposes.
1139	See functions: vmaCalculateStatistics(), vmaCalculatePoolStatistics().
1140
1141	Previous version of the statistics API provided averages, but they have been removed
1142	because they can be easily calculated as:
1143
1144	\code
1145	VkDeviceSize allocationSizeAvg = detailedStats.statistics.allocationBytes / detailedStats.statistics.allocationCount;
1146	VkDeviceSize unusedBytes = detailedStats.statistics.blockBytes - detailedStats.statistics.allocationBytes;
1147	VkDeviceSize unusedRangeSizeAvg = unusedBytes / detailedStats.unusedRangeCount;
1148	\endcode
1149	*/
1150	typedef struct VmaDetailedStatistics
1151	{
1152	/// Basic statistics.
1153	VmaStatistics statistics;
1154	/// Number of free ranges of memory between allocations.
1155	uint32_t unusedRangeCount;
1156	/// Smallest allocation size. `VK_WHOLE_SIZE` if there are 0 allocations.
1157	VkDeviceSize allocationSizeMin;
1158	/// Largest allocation size. 0 if there are 0 allocations.
1159	VkDeviceSize allocationSizeMax;
1160	/// Smallest empty range size. `VK_WHOLE_SIZE` if there are 0 empty ranges.
1161	VkDeviceSize unusedRangeSizeMin;
1162	/// Largest empty range size. 0 if there are 0 empty ranges.
1163	VkDeviceSize unusedRangeSizeMax;
1164	} VmaDetailedStatistics;
1165
1166	/** \brief General statistics from current state of the Allocator -
1167	total memory usage across all memory heaps and types.
1168
1169	These are slower to calculate. Use for debugging purposes.
1170	See function vmaCalculateStatistics().
1171	*/
1172	typedef struct VmaTotalStatistics
1173	{
1174	VmaDetailedStatistics memoryType[VK_MAX_MEMORY_TYPES];
1175	VmaDetailedStatistics memoryHeap[VK_MAX_MEMORY_HEAPS];
1176	VmaDetailedStatistics total;
1177	} VmaTotalStatistics;
1178
1179	/** \brief Statistics of current memory usage and available budget for a specific memory heap.
1180
1181	These are fast to calculate.
1182	See function vmaGetHeapBudgets().
1183	*/
1184	typedef struct VmaBudget
1185	{
1186	/** \brief Statistics fetched from the library.
1187	*/
1188	VmaStatistics statistics;
1189	/** \brief Estimated current memory usage of the program, in bytes.
1190
1191	Fetched from system using VK_EXT_memory_budget extension if enabled.
1192
1193	It might be different than `statistics.blockBytes` (usually higher) due to additional implicit objects
1194	also occupying the memory, like swapchain, pipelines, descriptor heaps, command buffers, or
1195	`VkDeviceMemory` blocks allocated outside of this library, if any.
1196	*/
1197	VkDeviceSize usage;
1198	/** \brief Estimated amount of memory available to the program, in bytes.
1199
1200	Fetched from system using VK_EXT_memory_budget extension if enabled.
1201
1202	It might be different (most probably smaller) than `VkMemoryHeap::size[heapIndex]` due to factors
1203	external to the program, decided by the operating system.
1204	Difference `budget - usage` is the amount of additional memory that can probably
1205	be allocated without problems. Exceeding the budget may result in various problems.
1206	*/
1207	VkDeviceSize budget;
1208	} VmaBudget;
1209
1210	/** @} */
1211
1212	/**
1213	\addtogroup group_alloc
1214	@{
1215	*/
1216
1217	/** \brief Parameters of new #VmaAllocation.
1218
1219	To be used with functions like vmaCreateBuffer(), vmaCreateImage(), and many others.
1220	*/
1221	typedef struct VmaAllocationCreateInfo
1222	{
1223	/// Use #VmaAllocationCreateFlagBits enum.
1224	VmaAllocationCreateFlags flags;
1225	/** \brief Intended usage of memory.
1226
1227	You can leave #VMA_MEMORY_USAGE_UNKNOWN if you specify memory requirements in other way. \n
1228	If `pool` is not null, this member is ignored.
1229	*/
1230	VmaMemoryUsage usage;
1231	/** \brief Flags that must be set in a Memory Type chosen for an allocation.
1232
1233	Leave 0 if you specify memory requirements in other way. \n
1234	If `pool` is not null, this member is ignored.*/
1235	VkMemoryPropertyFlags requiredFlags;
1236	/** \brief Flags that preferably should be set in a memory type chosen for an allocation.
1237
1238	Set to 0 if no additional flags are preferred. \n
1239	If `pool` is not null, this member is ignored. */
1240	VkMemoryPropertyFlags preferredFlags;
1241	/** \brief Bitmask containing one bit set for every memory type acceptable for this allocation.
1242
1243	Value 0 is equivalent to `UINT32_MAX` - it means any memory type is accepted if
1244	it meets other requirements specified by this structure, with no further
1245	restrictions on memory type index. \n
1246	If `pool` is not null, this member is ignored.
1247	*/
1248	uint32_t memoryTypeBits;
1249	/** \brief Pool that this allocation should be created in.
1250
1251	Leave `VK_NULL_HANDLE` to allocate from default pool. If not null, members:
1252	`usage`, `requiredFlags`, `preferredFlags`, `memoryTypeBits` are ignored.
1253	*/
1254	VmaPool VMA_NULLABLE pool;
1255	/** \brief Custom general-purpose pointer that will be stored in #VmaAllocation, can be read as VmaAllocationInfo::pUserData and changed using vmaSetAllocationUserData().
1256
1257	If #VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT is used, it must be either
1258	null or pointer to a null-terminated string. The string will be then copied to
1259	internal buffer, so it doesn't need to be valid after allocation call.
1260	*/
1261	void* VMA_NULLABLE pUserData;
1262	/** \brief A floating-point value between 0 and 1, indicating the priority of the allocation relative to other memory allocations.
1263
1264	It is used only when #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT flag was used during creation of the #VmaAllocator object
1265	and this allocation ends up as dedicated or is explicitly forced as dedicated using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
1266	Otherwise, it has the priority of a memory block where it is placed and this variable is ignored.
1267	*/
1268	float priority;
1269	} VmaAllocationCreateInfo;
1270
1271	/// Describes parameter of created #VmaPool.
1272	typedef struct VmaPoolCreateInfo
1273	{
1274	/** \brief Vulkan memory type index to allocate this pool from.
1275	*/
1276	uint32_t memoryTypeIndex;
1277	/** \brief Use combination of #VmaPoolCreateFlagBits.
1278	*/
1279	VmaPoolCreateFlags flags;
1280	/** \brief Size of a single `VkDeviceMemory` block to be allocated as part of this pool, in bytes. Optional.
1281
1282	Specify nonzero to set explicit, constant size of memory blocks used by this
1283	pool.
1284
1285	Leave 0 to use default and let the library manage block sizes automatically.
1286	Sizes of particular blocks may vary.
1287	In this case, the pool will also support dedicated allocations.
1288	*/
1289	VkDeviceSize blockSize;
1290	/** \brief Minimum number of blocks to be always allocated in this pool, even if they stay empty.
1291
1292	Set to 0 to have no preallocated blocks and allow the pool be completely empty.
1293	*/
1294	size_t minBlockCount;
1295	/** \brief Maximum number of blocks that can be allocated in this pool. Optional.
1296
1297	Set to 0 to use default, which is `SIZE_MAX`, which means no limit.
1298
1299	Set to same value as VmaPoolCreateInfo::minBlockCount to have fixed amount of memory allocated
1300	throughout whole lifetime of this pool.
1301	*/
1302	size_t maxBlockCount;
1303	/** \brief A floating-point value between 0 and 1, indicating the priority of the allocations in this pool relative to other memory allocations.
1304
1305	It is used only when #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT flag was used during creation of the #VmaAllocator object.
1306	Otherwise, this variable is ignored.
1307	*/
1308	float priority;
1309	/** \brief Additional minimum alignment to be used for all allocations created from this pool. Can be 0.
1310
1311	Leave 0 (default) not to impose any additional alignment. If not 0, it must be a power of two.
1312	It can be useful in cases where alignment returned by Vulkan by functions like `vkGetBufferMemoryRequirements` is not enough,
1313	e.g. when doing interop with OpenGL.
1314	*/
1315	VkDeviceSize minAllocationAlignment;
1316	/** \brief Additional `pNext` chain to be attached to `VkMemoryAllocateInfo` used for every allocation made by this pool. Optional.
1317
1318	Optional, can be null. If not null, it must point to a `pNext` chain of structures that can be attached to `VkMemoryAllocateInfo`.
1319	It can be useful for special needs such as adding `VkExportMemoryAllocateInfoKHR`.
1320	Structures pointed by this member must remain alive and unchanged for the whole lifetime of the custom pool.
1321
1322	Please note that some structures, e.g. `VkMemoryPriorityAllocateInfoEXT`, `VkMemoryDedicatedAllocateInfoKHR`,
1323	can be attached automatically by this library when using other, more convenient of its features.
1324	*/
1325	void* VMA_NULLABLE pMemoryAllocateNext;
1326	} VmaPoolCreateInfo;
1327
1328	/** @} */
1329
1330	/**
1331	\addtogroup group_alloc
1332	@{
1333	*/
1334
1335	/// Parameters of #VmaAllocation objects, that can be retrieved using function vmaGetAllocationInfo().
1336	typedef struct VmaAllocationInfo
1337	{
1338	/** \brief Memory type index that this allocation was allocated from.
1339
1340	It never changes.
1341	*/
1342	uint32_t memoryType;
1343	/** \brief Handle to Vulkan memory object.
1344
1345	Same memory object can be shared by multiple allocations.
1346
1347	It can change after the allocation is moved during \ref defragmentation.
1348	*/
1349	VkDeviceMemory VMA_NULLABLE_NON_DISPATCHABLE deviceMemory;
1350	/** \brief Offset in `VkDeviceMemory` object to the beginning of this allocation, in bytes. `(deviceMemory, offset)` pair is unique to this allocation.
1351
1352	You usually don't need to use this offset. If you create a buffer or an image together with the allocation using e.g. function
1353	vmaCreateBuffer(), vmaCreateImage(), functions that operate on these resources refer to the beginning of the buffer or image,
1354	not entire device memory block. Functions like vmaMapMemory(), vmaBindBufferMemory() also refer to the beginning of the allocation
1355	and apply this offset automatically.
1356
1357	It can change after the allocation is moved during \ref defragmentation.
1358	*/
1359	VkDeviceSize offset;
1360	/** \brief Size of this allocation, in bytes.
1361
1362	It never changes.
1363
1364	\note Allocation size returned in this variable may be greater than the size
1365	requested for the resource e.g. as `VkBufferCreateInfo::size`. Whole size of the
1366	allocation is accessible for operations on memory e.g. using a pointer after
1367	mapping with vmaMapMemory(), but operations on the resource e.g. using
1368	`vkCmdCopyBuffer` must be limited to the size of the resource.
1369	*/
1370	VkDeviceSize size;
1371	/** \brief Pointer to the beginning of this allocation as mapped data.
1372
1373	If the allocation hasn't been mapped using vmaMapMemory() and hasn't been
1374	created with #VMA_ALLOCATION_CREATE_MAPPED_BIT flag, this value is null.
1375
1376	It can change after call to vmaMapMemory(), vmaUnmapMemory().
1377	It can also change after the allocation is moved during \ref defragmentation.
1378	*/
1379	void* VMA_NULLABLE pMappedData;
1380	/** \brief Custom general-purpose pointer that was passed as VmaAllocationCreateInfo::pUserData or set using vmaSetAllocationUserData().
1381
1382	It can change after call to vmaSetAllocationUserData() for this allocation.
1383	*/
1384	void* VMA_NULLABLE pUserData;
1385	/** \brief Custom allocation name that was set with vmaSetAllocationName().
1386
1387	It can change after call to vmaSetAllocationName() for this allocation.
1388
1389	Another way to set custom name is to pass it in VmaAllocationCreateInfo::pUserData with
1390	additional flag #VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT set [DEPRECATED].
1391	*/
1392	const char* VMA_NULLABLE pName;
1393	} VmaAllocationInfo;
1394
1395	/** \brief Parameters for defragmentation.
1396
1397	To be used with function vmaBeginDefragmentation().
1398	*/
1399	typedef struct VmaDefragmentationInfo
1400	{
1401	/// \brief Use combination of #VmaDefragmentationFlagBits.
1402	VmaDefragmentationFlags flags;
1403	/** \brief Custom pool to be defragmented.
1404
1405	If null then default pools will undergo defragmentation process.
1406	*/
1407	VmaPool VMA_NULLABLE pool;
1408	/** \brief Maximum numbers of bytes that can be copied during single pass, while moving allocations to different places.
1409
1410	`0` means no limit.
1411	*/
1412	VkDeviceSize maxBytesPerPass;
1413	/** \brief Maximum number of allocations that can be moved during single pass to a different place.
1414
1415	`0` means no limit.
1416	*/
1417	uint32_t maxAllocationsPerPass;
1418	} VmaDefragmentationInfo;
1419
1420	/// Single move of an allocation to be done for defragmentation.
1421	typedef struct VmaDefragmentationMove
1422	{
1423	/// Operation to be performed on the allocation by vmaEndDefragmentationPass(). Default value is #VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY. You can modify it.
1424	VmaDefragmentationMoveOperation operation;
1425	/// Allocation that should be moved.
1426	VmaAllocation VMA_NOT_NULL srcAllocation;
1427	/** \brief Temporary allocation pointing to destination memory that will replace `srcAllocation`.
1428
1429	\warning Do not store this allocation in your data structures! It exists only temporarily, for the duration of the defragmentation pass,
1430	to be used for binding new buffer/image to the destination memory using e.g. vmaBindBufferMemory().
1431	vmaEndDefragmentationPass() will destroy it and make `srcAllocation` point to this memory.
1432	*/
1433	VmaAllocation VMA_NOT_NULL dstTmpAllocation;
1434	} VmaDefragmentationMove;
1435
1436	/** \brief Parameters for incremental defragmentation steps.
1437
1438	To be used with function vmaBeginDefragmentationPass().
1439	*/
1440	typedef struct VmaDefragmentationPassMoveInfo
1441	{
1442	/// Number of elements in the `pMoves` array.
1443	uint32_t moveCount;
1444	/** \brief Array of moves to be performed by the user in the current defragmentation pass.
1445
1446	Pointer to an array of `moveCount` elements, owned by VMA, created in vmaBeginDefragmentationPass(), destroyed in vmaEndDefragmentationPass().
1447
1448	For each element, you should:
1449
1450	1. Create a new buffer/image in the place pointed by VmaDefragmentationMove::dstMemory + VmaDefragmentationMove::dstOffset.
1451	2. Copy data from the VmaDefragmentationMove::srcAllocation e.g. using `vkCmdCopyBuffer`, `vkCmdCopyImage`.
1452	3. Make sure these commands finished executing on the GPU.
1453	4. Destroy the old buffer/image.
1454
1455	Only then you can finish defragmentation pass by calling vmaEndDefragmentationPass().
1456	After this call, the allocation will point to the new place in memory.
1457
1458	Alternatively, if you cannot move specific allocation, you can set VmaDefragmentationMove::operation to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
1459
1460	Alternatively, if you decide you want to completely remove the allocation:
1461
1462	1. Destroy its buffer/image.
1463	2. Set VmaDefragmentationMove::operation to #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY.
1464
1465	Then, after vmaEndDefragmentationPass() the allocation will be freed.
1466	*/
1467	VmaDefragmentationMove* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(moveCount) pMoves;
1468	} VmaDefragmentationPassMoveInfo;
1469
1470	/// Statistics returned for defragmentation process in function vmaEndDefragmentation().
1471	typedef struct VmaDefragmentationStats
1472	{
1473	/// Total number of bytes that have been copied while moving allocations to different places.
1474	VkDeviceSize bytesMoved;
1475	/// Total number of bytes that have been released to the system by freeing empty `VkDeviceMemory` objects.
1476	VkDeviceSize bytesFreed;
1477	/// Number of allocations that have been moved to different places.
1478	uint32_t allocationsMoved;
1479	/// Number of empty `VkDeviceMemory` objects that have been released to the system.
1480	uint32_t deviceMemoryBlocksFreed;
1481	} VmaDefragmentationStats;
1482
1483	/** @} */
1484
1485	/**
1486	\addtogroup group_virtual
1487	@{
1488	*/
1489
1490	/// Parameters of created #VmaVirtualBlock object to be passed to vmaCreateVirtualBlock().
1491	typedef struct VmaVirtualBlockCreateInfo
1492	{
1493	/** \brief Total size of the virtual block.
1494
1495	Sizes can be expressed in bytes or any units you want as long as you are consistent in using them.
1496	For example, if you allocate from some array of structures, 1 can mean single instance of entire structure.
1497	*/
1498	VkDeviceSize size;
1499
1500	/** \brief Use combination of #VmaVirtualBlockCreateFlagBits.
1501	*/
1502	VmaVirtualBlockCreateFlags flags;
1503
1504	/** \brief Custom CPU memory allocation callbacks. Optional.
1505
1506	Optional, can be null. When specified, they will be used for all CPU-side memory allocations.
1507	*/
1508	const VkAllocationCallbacks* VMA_NULLABLE pAllocationCallbacks;
1509	} VmaVirtualBlockCreateInfo;
1510
1511	/// Parameters of created virtual allocation to be passed to vmaVirtualAllocate().
1512	typedef struct VmaVirtualAllocationCreateInfo
1513	{
1514	/** \brief Size of the allocation.
1515
1516	Cannot be zero.
1517	*/
1518	VkDeviceSize size;
1519	/** \brief Required alignment of the allocation. Optional.
1520
1521	Must be power of two. Special value 0 has the same meaning as 1 - means no special alignment is required, so allocation can start at any offset.
1522	*/
1523	VkDeviceSize alignment;
1524	/** \brief Use combination of #VmaVirtualAllocationCreateFlagBits.
1525	*/
1526	VmaVirtualAllocationCreateFlags flags;
1527	/** \brief Custom pointer to be associated with the allocation. Optional.
1528
1529	It can be any value and can be used for user-defined purposes. It can be fetched or changed later.
1530	*/
1531	void* VMA_NULLABLE pUserData;
1532	} VmaVirtualAllocationCreateInfo;
1533
1534	/// Parameters of an existing virtual allocation, returned by vmaGetVirtualAllocationInfo().
1535	typedef struct VmaVirtualAllocationInfo
1536	{
1537	/** \brief Offset of the allocation.
1538
1539	Offset at which the allocation was made.
1540	*/
1541	VkDeviceSize offset;
1542	/** \brief Size of the allocation.
1543
1544	Same value as passed in VmaVirtualAllocationCreateInfo::size.
1545	*/
1546	VkDeviceSize size;
1547	/** \brief Custom pointer associated with the allocation.
1548
1549	Same value as passed in VmaVirtualAllocationCreateInfo::pUserData or to vmaSetVirtualAllocationUserData().
1550	*/
1551	void* VMA_NULLABLE pUserData;
1552	} VmaVirtualAllocationInfo;
1553
1554	/** @} */
1555
1556	#endif // _VMA_DATA_TYPES_DECLARATIONS
1557
1558	#ifndef _VMA_FUNCTION_HEADERS
1559
1560	/**
1561	\addtogroup group_init
1562	@{
1563	*/
1564
1565	/// Creates #VmaAllocator object.
1566	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAllocator(
1567	const VmaAllocatorCreateInfo* VMA_NOT_NULL pCreateInfo,
1568	VmaAllocator VMA_NULLABLE* VMA_NOT_NULL pAllocator);
1569
1570	/// Destroys allocator object.
1571	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyAllocator(
1572	VmaAllocator VMA_NULLABLE allocator);
1573
1574	/** \brief Returns information about existing #VmaAllocator object - handle to Vulkan device etc.
1575
1576	It might be useful if you want to keep just the #VmaAllocator handle and fetch other required handles to
1577	`VkPhysicalDevice`, `VkDevice` etc. every time using this function.
1578	*/
1579	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocatorInfo(
1580	VmaAllocator VMA_NOT_NULL allocator,
1581	VmaAllocatorInfo* VMA_NOT_NULL pAllocatorInfo);
1582
1583	/**
1584	PhysicalDeviceProperties are fetched from physicalDevice by the allocator.
1585	You can access it here, without fetching it again on your own.
1586	*/
1587	VMA_CALL_PRE void VMA_CALL_POST vmaGetPhysicalDeviceProperties(
1588	VmaAllocator VMA_NOT_NULL allocator,
1589	const VkPhysicalDeviceProperties* VMA_NULLABLE* VMA_NOT_NULL ppPhysicalDeviceProperties);
1590
1591	/**
1592	PhysicalDeviceMemoryProperties are fetched from physicalDevice by the allocator.
1593	You can access it here, without fetching it again on your own.
1594	*/
1595	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryProperties(
1596	VmaAllocator VMA_NOT_NULL allocator,
1597	const VkPhysicalDeviceMemoryProperties* VMA_NULLABLE* VMA_NOT_NULL ppPhysicalDeviceMemoryProperties);
1598
1599	/**
1600	\brief Given Memory Type Index, returns Property Flags of this memory type.
1601
1602	This is just a convenience function. Same information can be obtained using
1603	vmaGetMemoryProperties().
1604	*/
1605	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryTypeProperties(
1606	VmaAllocator VMA_NOT_NULL allocator,
1607	uint32_t memoryTypeIndex,
1608	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags);
1609
1610	/** \brief Sets index of the current frame.
1611	*/
1612	VMA_CALL_PRE void VMA_CALL_POST vmaSetCurrentFrameIndex(
1613	VmaAllocator VMA_NOT_NULL allocator,
1614	uint32_t frameIndex);
1615
1616	/** @} */
1617
1618	/**
1619	\addtogroup group_stats
1620	@{
1621	*/
1622
1623	/** \brief Retrieves statistics from current state of the Allocator.
1624
1625	This function is called "calculate" not "get" because it has to traverse all
1626	internal data structures, so it may be quite slow. Use it for debugging purposes.
1627	For faster but more brief statistics suitable to be called every frame or every allocation,
1628	use vmaGetHeapBudgets().
1629
1630	Note that when using allocator from multiple threads, returned information may immediately
1631	become outdated.
1632	*/
1633	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateStatistics(
1634	VmaAllocator VMA_NOT_NULL allocator,
1635	VmaTotalStatistics* VMA_NOT_NULL pStats);
1636
1637	/** \brief Retrieves information about current memory usage and budget for all memory heaps.
1638
1639	\param allocator
1640	\param[out] pBudgets Must point to array with number of elements at least equal to number of memory heaps in physical device used.
1641
1642	This function is called "get" not "calculate" because it is very fast, suitable to be called
1643	every frame or every allocation. For more detailed statistics use vmaCalculateStatistics().
1644
1645	Note that when using allocator from multiple threads, returned information may immediately
1646	become outdated.
1647	*/
1648	VMA_CALL_PRE void VMA_CALL_POST vmaGetHeapBudgets(
1649	VmaAllocator VMA_NOT_NULL allocator,
1650	VmaBudget* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL("VkPhysicalDeviceMemoryProperties::memoryHeapCount") pBudgets);
1651
1652	/** @} */
1653
1654	/**
1655	\addtogroup group_alloc
1656	@{
1657	*/
1658
1659	/**
1660	\brief Helps to find memoryTypeIndex, given memoryTypeBits and VmaAllocationCreateInfo.
1661
1662	This algorithm tries to find a memory type that:
1663
1664	- Is allowed by memoryTypeBits.
1665	- Contains all the flags from pAllocationCreateInfo->requiredFlags.
1666	- Matches intended usage.
1667	- Has as many flags from pAllocationCreateInfo->preferredFlags as possible.
1668
1669	\return Returns VK_ERROR_FEATURE_NOT_PRESENT if not found. Receiving such result
1670	from this function or any other allocating function probably means that your
1671	device doesn't support any memory type with requested features for the specific
1672	type of resource you want to use it for. Please check parameters of your
1673	resource, like image layout (OPTIMAL versus LINEAR) or mip level count.
1674	*/
1675	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndex(
1676	VmaAllocator VMA_NOT_NULL allocator,
1677	uint32_t memoryTypeBits,
1678	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1679	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1680
1681	/**
1682	\brief Helps to find memoryTypeIndex, given VkBufferCreateInfo and VmaAllocationCreateInfo.
1683
1684	It can be useful e.g. to determine value to be used as VmaPoolCreateInfo::memoryTypeIndex.
1685	It internally creates a temporary, dummy buffer that never has memory bound.
1686	*/
1687	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForBufferInfo(
1688	VmaAllocator VMA_NOT_NULL allocator,
1689	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
1690	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1691	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1692
1693	/**
1694	\brief Helps to find memoryTypeIndex, given VkImageCreateInfo and VmaAllocationCreateInfo.
1695
1696	It can be useful e.g. to determine value to be used as VmaPoolCreateInfo::memoryTypeIndex.
1697	It internally creates a temporary, dummy image that never has memory bound.
1698	*/
1699	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForImageInfo(
1700	VmaAllocator VMA_NOT_NULL allocator,
1701	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
1702	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
1703	uint32_t* VMA_NOT_NULL pMemoryTypeIndex);
1704
1705	/** \brief Allocates Vulkan device memory and creates #VmaPool object.
1706
1707	\param allocator Allocator object.
1708	\param pCreateInfo Parameters of pool to create.
1709	\param[out] pPool Handle to created pool.
1710	*/
1711	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreatePool(
1712	VmaAllocator VMA_NOT_NULL allocator,
1713	const VmaPoolCreateInfo* VMA_NOT_NULL pCreateInfo,
1714	VmaPool VMA_NULLABLE* VMA_NOT_NULL pPool);
1715
1716	/** \brief Destroys #VmaPool object and frees Vulkan device memory.
1717	*/
1718	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyPool(
1719	VmaAllocator VMA_NOT_NULL allocator,
1720	VmaPool VMA_NULLABLE pool);
1721
1722	/** @} */
1723
1724	/**
1725	\addtogroup group_stats
1726	@{
1727	*/
1728
1729	/** \brief Retrieves statistics of existing #VmaPool object.
1730
1731	\param allocator Allocator object.
1732	\param pool Pool object.
1733	\param[out] pPoolStats Statistics of specified pool.
1734	*/
1735	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolStatistics(
1736	VmaAllocator VMA_NOT_NULL allocator,
1737	VmaPool VMA_NOT_NULL pool,
1738	VmaStatistics* VMA_NOT_NULL pPoolStats);
1739
1740	/** \brief Retrieves detailed statistics of existing #VmaPool object.
1741
1742	\param allocator Allocator object.
1743	\param pool Pool object.
1744	\param[out] pPoolStats Statistics of specified pool.
1745	*/
1746	VMA_CALL_PRE void VMA_CALL_POST vmaCalculatePoolStatistics(
1747	VmaAllocator VMA_NOT_NULL allocator,
1748	VmaPool VMA_NOT_NULL pool,
1749	VmaDetailedStatistics* VMA_NOT_NULL pPoolStats);
1750
1751	/** @} */
1752
1753	/**
1754	\addtogroup group_alloc
1755	@{
1756	*/
1757
1758	/** \brief Checks magic number in margins around all allocations in given memory pool in search for corruptions.
1759
1760	Corruption detection is enabled only when `VMA_DEBUG_DETECT_CORRUPTION` macro is defined to nonzero,
1761	`VMA_DEBUG_MARGIN` is defined to nonzero and the pool is created in memory type that is
1762	`HOST_VISIBLE` and `HOST_COHERENT`. For more information, see [Corruption detection](@ref debugging_memory_usage_corruption_detection).
1763
1764	Possible return values:
1765
1766	- `VK_ERROR_FEATURE_NOT_PRESENT` - corruption detection is not enabled for specified pool.
1767	- `VK_SUCCESS` - corruption detection has been performed and succeeded.
1768	- `VK_ERROR_UNKNOWN` - corruption detection has been performed and found memory corruptions around one of the allocations.
1769	`VMA_ASSERT` is also fired in that case.
1770	- Other value: Error returned by Vulkan, e.g. memory mapping failure.
1771	*/
1772	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckPoolCorruption(
1773	VmaAllocator VMA_NOT_NULL allocator,
1774	VmaPool VMA_NOT_NULL pool);
1775
1776	/** \brief Retrieves name of a custom pool.
1777
1778	After the call `ppName` is either null or points to an internally-owned null-terminated string
1779	containing name of the pool that was previously set. The pointer becomes invalid when the pool is
1780	destroyed or its name is changed using vmaSetPoolName().
1781	*/
1782	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolName(
1783	VmaAllocator VMA_NOT_NULL allocator,
1784	VmaPool VMA_NOT_NULL pool,
1785	const char* VMA_NULLABLE* VMA_NOT_NULL ppName);
1786
1787	/** \brief Sets name of a custom pool.
1788
1789	`pName` can be either null or pointer to a null-terminated string with new name for the pool.
1790	Function makes internal copy of the string, so it can be changed or freed immediately after this call.
1791	*/
1792	VMA_CALL_PRE void VMA_CALL_POST vmaSetPoolName(
1793	VmaAllocator VMA_NOT_NULL allocator,
1794	VmaPool VMA_NOT_NULL pool,
1795	const char* VMA_NULLABLE pName);
1796
1797	/** \brief General purpose memory allocation.
1798
1799	\param allocator
1800	\param pVkMemoryRequirements
1801	\param pCreateInfo
1802	\param[out] pAllocation Handle to allocated memory.
1803	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1804
1805	You should free the memory using vmaFreeMemory() or vmaFreeMemoryPages().
1806
1807	It is recommended to use vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage(),
1808	vmaCreateBuffer(), vmaCreateImage() instead whenever possible.
1809	*/
1810	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemory(
1811	VmaAllocator VMA_NOT_NULL allocator,
1812	const VkMemoryRequirements* VMA_NOT_NULL pVkMemoryRequirements,
1813	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1814	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1815	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1816
1817	/** \brief General purpose memory allocation for multiple allocation objects at once.
1818
1819	\param allocator Allocator object.
1820	\param pVkMemoryRequirements Memory requirements for each allocation.
1821	\param pCreateInfo Creation parameters for each allocation.
1822	\param allocationCount Number of allocations to make.
1823	\param[out] pAllocations Pointer to array that will be filled with handles to created allocations.
1824	\param[out] pAllocationInfo Optional. Pointer to array that will be filled with parameters of created allocations.
1825
1826	You should free the memory using vmaFreeMemory() or vmaFreeMemoryPages().
1827
1828	Word "pages" is just a suggestion to use this function to allocate pieces of memory needed for sparse binding.
1829	It is just a general purpose allocation function able to make multiple allocations at once.
1830	It may be internally optimized to be more efficient than calling vmaAllocateMemory() `allocationCount` times.
1831
1832	All allocations are made using same parameters. All of them are created out of the same memory pool and type.
1833	If any allocation fails, all allocations already made within this function call are also freed, so that when
1834	returned result is not `VK_SUCCESS`, `pAllocation` array is always entirely filled with `VK_NULL_HANDLE`.
1835	*/
1836	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryPages(
1837	VmaAllocator VMA_NOT_NULL allocator,
1838	const VkMemoryRequirements* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pVkMemoryRequirements,
1839	const VmaAllocationCreateInfo* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pCreateInfo,
1840	size_t allocationCount,
1841	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pAllocations,
1842	VmaAllocationInfo* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) pAllocationInfo);
1843
1844	/** \brief Allocates memory suitable for given `VkBuffer`.
1845
1846	\param allocator
1847	\param buffer
1848	\param pCreateInfo
1849	\param[out] pAllocation Handle to allocated memory.
1850	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1851
1852	It only creates #VmaAllocation. To bind the memory to the buffer, use vmaBindBufferMemory().
1853
1854	This is a special-purpose function. In most cases you should use vmaCreateBuffer().
1855
1856	You must free the allocation using vmaFreeMemory() when no longer needed.
1857	*/
1858	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForBuffer(
1859	VmaAllocator VMA_NOT_NULL allocator,
1860	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer,
1861	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1862	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1863	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1864
1865	/** \brief Allocates memory suitable for given `VkImage`.
1866
1867	\param allocator
1868	\param image
1869	\param pCreateInfo
1870	\param[out] pAllocation Handle to allocated memory.
1871	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
1872
1873	It only creates #VmaAllocation. To bind the memory to the buffer, use vmaBindImageMemory().
1874
1875	This is a special-purpose function. In most cases you should use vmaCreateImage().
1876
1877	You must free the allocation using vmaFreeMemory() when no longer needed.
1878	*/
1879	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForImage(
1880	VmaAllocator VMA_NOT_NULL allocator,
1881	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image,
1882	const VmaAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
1883	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
1884	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
1885
1886	/** \brief Frees memory previously allocated using vmaAllocateMemory(), vmaAllocateMemoryForBuffer(), or vmaAllocateMemoryForImage().
1887
1888	Passing `VK_NULL_HANDLE` as `allocation` is valid. Such function call is just skipped.
1889	*/
1890	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemory(
1891	VmaAllocator VMA_NOT_NULL allocator,
1892	const VmaAllocation VMA_NULLABLE allocation);
1893
1894	/** \brief Frees memory and destroys multiple allocations.
1895
1896	Word "pages" is just a suggestion to use this function to free pieces of memory used for sparse binding.
1897	It is just a general purpose function to free memory and destroy allocations made using e.g. vmaAllocateMemory(),
1898	vmaAllocateMemoryPages() and other functions.
1899	It may be internally optimized to be more efficient than calling vmaFreeMemory() `allocationCount` times.
1900
1901	Allocations in `pAllocations` array can come from any memory pools and types.
1902	Passing `VK_NULL_HANDLE` as elements of `pAllocations` array is valid. Such entries are just skipped.
1903	*/
1904	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemoryPages(
1905	VmaAllocator VMA_NOT_NULL allocator,
1906	size_t allocationCount,
1907	const VmaAllocation VMA_NULLABLE* VMA_NOT_NULL VMA_LEN_IF_NOT_NULL(allocationCount) pAllocations);
1908
1909	/** \brief Returns current information about specified allocation.
1910
1911	Current paramteres of given allocation are returned in `pAllocationInfo`.
1912
1913	Although this function doesn't lock any mutex, so it should be quite efficient,
1914	you should avoid calling it too often.
1915	You can retrieve same VmaAllocationInfo structure while creating your resource, from function
1916	vmaCreateBuffer(), vmaCreateImage(). You can remember it if you are sure parameters don't change
1917	(e.g. due to defragmentation).
1918	*/
1919	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationInfo(
1920	VmaAllocator VMA_NOT_NULL allocator,
1921	VmaAllocation VMA_NOT_NULL allocation,
1922	VmaAllocationInfo* VMA_NOT_NULL pAllocationInfo);
1923
1924	/** \brief Sets pUserData in given allocation to new value.
1925
1926	The value of pointer `pUserData` is copied to allocation's `pUserData`.
1927	It is opaque, so you can use it however you want - e.g.
1928	as a pointer, ordinal number or some handle to you own data.
1929	*/
1930	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationUserData(
1931	VmaAllocator VMA_NOT_NULL allocator,
1932	VmaAllocation VMA_NOT_NULL allocation,
1933	void* VMA_NULLABLE pUserData);
1934
1935	/** \brief Sets pName in given allocation to new value.
1936
1937	`pName` must be either null, or pointer to a null-terminated string. The function
1938	makes local copy of the string and sets it as allocation's `pName`. String
1939	passed as pName doesn't need to be valid for whole lifetime of the allocation -
1940	you can free it after this call. String previously pointed by allocation's
1941	`pName` is freed from memory.
1942	*/
1943	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationName(
1944	VmaAllocator VMA_NOT_NULL allocator,
1945	VmaAllocation VMA_NOT_NULL allocation,
1946	const char* VMA_NULLABLE pName);
1947
1948	/**
1949	\brief Given an allocation, returns Property Flags of its memory type.
1950
1951	This is just a convenience function. Same information can be obtained using
1952	vmaGetAllocationInfo() + vmaGetMemoryProperties().
1953	*/
1954	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationMemoryProperties(
1955	VmaAllocator VMA_NOT_NULL allocator,
1956	VmaAllocation VMA_NOT_NULL allocation,
1957	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags);
1958
1959	/** \brief Maps memory represented by given allocation and returns pointer to it.
1960
1961	Maps memory represented by given allocation to make it accessible to CPU code.
1962	When succeeded, `*ppData` contains pointer to first byte of this memory.
1963
1964	\warning
1965	If the allocation is part of a bigger `VkDeviceMemory` block, returned pointer is
1966	correctly offsetted to the beginning of region assigned to this particular allocation.
1967	Unlike the result of `vkMapMemory`, it points to the allocation, not to the beginning of the whole block.
1968	You should not add VmaAllocationInfo::offset to it!
1969
1970	Mapping is internally reference-counted and synchronized, so despite raw Vulkan
1971	function `vkMapMemory()` cannot be used to map same block of `VkDeviceMemory`
1972	multiple times simultaneously, it is safe to call this function on allocations
1973	assigned to the same memory block. Actual Vulkan memory will be mapped on first
1974	mapping and unmapped on last unmapping.
1975
1976	If the function succeeded, you must call vmaUnmapMemory() to unmap the
1977	allocation when mapping is no longer needed or before freeing the allocation, at
1978	the latest.
1979
1980	It also safe to call this function multiple times on the same allocation. You
1981	must call vmaUnmapMemory() same number of times as you called vmaMapMemory().
1982
1983	It is also safe to call this function on allocation created with
1984	#VMA_ALLOCATION_CREATE_MAPPED_BIT flag. Its memory stays mapped all the time.
1985	You must still call vmaUnmapMemory() same number of times as you called
1986	vmaMapMemory(). You must not call vmaUnmapMemory() additional time to free the
1987	"0-th" mapping made automatically due to #VMA_ALLOCATION_CREATE_MAPPED_BIT flag.
1988
1989	This function fails when used on allocation made in memory type that is not
1990	`HOST_VISIBLE`.
1991
1992	This function doesn't automatically flush or invalidate caches.
1993	If the allocation is made from a memory types that is not `HOST_COHERENT`,
1994	you also need to use vmaInvalidateAllocation() / vmaFlushAllocation(), as required by Vulkan specification.
1995	*/
1996	VMA_CALL_PRE VkResult VMA_CALL_POST vmaMapMemory(
1997	VmaAllocator VMA_NOT_NULL allocator,
1998	VmaAllocation VMA_NOT_NULL allocation,
1999	void* VMA_NULLABLE* VMA_NOT_NULL ppData);
2000
2001	/** \brief Unmaps memory represented by given allocation, mapped previously using vmaMapMemory().
2002
2003	For details, see description of vmaMapMemory().
2004
2005	This function doesn't automatically flush or invalidate caches.
2006	If the allocation is made from a memory types that is not `HOST_COHERENT`,
2007	you also need to use vmaInvalidateAllocation() / vmaFlushAllocation(), as required by Vulkan specification.
2008	*/
2009	VMA_CALL_PRE void VMA_CALL_POST vmaUnmapMemory(
2010	VmaAllocator VMA_NOT_NULL allocator,
2011	VmaAllocation VMA_NOT_NULL allocation);
2012
2013	/** \brief Flushes memory of given allocation.
2014
2015	Calls `vkFlushMappedMemoryRanges()` for memory associated with given range of given allocation.
2016	It needs to be called after writing to a mapped memory for memory types that are not `HOST_COHERENT`.
2017	Unmap operation doesn't do that automatically.
2018
2019	- `offset` must be relative to the beginning of allocation.
2020	- `size` can be `VK_WHOLE_SIZE`. It means all memory from `offset` the the end of given allocation.
2021	- `offset` and `size` don't have to be aligned.
2022	They are internally rounded down/up to multiply of `nonCoherentAtomSize`.
2023	- If `size` is 0, this call is ignored.
2024	- If memory type that the `allocation` belongs to is not `HOST_VISIBLE` or it is `HOST_COHERENT`,
2025	this call is ignored.
2026
2027	Warning! `offset` and `size` are relative to the contents of given `allocation`.
2028	If you mean whole allocation, you can pass 0 and `VK_WHOLE_SIZE`, respectively.
2029	Do not pass allocation's offset as `offset`!!!
2030
2031	This function returns the `VkResult` from `vkFlushMappedMemoryRanges` if it is
2032	called, otherwise `VK_SUCCESS`.
2033	*/
2034	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocation(
2035	VmaAllocator VMA_NOT_NULL allocator,
2036	VmaAllocation VMA_NOT_NULL allocation,
2037	VkDeviceSize offset,
2038	VkDeviceSize size);
2039
2040	/** \brief Invalidates memory of given allocation.
2041
2042	Calls `vkInvalidateMappedMemoryRanges()` for memory associated with given range of given allocation.
2043	It needs to be called before reading from a mapped memory for memory types that are not `HOST_COHERENT`.
2044	Map operation doesn't do that automatically.
2045
2046	- `offset` must be relative to the beginning of allocation.
2047	- `size` can be `VK_WHOLE_SIZE`. It means all memory from `offset` the the end of given allocation.
2048	- `offset` and `size` don't have to be aligned.
2049	They are internally rounded down/up to multiply of `nonCoherentAtomSize`.
2050	- If `size` is 0, this call is ignored.
2051	- If memory type that the `allocation` belongs to is not `HOST_VISIBLE` or it is `HOST_COHERENT`,
2052	this call is ignored.
2053
2054	Warning! `offset` and `size` are relative to the contents of given `allocation`.
2055	If you mean whole allocation, you can pass 0 and `VK_WHOLE_SIZE`, respectively.
2056	Do not pass allocation's offset as `offset`!!!
2057
2058	This function returns the `VkResult` from `vkInvalidateMappedMemoryRanges` if
2059	it is called, otherwise `VK_SUCCESS`.
2060	*/
2061	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocation(
2062	VmaAllocator VMA_NOT_NULL allocator,
2063	VmaAllocation VMA_NOT_NULL allocation,
2064	VkDeviceSize offset,
2065	VkDeviceSize size);
2066
2067	/** \brief Flushes memory of given set of allocations.
2068
2069	Calls `vkFlushMappedMemoryRanges()` for memory associated with given ranges of given allocations.
2070	For more information, see documentation of vmaFlushAllocation().
2071
2072	\param allocator
2073	\param allocationCount
2074	\param allocations
2075	\param offsets If not null, it must point to an array of offsets of regions to flush, relative to the beginning of respective allocations. Null means all ofsets are zero.
2076	\param sizes If not null, it must point to an array of sizes of regions to flush in respective allocations. Null means `VK_WHOLE_SIZE` for all allocations.
2077
2078	This function returns the `VkResult` from `vkFlushMappedMemoryRanges` if it is
2079	called, otherwise `VK_SUCCESS`.
2080	*/
2081	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocations(
2082	VmaAllocator VMA_NOT_NULL allocator,
2083	uint32_t allocationCount,
2084	const VmaAllocation VMA_NOT_NULL* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) allocations,
2085	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) offsets,
2086	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) sizes);
2087
2088	/** \brief Invalidates memory of given set of allocations.
2089
2090	Calls `vkInvalidateMappedMemoryRanges()` for memory associated with given ranges of given allocations.
2091	For more information, see documentation of vmaInvalidateAllocation().
2092
2093	\param allocator
2094	\param allocationCount
2095	\param allocations
2096	\param offsets If not null, it must point to an array of offsets of regions to flush, relative to the beginning of respective allocations. Null means all ofsets are zero.
2097	\param sizes If not null, it must point to an array of sizes of regions to flush in respective allocations. Null means `VK_WHOLE_SIZE` for all allocations.
2098
2099	This function returns the `VkResult` from `vkInvalidateMappedMemoryRanges` if it is
2100	called, otherwise `VK_SUCCESS`.
2101	*/
2102	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocations(
2103	VmaAllocator VMA_NOT_NULL allocator,
2104	uint32_t allocationCount,
2105	const VmaAllocation VMA_NOT_NULL* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) allocations,
2106	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) offsets,
2107	const VkDeviceSize* VMA_NULLABLE VMA_LEN_IF_NOT_NULL(allocationCount) sizes);
2108
2109	/** \brief Checks magic number in margins around all allocations in given memory types (in both default and custom pools) in search for corruptions.
2110
2111	\param allocator
2112	\param memoryTypeBits Bit mask, where each bit set means that a memory type with that index should be checked.
2113
2114	Corruption detection is enabled only when `VMA_DEBUG_DETECT_CORRUPTION` macro is defined to nonzero,
2115	`VMA_DEBUG_MARGIN` is defined to nonzero and only for memory types that are
2116	`HOST_VISIBLE` and `HOST_COHERENT`. For more information, see [Corruption detection](@ref debugging_memory_usage_corruption_detection).
2117
2118	Possible return values:
2119
2120	- `VK_ERROR_FEATURE_NOT_PRESENT` - corruption detection is not enabled for any of specified memory types.
2121	- `VK_SUCCESS` - corruption detection has been performed and succeeded.
2122	- `VK_ERROR_UNKNOWN` - corruption detection has been performed and found memory corruptions around one of the allocations.
2123	`VMA_ASSERT` is also fired in that case.
2124	- Other value: Error returned by Vulkan, e.g. memory mapping failure.
2125	*/
2126	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckCorruption(
2127	VmaAllocator VMA_NOT_NULL allocator,
2128	uint32_t memoryTypeBits);
2129
2130	/** \brief Begins defragmentation process.
2131
2132	\param allocator Allocator object.
2133	\param pInfo Structure filled with parameters of defragmentation.
2134	\param[out] pContext Context object that must be passed to vmaEndDefragmentation() to finish defragmentation.
2135	\returns
2136	- `VK_SUCCESS` if defragmentation can begin.
2137	- `VK_ERROR_FEATURE_NOT_PRESENT` if defragmentation is not supported.
2138
2139	For more information about defragmentation, see documentation chapter:
2140	[Defragmentation](@ref defragmentation).
2141	*/
2142	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentation(
2143	VmaAllocator VMA_NOT_NULL allocator,
2144	const VmaDefragmentationInfo* VMA_NOT_NULL pInfo,
2145	VmaDefragmentationContext VMA_NULLABLE* VMA_NOT_NULL pContext);
2146
2147	/** \brief Ends defragmentation process.
2148
2149	\param allocator Allocator object.
2150	\param context Context object that has been created by vmaBeginDefragmentation().
2151	\param[out] pStats Optional stats for the defragmentation. Can be null.
2152
2153	Use this function to finish defragmentation started by vmaBeginDefragmentation().
2154	*/
2155	VMA_CALL_PRE void VMA_CALL_POST vmaEndDefragmentation(
2156	VmaAllocator VMA_NOT_NULL allocator,
2157	VmaDefragmentationContext VMA_NOT_NULL context,
2158	VmaDefragmentationStats* VMA_NULLABLE pStats);
2159
2160	/** \brief Starts single defragmentation pass.
2161
2162	\param allocator Allocator object.
2163	\param context Context object that has been created by vmaBeginDefragmentation().
2164	\param[out] pPassInfo Computed informations for current pass.
2165	\returns
2166	- `VK_SUCCESS` if no more moves are possible. Then you can omit call to vmaEndDefragmentationPass() and simply end whole defragmentation.
2167	- `VK_INCOMPLETE` if there are pending moves returned in `pPassInfo`. You need to perform them, call vmaEndDefragmentationPass(),
2168	and then preferably try another pass with vmaBeginDefragmentationPass().
2169	*/
2170	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentationPass(
2171	VmaAllocator VMA_NOT_NULL allocator,
2172	VmaDefragmentationContext VMA_NOT_NULL context,
2173	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo);
2174
2175	/** \brief Ends single defragmentation pass.
2176
2177	\param allocator Allocator object.
2178	\param context Context object that has been created by vmaBeginDefragmentation().
2179	\param pPassInfo Computed informations for current pass filled by vmaBeginDefragmentationPass() and possibly modified by you.
2180
2181	Returns `VK_SUCCESS` if no more moves are possible or `VK_INCOMPLETE` if more defragmentations are possible.
2182
2183	Ends incremental defragmentation pass and commits all defragmentation moves from `pPassInfo`.
2184	After this call:
2185
2186	- Allocations at `pPassInfo[i].srcAllocation` that had `pPassInfo[i].operation ==` #VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY
2187	(which is the default) will be pointing to the new destination place.
2188	- Allocation at `pPassInfo[i].srcAllocation` that had `pPassInfo[i].operation ==` #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY
2189	will be freed.
2190
2191	If no more moves are possible you can end whole defragmentation.
2192	*/
2193	VMA_CALL_PRE VkResult VMA_CALL_POST vmaEndDefragmentationPass(
2194	VmaAllocator VMA_NOT_NULL allocator,
2195	VmaDefragmentationContext VMA_NOT_NULL context,
2196	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo);
2197
2198	/** \brief Binds buffer to allocation.
2199
2200	Binds specified buffer to region of memory represented by specified allocation.
2201	Gets `VkDeviceMemory` handle and offset from the allocation.
2202	If you want to create a buffer, allocate memory for it and bind them together separately,
2203	you should use this function for binding instead of standard `vkBindBufferMemory()`,
2204	because it ensures proper synchronization so that when a `VkDeviceMemory` object is used by multiple
2205	allocations, calls to `vkBind*Memory()` or `vkMapMemory()` won't happen from multiple threads simultaneously
2206	(which is illegal in Vulkan).
2207
2208	It is recommended to use function vmaCreateBuffer() instead of this one.
2209	*/
2210	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory(
2211	VmaAllocator VMA_NOT_NULL allocator,
2212	VmaAllocation VMA_NOT_NULL allocation,
2213	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer);
2214
2215	/** \brief Binds buffer to allocation with additional parameters.
2216
2217	\param allocator
2218	\param allocation
2219	\param allocationLocalOffset Additional offset to be added while binding, relative to the beginning of the `allocation`. Normally it should be 0.
2220	\param buffer
2221	\param pNext A chain of structures to be attached to `VkBindBufferMemoryInfoKHR` structure used internally. Normally it should be null.
2222
2223	This function is similar to vmaBindBufferMemory(), but it provides additional parameters.
2224
2225	If `pNext` is not null, #VmaAllocator object must have been created with #VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT flag
2226	or with VmaAllocatorCreateInfo::vulkanApiVersion `>= VK_API_VERSION_1_1`. Otherwise the call fails.
2227	*/
2228	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory2(
2229	VmaAllocator VMA_NOT_NULL allocator,
2230	VmaAllocation VMA_NOT_NULL allocation,
2231	VkDeviceSize allocationLocalOffset,
2232	VkBuffer VMA_NOT_NULL_NON_DISPATCHABLE buffer,
2233	const void* VMA_NULLABLE pNext);
2234
2235	/** \brief Binds image to allocation.
2236
2237	Binds specified image to region of memory represented by specified allocation.
2238	Gets `VkDeviceMemory` handle and offset from the allocation.
2239	If you want to create an image, allocate memory for it and bind them together separately,
2240	you should use this function for binding instead of standard `vkBindImageMemory()`,
2241	because it ensures proper synchronization so that when a `VkDeviceMemory` object is used by multiple
2242	allocations, calls to `vkBind*Memory()` or `vkMapMemory()` won't happen from multiple threads simultaneously
2243	(which is illegal in Vulkan).
2244
2245	It is recommended to use function vmaCreateImage() instead of this one.
2246	*/
2247	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory(
2248	VmaAllocator VMA_NOT_NULL allocator,
2249	VmaAllocation VMA_NOT_NULL allocation,
2250	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image);
2251
2252	/** \brief Binds image to allocation with additional parameters.
2253
2254	\param allocator
2255	\param allocation
2256	\param allocationLocalOffset Additional offset to be added while binding, relative to the beginning of the `allocation`. Normally it should be 0.
2257	\param image
2258	\param pNext A chain of structures to be attached to `VkBindImageMemoryInfoKHR` structure used internally. Normally it should be null.
2259
2260	This function is similar to vmaBindImageMemory(), but it provides additional parameters.
2261
2262	If `pNext` is not null, #VmaAllocator object must have been created with #VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT flag
2263	or with VmaAllocatorCreateInfo::vulkanApiVersion `>= VK_API_VERSION_1_1`. Otherwise the call fails.
2264	*/
2265	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory2(
2266	VmaAllocator VMA_NOT_NULL allocator,
2267	VmaAllocation VMA_NOT_NULL allocation,
2268	VkDeviceSize allocationLocalOffset,
2269	VkImage VMA_NOT_NULL_NON_DISPATCHABLE image,
2270	const void* VMA_NULLABLE pNext);
2271
2272	/** \brief Creates a new `VkBuffer`, allocates and binds memory for it.
2273
2274	\param allocator
2275	\param pBufferCreateInfo
2276	\param pAllocationCreateInfo
2277	\param[out] pBuffer Buffer that was created.
2278	\param[out] pAllocation Allocation that was created.
2279	\param[out] pAllocationInfo Optional. Information about allocated memory. It can be later fetched using function vmaGetAllocationInfo().
2280
2281	This function automatically:
2282
2283	-# Creates buffer.
2284	-# Allocates appropriate memory for it.
2285	-# Binds the buffer with the memory.
2286
2287	If any of these operations fail, buffer and allocation are not created,
2288	returned value is negative error code, `*pBuffer` and `*pAllocation` are null.
2289
2290	If the function succeeded, you must destroy both buffer and allocation when you
2291	no longer need them using either convenience function vmaDestroyBuffer() or
2292	separately, using `vkDestroyBuffer()` and vmaFreeMemory().
2293
2294	If #VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag was used,
2295	VK_KHR_dedicated_allocation extension is used internally to query driver whether
2296	it requires or prefers the new buffer to have dedicated allocation. If yes,
2297	and if dedicated allocation is possible
2298	(#VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT is not used), it creates dedicated
2299	allocation for this buffer, just like when using
2300	#VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
2301
2302	\note This function creates a new `VkBuffer`. Sub-allocation of parts of one large buffer,
2303	although recommended as a good practice, is out of scope of this library and could be implemented
2304	by the user as a higher-level logic on top of VMA.
2305	*/
2306	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBuffer(
2307	VmaAllocator VMA_NOT_NULL allocator,
2308	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2309	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2310	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer,
2311	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2312	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2313
2314	/** \brief Creates a buffer with additional minimum alignment.
2315
2316	Similar to vmaCreateBuffer() but provides additional parameter `minAlignment` which allows to specify custom,
2317	minimum alignment to be used when placing the buffer inside a larger memory block, which may be needed e.g.
2318	for interop with OpenGL.
2319	*/
2320	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBufferWithAlignment(
2321	VmaAllocator VMA_NOT_NULL allocator,
2322	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2323	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2324	VkDeviceSize minAlignment,
2325	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer,
2326	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2327	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2328
2329	/** \brief Creates a new `VkBuffer`, binds already created memory for it.
2330
2331	\param allocator
2332	\param allocation Allocation that provides memory to be used for binding new buffer to it.
2333	\param pBufferCreateInfo
2334	\param[out] pBuffer Buffer that was created.
2335
2336	This function automatically:
2337
2338	-# Creates buffer.
2339	-# Binds the buffer with the supplied memory.
2340
2341	If any of these operations fail, buffer is not created,
2342	returned value is negative error code and `*pBuffer` is null.
2343
2344	If the function succeeded, you must destroy the buffer when you
2345	no longer need it using `vkDestroyBuffer()`. If you want to also destroy the corresponding
2346	allocation you can use convenience function vmaDestroyBuffer().
2347	*/
2348	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingBuffer(
2349	VmaAllocator VMA_NOT_NULL allocator,
2350	VmaAllocation VMA_NOT_NULL allocation,
2351	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
2352	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer);
2353
2354	/** \brief Destroys Vulkan buffer and frees allocated memory.
2355
2356	This is just a convenience function equivalent to:
2357
2358	\code
2359	vkDestroyBuffer(device, buffer, allocationCallbacks);
2360	vmaFreeMemory(allocator, allocation);
2361	\endcode
2362
2363	It it safe to pass null as buffer and/or allocation.
2364	*/
2365	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyBuffer(
2366	VmaAllocator VMA_NOT_NULL allocator,
2367	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE buffer,
2368	VmaAllocation VMA_NULLABLE allocation);
2369
2370	/// Function similar to vmaCreateBuffer().
2371	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateImage(
2372	VmaAllocator VMA_NOT_NULL allocator,
2373	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
2374	const VmaAllocationCreateInfo* VMA_NOT_NULL pAllocationCreateInfo,
2375	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage,
2376	VmaAllocation VMA_NULLABLE* VMA_NOT_NULL pAllocation,
2377	VmaAllocationInfo* VMA_NULLABLE pAllocationInfo);
2378
2379	/// Function similar to vmaCreateAliasingBuffer().
2380	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingImage(
2381	VmaAllocator VMA_NOT_NULL allocator,
2382	VmaAllocation VMA_NOT_NULL allocation,
2383	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
2384	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage);
2385
2386	/** \brief Destroys Vulkan image and frees allocated memory.
2387
2388	This is just a convenience function equivalent to:
2389
2390	\code
2391	vkDestroyImage(device, image, allocationCallbacks);
2392	vmaFreeMemory(allocator, allocation);
2393	\endcode
2394
2395	It it safe to pass null as image and/or allocation.
2396	*/
2397	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyImage(
2398	VmaAllocator VMA_NOT_NULL allocator,
2399	VkImage VMA_NULLABLE_NON_DISPATCHABLE image,
2400	VmaAllocation VMA_NULLABLE allocation);
2401
2402	/** @} */
2403
2404	/**
2405	\addtogroup group_virtual
2406	@{
2407	*/
2408
2409	/** \brief Creates new #VmaVirtualBlock object.
2410
2411	\param pCreateInfo Parameters for creation.
2412	\param[out] pVirtualBlock Returned virtual block object or `VMA_NULL` if creation failed.
2413	*/
2414	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateVirtualBlock(
2415	const VmaVirtualBlockCreateInfo* VMA_NOT_NULL pCreateInfo,
2416	VmaVirtualBlock VMA_NULLABLE* VMA_NOT_NULL pVirtualBlock);
2417
2418	/** \brief Destroys #VmaVirtualBlock object.
2419
2420	Please note that you should consciously handle virtual allocations that could remain unfreed in the block.
2421	You should either free them individually using vmaVirtualFree() or call vmaClearVirtualBlock()
2422	if you are sure this is what you want. If you do neither, an assert is called.
2423
2424	If you keep pointers to some additional metadata associated with your virtual allocations in their `pUserData`,
2425	don't forget to free them.
2426	*/
2427	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyVirtualBlock(
2428	VmaVirtualBlock VMA_NULLABLE virtualBlock);
2429
2430	/** \brief Returns true of the #VmaVirtualBlock is empty - contains 0 virtual allocations and has all its space available for new allocations.
2431	*/
2432	VMA_CALL_PRE VkBool32 VMA_CALL_POST vmaIsVirtualBlockEmpty(
2433	VmaVirtualBlock VMA_NOT_NULL virtualBlock);
2434
2435	/** \brief Returns information about a specific virtual allocation within a virtual block, like its size and `pUserData` pointer.
2436	*/
2437	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualAllocationInfo(
2438	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2439	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, VmaVirtualAllocationInfo* VMA_NOT_NULL pVirtualAllocInfo);
2440
2441	/** \brief Allocates new virtual allocation inside given #VmaVirtualBlock.
2442
2443	If the allocation fails due to not enough free space available, `VK_ERROR_OUT_OF_DEVICE_MEMORY` is returned
2444	(despite the function doesn't ever allocate actual GPU memory).
2445	`pAllocation` is then set to `VK_NULL_HANDLE` and `pOffset`, if not null, it set to `UINT64_MAX`.
2446
2447	\param virtualBlock Virtual block
2448	\param pCreateInfo Parameters for the allocation
2449	\param[out] pAllocation Returned handle of the new allocation
2450	\param[out] pOffset Returned offset of the new allocation. Optional, can be null.
2451	*/
2452	VMA_CALL_PRE VkResult VMA_CALL_POST vmaVirtualAllocate(
2453	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2454	const VmaVirtualAllocationCreateInfo* VMA_NOT_NULL pCreateInfo,
2455	VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pAllocation,
2456	VkDeviceSize* VMA_NULLABLE pOffset);
2457
2458	/** \brief Frees virtual allocation inside given #VmaVirtualBlock.
2459
2460	It is correct to call this function with `allocation == VK_NULL_HANDLE` - it does nothing.
2461	*/
2462	VMA_CALL_PRE void VMA_CALL_POST vmaVirtualFree(
2463	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2464	VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE allocation);
2465
2466	/** \brief Frees all virtual allocations inside given #VmaVirtualBlock.
2467
2468	You must either call this function or free each virtual allocation individually with vmaVirtualFree()
2469	before destroying a virtual block. Otherwise, an assert is called.
2470
2471	If you keep pointer to some additional metadata associated with your virtual allocation in its `pUserData`,
2472	don't forget to free it as well.
2473	*/
2474	VMA_CALL_PRE void VMA_CALL_POST vmaClearVirtualBlock(
2475	VmaVirtualBlock VMA_NOT_NULL virtualBlock);
2476
2477	/** \brief Changes custom pointer associated with given virtual allocation.
2478	*/
2479	VMA_CALL_PRE void VMA_CALL_POST vmaSetVirtualAllocationUserData(
2480	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2481	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation,
2482	void* VMA_NULLABLE pUserData);
2483
2484	/** \brief Calculates and returns statistics about virtual allocations and memory usage in given #VmaVirtualBlock.
2485
2486	This function is fast to call. For more detailed statistics, see vmaCalculateVirtualBlockStatistics().
2487	*/
2488	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualBlockStatistics(
2489	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2490	VmaStatistics* VMA_NOT_NULL pStats);
2491
2492	/** \brief Calculates and returns detailed statistics about virtual allocations and memory usage in given #VmaVirtualBlock.
2493
2494	This function is slow to call. Use for debugging purposes.
2495	For less detailed statistics, see vmaGetVirtualBlockStatistics().
2496	*/
2497	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateVirtualBlockStatistics(
2498	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2499	VmaDetailedStatistics* VMA_NOT_NULL pStats);
2500
2501	/** @} */
2502
2503	#if VMA_STATS_STRING_ENABLED
2504	/**
2505	\addtogroup group_stats
2506	@{
2507	*/
2508
2509	/** \brief Builds and returns a null-terminated string in JSON format with information about given #VmaVirtualBlock.
2510	\param virtualBlock Virtual block.
2511	\param[out] ppStatsString Returned string.
2512	\param detailedMap Pass `VK_FALSE` to only obtain statistics as returned by vmaCalculateVirtualBlockStatistics(). Pass `VK_TRUE` to also obtain full list of allocations and free spaces.
2513
2514	Returned string must be freed using vmaFreeVirtualBlockStatsString().
2515	*/
2516	VMA_CALL_PRE void VMA_CALL_POST vmaBuildVirtualBlockStatsString(
2517	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2518	char* VMA_NULLABLE* VMA_NOT_NULL ppStatsString,
2519	VkBool32 detailedMap);
2520
2521	/// Frees a string returned by vmaBuildVirtualBlockStatsString().
2522	VMA_CALL_PRE void VMA_CALL_POST vmaFreeVirtualBlockStatsString(
2523	VmaVirtualBlock VMA_NOT_NULL virtualBlock,
2524	char* VMA_NULLABLE pStatsString);
2525
2526	/** \brief Builds and returns statistics as a null-terminated string in JSON format.
2527	\param allocator
2528	\param[out] ppStatsString Must be freed using vmaFreeStatsString() function.
2529	\param detailedMap
2530	*/
2531	VMA_CALL_PRE void VMA_CALL_POST vmaBuildStatsString(
2532	VmaAllocator VMA_NOT_NULL allocator,
2533	char* VMA_NULLABLE* VMA_NOT_NULL ppStatsString,
2534	VkBool32 detailedMap);
2535
2536	VMA_CALL_PRE void VMA_CALL_POST vmaFreeStatsString(
2537	VmaAllocator VMA_NOT_NULL allocator,
2538	char* VMA_NULLABLE pStatsString);
2539
2540	/** @} */
2541
2542	#endif // VMA_STATS_STRING_ENABLED
2543
2544	#endif // _VMA_FUNCTION_HEADERS
2545
2546	#ifdef __cplusplus
2547	}
2548	#endif
2549
2550	#endif // AMD_VULKAN_MEMORY_ALLOCATOR_H
2551
2552	////////////////////////////////////////////////////////////////////////////////
2553	////////////////////////////////////////////////////////////////////////////////
2554	//
2555	// IMPLEMENTATION
2556	//
2557	////////////////////////////////////////////////////////////////////////////////
2558	////////////////////////////////////////////////////////////////////////////////
2559
2560	// For Visual Studio IntelliSense.
2561	#if defined(__cplusplus) && defined(__INTELLISENSE__)
2562	#define VMA_IMPLEMENTATION
2563	#endif
2564
2565	#ifdef VMA_IMPLEMENTATION
2566	#undef VMA_IMPLEMENTATION
2567
2568	#if defined(__GNUC__) && !defined(__clang__)
2569	#pragma GCC diagnostic push
2570	#pragma GCC diagnostic ignored "-Wunused-variable"
2571	#pragma GCC diagnostic ignored "-Wunused-parameter"
2572	#pragma GCC diagnostic ignored "-Wmissing-field-initializers"
2573	#pragma GCC diagnostic ignored "-Wparentheses"
2574	#pragma GCC diagnostic ignored "-Wimplicit-fallthrough"
2575	#elif defined(__clang__)
2576	#pragma clang diagnostic push
2577	#pragma clang diagnostic ignored "-Wunused-variable"
2578	#pragma clang diagnostic ignored "-Wunused-parameter"
2579	#pragma clang diagnostic ignored "-Wmissing-field-initializers"
2580	#pragma clang diagnostic ignored "-Wparentheses"
2581	#pragma clang diagnostic ignored "-Wimplicit-fallthrough"
2582	#pragma clang diagnostic ignored "-Wnullability-completeness"
2583	#endif
2584
2585	#include <cstdint>
2586	#include <cstdlib>
2587	#include <cstring>
2588	#include <utility>
2589	#include <type_traits>
2590
2591	#ifdef _MSC_VER
2592	#include <intrin.h> // For functions like __popcnt, _BitScanForward etc.
2593	#endif
2594	#if __cplusplus >= 202002L || _MSVC_LANG >= 202002L // C++20
2595	#include <bit> // For std::popcount
2596	#endif
2597
2598	/*******************************************************************************
2599	CONFIGURATION SECTION
2600
2601	Define some of these macros before each #include of this header or change them
2602	here if you need other then default behavior depending on your environment.
2603	*/
2604	#ifndef _VMA_CONFIGURATION
2605
2606	/*
2607	Define this macro to 1 to make the library fetch pointers to Vulkan functions
2608	internally, like:
2609
2610	vulkanFunctions.vkAllocateMemory = &vkAllocateMemory;
2611	*/
2612	#if !defined(VMA_STATIC_VULKAN_FUNCTIONS) && !defined(VK_NO_PROTOTYPES)
2613	#define VMA_STATIC_VULKAN_FUNCTIONS 1
2614	#endif
2615
2616	/*
2617	Define this macro to 1 to make the library fetch pointers to Vulkan functions
2618	internally, like:
2619
2620	vulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkGetDeviceProcAddr(device, "vkAllocateMemory");
2621
2622	To use this feature in new versions of VMA you now have to pass
2623	VmaVulkanFunctions::vkGetInstanceProcAddr and vkGetDeviceProcAddr as
2624	VmaAllocatorCreateInfo::pVulkanFunctions. Other members can be null.
2625	*/
2626	#if !defined(VMA_DYNAMIC_VULKAN_FUNCTIONS)
2627	#define VMA_DYNAMIC_VULKAN_FUNCTIONS 1
2628	#endif
2629
2630	#ifndef VMA_USE_STL_SHARED_MUTEX
2631	// Compiler conforms to C++17.
2632	#if __cplusplus >= 201703L
2633	#define VMA_USE_STL_SHARED_MUTEX 1
2634	// Visual studio defines __cplusplus properly only when passed additional parameter: /Zc:__cplusplus
2635	// Otherwise it is always 199711L, despite shared_mutex works since Visual Studio 2015 Update 2.
2636	#elif defined(_MSC_FULL_VER) && _MSC_FULL_VER >= 190023918 && __cplusplus == 199711L && _MSVC_LANG >= 201703L
2637	#define VMA_USE_STL_SHARED_MUTEX 1
2638	#else
2639	#define VMA_USE_STL_SHARED_MUTEX 0
2640	#endif
2641	#endif
2642
2643	/*
2644	Define this macro to include custom header files without having to edit this file directly, e.g.:
2645
2646	// Inside of "my_vma_configuration_user_includes.h":
2647
2648	#include "my_custom_assert.h" // for MY_CUSTOM_ASSERT
2649	#include "my_custom_min.h" // for my_custom_min
2650	#include <algorithm>
2651	#include <mutex>
2652
2653	// Inside a different file, which includes "vk_mem_alloc.h":
2654
2655	#define VMA_CONFIGURATION_USER_INCLUDES_H "my_vma_configuration_user_includes.h"
2656	#define VMA_ASSERT(expr) MY_CUSTOM_ASSERT(expr)
2657	#define VMA_MIN(v1, v2) (my_custom_min(v1, v2))
2658	#include "vk_mem_alloc.h"
2659	...
2660
2661	The following headers are used in this CONFIGURATION section only, so feel free to
2662	remove them if not needed.
2663	*/
2664	#if !defined(VMA_CONFIGURATION_USER_INCLUDES_H)
2665	#include <cassert> // for assert
2666	#include <algorithm> // for min, max
2667	#include <mutex>
2668	#else
2669	#include VMA_CONFIGURATION_USER_INCLUDES_H
2670	#endif
2671
2672	#ifndef VMA_NULL
2673	// Value used as null pointer. Define it to e.g.: nullptr, NULL, 0, (void*)0.
2674	#define VMA_NULL nullptr
2675	#endif
2676
2677	#if defined(__ANDROID_API__) && (__ANDROID_API__ < 16)
2678	#include <cstdlib>
2679	static void* vma_aligned_alloc(size_t alignment, size_t size)
2680	{
2681	// alignment must be >= sizeof(void*)
2682	if(alignment < sizeof(void*))
2683	{
2684	alignment = sizeof(void*);
2685	}
2686
2687	return memalign(alignment, size);
2688	}
2689	#elif defined(__APPLE__) || defined(__ANDROID__) || (defined(__linux__) && defined(__GLIBCXX__) && !defined(_GLIBCXX_HAVE_ALIGNED_ALLOC))
2690	#include <cstdlib>
2691
2692	#if defined(__APPLE__)
2693	#include <AvailabilityMacros.h>
2694	#endif
2695
2696	static void* vma_aligned_alloc(size_t alignment, size_t size)
2697	{
2698	// Unfortunately, aligned_alloc causes VMA to crash due to it returning null pointers. (At least under 11.4)
2699	// Therefore, for now disable this specific exception until a proper solution is found.
2700	//#if defined(__APPLE__) && (defined(MAC_OS_X_VERSION_10_16) || defined(__IPHONE_14_0))
2701	//#if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_16 || __IPHONE_OS_VERSION_MAX_ALLOWED >= __IPHONE_14_0
2702	// // For C++14, usr/include/malloc/_malloc.h declares aligned_alloc()) only
2703	// // with the MacOSX11.0 SDK in Xcode 12 (which is what adds
2704	// // MAC_OS_X_VERSION_10_16), even though the function is marked
2705	// // availabe for 10.15. That is why the preprocessor checks for 10.16 but
2706	// // the __builtin_available checks for 10.15.
2707	// // People who use C++17 could call aligned_alloc with the 10.15 SDK already.
2708	// if (__builtin_available(macOS 10.15, iOS 13, *))
2709	// return aligned_alloc(alignment, size);
2710	//#endif
2711	//#endif
2712
2713	// alignment must be >= sizeof(void*)
2714	if(alignment < sizeof(void*))
2715	{
2716	alignment = sizeof(void*);
2717	}
2718
2719	void *pointer;
2720	if(posix_memalign(&pointer, alignment, size) == 0)
2721	return pointer;
2722	return VMA_NULL;
2723	}
2724	#elif defined(_WIN32)
2725	static void* vma_aligned_alloc(size_t alignment, size_t size)
2726	{
2727	return _aligned_malloc(size, alignment);
2728	}
2729	#else
2730	static void* vma_aligned_alloc(size_t alignment, size_t size)
2731	{
2732	return aligned_alloc(alignment: alignment, size: size);
2733	}
2734	#endif
2735
2736	#if defined(_WIN32)
2737	static void vma_aligned_free(void* ptr)
2738	{
2739	_aligned_free(ptr);
2740	}
2741	#else
2742	static void vma_aligned_free(void* VMA_NULLABLE ptr)
2743	{
2744	free(ptr: ptr);
2745	}
2746	#endif
2747
2748	// If your compiler is not compatible with C++11 and definition of
2749	// aligned_alloc() function is missing, uncommeting following line may help:
2750
2751	//#include <malloc.h>
2752
2753	// Normal assert to check for programmer's errors, especially in Debug configuration.
2754	#ifndef VMA_ASSERT
2755	#ifdef NDEBUG
2756	#define VMA_ASSERT(expr)
2757	#else
2758	#define VMA_ASSERT(expr) assert(expr)
2759	#endif
2760	#endif
2761
2762	// Assert that will be called very often, like inside data structures e.g. operator[].
2763	// Making it non-empty can make program slow.
2764	#ifndef VMA_HEAVY_ASSERT
2765	#ifdef NDEBUG
2766	#define VMA_HEAVY_ASSERT(expr)
2767	#else
2768	#define VMA_HEAVY_ASSERT(expr) //VMA_ASSERT(expr)
2769	#endif
2770	#endif
2771
2772	#ifndef VMA_ALIGN_OF
2773	#define VMA_ALIGN_OF(type) (__alignof(type))
2774	#endif
2775
2776	#ifndef VMA_SYSTEM_ALIGNED_MALLOC
2777	#define VMA_SYSTEM_ALIGNED_MALLOC(size, alignment) vma_aligned_alloc((alignment), (size))
2778	#endif
2779
2780	#ifndef VMA_SYSTEM_ALIGNED_FREE
2781	// VMA_SYSTEM_FREE is the old name, but might have been defined by the user
2782	#if defined(VMA_SYSTEM_FREE)
2783	#define VMA_SYSTEM_ALIGNED_FREE(ptr) VMA_SYSTEM_FREE(ptr)
2784	#else
2785	#define VMA_SYSTEM_ALIGNED_FREE(ptr) vma_aligned_free(ptr)
2786	#endif
2787	#endif
2788
2789	#ifndef VMA_COUNT_BITS_SET
2790	// Returns number of bits set to 1 in (v)
2791	#define VMA_COUNT_BITS_SET(v) VmaCountBitsSet(v)
2792	#endif
2793
2794	#ifndef VMA_BITSCAN_LSB
2795	// Scans integer for index of first nonzero value from the Least Significant Bit (LSB). If mask is 0 then returns UINT8_MAX
2796	#define VMA_BITSCAN_LSB(mask) VmaBitScanLSB(mask)
2797	#endif
2798
2799	#ifndef VMA_BITSCAN_MSB
2800	// Scans integer for index of first nonzero value from the Most Significant Bit (MSB). If mask is 0 then returns UINT8_MAX
2801	#define VMA_BITSCAN_MSB(mask) VmaBitScanMSB(mask)
2802	#endif
2803
2804	#ifndef VMA_MIN
2805	#define VMA_MIN(v1, v2) ((std::min)((v1), (v2)))
2806	#endif
2807
2808	#ifndef VMA_MAX
2809	#define VMA_MAX(v1, v2) ((std::max)((v1), (v2)))
2810	#endif
2811
2812	#ifndef VMA_SWAP
2813	#define VMA_SWAP(v1, v2) std::swap((v1), (v2))
2814	#endif
2815
2816	#ifndef VMA_SORT
2817	#define VMA_SORT(beg, end, cmp) std::sort(beg, end, cmp)
2818	#endif
2819
2820	#ifndef VMA_DEBUG_LOG
2821	#define VMA_DEBUG_LOG(format, ...)
2822	/*
2823	#define VMA_DEBUG_LOG(format, ...) do { \
2824	printf(format, __VA_ARGS__); \
2825	printf("\n"); \
2826	} while(false)
2827	*/
2828	#endif
2829
2830	// Define this macro to 1 to enable functions: vmaBuildStatsString, vmaFreeStatsString.
2831	#if VMA_STATS_STRING_ENABLED
2832	static inline void VmaUint32ToStr(char* VMA_NOT_NULL outStr, size_t strLen, uint32_t num)
2833	{
2834	snprintf(s: outStr, maxlen: strLen, format: "%u", static_cast<unsigned int>(num));
2835	}
2836	static inline void VmaUint64ToStr(char* VMA_NOT_NULL outStr, size_t strLen, uint64_t num)
2837	{
2838	snprintf(s: outStr, maxlen: strLen, format: "%llu", static_cast<unsigned long long>(num));
2839	}
2840	static inline void VmaPtrToStr(char* VMA_NOT_NULL outStr, size_t strLen, const void* ptr)
2841	{
2842	snprintf(s: outStr, maxlen: strLen, format: "%p", ptr);
2843	}
2844	#endif
2845
2846	#ifndef VMA_MUTEX
2847	class VmaMutex
2848	{
2849	public:
2850	void Lock() { m_Mutex.lock(); }
2851	void Unlock() { m_Mutex.unlock(); }
2852	bool TryLock() { return m_Mutex.try_lock(); }
2853	private:
2854	std::mutex m_Mutex;
2855	};
2856	#define VMA_MUTEX VmaMutex
2857	#endif
2858
2859	// Read-write mutex, where "read" is shared access, "write" is exclusive access.
2860	#ifndef VMA_RW_MUTEX
2861	#if VMA_USE_STL_SHARED_MUTEX
2862	// Use std::shared_mutex from C++17.
2863	#include <shared_mutex>
2864	class VmaRWMutex
2865	{
2866	public:
2867	void LockRead() { m_Mutex.lock_shared(); }
2868	void UnlockRead() { m_Mutex.unlock_shared(); }
2869	bool TryLockRead() { return m_Mutex.try_lock_shared(); }
2870	void LockWrite() { m_Mutex.lock(); }
2871	void UnlockWrite() { m_Mutex.unlock(); }
2872	bool TryLockWrite() { return m_Mutex.try_lock(); }
2873	private:
2874	std::shared_mutex m_Mutex;
2875	};
2876	#define VMA_RW_MUTEX VmaRWMutex
2877	#elif defined(_WIN32) && defined(WINVER) && WINVER >= 0x0600 && !defined(__MINGW32__)
2878	// Use SRWLOCK from WinAPI.
2879	// Minimum supported client = Windows Vista, server = Windows Server 2008.
2880	class VmaRWMutex
2881	{
2882	public:
2883	VmaRWMutex() { InitializeSRWLock(&m_Lock); }
2884	void LockRead() { AcquireSRWLockShared(&m_Lock); }
2885	void UnlockRead() { ReleaseSRWLockShared(&m_Lock); }
2886	bool TryLockRead() { return TryAcquireSRWLockShared(&m_Lock) != FALSE; }
2887	void LockWrite() { AcquireSRWLockExclusive(&m_Lock); }
2888	void UnlockWrite() { ReleaseSRWLockExclusive(&m_Lock); }
2889	bool TryLockWrite() { return TryAcquireSRWLockExclusive(&m_Lock) != FALSE; }
2890	private:
2891	SRWLOCK m_Lock;
2892	};
2893	#define VMA_RW_MUTEX VmaRWMutex
2894	#else
2895	// Less efficient fallback: Use normal mutex.
2896	class VmaRWMutex
2897	{
2898	public:
2899	void LockRead() { m_Mutex.Lock(); }
2900	void UnlockRead() { m_Mutex.Unlock(); }
2901	bool TryLockRead() { return m_Mutex.TryLock(); }
2902	void LockWrite() { m_Mutex.Lock(); }
2903	void UnlockWrite() { m_Mutex.Unlock(); }
2904	bool TryLockWrite() { return m_Mutex.TryLock(); }
2905	private:
2906	VMA_MUTEX m_Mutex;
2907	};
2908	#define VMA_RW_MUTEX VmaRWMutex
2909	#endif // #if VMA_USE_STL_SHARED_MUTEX
2910	#endif // #ifndef VMA_RW_MUTEX
2911
2912	/*
2913	If providing your own implementation, you need to implement a subset of std::atomic.
2914	*/
2915	#ifndef VMA_ATOMIC_UINT32
2916	#include <atomic>
2917	#define VMA_ATOMIC_UINT32 std::atomic<uint32_t>
2918	#endif
2919
2920	#ifndef VMA_ATOMIC_UINT64
2921	#include <atomic>
2922	#define VMA_ATOMIC_UINT64 std::atomic<uint64_t>
2923	#endif
2924
2925	#ifndef VMA_DEBUG_ALWAYS_DEDICATED_MEMORY
2926	/**
2927	Every allocation will have its own memory block.
2928	Define to 1 for debugging purposes only.
2929	*/
2930	#define VMA_DEBUG_ALWAYS_DEDICATED_MEMORY (0)
2931	#endif
2932
2933	#ifndef VMA_MIN_ALIGNMENT
2934	/**
2935	Minimum alignment of all allocations, in bytes.
2936	Set to more than 1 for debugging purposes. Must be power of two.
2937	*/
2938	#ifdef VMA_DEBUG_ALIGNMENT // Old name
2939	#define VMA_MIN_ALIGNMENT VMA_DEBUG_ALIGNMENT
2940	#else
2941	#define VMA_MIN_ALIGNMENT (1)
2942	#endif
2943	#endif
2944
2945	#ifndef VMA_DEBUG_MARGIN
2946	/**
2947	Minimum margin after every allocation, in bytes.
2948	Set nonzero for debugging purposes only.
2949	*/
2950	#define VMA_DEBUG_MARGIN (0)
2951	#endif
2952
2953	#ifndef VMA_DEBUG_INITIALIZE_ALLOCATIONS
2954	/**
2955	Define this macro to 1 to automatically fill new allocations and destroyed
2956	allocations with some bit pattern.
2957	*/
2958	#define VMA_DEBUG_INITIALIZE_ALLOCATIONS (0)
2959	#endif
2960
2961	#ifndef VMA_DEBUG_DETECT_CORRUPTION
2962	/**
2963	Define this macro to 1 together with non-zero value of VMA_DEBUG_MARGIN to
2964	enable writing magic value to the margin after every allocation and
2965	validating it, so that memory corruptions (out-of-bounds writes) are detected.
2966	*/
2967	#define VMA_DEBUG_DETECT_CORRUPTION (0)
2968	#endif
2969
2970	#ifndef VMA_DEBUG_GLOBAL_MUTEX
2971	/**
2972	Set this to 1 for debugging purposes only, to enable single mutex protecting all
2973	entry calls to the library. Can be useful for debugging multithreading issues.
2974	*/
2975	#define VMA_DEBUG_GLOBAL_MUTEX (0)
2976	#endif
2977
2978	#ifndef VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY
2979	/**
2980	Minimum value for VkPhysicalDeviceLimits::bufferImageGranularity.
2981	Set to more than 1 for debugging purposes only. Must be power of two.
2982	*/
2983	#define VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY (1)
2984	#endif
2985
2986	#ifndef VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT
2987	/*
2988	Set this to 1 to make VMA never exceed VkPhysicalDeviceLimits::maxMemoryAllocationCount
2989	and return error instead of leaving up to Vulkan implementation what to do in such cases.
2990	*/
2991	#define VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT (0)
2992	#endif
2993
2994	#ifndef VMA_SMALL_HEAP_MAX_SIZE
2995	/// Maximum size of a memory heap in Vulkan to consider it "small".
2996	#define VMA_SMALL_HEAP_MAX_SIZE (1024ull * 1024 * 1024)
2997	#endif
2998
2999	#ifndef VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE
3000	/// Default size of a block allocated as single VkDeviceMemory from a "large" heap.
3001	#define VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE (256ull * 1024 * 1024)
3002	#endif
3003
3004	/*
3005	Mapping hysteresis is a logic that launches when vmaMapMemory/vmaUnmapMemory is called
3006	or a persistently mapped allocation is created and destroyed several times in a row.
3007	It keeps additional +1 mapping of a device memory block to prevent calling actual
3008	vkMapMemory/vkUnmapMemory too many times, which may improve performance and help
3009	tools like RenderDOc.
3010	*/
3011	#ifndef VMA_MAPPING_HYSTERESIS_ENABLED
3012	#define VMA_MAPPING_HYSTERESIS_ENABLED 1
3013	#endif
3014
3015	#ifndef VMA_CLASS_NO_COPY
3016	#define VMA_CLASS_NO_COPY(className) \
3017	private: \
3018	className(const className&) = delete; \
3019	className& operator=(const className&) = delete;
3020	#endif
3021
3022	#define VMA_VALIDATE(cond) do { if(!(cond)) { \
3023	VMA_ASSERT(0 && "Validation failed: " #cond); \
3024	return false; \
3025	} } while(false)
3026
3027	/*******************************************************************************
3028	END OF CONFIGURATION
3029	*/
3030	#endif // _VMA_CONFIGURATION
3031
3032
3033	static const uint8_t VMA_ALLOCATION_FILL_PATTERN_CREATED = 0xDC;
3034	static const uint8_t VMA_ALLOCATION_FILL_PATTERN_DESTROYED = 0xEF;
3035	// Decimal 2139416166, float NaN, little-endian binary 66 E6 84 7F.
3036	static const uint32_t VMA_CORRUPTION_DETECTION_MAGIC_VALUE = 0x7F84E666;
3037
3038	// Copy of some Vulkan definitions so we don't need to check their existence just to handle few constants.
3039	static const uint32_t VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY = 0x00000040;
3040	static const uint32_t VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY = 0x00000080;
3041	static const uint32_t VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY = 0x00020000;
3042	static const uint32_t VK_IMAGE_CREATE_DISJOINT_BIT_COPY = 0x00000200;
3043	static const int32_t VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT_COPY = 1000158000;
3044	static const uint32_t VMA_ALLOCATION_INTERNAL_STRATEGY_MIN_OFFSET = 0x10000000u;
3045	static const uint32_t VMA_ALLOCATION_TRY_COUNT = 32;
3046	static const uint32_t VMA_VENDOR_ID_AMD = 4098;
3047
3048	// This one is tricky. Vulkan specification defines this code as available since
3049	// Vulkan 1.0, but doesn't actually define it in Vulkan SDK earlier than 1.2.131.
3050	// See pull request #207.
3051	#define VK_ERROR_UNKNOWN_COPY ((VkResult)-13)
3052
3053
3054	#if VMA_STATS_STRING_ENABLED
3055	// Correspond to values of enum VmaSuballocationType.
3056	static const char* VMA_SUBALLOCATION_TYPE_NAMES[] =
3057	{
3058	"FREE",
3059	"UNKNOWN",
3060	"BUFFER",
3061	"IMAGE_UNKNOWN",
3062	"IMAGE_LINEAR",
3063	"IMAGE_OPTIMAL",
3064	};
3065	#endif
3066
3067	static VkAllocationCallbacks VmaEmptyAllocationCallbacks =
3068	{ VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL, VMA_NULL };
3069
3070
3071	#ifndef _VMA_ENUM_DECLARATIONS
3072
3073	enum VmaSuballocationType
3074	{
3075	VMA_SUBALLOCATION_TYPE_FREE = 0,
3076	VMA_SUBALLOCATION_TYPE_UNKNOWN = 1,
3077	VMA_SUBALLOCATION_TYPE_BUFFER = 2,
3078	VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN = 3,
3079	VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR = 4,
3080	VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL = 5,
3081	VMA_SUBALLOCATION_TYPE_MAX_ENUM = 0x7FFFFFFF
3082	};
3083
3084	enum VMA_CACHE_OPERATION
3085	{
3086	VMA_CACHE_FLUSH,
3087	VMA_CACHE_INVALIDATE
3088	};
3089
3090	enum class VmaAllocationRequestType
3091	{
3092	Normal,
3093	TLSF,
3094	// Used by "Linear" algorithm.
3095	UpperAddress,
3096	EndOf1st,
3097	EndOf2nd,
3098	};
3099
3100	#endif // _VMA_ENUM_DECLARATIONS
3101
3102	#ifndef _VMA_FORWARD_DECLARATIONS
3103	// Opaque handle used by allocation algorithms to identify single allocation in any conforming way.
3104	VK_DEFINE_NON_DISPATCHABLE_HANDLE(VmaAllocHandle);
3105
3106	struct VmaMutexLock;
3107	struct VmaMutexLockRead;
3108	struct VmaMutexLockWrite;
3109
3110	template<typename T>
3111	struct AtomicTransactionalIncrement;
3112
3113	template<typename T>
3114	struct VmaStlAllocator;
3115
3116	template<typename T, typename AllocatorT>
3117	class VmaVector;
3118
3119	template<typename T, typename AllocatorT, size_t N>
3120	class VmaSmallVector;
3121
3122	template<typename T>
3123	class VmaPoolAllocator;
3124
3125	template<typename T>
3126	struct VmaListItem;
3127
3128	template<typename T>
3129	class VmaRawList;
3130
3131	template<typename T, typename AllocatorT>
3132	class VmaList;
3133
3134	template<typename ItemTypeTraits>
3135	class VmaIntrusiveLinkedList;
3136
3137	// Unused in this version
3138	#if 0
3139	template<typename T1, typename T2>
3140	struct VmaPair;
3141	template<typename FirstT, typename SecondT>
3142	struct VmaPairFirstLess;
3143
3144	template<typename KeyT, typename ValueT>
3145	class VmaMap;
3146	#endif
3147
3148	#if VMA_STATS_STRING_ENABLED
3149	class VmaStringBuilder;
3150	class VmaJsonWriter;
3151	#endif
3152
3153	class VmaDeviceMemoryBlock;
3154
3155	struct VmaDedicatedAllocationListItemTraits;
3156	class VmaDedicatedAllocationList;
3157
3158	struct VmaSuballocation;
3159	struct VmaSuballocationOffsetLess;
3160	struct VmaSuballocationOffsetGreater;
3161	struct VmaSuballocationItemSizeLess;
3162
3163	typedef VmaList<VmaSuballocation, VmaStlAllocator<VmaSuballocation>> VmaSuballocationList;
3164
3165	struct VmaAllocationRequest;
3166
3167	class VmaBlockMetadata;
3168	class VmaBlockMetadata_Linear;
3169	class VmaBlockMetadata_TLSF;
3170
3171	class VmaBlockVector;
3172
3173	struct VmaPoolListItemTraits;
3174
3175	struct VmaCurrentBudgetData;
3176
3177	class VmaAllocationObjectAllocator;
3178
3179	#endif // _VMA_FORWARD_DECLARATIONS
3180
3181
3182	#ifndef _VMA_FUNCTIONS
3183
3184	/*
3185	Returns number of bits set to 1 in (v).
3186
3187	On specific platforms and compilers you can use instrinsics like:
3188
3189	Visual Studio:
3190	return __popcnt(v);
3191	GCC, Clang:
3192	return static_cast<uint32_t>(__builtin_popcount(v));
3193
3194	Define macro VMA_COUNT_BITS_SET to provide your optimized implementation.
3195	But you need to check in runtime whether user's CPU supports these, as some old processors don't.
3196	*/
3197	static inline uint32_t VmaCountBitsSet(uint32_t v)
3198	{
3199	#if __cplusplus >= 202002L || _MSVC_LANG >= 202002L // C++20
3200	return std::popcount(v);
3201	#else
3202	uint32_t c = v - ((v >> 1) & 0x55555555);
3203	c = ((c >> 2) & 0x33333333) + (c & 0x33333333);
3204	c = ((c >> 4) + c) & 0x0F0F0F0F;
3205	c = ((c >> 8) + c) & 0x00FF00FF;
3206	c = ((c >> 16) + c) & 0x0000FFFF;
3207	return c;
3208	#endif
3209	}
3210
3211	static inline uint8_t VmaBitScanLSB(uint64_t mask)
3212	{
3213	#if defined(_MSC_VER) && defined(_WIN64)
3214	unsigned long pos;
3215	if (_BitScanForward64(&pos, mask))
3216	return static_cast<uint8_t>(pos);
3217	return UINT8_MAX;
3218	#elif defined __GNUC__ || defined __clang__
3219	return static_cast<uint8_t>(__builtin_ffsll(mask)) - 1U;
3220	#else
3221	uint8_t pos = 0;
3222	uint64_t bit = 1;
3223	do
3224	{
3225	if (mask & bit)
3226	return pos;
3227	bit <<= 1;
3228	} while (pos++ < 63);
3229	return UINT8_MAX;
3230	#endif
3231	}
3232
3233	static inline uint8_t VmaBitScanLSB(uint32_t mask)
3234	{
3235	#ifdef _MSC_VER
3236	unsigned long pos;
3237	if (_BitScanForward(&pos, mask))
3238	return static_cast<uint8_t>(pos);
3239	return UINT8_MAX;
3240	#elif defined __GNUC__ || defined __clang__
3241	return static_cast<uint8_t>(__builtin_ffs(mask)) - 1U;
3242	#else
3243	uint8_t pos = 0;
3244	uint32_t bit = 1;
3245	do
3246	{
3247	if (mask & bit)
3248	return pos;
3249	bit <<= 1;
3250	} while (pos++ < 31);
3251	return UINT8_MAX;
3252	#endif
3253	}
3254
3255	static inline uint8_t VmaBitScanMSB(uint64_t mask)
3256	{
3257	#if defined(_MSC_VER) && defined(_WIN64)
3258	unsigned long pos;
3259	if (_BitScanReverse64(&pos, mask))
3260	return static_cast<uint8_t>(pos);
3261	#elif defined __GNUC__ || defined __clang__
3262	if (mask)
3263	return 63 - static_cast<uint8_t>(__builtin_clzll(mask));
3264	#else
3265	uint8_t pos = 63;
3266	uint64_t bit = 1ULL << 63;
3267	do
3268	{
3269	if (mask & bit)
3270	return pos;
3271	bit >>= 1;
3272	} while (pos-- > 0);
3273	#endif
3274	return UINT8_MAX;
3275	}
3276
3277	static inline uint8_t VmaBitScanMSB(uint32_t mask)
3278	{
3279	#ifdef _MSC_VER
3280	unsigned long pos;
3281	if (_BitScanReverse(&pos, mask))
3282	return static_cast<uint8_t>(pos);
3283	#elif defined __GNUC__ || defined __clang__
3284	if (mask)
3285	return 31 - static_cast<uint8_t>(__builtin_clz(mask));
3286	#else
3287	uint8_t pos = 31;
3288	uint32_t bit = 1UL << 31;
3289	do
3290	{
3291	if (mask & bit)
3292	return pos;
3293	bit >>= 1;
3294	} while (pos-- > 0);
3295	#endif
3296	return UINT8_MAX;
3297	}
3298
3299	/*
3300	Returns true if given number is a power of two.
3301	T must be unsigned integer number or signed integer but always nonnegative.
3302	For 0 returns true.
3303	*/
3304	template <typename T>
3305	inline bool VmaIsPow2(T x)
3306	{
3307	return (x & (x - 1)) == 0;
3308	}
3309
3310	// Aligns given value up to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 16.
3311	// Use types like uint32_t, uint64_t as T.
3312	template <typename T>
3313	static inline T VmaAlignUp(T val, T alignment)
3314	{
3315	VMA_HEAVY_ASSERT(VmaIsPow2(alignment));
3316	return (val + alignment - 1) & ~(alignment - 1);
3317	}
3318
3319	// Aligns given value down to nearest multiply of align value. For example: VmaAlignUp(11, 8) = 8.
3320	// Use types like uint32_t, uint64_t as T.
3321	template <typename T>
3322	static inline T VmaAlignDown(T val, T alignment)
3323	{
3324	VMA_HEAVY_ASSERT(VmaIsPow2(alignment));
3325	return val & ~(alignment - 1);
3326	}
3327
3328	// Division with mathematical rounding to nearest number.
3329	template <typename T>
3330	static inline T VmaRoundDiv(T x, T y)
3331	{
3332	return (x + (y / (T)2)) / y;
3333	}
3334
3335	// Divide by 'y' and round up to nearest integer.
3336	template <typename T>
3337	static inline T VmaDivideRoundingUp(T x, T y)
3338	{
3339	return (x + y - (T)1) / y;
3340	}
3341
3342	// Returns smallest power of 2 greater or equal to v.
3343	static inline uint32_t VmaNextPow2(uint32_t v)
3344	{
3345	v--;
3346	v |= v >> 1;
3347	v |= v >> 2;
3348	v |= v >> 4;
3349	v |= v >> 8;
3350	v |= v >> 16;
3351	v++;
3352	return v;
3353	}
3354
3355	static inline uint64_t VmaNextPow2(uint64_t v)
3356	{
3357	v--;
3358	v |= v >> 1;
3359	v |= v >> 2;
3360	v |= v >> 4;
3361	v |= v >> 8;
3362	v |= v >> 16;
3363	v |= v >> 32;
3364	v++;
3365	return v;
3366	}
3367
3368	// Returns largest power of 2 less or equal to v.
3369	static inline uint32_t VmaPrevPow2(uint32_t v)
3370	{
3371	v |= v >> 1;
3372	v |= v >> 2;
3373	v |= v >> 4;
3374	v |= v >> 8;
3375	v |= v >> 16;
3376	v = v ^ (v >> 1);
3377	return v;
3378	}
3379
3380	static inline uint64_t VmaPrevPow2(uint64_t v)
3381	{
3382	v |= v >> 1;
3383	v |= v >> 2;
3384	v |= v >> 4;
3385	v |= v >> 8;
3386	v |= v >> 16;
3387	v |= v >> 32;
3388	v = v ^ (v >> 1);
3389	return v;
3390	}
3391
3392	static inline bool VmaStrIsEmpty(const char* pStr)
3393	{
3394	return pStr == VMA_NULL || *pStr == '\0';
3395	}
3396
3397	/*
3398	Returns true if two memory blocks occupy overlapping pages.
3399	ResourceA must be in less memory offset than ResourceB.
3400
3401	Algorithm is based on "Vulkan 1.0.39 - A Specification (with all registered Vulkan extensions)"
3402	chapter 11.6 "Resource Memory Association", paragraph "Buffer-Image Granularity".
3403	*/
3404	static inline bool VmaBlocksOnSamePage(
3405	VkDeviceSize resourceAOffset,
3406	VkDeviceSize resourceASize,
3407	VkDeviceSize resourceBOffset,
3408	VkDeviceSize pageSize)
3409	{
3410	VMA_ASSERT(resourceAOffset + resourceASize <= resourceBOffset && resourceASize > 0 && pageSize > 0);
3411	VkDeviceSize resourceAEnd = resourceAOffset + resourceASize - 1;
3412	VkDeviceSize resourceAEndPage = resourceAEnd & ~(pageSize - 1);
3413	VkDeviceSize resourceBStart = resourceBOffset;
3414	VkDeviceSize resourceBStartPage = resourceBStart & ~(pageSize - 1);
3415	return resourceAEndPage == resourceBStartPage;
3416	}
3417
3418	/*
3419	Returns true if given suballocation types could conflict and must respect
3420	VkPhysicalDeviceLimits::bufferImageGranularity. They conflict if one is buffer
3421	or linear image and another one is optimal image. If type is unknown, behave
3422	conservatively.
3423	*/
3424	static inline bool VmaIsBufferImageGranularityConflict(
3425	VmaSuballocationType suballocType1,
3426	VmaSuballocationType suballocType2)
3427	{
3428	if (suballocType1 > suballocType2)
3429	{
3430	VMA_SWAP(suballocType1, suballocType2);
3431	}
3432
3433	switch (suballocType1)
3434	{
3435	case VMA_SUBALLOCATION_TYPE_FREE:
3436	return false;
3437	case VMA_SUBALLOCATION_TYPE_UNKNOWN:
3438	return true;
3439	case VMA_SUBALLOCATION_TYPE_BUFFER:
3440	return
3441	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
3442	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3443	case VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN:
3444	return
3445	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
3446	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR ||
3447	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3448	case VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR:
3449	return
3450	suballocType2 == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL;
3451	case VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL:
3452	return false;
3453	default:
3454	VMA_ASSERT(0);
3455	return true;
3456	}
3457	}
3458
3459	static void VmaWriteMagicValue(void* pData, VkDeviceSize offset)
3460	{
3461	#if VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_DETECT_CORRUPTION
3462	uint32_t* pDst = (uint32_t*)((char*)pData + offset);
3463	const size_t numberCount = VMA_DEBUG_MARGIN / sizeof(uint32_t);
3464	for (size_t i = 0; i < numberCount; ++i, ++pDst)
3465	{
3466	*pDst = VMA_CORRUPTION_DETECTION_MAGIC_VALUE;
3467	}
3468	#else
3469	// no-op
3470	#endif
3471	}
3472
3473	static bool VmaValidateMagicValue(const void* pData, VkDeviceSize offset)
3474	{
3475	#if VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_DETECT_CORRUPTION
3476	const uint32_t* pSrc = (const uint32_t*)((const char*)pData + offset);
3477	const size_t numberCount = VMA_DEBUG_MARGIN / sizeof(uint32_t);
3478	for (size_t i = 0; i < numberCount; ++i, ++pSrc)
3479	{
3480	if (*pSrc != VMA_CORRUPTION_DETECTION_MAGIC_VALUE)
3481	{
3482	return false;
3483	}
3484	}
3485	#endif
3486	return true;
3487	}
3488
3489	/*
3490	Fills structure with parameters of an example buffer to be used for transfers
3491	during GPU memory defragmentation.
3492	*/
3493	static void VmaFillGpuDefragmentationBufferCreateInfo(VkBufferCreateInfo& outBufCreateInfo)
3494	{
3495	memset(s: &outBufCreateInfo, c: 0, n: sizeof(outBufCreateInfo));
3496	outBufCreateInfo.sType = VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO;
3497	outBufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
3498	outBufCreateInfo.size = (VkDeviceSize)VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE; // Example size.
3499	}
3500
3501
3502	/*
3503	Performs binary search and returns iterator to first element that is greater or
3504	equal to (key), according to comparison (cmp).
3505
3506	Cmp should return true if first argument is less than second argument.
3507
3508	Returned value is the found element, if present in the collection or place where
3509	new element with value (key) should be inserted.
3510	*/
3511	template <typename CmpLess, typename IterT, typename KeyT>
3512	static IterT VmaBinaryFindFirstNotLess(IterT beg, IterT end, const KeyT& key, const CmpLess& cmp)
3513	{
3514	size_t down = 0, up = (end - beg);
3515	while (down < up)
3516	{
3517	const size_t mid = down + (up - down) / 2; // Overflow-safe midpoint calculation
3518	if (cmp(*(beg + mid), key))
3519	{
3520	down = mid + 1;
3521	}
3522	else
3523	{
3524	up = mid;
3525	}
3526	}
3527	return beg + down;
3528	}
3529
3530	template<typename CmpLess, typename IterT, typename KeyT>
3531	IterT VmaBinaryFindSorted(const IterT& beg, const IterT& end, const KeyT& value, const CmpLess& cmp)
3532	{
3533	IterT it = VmaBinaryFindFirstNotLess<CmpLess, IterT, KeyT>(
3534	beg, end, value, cmp);
3535	if (it == end ||
3536	(!cmp(*it, value) && !cmp(value, *it)))
3537	{
3538	return it;
3539	}
3540	return end;
3541	}
3542
3543	/*
3544	Returns true if all pointers in the array are not-null and unique.
3545	Warning! O(n^2) complexity. Use only inside VMA_HEAVY_ASSERT.
3546	T must be pointer type, e.g. VmaAllocation, VmaPool.
3547	*/
3548	template<typename T>
3549	static bool VmaValidatePointerArray(uint32_t count, const T* arr)
3550	{
3551	for (uint32_t i = 0; i < count; ++i)
3552	{
3553	const T iPtr = arr[i];
3554	if (iPtr == VMA_NULL)
3555	{
3556	return false;
3557	}
3558	for (uint32_t j = i + 1; j < count; ++j)
3559	{
3560	if (iPtr == arr[j])
3561	{
3562	return false;
3563	}
3564	}
3565	}
3566	return true;
3567	}
3568
3569	template<typename MainT, typename NewT>
3570	static inline void VmaPnextChainPushFront(MainT* mainStruct, NewT* newStruct)
3571	{
3572	newStruct->pNext = mainStruct->pNext;
3573	mainStruct->pNext = newStruct;
3574	}
3575
3576	// This is the main algorithm that guides the selection of a memory type best for an allocation -
3577	// converts usage to required/preferred/not preferred flags.
3578	static bool FindMemoryPreferences(
3579	bool isIntegratedGPU,
3580	const VmaAllocationCreateInfo& allocCreateInfo,
3581	VkFlags bufImgUsage, // VkBufferCreateInfo::usage or VkImageCreateInfo::usage. UINT32_MAX if unknown.
3582	VkMemoryPropertyFlags& outRequiredFlags,
3583	VkMemoryPropertyFlags& outPreferredFlags,
3584	VkMemoryPropertyFlags& outNotPreferredFlags)
3585	{
3586	outRequiredFlags = allocCreateInfo.requiredFlags;
3587	outPreferredFlags = allocCreateInfo.preferredFlags;
3588	outNotPreferredFlags = 0;
3589
3590	switch(allocCreateInfo.usage)
3591	{
3592	case VMA_MEMORY_USAGE_UNKNOWN:
3593	break;
3594	case VMA_MEMORY_USAGE_GPU_ONLY:
3595	if(!isIntegratedGPU || (outPreferredFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
3596	{
3597	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3598	}
3599	break;
3600	case VMA_MEMORY_USAGE_CPU_ONLY:
3601	outRequiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
3602	break;
3603	case VMA_MEMORY_USAGE_CPU_TO_GPU:
3604	outRequiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3605	if(!isIntegratedGPU || (outPreferredFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
3606	{
3607	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3608	}
3609	break;
3610	case VMA_MEMORY_USAGE_GPU_TO_CPU:
3611	outRequiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3612	outPreferredFlags |= VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3613	break;
3614	case VMA_MEMORY_USAGE_CPU_COPY:
3615	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3616	break;
3617	case VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED:
3618	outRequiredFlags |= VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT;
3619	break;
3620	case VMA_MEMORY_USAGE_AUTO:
3621	case VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE:
3622	case VMA_MEMORY_USAGE_AUTO_PREFER_HOST:
3623	{
3624	if(bufImgUsage == UINT32_MAX)
3625	{
3626	VMA_ASSERT(0 && "VMA_MEMORY_USAGE_AUTO* values can only be used with functions like vmaCreateBuffer, vmaCreateImage so that the details of the created resource are known.");
3627	return false;
3628	}
3629	// This relies on values of VK_IMAGE_USAGE_TRANSFER* being the same VK_BUFFER_IMAGE_TRANSFER*.
3630	const bool deviceAccess = (bufImgUsage & ~(VK_BUFFER_USAGE_TRANSFER_DST_BIT | VK_BUFFER_USAGE_TRANSFER_SRC_BIT)) != 0;
3631	const bool hostAccessSequentialWrite = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT) != 0;
3632	const bool hostAccessRandom = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT) != 0;
3633	const bool hostAccessAllowTransferInstead = (allocCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT) != 0;
3634	const bool preferDevice = allocCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE;
3635	const bool preferHost = allocCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_HOST;
3636
3637	// CPU random access - e.g. a buffer written to or transferred from GPU to read back on CPU.
3638	if(hostAccessRandom)
3639	{
3640	if(!isIntegratedGPU && deviceAccess && hostAccessAllowTransferInstead && !preferHost)
3641	{
3642	// Nice if it will end up in HOST_VISIBLE, but more importantly prefer DEVICE_LOCAL.
3643	// Omitting HOST_VISIBLE here is intentional.
3644	// In case there is DEVICE_LOCAL | HOST_VISIBLE | HOST_CACHED, it will pick that one.
3645	// Otherwise, this will give same weight to DEVICE_LOCAL as HOST_VISIBLE | HOST_CACHED and select the former if occurs first on the list.
3646	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3647	}
3648	else
3649	{
3650	// Always CPU memory, cached.
3651	outRequiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3652	}
3653	}
3654	// CPU sequential write - may be CPU or host-visible GPU memory, uncached and write-combined.
3655	else if(hostAccessSequentialWrite)
3656	{
3657	// Want uncached and write-combined.
3658	outNotPreferredFlags |= VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
3659
3660	if(!isIntegratedGPU && deviceAccess && hostAccessAllowTransferInstead && !preferHost)
3661	{
3662	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT | VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3663	}
3664	else
3665	{
3666	outRequiredFlags |= VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
3667	// Direct GPU access, CPU sequential write (e.g. a dynamic uniform buffer updated every frame)
3668	if(deviceAccess)
3669	{
3670	// Could go to CPU memory or GPU BAR/unified. Up to the user to decide. If no preference, choose GPU memory.
3671	if(preferHost)
3672	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3673	else
3674	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3675	}
3676	// GPU no direct access, CPU sequential write (e.g. an upload buffer to be transferred to the GPU)
3677	else
3678	{
3679	// Could go to CPU memory or GPU BAR/unified. Up to the user to decide. If no preference, choose CPU memory.
3680	if(preferDevice)
3681	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3682	else
3683	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3684	}
3685	}
3686	}
3687	// No CPU access
3688	else
3689	{
3690	// GPU access, no CPU access (e.g. a color attachment image) - prefer GPU memory
3691	if(deviceAccess)
3692	{
3693	// ...unless there is a clear preference from the user not to do so.
3694	if(preferHost)
3695	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3696	else
3697	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3698	}
3699	// No direct GPU access, no CPU access, just transfers.
3700	// It may be staging copy intended for e.g. preserving image for next frame (then better GPU memory) or
3701	// a "swap file" copy to free some GPU memory (then better CPU memory).
3702	// Up to the user to decide. If no preferece, assume the former and choose GPU memory.
3703	if(preferHost)
3704	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3705	else
3706	outPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
3707	}
3708	break;
3709	}
3710	default:
3711	VMA_ASSERT(0);
3712	}
3713
3714	// Avoid DEVICE_COHERENT unless explicitly requested.
3715	if(((allocCreateInfo.requiredFlags | allocCreateInfo.preferredFlags) &
3716	(VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY | VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY)) == 0)
3717	{
3718	outNotPreferredFlags |= VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY;
3719	}
3720
3721	return true;
3722	}
3723
3724	////////////////////////////////////////////////////////////////////////////////
3725	// Memory allocation
3726
3727	static void* VmaMalloc(const VkAllocationCallbacks* pAllocationCallbacks, size_t size, size_t alignment)
3728	{
3729	void* result = VMA_NULL;
3730	if ((pAllocationCallbacks != VMA_NULL) &&
3731	(pAllocationCallbacks->pfnAllocation != VMA_NULL))
3732	{
3733	result = (*pAllocationCallbacks->pfnAllocation)(
3734	pAllocationCallbacks->pUserData,
3735	size,
3736	alignment,
3737	VK_SYSTEM_ALLOCATION_SCOPE_OBJECT);
3738	}
3739	else
3740	{
3741	result = VMA_SYSTEM_ALIGNED_MALLOC(size, alignment);
3742	}
3743	VMA_ASSERT(result != VMA_NULL && "CPU memory allocation failed.");
3744	return result;
3745	}
3746
3747	static void VmaFree(const VkAllocationCallbacks* pAllocationCallbacks, void* ptr)
3748	{
3749	if ((pAllocationCallbacks != VMA_NULL) &&
3750	(pAllocationCallbacks->pfnFree != VMA_NULL))
3751	{
3752	(*pAllocationCallbacks->pfnFree)(pAllocationCallbacks->pUserData, ptr);
3753	}
3754	else
3755	{
3756	VMA_SYSTEM_ALIGNED_FREE(ptr);
3757	}
3758	}
3759
3760	template<typename T>
3761	static T* VmaAllocate(const VkAllocationCallbacks* pAllocationCallbacks)
3762	{
3763	return (T*)VmaMalloc(pAllocationCallbacks, size: sizeof(T), VMA_ALIGN_OF(T));
3764	}
3765
3766	template<typename T>
3767	static T* VmaAllocateArray(const VkAllocationCallbacks* pAllocationCallbacks, size_t count)
3768	{
3769	return (T*)VmaMalloc(pAllocationCallbacks, size: sizeof(T) * count, VMA_ALIGN_OF(T));
3770	}
3771
3772	#define vma_new(allocator, type) new(VmaAllocate<type>(allocator))(type)
3773
3774	#define vma_new_array(allocator, type, count) new(VmaAllocateArray<type>((allocator), (count)))(type)
3775
3776	template<typename T>
3777	static void vma_delete(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr)
3778	{
3779	ptr->~T();
3780	VmaFree(pAllocationCallbacks, ptr);
3781	}
3782
3783	template<typename T>
3784	static void vma_delete_array(const VkAllocationCallbacks* pAllocationCallbacks, T* ptr, size_t count)
3785	{
3786	if (ptr != VMA_NULL)
3787	{
3788	for (size_t i = count; i--; )
3789	{
3790	ptr[i].~T();
3791	}
3792	VmaFree(pAllocationCallbacks, ptr);
3793	}
3794	}
3795
3796	static char* VmaCreateStringCopy(const VkAllocationCallbacks* allocs, const char* srcStr)
3797	{
3798	if (srcStr != VMA_NULL)
3799	{
3800	const size_t len = strlen(s: srcStr);
3801	char* const result = vma_new_array(allocs, char, len + 1);
3802	memcpy(dest: result, src: srcStr, n: len + 1);
3803	return result;
3804	}
3805	return VMA_NULL;
3806	}
3807
3808	#if VMA_STATS_STRING_ENABLED
3809	static char* VmaCreateStringCopy(const VkAllocationCallbacks* allocs, const char* srcStr, size_t strLen)
3810	{
3811	if (srcStr != VMA_NULL)
3812	{
3813	char* const result = vma_new_array(allocs, char, strLen + 1);
3814	memcpy(dest: result, src: srcStr, n: strLen);
3815	result[strLen] = '\0';
3816	return result;
3817	}
3818	return VMA_NULL;
3819	}
3820	#endif // VMA_STATS_STRING_ENABLED
3821
3822	static void VmaFreeString(const VkAllocationCallbacks* allocs, char* str)
3823	{
3824	if (str != VMA_NULL)
3825	{
3826	const size_t len = strlen(s: str);
3827	vma_delete_array(pAllocationCallbacks: allocs, ptr: str, count: len + 1);
3828	}
3829	}
3830
3831	template<typename CmpLess, typename VectorT>
3832	size_t VmaVectorInsertSorted(VectorT& vector, const typename VectorT::value_type& value)
3833	{
3834	const size_t indexToInsert = VmaBinaryFindFirstNotLess(
3835	vector.data(),
3836	vector.data() + vector.size(),
3837	value,
3838	CmpLess()) - vector.data();
3839	VmaVectorInsert(vector, indexToInsert, value);
3840	return indexToInsert;
3841	}
3842
3843	template<typename CmpLess, typename VectorT>
3844	bool VmaVectorRemoveSorted(VectorT& vector, const typename VectorT::value_type& value)
3845	{
3846	CmpLess comparator;
3847	typename VectorT::iterator it = VmaBinaryFindFirstNotLess(
3848	vector.begin(),
3849	vector.end(),
3850	value,
3851	comparator);
3852	if ((it != vector.end()) && !comparator(*it, value) && !comparator(value, *it))
3853	{
3854	size_t indexToRemove = it - vector.begin();
3855	VmaVectorRemove(vector, indexToRemove);
3856	return true;
3857	}
3858	return false;
3859	}
3860	#endif // _VMA_FUNCTIONS
3861
3862	#ifndef _VMA_STATISTICS_FUNCTIONS
3863
3864	static void VmaClearStatistics(VmaStatistics& outStats)
3865	{
3866	outStats.blockCount = 0;
3867	outStats.allocationCount = 0;
3868	outStats.blockBytes = 0;
3869	outStats.allocationBytes = 0;
3870	}
3871
3872	static void VmaAddStatistics(VmaStatistics& inoutStats, const VmaStatistics& src)
3873	{
3874	inoutStats.blockCount += src.blockCount;
3875	inoutStats.allocationCount += src.allocationCount;
3876	inoutStats.blockBytes += src.blockBytes;
3877	inoutStats.allocationBytes += src.allocationBytes;
3878	}
3879
3880	static void VmaClearDetailedStatistics(VmaDetailedStatistics& outStats)
3881	{
3882	VmaClearStatistics(outStats&: outStats.statistics);
3883	outStats.unusedRangeCount = 0;
3884	outStats.allocationSizeMin = VK_WHOLE_SIZE;
3885	outStats.allocationSizeMax = 0;
3886	outStats.unusedRangeSizeMin = VK_WHOLE_SIZE;
3887	outStats.unusedRangeSizeMax = 0;
3888	}
3889
3890	static void VmaAddDetailedStatisticsAllocation(VmaDetailedStatistics& inoutStats, VkDeviceSize size)
3891	{
3892	inoutStats.statistics.allocationCount++;
3893	inoutStats.statistics.allocationBytes += size;
3894	inoutStats.allocationSizeMin = VMA_MIN(inoutStats.allocationSizeMin, size);
3895	inoutStats.allocationSizeMax = VMA_MAX(inoutStats.allocationSizeMax, size);
3896	}
3897
3898	static void VmaAddDetailedStatisticsUnusedRange(VmaDetailedStatistics& inoutStats, VkDeviceSize size)
3899	{
3900	inoutStats.unusedRangeCount++;
3901	inoutStats.unusedRangeSizeMin = VMA_MIN(inoutStats.unusedRangeSizeMin, size);
3902	inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, size);
3903	}
3904
3905	static void VmaAddDetailedStatistics(VmaDetailedStatistics& inoutStats, const VmaDetailedStatistics& src)
3906	{
3907	VmaAddStatistics(inoutStats&: inoutStats.statistics, src: src.statistics);
3908	inoutStats.unusedRangeCount += src.unusedRangeCount;
3909	inoutStats.allocationSizeMin = VMA_MIN(inoutStats.allocationSizeMin, src.allocationSizeMin);
3910	inoutStats.allocationSizeMax = VMA_MAX(inoutStats.allocationSizeMax, src.allocationSizeMax);
3911	inoutStats.unusedRangeSizeMin = VMA_MIN(inoutStats.unusedRangeSizeMin, src.unusedRangeSizeMin);
3912	inoutStats.unusedRangeSizeMax = VMA_MAX(inoutStats.unusedRangeSizeMax, src.unusedRangeSizeMax);
3913	}
3914
3915	#endif // _VMA_STATISTICS_FUNCTIONS
3916
3917	#ifndef _VMA_MUTEX_LOCK
3918	// Helper RAII class to lock a mutex in constructor and unlock it in destructor (at the end of scope).
3919	struct VmaMutexLock
3920	{
3921	VMA_CLASS_NO_COPY(VmaMutexLock)
3922	public:
3923	VmaMutexLock(VMA_MUTEX& mutex, bool useMutex = true) :
3924	m_pMutex(useMutex ? &mutex : VMA_NULL)
3925	{
3926	if (m_pMutex) { m_pMutex->Lock(); }
3927	}
3928	~VmaMutexLock() { if (m_pMutex) { m_pMutex->Unlock(); } }
3929
3930	private:
3931	VMA_MUTEX* m_pMutex;
3932	};
3933
3934	// Helper RAII class to lock a RW mutex in constructor and unlock it in destructor (at the end of scope), for reading.
3935	struct VmaMutexLockRead
3936	{
3937	VMA_CLASS_NO_COPY(VmaMutexLockRead)
3938	public:
3939	VmaMutexLockRead(VMA_RW_MUTEX& mutex, bool useMutex) :
3940	m_pMutex(useMutex ? &mutex : VMA_NULL)
3941	{
3942	if (m_pMutex) { m_pMutex->LockRead(); }
3943	}
3944	~VmaMutexLockRead() { if (m_pMutex) { m_pMutex->UnlockRead(); } }
3945
3946	private:
3947	VMA_RW_MUTEX* m_pMutex;
3948	};
3949
3950	// Helper RAII class to lock a RW mutex in constructor and unlock it in destructor (at the end of scope), for writing.
3951	struct VmaMutexLockWrite
3952	{
3953	VMA_CLASS_NO_COPY(VmaMutexLockWrite)
3954	public:
3955	VmaMutexLockWrite(VMA_RW_MUTEX& mutex, bool useMutex)
3956	: m_pMutex(useMutex ? &mutex : VMA_NULL)
3957	{
3958	if (m_pMutex) { m_pMutex->LockWrite(); }
3959	}
3960	~VmaMutexLockWrite() { if (m_pMutex) { m_pMutex->UnlockWrite(); } }
3961
3962	private:
3963	VMA_RW_MUTEX* m_pMutex;
3964	};
3965
3966	#if VMA_DEBUG_GLOBAL_MUTEX
3967	static VMA_MUTEX gDebugGlobalMutex;
3968	#define VMA_DEBUG_GLOBAL_MUTEX_LOCK VmaMutexLock debugGlobalMutexLock(gDebugGlobalMutex, true);
3969	#else
3970	#define VMA_DEBUG_GLOBAL_MUTEX_LOCK
3971	#endif
3972	#endif // _VMA_MUTEX_LOCK
3973
3974	#ifndef _VMA_ATOMIC_TRANSACTIONAL_INCREMENT
3975	// An object that increments given atomic but decrements it back in the destructor unless Commit() is called.
3976	template<typename T>
3977	struct AtomicTransactionalIncrement
3978	{
3979	public:
3980	typedef std::atomic<T> AtomicT;
3981
3982	~AtomicTransactionalIncrement()
3983	{
3984	if(m_Atomic)
3985	--(*m_Atomic);
3986	}
3987
3988	void Commit() { m_Atomic = nullptr; }
3989	T Increment(AtomicT* atomic)
3990	{
3991	m_Atomic = atomic;
3992	return m_Atomic->fetch_add(1);
3993	}
3994
3995	private:
3996	AtomicT* m_Atomic = nullptr;
3997	};
3998	#endif // _VMA_ATOMIC_TRANSACTIONAL_INCREMENT
3999
4000	#ifndef _VMA_STL_ALLOCATOR
4001	// STL-compatible allocator.
4002	template<typename T>
4003	struct VmaStlAllocator
4004	{
4005	const VkAllocationCallbacks* const m_pCallbacks;
4006	typedef T value_type;
4007
4008	VmaStlAllocator(const VkAllocationCallbacks* pCallbacks) : m_pCallbacks(pCallbacks) {}
4009	template<typename U>
4010	VmaStlAllocator(const VmaStlAllocator<U>& src) : m_pCallbacks(src.m_pCallbacks) {}
4011	VmaStlAllocator(const VmaStlAllocator&) = default;
4012	VmaStlAllocator& operator=(const VmaStlAllocator&) = delete;
4013
4014	T* allocate(size_t n) { return VmaAllocateArray<T>(m_pCallbacks, n); }
4015	void deallocate(T* p, size_t n) { VmaFree(m_pCallbacks, p); }
4016
4017	template<typename U>
4018	bool operator==(const VmaStlAllocator<U>& rhs) const
4019	{
4020	return m_pCallbacks == rhs.m_pCallbacks;
4021	}
4022	template<typename U>
4023	bool operator!=(const VmaStlAllocator<U>& rhs) const
4024	{
4025	return m_pCallbacks != rhs.m_pCallbacks;
4026	}
4027	};
4028	#endif // _VMA_STL_ALLOCATOR
4029
4030	#ifndef _VMA_VECTOR
4031	/* Class with interface compatible with subset of std::vector.
4032	T must be POD because constructors and destructors are not called and memcpy is
4033	used for these objects. */
4034	template<typename T, typename AllocatorT>
4035	class VmaVector
4036	{
4037	public:
4038	typedef T value_type;
4039	typedef T* iterator;
4040	typedef const T* const_iterator;
4041
4042	VmaVector(const AllocatorT& allocator);
4043	VmaVector(size_t count, const AllocatorT& allocator);
4044	// This version of the constructor is here for compatibility with pre-C++14 std::vector.
4045	// value is unused.
4046	VmaVector(size_t count, const T& value, const AllocatorT& allocator) : VmaVector(count, allocator) {}
4047	VmaVector(const VmaVector<T, AllocatorT>& src);
4048	VmaVector& operator=(const VmaVector& rhs);
4049	~VmaVector() { VmaFree(m_Allocator.m_pCallbacks, m_pArray); }
4050
4051	bool empty() const { return m_Count == 0; }
4052	size_t size() const { return m_Count; }
4053	T* data() { return m_pArray; }
4054	T& front() { VMA_HEAVY_ASSERT(m_Count > 0); return m_pArray[0]; }
4055	T& back() { VMA_HEAVY_ASSERT(m_Count > 0); return m_pArray[m_Count - 1]; }
4056	const T* data() const { return m_pArray; }
4057	const T& front() const { VMA_HEAVY_ASSERT(m_Count > 0); return m_pArray[0]; }
4058	const T& back() const { VMA_HEAVY_ASSERT(m_Count > 0); return m_pArray[m_Count - 1]; }
4059
4060	iterator begin() { return m_pArray; }
4061	iterator end() { return m_pArray + m_Count; }
4062	const_iterator cbegin() const { return m_pArray; }
4063	const_iterator cend() const { return m_pArray + m_Count; }
4064	const_iterator begin() const { return cbegin(); }
4065	const_iterator end() const { return cend(); }
4066
4067	void pop_front() { VMA_HEAVY_ASSERT(m_Count > 0); remove(index: 0); }
4068	void pop_back() { VMA_HEAVY_ASSERT(m_Count > 0); resize(newCount: size() - 1); }
4069	void push_front(const T& src) { insert(index: 0, src); }
4070
4071	void push_back(const T& src);
4072	void reserve(size_t newCapacity, bool freeMemory = false);
4073	void resize(size_t newCount);
4074	void clear() { resize(newCount: 0); }
4075	void shrink_to_fit();
4076	void insert(size_t index, const T& src);
4077	void remove(size_t index);
4078
4079	T& operator[](size_t index) { VMA_HEAVY_ASSERT(index < m_Count); return m_pArray[index]; }
4080	const T& operator[](size_t index) const { VMA_HEAVY_ASSERT(index < m_Count); return m_pArray[index]; }
4081
4082	private:
4083	AllocatorT m_Allocator;
4084	T* m_pArray;
4085	size_t m_Count;
4086	size_t m_Capacity;
4087	};
4088
4089	#ifndef _VMA_VECTOR_FUNCTIONS
4090	template<typename T, typename AllocatorT>
4091	VmaVector<T, AllocatorT>::VmaVector(const AllocatorT& allocator)
4092	: m_Allocator(allocator),
4093	m_pArray(VMA_NULL),
4094	m_Count(0),
4095	m_Capacity(0) {}
4096
4097	template<typename T, typename AllocatorT>
4098	VmaVector<T, AllocatorT>::VmaVector(size_t count, const AllocatorT& allocator)
4099	: m_Allocator(allocator),
4100	m_pArray(count ? (T*)VmaAllocateArray<T>(allocator.m_pCallbacks, count) : VMA_NULL),
4101	m_Count(count),
4102	m_Capacity(count) {}
4103
4104	template<typename T, typename AllocatorT>
4105	VmaVector<T, AllocatorT>::VmaVector(const VmaVector& src)
4106	: m_Allocator(src.m_Allocator),
4107	m_pArray(src.m_Count ? (T*)VmaAllocateArray<T>(src.m_Allocator.m_pCallbacks, src.m_Count) : VMA_NULL),
4108	m_Count(src.m_Count),
4109	m_Capacity(src.m_Count)
4110	{
4111	if (m_Count != 0)
4112	{
4113	memcpy(m_pArray, src.m_pArray, m_Count * sizeof(T));
4114	}
4115	}
4116
4117	template<typename T, typename AllocatorT>
4118	VmaVector<T, AllocatorT>& VmaVector<T, AllocatorT>::operator=(const VmaVector& rhs)
4119	{
4120	if (&rhs != this)
4121	{
4122	resize(newCount: rhs.m_Count);
4123	if (m_Count != 0)
4124	{
4125	memcpy(m_pArray, rhs.m_pArray, m_Count * sizeof(T));
4126	}
4127	}
4128	return *this;
4129	}
4130
4131	template<typename T, typename AllocatorT>
4132	void VmaVector<T, AllocatorT>::push_back(const T& src)
4133	{
4134	const size_t newIndex = size();
4135	resize(newCount: newIndex + 1);
4136	m_pArray[newIndex] = src;
4137	}
4138
4139	template<typename T, typename AllocatorT>
4140	void VmaVector<T, AllocatorT>::reserve(size_t newCapacity, bool freeMemory)
4141	{
4142	newCapacity = VMA_MAX(newCapacity, m_Count);
4143
4144	if ((newCapacity < m_Capacity) && !freeMemory)
4145	{
4146	newCapacity = m_Capacity;
4147	}
4148
4149	if (newCapacity != m_Capacity)
4150	{
4151	T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator, newCapacity) : VMA_NULL;
4152	if (m_Count != 0)
4153	{
4154	memcpy(newArray, m_pArray, m_Count * sizeof(T));
4155	}
4156	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4157	m_Capacity = newCapacity;
4158	m_pArray = newArray;
4159	}
4160	}
4161
4162	template<typename T, typename AllocatorT>
4163	void VmaVector<T, AllocatorT>::resize(size_t newCount)
4164	{
4165	size_t newCapacity = m_Capacity;
4166	if (newCount > m_Capacity)
4167	{
4168	newCapacity = VMA_MAX(newCount, VMA_MAX(m_Capacity * 3 / 2, (size_t)8));
4169	}
4170
4171	if (newCapacity != m_Capacity)
4172	{
4173	T* const newArray = newCapacity ? VmaAllocateArray<T>(m_Allocator.m_pCallbacks, newCapacity) : VMA_NULL;
4174	const size_t elementsToCopy = VMA_MIN(m_Count, newCount);
4175	if (elementsToCopy != 0)
4176	{
4177	memcpy(newArray, m_pArray, elementsToCopy * sizeof(T));
4178	}
4179	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4180	m_Capacity = newCapacity;
4181	m_pArray = newArray;
4182	}
4183
4184	m_Count = newCount;
4185	}
4186
4187	template<typename T, typename AllocatorT>
4188	void VmaVector<T, AllocatorT>::shrink_to_fit()
4189	{
4190	if (m_Capacity > m_Count)
4191	{
4192	T* newArray = VMA_NULL;
4193	if (m_Count > 0)
4194	{
4195	newArray = VmaAllocateArray<T>(m_Allocator.m_pCallbacks, m_Count);
4196	memcpy(newArray, m_pArray, m_Count * sizeof(T));
4197	}
4198	VmaFree(m_Allocator.m_pCallbacks, m_pArray);
4199	m_Capacity = m_Count;
4200	m_pArray = newArray;
4201	}
4202	}
4203
4204	template<typename T, typename AllocatorT>
4205	void VmaVector<T, AllocatorT>::insert(size_t index, const T& src)
4206	{
4207	VMA_HEAVY_ASSERT(index <= m_Count);
4208	const size_t oldCount = size();
4209	resize(newCount: oldCount + 1);
4210	if (index < oldCount)
4211	{
4212	memmove(m_pArray + (index + 1), m_pArray + index, (oldCount - index) * sizeof(T));
4213	}
4214	m_pArray[index] = src;
4215	}
4216
4217	template<typename T, typename AllocatorT>
4218	void VmaVector<T, AllocatorT>::remove(size_t index)
4219	{
4220	VMA_HEAVY_ASSERT(index < m_Count);
4221	const size_t oldCount = size();
4222	if (index < oldCount - 1)
4223	{
4224	memmove(m_pArray + index, m_pArray + (index + 1), (oldCount - index - 1) * sizeof(T));
4225	}
4226	resize(newCount: oldCount - 1);
4227	}
4228	#endif // _VMA_VECTOR_FUNCTIONS
4229
4230	template<typename T, typename allocatorT>
4231	static void VmaVectorInsert(VmaVector<T, allocatorT>& vec, size_t index, const T& item)
4232	{
4233	vec.insert(index, item);
4234	}
4235
4236	template<typename T, typename allocatorT>
4237	static void VmaVectorRemove(VmaVector<T, allocatorT>& vec, size_t index)
4238	{
4239	vec.remove(index);
4240	}
4241	#endif // _VMA_VECTOR
4242
4243	#ifndef _VMA_SMALL_VECTOR
4244	/*
4245	This is a vector (a variable-sized array), optimized for the case when the array is small.
4246
4247	It contains some number of elements in-place, which allows it to avoid heap allocation
4248	when the actual number of elements is below that threshold. This allows normal "small"
4249	cases to be fast without losing generality for large inputs.
4250	*/
4251	template<typename T, typename AllocatorT, size_t N>
4252	class VmaSmallVector
4253	{
4254	public:
4255	typedef T value_type;
4256	typedef T* iterator;
4257
4258	VmaSmallVector(const AllocatorT& allocator);
4259	VmaSmallVector(size_t count, const AllocatorT& allocator);
4260	template<typename SrcT, typename SrcAllocatorT, size_t SrcN>
4261	VmaSmallVector(const VmaSmallVector<SrcT, SrcAllocatorT, SrcN>&) = delete;
4262	template<typename SrcT, typename SrcAllocatorT, size_t SrcN>
4263	VmaSmallVector<T, AllocatorT, N>& operator=(const VmaSmallVector<SrcT, SrcAllocatorT, SrcN>&) = delete;
4264	~VmaSmallVector() = default;
4265
4266	bool empty() const { return m_Count == 0; }
4267	size_t size() const { return m_Count; }
4268	T* data() { return m_Count > N ? m_DynamicArray.data() : m_StaticArray; }
4269	T& front() { VMA_HEAVY_ASSERT(m_Count > 0); return data()[0]; }
4270	T& back() { VMA_HEAVY_ASSERT(m_Count > 0); return data()[m_Count - 1]; }
4271	const T* data() const { return m_Count > N ? m_DynamicArray.data() : m_StaticArray; }
4272	const T& front() const { VMA_HEAVY_ASSERT(m_Count > 0); return data()[0]; }
4273	const T& back() const { VMA_HEAVY_ASSERT(m_Count > 0); return data()[m_Count - 1]; }
4274
4275	iterator begin() { return data(); }
4276	iterator end() { return data() + m_Count; }
4277
4278	void pop_front() { VMA_HEAVY_ASSERT(m_Count > 0); remove(index: 0); }
4279	void pop_back() { VMA_HEAVY_ASSERT(m_Count > 0); resize(newCount: size() - 1); }
4280	void push_front(const T& src) { insert(index: 0, src); }
4281
4282	void push_back(const T& src);
4283	void resize(size_t newCount, bool freeMemory = false);
4284	void clear(bool freeMemory = false);
4285	void insert(size_t index, const T& src);
4286	void remove(size_t index);
4287
4288	T& operator[](size_t index) { VMA_HEAVY_ASSERT(index < m_Count); return data()[index]; }
4289	const T& operator[](size_t index) const { VMA_HEAVY_ASSERT(index < m_Count); return data()[index]; }
4290
4291	private:
4292	size_t m_Count;
4293	T m_StaticArray[N]; // Used when m_Size <= N
4294	VmaVector<T, AllocatorT> m_DynamicArray; // Used when m_Size > N
4295	};
4296
4297	#ifndef _VMA_SMALL_VECTOR_FUNCTIONS
4298	template<typename T, typename AllocatorT, size_t N>
4299	VmaSmallVector<T, AllocatorT, N>::VmaSmallVector(const AllocatorT& allocator)
4300	: m_Count(0),
4301	m_DynamicArray(allocator) {}
4302
4303	template<typename T, typename AllocatorT, size_t N>
4304	VmaSmallVector<T, AllocatorT, N>::VmaSmallVector(size_t count, const AllocatorT& allocator)
4305	: m_Count(count),
4306	m_DynamicArray(count > N ? count : 0, allocator) {}
4307
4308	template<typename T, typename AllocatorT, size_t N>
4309	void VmaSmallVector<T, AllocatorT, N>::push_back(const T& src)
4310	{
4311	const size_t newIndex = size();
4312	resize(newCount: newIndex + 1);
4313	data()[newIndex] = src;
4314	}
4315
4316	template<typename T, typename AllocatorT, size_t N>
4317	void VmaSmallVector<T, AllocatorT, N>::resize(size_t newCount, bool freeMemory)
4318	{
4319	if (newCount > N && m_Count > N)
4320	{
4321	// Any direction, staying in m_DynamicArray
4322	m_DynamicArray.resize(newCount);
4323	if (freeMemory)
4324	{
4325	m_DynamicArray.shrink_to_fit();
4326	}
4327	}
4328	else if (newCount > N && m_Count <= N)
4329	{
4330	// Growing, moving from m_StaticArray to m_DynamicArray
4331	m_DynamicArray.resize(newCount);
4332	if (m_Count > 0)
4333	{
4334	memcpy(m_DynamicArray.data(), m_StaticArray, m_Count * sizeof(T));
4335	}
4336	}
4337	else if (newCount <= N && m_Count > N)
4338	{
4339	// Shrinking, moving from m_DynamicArray to m_StaticArray
4340	if (newCount > 0)
4341	{
4342	memcpy(m_StaticArray, m_DynamicArray.data(), newCount * sizeof(T));
4343	}
4344	m_DynamicArray.resize(0);
4345	if (freeMemory)
4346	{
4347	m_DynamicArray.shrink_to_fit();
4348	}
4349	}
4350	else
4351	{
4352	// Any direction, staying in m_StaticArray - nothing to do here
4353	}
4354	m_Count = newCount;
4355	}
4356
4357	template<typename T, typename AllocatorT, size_t N>
4358	void VmaSmallVector<T, AllocatorT, N>::clear(bool freeMemory)
4359	{
4360	m_DynamicArray.clear();
4361	if (freeMemory)
4362	{
4363	m_DynamicArray.shrink_to_fit();
4364	}
4365	m_Count = 0;
4366	}
4367
4368	template<typename T, typename AllocatorT, size_t N>
4369	void VmaSmallVector<T, AllocatorT, N>::insert(size_t index, const T& src)
4370	{
4371	VMA_HEAVY_ASSERT(index <= m_Count);
4372	const size_t oldCount = size();
4373	resize(newCount: oldCount + 1);
4374	T* const dataPtr = data();
4375	if (index < oldCount)
4376	{
4377	// I know, this could be more optimal for case where memmove can be memcpy directly from m_StaticArray to m_DynamicArray.
4378	memmove(dataPtr + (index + 1), dataPtr + index, (oldCount - index) * sizeof(T));
4379	}
4380	dataPtr[index] = src;
4381	}
4382
4383	template<typename T, typename AllocatorT, size_t N>
4384	void VmaSmallVector<T, AllocatorT, N>::remove(size_t index)
4385	{
4386	VMA_HEAVY_ASSERT(index < m_Count);
4387	const size_t oldCount = size();
4388	if (index < oldCount - 1)
4389	{
4390	// I know, this could be more optimal for case where memmove can be memcpy directly from m_DynamicArray to m_StaticArray.
4391	T* const dataPtr = data();
4392	memmove(dataPtr + index, dataPtr + (index + 1), (oldCount - index - 1) * sizeof(T));
4393	}
4394	resize(newCount: oldCount - 1);
4395	}
4396	#endif // _VMA_SMALL_VECTOR_FUNCTIONS
4397	#endif // _VMA_SMALL_VECTOR
4398
4399	#ifndef _VMA_POOL_ALLOCATOR
4400	/*
4401	Allocator for objects of type T using a list of arrays (pools) to speed up
4402	allocation. Number of elements that can be allocated is not bounded because
4403	allocator can create multiple blocks.
4404	*/
4405	template<typename T>
4406	class VmaPoolAllocator
4407	{
4408	VMA_CLASS_NO_COPY(VmaPoolAllocator)
4409	public:
4410	VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, uint32_t firstBlockCapacity);
4411	~VmaPoolAllocator();
4412	template<typename... Types> T* Alloc(Types&&... args);
4413	void Free(T* ptr);
4414
4415	private:
4416	union Item
4417	{
4418	uint32_t NextFreeIndex;
4419	alignas(T) char Value[sizeof(T)];
4420	};
4421	struct ItemBlock
4422	{
4423	Item* pItems;
4424	uint32_t Capacity;
4425	uint32_t FirstFreeIndex;
4426	};
4427
4428	const VkAllocationCallbacks* m_pAllocationCallbacks;
4429	const uint32_t m_FirstBlockCapacity;
4430	VmaVector<ItemBlock, VmaStlAllocator<ItemBlock>> m_ItemBlocks;
4431
4432	ItemBlock& CreateNewBlock();
4433	};
4434
4435	#ifndef _VMA_POOL_ALLOCATOR_FUNCTIONS
4436	template<typename T>
4437	VmaPoolAllocator<T>::VmaPoolAllocator(const VkAllocationCallbacks* pAllocationCallbacks, uint32_t firstBlockCapacity)
4438	: m_pAllocationCallbacks(pAllocationCallbacks),
4439	m_FirstBlockCapacity(firstBlockCapacity),
4440	m_ItemBlocks(VmaStlAllocator<ItemBlock>(pAllocationCallbacks))
4441	{
4442	VMA_ASSERT(m_FirstBlockCapacity > 1);
4443	}
4444
4445	template<typename T>
4446	VmaPoolAllocator<T>::~VmaPoolAllocator()
4447	{
4448	for (size_t i = m_ItemBlocks.size(); i--;)
4449	vma_delete_array(m_pAllocationCallbacks, m_ItemBlocks[i].pItems, m_ItemBlocks[i].Capacity);
4450	m_ItemBlocks.clear();
4451	}
4452
4453	template<typename T>
4454	template<typename... Types> T* VmaPoolAllocator<T>::Alloc(Types&&... args)
4455	{
4456	for (size_t i = m_ItemBlocks.size(); i--; )
4457	{
4458	ItemBlock& block = m_ItemBlocks[i];
4459	// This block has some free items: Use first one.
4460	if (block.FirstFreeIndex != UINT32_MAX)
4461	{
4462	Item* const pItem = &block.pItems[block.FirstFreeIndex];
4463	block.FirstFreeIndex = pItem->NextFreeIndex;
4464	T* result = (T*)&pItem->Value;
4465	new(result)T(std::forward<Types>(args)...); // Explicit constructor call.
4466	return result;
4467	}
4468	}
4469
4470	// No block has free item: Create new one and use it.
4471	ItemBlock& newBlock = CreateNewBlock();
4472	Item* const pItem = &newBlock.pItems[0];
4473	newBlock.FirstFreeIndex = pItem->NextFreeIndex;
4474	T* result = (T*)&pItem->Value;
4475	new(result) T(std::forward<Types>(args)...); // Explicit constructor call.
4476	return result;
4477	}
4478
4479	template<typename T>
4480	void VmaPoolAllocator<T>::Free(T* ptr)
4481	{
4482	// Search all memory blocks to find ptr.
4483	for (size_t i = m_ItemBlocks.size(); i--; )
4484	{
4485	ItemBlock& block = m_ItemBlocks[i];
4486
4487	// Casting to union.
4488	Item* pItemPtr;
4489	memcpy(&pItemPtr, &ptr, sizeof(pItemPtr));
4490
4491	// Check if pItemPtr is in address range of this block.
4492	if ((pItemPtr >= block.pItems) && (pItemPtr < block.pItems + block.Capacity))
4493	{
4494	ptr->~T(); // Explicit destructor call.
4495	const uint32_t index = static_cast<uint32_t>(pItemPtr - block.pItems);
4496	pItemPtr->NextFreeIndex = block.FirstFreeIndex;
4497	block.FirstFreeIndex = index;
4498	return;
4499	}
4500	}
4501	VMA_ASSERT(0 && "Pointer doesn't belong to this memory pool.");
4502	}
4503
4504	template<typename T>
4505	typename VmaPoolAllocator<T>::ItemBlock& VmaPoolAllocator<T>::CreateNewBlock()
4506	{
4507	const uint32_t newBlockCapacity = m_ItemBlocks.empty() ?
4508	m_FirstBlockCapacity : m_ItemBlocks.back().Capacity * 3 / 2;
4509
4510	const ItemBlock newBlock =
4511	{
4512	vma_new_array(m_pAllocationCallbacks, Item, newBlockCapacity),
4513	newBlockCapacity,
4514	0
4515	};
4516
4517	m_ItemBlocks.push_back(newBlock);
4518
4519	// Setup singly-linked list of all free items in this block.
4520	for (uint32_t i = 0; i < newBlockCapacity - 1; ++i)
4521	newBlock.pItems[i].NextFreeIndex = i + 1;
4522	newBlock.pItems[newBlockCapacity - 1].NextFreeIndex = UINT32_MAX;
4523	return m_ItemBlocks.back();
4524	}
4525	#endif // _VMA_POOL_ALLOCATOR_FUNCTIONS
4526	#endif // _VMA_POOL_ALLOCATOR
4527
4528	#ifndef _VMA_RAW_LIST
4529	template<typename T>
4530	struct VmaListItem
4531	{
4532	VmaListItem* pPrev;
4533	VmaListItem* pNext;
4534	T Value;
4535	};
4536
4537	// Doubly linked list.
4538	template<typename T>
4539	class VmaRawList
4540	{
4541	VMA_CLASS_NO_COPY(VmaRawList)
4542	public:
4543	typedef VmaListItem<T> ItemType;
4544
4545	VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks);
4546	// Intentionally not calling Clear, because that would be unnecessary
4547	// computations to return all items to m_ItemAllocator as free.
4548	~VmaRawList() = default;
4549
4550	size_t GetCount() const { return m_Count; }
4551	bool IsEmpty() const { return m_Count == 0; }
4552
4553	ItemType* Front() { return m_pFront; }
4554	ItemType* Back() { return m_pBack; }
4555	const ItemType* Front() const { return m_pFront; }
4556	const ItemType* Back() const { return m_pBack; }
4557
4558	ItemType* PushFront();
4559	ItemType* PushBack();
4560	ItemType* PushFront(const T& value);
4561	ItemType* PushBack(const T& value);
4562	void PopFront();
4563	void PopBack();
4564
4565	// Item can be null - it means PushBack.
4566	ItemType* InsertBefore(ItemType* pItem);
4567	// Item can be null - it means PushFront.
4568	ItemType* InsertAfter(ItemType* pItem);
4569	ItemType* InsertBefore(ItemType* pItem, const T& value);
4570	ItemType* InsertAfter(ItemType* pItem, const T& value);
4571
4572	void Clear();
4573	void Remove(ItemType* pItem);
4574
4575	private:
4576	const VkAllocationCallbacks* const m_pAllocationCallbacks;
4577	VmaPoolAllocator<ItemType> m_ItemAllocator;
4578	ItemType* m_pFront;
4579	ItemType* m_pBack;
4580	size_t m_Count;
4581	};
4582
4583	#ifndef _VMA_RAW_LIST_FUNCTIONS
4584	template<typename T>
4585	VmaRawList<T>::VmaRawList(const VkAllocationCallbacks* pAllocationCallbacks)
4586	: m_pAllocationCallbacks(pAllocationCallbacks),
4587	m_ItemAllocator(pAllocationCallbacks, 128),
4588	m_pFront(VMA_NULL),
4589	m_pBack(VMA_NULL),
4590	m_Count(0) {}
4591
4592	template<typename T>
4593	VmaListItem<T>* VmaRawList<T>::PushFront()
4594	{
4595	ItemType* const pNewItem = m_ItemAllocator.Alloc();
4596	pNewItem->pPrev = VMA_NULL;
4597	if (IsEmpty())
4598	{
4599	pNewItem->pNext = VMA_NULL;
4600	m_pFront = pNewItem;
4601	m_pBack = pNewItem;
4602	m_Count = 1;
4603	}
4604	else
4605	{
4606	pNewItem->pNext = m_pFront;
4607	m_pFront->pPrev = pNewItem;
4608	m_pFront = pNewItem;
4609	++m_Count;
4610	}
4611	return pNewItem;
4612	}
4613
4614	template<typename T>
4615	VmaListItem<T>* VmaRawList<T>::PushBack()
4616	{
4617	ItemType* const pNewItem = m_ItemAllocator.Alloc();
4618	pNewItem->pNext = VMA_NULL;
4619	if(IsEmpty())
4620	{
4621	pNewItem->pPrev = VMA_NULL;
4622	m_pFront = pNewItem;
4623	m_pBack = pNewItem;
4624	m_Count = 1;
4625	}
4626	else
4627	{
4628	pNewItem->pPrev = m_pBack;
4629	m_pBack->pNext = pNewItem;
4630	m_pBack = pNewItem;
4631	++m_Count;
4632	}
4633	return pNewItem;
4634	}
4635
4636	template<typename T>
4637	VmaListItem<T>* VmaRawList<T>::PushFront(const T& value)
4638	{
4639	ItemType* const pNewItem = PushFront();
4640	pNewItem->Value = value;
4641	return pNewItem;
4642	}
4643
4644	template<typename T>
4645	VmaListItem<T>* VmaRawList<T>::PushBack(const T& value)
4646	{
4647	ItemType* const pNewItem = PushBack();
4648	pNewItem->Value = value;
4649	return pNewItem;
4650	}
4651
4652	template<typename T>
4653	void VmaRawList<T>::PopFront()
4654	{
4655	VMA_HEAVY_ASSERT(m_Count > 0);
4656	ItemType* const pFrontItem = m_pFront;
4657	ItemType* const pNextItem = pFrontItem->pNext;
4658	if (pNextItem != VMA_NULL)
4659	{
4660	pNextItem->pPrev = VMA_NULL;
4661	}
4662	m_pFront = pNextItem;
4663	m_ItemAllocator.Free(pFrontItem);
4664	--m_Count;
4665	}
4666
4667	template<typename T>
4668	void VmaRawList<T>::PopBack()
4669	{
4670	VMA_HEAVY_ASSERT(m_Count > 0);
4671	ItemType* const pBackItem = m_pBack;
4672	ItemType* const pPrevItem = pBackItem->pPrev;
4673	if(pPrevItem != VMA_NULL)
4674	{
4675	pPrevItem->pNext = VMA_NULL;
4676	}
4677	m_pBack = pPrevItem;
4678	m_ItemAllocator.Free(pBackItem);
4679	--m_Count;
4680	}
4681
4682	template<typename T>
4683	void VmaRawList<T>::Clear()
4684	{
4685	if (IsEmpty() == false)
4686	{
4687	ItemType* pItem = m_pBack;
4688	while (pItem != VMA_NULL)
4689	{
4690	ItemType* const pPrevItem = pItem->pPrev;
4691	m_ItemAllocator.Free(pItem);
4692	pItem = pPrevItem;
4693	}
4694	m_pFront = VMA_NULL;
4695	m_pBack = VMA_NULL;
4696	m_Count = 0;
4697	}
4698	}
4699
4700	template<typename T>
4701	void VmaRawList<T>::Remove(ItemType* pItem)
4702	{
4703	VMA_HEAVY_ASSERT(pItem != VMA_NULL);
4704	VMA_HEAVY_ASSERT(m_Count > 0);
4705
4706	if(pItem->pPrev != VMA_NULL)
4707	{
4708	pItem->pPrev->pNext = pItem->pNext;
4709	}
4710	else
4711	{
4712	VMA_HEAVY_ASSERT(m_pFront == pItem);
4713	m_pFront = pItem->pNext;
4714	}
4715
4716	if(pItem->pNext != VMA_NULL)
4717	{
4718	pItem->pNext->pPrev = pItem->pPrev;
4719	}
4720	else
4721	{
4722	VMA_HEAVY_ASSERT(m_pBack == pItem);
4723	m_pBack = pItem->pPrev;
4724	}
4725
4726	m_ItemAllocator.Free(pItem);
4727	--m_Count;
4728	}
4729
4730	template<typename T>
4731	VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem)
4732	{
4733	if(pItem != VMA_NULL)
4734	{
4735	ItemType* const prevItem = pItem->pPrev;
4736	ItemType* const newItem = m_ItemAllocator.Alloc();
4737	newItem->pPrev = prevItem;
4738	newItem->pNext = pItem;
4739	pItem->pPrev = newItem;
4740	if(prevItem != VMA_NULL)
4741	{
4742	prevItem->pNext = newItem;
4743	}
4744	else
4745	{
4746	VMA_HEAVY_ASSERT(m_pFront == pItem);
4747	m_pFront = newItem;
4748	}
4749	++m_Count;
4750	return newItem;
4751	}
4752	else
4753	return PushBack();
4754	}
4755
4756	template<typename T>
4757	VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem)
4758	{
4759	if(pItem != VMA_NULL)
4760	{
4761	ItemType* const nextItem = pItem->pNext;
4762	ItemType* const newItem = m_ItemAllocator.Alloc();
4763	newItem->pNext = nextItem;
4764	newItem->pPrev = pItem;
4765	pItem->pNext = newItem;
4766	if(nextItem != VMA_NULL)
4767	{
4768	nextItem->pPrev = newItem;
4769	}
4770	else
4771	{
4772	VMA_HEAVY_ASSERT(m_pBack == pItem);
4773	m_pBack = newItem;
4774	}
4775	++m_Count;
4776	return newItem;
4777	}
4778	else
4779	return PushFront();
4780	}
4781
4782	template<typename T>
4783	VmaListItem<T>* VmaRawList<T>::InsertBefore(ItemType* pItem, const T& value)
4784	{
4785	ItemType* const newItem = InsertBefore(pItem);
4786	newItem->Value = value;
4787	return newItem;
4788	}
4789
4790	template<typename T>
4791	VmaListItem<T>* VmaRawList<T>::InsertAfter(ItemType* pItem, const T& value)
4792	{
4793	ItemType* const newItem = InsertAfter(pItem);
4794	newItem->Value = value;
4795	return newItem;
4796	}
4797	#endif // _VMA_RAW_LIST_FUNCTIONS
4798	#endif // _VMA_RAW_LIST
4799
4800	#ifndef _VMA_LIST
4801	template<typename T, typename AllocatorT>
4802	class VmaList
4803	{
4804	VMA_CLASS_NO_COPY(VmaList)
4805	public:
4806	class reverse_iterator;
4807	class const_iterator;
4808	class const_reverse_iterator;
4809
4810	class iterator
4811	{
4812	friend class const_iterator;
4813	friend class VmaList<T, AllocatorT>;
4814	public:
4815	iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4816	iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4817
4818	T& operator*() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4819	T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4820
4821	bool operator==(const iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4822	bool operator!=(const iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4823
4824	iterator operator++(int) { iterator result = *this; ++*this; return result; }
4825	iterator operator--(int) { iterator result = *this; --*this; return result; }
4826
4827	iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pNext; return *this; }
4828	iterator& operator--();
4829
4830	private:
4831	VmaRawList<T>* m_pList;
4832	VmaListItem<T>* m_pItem;
4833
4834	iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4835	};
4836	class reverse_iterator
4837	{
4838	friend class const_reverse_iterator;
4839	friend class VmaList<T, AllocatorT>;
4840	public:
4841	reverse_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4842	reverse_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4843
4844	T& operator*() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4845	T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4846
4847	bool operator==(const reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4848	bool operator!=(const reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4849
4850	reverse_iterator operator++(int) { reverse_iterator result = *this; ++* this; return result; }
4851	reverse_iterator operator--(int) { reverse_iterator result = *this; --* this; return result; }
4852
4853	reverse_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pPrev; return *this; }
4854	reverse_iterator& operator--();
4855
4856	private:
4857	VmaRawList<T>* m_pList;
4858	VmaListItem<T>* m_pItem;
4859
4860	reverse_iterator(VmaRawList<T>* pList, VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4861	};
4862	class const_iterator
4863	{
4864	friend class VmaList<T, AllocatorT>;
4865	public:
4866	const_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4867	const_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4868	const_iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4869
4870	iterator drop_const() { return { const_cast<VmaRawList<T>*>(m_pList), const_cast<VmaListItem<T>*>(m_pItem) }; }
4871
4872	const T& operator*() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4873	const T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4874
4875	bool operator==(const const_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4876	bool operator!=(const const_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4877
4878	const_iterator operator++(int) { const_iterator result = *this; ++* this; return result; }
4879	const_iterator operator--(int) { const_iterator result = *this; --* this; return result; }
4880
4881	const_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pNext; return *this; }
4882	const_iterator& operator--();
4883
4884	private:
4885	const VmaRawList<T>* m_pList;
4886	const VmaListItem<T>* m_pItem;
4887
4888	const_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4889	};
4890	class const_reverse_iterator
4891	{
4892	friend class VmaList<T, AllocatorT>;
4893	public:
4894	const_reverse_iterator() : m_pList(VMA_NULL), m_pItem(VMA_NULL) {}
4895	const_reverse_iterator(const reverse_iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4896	const_reverse_iterator(const iterator& src) : m_pList(src.m_pList), m_pItem(src.m_pItem) {}
4897
4898	reverse_iterator drop_const() { return { const_cast<VmaRawList<T>*>(m_pList), const_cast<VmaListItem<T>*>(m_pItem) }; }
4899
4900	const T& operator*() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return m_pItem->Value; }
4901	const T* operator->() const { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); return &m_pItem->Value; }
4902
4903	bool operator==(const const_reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem == rhs.m_pItem; }
4904	bool operator!=(const const_reverse_iterator& rhs) const { VMA_HEAVY_ASSERT(m_pList == rhs.m_pList); return m_pItem != rhs.m_pItem; }
4905
4906	const_reverse_iterator operator++(int) { const_reverse_iterator result = *this; ++* this; return result; }
4907	const_reverse_iterator operator--(int) { const_reverse_iterator result = *this; --* this; return result; }
4908
4909	const_reverse_iterator& operator++() { VMA_HEAVY_ASSERT(m_pItem != VMA_NULL); m_pItem = m_pItem->pPrev; return *this; }
4910	const_reverse_iterator& operator--();
4911
4912	private:
4913	const VmaRawList<T>* m_pList;
4914	const VmaListItem<T>* m_pItem;
4915
4916	const_reverse_iterator(const VmaRawList<T>* pList, const VmaListItem<T>* pItem) : m_pList(pList), m_pItem(pItem) {}
4917	};
4918
4919	VmaList(const AllocatorT& allocator) : m_RawList(allocator.m_pCallbacks) {}
4920
4921	bool empty() const { return m_RawList.IsEmpty(); }
4922	size_t size() const { return m_RawList.GetCount(); }
4923
4924	iterator begin() { return iterator(&m_RawList, m_RawList.Front()); }
4925	iterator end() { return iterator(&m_RawList, VMA_NULL); }
4926
4927	const_iterator cbegin() const { return const_iterator(&m_RawList, m_RawList.Front()); }
4928	const_iterator cend() const { return const_iterator(&m_RawList, VMA_NULL); }
4929
4930	const_iterator begin() const { return cbegin(); }
4931	const_iterator end() const { return cend(); }
4932
4933	reverse_iterator rbegin() { return reverse_iterator(&m_RawList, m_RawList.Back()); }
4934	reverse_iterator rend() { return reverse_iterator(&m_RawList, VMA_NULL); }
4935
4936	const_reverse_iterator crbegin() const { return const_reverse_iterator(&m_RawList, m_RawList.Back()); }
4937	const_reverse_iterator crend() const { return const_reverse_iterator(&m_RawList, VMA_NULL); }
4938
4939	const_reverse_iterator rbegin() const { return crbegin(); }
4940	const_reverse_iterator rend() const { return crend(); }
4941
4942	void push_back(const T& value) { m_RawList.PushBack(value); }
4943	iterator insert(iterator it, const T& value) { return iterator(&m_RawList, m_RawList.InsertBefore(it.m_pItem, value)); }
4944
4945	void clear() { m_RawList.Clear(); }
4946	void erase(iterator it) { m_RawList.Remove(it.m_pItem); }
4947
4948	private:
4949	VmaRawList<T> m_RawList;
4950	};
4951
4952	#ifndef _VMA_LIST_FUNCTIONS
4953	template<typename T, typename AllocatorT>
4954	typename VmaList<T, AllocatorT>::iterator& VmaList<T, AllocatorT>::iterator::operator--()
4955	{
4956	if (m_pItem != VMA_NULL)
4957	{
4958	m_pItem = m_pItem->pPrev;
4959	}
4960	else
4961	{
4962	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4963	m_pItem = m_pList->Back();
4964	}
4965	return *this;
4966	}
4967
4968	template<typename T, typename AllocatorT>
4969	typename VmaList<T, AllocatorT>::reverse_iterator& VmaList<T, AllocatorT>::reverse_iterator::operator--()
4970	{
4971	if (m_pItem != VMA_NULL)
4972	{
4973	m_pItem = m_pItem->pNext;
4974	}
4975	else
4976	{
4977	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4978	m_pItem = m_pList->Front();
4979	}
4980	return *this;
4981	}
4982
4983	template<typename T, typename AllocatorT>
4984	typename VmaList<T, AllocatorT>::const_iterator& VmaList<T, AllocatorT>::const_iterator::operator--()
4985	{
4986	if (m_pItem != VMA_NULL)
4987	{
4988	m_pItem = m_pItem->pPrev;
4989	}
4990	else
4991	{
4992	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
4993	m_pItem = m_pList->Back();
4994	}
4995	return *this;
4996	}
4997
4998	template<typename T, typename AllocatorT>
4999	typename VmaList<T, AllocatorT>::const_reverse_iterator& VmaList<T, AllocatorT>::const_reverse_iterator::operator--()
5000	{
5001	if (m_pItem != VMA_NULL)
5002	{
5003	m_pItem = m_pItem->pNext;
5004	}
5005	else
5006	{
5007	VMA_HEAVY_ASSERT(!m_pList->IsEmpty());
5008	m_pItem = m_pList->Back();
5009	}
5010	return *this;
5011	}
5012	#endif // _VMA_LIST_FUNCTIONS
5013	#endif // _VMA_LIST
5014
5015	#ifndef _VMA_INTRUSIVE_LINKED_LIST
5016	/*
5017	Expected interface of ItemTypeTraits:
5018	struct MyItemTypeTraits
5019	{
5020	typedef MyItem ItemType;
5021	static ItemType* GetPrev(const ItemType* item) { return item->myPrevPtr; }
5022	static ItemType* GetNext(const ItemType* item) { return item->myNextPtr; }
5023	static ItemType*& AccessPrev(ItemType* item) { return item->myPrevPtr; }
5024	static ItemType*& AccessNext(ItemType* item) { return item->myNextPtr; }
5025	};
5026	*/
5027	template<typename ItemTypeTraits>
5028	class VmaIntrusiveLinkedList
5029	{
5030	public:
5031	typedef typename ItemTypeTraits::ItemType ItemType;
5032	static ItemType* GetPrev(const ItemType* item) { return ItemTypeTraits::GetPrev(item); }
5033	static ItemType* GetNext(const ItemType* item) { return ItemTypeTraits::GetNext(item); }
5034
5035	// Movable, not copyable.
5036	VmaIntrusiveLinkedList() = default;
5037	VmaIntrusiveLinkedList(VmaIntrusiveLinkedList && src);
5038	VmaIntrusiveLinkedList(const VmaIntrusiveLinkedList&) = delete;
5039	VmaIntrusiveLinkedList& operator=(VmaIntrusiveLinkedList&& src);
5040	VmaIntrusiveLinkedList& operator=(const VmaIntrusiveLinkedList&) = delete;
5041	~VmaIntrusiveLinkedList() { VMA_HEAVY_ASSERT(IsEmpty()); }
5042
5043	size_t GetCount() const { return m_Count; }
5044	bool IsEmpty() const { return m_Count == 0; }
5045	ItemType* Front() { return m_Front; }
5046	ItemType* Back() { return m_Back; }
5047	const ItemType* Front() const { return m_Front; }
5048	const ItemType* Back() const { return m_Back; }
5049
5050	void PushBack(ItemType* item);
5051	void PushFront(ItemType* item);
5052	ItemType* PopBack();
5053	ItemType* PopFront();
5054
5055	// MyItem can be null - it means PushBack.
5056	void InsertBefore(ItemType* existingItem, ItemType* newItem);
5057	// MyItem can be null - it means PushFront.
5058	void InsertAfter(ItemType* existingItem, ItemType* newItem);
5059	void Remove(ItemType* item);
5060	void RemoveAll();
5061
5062	private:
5063	ItemType* m_Front = VMA_NULL;
5064	ItemType* m_Back = VMA_NULL;
5065	size_t m_Count = 0;
5066	};
5067
5068	#ifndef _VMA_INTRUSIVE_LINKED_LIST_FUNCTIONS
5069	template<typename ItemTypeTraits>
5070	VmaIntrusiveLinkedList<ItemTypeTraits>::VmaIntrusiveLinkedList(VmaIntrusiveLinkedList&& src)
5071	: m_Front(src.m_Front), m_Back(src.m_Back), m_Count(src.m_Count)
5072	{
5073	src.m_Front = src.m_Back = VMA_NULL;
5074	src.m_Count = 0;
5075	}
5076
5077	template<typename ItemTypeTraits>
5078	VmaIntrusiveLinkedList<ItemTypeTraits>& VmaIntrusiveLinkedList<ItemTypeTraits>::operator=(VmaIntrusiveLinkedList&& src)
5079	{
5080	if (&src != this)
5081	{
5082	VMA_HEAVY_ASSERT(IsEmpty());
5083	m_Front = src.m_Front;
5084	m_Back = src.m_Back;
5085	m_Count = src.m_Count;
5086	src.m_Front = src.m_Back = VMA_NULL;
5087	src.m_Count = 0;
5088	}
5089	return *this;
5090	}
5091
5092	template<typename ItemTypeTraits>
5093	void VmaIntrusiveLinkedList<ItemTypeTraits>::PushBack(ItemType* item)
5094	{
5095	VMA_HEAVY_ASSERT(ItemTypeTraits::GetPrev(item) == VMA_NULL && ItemTypeTraits::GetNext(item) == VMA_NULL);
5096	if (IsEmpty())
5097	{
5098	m_Front = item;
5099	m_Back = item;
5100	m_Count = 1;
5101	}
5102	else
5103	{
5104	ItemTypeTraits::AccessPrev(item) = m_Back;
5105	ItemTypeTraits::AccessNext(m_Back) = item;
5106	m_Back = item;
5107	++m_Count;
5108	}
5109	}
5110
5111	template<typename ItemTypeTraits>
5112	void VmaIntrusiveLinkedList<ItemTypeTraits>::PushFront(ItemType* item)
5113	{
5114	VMA_HEAVY_ASSERT(ItemTypeTraits::GetPrev(item) == VMA_NULL && ItemTypeTraits::GetNext(item) == VMA_NULL);
5115	if (IsEmpty())
5116	{
5117	m_Front = item;
5118	m_Back = item;
5119	m_Count = 1;
5120	}
5121	else
5122	{
5123	ItemTypeTraits::AccessNext(item) = m_Front;
5124	ItemTypeTraits::AccessPrev(m_Front) = item;
5125	m_Front = item;
5126	++m_Count;
5127	}
5128	}
5129
5130	template<typename ItemTypeTraits>
5131	typename VmaIntrusiveLinkedList<ItemTypeTraits>::ItemType* VmaIntrusiveLinkedList<ItemTypeTraits>::PopBack()
5132	{
5133	VMA_HEAVY_ASSERT(m_Count > 0);
5134	ItemType* const backItem = m_Back;
5135	ItemType* const prevItem = ItemTypeTraits::GetPrev(backItem);
5136	if (prevItem != VMA_NULL)
5137	{
5138	ItemTypeTraits::AccessNext(prevItem) = VMA_NULL;
5139	}
5140	m_Back = prevItem;
5141	--m_Count;
5142	ItemTypeTraits::AccessPrev(backItem) = VMA_NULL;
5143	ItemTypeTraits::AccessNext(backItem) = VMA_NULL;
5144	return backItem;
5145	}
5146
5147	template<typename ItemTypeTraits>
5148	typename VmaIntrusiveLinkedList<ItemTypeTraits>::ItemType* VmaIntrusiveLinkedList<ItemTypeTraits>::PopFront()
5149	{
5150	VMA_HEAVY_ASSERT(m_Count > 0);
5151	ItemType* const frontItem = m_Front;
5152	ItemType* const nextItem = ItemTypeTraits::GetNext(frontItem);
5153	if (nextItem != VMA_NULL)
5154	{
5155	ItemTypeTraits::AccessPrev(nextItem) = VMA_NULL;
5156	}
5157	m_Front = nextItem;
5158	--m_Count;
5159	ItemTypeTraits::AccessPrev(frontItem) = VMA_NULL;
5160	ItemTypeTraits::AccessNext(frontItem) = VMA_NULL;
5161	return frontItem;
5162	}
5163
5164	template<typename ItemTypeTraits>
5165	void VmaIntrusiveLinkedList<ItemTypeTraits>::InsertBefore(ItemType* existingItem, ItemType* newItem)
5166	{
5167	VMA_HEAVY_ASSERT(newItem != VMA_NULL && ItemTypeTraits::GetPrev(newItem) == VMA_NULL && ItemTypeTraits::GetNext(newItem) == VMA_NULL);
5168	if (existingItem != VMA_NULL)
5169	{
5170	ItemType* const prevItem = ItemTypeTraits::GetPrev(existingItem);
5171	ItemTypeTraits::AccessPrev(newItem) = prevItem;
5172	ItemTypeTraits::AccessNext(newItem) = existingItem;
5173	ItemTypeTraits::AccessPrev(existingItem) = newItem;
5174	if (prevItem != VMA_NULL)
5175	{
5176	ItemTypeTraits::AccessNext(prevItem) = newItem;
5177	}
5178	else
5179	{
5180	VMA_HEAVY_ASSERT(m_Front == existingItem);
5181	m_Front = newItem;
5182	}
5183	++m_Count;
5184	}
5185	else
5186	PushBack(item: newItem);
5187	}
5188
5189	template<typename ItemTypeTraits>
5190	void VmaIntrusiveLinkedList<ItemTypeTraits>::InsertAfter(ItemType* existingItem, ItemType* newItem)
5191	{
5192	VMA_HEAVY_ASSERT(newItem != VMA_NULL && ItemTypeTraits::GetPrev(newItem) == VMA_NULL && ItemTypeTraits::GetNext(newItem) == VMA_NULL);
5193	if (existingItem != VMA_NULL)
5194	{
5195	ItemType* const nextItem = ItemTypeTraits::GetNext(existingItem);
5196	ItemTypeTraits::AccessNext(newItem) = nextItem;
5197	ItemTypeTraits::AccessPrev(newItem) = existingItem;
5198	ItemTypeTraits::AccessNext(existingItem) = newItem;
5199	if (nextItem != VMA_NULL)
5200	{
5201	ItemTypeTraits::AccessPrev(nextItem) = newItem;
5202	}
5203	else
5204	{
5205	VMA_HEAVY_ASSERT(m_Back == existingItem);
5206	m_Back = newItem;
5207	}
5208	++m_Count;
5209	}
5210	else
5211	return PushFront(item: newItem);
5212	}
5213
5214	template<typename ItemTypeTraits>
5215	void VmaIntrusiveLinkedList<ItemTypeTraits>::Remove(ItemType* item)
5216	{
5217	VMA_HEAVY_ASSERT(item != VMA_NULL && m_Count > 0);
5218	if (ItemTypeTraits::GetPrev(item) != VMA_NULL)
5219	{
5220	ItemTypeTraits::AccessNext(ItemTypeTraits::AccessPrev(item)) = ItemTypeTraits::GetNext(item);
5221	}
5222	else
5223	{
5224	VMA_HEAVY_ASSERT(m_Front == item);
5225	m_Front = ItemTypeTraits::GetNext(item);
5226	}
5227
5228	if (ItemTypeTraits::GetNext(item) != VMA_NULL)
5229	{
5230	ItemTypeTraits::AccessPrev(ItemTypeTraits::AccessNext(item)) = ItemTypeTraits::GetPrev(item);
5231	}
5232	else
5233	{
5234	VMA_HEAVY_ASSERT(m_Back == item);
5235	m_Back = ItemTypeTraits::GetPrev(item);
5236	}
5237	ItemTypeTraits::AccessPrev(item) = VMA_NULL;
5238	ItemTypeTraits::AccessNext(item) = VMA_NULL;
5239	--m_Count;
5240	}
5241
5242	template<typename ItemTypeTraits>
5243	void VmaIntrusiveLinkedList<ItemTypeTraits>::RemoveAll()
5244	{
5245	if (!IsEmpty())
5246	{
5247	ItemType* item = m_Back;
5248	while (item != VMA_NULL)
5249	{
5250	ItemType* const prevItem = ItemTypeTraits::AccessPrev(item);
5251	ItemTypeTraits::AccessPrev(item) = VMA_NULL;
5252	ItemTypeTraits::AccessNext(item) = VMA_NULL;
5253	item = prevItem;
5254	}
5255	m_Front = VMA_NULL;
5256	m_Back = VMA_NULL;
5257	m_Count = 0;
5258	}
5259	}
5260	#endif // _VMA_INTRUSIVE_LINKED_LIST_FUNCTIONS
5261	#endif // _VMA_INTRUSIVE_LINKED_LIST
5262
5263	// Unused in this version.
5264	#if 0
5265
5266	#ifndef _VMA_PAIR
5267	template<typename T1, typename T2>
5268	struct VmaPair
5269	{
5270	T1 first;
5271	T2 second;
5272
5273	VmaPair() : first(), second() {}
5274	VmaPair(const T1& firstSrc, const T2& secondSrc) : first(firstSrc), second(secondSrc) {}
5275	};
5276
5277	template<typename FirstT, typename SecondT>
5278	struct VmaPairFirstLess
5279	{
5280	bool operator()(const VmaPair<FirstT, SecondT>& lhs, const VmaPair<FirstT, SecondT>& rhs) const
5281	{
5282	return lhs.first < rhs.first;
5283	}
5284	bool operator()(const VmaPair<FirstT, SecondT>& lhs, const FirstT& rhsFirst) const
5285	{
5286	return lhs.first < rhsFirst;
5287	}
5288	};
5289	#endif // _VMA_PAIR
5290
5291	#ifndef _VMA_MAP
5292	/* Class compatible with subset of interface of std::unordered_map.
5293	KeyT, ValueT must be POD because they will be stored in VmaVector.
5294	*/
5295	template<typename KeyT, typename ValueT>
5296	class VmaMap
5297	{
5298	public:
5299	typedef VmaPair<KeyT, ValueT> PairType;
5300	typedef PairType* iterator;
5301
5302	VmaMap(const VmaStlAllocator<PairType>& allocator) : m_Vector(allocator) {}
5303
5304	iterator begin() { return m_Vector.begin(); }
5305	iterator end() { return m_Vector.end(); }
5306	size_t size() { return m_Vector.size(); }
5307
5308	void insert(const PairType& pair);
5309	iterator find(const KeyT& key);
5310	void erase(iterator it);
5311
5312	private:
5313	VmaVector< PairType, VmaStlAllocator<PairType>> m_Vector;
5314	};
5315
5316	#ifndef _VMA_MAP_FUNCTIONS
5317	template<typename KeyT, typename ValueT>
5318	void VmaMap<KeyT, ValueT>::insert(const PairType& pair)
5319	{
5320	const size_t indexToInsert = VmaBinaryFindFirstNotLess(
5321	m_Vector.data(),
5322	m_Vector.data() + m_Vector.size(),
5323	pair,
5324	VmaPairFirstLess<KeyT, ValueT>()) - m_Vector.data();
5325	VmaVectorInsert(m_Vector, indexToInsert, pair);
5326	}
5327
5328	template<typename KeyT, typename ValueT>
5329	VmaPair<KeyT, ValueT>* VmaMap<KeyT, ValueT>::find(const KeyT& key)
5330	{
5331	PairType* it = VmaBinaryFindFirstNotLess(
5332	m_Vector.data(),
5333	m_Vector.data() + m_Vector.size(),
5334	key,
5335	VmaPairFirstLess<KeyT, ValueT>());
5336	if ((it != m_Vector.end()) && (it->first == key))
5337	{
5338	return it;
5339	}
5340	else
5341	{
5342	return m_Vector.end();
5343	}
5344	}
5345
5346	template<typename KeyT, typename ValueT>
5347	void VmaMap<KeyT, ValueT>::erase(iterator it)
5348	{
5349	VmaVectorRemove(m_Vector, it - m_Vector.begin());
5350	}
5351	#endif // _VMA_MAP_FUNCTIONS
5352	#endif // _VMA_MAP
5353
5354	#endif // #if 0
5355
5356	#if !defined(_VMA_STRING_BUILDER) && VMA_STATS_STRING_ENABLED
5357	class VmaStringBuilder
5358	{
5359	public:
5360	VmaStringBuilder(const VkAllocationCallbacks* allocationCallbacks) : m_Data(VmaStlAllocator<char>(allocationCallbacks)) {}
5361	~VmaStringBuilder() = default;
5362
5363	size_t GetLength() const { return m_Data.size(); }
5364	const char* GetData() const { return m_Data.data(); }
5365	void AddNewLine() { Add(ch: '\n'); }
5366	void Add(char ch) { m_Data.push_back(src: ch); }
5367
5368	void Add(const char* pStr);
5369	void AddNumber(uint32_t num);
5370	void AddNumber(uint64_t num);
5371	void AddPointer(const void* ptr);
5372
5373	private:
5374	VmaVector<char, VmaStlAllocator<char>> m_Data;
5375	};
5376
5377	#ifndef _VMA_STRING_BUILDER_FUNCTIONS
5378	void VmaStringBuilder::Add(const char* pStr)
5379	{
5380	const size_t strLen = strlen(s: pStr);
5381	if (strLen > 0)
5382	{
5383	const size_t oldCount = m_Data.size();
5384	m_Data.resize(newCount: oldCount + strLen);
5385	memcpy(dest: m_Data.data() + oldCount, src: pStr, n: strLen);
5386	}
5387	}
5388
5389	void VmaStringBuilder::AddNumber(uint32_t num)
5390	{
5391	char buf[11];
5392	buf[10] = '\0';
5393	char* p = &buf[10];
5394	do
5395	{
5396	*--p = '0' + (num % 10);
5397	num /= 10;
5398	} while (num);
5399	Add(pStr: p);
5400	}
5401
5402	void VmaStringBuilder::AddNumber(uint64_t num)
5403	{
5404	char buf[21];
5405	buf[20] = '\0';
5406	char* p = &buf[20];
5407	do
5408	{
5409	*--p = '0' + (num % 10);
5410	num /= 10;
5411	} while (num);
5412	Add(pStr: p);
5413	}
5414
5415	void VmaStringBuilder::AddPointer(const void* ptr)
5416	{
5417	char buf[21];
5418	VmaPtrToStr(outStr: buf, strLen: sizeof(buf), ptr);
5419	Add(pStr: buf);
5420	}
5421	#endif //_VMA_STRING_BUILDER_FUNCTIONS
5422	#endif // _VMA_STRING_BUILDER
5423
5424	#if !defined(_VMA_JSON_WRITER) && VMA_STATS_STRING_ENABLED
5425	/*
5426	Allows to conveniently build a correct JSON document to be written to the
5427	VmaStringBuilder passed to the constructor.
5428	*/
5429	class VmaJsonWriter
5430	{
5431	VMA_CLASS_NO_COPY(VmaJsonWriter)
5432	public:
5433	// sb - string builder to write the document to. Must remain alive for the whole lifetime of this object.
5434	VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb);
5435	~VmaJsonWriter();
5436
5437	// Begins object by writing "{".
5438	// Inside an object, you must call pairs of WriteString and a value, e.g.:
5439	// j.BeginObject(true); j.WriteString("A"); j.WriteNumber(1); j.WriteString("B"); j.WriteNumber(2); j.EndObject();
5440	// Will write: { "A": 1, "B": 2 }
5441	void BeginObject(bool singleLine = false);
5442	// Ends object by writing "}".
5443	void EndObject();
5444
5445	// Begins array by writing "[".
5446	// Inside an array, you can write a sequence of any values.
5447	void BeginArray(bool singleLine = false);
5448	// Ends array by writing "[".
5449	void EndArray();
5450
5451	// Writes a string value inside "".
5452	// pStr can contain any ANSI characters, including '"', new line etc. - they will be properly escaped.
5453	void WriteString(const char* pStr);
5454
5455	// Begins writing a string value.
5456	// Call BeginString, ContinueString, ContinueString, ..., EndString instead of
5457	// WriteString to conveniently build the string content incrementally, made of
5458	// parts including numbers.
5459	void BeginString(const char* pStr = VMA_NULL);
5460	// Posts next part of an open string.
5461	void ContinueString(const char* pStr);
5462	// Posts next part of an open string. The number is converted to decimal characters.
5463	void ContinueString(uint32_t n);
5464	void ContinueString(uint64_t n);
5465	void ContinueString_Size(size_t n);
5466	// Posts next part of an open string. Pointer value is converted to characters
5467	// using "%p" formatting - shown as hexadecimal number, e.g.: 000000081276Ad00
5468	void ContinueString_Pointer(const void* ptr);
5469	// Ends writing a string value by writing '"'.
5470	void EndString(const char* pStr = VMA_NULL);
5471
5472	// Writes a number value.
5473	void WriteNumber(uint32_t n);
5474	void WriteNumber(uint64_t n);
5475	void WriteSize(size_t n);
5476	// Writes a boolean value - false or true.
5477	void WriteBool(bool b);
5478	// Writes a null value.
5479	void WriteNull();
5480
5481	private:
5482	enum COLLECTION_TYPE
5483	{
5484	COLLECTION_TYPE_OBJECT,
5485	COLLECTION_TYPE_ARRAY,
5486	};
5487	struct StackItem
5488	{
5489	COLLECTION_TYPE type;
5490	uint32_t valueCount;
5491	bool singleLineMode;
5492	};
5493
5494	static const char* const INDENT;
5495
5496	VmaStringBuilder& m_SB;
5497	VmaVector< StackItem, VmaStlAllocator<StackItem> > m_Stack;
5498	bool m_InsideString;
5499
5500	// Write size_t for less than 64bits
5501	void WriteSize(size_t n, std::integral_constant<bool, false>) { m_SB.AddNumber(num: static_cast<uint32_t>(n)); }
5502	// Write size_t for 64bits
5503	void WriteSize(size_t n, std::integral_constant<bool, true>) { m_SB.AddNumber(num: static_cast<uint64_t>(n)); }
5504
5505	void BeginValue(bool isString);
5506	void WriteIndent(bool oneLess = false);
5507	};
5508	const char* const VmaJsonWriter::INDENT = " ";
5509
5510	#ifndef _VMA_JSON_WRITER_FUNCTIONS
5511	VmaJsonWriter::VmaJsonWriter(const VkAllocationCallbacks* pAllocationCallbacks, VmaStringBuilder& sb)
5512	: m_SB(sb),
5513	m_Stack(VmaStlAllocator<StackItem>(pAllocationCallbacks)),
5514	m_InsideString(false) {}
5515
5516	VmaJsonWriter::~VmaJsonWriter()
5517	{
5518	VMA_ASSERT(!m_InsideString);
5519	VMA_ASSERT(m_Stack.empty());
5520	}
5521
5522	void VmaJsonWriter::BeginObject(bool singleLine)
5523	{
5524	VMA_ASSERT(!m_InsideString);
5525
5526	BeginValue(isString: false);
5527	m_SB.Add(ch: '{');
5528
5529	StackItem item;
5530	item.type = COLLECTION_TYPE_OBJECT;
5531	item.valueCount = 0;
5532	item.singleLineMode = singleLine;
5533	m_Stack.push_back(src: item);
5534	}
5535
5536	void VmaJsonWriter::EndObject()
5537	{
5538	VMA_ASSERT(!m_InsideString);
5539
5540	WriteIndent(oneLess: true);
5541	m_SB.Add(ch: '}');
5542
5543	VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_OBJECT);
5544	m_Stack.pop_back();
5545	}
5546
5547	void VmaJsonWriter::BeginArray(bool singleLine)
5548	{
5549	VMA_ASSERT(!m_InsideString);
5550
5551	BeginValue(isString: false);
5552	m_SB.Add(ch: '[');
5553
5554	StackItem item;
5555	item.type = COLLECTION_TYPE_ARRAY;
5556	item.valueCount = 0;
5557	item.singleLineMode = singleLine;
5558	m_Stack.push_back(src: item);
5559	}
5560
5561	void VmaJsonWriter::EndArray()
5562	{
5563	VMA_ASSERT(!m_InsideString);
5564
5565	WriteIndent(oneLess: true);
5566	m_SB.Add(ch: ']');
5567
5568	VMA_ASSERT(!m_Stack.empty() && m_Stack.back().type == COLLECTION_TYPE_ARRAY);
5569	m_Stack.pop_back();
5570	}
5571
5572	void VmaJsonWriter::WriteString(const char* pStr)
5573	{
5574	BeginString(pStr);
5575	EndString();
5576	}
5577
5578	void VmaJsonWriter::BeginString(const char* pStr)
5579	{
5580	VMA_ASSERT(!m_InsideString);
5581
5582	BeginValue(isString: true);
5583	m_SB.Add(ch: '"');
5584	m_InsideString = true;
5585	if (pStr != VMA_NULL && pStr[0] != '\0')
5586	{
5587	ContinueString(pStr);
5588	}
5589	}
5590
5591	void VmaJsonWriter::ContinueString(const char* pStr)
5592	{
5593	VMA_ASSERT(m_InsideString);
5594
5595	const size_t strLen = strlen(s: pStr);
5596	for (size_t i = 0; i < strLen; ++i)
5597	{
5598	char ch = pStr[i];
5599	if (ch == '\\')
5600	{
5601	m_SB.Add(pStr: "\\\\");
5602	}
5603	else if (ch == '"')
5604	{
5605	m_SB.Add(pStr: "\\\"");
5606	}
5607	else if (ch >= 32)
5608	{
5609	m_SB.Add(ch);
5610	}
5611	else switch (ch)
5612	{
5613	case '\b':
5614	m_SB.Add(pStr: "\\b");
5615	break;
5616	case '\f':
5617	m_SB.Add(pStr: "\\f");
5618	break;
5619	case '\n':
5620	m_SB.Add(pStr: "\\n");
5621	break;
5622	case '\r':
5623	m_SB.Add(pStr: "\\r");
5624	break;
5625	case '\t':
5626	m_SB.Add(pStr: "\\t");
5627	break;
5628	default:
5629	VMA_ASSERT(0 && "Character not currently supported.");
5630	break;
5631	}
5632	}
5633	}
5634
5635	void VmaJsonWriter::ContinueString(uint32_t n)
5636	{
5637	VMA_ASSERT(m_InsideString);
5638	m_SB.AddNumber(num: n);
5639	}
5640
5641	void VmaJsonWriter::ContinueString(uint64_t n)
5642	{
5643	VMA_ASSERT(m_InsideString);
5644	m_SB.AddNumber(num: n);
5645	}
5646
5647	void VmaJsonWriter::ContinueString_Size(size_t n)
5648	{
5649	VMA_ASSERT(m_InsideString);
5650	// Fix for AppleClang incorrect type casting
5651	// TODO: Change to if constexpr when C++17 used as minimal standard
5652	WriteSize(n, std::is_same<size_t, uint64_t>{});
5653	}
5654
5655	void VmaJsonWriter::ContinueString_Pointer(const void* ptr)
5656	{
5657	VMA_ASSERT(m_InsideString);
5658	m_SB.AddPointer(ptr);
5659	}
5660
5661	void VmaJsonWriter::EndString(const char* pStr)
5662	{
5663	VMA_ASSERT(m_InsideString);
5664	if (pStr != VMA_NULL && pStr[0] != '\0')
5665	{
5666	ContinueString(pStr);
5667	}
5668	m_SB.Add(ch: '"');
5669	m_InsideString = false;
5670	}
5671
5672	void VmaJsonWriter::WriteNumber(uint32_t n)
5673	{
5674	VMA_ASSERT(!m_InsideString);
5675	BeginValue(isString: false);
5676	m_SB.AddNumber(num: n);
5677	}
5678
5679	void VmaJsonWriter::WriteNumber(uint64_t n)
5680	{
5681	VMA_ASSERT(!m_InsideString);
5682	BeginValue(isString: false);
5683	m_SB.AddNumber(num: n);
5684	}
5685
5686	void VmaJsonWriter::WriteSize(size_t n)
5687	{
5688	VMA_ASSERT(!m_InsideString);
5689	BeginValue(isString: false);
5690	// Fix for AppleClang incorrect type casting
5691	// TODO: Change to if constexpr when C++17 used as minimal standard
5692	WriteSize(n, std::is_same<size_t, uint64_t>{});
5693	}
5694
5695	void VmaJsonWriter::WriteBool(bool b)
5696	{
5697	VMA_ASSERT(!m_InsideString);
5698	BeginValue(isString: false);
5699	m_SB.Add(pStr: b ? "true" : "false");
5700	}
5701
5702	void VmaJsonWriter::WriteNull()
5703	{
5704	VMA_ASSERT(!m_InsideString);
5705	BeginValue(isString: false);
5706	m_SB.Add(pStr: "null");
5707	}
5708
5709	void VmaJsonWriter::BeginValue(bool isString)
5710	{
5711	if (!m_Stack.empty())
5712	{
5713	StackItem& currItem = m_Stack.back();
5714	if (currItem.type == COLLECTION_TYPE_OBJECT &&
5715	currItem.valueCount % 2 == 0)
5716	{
5717	VMA_ASSERT(isString);
5718	}
5719
5720	if (currItem.type == COLLECTION_TYPE_OBJECT &&
5721	currItem.valueCount % 2 != 0)
5722	{
5723	m_SB.Add(pStr: ": ");
5724	}
5725	else if (currItem.valueCount > 0)
5726	{
5727	m_SB.Add(pStr: ", ");
5728	WriteIndent();
5729	}
5730	else
5731	{
5732	WriteIndent();
5733	}
5734	++currItem.valueCount;
5735	}
5736	}
5737
5738	void VmaJsonWriter::WriteIndent(bool oneLess)
5739	{
5740	if (!m_Stack.empty() && !m_Stack.back().singleLineMode)
5741	{
5742	m_SB.AddNewLine();
5743
5744	size_t count = m_Stack.size();
5745	if (count > 0 && oneLess)
5746	{
5747	--count;
5748	}
5749	for (size_t i = 0; i < count; ++i)
5750	{
5751	m_SB.Add(pStr: INDENT);
5752	}
5753	}
5754	}
5755	#endif // _VMA_JSON_WRITER_FUNCTIONS
5756
5757	static void VmaPrintDetailedStatistics(VmaJsonWriter& json, const VmaDetailedStatistics& stat)
5758	{
5759	json.BeginObject();
5760
5761	json.WriteString(pStr: "BlockCount");
5762	json.WriteNumber(n: stat.statistics.blockCount);
5763	json.WriteString(pStr: "BlockBytes");
5764	json.WriteNumber(n: stat.statistics.blockBytes);
5765	json.WriteString(pStr: "AllocationCount");
5766	json.WriteNumber(n: stat.statistics.allocationCount);
5767	json.WriteString(pStr: "AllocationBytes");
5768	json.WriteNumber(n: stat.statistics.allocationBytes);
5769	json.WriteString(pStr: "UnusedRangeCount");
5770	json.WriteNumber(n: stat.unusedRangeCount);
5771
5772	if (stat.statistics.allocationCount > 1)
5773	{
5774	json.WriteString(pStr: "AllocationSizeMin");
5775	json.WriteNumber(n: stat.allocationSizeMin);
5776	json.WriteString(pStr: "AllocationSizeMax");
5777	json.WriteNumber(n: stat.allocationSizeMax);
5778	}
5779	if (stat.unusedRangeCount > 1)
5780	{
5781	json.WriteString(pStr: "UnusedRangeSizeMin");
5782	json.WriteNumber(n: stat.unusedRangeSizeMin);
5783	json.WriteString(pStr: "UnusedRangeSizeMax");
5784	json.WriteNumber(n: stat.unusedRangeSizeMax);
5785	}
5786	json.EndObject();
5787	}
5788	#endif // _VMA_JSON_WRITER
5789
5790	#ifndef _VMA_MAPPING_HYSTERESIS
5791
5792	class VmaMappingHysteresis
5793	{
5794	VMA_CLASS_NO_COPY(VmaMappingHysteresis)
5795	public:
5796	VmaMappingHysteresis() = default;
5797
5798	uint32_t GetExtraMapping() const { return m_ExtraMapping; }
5799
5800	// Call when Map was called.
5801	// Returns true if switched to extra +1 mapping reference count.
5802	bool PostMap()
5803	{
5804	#if VMA_MAPPING_HYSTERESIS_ENABLED
5805	if(m_ExtraMapping == 0)
5806	{
5807	++m_MajorCounter;
5808	if(m_MajorCounter >= COUNTER_MIN_EXTRA_MAPPING)
5809	{
5810	m_ExtraMapping = 1;
5811	m_MajorCounter = 0;
5812	m_MinorCounter = 0;
5813	return true;
5814	}
5815	}
5816	else // m_ExtraMapping == 1
5817	PostMinorCounter();
5818	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5819	return false;
5820	}
5821
5822	// Call when Unmap was called.
5823	void PostUnmap()
5824	{
5825	#if VMA_MAPPING_HYSTERESIS_ENABLED
5826	if(m_ExtraMapping == 0)
5827	++m_MajorCounter;
5828	else // m_ExtraMapping == 1
5829	PostMinorCounter();
5830	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5831	}
5832
5833	// Call when allocation was made from the memory block.
5834	void PostAlloc()
5835	{
5836	#if VMA_MAPPING_HYSTERESIS_ENABLED
5837	if(m_ExtraMapping == 1)
5838	++m_MajorCounter;
5839	else // m_ExtraMapping == 0
5840	PostMinorCounter();
5841	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5842	}
5843
5844	// Call when allocation was freed from the memory block.
5845	// Returns true if switched to extra -1 mapping reference count.
5846	bool PostFree()
5847	{
5848	#if VMA_MAPPING_HYSTERESIS_ENABLED
5849	if(m_ExtraMapping == 1)
5850	{
5851	++m_MajorCounter;
5852	if(m_MajorCounter >= COUNTER_MIN_EXTRA_MAPPING &&
5853	m_MajorCounter > m_MinorCounter + 1)
5854	{
5855	m_ExtraMapping = 0;
5856	m_MajorCounter = 0;
5857	m_MinorCounter = 0;
5858	return true;
5859	}
5860	}
5861	else // m_ExtraMapping == 0
5862	PostMinorCounter();
5863	#endif // #if VMA_MAPPING_HYSTERESIS_ENABLED
5864	return false;
5865	}
5866
5867	private:
5868	static const int32_t COUNTER_MIN_EXTRA_MAPPING = 7;
5869
5870	uint32_t m_MinorCounter = 0;
5871	uint32_t m_MajorCounter = 0;
5872	uint32_t m_ExtraMapping = 0; // 0 or 1.
5873
5874	void PostMinorCounter()
5875	{
5876	if(m_MinorCounter < m_MajorCounter)
5877	{
5878	++m_MinorCounter;
5879	}
5880	else if(m_MajorCounter > 0)
5881	{
5882	--m_MajorCounter;
5883	--m_MinorCounter;
5884	}
5885	}
5886	};
5887
5888	#endif // _VMA_MAPPING_HYSTERESIS
5889
5890	#ifndef _VMA_DEVICE_MEMORY_BLOCK
5891	/*
5892	Represents a single block of device memory (`VkDeviceMemory`) with all the
5893	data about its regions (aka suballocations, #VmaAllocation), assigned and free.
5894
5895	Thread-safety:
5896	- Access to m_pMetadata must be externally synchronized.
5897	- Map, Unmap, Bind* are synchronized internally.
5898	*/
5899	class VmaDeviceMemoryBlock
5900	{
5901	VMA_CLASS_NO_COPY(VmaDeviceMemoryBlock)
5902	public:
5903	VmaBlockMetadata* m_pMetadata;
5904
5905	VmaDeviceMemoryBlock(VmaAllocator hAllocator);
5906	~VmaDeviceMemoryBlock();
5907
5908	// Always call after construction.
5909	void Init(
5910	VmaAllocator hAllocator,
5911	VmaPool hParentPool,
5912	uint32_t newMemoryTypeIndex,
5913	VkDeviceMemory newMemory,
5914	VkDeviceSize newSize,
5915	uint32_t id,
5916	uint32_t algorithm,
5917	VkDeviceSize bufferImageGranularity);
5918	// Always call before destruction.
5919	void Destroy(VmaAllocator allocator);
5920
5921	VmaPool GetParentPool() const { return m_hParentPool; }
5922	VkDeviceMemory GetDeviceMemory() const { return m_hMemory; }
5923	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
5924	uint32_t GetId() const { return m_Id; }
5925	void* GetMappedData() const { return m_pMappedData; }
5926	uint32_t GetMapRefCount() const { return m_MapCount; }
5927
5928	// Call when allocation/free was made from m_pMetadata.
5929	// Used for m_MappingHysteresis.
5930	void PostAlloc() { m_MappingHysteresis.PostAlloc(); }
5931	void PostFree(VmaAllocator hAllocator);
5932
5933	// Validates all data structures inside this object. If not valid, returns false.
5934	bool Validate() const;
5935	VkResult CheckCorruption(VmaAllocator hAllocator);
5936
5937	// ppData can be null.
5938	VkResult Map(VmaAllocator hAllocator, uint32_t count, void** ppData);
5939	void Unmap(VmaAllocator hAllocator, uint32_t count);
5940
5941	VkResult WriteMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize);
5942	VkResult ValidateMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize);
5943
5944	VkResult BindBufferMemory(
5945	const VmaAllocator hAllocator,
5946	const VmaAllocation hAllocation,
5947	VkDeviceSize allocationLocalOffset,
5948	VkBuffer hBuffer,
5949	const void* pNext);
5950	VkResult BindImageMemory(
5951	const VmaAllocator hAllocator,
5952	const VmaAllocation hAllocation,
5953	VkDeviceSize allocationLocalOffset,
5954	VkImage hImage,
5955	const void* pNext);
5956
5957	private:
5958	VmaPool m_hParentPool; // VK_NULL_HANDLE if not belongs to custom pool.
5959	uint32_t m_MemoryTypeIndex;
5960	uint32_t m_Id;
5961	VkDeviceMemory m_hMemory;
5962
5963	/*
5964	Protects access to m_hMemory so it is not used by multiple threads simultaneously, e.g. vkMapMemory, vkBindBufferMemory.
5965	Also protects m_MapCount, m_pMappedData.
5966	Allocations, deallocations, any change in m_pMetadata is protected by parent's VmaBlockVector::m_Mutex.
5967	*/
5968	VMA_MUTEX m_MapAndBindMutex;
5969	VmaMappingHysteresis m_MappingHysteresis;
5970	uint32_t m_MapCount;
5971	void* m_pMappedData;
5972	};
5973	#endif // _VMA_DEVICE_MEMORY_BLOCK
5974
5975	#ifndef _VMA_ALLOCATION_T
5976	struct VmaAllocation_T
5977	{
5978	friend struct VmaDedicatedAllocationListItemTraits;
5979
5980	enum FLAGS
5981	{
5982	FLAG_PERSISTENT_MAP = 0x01,
5983	FLAG_MAPPING_ALLOWED = 0x02,
5984	};
5985
5986	public:
5987	enum ALLOCATION_TYPE
5988	{
5989	ALLOCATION_TYPE_NONE,
5990	ALLOCATION_TYPE_BLOCK,
5991	ALLOCATION_TYPE_DEDICATED,
5992	};
5993
5994	// This struct is allocated using VmaPoolAllocator.
5995	VmaAllocation_T(bool mappingAllowed);
5996	~VmaAllocation_T();
5997
5998	void InitBlockAllocation(
5999	VmaDeviceMemoryBlock* block,
6000	VmaAllocHandle allocHandle,
6001	VkDeviceSize alignment,
6002	VkDeviceSize size,
6003	uint32_t memoryTypeIndex,
6004	VmaSuballocationType suballocationType,
6005	bool mapped);
6006	// pMappedData not null means allocation is created with MAPPED flag.
6007	void InitDedicatedAllocation(
6008	VmaPool hParentPool,
6009	uint32_t memoryTypeIndex,
6010	VkDeviceMemory hMemory,
6011	VmaSuballocationType suballocationType,
6012	void* pMappedData,
6013	VkDeviceSize size);
6014
6015	ALLOCATION_TYPE GetType() const { return (ALLOCATION_TYPE)m_Type; }
6016	VkDeviceSize GetAlignment() const { return m_Alignment; }
6017	VkDeviceSize GetSize() const { return m_Size; }
6018	void* GetUserData() const { return m_pUserData; }
6019	const char* GetName() const { return m_pName; }
6020	VmaSuballocationType GetSuballocationType() const { return (VmaSuballocationType)m_SuballocationType; }
6021
6022	VmaDeviceMemoryBlock* GetBlock() const { VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK); return m_BlockAllocation.m_Block; }
6023	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
6024	bool IsPersistentMap() const { return (m_Flags & FLAG_PERSISTENT_MAP) != 0; }
6025	bool IsMappingAllowed() const { return (m_Flags & FLAG_MAPPING_ALLOWED) != 0; }
6026
6027	void SetUserData(VmaAllocator hAllocator, void* pUserData) { m_pUserData = pUserData; }
6028	void SetName(VmaAllocator hAllocator, const char* pName);
6029	void FreeName(VmaAllocator hAllocator);
6030	uint8_t SwapBlockAllocation(VmaAllocator hAllocator, VmaAllocation allocation);
6031	VmaAllocHandle GetAllocHandle() const;
6032	VkDeviceSize GetOffset() const;
6033	VmaPool GetParentPool() const;
6034	VkDeviceMemory GetMemory() const;
6035	void* GetMappedData() const;
6036
6037	void BlockAllocMap();
6038	void BlockAllocUnmap();
6039	VkResult DedicatedAllocMap(VmaAllocator hAllocator, void** ppData);
6040	void DedicatedAllocUnmap(VmaAllocator hAllocator);
6041
6042	#if VMA_STATS_STRING_ENABLED
6043	uint32_t GetBufferImageUsage() const { return m_BufferImageUsage; }
6044
6045	void InitBufferImageUsage(uint32_t bufferImageUsage);
6046	void PrintParameters(class VmaJsonWriter& json) const;
6047	#endif
6048
6049	private:
6050	// Allocation out of VmaDeviceMemoryBlock.
6051	struct BlockAllocation
6052	{
6053	VmaDeviceMemoryBlock* m_Block;
6054	VmaAllocHandle m_AllocHandle;
6055	};
6056	// Allocation for an object that has its own private VkDeviceMemory.
6057	struct DedicatedAllocation
6058	{
6059	VmaPool m_hParentPool; // VK_NULL_HANDLE if not belongs to custom pool.
6060	VkDeviceMemory m_hMemory;
6061	void* m_pMappedData; // Not null means memory is mapped.
6062	VmaAllocation_T* m_Prev;
6063	VmaAllocation_T* m_Next;
6064	};
6065	union
6066	{
6067	// Allocation out of VmaDeviceMemoryBlock.
6068	BlockAllocation m_BlockAllocation;
6069	// Allocation for an object that has its own private VkDeviceMemory.
6070	DedicatedAllocation m_DedicatedAllocation;
6071	};
6072
6073	VkDeviceSize m_Alignment;
6074	VkDeviceSize m_Size;
6075	void* m_pUserData;
6076	char* m_pName;
6077	uint32_t m_MemoryTypeIndex;
6078	uint8_t m_Type; // ALLOCATION_TYPE
6079	uint8_t m_SuballocationType; // VmaSuballocationType
6080	// Reference counter for vmaMapMemory()/vmaUnmapMemory().
6081	uint8_t m_MapCount;
6082	uint8_t m_Flags; // enum FLAGS
6083	#if VMA_STATS_STRING_ENABLED
6084	uint32_t m_BufferImageUsage; // 0 if unknown.
6085	#endif
6086	};
6087	#endif // _VMA_ALLOCATION_T
6088
6089	#ifndef _VMA_DEDICATED_ALLOCATION_LIST_ITEM_TRAITS
6090	struct VmaDedicatedAllocationListItemTraits
6091	{
6092	typedef VmaAllocation_T ItemType;
6093
6094	static ItemType* GetPrev(const ItemType* item)
6095	{
6096	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6097	return item->m_DedicatedAllocation.m_Prev;
6098	}
6099	static ItemType* GetNext(const ItemType* item)
6100	{
6101	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6102	return item->m_DedicatedAllocation.m_Next;
6103	}
6104	static ItemType*& AccessPrev(ItemType* item)
6105	{
6106	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6107	return item->m_DedicatedAllocation.m_Prev;
6108	}
6109	static ItemType*& AccessNext(ItemType* item)
6110	{
6111	VMA_HEAVY_ASSERT(item->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
6112	return item->m_DedicatedAllocation.m_Next;
6113	}
6114	};
6115	#endif // _VMA_DEDICATED_ALLOCATION_LIST_ITEM_TRAITS
6116
6117	#ifndef _VMA_DEDICATED_ALLOCATION_LIST
6118	/*
6119	Stores linked list of VmaAllocation_T objects.
6120	Thread-safe, synchronized internally.
6121	*/
6122	class VmaDedicatedAllocationList
6123	{
6124	public:
6125	VmaDedicatedAllocationList() {}
6126	~VmaDedicatedAllocationList();
6127
6128	void Init(bool useMutex) { m_UseMutex = useMutex; }
6129	bool Validate();
6130
6131	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats);
6132	void AddStatistics(VmaStatistics& inoutStats);
6133	#if VMA_STATS_STRING_ENABLED
6134	// Writes JSON array with the list of allocations.
6135	void BuildStatsString(VmaJsonWriter& json);
6136	#endif
6137
6138	bool IsEmpty();
6139	void Register(VmaAllocation alloc);
6140	void Unregister(VmaAllocation alloc);
6141
6142	private:
6143	typedef VmaIntrusiveLinkedList<VmaDedicatedAllocationListItemTraits> DedicatedAllocationLinkedList;
6144
6145	bool m_UseMutex = true;
6146	VMA_RW_MUTEX m_Mutex;
6147	DedicatedAllocationLinkedList m_AllocationList;
6148	};
6149
6150	#ifndef _VMA_DEDICATED_ALLOCATION_LIST_FUNCTIONS
6151
6152	VmaDedicatedAllocationList::~VmaDedicatedAllocationList()
6153	{
6154	VMA_HEAVY_ASSERT(Validate());
6155
6156	if (!m_AllocationList.IsEmpty())
6157	{
6158	VMA_ASSERT(false && "Unfreed dedicated allocations found!");
6159	}
6160	}
6161
6162	bool VmaDedicatedAllocationList::Validate()
6163	{
6164	const size_t declaredCount = m_AllocationList.GetCount();
6165	size_t actualCount = 0;
6166	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6167	for (VmaAllocation alloc = m_AllocationList.Front();
6168	alloc != VMA_NULL; alloc = m_AllocationList.GetNext(item: alloc))
6169	{
6170	++actualCount;
6171	}
6172	VMA_VALIDATE(actualCount == declaredCount);
6173
6174	return true;
6175	}
6176
6177	void VmaDedicatedAllocationList::AddDetailedStatistics(VmaDetailedStatistics& inoutStats)
6178	{
6179	for(auto* item = m_AllocationList.Front(); item != nullptr; item = DedicatedAllocationLinkedList::GetNext(item))
6180	{
6181	const VkDeviceSize size = item->GetSize();
6182	inoutStats.statistics.blockCount++;
6183	inoutStats.statistics.blockBytes += size;
6184	VmaAddDetailedStatisticsAllocation(inoutStats, size: item->GetSize());
6185	}
6186	}
6187
6188	void VmaDedicatedAllocationList::AddStatistics(VmaStatistics& inoutStats)
6189	{
6190	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6191
6192	const uint32_t allocCount = (uint32_t)m_AllocationList.GetCount();
6193	inoutStats.blockCount += allocCount;
6194	inoutStats.allocationCount += allocCount;
6195
6196	for(auto* item = m_AllocationList.Front(); item != nullptr; item = DedicatedAllocationLinkedList::GetNext(item))
6197	{
6198	const VkDeviceSize size = item->GetSize();
6199	inoutStats.blockBytes += size;
6200	inoutStats.allocationBytes += size;
6201	}
6202	}
6203
6204	#if VMA_STATS_STRING_ENABLED
6205	void VmaDedicatedAllocationList::BuildStatsString(VmaJsonWriter& json)
6206	{
6207	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6208	json.BeginArray();
6209	for (VmaAllocation alloc = m_AllocationList.Front();
6210	alloc != VMA_NULL; alloc = m_AllocationList.GetNext(item: alloc))
6211	{
6212	json.BeginObject(singleLine: true);
6213	alloc->PrintParameters(json);
6214	json.EndObject();
6215	}
6216	json.EndArray();
6217	}
6218	#endif // VMA_STATS_STRING_ENABLED
6219
6220	bool VmaDedicatedAllocationList::IsEmpty()
6221	{
6222	VmaMutexLockRead lock(m_Mutex, m_UseMutex);
6223	return m_AllocationList.IsEmpty();
6224	}
6225
6226	void VmaDedicatedAllocationList::Register(VmaAllocation alloc)
6227	{
6228	VmaMutexLockWrite lock(m_Mutex, m_UseMutex);
6229	m_AllocationList.PushBack(item: alloc);
6230	}
6231
6232	void VmaDedicatedAllocationList::Unregister(VmaAllocation alloc)
6233	{
6234	VmaMutexLockWrite lock(m_Mutex, m_UseMutex);
6235	m_AllocationList.Remove(item: alloc);
6236	}
6237	#endif // _VMA_DEDICATED_ALLOCATION_LIST_FUNCTIONS
6238	#endif // _VMA_DEDICATED_ALLOCATION_LIST
6239
6240	#ifndef _VMA_SUBALLOCATION
6241	/*
6242	Represents a region of VmaDeviceMemoryBlock that is either assigned and returned as
6243	allocated memory block or free.
6244	*/
6245	struct VmaSuballocation
6246	{
6247	VkDeviceSize offset;
6248	VkDeviceSize size;
6249	void* userData;
6250	VmaSuballocationType type;
6251	};
6252
6253	// Comparator for offsets.
6254	struct VmaSuballocationOffsetLess
6255	{
6256	bool operator()(const VmaSuballocation& lhs, const VmaSuballocation& rhs) const
6257	{
6258	return lhs.offset < rhs.offset;
6259	}
6260	};
6261
6262	struct VmaSuballocationOffsetGreater
6263	{
6264	bool operator()(const VmaSuballocation& lhs, const VmaSuballocation& rhs) const
6265	{
6266	return lhs.offset > rhs.offset;
6267	}
6268	};
6269
6270	struct VmaSuballocationItemSizeLess
6271	{
6272	bool operator()(const VmaSuballocationList::iterator lhs,
6273	const VmaSuballocationList::iterator rhs) const
6274	{
6275	return lhs->size < rhs->size;
6276	}
6277
6278	bool operator()(const VmaSuballocationList::iterator lhs,
6279	VkDeviceSize rhsSize) const
6280	{
6281	return lhs->size < rhsSize;
6282	}
6283	};
6284	#endif // _VMA_SUBALLOCATION
6285
6286	#ifndef _VMA_ALLOCATION_REQUEST
6287	/*
6288	Parameters of planned allocation inside a VmaDeviceMemoryBlock.
6289	item points to a FREE suballocation.
6290	*/
6291	struct VmaAllocationRequest
6292	{
6293	VmaAllocHandle allocHandle;
6294	VkDeviceSize size;
6295	VmaSuballocationList::iterator item;
6296	void* customData;
6297	uint64_t algorithmData;
6298	VmaAllocationRequestType type;
6299	};
6300	#endif // _VMA_ALLOCATION_REQUEST
6301
6302	#ifndef _VMA_BLOCK_METADATA
6303	/*
6304	Data structure used for bookkeeping of allocations and unused ranges of memory
6305	in a single VkDeviceMemory block.
6306	*/
6307	class VmaBlockMetadata
6308	{
6309	public:
6310	// pAllocationCallbacks, if not null, must be owned externally - alive and unchanged for the whole lifetime of this object.
6311	VmaBlockMetadata(const VkAllocationCallbacks* pAllocationCallbacks,
6312	VkDeviceSize bufferImageGranularity, bool isVirtual);
6313	virtual ~VmaBlockMetadata() = default;
6314
6315	virtual void Init(VkDeviceSize size) { m_Size = size; }
6316	bool IsVirtual() const { return m_IsVirtual; }
6317	VkDeviceSize GetSize() const { return m_Size; }
6318
6319	// Validates all data structures inside this object. If not valid, returns false.
6320	virtual bool Validate() const = 0;
6321	virtual size_t GetAllocationCount() const = 0;
6322	virtual size_t GetFreeRegionsCount() const = 0;
6323	virtual VkDeviceSize GetSumFreeSize() const = 0;
6324	// Returns true if this block is empty - contains only single free suballocation.
6325	virtual bool IsEmpty() const = 0;
6326	virtual void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) = 0;
6327	virtual VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const = 0;
6328	virtual void* GetAllocationUserData(VmaAllocHandle allocHandle) const = 0;
6329
6330	virtual VmaAllocHandle GetAllocationListBegin() const = 0;
6331	virtual VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const = 0;
6332	virtual VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const = 0;
6333
6334	// Shouldn't modify blockCount.
6335	virtual void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const = 0;
6336	virtual void AddStatistics(VmaStatistics& inoutStats) const = 0;
6337
6338	#if VMA_STATS_STRING_ENABLED
6339	virtual void PrintDetailedMap(class VmaJsonWriter& json) const = 0;
6340	#endif
6341
6342	// Tries to find a place for suballocation with given parameters inside this block.
6343	// If succeeded, fills pAllocationRequest and returns true.
6344	// If failed, returns false.
6345	virtual bool CreateAllocationRequest(
6346	VkDeviceSize allocSize,
6347	VkDeviceSize allocAlignment,
6348	bool upperAddress,
6349	VmaSuballocationType allocType,
6350	// Always one of VMA_ALLOCATION_CREATE_STRATEGY_* or VMA_ALLOCATION_INTERNAL_STRATEGY_* flags.
6351	uint32_t strategy,
6352	VmaAllocationRequest* pAllocationRequest) = 0;
6353
6354	virtual VkResult CheckCorruption(const void* pBlockData) = 0;
6355
6356	// Makes actual allocation based on request. Request must already be checked and valid.
6357	virtual void Alloc(
6358	const VmaAllocationRequest& request,
6359	VmaSuballocationType type,
6360	void* userData) = 0;
6361
6362	// Frees suballocation assigned to given memory region.
6363	virtual void Free(VmaAllocHandle allocHandle) = 0;
6364
6365	// Frees all allocations.
6366	// Careful! Don't call it if there are VmaAllocation objects owned by userData of cleared allocations!
6367	virtual void Clear() = 0;
6368
6369	virtual void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) = 0;
6370	virtual void DebugLogAllAllocations() const = 0;
6371
6372	protected:
6373	const VkAllocationCallbacks* GetAllocationCallbacks() const { return m_pAllocationCallbacks; }
6374	VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
6375	VkDeviceSize GetDebugMargin() const { return IsVirtual() ? 0 : VMA_DEBUG_MARGIN; }
6376
6377	void DebugLogAllocation(VkDeviceSize offset, VkDeviceSize size, void* userData) const;
6378	#if VMA_STATS_STRING_ENABLED
6379	// mapRefCount == UINT32_MAX means unspecified.
6380	void PrintDetailedMap_Begin(class VmaJsonWriter& json,
6381	VkDeviceSize unusedBytes,
6382	size_t allocationCount,
6383	size_t unusedRangeCount) const;
6384	void PrintDetailedMap_Allocation(class VmaJsonWriter& json,
6385	VkDeviceSize offset, VkDeviceSize size, void* userData) const;
6386	void PrintDetailedMap_UnusedRange(class VmaJsonWriter& json,
6387	VkDeviceSize offset,
6388	VkDeviceSize size) const;
6389	void PrintDetailedMap_End(class VmaJsonWriter& json) const;
6390	#endif
6391
6392	private:
6393	VkDeviceSize m_Size;
6394	const VkAllocationCallbacks* m_pAllocationCallbacks;
6395	const VkDeviceSize m_BufferImageGranularity;
6396	const bool m_IsVirtual;
6397	};
6398
6399	#ifndef _VMA_BLOCK_METADATA_FUNCTIONS
6400	VmaBlockMetadata::VmaBlockMetadata(const VkAllocationCallbacks* pAllocationCallbacks,
6401	VkDeviceSize bufferImageGranularity, bool isVirtual)
6402	: m_Size(0),
6403	m_pAllocationCallbacks(pAllocationCallbacks),
6404	m_BufferImageGranularity(bufferImageGranularity),
6405	m_IsVirtual(isVirtual) {}
6406
6407	void VmaBlockMetadata::DebugLogAllocation(VkDeviceSize offset, VkDeviceSize size, void* userData) const
6408	{
6409	if (IsVirtual())
6410	{
6411	VMA_DEBUG_LOG("UNFREED VIRTUAL ALLOCATION; Offset: %llu; Size: %llu; UserData: %p", offset, size, userData);
6412	}
6413	else
6414	{
6415	VMA_ASSERT(userData != VMA_NULL);
6416	VmaAllocation allocation = reinterpret_cast<VmaAllocation>(userData);
6417
6418	userData = allocation->GetUserData();
6419	const char* name = allocation->GetName();
6420
6421	#if VMA_STATS_STRING_ENABLED
6422	VMA_DEBUG_LOG("UNFREED ALLOCATION; Offset: %llu; Size: %llu; UserData: %p; Name: %s; Type: %s; Usage: %u",
6423	offset, size, userData, name ? name : "vma_empty",
6424	VMA_SUBALLOCATION_TYPE_NAMES[allocation->GetSuballocationType()],
6425	allocation->GetBufferImageUsage());
6426	#else
6427	VMA_DEBUG_LOG("UNFREED ALLOCATION; Offset: %llu; Size: %llu; UserData: %p; Name: %s; Type: %u",
6428	offset, size, userData, name ? name : "vma_empty",
6429	(uint32_t)allocation->GetSuballocationType());
6430	#endif // VMA_STATS_STRING_ENABLED
6431	}
6432
6433	}
6434
6435	#if VMA_STATS_STRING_ENABLED
6436	void VmaBlockMetadata::PrintDetailedMap_Begin(class VmaJsonWriter& json,
6437	VkDeviceSize unusedBytes, size_t allocationCount, size_t unusedRangeCount) const
6438	{
6439	json.WriteString(pStr: "TotalBytes");
6440	json.WriteNumber(n: GetSize());
6441
6442	json.WriteString(pStr: "UnusedBytes");
6443	json.WriteSize(n: unusedBytes);
6444
6445	json.WriteString(pStr: "Allocations");
6446	json.WriteSize(n: allocationCount);
6447
6448	json.WriteString(pStr: "UnusedRanges");
6449	json.WriteSize(n: unusedRangeCount);
6450
6451	json.WriteString(pStr: "Suballocations");
6452	json.BeginArray();
6453	}
6454
6455	void VmaBlockMetadata::PrintDetailedMap_Allocation(class VmaJsonWriter& json,
6456	VkDeviceSize offset, VkDeviceSize size, void* userData) const
6457	{
6458	json.BeginObject(singleLine: true);
6459
6460	json.WriteString(pStr: "Offset");
6461	json.WriteNumber(n: offset);
6462
6463	if (IsVirtual())
6464	{
6465	json.WriteString(pStr: "Size");
6466	json.WriteNumber(n: size);
6467	if (userData)
6468	{
6469	json.WriteString(pStr: "CustomData");
6470	json.BeginString();
6471	json.ContinueString_Pointer(ptr: userData);
6472	json.EndString();
6473	}
6474	}
6475	else
6476	{
6477	((VmaAllocation)userData)->PrintParameters(json);
6478	}
6479
6480	json.EndObject();
6481	}
6482
6483	void VmaBlockMetadata::PrintDetailedMap_UnusedRange(class VmaJsonWriter& json,
6484	VkDeviceSize offset, VkDeviceSize size) const
6485	{
6486	json.BeginObject(singleLine: true);
6487
6488	json.WriteString(pStr: "Offset");
6489	json.WriteNumber(n: offset);
6490
6491	json.WriteString(pStr: "Type");
6492	json.WriteString(pStr: VMA_SUBALLOCATION_TYPE_NAMES[VMA_SUBALLOCATION_TYPE_FREE]);
6493
6494	json.WriteString(pStr: "Size");
6495	json.WriteNumber(n: size);
6496
6497	json.EndObject();
6498	}
6499
6500	void VmaBlockMetadata::PrintDetailedMap_End(class VmaJsonWriter& json) const
6501	{
6502	json.EndArray();
6503	}
6504	#endif // VMA_STATS_STRING_ENABLED
6505	#endif // _VMA_BLOCK_METADATA_FUNCTIONS
6506	#endif // _VMA_BLOCK_METADATA
6507
6508	#ifndef _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY
6509	// Before deleting object of this class remember to call 'Destroy()'
6510	class VmaBlockBufferImageGranularity final
6511	{
6512	public:
6513	struct ValidationContext
6514	{
6515	const VkAllocationCallbacks* allocCallbacks;
6516	uint16_t* pageAllocs;
6517	};
6518
6519	VmaBlockBufferImageGranularity(VkDeviceSize bufferImageGranularity);
6520	~VmaBlockBufferImageGranularity();
6521
6522	bool IsEnabled() const { return m_BufferImageGranularity > MAX_LOW_BUFFER_IMAGE_GRANULARITY; }
6523
6524	void Init(const VkAllocationCallbacks* pAllocationCallbacks, VkDeviceSize size);
6525	// Before destroying object you must call free it's memory
6526	void Destroy(const VkAllocationCallbacks* pAllocationCallbacks);
6527
6528	void RoundupAllocRequest(VmaSuballocationType allocType,
6529	VkDeviceSize& inOutAllocSize,
6530	VkDeviceSize& inOutAllocAlignment) const;
6531
6532	bool CheckConflictAndAlignUp(VkDeviceSize& inOutAllocOffset,
6533	VkDeviceSize allocSize,
6534	VkDeviceSize blockOffset,
6535	VkDeviceSize blockSize,
6536	VmaSuballocationType allocType) const;
6537
6538	void AllocPages(uint8_t allocType, VkDeviceSize offset, VkDeviceSize size);
6539	void FreePages(VkDeviceSize offset, VkDeviceSize size);
6540	void Clear();
6541
6542	ValidationContext StartValidation(const VkAllocationCallbacks* pAllocationCallbacks,
6543	bool isVirutal) const;
6544	bool Validate(ValidationContext& ctx, VkDeviceSize offset, VkDeviceSize size) const;
6545	bool FinishValidation(ValidationContext& ctx) const;
6546
6547	private:
6548	static const uint16_t MAX_LOW_BUFFER_IMAGE_GRANULARITY = 256;
6549
6550	struct RegionInfo
6551	{
6552	uint8_t allocType;
6553	uint16_t allocCount;
6554	};
6555
6556	VkDeviceSize m_BufferImageGranularity;
6557	uint32_t m_RegionCount;
6558	RegionInfo* m_RegionInfo;
6559
6560	uint32_t GetStartPage(VkDeviceSize offset) const { return OffsetToPageIndex(offset: offset & ~(m_BufferImageGranularity - 1)); }
6561	uint32_t GetEndPage(VkDeviceSize offset, VkDeviceSize size) const { return OffsetToPageIndex(offset: (offset + size - 1) & ~(m_BufferImageGranularity - 1)); }
6562
6563	uint32_t OffsetToPageIndex(VkDeviceSize offset) const;
6564	void AllocPage(RegionInfo& page, uint8_t allocType);
6565	};
6566
6567	#ifndef _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY_FUNCTIONS
6568	VmaBlockBufferImageGranularity::VmaBlockBufferImageGranularity(VkDeviceSize bufferImageGranularity)
6569	: m_BufferImageGranularity(bufferImageGranularity),
6570	m_RegionCount(0),
6571	m_RegionInfo(VMA_NULL) {}
6572
6573	VmaBlockBufferImageGranularity::~VmaBlockBufferImageGranularity()
6574	{
6575	VMA_ASSERT(m_RegionInfo == VMA_NULL && "Free not called before destroying object!");
6576	}
6577
6578	void VmaBlockBufferImageGranularity::Init(const VkAllocationCallbacks* pAllocationCallbacks, VkDeviceSize size)
6579	{
6580	if (IsEnabled())
6581	{
6582	m_RegionCount = static_cast<uint32_t>(VmaDivideRoundingUp(x: size, y: m_BufferImageGranularity));
6583	m_RegionInfo = vma_new_array(pAllocationCallbacks, RegionInfo, m_RegionCount);
6584	memset(s: m_RegionInfo, c: 0, n: m_RegionCount * sizeof(RegionInfo));
6585	}
6586	}
6587
6588	void VmaBlockBufferImageGranularity::Destroy(const VkAllocationCallbacks* pAllocationCallbacks)
6589	{
6590	if (m_RegionInfo)
6591	{
6592	vma_delete_array(pAllocationCallbacks, ptr: m_RegionInfo, count: m_RegionCount);
6593	m_RegionInfo = VMA_NULL;
6594	}
6595	}
6596
6597	void VmaBlockBufferImageGranularity::RoundupAllocRequest(VmaSuballocationType allocType,
6598	VkDeviceSize& inOutAllocSize,
6599	VkDeviceSize& inOutAllocAlignment) const
6600	{
6601	if (m_BufferImageGranularity > 1 &&
6602	m_BufferImageGranularity <= MAX_LOW_BUFFER_IMAGE_GRANULARITY)
6603	{
6604	if (allocType == VMA_SUBALLOCATION_TYPE_UNKNOWN ||
6605	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
6606	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL)
6607	{
6608	inOutAllocAlignment = VMA_MAX(inOutAllocAlignment, m_BufferImageGranularity);
6609	inOutAllocSize = VmaAlignUp(val: inOutAllocSize, alignment: m_BufferImageGranularity);
6610	}
6611	}
6612	}
6613
6614	bool VmaBlockBufferImageGranularity::CheckConflictAndAlignUp(VkDeviceSize& inOutAllocOffset,
6615	VkDeviceSize allocSize,
6616	VkDeviceSize blockOffset,
6617	VkDeviceSize blockSize,
6618	VmaSuballocationType allocType) const
6619	{
6620	if (IsEnabled())
6621	{
6622	uint32_t startPage = GetStartPage(offset: inOutAllocOffset);
6623	if (m_RegionInfo[startPage].allocCount > 0 &&
6624	VmaIsBufferImageGranularityConflict(suballocType1: static_cast<VmaSuballocationType>(m_RegionInfo[startPage].allocType), suballocType2: allocType))
6625	{
6626	inOutAllocOffset = VmaAlignUp(val: inOutAllocOffset, alignment: m_BufferImageGranularity);
6627	if (blockSize < allocSize + inOutAllocOffset - blockOffset)
6628	return true;
6629	++startPage;
6630	}
6631	uint32_t endPage = GetEndPage(offset: inOutAllocOffset, size: allocSize);
6632	if (endPage != startPage &&
6633	m_RegionInfo[endPage].allocCount > 0 &&
6634	VmaIsBufferImageGranularityConflict(suballocType1: static_cast<VmaSuballocationType>(m_RegionInfo[endPage].allocType), suballocType2: allocType))
6635	{
6636	return true;
6637	}
6638	}
6639	return false;
6640	}
6641
6642	void VmaBlockBufferImageGranularity::AllocPages(uint8_t allocType, VkDeviceSize offset, VkDeviceSize size)
6643	{
6644	if (IsEnabled())
6645	{
6646	uint32_t startPage = GetStartPage(offset);
6647	AllocPage(page&: m_RegionInfo[startPage], allocType);
6648
6649	uint32_t endPage = GetEndPage(offset, size);
6650	if (startPage != endPage)
6651	AllocPage(page&: m_RegionInfo[endPage], allocType);
6652	}
6653	}
6654
6655	void VmaBlockBufferImageGranularity::FreePages(VkDeviceSize offset, VkDeviceSize size)
6656	{
6657	if (IsEnabled())
6658	{
6659	uint32_t startPage = GetStartPage(offset);
6660	--m_RegionInfo[startPage].allocCount;
6661	if (m_RegionInfo[startPage].allocCount == 0)
6662	m_RegionInfo[startPage].allocType = VMA_SUBALLOCATION_TYPE_FREE;
6663	uint32_t endPage = GetEndPage(offset, size);
6664	if (startPage != endPage)
6665	{
6666	--m_RegionInfo[endPage].allocCount;
6667	if (m_RegionInfo[endPage].allocCount == 0)
6668	m_RegionInfo[endPage].allocType = VMA_SUBALLOCATION_TYPE_FREE;
6669	}
6670	}
6671	}
6672
6673	void VmaBlockBufferImageGranularity::Clear()
6674	{
6675	if (m_RegionInfo)
6676	memset(s: m_RegionInfo, c: 0, n: m_RegionCount * sizeof(RegionInfo));
6677	}
6678
6679	VmaBlockBufferImageGranularity::ValidationContext VmaBlockBufferImageGranularity::StartValidation(
6680	const VkAllocationCallbacks* pAllocationCallbacks, bool isVirutal) const
6681	{
6682	ValidationContext ctx{ .allocCallbacks: pAllocationCallbacks, VMA_NULL };
6683	if (!isVirutal && IsEnabled())
6684	{
6685	ctx.pageAllocs = vma_new_array(pAllocationCallbacks, uint16_t, m_RegionCount);
6686	memset(s: ctx.pageAllocs, c: 0, n: m_RegionCount * sizeof(uint16_t));
6687	}
6688	return ctx;
6689	}
6690
6691	bool VmaBlockBufferImageGranularity::Validate(ValidationContext& ctx,
6692	VkDeviceSize offset, VkDeviceSize size) const
6693	{
6694	if (IsEnabled())
6695	{
6696	uint32_t start = GetStartPage(offset);
6697	++ctx.pageAllocs[start];
6698	VMA_VALIDATE(m_RegionInfo[start].allocCount > 0);
6699
6700	uint32_t end = GetEndPage(offset, size);
6701	if (start != end)
6702	{
6703	++ctx.pageAllocs[end];
6704	VMA_VALIDATE(m_RegionInfo[end].allocCount > 0);
6705	}
6706	}
6707	return true;
6708	}
6709
6710	bool VmaBlockBufferImageGranularity::FinishValidation(ValidationContext& ctx) const
6711	{
6712	// Check proper page structure
6713	if (IsEnabled())
6714	{
6715	VMA_ASSERT(ctx.pageAllocs != VMA_NULL && "Validation context not initialized!");
6716
6717	for (uint32_t page = 0; page < m_RegionCount; ++page)
6718	{
6719	VMA_VALIDATE(ctx.pageAllocs[page] == m_RegionInfo[page].allocCount);
6720	}
6721	vma_delete_array(pAllocationCallbacks: ctx.allocCallbacks, ptr: ctx.pageAllocs, count: m_RegionCount);
6722	ctx.pageAllocs = VMA_NULL;
6723	}
6724	return true;
6725	}
6726
6727	uint32_t VmaBlockBufferImageGranularity::OffsetToPageIndex(VkDeviceSize offset) const
6728	{
6729	return static_cast<uint32_t>(offset >> VMA_BITSCAN_MSB(m_BufferImageGranularity));
6730	}
6731
6732	void VmaBlockBufferImageGranularity::AllocPage(RegionInfo& page, uint8_t allocType)
6733	{
6734	// When current alloc type is free then it can be overriden by new type
6735	if (page.allocCount == 0 || (page.allocCount > 0 && page.allocType == VMA_SUBALLOCATION_TYPE_FREE))
6736	page.allocType = allocType;
6737
6738	++page.allocCount;
6739	}
6740	#endif // _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY_FUNCTIONS
6741	#endif // _VMA_BLOCK_BUFFER_IMAGE_GRANULARITY
6742
6743	#if 0
6744	#ifndef _VMA_BLOCK_METADATA_GENERIC
6745	class VmaBlockMetadata_Generic : public VmaBlockMetadata
6746	{
6747	friend class VmaDefragmentationAlgorithm_Generic;
6748	friend class VmaDefragmentationAlgorithm_Fast;
6749	VMA_CLASS_NO_COPY(VmaBlockMetadata_Generic)
6750	public:
6751	VmaBlockMetadata_Generic(const VkAllocationCallbacks* pAllocationCallbacks,
6752	VkDeviceSize bufferImageGranularity, bool isVirtual);
6753	virtual ~VmaBlockMetadata_Generic() = default;
6754
6755	size_t GetAllocationCount() const override { return m_Suballocations.size() - m_FreeCount; }
6756	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize; }
6757	bool IsEmpty() const override { return (m_Suballocations.size() == 1) && (m_FreeCount == 1); }
6758	void Free(VmaAllocHandle allocHandle) override { FreeSuballocation(FindAtOffset((VkDeviceSize)allocHandle - 1)); }
6759	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - 1; };
6760
6761	void Init(VkDeviceSize size) override;
6762	bool Validate() const override;
6763
6764	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
6765	void AddStatistics(VmaStatistics& inoutStats) const override;
6766
6767	#if VMA_STATS_STRING_ENABLED
6768	void PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const override;
6769	#endif
6770
6771	bool CreateAllocationRequest(
6772	VkDeviceSize allocSize,
6773	VkDeviceSize allocAlignment,
6774	bool upperAddress,
6775	VmaSuballocationType allocType,
6776	uint32_t strategy,
6777	VmaAllocationRequest* pAllocationRequest) override;
6778
6779	VkResult CheckCorruption(const void* pBlockData) override;
6780
6781	void Alloc(
6782	const VmaAllocationRequest& request,
6783	VmaSuballocationType type,
6784	void* userData) override;
6785
6786	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
6787	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
6788	VmaAllocHandle GetAllocationListBegin() const override;
6789	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
6790	void Clear() override;
6791	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
6792	void DebugLogAllAllocations() const override;
6793
6794	private:
6795	uint32_t m_FreeCount;
6796	VkDeviceSize m_SumFreeSize;
6797	VmaSuballocationList m_Suballocations;
6798	// Suballocations that are free. Sorted by size, ascending.
6799	VmaVector<VmaSuballocationList::iterator, VmaStlAllocator<VmaSuballocationList::iterator>> m_FreeSuballocationsBySize;
6800
6801	VkDeviceSize AlignAllocationSize(VkDeviceSize size) const { return IsVirtual() ? size : VmaAlignUp(size, (VkDeviceSize)16); }
6802
6803	VmaSuballocationList::iterator FindAtOffset(VkDeviceSize offset) const;
6804	bool ValidateFreeSuballocationList() const;
6805
6806	// Checks if requested suballocation with given parameters can be placed in given pFreeSuballocItem.
6807	// If yes, fills pOffset and returns true. If no, returns false.
6808	bool CheckAllocation(
6809	VkDeviceSize allocSize,
6810	VkDeviceSize allocAlignment,
6811	VmaSuballocationType allocType,
6812	VmaSuballocationList::const_iterator suballocItem,
6813	VmaAllocHandle* pAllocHandle) const;
6814
6815	// Given free suballocation, it merges it with following one, which must also be free.
6816	void MergeFreeWithNext(VmaSuballocationList::iterator item);
6817	// Releases given suballocation, making it free.
6818	// Merges it with adjacent free suballocations if applicable.
6819	// Returns iterator to new free suballocation at this place.
6820	VmaSuballocationList::iterator FreeSuballocation(VmaSuballocationList::iterator suballocItem);
6821	// Given free suballocation, it inserts it into sorted list of
6822	// m_FreeSuballocationsBySize if it is suitable.
6823	void RegisterFreeSuballocation(VmaSuballocationList::iterator item);
6824	// Given free suballocation, it removes it from sorted list of
6825	// m_FreeSuballocationsBySize if it is suitable.
6826	void UnregisterFreeSuballocation(VmaSuballocationList::iterator item);
6827	};
6828
6829	#ifndef _VMA_BLOCK_METADATA_GENERIC_FUNCTIONS
6830	VmaBlockMetadata_Generic::VmaBlockMetadata_Generic(const VkAllocationCallbacks* pAllocationCallbacks,
6831	VkDeviceSize bufferImageGranularity, bool isVirtual)
6832	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
6833	m_FreeCount(0),
6834	m_SumFreeSize(0),
6835	m_Suballocations(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
6836	m_FreeSuballocationsBySize(VmaStlAllocator<VmaSuballocationList::iterator>(pAllocationCallbacks)) {}
6837
6838	void VmaBlockMetadata_Generic::Init(VkDeviceSize size)
6839	{
6840	VmaBlockMetadata::Init(size);
6841
6842	m_FreeCount = 1;
6843	m_SumFreeSize = size;
6844
6845	VmaSuballocation suballoc = {};
6846	suballoc.offset = 0;
6847	suballoc.size = size;
6848	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
6849
6850	m_Suballocations.push_back(suballoc);
6851	m_FreeSuballocationsBySize.push_back(m_Suballocations.begin());
6852	}
6853
6854	bool VmaBlockMetadata_Generic::Validate() const
6855	{
6856	VMA_VALIDATE(!m_Suballocations.empty());
6857
6858	// Expected offset of new suballocation as calculated from previous ones.
6859	VkDeviceSize calculatedOffset = 0;
6860	// Expected number of free suballocations as calculated from traversing their list.
6861	uint32_t calculatedFreeCount = 0;
6862	// Expected sum size of free suballocations as calculated from traversing their list.
6863	VkDeviceSize calculatedSumFreeSize = 0;
6864	// Expected number of free suballocations that should be registered in
6865	// m_FreeSuballocationsBySize calculated from traversing their list.
6866	size_t freeSuballocationsToRegister = 0;
6867	// True if previous visited suballocation was free.
6868	bool prevFree = false;
6869
6870	const VkDeviceSize debugMargin = GetDebugMargin();
6871
6872	for (const auto& subAlloc : m_Suballocations)
6873	{
6874	// Actual offset of this suballocation doesn't match expected one.
6875	VMA_VALIDATE(subAlloc.offset == calculatedOffset);
6876
6877	const bool currFree = (subAlloc.type == VMA_SUBALLOCATION_TYPE_FREE);
6878	// Two adjacent free suballocations are invalid. They should be merged.
6879	VMA_VALIDATE(!prevFree || !currFree);
6880
6881	VmaAllocation alloc = (VmaAllocation)subAlloc.userData;
6882	if (!IsVirtual())
6883	{
6884	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
6885	}
6886
6887	if (currFree)
6888	{
6889	calculatedSumFreeSize += subAlloc.size;
6890	++calculatedFreeCount;
6891	++freeSuballocationsToRegister;
6892
6893	// Margin required between allocations - every free space must be at least that large.
6894	VMA_VALIDATE(subAlloc.size >= debugMargin);
6895	}
6896	else
6897	{
6898	if (!IsVirtual())
6899	{
6900	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == subAlloc.offset + 1);
6901	VMA_VALIDATE(alloc->GetSize() == subAlloc.size);
6902	}
6903
6904	// Margin required between allocations - previous allocation must be free.
6905	VMA_VALIDATE(debugMargin == 0 || prevFree);
6906	}
6907
6908	calculatedOffset += subAlloc.size;
6909	prevFree = currFree;
6910	}
6911
6912	// Number of free suballocations registered in m_FreeSuballocationsBySize doesn't
6913	// match expected one.
6914	VMA_VALIDATE(m_FreeSuballocationsBySize.size() == freeSuballocationsToRegister);
6915
6916	VkDeviceSize lastSize = 0;
6917	for (size_t i = 0; i < m_FreeSuballocationsBySize.size(); ++i)
6918	{
6919	VmaSuballocationList::iterator suballocItem = m_FreeSuballocationsBySize[i];
6920
6921	// Only free suballocations can be registered in m_FreeSuballocationsBySize.
6922	VMA_VALIDATE(suballocItem->type == VMA_SUBALLOCATION_TYPE_FREE);
6923	// They must be sorted by size ascending.
6924	VMA_VALIDATE(suballocItem->size >= lastSize);
6925
6926	lastSize = suballocItem->size;
6927	}
6928
6929	// Check if totals match calculated values.
6930	VMA_VALIDATE(ValidateFreeSuballocationList());
6931	VMA_VALIDATE(calculatedOffset == GetSize());
6932	VMA_VALIDATE(calculatedSumFreeSize == m_SumFreeSize);
6933	VMA_VALIDATE(calculatedFreeCount == m_FreeCount);
6934
6935	return true;
6936	}
6937
6938	void VmaBlockMetadata_Generic::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
6939	{
6940	const uint32_t rangeCount = (uint32_t)m_Suballocations.size();
6941	inoutStats.statistics.blockCount++;
6942	inoutStats.statistics.blockBytes += GetSize();
6943
6944	for (const auto& suballoc : m_Suballocations)
6945	{
6946	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
6947	VmaAddDetailedStatisticsAllocation(inoutStats, suballoc.size);
6948	else
6949	VmaAddDetailedStatisticsUnusedRange(inoutStats, suballoc.size);
6950	}
6951	}
6952
6953	void VmaBlockMetadata_Generic::AddStatistics(VmaStatistics& inoutStats) const
6954	{
6955	inoutStats.blockCount++;
6956	inoutStats.allocationCount += (uint32_t)m_Suballocations.size() - m_FreeCount;
6957	inoutStats.blockBytes += GetSize();
6958	inoutStats.allocationBytes += GetSize() - m_SumFreeSize;
6959	}
6960
6961	#if VMA_STATS_STRING_ENABLED
6962	void VmaBlockMetadata_Generic::PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const
6963	{
6964	PrintDetailedMap_Begin(json,
6965	m_SumFreeSize, // unusedBytes
6966	m_Suballocations.size() - (size_t)m_FreeCount, // allocationCount
6967	m_FreeCount, // unusedRangeCount
6968	mapRefCount);
6969
6970	for (const auto& suballoc : m_Suballocations)
6971	{
6972	if (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE)
6973	{
6974	PrintDetailedMap_UnusedRange(json, suballoc.offset, suballoc.size);
6975	}
6976	else
6977	{
6978	PrintDetailedMap_Allocation(json, suballoc.offset, suballoc.size, suballoc.userData);
6979	}
6980	}
6981
6982	PrintDetailedMap_End(json);
6983	}
6984	#endif // VMA_STATS_STRING_ENABLED
6985
6986	bool VmaBlockMetadata_Generic::CreateAllocationRequest(
6987	VkDeviceSize allocSize,
6988	VkDeviceSize allocAlignment,
6989	bool upperAddress,
6990	VmaSuballocationType allocType,
6991	uint32_t strategy,
6992	VmaAllocationRequest* pAllocationRequest)
6993	{
6994	VMA_ASSERT(allocSize > 0);
6995	VMA_ASSERT(!upperAddress);
6996	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
6997	VMA_ASSERT(pAllocationRequest != VMA_NULL);
6998	VMA_HEAVY_ASSERT(Validate());
6999
7000	allocSize = AlignAllocationSize(allocSize);
7001
7002	pAllocationRequest->type = VmaAllocationRequestType::Normal;
7003	pAllocationRequest->size = allocSize;
7004
7005	const VkDeviceSize debugMargin = GetDebugMargin();
7006
7007	// There is not enough total free space in this block to fulfill the request: Early return.
7008	if (m_SumFreeSize < allocSize + debugMargin)
7009	{
7010	return false;
7011	}
7012
7013	// New algorithm, efficiently searching freeSuballocationsBySize.
7014	const size_t freeSuballocCount = m_FreeSuballocationsBySize.size();
7015	if (freeSuballocCount > 0)
7016	{
7017	if (strategy == 0 ||
7018	strategy == VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT)
7019	{
7020	// Find first free suballocation with size not less than allocSize + debugMargin.
7021	VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
7022	m_FreeSuballocationsBySize.data(),
7023	m_FreeSuballocationsBySize.data() + freeSuballocCount,
7024	allocSize + debugMargin,
7025	VmaSuballocationItemSizeLess());
7026	size_t index = it - m_FreeSuballocationsBySize.data();
7027	for (; index < freeSuballocCount; ++index)
7028	{
7029	if (CheckAllocation(
7030	allocSize,
7031	allocAlignment,
7032	allocType,
7033	m_FreeSuballocationsBySize[index],
7034	&pAllocationRequest->allocHandle))
7035	{
7036	pAllocationRequest->item = m_FreeSuballocationsBySize[index];
7037	return true;
7038	}
7039	}
7040	}
7041	else if (strategy == VMA_ALLOCATION_INTERNAL_STRATEGY_MIN_OFFSET)
7042	{
7043	for (VmaSuballocationList::iterator it = m_Suballocations.begin();
7044	it != m_Suballocations.end();
7045	++it)
7046	{
7047	if (it->type == VMA_SUBALLOCATION_TYPE_FREE && CheckAllocation(
7048	allocSize,
7049	allocAlignment,
7050	allocType,
7051	it,
7052	&pAllocationRequest->allocHandle))
7053	{
7054	pAllocationRequest->item = it;
7055	return true;
7056	}
7057	}
7058	}
7059	else
7060	{
7061	VMA_ASSERT(strategy & (VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT | VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT ));
7062	// Search staring from biggest suballocations.
7063	for (size_t index = freeSuballocCount; index--; )
7064	{
7065	if (CheckAllocation(
7066	allocSize,
7067	allocAlignment,
7068	allocType,
7069	m_FreeSuballocationsBySize[index],
7070	&pAllocationRequest->allocHandle))
7071	{
7072	pAllocationRequest->item = m_FreeSuballocationsBySize[index];
7073	return true;
7074	}
7075	}
7076	}
7077	}
7078
7079	return false;
7080	}
7081
7082	VkResult VmaBlockMetadata_Generic::CheckCorruption(const void* pBlockData)
7083	{
7084	for (auto& suballoc : m_Suballocations)
7085	{
7086	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7087	{
7088	if (!VmaValidateMagicValue(pBlockData, suballoc.offset + suballoc.size))
7089	{
7090	VMA_ASSERT(0 && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
7091	return VK_ERROR_UNKNOWN_COPY;
7092	}
7093	}
7094	}
7095
7096	return VK_SUCCESS;
7097	}
7098
7099	void VmaBlockMetadata_Generic::Alloc(
7100	const VmaAllocationRequest& request,
7101	VmaSuballocationType type,
7102	void* userData)
7103	{
7104	VMA_ASSERT(request.type == VmaAllocationRequestType::Normal);
7105	VMA_ASSERT(request.item != m_Suballocations.end());
7106	VmaSuballocation& suballoc = *request.item;
7107	// Given suballocation is a free block.
7108	VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7109
7110	// Given offset is inside this suballocation.
7111	VMA_ASSERT((VkDeviceSize)request.allocHandle - 1 >= suballoc.offset);
7112	const VkDeviceSize paddingBegin = (VkDeviceSize)request.allocHandle - suballoc.offset - 1;
7113	VMA_ASSERT(suballoc.size >= paddingBegin + request.size);
7114	const VkDeviceSize paddingEnd = suballoc.size - paddingBegin - request.size;
7115
7116	// Unregister this free suballocation from m_FreeSuballocationsBySize and update
7117	// it to become used.
7118	UnregisterFreeSuballocation(request.item);
7119
7120	suballoc.offset = (VkDeviceSize)request.allocHandle - 1;
7121	suballoc.size = request.size;
7122	suballoc.type = type;
7123	suballoc.userData = userData;
7124
7125	// If there are any free bytes remaining at the end, insert new free suballocation after current one.
7126	if (paddingEnd)
7127	{
7128	VmaSuballocation paddingSuballoc = {};
7129	paddingSuballoc.offset = suballoc.offset + suballoc.size;
7130	paddingSuballoc.size = paddingEnd;
7131	paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7132	VmaSuballocationList::iterator next = request.item;
7133	++next;
7134	const VmaSuballocationList::iterator paddingEndItem =
7135	m_Suballocations.insert(next, paddingSuballoc);
7136	RegisterFreeSuballocation(paddingEndItem);
7137	}
7138
7139	// If there are any free bytes remaining at the beginning, insert new free suballocation before current one.
7140	if (paddingBegin)
7141	{
7142	VmaSuballocation paddingSuballoc = {};
7143	paddingSuballoc.offset = suballoc.offset - paddingBegin;
7144	paddingSuballoc.size = paddingBegin;
7145	paddingSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7146	const VmaSuballocationList::iterator paddingBeginItem =
7147	m_Suballocations.insert(request.item, paddingSuballoc);
7148	RegisterFreeSuballocation(paddingBeginItem);
7149	}
7150
7151	// Update totals.
7152	m_FreeCount = m_FreeCount - 1;
7153	if (paddingBegin > 0)
7154	{
7155	++m_FreeCount;
7156	}
7157	if (paddingEnd > 0)
7158	{
7159	++m_FreeCount;
7160	}
7161	m_SumFreeSize -= request.size;
7162	}
7163
7164	void VmaBlockMetadata_Generic::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
7165	{
7166	outInfo.offset = (VkDeviceSize)allocHandle - 1;
7167	const VmaSuballocation& suballoc = *FindAtOffset(outInfo.offset);
7168	outInfo.size = suballoc.size;
7169	outInfo.pUserData = suballoc.userData;
7170	}
7171
7172	void* VmaBlockMetadata_Generic::GetAllocationUserData(VmaAllocHandle allocHandle) const
7173	{
7174	return FindAtOffset((VkDeviceSize)allocHandle - 1)->userData;
7175	}
7176
7177	VmaAllocHandle VmaBlockMetadata_Generic::GetAllocationListBegin() const
7178	{
7179	if (IsEmpty())
7180	return VK_NULL_HANDLE;
7181
7182	for (const auto& suballoc : m_Suballocations)
7183	{
7184	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7185	return (VmaAllocHandle)(suballoc.offset + 1);
7186	}
7187	VMA_ASSERT(false && "Should contain at least 1 allocation!");
7188	return VK_NULL_HANDLE;
7189	}
7190
7191	VmaAllocHandle VmaBlockMetadata_Generic::GetNextAllocation(VmaAllocHandle prevAlloc) const
7192	{
7193	VmaSuballocationList::const_iterator prev = FindAtOffset((VkDeviceSize)prevAlloc - 1);
7194
7195	for (VmaSuballocationList::const_iterator it = ++prev; it != m_Suballocations.end(); ++it)
7196	{
7197	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
7198	return (VmaAllocHandle)(it->offset + 1);
7199	}
7200	return VK_NULL_HANDLE;
7201	}
7202
7203	void VmaBlockMetadata_Generic::Clear()
7204	{
7205	const VkDeviceSize size = GetSize();
7206
7207	VMA_ASSERT(IsVirtual());
7208	m_FreeCount = 1;
7209	m_SumFreeSize = size;
7210	m_Suballocations.clear();
7211	m_FreeSuballocationsBySize.clear();
7212
7213	VmaSuballocation suballoc = {};
7214	suballoc.offset = 0;
7215	suballoc.size = size;
7216	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7217	m_Suballocations.push_back(suballoc);
7218
7219	m_FreeSuballocationsBySize.push_back(m_Suballocations.begin());
7220	}
7221
7222	void VmaBlockMetadata_Generic::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
7223	{
7224	VmaSuballocation& suballoc = *FindAtOffset((VkDeviceSize)allocHandle - 1);
7225	suballoc.userData = userData;
7226	}
7227
7228	void VmaBlockMetadata_Generic::DebugLogAllAllocations() const
7229	{
7230	for (const auto& suballoc : m_Suballocations)
7231	{
7232	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
7233	DebugLogAllocation(suballoc.offset, suballoc.size, suballoc.userData);
7234	}
7235	}
7236
7237	VmaSuballocationList::iterator VmaBlockMetadata_Generic::FindAtOffset(VkDeviceSize offset) const
7238	{
7239	VMA_HEAVY_ASSERT(!m_Suballocations.empty());
7240	const VkDeviceSize last = m_Suballocations.rbegin()->offset;
7241	if (last == offset)
7242	return m_Suballocations.rbegin().drop_const();
7243	const VkDeviceSize first = m_Suballocations.begin()->offset;
7244	if (first == offset)
7245	return m_Suballocations.begin().drop_const();
7246
7247	const size_t suballocCount = m_Suballocations.size();
7248	const VkDeviceSize step = (last - first + m_Suballocations.begin()->size) / suballocCount;
7249	auto findSuballocation = [&](auto begin, auto end) -> VmaSuballocationList::iterator
7250	{
7251	for (auto suballocItem = begin;
7252	suballocItem != end;
7253	++suballocItem)
7254	{
7255	if (suballocItem->offset == offset)
7256	return suballocItem.drop_const();
7257	}
7258	VMA_ASSERT(false && "Not found!");
7259	return m_Suballocations.end().drop_const();
7260	};
7261	// If requested offset is closer to the end of range, search from the end
7262	if (offset - first > suballocCount * step / 2)
7263	{
7264	return findSuballocation(m_Suballocations.rbegin(), m_Suballocations.rend());
7265	}
7266	return findSuballocation(m_Suballocations.begin(), m_Suballocations.end());
7267	}
7268
7269	bool VmaBlockMetadata_Generic::ValidateFreeSuballocationList() const
7270	{
7271	VkDeviceSize lastSize = 0;
7272	for (size_t i = 0, count = m_FreeSuballocationsBySize.size(); i < count; ++i)
7273	{
7274	const VmaSuballocationList::iterator it = m_FreeSuballocationsBySize[i];
7275
7276	VMA_VALIDATE(it->type == VMA_SUBALLOCATION_TYPE_FREE);
7277	VMA_VALIDATE(it->size >= lastSize);
7278	lastSize = it->size;
7279	}
7280	return true;
7281	}
7282
7283	bool VmaBlockMetadata_Generic::CheckAllocation(
7284	VkDeviceSize allocSize,
7285	VkDeviceSize allocAlignment,
7286	VmaSuballocationType allocType,
7287	VmaSuballocationList::const_iterator suballocItem,
7288	VmaAllocHandle* pAllocHandle) const
7289	{
7290	VMA_ASSERT(allocSize > 0);
7291	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
7292	VMA_ASSERT(suballocItem != m_Suballocations.cend());
7293	VMA_ASSERT(pAllocHandle != VMA_NULL);
7294
7295	const VkDeviceSize debugMargin = GetDebugMargin();
7296	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
7297
7298	const VmaSuballocation& suballoc = *suballocItem;
7299	VMA_ASSERT(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7300
7301	// Size of this suballocation is too small for this request: Early return.
7302	if (suballoc.size < allocSize)
7303	{
7304	return false;
7305	}
7306
7307	// Start from offset equal to beginning of this suballocation.
7308	VkDeviceSize offset = suballoc.offset + (suballocItem == m_Suballocations.cbegin() ? 0 : GetDebugMargin());
7309
7310	// Apply debugMargin from the end of previous alloc.
7311	if (debugMargin > 0)
7312	{
7313	offset += debugMargin;
7314	}
7315
7316	// Apply alignment.
7317	offset = VmaAlignUp(offset, allocAlignment);
7318
7319	// Check previous suballocations for BufferImageGranularity conflicts.
7320	// Make bigger alignment if necessary.
7321	if (bufferImageGranularity > 1 && bufferImageGranularity != allocAlignment)
7322	{
7323	bool bufferImageGranularityConflict = false;
7324	VmaSuballocationList::const_iterator prevSuballocItem = suballocItem;
7325	while (prevSuballocItem != m_Suballocations.cbegin())
7326	{
7327	--prevSuballocItem;
7328	const VmaSuballocation& prevSuballoc = *prevSuballocItem;
7329	if (VmaBlocksOnSamePage(prevSuballoc.offset, prevSuballoc.size, offset, bufferImageGranularity))
7330	{
7331	if (VmaIsBufferImageGranularityConflict(prevSuballoc.type, allocType))
7332	{
7333	bufferImageGranularityConflict = true;
7334	break;
7335	}
7336	}
7337	else
7338	// Already on previous page.
7339	break;
7340	}
7341	if (bufferImageGranularityConflict)
7342	{
7343	offset = VmaAlignUp(offset, bufferImageGranularity);
7344	}
7345	}
7346
7347	// Calculate padding at the beginning based on current offset.
7348	const VkDeviceSize paddingBegin = offset - suballoc.offset;
7349
7350	// Fail if requested size plus margin after is bigger than size of this suballocation.
7351	if (paddingBegin + allocSize + debugMargin > suballoc.size)
7352	{
7353	return false;
7354	}
7355
7356	// Check next suballocations for BufferImageGranularity conflicts.
7357	// If conflict exists, allocation cannot be made here.
7358	if (allocSize % bufferImageGranularity || offset % bufferImageGranularity)
7359	{
7360	VmaSuballocationList::const_iterator nextSuballocItem = suballocItem;
7361	++nextSuballocItem;
7362	while (nextSuballocItem != m_Suballocations.cend())
7363	{
7364	const VmaSuballocation& nextSuballoc = *nextSuballocItem;
7365	if (VmaBlocksOnSamePage(offset, allocSize, nextSuballoc.offset, bufferImageGranularity))
7366	{
7367	if (VmaIsBufferImageGranularityConflict(allocType, nextSuballoc.type))
7368	{
7369	return false;
7370	}
7371	}
7372	else
7373	{
7374	// Already on next page.
7375	break;
7376	}
7377	++nextSuballocItem;
7378	}
7379	}
7380
7381	*pAllocHandle = (VmaAllocHandle)(offset + 1);
7382	// All tests passed: Success. pAllocHandle is already filled.
7383	return true;
7384	}
7385
7386	void VmaBlockMetadata_Generic::MergeFreeWithNext(VmaSuballocationList::iterator item)
7387	{
7388	VMA_ASSERT(item != m_Suballocations.end());
7389	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7390
7391	VmaSuballocationList::iterator nextItem = item;
7392	++nextItem;
7393	VMA_ASSERT(nextItem != m_Suballocations.end());
7394	VMA_ASSERT(nextItem->type == VMA_SUBALLOCATION_TYPE_FREE);
7395
7396	item->size += nextItem->size;
7397	--m_FreeCount;
7398	m_Suballocations.erase(nextItem);
7399	}
7400
7401	VmaSuballocationList::iterator VmaBlockMetadata_Generic::FreeSuballocation(VmaSuballocationList::iterator suballocItem)
7402	{
7403	// Change this suballocation to be marked as free.
7404	VmaSuballocation& suballoc = *suballocItem;
7405	suballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
7406	suballoc.userData = VMA_NULL;
7407
7408	// Update totals.
7409	++m_FreeCount;
7410	m_SumFreeSize += suballoc.size;
7411
7412	// Merge with previous and/or next suballocation if it's also free.
7413	bool mergeWithNext = false;
7414	bool mergeWithPrev = false;
7415
7416	VmaSuballocationList::iterator nextItem = suballocItem;
7417	++nextItem;
7418	if ((nextItem != m_Suballocations.end()) && (nextItem->type == VMA_SUBALLOCATION_TYPE_FREE))
7419	{
7420	mergeWithNext = true;
7421	}
7422
7423	VmaSuballocationList::iterator prevItem = suballocItem;
7424	if (suballocItem != m_Suballocations.begin())
7425	{
7426	--prevItem;
7427	if (prevItem->type == VMA_SUBALLOCATION_TYPE_FREE)
7428	{
7429	mergeWithPrev = true;
7430	}
7431	}
7432
7433	if (mergeWithNext)
7434	{
7435	UnregisterFreeSuballocation(nextItem);
7436	MergeFreeWithNext(suballocItem);
7437	}
7438
7439	if (mergeWithPrev)
7440	{
7441	UnregisterFreeSuballocation(prevItem);
7442	MergeFreeWithNext(prevItem);
7443	RegisterFreeSuballocation(prevItem);
7444	return prevItem;
7445	}
7446	else
7447	{
7448	RegisterFreeSuballocation(suballocItem);
7449	return suballocItem;
7450	}
7451	}
7452
7453	void VmaBlockMetadata_Generic::RegisterFreeSuballocation(VmaSuballocationList::iterator item)
7454	{
7455	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7456	VMA_ASSERT(item->size > 0);
7457
7458	// You may want to enable this validation at the beginning or at the end of
7459	// this function, depending on what do you want to check.
7460	VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7461
7462	if (m_FreeSuballocationsBySize.empty())
7463	{
7464	m_FreeSuballocationsBySize.push_back(item);
7465	}
7466	else
7467	{
7468	VmaVectorInsertSorted<VmaSuballocationItemSizeLess>(m_FreeSuballocationsBySize, item);
7469	}
7470
7471	//VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7472	}
7473
7474	void VmaBlockMetadata_Generic::UnregisterFreeSuballocation(VmaSuballocationList::iterator item)
7475	{
7476	VMA_ASSERT(item->type == VMA_SUBALLOCATION_TYPE_FREE);
7477	VMA_ASSERT(item->size > 0);
7478
7479	// You may want to enable this validation at the beginning or at the end of
7480	// this function, depending on what do you want to check.
7481	VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7482
7483	VmaSuballocationList::iterator* const it = VmaBinaryFindFirstNotLess(
7484	m_FreeSuballocationsBySize.data(),
7485	m_FreeSuballocationsBySize.data() + m_FreeSuballocationsBySize.size(),
7486	item,
7487	VmaSuballocationItemSizeLess());
7488	for (size_t index = it - m_FreeSuballocationsBySize.data();
7489	index < m_FreeSuballocationsBySize.size();
7490	++index)
7491	{
7492	if (m_FreeSuballocationsBySize[index] == item)
7493	{
7494	VmaVectorRemove(m_FreeSuballocationsBySize, index);
7495	return;
7496	}
7497	VMA_ASSERT((m_FreeSuballocationsBySize[index]->size == item->size) && "Not found.");
7498	}
7499	VMA_ASSERT(0 && "Not found.");
7500
7501	//VMA_HEAVY_ASSERT(ValidateFreeSuballocationList());
7502	}
7503	#endif // _VMA_BLOCK_METADATA_GENERIC_FUNCTIONS
7504	#endif // _VMA_BLOCK_METADATA_GENERIC
7505	#endif // #if 0
7506
7507	#ifndef _VMA_BLOCK_METADATA_LINEAR
7508	/*
7509	Allocations and their references in internal data structure look like this:
7510
7511	if(m_2ndVectorMode == SECOND_VECTOR_EMPTY):
7512
7513	0 +-------+
7514	| |
7515	| |
7516	| |
7517	+-------+
7518	| Alloc | 1st[m_1stNullItemsBeginCount]
7519	+-------+
7520	| Alloc | 1st[m_1stNullItemsBeginCount + 1]
7521	+-------+
7522	| ... |
7523	+-------+
7524	| Alloc | 1st[1st.size() - 1]
7525	+-------+
7526	| |
7527	| |
7528	| |
7529	GetSize() +-------+
7530
7531	if(m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER):
7532
7533	0 +-------+
7534	| Alloc | 2nd[0]
7535	+-------+
7536	| Alloc | 2nd[1]
7537	+-------+
7538	| ... |
7539	+-------+
7540	| Alloc | 2nd[2nd.size() - 1]
7541	+-------+
7542	| |
7543	| |
7544	| |
7545	+-------+
7546	| Alloc | 1st[m_1stNullItemsBeginCount]
7547	+-------+
7548	| Alloc | 1st[m_1stNullItemsBeginCount + 1]
7549	+-------+
7550	| ... |
7551	+-------+
7552	| Alloc | 1st[1st.size() - 1]
7553	+-------+
7554	| |
7555	GetSize() +-------+
7556
7557	if(m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK):
7558
7559	0 +-------+
7560	| |
7561	| |
7562	| |
7563	+-------+
7564	| Alloc | 1st[m_1stNullItemsBeginCount]
7565	+-------+
7566	| Alloc | 1st[m_1stNullItemsBeginCount + 1]
7567	+-------+
7568	| ... |
7569	+-------+
7570	| Alloc | 1st[1st.size() - 1]
7571	+-------+
7572	| |
7573	| |
7574	| |
7575	+-------+
7576	| Alloc | 2nd[2nd.size() - 1]
7577	+-------+
7578	| ... |
7579	+-------+
7580	| Alloc | 2nd[1]
7581	+-------+
7582	| Alloc | 2nd[0]
7583	GetSize() +-------+
7584
7585	*/
7586	class VmaBlockMetadata_Linear : public VmaBlockMetadata
7587	{
7588	VMA_CLASS_NO_COPY(VmaBlockMetadata_Linear)
7589	public:
7590	VmaBlockMetadata_Linear(const VkAllocationCallbacks* pAllocationCallbacks,
7591	VkDeviceSize bufferImageGranularity, bool isVirtual);
7592	virtual ~VmaBlockMetadata_Linear() = default;
7593
7594	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize; }
7595	bool IsEmpty() const override { return GetAllocationCount() == 0; }
7596	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - 1; };
7597
7598	void Init(VkDeviceSize size) override;
7599	bool Validate() const override;
7600	size_t GetAllocationCount() const override;
7601	size_t GetFreeRegionsCount() const override;
7602
7603	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
7604	void AddStatistics(VmaStatistics& inoutStats) const override;
7605
7606	#if VMA_STATS_STRING_ENABLED
7607	void PrintDetailedMap(class VmaJsonWriter& json) const override;
7608	#endif
7609
7610	bool CreateAllocationRequest(
7611	VkDeviceSize allocSize,
7612	VkDeviceSize allocAlignment,
7613	bool upperAddress,
7614	VmaSuballocationType allocType,
7615	uint32_t strategy,
7616	VmaAllocationRequest* pAllocationRequest) override;
7617
7618	VkResult CheckCorruption(const void* pBlockData) override;
7619
7620	void Alloc(
7621	const VmaAllocationRequest& request,
7622	VmaSuballocationType type,
7623	void* userData) override;
7624
7625	void Free(VmaAllocHandle allocHandle) override;
7626	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
7627	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
7628	VmaAllocHandle GetAllocationListBegin() const override;
7629	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
7630	VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const override;
7631	void Clear() override;
7632	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
7633	void DebugLogAllAllocations() const override;
7634
7635	private:
7636	/*
7637	There are two suballocation vectors, used in ping-pong way.
7638	The one with index m_1stVectorIndex is called 1st.
7639	The one with index (m_1stVectorIndex ^ 1) is called 2nd.
7640	2nd can be non-empty only when 1st is not empty.
7641	When 2nd is not empty, m_2ndVectorMode indicates its mode of operation.
7642	*/
7643	typedef VmaVector<VmaSuballocation, VmaStlAllocator<VmaSuballocation>> SuballocationVectorType;
7644
7645	enum SECOND_VECTOR_MODE
7646	{
7647	SECOND_VECTOR_EMPTY,
7648	/*
7649	Suballocations in 2nd vector are created later than the ones in 1st, but they
7650	all have smaller offset.
7651	*/
7652	SECOND_VECTOR_RING_BUFFER,
7653	/*
7654	Suballocations in 2nd vector are upper side of double stack.
7655	They all have offsets higher than those in 1st vector.
7656	Top of this stack means smaller offsets, but higher indices in this vector.
7657	*/
7658	SECOND_VECTOR_DOUBLE_STACK,
7659	};
7660
7661	VkDeviceSize m_SumFreeSize;
7662	SuballocationVectorType m_Suballocations0, m_Suballocations1;
7663	uint32_t m_1stVectorIndex;
7664	SECOND_VECTOR_MODE m_2ndVectorMode;
7665	// Number of items in 1st vector with hAllocation = null at the beginning.
7666	size_t m_1stNullItemsBeginCount;
7667	// Number of other items in 1st vector with hAllocation = null somewhere in the middle.
7668	size_t m_1stNullItemsMiddleCount;
7669	// Number of items in 2nd vector with hAllocation = null.
7670	size_t m_2ndNullItemsCount;
7671
7672	SuballocationVectorType& AccessSuballocations1st() { return m_1stVectorIndex ? m_Suballocations1 : m_Suballocations0; }
7673	SuballocationVectorType& AccessSuballocations2nd() { return m_1stVectorIndex ? m_Suballocations0 : m_Suballocations1; }
7674	const SuballocationVectorType& AccessSuballocations1st() const { return m_1stVectorIndex ? m_Suballocations1 : m_Suballocations0; }
7675	const SuballocationVectorType& AccessSuballocations2nd() const { return m_1stVectorIndex ? m_Suballocations0 : m_Suballocations1; }
7676
7677	VmaSuballocation& FindSuballocation(VkDeviceSize offset) const;
7678	bool ShouldCompact1st() const;
7679	void CleanupAfterFree();
7680
7681	bool CreateAllocationRequest_LowerAddress(
7682	VkDeviceSize allocSize,
7683	VkDeviceSize allocAlignment,
7684	VmaSuballocationType allocType,
7685	uint32_t strategy,
7686	VmaAllocationRequest* pAllocationRequest);
7687	bool CreateAllocationRequest_UpperAddress(
7688	VkDeviceSize allocSize,
7689	VkDeviceSize allocAlignment,
7690	VmaSuballocationType allocType,
7691	uint32_t strategy,
7692	VmaAllocationRequest* pAllocationRequest);
7693	};
7694
7695	#ifndef _VMA_BLOCK_METADATA_LINEAR_FUNCTIONS
7696	VmaBlockMetadata_Linear::VmaBlockMetadata_Linear(const VkAllocationCallbacks* pAllocationCallbacks,
7697	VkDeviceSize bufferImageGranularity, bool isVirtual)
7698	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
7699	m_SumFreeSize(0),
7700	m_Suballocations0(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
7701	m_Suballocations1(VmaStlAllocator<VmaSuballocation>(pAllocationCallbacks)),
7702	m_1stVectorIndex(0),
7703	m_2ndVectorMode(SECOND_VECTOR_EMPTY),
7704	m_1stNullItemsBeginCount(0),
7705	m_1stNullItemsMiddleCount(0),
7706	m_2ndNullItemsCount(0) {}
7707
7708	void VmaBlockMetadata_Linear::Init(VkDeviceSize size)
7709	{
7710	VmaBlockMetadata::Init(size);
7711	m_SumFreeSize = size;
7712	}
7713
7714	bool VmaBlockMetadata_Linear::Validate() const
7715	{
7716	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
7717	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
7718
7719	VMA_VALIDATE(suballocations2nd.empty() == (m_2ndVectorMode == SECOND_VECTOR_EMPTY));
7720	VMA_VALIDATE(!suballocations1st.empty() ||
7721	suballocations2nd.empty() ||
7722	m_2ndVectorMode != SECOND_VECTOR_RING_BUFFER);
7723
7724	if (!suballocations1st.empty())
7725	{
7726	// Null item at the beginning should be accounted into m_1stNullItemsBeginCount.
7727	VMA_VALIDATE(suballocations1st[m_1stNullItemsBeginCount].type != VMA_SUBALLOCATION_TYPE_FREE);
7728	// Null item at the end should be just pop_back().
7729	VMA_VALIDATE(suballocations1st.back().type != VMA_SUBALLOCATION_TYPE_FREE);
7730	}
7731	if (!suballocations2nd.empty())
7732	{
7733	// Null item at the end should be just pop_back().
7734	VMA_VALIDATE(suballocations2nd.back().type != VMA_SUBALLOCATION_TYPE_FREE);
7735	}
7736
7737	VMA_VALIDATE(m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount <= suballocations1st.size());
7738	VMA_VALIDATE(m_2ndNullItemsCount <= suballocations2nd.size());
7739
7740	VkDeviceSize sumUsedSize = 0;
7741	const size_t suballoc1stCount = suballocations1st.size();
7742	const VkDeviceSize debugMargin = GetDebugMargin();
7743	VkDeviceSize offset = 0;
7744
7745	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
7746	{
7747	const size_t suballoc2ndCount = suballocations2nd.size();
7748	size_t nullItem2ndCount = 0;
7749	for (size_t i = 0; i < suballoc2ndCount; ++i)
7750	{
7751	const VmaSuballocation& suballoc = suballocations2nd[i];
7752	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7753
7754	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7755	if (!IsVirtual())
7756	{
7757	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7758	}
7759	VMA_VALIDATE(suballoc.offset >= offset);
7760
7761	if (!currFree)
7762	{
7763	if (!IsVirtual())
7764	{
7765	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + 1);
7766	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7767	}
7768	sumUsedSize += suballoc.size;
7769	}
7770	else
7771	{
7772	++nullItem2ndCount;
7773	}
7774
7775	offset = suballoc.offset + suballoc.size + debugMargin;
7776	}
7777
7778	VMA_VALIDATE(nullItem2ndCount == m_2ndNullItemsCount);
7779	}
7780
7781	for (size_t i = 0; i < m_1stNullItemsBeginCount; ++i)
7782	{
7783	const VmaSuballocation& suballoc = suballocations1st[i];
7784	VMA_VALIDATE(suballoc.type == VMA_SUBALLOCATION_TYPE_FREE &&
7785	suballoc.userData == VMA_NULL);
7786	}
7787
7788	size_t nullItem1stCount = m_1stNullItemsBeginCount;
7789
7790	for (size_t i = m_1stNullItemsBeginCount; i < suballoc1stCount; ++i)
7791	{
7792	const VmaSuballocation& suballoc = suballocations1st[i];
7793	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7794
7795	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7796	if (!IsVirtual())
7797	{
7798	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7799	}
7800	VMA_VALIDATE(suballoc.offset >= offset);
7801	VMA_VALIDATE(i >= m_1stNullItemsBeginCount || currFree);
7802
7803	if (!currFree)
7804	{
7805	if (!IsVirtual())
7806	{
7807	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + 1);
7808	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7809	}
7810	sumUsedSize += suballoc.size;
7811	}
7812	else
7813	{
7814	++nullItem1stCount;
7815	}
7816
7817	offset = suballoc.offset + suballoc.size + debugMargin;
7818	}
7819	VMA_VALIDATE(nullItem1stCount == m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount);
7820
7821	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
7822	{
7823	const size_t suballoc2ndCount = suballocations2nd.size();
7824	size_t nullItem2ndCount = 0;
7825	for (size_t i = suballoc2ndCount; i--; )
7826	{
7827	const VmaSuballocation& suballoc = suballocations2nd[i];
7828	const bool currFree = (suballoc.type == VMA_SUBALLOCATION_TYPE_FREE);
7829
7830	VmaAllocation const alloc = (VmaAllocation)suballoc.userData;
7831	if (!IsVirtual())
7832	{
7833	VMA_VALIDATE(currFree == (alloc == VK_NULL_HANDLE));
7834	}
7835	VMA_VALIDATE(suballoc.offset >= offset);
7836
7837	if (!currFree)
7838	{
7839	if (!IsVirtual())
7840	{
7841	VMA_VALIDATE((VkDeviceSize)alloc->GetAllocHandle() == suballoc.offset + 1);
7842	VMA_VALIDATE(alloc->GetSize() == suballoc.size);
7843	}
7844	sumUsedSize += suballoc.size;
7845	}
7846	else
7847	{
7848	++nullItem2ndCount;
7849	}
7850
7851	offset = suballoc.offset + suballoc.size + debugMargin;
7852	}
7853
7854	VMA_VALIDATE(nullItem2ndCount == m_2ndNullItemsCount);
7855	}
7856
7857	VMA_VALIDATE(offset <= GetSize());
7858	VMA_VALIDATE(m_SumFreeSize == GetSize() - sumUsedSize);
7859
7860	return true;
7861	}
7862
7863	size_t VmaBlockMetadata_Linear::GetAllocationCount() const
7864	{
7865	return AccessSuballocations1st().size() - m_1stNullItemsBeginCount - m_1stNullItemsMiddleCount +
7866	AccessSuballocations2nd().size() - m_2ndNullItemsCount;
7867	}
7868
7869	size_t VmaBlockMetadata_Linear::GetFreeRegionsCount() const
7870	{
7871	// Function only used for defragmentation, which is disabled for this algorithm
7872	VMA_ASSERT(0);
7873	return SIZE_MAX;
7874	}
7875
7876	void VmaBlockMetadata_Linear::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
7877	{
7878	const VkDeviceSize size = GetSize();
7879	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
7880	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
7881	const size_t suballoc1stCount = suballocations1st.size();
7882	const size_t suballoc2ndCount = suballocations2nd.size();
7883
7884	inoutStats.statistics.blockCount++;
7885	inoutStats.statistics.blockBytes += size;
7886
7887	VkDeviceSize lastOffset = 0;
7888
7889	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
7890	{
7891	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
7892	size_t nextAlloc2ndIndex = 0;
7893	while (lastOffset < freeSpace2ndTo1stEnd)
7894	{
7895	// Find next non-null allocation or move nextAllocIndex to the end.
7896	while (nextAlloc2ndIndex < suballoc2ndCount &&
7897	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
7898	{
7899	++nextAlloc2ndIndex;
7900	}
7901
7902	// Found non-null allocation.
7903	if (nextAlloc2ndIndex < suballoc2ndCount)
7904	{
7905	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
7906
7907	// 1. Process free space before this allocation.
7908	if (lastOffset < suballoc.offset)
7909	{
7910	// There is free space from lastOffset to suballoc.offset.
7911	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
7912	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
7913	}
7914
7915	// 2. Process this allocation.
7916	// There is allocation with suballoc.offset, suballoc.size.
7917	VmaAddDetailedStatisticsAllocation(inoutStats, size: suballoc.size);
7918
7919	// 3. Prepare for next iteration.
7920	lastOffset = suballoc.offset + suballoc.size;
7921	++nextAlloc2ndIndex;
7922	}
7923	// We are at the end.
7924	else
7925	{
7926	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
7927	if (lastOffset < freeSpace2ndTo1stEnd)
7928	{
7929	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
7930	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
7931	}
7932
7933	// End of loop.
7934	lastOffset = freeSpace2ndTo1stEnd;
7935	}
7936	}
7937	}
7938
7939	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
7940	const VkDeviceSize freeSpace1stTo2ndEnd =
7941	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
7942	while (lastOffset < freeSpace1stTo2ndEnd)
7943	{
7944	// Find next non-null allocation or move nextAllocIndex to the end.
7945	while (nextAlloc1stIndex < suballoc1stCount &&
7946	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
7947	{
7948	++nextAlloc1stIndex;
7949	}
7950
7951	// Found non-null allocation.
7952	if (nextAlloc1stIndex < suballoc1stCount)
7953	{
7954	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
7955
7956	// 1. Process free space before this allocation.
7957	if (lastOffset < suballoc.offset)
7958	{
7959	// There is free space from lastOffset to suballoc.offset.
7960	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
7961	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
7962	}
7963
7964	// 2. Process this allocation.
7965	// There is allocation with suballoc.offset, suballoc.size.
7966	VmaAddDetailedStatisticsAllocation(inoutStats, size: suballoc.size);
7967
7968	// 3. Prepare for next iteration.
7969	lastOffset = suballoc.offset + suballoc.size;
7970	++nextAlloc1stIndex;
7971	}
7972	// We are at the end.
7973	else
7974	{
7975	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
7976	if (lastOffset < freeSpace1stTo2ndEnd)
7977	{
7978	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
7979	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
7980	}
7981
7982	// End of loop.
7983	lastOffset = freeSpace1stTo2ndEnd;
7984	}
7985	}
7986
7987	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
7988	{
7989	size_t nextAlloc2ndIndex = suballocations2nd.size() - 1;
7990	while (lastOffset < size)
7991	{
7992	// Find next non-null allocation or move nextAllocIndex to the end.
7993	while (nextAlloc2ndIndex != SIZE_MAX &&
7994	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
7995	{
7996	--nextAlloc2ndIndex;
7997	}
7998
7999	// Found non-null allocation.
8000	if (nextAlloc2ndIndex != SIZE_MAX)
8001	{
8002	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8003
8004	// 1. Process free space before this allocation.
8005	if (lastOffset < suballoc.offset)
8006	{
8007	// There is free space from lastOffset to suballoc.offset.
8008	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8009	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
8010	}
8011
8012	// 2. Process this allocation.
8013	// There is allocation with suballoc.offset, suballoc.size.
8014	VmaAddDetailedStatisticsAllocation(inoutStats, size: suballoc.size);
8015
8016	// 3. Prepare for next iteration.
8017	lastOffset = suballoc.offset + suballoc.size;
8018	--nextAlloc2ndIndex;
8019	}
8020	// We are at the end.
8021	else
8022	{
8023	// There is free space from lastOffset to size.
8024	if (lastOffset < size)
8025	{
8026	const VkDeviceSize unusedRangeSize = size - lastOffset;
8027	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: unusedRangeSize);
8028	}
8029
8030	// End of loop.
8031	lastOffset = size;
8032	}
8033	}
8034	}
8035	}
8036
8037	void VmaBlockMetadata_Linear::AddStatistics(VmaStatistics& inoutStats) const
8038	{
8039	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8040	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8041	const VkDeviceSize size = GetSize();
8042	const size_t suballoc1stCount = suballocations1st.size();
8043	const size_t suballoc2ndCount = suballocations2nd.size();
8044
8045	inoutStats.blockCount++;
8046	inoutStats.blockBytes += size;
8047	inoutStats.allocationBytes += size - m_SumFreeSize;
8048
8049	VkDeviceSize lastOffset = 0;
8050
8051	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8052	{
8053	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8054	size_t nextAlloc2ndIndex = m_1stNullItemsBeginCount;
8055	while (lastOffset < freeSpace2ndTo1stEnd)
8056	{
8057	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8058	while (nextAlloc2ndIndex < suballoc2ndCount &&
8059	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8060	{
8061	++nextAlloc2ndIndex;
8062	}
8063
8064	// Found non-null allocation.
8065	if (nextAlloc2ndIndex < suballoc2ndCount)
8066	{
8067	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8068
8069	// 1. Process free space before this allocation.
8070	if (lastOffset < suballoc.offset)
8071	{
8072	// There is free space from lastOffset to suballoc.offset.
8073	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8074	}
8075
8076	// 2. Process this allocation.
8077	// There is allocation with suballoc.offset, suballoc.size.
8078	++inoutStats.allocationCount;
8079
8080	// 3. Prepare for next iteration.
8081	lastOffset = suballoc.offset + suballoc.size;
8082	++nextAlloc2ndIndex;
8083	}
8084	// We are at the end.
8085	else
8086	{
8087	if (lastOffset < freeSpace2ndTo1stEnd)
8088	{
8089	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8090	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
8091	}
8092
8093	// End of loop.
8094	lastOffset = freeSpace2ndTo1stEnd;
8095	}
8096	}
8097	}
8098
8099	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
8100	const VkDeviceSize freeSpace1stTo2ndEnd =
8101	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
8102	while (lastOffset < freeSpace1stTo2ndEnd)
8103	{
8104	// Find next non-null allocation or move nextAllocIndex to the end.
8105	while (nextAlloc1stIndex < suballoc1stCount &&
8106	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8107	{
8108	++nextAlloc1stIndex;
8109	}
8110
8111	// Found non-null allocation.
8112	if (nextAlloc1stIndex < suballoc1stCount)
8113	{
8114	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8115
8116	// 1. Process free space before this allocation.
8117	if (lastOffset < suballoc.offset)
8118	{
8119	// There is free space from lastOffset to suballoc.offset.
8120	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8121	}
8122
8123	// 2. Process this allocation.
8124	// There is allocation with suballoc.offset, suballoc.size.
8125	++inoutStats.allocationCount;
8126
8127	// 3. Prepare for next iteration.
8128	lastOffset = suballoc.offset + suballoc.size;
8129	++nextAlloc1stIndex;
8130	}
8131	// We are at the end.
8132	else
8133	{
8134	if (lastOffset < freeSpace1stTo2ndEnd)
8135	{
8136	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8137	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
8138	}
8139
8140	// End of loop.
8141	lastOffset = freeSpace1stTo2ndEnd;
8142	}
8143	}
8144
8145	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8146	{
8147	size_t nextAlloc2ndIndex = suballocations2nd.size() - 1;
8148	while (lastOffset < size)
8149	{
8150	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8151	while (nextAlloc2ndIndex != SIZE_MAX &&
8152	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8153	{
8154	--nextAlloc2ndIndex;
8155	}
8156
8157	// Found non-null allocation.
8158	if (nextAlloc2ndIndex != SIZE_MAX)
8159	{
8160	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8161
8162	// 1. Process free space before this allocation.
8163	if (lastOffset < suballoc.offset)
8164	{
8165	// There is free space from lastOffset to suballoc.offset.
8166	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8167	}
8168
8169	// 2. Process this allocation.
8170	// There is allocation with suballoc.offset, suballoc.size.
8171	++inoutStats.allocationCount;
8172
8173	// 3. Prepare for next iteration.
8174	lastOffset = suballoc.offset + suballoc.size;
8175	--nextAlloc2ndIndex;
8176	}
8177	// We are at the end.
8178	else
8179	{
8180	if (lastOffset < size)
8181	{
8182	// There is free space from lastOffset to size.
8183	const VkDeviceSize unusedRangeSize = size - lastOffset;
8184	}
8185
8186	// End of loop.
8187	lastOffset = size;
8188	}
8189	}
8190	}
8191	}
8192
8193	#if VMA_STATS_STRING_ENABLED
8194	void VmaBlockMetadata_Linear::PrintDetailedMap(class VmaJsonWriter& json) const
8195	{
8196	const VkDeviceSize size = GetSize();
8197	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8198	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8199	const size_t suballoc1stCount = suballocations1st.size();
8200	const size_t suballoc2ndCount = suballocations2nd.size();
8201
8202	// FIRST PASS
8203
8204	size_t unusedRangeCount = 0;
8205	VkDeviceSize usedBytes = 0;
8206
8207	VkDeviceSize lastOffset = 0;
8208
8209	size_t alloc2ndCount = 0;
8210	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8211	{
8212	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8213	size_t nextAlloc2ndIndex = 0;
8214	while (lastOffset < freeSpace2ndTo1stEnd)
8215	{
8216	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8217	while (nextAlloc2ndIndex < suballoc2ndCount &&
8218	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8219	{
8220	++nextAlloc2ndIndex;
8221	}
8222
8223	// Found non-null allocation.
8224	if (nextAlloc2ndIndex < suballoc2ndCount)
8225	{
8226	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8227
8228	// 1. Process free space before this allocation.
8229	if (lastOffset < suballoc.offset)
8230	{
8231	// There is free space from lastOffset to suballoc.offset.
8232	++unusedRangeCount;
8233	}
8234
8235	// 2. Process this allocation.
8236	// There is allocation with suballoc.offset, suballoc.size.
8237	++alloc2ndCount;
8238	usedBytes += suballoc.size;
8239
8240	// 3. Prepare for next iteration.
8241	lastOffset = suballoc.offset + suballoc.size;
8242	++nextAlloc2ndIndex;
8243	}
8244	// We are at the end.
8245	else
8246	{
8247	if (lastOffset < freeSpace2ndTo1stEnd)
8248	{
8249	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8250	++unusedRangeCount;
8251	}
8252
8253	// End of loop.
8254	lastOffset = freeSpace2ndTo1stEnd;
8255	}
8256	}
8257	}
8258
8259	size_t nextAlloc1stIndex = m_1stNullItemsBeginCount;
8260	size_t alloc1stCount = 0;
8261	const VkDeviceSize freeSpace1stTo2ndEnd =
8262	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ? suballocations2nd.back().offset : size;
8263	while (lastOffset < freeSpace1stTo2ndEnd)
8264	{
8265	// Find next non-null allocation or move nextAllocIndex to the end.
8266	while (nextAlloc1stIndex < suballoc1stCount &&
8267	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8268	{
8269	++nextAlloc1stIndex;
8270	}
8271
8272	// Found non-null allocation.
8273	if (nextAlloc1stIndex < suballoc1stCount)
8274	{
8275	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8276
8277	// 1. Process free space before this allocation.
8278	if (lastOffset < suballoc.offset)
8279	{
8280	// There is free space from lastOffset to suballoc.offset.
8281	++unusedRangeCount;
8282	}
8283
8284	// 2. Process this allocation.
8285	// There is allocation with suballoc.offset, suballoc.size.
8286	++alloc1stCount;
8287	usedBytes += suballoc.size;
8288
8289	// 3. Prepare for next iteration.
8290	lastOffset = suballoc.offset + suballoc.size;
8291	++nextAlloc1stIndex;
8292	}
8293	// We are at the end.
8294	else
8295	{
8296	if (lastOffset < size)
8297	{
8298	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8299	++unusedRangeCount;
8300	}
8301
8302	// End of loop.
8303	lastOffset = freeSpace1stTo2ndEnd;
8304	}
8305	}
8306
8307	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8308	{
8309	size_t nextAlloc2ndIndex = suballocations2nd.size() - 1;
8310	while (lastOffset < size)
8311	{
8312	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8313	while (nextAlloc2ndIndex != SIZE_MAX &&
8314	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8315	{
8316	--nextAlloc2ndIndex;
8317	}
8318
8319	// Found non-null allocation.
8320	if (nextAlloc2ndIndex != SIZE_MAX)
8321	{
8322	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8323
8324	// 1. Process free space before this allocation.
8325	if (lastOffset < suballoc.offset)
8326	{
8327	// There is free space from lastOffset to suballoc.offset.
8328	++unusedRangeCount;
8329	}
8330
8331	// 2. Process this allocation.
8332	// There is allocation with suballoc.offset, suballoc.size.
8333	++alloc2ndCount;
8334	usedBytes += suballoc.size;
8335
8336	// 3. Prepare for next iteration.
8337	lastOffset = suballoc.offset + suballoc.size;
8338	--nextAlloc2ndIndex;
8339	}
8340	// We are at the end.
8341	else
8342	{
8343	if (lastOffset < size)
8344	{
8345	// There is free space from lastOffset to size.
8346	++unusedRangeCount;
8347	}
8348
8349	// End of loop.
8350	lastOffset = size;
8351	}
8352	}
8353	}
8354
8355	const VkDeviceSize unusedBytes = size - usedBytes;
8356	PrintDetailedMap_Begin(json, unusedBytes, allocationCount: alloc1stCount + alloc2ndCount, unusedRangeCount);
8357
8358	// SECOND PASS
8359	lastOffset = 0;
8360
8361	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8362	{
8363	const VkDeviceSize freeSpace2ndTo1stEnd = suballocations1st[m_1stNullItemsBeginCount].offset;
8364	size_t nextAlloc2ndIndex = 0;
8365	while (lastOffset < freeSpace2ndTo1stEnd)
8366	{
8367	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8368	while (nextAlloc2ndIndex < suballoc2ndCount &&
8369	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8370	{
8371	++nextAlloc2ndIndex;
8372	}
8373
8374	// Found non-null allocation.
8375	if (nextAlloc2ndIndex < suballoc2ndCount)
8376	{
8377	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8378
8379	// 1. Process free space before this allocation.
8380	if (lastOffset < suballoc.offset)
8381	{
8382	// There is free space from lastOffset to suballoc.offset.
8383	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8384	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8385	}
8386
8387	// 2. Process this allocation.
8388	// There is allocation with suballoc.offset, suballoc.size.
8389	PrintDetailedMap_Allocation(json, offset: suballoc.offset, size: suballoc.size, userData: suballoc.userData);
8390
8391	// 3. Prepare for next iteration.
8392	lastOffset = suballoc.offset + suballoc.size;
8393	++nextAlloc2ndIndex;
8394	}
8395	// We are at the end.
8396	else
8397	{
8398	if (lastOffset < freeSpace2ndTo1stEnd)
8399	{
8400	// There is free space from lastOffset to freeSpace2ndTo1stEnd.
8401	const VkDeviceSize unusedRangeSize = freeSpace2ndTo1stEnd - lastOffset;
8402	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8403	}
8404
8405	// End of loop.
8406	lastOffset = freeSpace2ndTo1stEnd;
8407	}
8408	}
8409	}
8410
8411	nextAlloc1stIndex = m_1stNullItemsBeginCount;
8412	while (lastOffset < freeSpace1stTo2ndEnd)
8413	{
8414	// Find next non-null allocation or move nextAllocIndex to the end.
8415	while (nextAlloc1stIndex < suballoc1stCount &&
8416	suballocations1st[nextAlloc1stIndex].userData == VMA_NULL)
8417	{
8418	++nextAlloc1stIndex;
8419	}
8420
8421	// Found non-null allocation.
8422	if (nextAlloc1stIndex < suballoc1stCount)
8423	{
8424	const VmaSuballocation& suballoc = suballocations1st[nextAlloc1stIndex];
8425
8426	// 1. Process free space before this allocation.
8427	if (lastOffset < suballoc.offset)
8428	{
8429	// There is free space from lastOffset to suballoc.offset.
8430	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8431	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8432	}
8433
8434	// 2. Process this allocation.
8435	// There is allocation with suballoc.offset, suballoc.size.
8436	PrintDetailedMap_Allocation(json, offset: suballoc.offset, size: suballoc.size, userData: suballoc.userData);
8437
8438	// 3. Prepare for next iteration.
8439	lastOffset = suballoc.offset + suballoc.size;
8440	++nextAlloc1stIndex;
8441	}
8442	// We are at the end.
8443	else
8444	{
8445	if (lastOffset < freeSpace1stTo2ndEnd)
8446	{
8447	// There is free space from lastOffset to freeSpace1stTo2ndEnd.
8448	const VkDeviceSize unusedRangeSize = freeSpace1stTo2ndEnd - lastOffset;
8449	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8450	}
8451
8452	// End of loop.
8453	lastOffset = freeSpace1stTo2ndEnd;
8454	}
8455	}
8456
8457	if (m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8458	{
8459	size_t nextAlloc2ndIndex = suballocations2nd.size() - 1;
8460	while (lastOffset < size)
8461	{
8462	// Find next non-null allocation or move nextAlloc2ndIndex to the end.
8463	while (nextAlloc2ndIndex != SIZE_MAX &&
8464	suballocations2nd[nextAlloc2ndIndex].userData == VMA_NULL)
8465	{
8466	--nextAlloc2ndIndex;
8467	}
8468
8469	// Found non-null allocation.
8470	if (nextAlloc2ndIndex != SIZE_MAX)
8471	{
8472	const VmaSuballocation& suballoc = suballocations2nd[nextAlloc2ndIndex];
8473
8474	// 1. Process free space before this allocation.
8475	if (lastOffset < suballoc.offset)
8476	{
8477	// There is free space from lastOffset to suballoc.offset.
8478	const VkDeviceSize unusedRangeSize = suballoc.offset - lastOffset;
8479	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8480	}
8481
8482	// 2. Process this allocation.
8483	// There is allocation with suballoc.offset, suballoc.size.
8484	PrintDetailedMap_Allocation(json, offset: suballoc.offset, size: suballoc.size, userData: suballoc.userData);
8485
8486	// 3. Prepare for next iteration.
8487	lastOffset = suballoc.offset + suballoc.size;
8488	--nextAlloc2ndIndex;
8489	}
8490	// We are at the end.
8491	else
8492	{
8493	if (lastOffset < size)
8494	{
8495	// There is free space from lastOffset to size.
8496	const VkDeviceSize unusedRangeSize = size - lastOffset;
8497	PrintDetailedMap_UnusedRange(json, offset: lastOffset, size: unusedRangeSize);
8498	}
8499
8500	// End of loop.
8501	lastOffset = size;
8502	}
8503	}
8504	}
8505
8506	PrintDetailedMap_End(json);
8507	}
8508	#endif // VMA_STATS_STRING_ENABLED
8509
8510	bool VmaBlockMetadata_Linear::CreateAllocationRequest(
8511	VkDeviceSize allocSize,
8512	VkDeviceSize allocAlignment,
8513	bool upperAddress,
8514	VmaSuballocationType allocType,
8515	uint32_t strategy,
8516	VmaAllocationRequest* pAllocationRequest)
8517	{
8518	VMA_ASSERT(allocSize > 0);
8519	VMA_ASSERT(allocType != VMA_SUBALLOCATION_TYPE_FREE);
8520	VMA_ASSERT(pAllocationRequest != VMA_NULL);
8521	VMA_HEAVY_ASSERT(Validate());
8522	pAllocationRequest->size = allocSize;
8523	return upperAddress ?
8524	CreateAllocationRequest_UpperAddress(
8525	allocSize, allocAlignment, allocType, strategy, pAllocationRequest) :
8526	CreateAllocationRequest_LowerAddress(
8527	allocSize, allocAlignment, allocType, strategy, pAllocationRequest);
8528	}
8529
8530	VkResult VmaBlockMetadata_Linear::CheckCorruption(const void* pBlockData)
8531	{
8532	VMA_ASSERT(!IsVirtual());
8533	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8534	for (size_t i = m_1stNullItemsBeginCount, count = suballocations1st.size(); i < count; ++i)
8535	{
8536	const VmaSuballocation& suballoc = suballocations1st[i];
8537	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
8538	{
8539	if (!VmaValidateMagicValue(pData: pBlockData, offset: suballoc.offset + suballoc.size))
8540	{
8541	VMA_ASSERT(0 && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
8542	return VK_ERROR_UNKNOWN_COPY;
8543	}
8544	}
8545	}
8546
8547	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8548	for (size_t i = 0, count = suballocations2nd.size(); i < count; ++i)
8549	{
8550	const VmaSuballocation& suballoc = suballocations2nd[i];
8551	if (suballoc.type != VMA_SUBALLOCATION_TYPE_FREE)
8552	{
8553	if (!VmaValidateMagicValue(pData: pBlockData, offset: suballoc.offset + suballoc.size))
8554	{
8555	VMA_ASSERT(0 && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
8556	return VK_ERROR_UNKNOWN_COPY;
8557	}
8558	}
8559	}
8560
8561	return VK_SUCCESS;
8562	}
8563
8564	void VmaBlockMetadata_Linear::Alloc(
8565	const VmaAllocationRequest& request,
8566	VmaSuballocationType type,
8567	void* userData)
8568	{
8569	const VkDeviceSize offset = (VkDeviceSize)request.allocHandle - 1;
8570	const VmaSuballocation newSuballoc = { .offset: offset, .size: request.size, .userData: userData, .type: type };
8571
8572	switch (request.type)
8573	{
8574	case VmaAllocationRequestType::UpperAddress:
8575	{
8576	VMA_ASSERT(m_2ndVectorMode != SECOND_VECTOR_RING_BUFFER &&
8577	"CRITICAL ERROR: Trying to use linear allocator as double stack while it was already used as ring buffer.");
8578	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8579	suballocations2nd.push_back(src: newSuballoc);
8580	m_2ndVectorMode = SECOND_VECTOR_DOUBLE_STACK;
8581	}
8582	break;
8583	case VmaAllocationRequestType::EndOf1st:
8584	{
8585	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8586
8587	VMA_ASSERT(suballocations1st.empty() ||
8588	offset >= suballocations1st.back().offset + suballocations1st.back().size);
8589	// Check if it fits before the end of the block.
8590	VMA_ASSERT(offset + request.size <= GetSize());
8591
8592	suballocations1st.push_back(src: newSuballoc);
8593	}
8594	break;
8595	case VmaAllocationRequestType::EndOf2nd:
8596	{
8597	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8598	// New allocation at the end of 2-part ring buffer, so before first allocation from 1st vector.
8599	VMA_ASSERT(!suballocations1st.empty() &&
8600	offset + request.size <= suballocations1st[m_1stNullItemsBeginCount].offset);
8601	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8602
8603	switch (m_2ndVectorMode)
8604	{
8605	case SECOND_VECTOR_EMPTY:
8606	// First allocation from second part ring buffer.
8607	VMA_ASSERT(suballocations2nd.empty());
8608	m_2ndVectorMode = SECOND_VECTOR_RING_BUFFER;
8609	break;
8610	case SECOND_VECTOR_RING_BUFFER:
8611	// 2-part ring buffer is already started.
8612	VMA_ASSERT(!suballocations2nd.empty());
8613	break;
8614	case SECOND_VECTOR_DOUBLE_STACK:
8615	VMA_ASSERT(0 && "CRITICAL ERROR: Trying to use linear allocator as ring buffer while it was already used as double stack.");
8616	break;
8617	default:
8618	VMA_ASSERT(0);
8619	}
8620
8621	suballocations2nd.push_back(src: newSuballoc);
8622	}
8623	break;
8624	default:
8625	VMA_ASSERT(0 && "CRITICAL INTERNAL ERROR.");
8626	}
8627
8628	m_SumFreeSize -= newSuballoc.size;
8629	}
8630
8631	void VmaBlockMetadata_Linear::Free(VmaAllocHandle allocHandle)
8632	{
8633	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8634	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8635	VkDeviceSize offset = (VkDeviceSize)allocHandle - 1;
8636
8637	if (!suballocations1st.empty())
8638	{
8639	// First allocation: Mark it as next empty at the beginning.
8640	VmaSuballocation& firstSuballoc = suballocations1st[m_1stNullItemsBeginCount];
8641	if (firstSuballoc.offset == offset)
8642	{
8643	firstSuballoc.type = VMA_SUBALLOCATION_TYPE_FREE;
8644	firstSuballoc.userData = VMA_NULL;
8645	m_SumFreeSize += firstSuballoc.size;
8646	++m_1stNullItemsBeginCount;
8647	CleanupAfterFree();
8648	return;
8649	}
8650	}
8651
8652	// Last allocation in 2-part ring buffer or top of upper stack (same logic).
8653	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER ||
8654	m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8655	{
8656	VmaSuballocation& lastSuballoc = suballocations2nd.back();
8657	if (lastSuballoc.offset == offset)
8658	{
8659	m_SumFreeSize += lastSuballoc.size;
8660	suballocations2nd.pop_back();
8661	CleanupAfterFree();
8662	return;
8663	}
8664	}
8665	// Last allocation in 1st vector.
8666	else if (m_2ndVectorMode == SECOND_VECTOR_EMPTY)
8667	{
8668	VmaSuballocation& lastSuballoc = suballocations1st.back();
8669	if (lastSuballoc.offset == offset)
8670	{
8671	m_SumFreeSize += lastSuballoc.size;
8672	suballocations1st.pop_back();
8673	CleanupAfterFree();
8674	return;
8675	}
8676	}
8677
8678	VmaSuballocation refSuballoc;
8679	refSuballoc.offset = offset;
8680	// Rest of members stays uninitialized intentionally for better performance.
8681
8682	// Item from the middle of 1st vector.
8683	{
8684	const SuballocationVectorType::iterator it = VmaBinaryFindSorted(
8685	beg: suballocations1st.begin() + m_1stNullItemsBeginCount,
8686	end: suballocations1st.end(),
8687	value: refSuballoc,
8688	cmp: VmaSuballocationOffsetLess());
8689	if (it != suballocations1st.end())
8690	{
8691	it->type = VMA_SUBALLOCATION_TYPE_FREE;
8692	it->userData = VMA_NULL;
8693	++m_1stNullItemsMiddleCount;
8694	m_SumFreeSize += it->size;
8695	CleanupAfterFree();
8696	return;
8697	}
8698	}
8699
8700	if (m_2ndVectorMode != SECOND_VECTOR_EMPTY)
8701	{
8702	// Item from the middle of 2nd vector.
8703	const SuballocationVectorType::iterator it = m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER ?
8704	VmaBinaryFindSorted(beg: suballocations2nd.begin(), end: suballocations2nd.end(), value: refSuballoc, cmp: VmaSuballocationOffsetLess()) :
8705	VmaBinaryFindSorted(beg: suballocations2nd.begin(), end: suballocations2nd.end(), value: refSuballoc, cmp: VmaSuballocationOffsetGreater());
8706	if (it != suballocations2nd.end())
8707	{
8708	it->type = VMA_SUBALLOCATION_TYPE_FREE;
8709	it->userData = VMA_NULL;
8710	++m_2ndNullItemsCount;
8711	m_SumFreeSize += it->size;
8712	CleanupAfterFree();
8713	return;
8714	}
8715	}
8716
8717	VMA_ASSERT(0 && "Allocation to free not found in linear allocator!");
8718	}
8719
8720	void VmaBlockMetadata_Linear::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
8721	{
8722	outInfo.offset = (VkDeviceSize)allocHandle - 1;
8723	VmaSuballocation& suballoc = FindSuballocation(offset: outInfo.offset);
8724	outInfo.size = suballoc.size;
8725	outInfo.pUserData = suballoc.userData;
8726	}
8727
8728	void* VmaBlockMetadata_Linear::GetAllocationUserData(VmaAllocHandle allocHandle) const
8729	{
8730	return FindSuballocation(offset: (VkDeviceSize)allocHandle - 1).userData;
8731	}
8732
8733	VmaAllocHandle VmaBlockMetadata_Linear::GetAllocationListBegin() const
8734	{
8735	// Function only used for defragmentation, which is disabled for this algorithm
8736	VMA_ASSERT(0);
8737	return VK_NULL_HANDLE;
8738	}
8739
8740	VmaAllocHandle VmaBlockMetadata_Linear::GetNextAllocation(VmaAllocHandle prevAlloc) const
8741	{
8742	// Function only used for defragmentation, which is disabled for this algorithm
8743	VMA_ASSERT(0);
8744	return VK_NULL_HANDLE;
8745	}
8746
8747	VkDeviceSize VmaBlockMetadata_Linear::GetNextFreeRegionSize(VmaAllocHandle alloc) const
8748	{
8749	// Function only used for defragmentation, which is disabled for this algorithm
8750	VMA_ASSERT(0);
8751	return 0;
8752	}
8753
8754	void VmaBlockMetadata_Linear::Clear()
8755	{
8756	m_SumFreeSize = GetSize();
8757	m_Suballocations0.clear();
8758	m_Suballocations1.clear();
8759	// Leaving m_1stVectorIndex unchanged - it doesn't matter.
8760	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8761	m_1stNullItemsBeginCount = 0;
8762	m_1stNullItemsMiddleCount = 0;
8763	m_2ndNullItemsCount = 0;
8764	}
8765
8766	void VmaBlockMetadata_Linear::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
8767	{
8768	VmaSuballocation& suballoc = FindSuballocation(offset: (VkDeviceSize)allocHandle - 1);
8769	suballoc.userData = userData;
8770	}
8771
8772	void VmaBlockMetadata_Linear::DebugLogAllAllocations() const
8773	{
8774	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8775	for (auto it = suballocations1st.begin() + m_1stNullItemsBeginCount; it != suballocations1st.end(); ++it)
8776	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
8777	DebugLogAllocation(offset: it->offset, size: it->size, userData: it->userData);
8778
8779	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8780	for (auto it = suballocations2nd.begin(); it != suballocations2nd.end(); ++it)
8781	if (it->type != VMA_SUBALLOCATION_TYPE_FREE)
8782	DebugLogAllocation(offset: it->offset, size: it->size, userData: it->userData);
8783	}
8784
8785	VmaSuballocation& VmaBlockMetadata_Linear::FindSuballocation(VkDeviceSize offset) const
8786	{
8787	const SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8788	const SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8789
8790	VmaSuballocation refSuballoc;
8791	refSuballoc.offset = offset;
8792	// Rest of members stays uninitialized intentionally for better performance.
8793
8794	// Item from the 1st vector.
8795	{
8796	SuballocationVectorType::const_iterator it = VmaBinaryFindSorted(
8797	beg: suballocations1st.begin() + m_1stNullItemsBeginCount,
8798	end: suballocations1st.end(),
8799	value: refSuballoc,
8800	cmp: VmaSuballocationOffsetLess());
8801	if (it != suballocations1st.end())
8802	{
8803	return const_cast<VmaSuballocation&>(*it);
8804	}
8805	}
8806
8807	if (m_2ndVectorMode != SECOND_VECTOR_EMPTY)
8808	{
8809	// Rest of members stays uninitialized intentionally for better performance.
8810	SuballocationVectorType::const_iterator it = m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER ?
8811	VmaBinaryFindSorted(beg: suballocations2nd.begin(), end: suballocations2nd.end(), value: refSuballoc, cmp: VmaSuballocationOffsetLess()) :
8812	VmaBinaryFindSorted(beg: suballocations2nd.begin(), end: suballocations2nd.end(), value: refSuballoc, cmp: VmaSuballocationOffsetGreater());
8813	if (it != suballocations2nd.end())
8814	{
8815	return const_cast<VmaSuballocation&>(*it);
8816	}
8817	}
8818
8819	VMA_ASSERT(0 && "Allocation not found in linear allocator!");
8820	return const_cast<VmaSuballocation&>(suballocations1st.back()); // Should never occur.
8821	}
8822
8823	bool VmaBlockMetadata_Linear::ShouldCompact1st() const
8824	{
8825	const size_t nullItemCount = m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount;
8826	const size_t suballocCount = AccessSuballocations1st().size();
8827	return suballocCount > 32 && nullItemCount * 2 >= (suballocCount - nullItemCount) * 3;
8828	}
8829
8830	void VmaBlockMetadata_Linear::CleanupAfterFree()
8831	{
8832	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8833	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8834
8835	if (IsEmpty())
8836	{
8837	suballocations1st.clear();
8838	suballocations2nd.clear();
8839	m_1stNullItemsBeginCount = 0;
8840	m_1stNullItemsMiddleCount = 0;
8841	m_2ndNullItemsCount = 0;
8842	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8843	}
8844	else
8845	{
8846	const size_t suballoc1stCount = suballocations1st.size();
8847	const size_t nullItem1stCount = m_1stNullItemsBeginCount + m_1stNullItemsMiddleCount;
8848	VMA_ASSERT(nullItem1stCount <= suballoc1stCount);
8849
8850	// Find more null items at the beginning of 1st vector.
8851	while (m_1stNullItemsBeginCount < suballoc1stCount &&
8852	suballocations1st[m_1stNullItemsBeginCount].type == VMA_SUBALLOCATION_TYPE_FREE)
8853	{
8854	++m_1stNullItemsBeginCount;
8855	--m_1stNullItemsMiddleCount;
8856	}
8857
8858	// Find more null items at the end of 1st vector.
8859	while (m_1stNullItemsMiddleCount > 0 &&
8860	suballocations1st.back().type == VMA_SUBALLOCATION_TYPE_FREE)
8861	{
8862	--m_1stNullItemsMiddleCount;
8863	suballocations1st.pop_back();
8864	}
8865
8866	// Find more null items at the end of 2nd vector.
8867	while (m_2ndNullItemsCount > 0 &&
8868	suballocations2nd.back().type == VMA_SUBALLOCATION_TYPE_FREE)
8869	{
8870	--m_2ndNullItemsCount;
8871	suballocations2nd.pop_back();
8872	}
8873
8874	// Find more null items at the beginning of 2nd vector.
8875	while (m_2ndNullItemsCount > 0 &&
8876	suballocations2nd[0].type == VMA_SUBALLOCATION_TYPE_FREE)
8877	{
8878	--m_2ndNullItemsCount;
8879	VmaVectorRemove(vec&: suballocations2nd, index: 0);
8880	}
8881
8882	if (ShouldCompact1st())
8883	{
8884	const size_t nonNullItemCount = suballoc1stCount - nullItem1stCount;
8885	size_t srcIndex = m_1stNullItemsBeginCount;
8886	for (size_t dstIndex = 0; dstIndex < nonNullItemCount; ++dstIndex)
8887	{
8888	while (suballocations1st[srcIndex].type == VMA_SUBALLOCATION_TYPE_FREE)
8889	{
8890	++srcIndex;
8891	}
8892	if (dstIndex != srcIndex)
8893	{
8894	suballocations1st[dstIndex] = suballocations1st[srcIndex];
8895	}
8896	++srcIndex;
8897	}
8898	suballocations1st.resize(newCount: nonNullItemCount);
8899	m_1stNullItemsBeginCount = 0;
8900	m_1stNullItemsMiddleCount = 0;
8901	}
8902
8903	// 2nd vector became empty.
8904	if (suballocations2nd.empty())
8905	{
8906	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8907	}
8908
8909	// 1st vector became empty.
8910	if (suballocations1st.size() - m_1stNullItemsBeginCount == 0)
8911	{
8912	suballocations1st.clear();
8913	m_1stNullItemsBeginCount = 0;
8914
8915	if (!suballocations2nd.empty() && m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
8916	{
8917	// Swap 1st with 2nd. Now 2nd is empty.
8918	m_2ndVectorMode = SECOND_VECTOR_EMPTY;
8919	m_1stNullItemsMiddleCount = m_2ndNullItemsCount;
8920	while (m_1stNullItemsBeginCount < suballocations2nd.size() &&
8921	suballocations2nd[m_1stNullItemsBeginCount].type == VMA_SUBALLOCATION_TYPE_FREE)
8922	{
8923	++m_1stNullItemsBeginCount;
8924	--m_1stNullItemsMiddleCount;
8925	}
8926	m_2ndNullItemsCount = 0;
8927	m_1stVectorIndex ^= 1;
8928	}
8929	}
8930	}
8931
8932	VMA_HEAVY_ASSERT(Validate());
8933	}
8934
8935	bool VmaBlockMetadata_Linear::CreateAllocationRequest_LowerAddress(
8936	VkDeviceSize allocSize,
8937	VkDeviceSize allocAlignment,
8938	VmaSuballocationType allocType,
8939	uint32_t strategy,
8940	VmaAllocationRequest* pAllocationRequest)
8941	{
8942	const VkDeviceSize blockSize = GetSize();
8943	const VkDeviceSize debugMargin = GetDebugMargin();
8944	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
8945	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
8946	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
8947
8948	if (m_2ndVectorMode == SECOND_VECTOR_EMPTY || m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
8949	{
8950	// Try to allocate at the end of 1st vector.
8951
8952	VkDeviceSize resultBaseOffset = 0;
8953	if (!suballocations1st.empty())
8954	{
8955	const VmaSuballocation& lastSuballoc = suballocations1st.back();
8956	resultBaseOffset = lastSuballoc.offset + lastSuballoc.size + debugMargin;
8957	}
8958
8959	// Start from offset equal to beginning of free space.
8960	VkDeviceSize resultOffset = resultBaseOffset;
8961
8962	// Apply alignment.
8963	resultOffset = VmaAlignUp(val: resultOffset, alignment: allocAlignment);
8964
8965	// Check previous suballocations for BufferImageGranularity conflicts.
8966	// Make bigger alignment if necessary.
8967	if (bufferImageGranularity > 1 && bufferImageGranularity != allocAlignment && !suballocations1st.empty())
8968	{
8969	bool bufferImageGranularityConflict = false;
8970	for (size_t prevSuballocIndex = suballocations1st.size(); prevSuballocIndex--; )
8971	{
8972	const VmaSuballocation& prevSuballoc = suballocations1st[prevSuballocIndex];
8973	if (VmaBlocksOnSamePage(resourceAOffset: prevSuballoc.offset, resourceASize: prevSuballoc.size, resourceBOffset: resultOffset, pageSize: bufferImageGranularity))
8974	{
8975	if (VmaIsBufferImageGranularityConflict(suballocType1: prevSuballoc.type, suballocType2: allocType))
8976	{
8977	bufferImageGranularityConflict = true;
8978	break;
8979	}
8980	}
8981	else
8982	// Already on previous page.
8983	break;
8984	}
8985	if (bufferImageGranularityConflict)
8986	{
8987	resultOffset = VmaAlignUp(val: resultOffset, alignment: bufferImageGranularity);
8988	}
8989	}
8990
8991	const VkDeviceSize freeSpaceEnd = m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK ?
8992	suballocations2nd.back().offset : blockSize;
8993
8994	// There is enough free space at the end after alignment.
8995	if (resultOffset + allocSize + debugMargin <= freeSpaceEnd)
8996	{
8997	// Check next suballocations for BufferImageGranularity conflicts.
8998	// If conflict exists, allocation cannot be made here.
8999	if ((allocSize % bufferImageGranularity || resultOffset % bufferImageGranularity) && m_2ndVectorMode == SECOND_VECTOR_DOUBLE_STACK)
9000	{
9001	for (size_t nextSuballocIndex = suballocations2nd.size(); nextSuballocIndex--; )
9002	{
9003	const VmaSuballocation& nextSuballoc = suballocations2nd[nextSuballocIndex];
9004	if (VmaBlocksOnSamePage(resourceAOffset: resultOffset, resourceASize: allocSize, resourceBOffset: nextSuballoc.offset, pageSize: bufferImageGranularity))
9005	{
9006	if (VmaIsBufferImageGranularityConflict(suballocType1: allocType, suballocType2: nextSuballoc.type))
9007	{
9008	return false;
9009	}
9010	}
9011	else
9012	{
9013	// Already on previous page.
9014	break;
9015	}
9016	}
9017	}
9018
9019	// All tests passed: Success.
9020	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + 1);
9021	// pAllocationRequest->item, customData unused.
9022	pAllocationRequest->type = VmaAllocationRequestType::EndOf1st;
9023	return true;
9024	}
9025	}
9026
9027	// Wrap-around to end of 2nd vector. Try to allocate there, watching for the
9028	// beginning of 1st vector as the end of free space.
9029	if (m_2ndVectorMode == SECOND_VECTOR_EMPTY || m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
9030	{
9031	VMA_ASSERT(!suballocations1st.empty());
9032
9033	VkDeviceSize resultBaseOffset = 0;
9034	if (!suballocations2nd.empty())
9035	{
9036	const VmaSuballocation& lastSuballoc = suballocations2nd.back();
9037	resultBaseOffset = lastSuballoc.offset + lastSuballoc.size + debugMargin;
9038	}
9039
9040	// Start from offset equal to beginning of free space.
9041	VkDeviceSize resultOffset = resultBaseOffset;
9042
9043	// Apply alignment.
9044	resultOffset = VmaAlignUp(val: resultOffset, alignment: allocAlignment);
9045
9046	// Check previous suballocations for BufferImageGranularity conflicts.
9047	// Make bigger alignment if necessary.
9048	if (bufferImageGranularity > 1 && bufferImageGranularity != allocAlignment && !suballocations2nd.empty())
9049	{
9050	bool bufferImageGranularityConflict = false;
9051	for (size_t prevSuballocIndex = suballocations2nd.size(); prevSuballocIndex--; )
9052	{
9053	const VmaSuballocation& prevSuballoc = suballocations2nd[prevSuballocIndex];
9054	if (VmaBlocksOnSamePage(resourceAOffset: prevSuballoc.offset, resourceASize: prevSuballoc.size, resourceBOffset: resultOffset, pageSize: bufferImageGranularity))
9055	{
9056	if (VmaIsBufferImageGranularityConflict(suballocType1: prevSuballoc.type, suballocType2: allocType))
9057	{
9058	bufferImageGranularityConflict = true;
9059	break;
9060	}
9061	}
9062	else
9063	// Already on previous page.
9064	break;
9065	}
9066	if (bufferImageGranularityConflict)
9067	{
9068	resultOffset = VmaAlignUp(val: resultOffset, alignment: bufferImageGranularity);
9069	}
9070	}
9071
9072	size_t index1st = m_1stNullItemsBeginCount;
9073
9074	// There is enough free space at the end after alignment.
9075	if ((index1st == suballocations1st.size() && resultOffset + allocSize + debugMargin <= blockSize) ||
9076	(index1st < suballocations1st.size() && resultOffset + allocSize + debugMargin <= suballocations1st[index1st].offset))
9077	{
9078	// Check next suballocations for BufferImageGranularity conflicts.
9079	// If conflict exists, allocation cannot be made here.
9080	if (allocSize % bufferImageGranularity || resultOffset % bufferImageGranularity)
9081	{
9082	for (size_t nextSuballocIndex = index1st;
9083	nextSuballocIndex < suballocations1st.size();
9084	nextSuballocIndex++)
9085	{
9086	const VmaSuballocation& nextSuballoc = suballocations1st[nextSuballocIndex];
9087	if (VmaBlocksOnSamePage(resourceAOffset: resultOffset, resourceASize: allocSize, resourceBOffset: nextSuballoc.offset, pageSize: bufferImageGranularity))
9088	{
9089	if (VmaIsBufferImageGranularityConflict(suballocType1: allocType, suballocType2: nextSuballoc.type))
9090	{
9091	return false;
9092	}
9093	}
9094	else
9095	{
9096	// Already on next page.
9097	break;
9098	}
9099	}
9100	}
9101
9102	// All tests passed: Success.
9103	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + 1);
9104	pAllocationRequest->type = VmaAllocationRequestType::EndOf2nd;
9105	// pAllocationRequest->item, customData unused.
9106	return true;
9107	}
9108	}
9109
9110	return false;
9111	}
9112
9113	bool VmaBlockMetadata_Linear::CreateAllocationRequest_UpperAddress(
9114	VkDeviceSize allocSize,
9115	VkDeviceSize allocAlignment,
9116	VmaSuballocationType allocType,
9117	uint32_t strategy,
9118	VmaAllocationRequest* pAllocationRequest)
9119	{
9120	const VkDeviceSize blockSize = GetSize();
9121	const VkDeviceSize bufferImageGranularity = GetBufferImageGranularity();
9122	SuballocationVectorType& suballocations1st = AccessSuballocations1st();
9123	SuballocationVectorType& suballocations2nd = AccessSuballocations2nd();
9124
9125	if (m_2ndVectorMode == SECOND_VECTOR_RING_BUFFER)
9126	{
9127	VMA_ASSERT(0 && "Trying to use pool with linear algorithm as double stack, while it is already being used as ring buffer.");
9128	return false;
9129	}
9130
9131	// Try to allocate before 2nd.back(), or end of block if 2nd.empty().
9132	if (allocSize > blockSize)
9133	{
9134	return false;
9135	}
9136	VkDeviceSize resultBaseOffset = blockSize - allocSize;
9137	if (!suballocations2nd.empty())
9138	{
9139	const VmaSuballocation& lastSuballoc = suballocations2nd.back();
9140	resultBaseOffset = lastSuballoc.offset - allocSize;
9141	if (allocSize > lastSuballoc.offset)
9142	{
9143	return false;
9144	}
9145	}
9146
9147	// Start from offset equal to end of free space.
9148	VkDeviceSize resultOffset = resultBaseOffset;
9149
9150	const VkDeviceSize debugMargin = GetDebugMargin();
9151
9152	// Apply debugMargin at the end.
9153	if (debugMargin > 0)
9154	{
9155	if (resultOffset < debugMargin)
9156	{
9157	return false;
9158	}
9159	resultOffset -= debugMargin;
9160	}
9161
9162	// Apply alignment.
9163	resultOffset = VmaAlignDown(val: resultOffset, alignment: allocAlignment);
9164
9165	// Check next suballocations from 2nd for BufferImageGranularity conflicts.
9166	// Make bigger alignment if necessary.
9167	if (bufferImageGranularity > 1 && bufferImageGranularity != allocAlignment && !suballocations2nd.empty())
9168	{
9169	bool bufferImageGranularityConflict = false;
9170	for (size_t nextSuballocIndex = suballocations2nd.size(); nextSuballocIndex--; )
9171	{
9172	const VmaSuballocation& nextSuballoc = suballocations2nd[nextSuballocIndex];
9173	if (VmaBlocksOnSamePage(resourceAOffset: resultOffset, resourceASize: allocSize, resourceBOffset: nextSuballoc.offset, pageSize: bufferImageGranularity))
9174	{
9175	if (VmaIsBufferImageGranularityConflict(suballocType1: nextSuballoc.type, suballocType2: allocType))
9176	{
9177	bufferImageGranularityConflict = true;
9178	break;
9179	}
9180	}
9181	else
9182	// Already on previous page.
9183	break;
9184	}
9185	if (bufferImageGranularityConflict)
9186	{
9187	resultOffset = VmaAlignDown(val: resultOffset, alignment: bufferImageGranularity);
9188	}
9189	}
9190
9191	// There is enough free space.
9192	const VkDeviceSize endOf1st = !suballocations1st.empty() ?
9193	suballocations1st.back().offset + suballocations1st.back().size :
9194	0;
9195	if (endOf1st + debugMargin <= resultOffset)
9196	{
9197	// Check previous suballocations for BufferImageGranularity conflicts.
9198	// If conflict exists, allocation cannot be made here.
9199	if (bufferImageGranularity > 1)
9200	{
9201	for (size_t prevSuballocIndex = suballocations1st.size(); prevSuballocIndex--; )
9202	{
9203	const VmaSuballocation& prevSuballoc = suballocations1st[prevSuballocIndex];
9204	if (VmaBlocksOnSamePage(resourceAOffset: prevSuballoc.offset, resourceASize: prevSuballoc.size, resourceBOffset: resultOffset, pageSize: bufferImageGranularity))
9205	{
9206	if (VmaIsBufferImageGranularityConflict(suballocType1: allocType, suballocType2: prevSuballoc.type))
9207	{
9208	return false;
9209	}
9210	}
9211	else
9212	{
9213	// Already on next page.
9214	break;
9215	}
9216	}
9217	}
9218
9219	// All tests passed: Success.
9220	pAllocationRequest->allocHandle = (VmaAllocHandle)(resultOffset + 1);
9221	// pAllocationRequest->item unused.
9222	pAllocationRequest->type = VmaAllocationRequestType::UpperAddress;
9223	return true;
9224	}
9225
9226	return false;
9227	}
9228	#endif // _VMA_BLOCK_METADATA_LINEAR_FUNCTIONS
9229	#endif // _VMA_BLOCK_METADATA_LINEAR
9230
9231	#if 0
9232	#ifndef _VMA_BLOCK_METADATA_BUDDY
9233	/*
9234	- GetSize() is the original size of allocated memory block.
9235	- m_UsableSize is this size aligned down to a power of two.
9236	All allocations and calculations happen relative to m_UsableSize.
9237	- GetUnusableSize() is the difference between them.
9238	It is reported as separate, unused range, not available for allocations.
9239
9240	Node at level 0 has size = m_UsableSize.
9241	Each next level contains nodes with size 2 times smaller than current level.
9242	m_LevelCount is the maximum number of levels to use in the current object.
9243	*/
9244	class VmaBlockMetadata_Buddy : public VmaBlockMetadata
9245	{
9246	VMA_CLASS_NO_COPY(VmaBlockMetadata_Buddy)
9247	public:
9248	VmaBlockMetadata_Buddy(const VkAllocationCallbacks* pAllocationCallbacks,
9249	VkDeviceSize bufferImageGranularity, bool isVirtual);
9250	virtual ~VmaBlockMetadata_Buddy();
9251
9252	size_t GetAllocationCount() const override { return m_AllocationCount; }
9253	VkDeviceSize GetSumFreeSize() const override { return m_SumFreeSize + GetUnusableSize(); }
9254	bool IsEmpty() const override { return m_Root->type == Node::TYPE_FREE; }
9255	VkResult CheckCorruption(const void* pBlockData) override { return VK_ERROR_FEATURE_NOT_PRESENT; }
9256	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return (VkDeviceSize)allocHandle - 1; };
9257	void DebugLogAllAllocations() const override { DebugLogAllAllocationNode(m_Root, 0); }
9258
9259	void Init(VkDeviceSize size) override;
9260	bool Validate() const override;
9261
9262	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
9263	void AddStatistics(VmaStatistics& inoutStats) const override;
9264
9265	#if VMA_STATS_STRING_ENABLED
9266	void PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const override;
9267	#endif
9268
9269	bool CreateAllocationRequest(
9270	VkDeviceSize allocSize,
9271	VkDeviceSize allocAlignment,
9272	bool upperAddress,
9273	VmaSuballocationType allocType,
9274	uint32_t strategy,
9275	VmaAllocationRequest* pAllocationRequest) override;
9276
9277	void Alloc(
9278	const VmaAllocationRequest& request,
9279	VmaSuballocationType type,
9280	void* userData) override;
9281
9282	void Free(VmaAllocHandle allocHandle) override;
9283	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
9284	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
9285	VmaAllocHandle GetAllocationListBegin() const override;
9286	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
9287	void Clear() override;
9288	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
9289
9290	private:
9291	static const size_t MAX_LEVELS = 48;
9292
9293	struct ValidationContext
9294	{
9295	size_t calculatedAllocationCount = 0;
9296	size_t calculatedFreeCount = 0;
9297	VkDeviceSize calculatedSumFreeSize = 0;
9298	};
9299	struct Node
9300	{
9301	VkDeviceSize offset;
9302	enum TYPE
9303	{
9304	TYPE_FREE,
9305	TYPE_ALLOCATION,
9306	TYPE_SPLIT,
9307	TYPE_COUNT
9308	} type;
9309	Node* parent;
9310	Node* buddy;
9311
9312	union
9313	{
9314	struct
9315	{
9316	Node* prev;
9317	Node* next;
9318	} free;
9319	struct
9320	{
9321	void* userData;
9322	} allocation;
9323	struct
9324	{
9325	Node* leftChild;
9326	} split;
9327	};
9328	};
9329
9330	// Size of the memory block aligned down to a power of two.
9331	VkDeviceSize m_UsableSize;
9332	uint32_t m_LevelCount;
9333	VmaPoolAllocator<Node> m_NodeAllocator;
9334	Node* m_Root;
9335	struct
9336	{
9337	Node* front;
9338	Node* back;
9339	} m_FreeList[MAX_LEVELS];
9340
9341	// Number of nodes in the tree with type == TYPE_ALLOCATION.
9342	size_t m_AllocationCount;
9343	// Number of nodes in the tree with type == TYPE_FREE.
9344	size_t m_FreeCount;
9345	// Doesn't include space wasted due to internal fragmentation - allocation sizes are just aligned up to node sizes.
9346	// Doesn't include unusable size.
9347	VkDeviceSize m_SumFreeSize;
9348
9349	VkDeviceSize GetUnusableSize() const { return GetSize() - m_UsableSize; }
9350	VkDeviceSize LevelToNodeSize(uint32_t level) const { return m_UsableSize >> level; }
9351
9352	VkDeviceSize AlignAllocationSize(VkDeviceSize size) const
9353	{
9354	if (!IsVirtual())
9355	{
9356	size = VmaAlignUp(size, (VkDeviceSize)16);
9357	}
9358	return VmaNextPow2(size);
9359	}
9360	Node* FindAllocationNode(VkDeviceSize offset, uint32_t& outLevel) const;
9361	void DeleteNodeChildren(Node* node);
9362	bool ValidateNode(ValidationContext& ctx, const Node* parent, const Node* curr, uint32_t level, VkDeviceSize levelNodeSize) const;
9363	uint32_t AllocSizeToLevel(VkDeviceSize allocSize) const;
9364	void AddNodeToDetailedStatistics(VmaDetailedStatistics& inoutStats, const Node* node, VkDeviceSize levelNodeSize) const;
9365	// Adds node to the front of FreeList at given level.
9366	// node->type must be FREE.
9367	// node->free.prev, next can be undefined.
9368	void AddToFreeListFront(uint32_t level, Node* node);
9369	// Removes node from FreeList at given level.
9370	// node->type must be FREE.
9371	// node->free.prev, next stay untouched.
9372	void RemoveFromFreeList(uint32_t level, Node* node);
9373	void DebugLogAllAllocationNode(Node* node, uint32_t level) const;
9374
9375	#if VMA_STATS_STRING_ENABLED
9376	void PrintDetailedMapNode(class VmaJsonWriter& json, const Node* node, VkDeviceSize levelNodeSize) const;
9377	#endif
9378	};
9379
9380	#ifndef _VMA_BLOCK_METADATA_BUDDY_FUNCTIONS
9381	VmaBlockMetadata_Buddy::VmaBlockMetadata_Buddy(const VkAllocationCallbacks* pAllocationCallbacks,
9382	VkDeviceSize bufferImageGranularity, bool isVirtual)
9383	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
9384	m_NodeAllocator(pAllocationCallbacks, 32), // firstBlockCapacity
9385	m_Root(VMA_NULL),
9386	m_AllocationCount(0),
9387	m_FreeCount(1),
9388	m_SumFreeSize(0)
9389	{
9390	memset(m_FreeList, 0, sizeof(m_FreeList));
9391	}
9392
9393	VmaBlockMetadata_Buddy::~VmaBlockMetadata_Buddy()
9394	{
9395	DeleteNodeChildren(m_Root);
9396	m_NodeAllocator.Free(m_Root);
9397	}
9398
9399	void VmaBlockMetadata_Buddy::Init(VkDeviceSize size)
9400	{
9401	VmaBlockMetadata::Init(size);
9402
9403	m_UsableSize = VmaPrevPow2(size);
9404	m_SumFreeSize = m_UsableSize;
9405
9406	// Calculate m_LevelCount.
9407	const VkDeviceSize minNodeSize = IsVirtual() ? 1 : 16;
9408	m_LevelCount = 1;
9409	while (m_LevelCount < MAX_LEVELS &&
9410	LevelToNodeSize(m_LevelCount) >= minNodeSize)
9411	{
9412	++m_LevelCount;
9413	}
9414
9415	Node* rootNode = m_NodeAllocator.Alloc();
9416	rootNode->offset = 0;
9417	rootNode->type = Node::TYPE_FREE;
9418	rootNode->parent = VMA_NULL;
9419	rootNode->buddy = VMA_NULL;
9420
9421	m_Root = rootNode;
9422	AddToFreeListFront(0, rootNode);
9423	}
9424
9425	bool VmaBlockMetadata_Buddy::Validate() const
9426	{
9427	// Validate tree.
9428	ValidationContext ctx;
9429	if (!ValidateNode(ctx, VMA_NULL, m_Root, 0, LevelToNodeSize(0)))
9430	{
9431	VMA_VALIDATE(false && "ValidateNode failed.");
9432	}
9433	VMA_VALIDATE(m_AllocationCount == ctx.calculatedAllocationCount);
9434	VMA_VALIDATE(m_SumFreeSize == ctx.calculatedSumFreeSize);
9435
9436	// Validate free node lists.
9437	for (uint32_t level = 0; level < m_LevelCount; ++level)
9438	{
9439	VMA_VALIDATE(m_FreeList[level].front == VMA_NULL ||
9440	m_FreeList[level].front->free.prev == VMA_NULL);
9441
9442	for (Node* node = m_FreeList[level].front;
9443	node != VMA_NULL;
9444	node = node->free.next)
9445	{
9446	VMA_VALIDATE(node->type == Node::TYPE_FREE);
9447
9448	if (node->free.next == VMA_NULL)
9449	{
9450	VMA_VALIDATE(m_FreeList[level].back == node);
9451	}
9452	else
9453	{
9454	VMA_VALIDATE(node->free.next->free.prev == node);
9455	}
9456	}
9457	}
9458
9459	// Validate that free lists ar higher levels are empty.
9460	for (uint32_t level = m_LevelCount; level < MAX_LEVELS; ++level)
9461	{
9462	VMA_VALIDATE(m_FreeList[level].front == VMA_NULL && m_FreeList[level].back == VMA_NULL);
9463	}
9464
9465	return true;
9466	}
9467
9468	void VmaBlockMetadata_Buddy::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
9469	{
9470	inoutStats.statistics.blockCount++;
9471	inoutStats.statistics.blockBytes += GetSize();
9472
9473	AddNodeToDetailedStatistics(inoutStats, m_Root, LevelToNodeSize(0));
9474
9475	const VkDeviceSize unusableSize = GetUnusableSize();
9476	if (unusableSize > 0)
9477	VmaAddDetailedStatisticsUnusedRange(inoutStats, unusableSize);
9478	}
9479
9480	void VmaBlockMetadata_Buddy::AddStatistics(VmaStatistics& inoutStats) const
9481	{
9482	inoutStats.blockCount++;
9483	inoutStats.allocationCount += (uint32_t)m_AllocationCount;
9484	inoutStats.blockBytes += GetSize();
9485	inoutStats.allocationBytes += GetSize() - m_SumFreeSize;
9486	}
9487
9488	#if VMA_STATS_STRING_ENABLED
9489	void VmaBlockMetadata_Buddy::PrintDetailedMap(class VmaJsonWriter& json, uint32_t mapRefCount) const
9490	{
9491	VmaDetailedStatistics stats;
9492	VmaClearDetailedStatistics(stats);
9493	AddDetailedStatistics(stats);
9494
9495	PrintDetailedMap_Begin(
9496	json,
9497	stats.statistics.blockBytes - stats.statistics.allocationBytes,
9498	stats.statistics.allocationCount,
9499	stats.unusedRangeCount,
9500	mapRefCount);
9501
9502	PrintDetailedMapNode(json, m_Root, LevelToNodeSize(0));
9503
9504	const VkDeviceSize unusableSize = GetUnusableSize();
9505	if (unusableSize > 0)
9506	{
9507	PrintDetailedMap_UnusedRange(json,
9508	m_UsableSize, // offset
9509	unusableSize); // size
9510	}
9511
9512	PrintDetailedMap_End(json);
9513	}
9514	#endif // VMA_STATS_STRING_ENABLED
9515
9516	bool VmaBlockMetadata_Buddy::CreateAllocationRequest(
9517	VkDeviceSize allocSize,
9518	VkDeviceSize allocAlignment,
9519	bool upperAddress,
9520	VmaSuballocationType allocType,
9521	uint32_t strategy,
9522	VmaAllocationRequest* pAllocationRequest)
9523	{
9524	VMA_ASSERT(!upperAddress && "VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT can be used only with linear algorithm.");
9525
9526	allocSize = AlignAllocationSize(allocSize);
9527
9528	// Simple way to respect bufferImageGranularity. May be optimized some day.
9529	// Whenever it might be an OPTIMAL image...
9530	if (allocType == VMA_SUBALLOCATION_TYPE_UNKNOWN ||
9531	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN ||
9532	allocType == VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL)
9533	{
9534	allocAlignment = VMA_MAX(allocAlignment, GetBufferImageGranularity());
9535	allocSize = VmaAlignUp(allocSize, GetBufferImageGranularity());
9536	}
9537
9538	if (allocSize > m_UsableSize)
9539	{
9540	return false;
9541	}
9542
9543	const uint32_t targetLevel = AllocSizeToLevel(allocSize);
9544	for (uint32_t level = targetLevel; level--; )
9545	{
9546	for (Node* freeNode = m_FreeList[level].front;
9547	freeNode != VMA_NULL;
9548	freeNode = freeNode->free.next)
9549	{
9550	if (freeNode->offset % allocAlignment == 0)
9551	{
9552	pAllocationRequest->type = VmaAllocationRequestType::Normal;
9553	pAllocationRequest->allocHandle = (VmaAllocHandle)(freeNode->offset + 1);
9554	pAllocationRequest->size = allocSize;
9555	pAllocationRequest->customData = (void*)(uintptr_t)level;
9556	return true;
9557	}
9558	}
9559	}
9560
9561	return false;
9562	}
9563
9564	void VmaBlockMetadata_Buddy::Alloc(
9565	const VmaAllocationRequest& request,
9566	VmaSuballocationType type,
9567	void* userData)
9568	{
9569	VMA_ASSERT(request.type == VmaAllocationRequestType::Normal);
9570
9571	const uint32_t targetLevel = AllocSizeToLevel(request.size);
9572	uint32_t currLevel = (uint32_t)(uintptr_t)request.customData;
9573
9574	Node* currNode = m_FreeList[currLevel].front;
9575	VMA_ASSERT(currNode != VMA_NULL && currNode->type == Node::TYPE_FREE);
9576	const VkDeviceSize offset = (VkDeviceSize)request.allocHandle - 1;
9577	while (currNode->offset != offset)
9578	{
9579	currNode = currNode->free.next;
9580	VMA_ASSERT(currNode != VMA_NULL && currNode->type == Node::TYPE_FREE);
9581	}
9582
9583	// Go down, splitting free nodes.
9584	while (currLevel < targetLevel)
9585	{
9586	// currNode is already first free node at currLevel.
9587	// Remove it from list of free nodes at this currLevel.
9588	RemoveFromFreeList(currLevel, currNode);
9589
9590	const uint32_t childrenLevel = currLevel + 1;
9591
9592	// Create two free sub-nodes.
9593	Node* leftChild = m_NodeAllocator.Alloc();
9594	Node* rightChild = m_NodeAllocator.Alloc();
9595
9596	leftChild->offset = currNode->offset;
9597	leftChild->type = Node::TYPE_FREE;
9598	leftChild->parent = currNode;
9599	leftChild->buddy = rightChild;
9600
9601	rightChild->offset = currNode->offset + LevelToNodeSize(childrenLevel);
9602	rightChild->type = Node::TYPE_FREE;
9603	rightChild->parent = currNode;
9604	rightChild->buddy = leftChild;
9605
9606	// Convert current currNode to split type.
9607	currNode->type = Node::TYPE_SPLIT;
9608	currNode->split.leftChild = leftChild;
9609
9610	// Add child nodes to free list. Order is important!
9611	AddToFreeListFront(childrenLevel, rightChild);
9612	AddToFreeListFront(childrenLevel, leftChild);
9613
9614	++m_FreeCount;
9615	++currLevel;
9616	currNode = m_FreeList[currLevel].front;
9617
9618	/*
9619	We can be sure that currNode, as left child of node previously split,
9620	also fulfills the alignment requirement.
9621	*/
9622	}
9623
9624	// Remove from free list.
9625	VMA_ASSERT(currLevel == targetLevel &&
9626	currNode != VMA_NULL &&
9627	currNode->type == Node::TYPE_FREE);
9628	RemoveFromFreeList(currLevel, currNode);
9629
9630	// Convert to allocation node.
9631	currNode->type = Node::TYPE_ALLOCATION;
9632	currNode->allocation.userData = userData;
9633
9634	++m_AllocationCount;
9635	--m_FreeCount;
9636	m_SumFreeSize -= request.size;
9637	}
9638
9639	void VmaBlockMetadata_Buddy::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
9640	{
9641	uint32_t level = 0;
9642	outInfo.offset = (VkDeviceSize)allocHandle - 1;
9643	const Node* const node = FindAllocationNode(outInfo.offset, level);
9644	outInfo.size = LevelToNodeSize(level);
9645	outInfo.pUserData = node->allocation.userData;
9646	}
9647
9648	void* VmaBlockMetadata_Buddy::GetAllocationUserData(VmaAllocHandle allocHandle) const
9649	{
9650	uint32_t level = 0;
9651	const Node* const node = FindAllocationNode((VkDeviceSize)allocHandle - 1, level);
9652	return node->allocation.userData;
9653	}
9654
9655	VmaAllocHandle VmaBlockMetadata_Buddy::GetAllocationListBegin() const
9656	{
9657	// Function only used for defragmentation, which is disabled for this algorithm
9658	return VK_NULL_HANDLE;
9659	}
9660
9661	VmaAllocHandle VmaBlockMetadata_Buddy::GetNextAllocation(VmaAllocHandle prevAlloc) const
9662	{
9663	// Function only used for defragmentation, which is disabled for this algorithm
9664	return VK_NULL_HANDLE;
9665	}
9666
9667	void VmaBlockMetadata_Buddy::DeleteNodeChildren(Node* node)
9668	{
9669	if (node->type == Node::TYPE_SPLIT)
9670	{
9671	DeleteNodeChildren(node->split.leftChild->buddy);
9672	DeleteNodeChildren(node->split.leftChild);
9673	const VkAllocationCallbacks* allocationCallbacks = GetAllocationCallbacks();
9674	m_NodeAllocator.Free(node->split.leftChild->buddy);
9675	m_NodeAllocator.Free(node->split.leftChild);
9676	}
9677	}
9678
9679	void VmaBlockMetadata_Buddy::Clear()
9680	{
9681	DeleteNodeChildren(m_Root);
9682	m_Root->type = Node::TYPE_FREE;
9683	m_AllocationCount = 0;
9684	m_FreeCount = 1;
9685	m_SumFreeSize = m_UsableSize;
9686	}
9687
9688	void VmaBlockMetadata_Buddy::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
9689	{
9690	uint32_t level = 0;
9691	Node* const node = FindAllocationNode((VkDeviceSize)allocHandle - 1, level);
9692	node->allocation.userData = userData;
9693	}
9694
9695	VmaBlockMetadata_Buddy::Node* VmaBlockMetadata_Buddy::FindAllocationNode(VkDeviceSize offset, uint32_t& outLevel) const
9696	{
9697	Node* node = m_Root;
9698	VkDeviceSize nodeOffset = 0;
9699	outLevel = 0;
9700	VkDeviceSize levelNodeSize = LevelToNodeSize(0);
9701	while (node->type == Node::TYPE_SPLIT)
9702	{
9703	const VkDeviceSize nextLevelNodeSize = levelNodeSize >> 1;
9704	if (offset < nodeOffset + nextLevelNodeSize)
9705	{
9706	node = node->split.leftChild;
9707	}
9708	else
9709	{
9710	node = node->split.leftChild->buddy;
9711	nodeOffset += nextLevelNodeSize;
9712	}
9713	++outLevel;
9714	levelNodeSize = nextLevelNodeSize;
9715	}
9716
9717	VMA_ASSERT(node != VMA_NULL && node->type == Node::TYPE_ALLOCATION);
9718	return node;
9719	}
9720
9721	bool VmaBlockMetadata_Buddy::ValidateNode(ValidationContext& ctx, const Node* parent, const Node* curr, uint32_t level, VkDeviceSize levelNodeSize) const
9722	{
9723	VMA_VALIDATE(level < m_LevelCount);
9724	VMA_VALIDATE(curr->parent == parent);
9725	VMA_VALIDATE((curr->buddy == VMA_NULL) == (parent == VMA_NULL));
9726	VMA_VALIDATE(curr->buddy == VMA_NULL || curr->buddy->buddy == curr);
9727	switch (curr->type)
9728	{
9729	case Node::TYPE_FREE:
9730	// curr->free.prev, next are validated separately.
9731	ctx.calculatedSumFreeSize += levelNodeSize;
9732	++ctx.calculatedFreeCount;
9733	break;
9734	case Node::TYPE_ALLOCATION:
9735	++ctx.calculatedAllocationCount;
9736	if (!IsVirtual())
9737	{
9738	VMA_VALIDATE(curr->allocation.userData != VMA_NULL);
9739	}
9740	break;
9741	case Node::TYPE_SPLIT:
9742	{
9743	const uint32_t childrenLevel = level + 1;
9744	const VkDeviceSize childrenLevelNodeSize = levelNodeSize >> 1;
9745	const Node* const leftChild = curr->split.leftChild;
9746	VMA_VALIDATE(leftChild != VMA_NULL);
9747	VMA_VALIDATE(leftChild->offset == curr->offset);
9748	if (!ValidateNode(ctx, curr, leftChild, childrenLevel, childrenLevelNodeSize))
9749	{
9750	VMA_VALIDATE(false && "ValidateNode for left child failed.");
9751	}
9752	const Node* const rightChild = leftChild->buddy;
9753	VMA_VALIDATE(rightChild->offset == curr->offset + childrenLevelNodeSize);
9754	if (!ValidateNode(ctx, curr, rightChild, childrenLevel, childrenLevelNodeSize))
9755	{
9756	VMA_VALIDATE(false && "ValidateNode for right child failed.");
9757	}
9758	}
9759	break;
9760	default:
9761	return false;
9762	}
9763
9764	return true;
9765	}
9766
9767	uint32_t VmaBlockMetadata_Buddy::AllocSizeToLevel(VkDeviceSize allocSize) const
9768	{
9769	// I know this could be optimized somehow e.g. by using std::log2p1 from C++20.
9770	uint32_t level = 0;
9771	VkDeviceSize currLevelNodeSize = m_UsableSize;
9772	VkDeviceSize nextLevelNodeSize = currLevelNodeSize >> 1;
9773	while (allocSize <= nextLevelNodeSize && level + 1 < m_LevelCount)
9774	{
9775	++level;
9776	currLevelNodeSize >>= 1;
9777	nextLevelNodeSize >>= 1;
9778	}
9779	return level;
9780	}
9781
9782	void VmaBlockMetadata_Buddy::Free(VmaAllocHandle allocHandle)
9783	{
9784	uint32_t level = 0;
9785	Node* node = FindAllocationNode((VkDeviceSize)allocHandle - 1, level);
9786
9787	++m_FreeCount;
9788	--m_AllocationCount;
9789	m_SumFreeSize += LevelToNodeSize(level);
9790
9791	node->type = Node::TYPE_FREE;
9792
9793	// Join free nodes if possible.
9794	while (level > 0 && node->buddy->type == Node::TYPE_FREE)
9795	{
9796	RemoveFromFreeList(level, node->buddy);
9797	Node* const parent = node->parent;
9798
9799	m_NodeAllocator.Free(node->buddy);
9800	m_NodeAllocator.Free(node);
9801	parent->type = Node::TYPE_FREE;
9802
9803	node = parent;
9804	--level;
9805	--m_FreeCount;
9806	}
9807
9808	AddToFreeListFront(level, node);
9809	}
9810
9811	void VmaBlockMetadata_Buddy::AddNodeToDetailedStatistics(VmaDetailedStatistics& inoutStats, const Node* node, VkDeviceSize levelNodeSize) const
9812	{
9813	switch (node->type)
9814	{
9815	case Node::TYPE_FREE:
9816	VmaAddDetailedStatisticsUnusedRange(inoutStats, levelNodeSize);
9817	break;
9818	case Node::TYPE_ALLOCATION:
9819	VmaAddDetailedStatisticsAllocation(inoutStats, levelNodeSize);
9820	break;
9821	case Node::TYPE_SPLIT:
9822	{
9823	const VkDeviceSize childrenNodeSize = levelNodeSize / 2;
9824	const Node* const leftChild = node->split.leftChild;
9825	AddNodeToDetailedStatistics(inoutStats, leftChild, childrenNodeSize);
9826	const Node* const rightChild = leftChild->buddy;
9827	AddNodeToDetailedStatistics(inoutStats, rightChild, childrenNodeSize);
9828	}
9829	break;
9830	default:
9831	VMA_ASSERT(0);
9832	}
9833	}
9834
9835	void VmaBlockMetadata_Buddy::AddToFreeListFront(uint32_t level, Node* node)
9836	{
9837	VMA_ASSERT(node->type == Node::TYPE_FREE);
9838
9839	// List is empty.
9840	Node* const frontNode = m_FreeList[level].front;
9841	if (frontNode == VMA_NULL)
9842	{
9843	VMA_ASSERT(m_FreeList[level].back == VMA_NULL);
9844	node->free.prev = node->free.next = VMA_NULL;
9845	m_FreeList[level].front = m_FreeList[level].back = node;
9846	}
9847	else
9848	{
9849	VMA_ASSERT(frontNode->free.prev == VMA_NULL);
9850	node->free.prev = VMA_NULL;
9851	node->free.next = frontNode;
9852	frontNode->free.prev = node;
9853	m_FreeList[level].front = node;
9854	}
9855	}
9856
9857	void VmaBlockMetadata_Buddy::RemoveFromFreeList(uint32_t level, Node* node)
9858	{
9859	VMA_ASSERT(m_FreeList[level].front != VMA_NULL);
9860
9861	// It is at the front.
9862	if (node->free.prev == VMA_NULL)
9863	{
9864	VMA_ASSERT(m_FreeList[level].front == node);
9865	m_FreeList[level].front = node->free.next;
9866	}
9867	else
9868	{
9869	Node* const prevFreeNode = node->free.prev;
9870	VMA_ASSERT(prevFreeNode->free.next == node);
9871	prevFreeNode->free.next = node->free.next;
9872	}
9873
9874	// It is at the back.
9875	if (node->free.next == VMA_NULL)
9876	{
9877	VMA_ASSERT(m_FreeList[level].back == node);
9878	m_FreeList[level].back = node->free.prev;
9879	}
9880	else
9881	{
9882	Node* const nextFreeNode = node->free.next;
9883	VMA_ASSERT(nextFreeNode->free.prev == node);
9884	nextFreeNode->free.prev = node->free.prev;
9885	}
9886	}
9887
9888	void VmaBlockMetadata_Buddy::DebugLogAllAllocationNode(Node* node, uint32_t level) const
9889	{
9890	switch (node->type)
9891	{
9892	case Node::TYPE_FREE:
9893	break;
9894	case Node::TYPE_ALLOCATION:
9895	DebugLogAllocation(node->offset, LevelToNodeSize(level), node->allocation.userData);
9896	break;
9897	case Node::TYPE_SPLIT:
9898	{
9899	++level;
9900	DebugLogAllAllocationNode(node->split.leftChild, level);
9901	DebugLogAllAllocationNode(node->split.leftChild->buddy, level);
9902	}
9903	break;
9904	default:
9905	VMA_ASSERT(0);
9906	}
9907	}
9908
9909	#if VMA_STATS_STRING_ENABLED
9910	void VmaBlockMetadata_Buddy::PrintDetailedMapNode(class VmaJsonWriter& json, const Node* node, VkDeviceSize levelNodeSize) const
9911	{
9912	switch (node->type)
9913	{
9914	case Node::TYPE_FREE:
9915	PrintDetailedMap_UnusedRange(json, node->offset, levelNodeSize);
9916	break;
9917	case Node::TYPE_ALLOCATION:
9918	PrintDetailedMap_Allocation(json, node->offset, levelNodeSize, node->allocation.userData);
9919	break;
9920	case Node::TYPE_SPLIT:
9921	{
9922	const VkDeviceSize childrenNodeSize = levelNodeSize / 2;
9923	const Node* const leftChild = node->split.leftChild;
9924	PrintDetailedMapNode(json, leftChild, childrenNodeSize);
9925	const Node* const rightChild = leftChild->buddy;
9926	PrintDetailedMapNode(json, rightChild, childrenNodeSize);
9927	}
9928	break;
9929	default:
9930	VMA_ASSERT(0);
9931	}
9932	}
9933	#endif // VMA_STATS_STRING_ENABLED
9934	#endif // _VMA_BLOCK_METADATA_BUDDY_FUNCTIONS
9935	#endif // _VMA_BLOCK_METADATA_BUDDY
9936	#endif // #if 0
9937
9938	#ifndef _VMA_BLOCK_METADATA_TLSF
9939	// To not search current larger region if first allocation won't succeed and skip to smaller range
9940	// use with VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT as strategy in CreateAllocationRequest().
9941	// When fragmentation and reusal of previous blocks doesn't matter then use with
9942	// VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT for fastest alloc time possible.
9943	class VmaBlockMetadata_TLSF : public VmaBlockMetadata
9944	{
9945	VMA_CLASS_NO_COPY(VmaBlockMetadata_TLSF)
9946	public:
9947	VmaBlockMetadata_TLSF(const VkAllocationCallbacks* pAllocationCallbacks,
9948	VkDeviceSize bufferImageGranularity, bool isVirtual);
9949	virtual ~VmaBlockMetadata_TLSF();
9950
9951	size_t GetAllocationCount() const override { return m_AllocCount; }
9952	size_t GetFreeRegionsCount() const override { return m_BlocksFreeCount + 1; }
9953	VkDeviceSize GetSumFreeSize() const override { return m_BlocksFreeSize + m_NullBlock->size; }
9954	bool IsEmpty() const override { return m_NullBlock->offset == 0; }
9955	VkDeviceSize GetAllocationOffset(VmaAllocHandle allocHandle) const override { return ((Block*)allocHandle)->offset; };
9956
9957	void Init(VkDeviceSize size) override;
9958	bool Validate() const override;
9959
9960	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const override;
9961	void AddStatistics(VmaStatistics& inoutStats) const override;
9962
9963	#if VMA_STATS_STRING_ENABLED
9964	void PrintDetailedMap(class VmaJsonWriter& json) const override;
9965	#endif
9966
9967	bool CreateAllocationRequest(
9968	VkDeviceSize allocSize,
9969	VkDeviceSize allocAlignment,
9970	bool upperAddress,
9971	VmaSuballocationType allocType,
9972	uint32_t strategy,
9973	VmaAllocationRequest* pAllocationRequest) override;
9974
9975	VkResult CheckCorruption(const void* pBlockData) override;
9976	void Alloc(
9977	const VmaAllocationRequest& request,
9978	VmaSuballocationType type,
9979	void* userData) override;
9980
9981	void Free(VmaAllocHandle allocHandle) override;
9982	void GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo) override;
9983	void* GetAllocationUserData(VmaAllocHandle allocHandle) const override;
9984	VmaAllocHandle GetAllocationListBegin() const override;
9985	VmaAllocHandle GetNextAllocation(VmaAllocHandle prevAlloc) const override;
9986	VkDeviceSize GetNextFreeRegionSize(VmaAllocHandle alloc) const override;
9987	void Clear() override;
9988	void SetAllocationUserData(VmaAllocHandle allocHandle, void* userData) override;
9989	void DebugLogAllAllocations() const override;
9990
9991	private:
9992	// According to original paper it should be preferable 4 or 5:
9993	// M. Masmano, I. Ripoll, A. Crespo, and J. Real "TLSF: a New Dynamic Memory Allocator for Real-Time Systems"
9994	// http://www.gii.upv.es/tlsf/files/ecrts04_tlsf.pdf
9995	static const uint8_t SECOND_LEVEL_INDEX = 5;
9996	static const uint16_t SMALL_BUFFER_SIZE = 256;
9997	static const uint32_t INITIAL_BLOCK_ALLOC_COUNT = 16;
9998	static const uint8_t MEMORY_CLASS_SHIFT = 7;
9999	static const uint8_t MAX_MEMORY_CLASSES = 65 - MEMORY_CLASS_SHIFT;
10000
10001	class Block
10002	{
10003	public:
10004	VkDeviceSize offset;
10005	VkDeviceSize size;
10006	Block* prevPhysical;
10007	Block* nextPhysical;
10008
10009	void MarkFree() { prevFree = VMA_NULL; }
10010	void MarkTaken() { prevFree = this; }
10011	bool IsFree() const { return prevFree != this; }
10012	void*& UserData() { VMA_HEAVY_ASSERT(!IsFree()); return userData; }
10013	Block*& PrevFree() { return prevFree; }
10014	Block*& NextFree() { VMA_HEAVY_ASSERT(IsFree()); return nextFree; }
10015
10016	private:
10017	Block* prevFree; // Address of the same block here indicates that block is taken
10018	union
10019	{
10020	Block* nextFree;
10021	void* userData;
10022	};
10023	};
10024
10025	size_t m_AllocCount;
10026	// Total number of free blocks besides null block
10027	size_t m_BlocksFreeCount;
10028	// Total size of free blocks excluding null block
10029	VkDeviceSize m_BlocksFreeSize;
10030	uint32_t m_IsFreeBitmap;
10031	uint8_t m_MemoryClasses;
10032	uint32_t m_InnerIsFreeBitmap[MAX_MEMORY_CLASSES];
10033	uint32_t m_ListsCount;
10034	/*
10035	* 0: 0-3 lists for small buffers
10036	* 1+: 0-(2^SLI-1) lists for normal buffers
10037	*/
10038	Block** m_FreeList;
10039	VmaPoolAllocator<Block> m_BlockAllocator;
10040	Block* m_NullBlock;
10041	VmaBlockBufferImageGranularity m_GranularityHandler;
10042
10043	uint8_t SizeToMemoryClass(VkDeviceSize size) const;
10044	uint16_t SizeToSecondIndex(VkDeviceSize size, uint8_t memoryClass) const;
10045	uint32_t GetListIndex(uint8_t memoryClass, uint16_t secondIndex) const;
10046	uint32_t GetListIndex(VkDeviceSize size) const;
10047
10048	void RemoveFreeBlock(Block* block);
10049	void InsertFreeBlock(Block* block);
10050	void MergeBlock(Block* block, Block* prev);
10051
10052	Block* FindFreeBlock(VkDeviceSize size, uint32_t& listIndex) const;
10053	bool CheckBlock(
10054	Block& block,
10055	uint32_t listIndex,
10056	VkDeviceSize allocSize,
10057	VkDeviceSize allocAlignment,
10058	VmaSuballocationType allocType,
10059	VmaAllocationRequest* pAllocationRequest);
10060	};
10061
10062	#ifndef _VMA_BLOCK_METADATA_TLSF_FUNCTIONS
10063	VmaBlockMetadata_TLSF::VmaBlockMetadata_TLSF(const VkAllocationCallbacks* pAllocationCallbacks,
10064	VkDeviceSize bufferImageGranularity, bool isVirtual)
10065	: VmaBlockMetadata(pAllocationCallbacks, bufferImageGranularity, isVirtual),
10066	m_AllocCount(0),
10067	m_BlocksFreeCount(0),
10068	m_BlocksFreeSize(0),
10069	m_IsFreeBitmap(0),
10070	m_MemoryClasses(0),
10071	m_ListsCount(0),
10072	m_FreeList(VMA_NULL),
10073	m_BlockAllocator(pAllocationCallbacks, INITIAL_BLOCK_ALLOC_COUNT),
10074	m_NullBlock(VMA_NULL),
10075	m_GranularityHandler(bufferImageGranularity) {}
10076
10077	VmaBlockMetadata_TLSF::~VmaBlockMetadata_TLSF()
10078	{
10079	if (m_FreeList)
10080	vma_delete_array(pAllocationCallbacks: GetAllocationCallbacks(), ptr: m_FreeList, count: m_ListsCount);
10081	m_GranularityHandler.Destroy(pAllocationCallbacks: GetAllocationCallbacks());
10082	}
10083
10084	void VmaBlockMetadata_TLSF::Init(VkDeviceSize size)
10085	{
10086	VmaBlockMetadata::Init(size);
10087
10088	if (!IsVirtual())
10089	m_GranularityHandler.Init(pAllocationCallbacks: GetAllocationCallbacks(), size);
10090
10091	m_NullBlock = m_BlockAllocator.Alloc();
10092	m_NullBlock->size = size;
10093	m_NullBlock->offset = 0;
10094	m_NullBlock->prevPhysical = VMA_NULL;
10095	m_NullBlock->nextPhysical = VMA_NULL;
10096	m_NullBlock->MarkFree();
10097	m_NullBlock->NextFree() = VMA_NULL;
10098	m_NullBlock->PrevFree() = VMA_NULL;
10099	uint8_t memoryClass = SizeToMemoryClass(size);
10100	uint16_t sli = SizeToSecondIndex(size, memoryClass);
10101	m_ListsCount = (memoryClass == 0 ? 0 : (memoryClass - 1) * (1UL << SECOND_LEVEL_INDEX) + sli) + 1;
10102	if (IsVirtual())
10103	m_ListsCount += 1UL << SECOND_LEVEL_INDEX;
10104	else
10105	m_ListsCount += 4;
10106
10107	m_MemoryClasses = memoryClass + 2;
10108	memset(s: m_InnerIsFreeBitmap, c: 0, n: MAX_MEMORY_CLASSES * sizeof(uint32_t));
10109
10110	m_FreeList = vma_new_array(GetAllocationCallbacks(), Block*, m_ListsCount);
10111	memset(s: m_FreeList, c: 0, n: m_ListsCount * sizeof(Block*));
10112	}
10113
10114	bool VmaBlockMetadata_TLSF::Validate() const
10115	{
10116	VMA_VALIDATE(GetSumFreeSize() <= GetSize());
10117
10118	VkDeviceSize calculatedSize = m_NullBlock->size;
10119	VkDeviceSize calculatedFreeSize = m_NullBlock->size;
10120	size_t allocCount = 0;
10121	size_t freeCount = 0;
10122
10123	// Check integrity of free lists
10124	for (uint32_t list = 0; list < m_ListsCount; ++list)
10125	{
10126	Block* block = m_FreeList[list];
10127	if (block != VMA_NULL)
10128	{
10129	VMA_VALIDATE(block->IsFree());
10130	VMA_VALIDATE(block->PrevFree() == VMA_NULL);
10131	while (block->NextFree())
10132	{
10133	VMA_VALIDATE(block->NextFree()->IsFree());
10134	VMA_VALIDATE(block->NextFree()->PrevFree() == block);
10135	block = block->NextFree();
10136	}
10137	}
10138	}
10139
10140	VkDeviceSize nextOffset = m_NullBlock->offset;
10141	auto validateCtx = m_GranularityHandler.StartValidation(pAllocationCallbacks: GetAllocationCallbacks(), isVirutal: IsVirtual());
10142
10143	VMA_VALIDATE(m_NullBlock->nextPhysical == VMA_NULL);
10144	if (m_NullBlock->prevPhysical)
10145	{
10146	VMA_VALIDATE(m_NullBlock->prevPhysical->nextPhysical == m_NullBlock);
10147	}
10148	// Check all blocks
10149	for (Block* prev = m_NullBlock->prevPhysical; prev != VMA_NULL; prev = prev->prevPhysical)
10150	{
10151	VMA_VALIDATE(prev->offset + prev->size == nextOffset);
10152	nextOffset = prev->offset;
10153	calculatedSize += prev->size;
10154
10155	uint32_t listIndex = GetListIndex(size: prev->size);
10156	if (prev->IsFree())
10157	{
10158	++freeCount;
10159	// Check if free block belongs to free list
10160	Block* freeBlock = m_FreeList[listIndex];
10161	VMA_VALIDATE(freeBlock != VMA_NULL);
10162
10163	bool found = false;
10164	do
10165	{
10166	if (freeBlock == prev)
10167	found = true;
10168
10169	freeBlock = freeBlock->NextFree();
10170	} while (!found && freeBlock != VMA_NULL);
10171
10172	VMA_VALIDATE(found);
10173	calculatedFreeSize += prev->size;
10174	}
10175	else
10176	{
10177	++allocCount;
10178	// Check if taken block is not on a free list
10179	Block* freeBlock = m_FreeList[listIndex];
10180	while (freeBlock)
10181	{
10182	VMA_VALIDATE(freeBlock != prev);
10183	freeBlock = freeBlock->NextFree();
10184	}
10185
10186	if (!IsVirtual())
10187	{
10188	VMA_VALIDATE(m_GranularityHandler.Validate(validateCtx, prev->offset, prev->size));
10189	}
10190	}
10191
10192	if (prev->prevPhysical)
10193	{
10194	VMA_VALIDATE(prev->prevPhysical->nextPhysical == prev);
10195	}
10196	}
10197
10198	if (!IsVirtual())
10199	{
10200	VMA_VALIDATE(m_GranularityHandler.FinishValidation(validateCtx));
10201	}
10202
10203	VMA_VALIDATE(nextOffset == 0);
10204	VMA_VALIDATE(calculatedSize == GetSize());
10205	VMA_VALIDATE(calculatedFreeSize == GetSumFreeSize());
10206	VMA_VALIDATE(allocCount == m_AllocCount);
10207	VMA_VALIDATE(freeCount == m_BlocksFreeCount);
10208
10209	return true;
10210	}
10211
10212	void VmaBlockMetadata_TLSF::AddDetailedStatistics(VmaDetailedStatistics& inoutStats) const
10213	{
10214	inoutStats.statistics.blockCount++;
10215	inoutStats.statistics.blockBytes += GetSize();
10216	if (m_NullBlock->size > 0)
10217	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: m_NullBlock->size);
10218
10219	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10220	{
10221	if (block->IsFree())
10222	VmaAddDetailedStatisticsUnusedRange(inoutStats, size: block->size);
10223	else
10224	VmaAddDetailedStatisticsAllocation(inoutStats, size: block->size);
10225	}
10226	}
10227
10228	void VmaBlockMetadata_TLSF::AddStatistics(VmaStatistics& inoutStats) const
10229	{
10230	inoutStats.blockCount++;
10231	inoutStats.allocationCount += (uint32_t)m_AllocCount;
10232	inoutStats.blockBytes += GetSize();
10233	inoutStats.allocationBytes += GetSize() - GetSumFreeSize();
10234	}
10235
10236	#if VMA_STATS_STRING_ENABLED
10237	void VmaBlockMetadata_TLSF::PrintDetailedMap(class VmaJsonWriter& json) const
10238	{
10239	size_t blockCount = m_AllocCount + m_BlocksFreeCount;
10240	VmaStlAllocator<Block*> allocator(GetAllocationCallbacks());
10241	VmaVector<Block*, VmaStlAllocator<Block*>> blockList(blockCount, allocator);
10242
10243	size_t i = blockCount;
10244	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10245	{
10246	blockList[--i] = block;
10247	}
10248	VMA_ASSERT(i == 0);
10249
10250	VmaDetailedStatistics stats;
10251	VmaClearDetailedStatistics(outStats&: stats);
10252	AddDetailedStatistics(inoutStats&: stats);
10253
10254	PrintDetailedMap_Begin(json,
10255	unusedBytes: stats.statistics.blockBytes - stats.statistics.allocationBytes,
10256	allocationCount: stats.statistics.allocationCount,
10257	unusedRangeCount: stats.unusedRangeCount);
10258
10259	for (; i < blockCount; ++i)
10260	{
10261	Block* block = blockList[i];
10262	if (block->IsFree())
10263	PrintDetailedMap_UnusedRange(json, offset: block->offset, size: block->size);
10264	else
10265	PrintDetailedMap_Allocation(json, offset: block->offset, size: block->size, userData: block->UserData());
10266	}
10267	if (m_NullBlock->size > 0)
10268	PrintDetailedMap_UnusedRange(json, offset: m_NullBlock->offset, size: m_NullBlock->size);
10269
10270	PrintDetailedMap_End(json);
10271	}
10272	#endif
10273
10274	bool VmaBlockMetadata_TLSF::CreateAllocationRequest(
10275	VkDeviceSize allocSize,
10276	VkDeviceSize allocAlignment,
10277	bool upperAddress,
10278	VmaSuballocationType allocType,
10279	uint32_t strategy,
10280	VmaAllocationRequest* pAllocationRequest)
10281	{
10282	VMA_ASSERT(allocSize > 0 && "Cannot allocate empty block!");
10283	VMA_ASSERT(!upperAddress && "VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT can be used only with linear algorithm.");
10284
10285	// For small granularity round up
10286	if (!IsVirtual())
10287	m_GranularityHandler.RoundupAllocRequest(allocType, inOutAllocSize&: allocSize, inOutAllocAlignment&: allocAlignment);
10288
10289	allocSize += GetDebugMargin();
10290	// Quick check for too small pool
10291	if (allocSize > GetSumFreeSize())
10292	return false;
10293
10294	// If no free blocks in pool then check only null block
10295	if (m_BlocksFreeCount == 0)
10296	return CheckBlock(block&: *m_NullBlock, listIndex: m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest);
10297
10298	// Round up to the next block
10299	VkDeviceSize sizeForNextList = allocSize;
10300	VkDeviceSize smallSizeStep = SMALL_BUFFER_SIZE / (IsVirtual() ? 1 << SECOND_LEVEL_INDEX : 4);
10301	if (allocSize > SMALL_BUFFER_SIZE)
10302	{
10303	sizeForNextList += (1ULL << (VMA_BITSCAN_MSB(allocSize) - SECOND_LEVEL_INDEX));
10304	}
10305	else if (allocSize > SMALL_BUFFER_SIZE - smallSizeStep)
10306	sizeForNextList = SMALL_BUFFER_SIZE + 1;
10307	else
10308	sizeForNextList += smallSizeStep;
10309
10310	uint32_t nextListIndex = 0;
10311	uint32_t prevListIndex = 0;
10312	Block* nextListBlock = VMA_NULL;
10313	Block* prevListBlock = VMA_NULL;
10314
10315	// Check blocks according to strategies
10316	if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT)
10317	{
10318	// Quick check for larger block first
10319	nextListBlock = FindFreeBlock(size: sizeForNextList, listIndex&: nextListIndex);
10320	if (nextListBlock != VMA_NULL && CheckBlock(block&: *nextListBlock, listIndex: nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10321	return true;
10322
10323	// If not fitted then null block
10324	if (CheckBlock(block&: *m_NullBlock, listIndex: m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10325	return true;
10326
10327	// Null block failed, search larger bucket
10328	while (nextListBlock)
10329	{
10330	if (CheckBlock(block&: *nextListBlock, listIndex: nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10331	return true;
10332	nextListBlock = nextListBlock->NextFree();
10333	}
10334
10335	// Failed again, check best fit bucket
10336	prevListBlock = FindFreeBlock(size: allocSize, listIndex&: prevListIndex);
10337	while (prevListBlock)
10338	{
10339	if (CheckBlock(block&: *prevListBlock, listIndex: prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10340	return true;
10341	prevListBlock = prevListBlock->NextFree();
10342	}
10343	}
10344	else if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_MEMORY_BIT)
10345	{
10346	// Check best fit bucket
10347	prevListBlock = FindFreeBlock(size: allocSize, listIndex&: prevListIndex);
10348	while (prevListBlock)
10349	{
10350	if (CheckBlock(block&: *prevListBlock, listIndex: prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10351	return true;
10352	prevListBlock = prevListBlock->NextFree();
10353	}
10354
10355	// If failed check null block
10356	if (CheckBlock(block&: *m_NullBlock, listIndex: m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10357	return true;
10358
10359	// Check larger bucket
10360	nextListBlock = FindFreeBlock(size: sizeForNextList, listIndex&: nextListIndex);
10361	while (nextListBlock)
10362	{
10363	if (CheckBlock(block&: *nextListBlock, listIndex: nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10364	return true;
10365	nextListBlock = nextListBlock->NextFree();
10366	}
10367	}
10368	else if (strategy & VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT )
10369	{
10370	// Perform search from the start
10371	VmaStlAllocator<Block*> allocator(GetAllocationCallbacks());
10372	VmaVector<Block*, VmaStlAllocator<Block*>> blockList(m_BlocksFreeCount, allocator);
10373
10374	size_t i = m_BlocksFreeCount;
10375	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10376	{
10377	if (block->IsFree() && block->size >= allocSize)
10378	blockList[--i] = block;
10379	}
10380
10381	for (; i < m_BlocksFreeCount; ++i)
10382	{
10383	Block& block = *blockList[i];
10384	if (CheckBlock(block, listIndex: GetListIndex(size: block.size), allocSize, allocAlignment, allocType, pAllocationRequest))
10385	return true;
10386	}
10387
10388	// If failed check null block
10389	if (CheckBlock(block&: *m_NullBlock, listIndex: m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10390	return true;
10391
10392	// Whole range searched, no more memory
10393	return false;
10394	}
10395	else
10396	{
10397	// Check larger bucket
10398	nextListBlock = FindFreeBlock(size: sizeForNextList, listIndex&: nextListIndex);
10399	while (nextListBlock)
10400	{
10401	if (CheckBlock(block&: *nextListBlock, listIndex: nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10402	return true;
10403	nextListBlock = nextListBlock->NextFree();
10404	}
10405
10406	// If failed check null block
10407	if (CheckBlock(block&: *m_NullBlock, listIndex: m_ListsCount, allocSize, allocAlignment, allocType, pAllocationRequest))
10408	return true;
10409
10410	// Check best fit bucket
10411	prevListBlock = FindFreeBlock(size: allocSize, listIndex&: prevListIndex);
10412	while (prevListBlock)
10413	{
10414	if (CheckBlock(block&: *prevListBlock, listIndex: prevListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10415	return true;
10416	prevListBlock = prevListBlock->NextFree();
10417	}
10418	}
10419
10420	// Worst case, full search has to be done
10421	while (++nextListIndex < m_ListsCount)
10422	{
10423	nextListBlock = m_FreeList[nextListIndex];
10424	while (nextListBlock)
10425	{
10426	if (CheckBlock(block&: *nextListBlock, listIndex: nextListIndex, allocSize, allocAlignment, allocType, pAllocationRequest))
10427	return true;
10428	nextListBlock = nextListBlock->NextFree();
10429	}
10430	}
10431
10432	// No more memory sadly
10433	return false;
10434	}
10435
10436	VkResult VmaBlockMetadata_TLSF::CheckCorruption(const void* pBlockData)
10437	{
10438	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10439	{
10440	if (!block->IsFree())
10441	{
10442	if (!VmaValidateMagicValue(pData: pBlockData, offset: block->offset + block->size))
10443	{
10444	VMA_ASSERT(0 && "MEMORY CORRUPTION DETECTED AFTER VALIDATED ALLOCATION!");
10445	return VK_ERROR_UNKNOWN_COPY;
10446	}
10447	}
10448	}
10449
10450	return VK_SUCCESS;
10451	}
10452
10453	void VmaBlockMetadata_TLSF::Alloc(
10454	const VmaAllocationRequest& request,
10455	VmaSuballocationType type,
10456	void* userData)
10457	{
10458	VMA_ASSERT(request.type == VmaAllocationRequestType::TLSF);
10459
10460	// Get block and pop it from the free list
10461	Block* currentBlock = (Block*)request.allocHandle;
10462	VkDeviceSize offset = request.algorithmData;
10463	VMA_ASSERT(currentBlock != VMA_NULL);
10464	VMA_ASSERT(currentBlock->offset <= offset);
10465
10466	if (currentBlock != m_NullBlock)
10467	RemoveFreeBlock(block: currentBlock);
10468
10469	VkDeviceSize debugMargin = GetDebugMargin();
10470	VkDeviceSize misssingAlignment = offset - currentBlock->offset;
10471
10472	// Append missing alignment to prev block or create new one
10473	if (misssingAlignment)
10474	{
10475	Block* prevBlock = currentBlock->prevPhysical;
10476	VMA_ASSERT(prevBlock != VMA_NULL && "There should be no missing alignment at offset 0!");
10477
10478	if (prevBlock->IsFree() && prevBlock->size != debugMargin)
10479	{
10480	uint32_t oldList = GetListIndex(size: prevBlock->size);
10481	prevBlock->size += misssingAlignment;
10482	// Check if new size crosses list bucket
10483	if (oldList != GetListIndex(size: prevBlock->size))
10484	{
10485	prevBlock->size -= misssingAlignment;
10486	RemoveFreeBlock(block: prevBlock);
10487	prevBlock->size += misssingAlignment;
10488	InsertFreeBlock(block: prevBlock);
10489	}
10490	else
10491	m_BlocksFreeSize += misssingAlignment;
10492	}
10493	else
10494	{
10495	Block* newBlock = m_BlockAllocator.Alloc();
10496	currentBlock->prevPhysical = newBlock;
10497	prevBlock->nextPhysical = newBlock;
10498	newBlock->prevPhysical = prevBlock;
10499	newBlock->nextPhysical = currentBlock;
10500	newBlock->size = misssingAlignment;
10501	newBlock->offset = currentBlock->offset;
10502	newBlock->MarkTaken();
10503
10504	InsertFreeBlock(block: newBlock);
10505	}
10506
10507	currentBlock->size -= misssingAlignment;
10508	currentBlock->offset += misssingAlignment;
10509	}
10510
10511	VkDeviceSize size = request.size + debugMargin;
10512	if (currentBlock->size == size)
10513	{
10514	if (currentBlock == m_NullBlock)
10515	{
10516	// Setup new null block
10517	m_NullBlock = m_BlockAllocator.Alloc();
10518	m_NullBlock->size = 0;
10519	m_NullBlock->offset = currentBlock->offset + size;
10520	m_NullBlock->prevPhysical = currentBlock;
10521	m_NullBlock->nextPhysical = VMA_NULL;
10522	m_NullBlock->MarkFree();
10523	m_NullBlock->PrevFree() = VMA_NULL;
10524	m_NullBlock->NextFree() = VMA_NULL;
10525	currentBlock->nextPhysical = m_NullBlock;
10526	currentBlock->MarkTaken();
10527	}
10528	}
10529	else
10530	{
10531	VMA_ASSERT(currentBlock->size > size && "Proper block already found, shouldn't find smaller one!");
10532
10533	// Create new free block
10534	Block* newBlock = m_BlockAllocator.Alloc();
10535	newBlock->size = currentBlock->size - size;
10536	newBlock->offset = currentBlock->offset + size;
10537	newBlock->prevPhysical = currentBlock;
10538	newBlock->nextPhysical = currentBlock->nextPhysical;
10539	currentBlock->nextPhysical = newBlock;
10540	currentBlock->size = size;
10541
10542	if (currentBlock == m_NullBlock)
10543	{
10544	m_NullBlock = newBlock;
10545	m_NullBlock->MarkFree();
10546	m_NullBlock->NextFree() = VMA_NULL;
10547	m_NullBlock->PrevFree() = VMA_NULL;
10548	currentBlock->MarkTaken();
10549	}
10550	else
10551	{
10552	newBlock->nextPhysical->prevPhysical = newBlock;
10553	newBlock->MarkTaken();
10554	InsertFreeBlock(block: newBlock);
10555	}
10556	}
10557	currentBlock->UserData() = userData;
10558
10559	if (debugMargin > 0)
10560	{
10561	currentBlock->size -= debugMargin;
10562	Block* newBlock = m_BlockAllocator.Alloc();
10563	newBlock->size = debugMargin;
10564	newBlock->offset = currentBlock->offset + currentBlock->size;
10565	newBlock->prevPhysical = currentBlock;
10566	newBlock->nextPhysical = currentBlock->nextPhysical;
10567	newBlock->MarkTaken();
10568	currentBlock->nextPhysical->prevPhysical = newBlock;
10569	currentBlock->nextPhysical = newBlock;
10570	InsertFreeBlock(block: newBlock);
10571	}
10572
10573	if (!IsVirtual())
10574	m_GranularityHandler.AllocPages(allocType: (uint8_t)(uintptr_t)request.customData,
10575	offset: currentBlock->offset, size: currentBlock->size);
10576	++m_AllocCount;
10577	}
10578
10579	void VmaBlockMetadata_TLSF::Free(VmaAllocHandle allocHandle)
10580	{
10581	Block* block = (Block*)allocHandle;
10582	Block* next = block->nextPhysical;
10583	VMA_ASSERT(!block->IsFree() && "Block is already free!");
10584
10585	if (!IsVirtual())
10586	m_GranularityHandler.FreePages(offset: block->offset, size: block->size);
10587	--m_AllocCount;
10588
10589	VkDeviceSize debugMargin = GetDebugMargin();
10590	if (debugMargin > 0)
10591	{
10592	RemoveFreeBlock(block: next);
10593	MergeBlock(block: next, prev: block);
10594	block = next;
10595	next = next->nextPhysical;
10596	}
10597
10598	// Try merging
10599	Block* prev = block->prevPhysical;
10600	if (prev != VMA_NULL && prev->IsFree() && prev->size != debugMargin)
10601	{
10602	RemoveFreeBlock(block: prev);
10603	MergeBlock(block, prev);
10604	}
10605
10606	if (!next->IsFree())
10607	InsertFreeBlock(block);
10608	else if (next == m_NullBlock)
10609	MergeBlock(block: m_NullBlock, prev: block);
10610	else
10611	{
10612	RemoveFreeBlock(block: next);
10613	MergeBlock(block: next, prev: block);
10614	InsertFreeBlock(block: next);
10615	}
10616	}
10617
10618	void VmaBlockMetadata_TLSF::GetAllocationInfo(VmaAllocHandle allocHandle, VmaVirtualAllocationInfo& outInfo)
10619	{
10620	Block* block = (Block*)allocHandle;
10621	VMA_ASSERT(!block->IsFree() && "Cannot get allocation info for free block!");
10622	outInfo.offset = block->offset;
10623	outInfo.size = block->size;
10624	outInfo.pUserData = block->UserData();
10625	}
10626
10627	void* VmaBlockMetadata_TLSF::GetAllocationUserData(VmaAllocHandle allocHandle) const
10628	{
10629	Block* block = (Block*)allocHandle;
10630	VMA_ASSERT(!block->IsFree() && "Cannot get user data for free block!");
10631	return block->UserData();
10632	}
10633
10634	VmaAllocHandle VmaBlockMetadata_TLSF::GetAllocationListBegin() const
10635	{
10636	if (m_AllocCount == 0)
10637	return VK_NULL_HANDLE;
10638
10639	for (Block* block = m_NullBlock->prevPhysical; block; block = block->prevPhysical)
10640	{
10641	if (!block->IsFree())
10642	return (VmaAllocHandle)block;
10643	}
10644	VMA_ASSERT(false && "If m_AllocCount > 0 then should find any allocation!");
10645	return VK_NULL_HANDLE;
10646	}
10647
10648	VmaAllocHandle VmaBlockMetadata_TLSF::GetNextAllocation(VmaAllocHandle prevAlloc) const
10649	{
10650	Block* startBlock = (Block*)prevAlloc;
10651	VMA_ASSERT(!startBlock->IsFree() && "Incorrect block!");
10652
10653	for (Block* block = startBlock->prevPhysical; block; block = block->prevPhysical)
10654	{
10655	if (!block->IsFree())
10656	return (VmaAllocHandle)block;
10657	}
10658	return VK_NULL_HANDLE;
10659	}
10660
10661	VkDeviceSize VmaBlockMetadata_TLSF::GetNextFreeRegionSize(VmaAllocHandle alloc) const
10662	{
10663	Block* block = (Block*)alloc;
10664	VMA_ASSERT(!block->IsFree() && "Incorrect block!");
10665
10666	if (block->prevPhysical)
10667	return block->prevPhysical->IsFree() ? block->prevPhysical->size : 0;
10668	return 0;
10669	}
10670
10671	void VmaBlockMetadata_TLSF::Clear()
10672	{
10673	m_AllocCount = 0;
10674	m_BlocksFreeCount = 0;
10675	m_BlocksFreeSize = 0;
10676	m_IsFreeBitmap = 0;
10677	m_NullBlock->offset = 0;
10678	m_NullBlock->size = GetSize();
10679	Block* block = m_NullBlock->prevPhysical;
10680	m_NullBlock->prevPhysical = VMA_NULL;
10681	while (block)
10682	{
10683	Block* prev = block->prevPhysical;
10684	m_BlockAllocator.Free(ptr: block);
10685	block = prev;
10686	}
10687	memset(s: m_FreeList, c: 0, n: m_ListsCount * sizeof(Block*));
10688	memset(s: m_InnerIsFreeBitmap, c: 0, n: m_MemoryClasses * sizeof(uint32_t));
10689	m_GranularityHandler.Clear();
10690	}
10691
10692	void VmaBlockMetadata_TLSF::SetAllocationUserData(VmaAllocHandle allocHandle, void* userData)
10693	{
10694	Block* block = (Block*)allocHandle;
10695	VMA_ASSERT(!block->IsFree() && "Trying to set user data for not allocated block!");
10696	block->UserData() = userData;
10697	}
10698
10699	void VmaBlockMetadata_TLSF::DebugLogAllAllocations() const
10700	{
10701	for (Block* block = m_NullBlock->prevPhysical; block != VMA_NULL; block = block->prevPhysical)
10702	if (!block->IsFree())
10703	DebugLogAllocation(offset: block->offset, size: block->size, userData: block->UserData());
10704	}
10705
10706	uint8_t VmaBlockMetadata_TLSF::SizeToMemoryClass(VkDeviceSize size) const
10707	{
10708	if (size > SMALL_BUFFER_SIZE)
10709	return VMA_BITSCAN_MSB(size) - MEMORY_CLASS_SHIFT;
10710	return 0;
10711	}
10712
10713	uint16_t VmaBlockMetadata_TLSF::SizeToSecondIndex(VkDeviceSize size, uint8_t memoryClass) const
10714	{
10715	if (memoryClass == 0)
10716	{
10717	if (IsVirtual())
10718	return static_cast<uint16_t>((size - 1) / 8);
10719	else
10720	return static_cast<uint16_t>((size - 1) / 64);
10721	}
10722	return static_cast<uint16_t>((size >> (memoryClass + MEMORY_CLASS_SHIFT - SECOND_LEVEL_INDEX)) ^ (1U << SECOND_LEVEL_INDEX));
10723	}
10724
10725	uint32_t VmaBlockMetadata_TLSF::GetListIndex(uint8_t memoryClass, uint16_t secondIndex) const
10726	{
10727	if (memoryClass == 0)
10728	return secondIndex;
10729
10730	const uint32_t index = static_cast<uint32_t>(memoryClass - 1) * (1 << SECOND_LEVEL_INDEX) + secondIndex;
10731	if (IsVirtual())
10732	return index + (1 << SECOND_LEVEL_INDEX);
10733	else
10734	return index + 4;
10735	}
10736
10737	uint32_t VmaBlockMetadata_TLSF::GetListIndex(VkDeviceSize size) const
10738	{
10739	uint8_t memoryClass = SizeToMemoryClass(size);
10740	return GetListIndex(memoryClass, secondIndex: SizeToSecondIndex(size, memoryClass));
10741	}
10742
10743	void VmaBlockMetadata_TLSF::RemoveFreeBlock(Block* block)
10744	{
10745	VMA_ASSERT(block != m_NullBlock);
10746	VMA_ASSERT(block->IsFree());
10747
10748	if (block->NextFree() != VMA_NULL)
10749	block->NextFree()->PrevFree() = block->PrevFree();
10750	if (block->PrevFree() != VMA_NULL)
10751	block->PrevFree()->NextFree() = block->NextFree();
10752	else
10753	{
10754	uint8_t memClass = SizeToMemoryClass(size: block->size);
10755	uint16_t secondIndex = SizeToSecondIndex(size: block->size, memoryClass: memClass);
10756	uint32_t index = GetListIndex(memoryClass: memClass, secondIndex);
10757	VMA_ASSERT(m_FreeList[index] == block);
10758	m_FreeList[index] = block->NextFree();
10759	if (block->NextFree() == VMA_NULL)
10760	{
10761	m_InnerIsFreeBitmap[memClass] &= ~(1U << secondIndex);
10762	if (m_InnerIsFreeBitmap[memClass] == 0)
10763	m_IsFreeBitmap &= ~(1UL << memClass);
10764	}
10765	}
10766	block->MarkTaken();
10767	block->UserData() = VMA_NULL;
10768	--m_BlocksFreeCount;
10769	m_BlocksFreeSize -= block->size;
10770	}
10771
10772	void VmaBlockMetadata_TLSF::InsertFreeBlock(Block* block)
10773	{
10774	VMA_ASSERT(block != m_NullBlock);
10775	VMA_ASSERT(!block->IsFree() && "Cannot insert block twice!");
10776
10777	uint8_t memClass = SizeToMemoryClass(size: block->size);
10778	uint16_t secondIndex = SizeToSecondIndex(size: block->size, memoryClass: memClass);
10779	uint32_t index = GetListIndex(memoryClass: memClass, secondIndex);
10780	VMA_ASSERT(index < m_ListsCount);
10781	block->PrevFree() = VMA_NULL;
10782	block->NextFree() = m_FreeList[index];
10783	m_FreeList[index] = block;
10784	if (block->NextFree() != VMA_NULL)
10785	block->NextFree()->PrevFree() = block;
10786	else
10787	{
10788	m_InnerIsFreeBitmap[memClass] |= 1U << secondIndex;
10789	m_IsFreeBitmap |= 1UL << memClass;
10790	}
10791	++m_BlocksFreeCount;
10792	m_BlocksFreeSize += block->size;
10793	}
10794
10795	void VmaBlockMetadata_TLSF::MergeBlock(Block* block, Block* prev)
10796	{
10797	VMA_ASSERT(block->prevPhysical == prev && "Cannot merge seperate physical regions!");
10798	VMA_ASSERT(!prev->IsFree() && "Cannot merge block that belongs to free list!");
10799
10800	block->offset = prev->offset;
10801	block->size += prev->size;
10802	block->prevPhysical = prev->prevPhysical;
10803	if (block->prevPhysical)
10804	block->prevPhysical->nextPhysical = block;
10805	m_BlockAllocator.Free(ptr: prev);
10806	}
10807
10808	VmaBlockMetadata_TLSF::Block* VmaBlockMetadata_TLSF::FindFreeBlock(VkDeviceSize size, uint32_t& listIndex) const
10809	{
10810	uint8_t memoryClass = SizeToMemoryClass(size);
10811	uint32_t innerFreeMap = m_InnerIsFreeBitmap[memoryClass] & (~0U << SizeToSecondIndex(size, memoryClass));
10812	if (!innerFreeMap)
10813	{
10814	// Check higher levels for avaiable blocks
10815	uint32_t freeMap = m_IsFreeBitmap & (~0UL << (memoryClass + 1));
10816	if (!freeMap)
10817	return VMA_NULL; // No more memory avaible
10818
10819	// Find lowest free region
10820	memoryClass = VMA_BITSCAN_LSB(freeMap);
10821	innerFreeMap = m_InnerIsFreeBitmap[memoryClass];
10822	VMA_ASSERT(innerFreeMap != 0);
10823	}
10824	// Find lowest free subregion
10825	listIndex = GetListIndex(memoryClass, VMA_BITSCAN_LSB(innerFreeMap));
10826	VMA_ASSERT(m_FreeList[listIndex]);
10827	return m_FreeList[listIndex];
10828	}
10829
10830	bool VmaBlockMetadata_TLSF::CheckBlock(
10831	Block& block,
10832	uint32_t listIndex,
10833	VkDeviceSize allocSize,
10834	VkDeviceSize allocAlignment,
10835	VmaSuballocationType allocType,
10836	VmaAllocationRequest* pAllocationRequest)
10837	{
10838	VMA_ASSERT(block.IsFree() && "Block is already taken!");
10839
10840	VkDeviceSize alignedOffset = VmaAlignUp(val: block.offset, alignment: allocAlignment);
10841	if (block.size < allocSize + alignedOffset - block.offset)
10842	return false;
10843
10844	// Check for granularity conflicts
10845	if (!IsVirtual() &&
10846	m_GranularityHandler.CheckConflictAndAlignUp(inOutAllocOffset&: alignedOffset, allocSize, blockOffset: block.offset, blockSize: block.size, allocType))
10847	return false;
10848
10849	// Alloc successful
10850	pAllocationRequest->type = VmaAllocationRequestType::TLSF;
10851	pAllocationRequest->allocHandle = (VmaAllocHandle)&block;
10852	pAllocationRequest->size = allocSize - GetDebugMargin();
10853	pAllocationRequest->customData = (void*)allocType;
10854	pAllocationRequest->algorithmData = alignedOffset;
10855
10856	// Place block at the start of list if it's normal block
10857	if (listIndex != m_ListsCount && block.PrevFree())
10858	{
10859	block.PrevFree()->NextFree() = block.NextFree();
10860	if (block.NextFree())
10861	block.NextFree()->PrevFree() = block.PrevFree();
10862	block.PrevFree() = VMA_NULL;
10863	block.NextFree() = m_FreeList[listIndex];
10864	m_FreeList[listIndex] = &block;
10865	if (block.NextFree())
10866	block.NextFree()->PrevFree() = &block;
10867	}
10868
10869	return true;
10870	}
10871	#endif // _VMA_BLOCK_METADATA_TLSF_FUNCTIONS
10872	#endif // _VMA_BLOCK_METADATA_TLSF
10873
10874	#ifndef _VMA_BLOCK_VECTOR
10875	/*
10876	Sequence of VmaDeviceMemoryBlock. Represents memory blocks allocated for a specific
10877	Vulkan memory type.
10878
10879	Synchronized internally with a mutex.
10880	*/
10881	class VmaBlockVector
10882	{
10883	friend struct VmaDefragmentationContext_T;
10884	VMA_CLASS_NO_COPY(VmaBlockVector)
10885	public:
10886	VmaBlockVector(
10887	VmaAllocator hAllocator,
10888	VmaPool hParentPool,
10889	uint32_t memoryTypeIndex,
10890	VkDeviceSize preferredBlockSize,
10891	size_t minBlockCount,
10892	size_t maxBlockCount,
10893	VkDeviceSize bufferImageGranularity,
10894	bool explicitBlockSize,
10895	uint32_t algorithm,
10896	float priority,
10897	VkDeviceSize minAllocationAlignment,
10898	void* pMemoryAllocateNext);
10899	~VmaBlockVector();
10900
10901	VmaAllocator GetAllocator() const { return m_hAllocator; }
10902	VmaPool GetParentPool() const { return m_hParentPool; }
10903	bool IsCustomPool() const { return m_hParentPool != VMA_NULL; }
10904	uint32_t GetMemoryTypeIndex() const { return m_MemoryTypeIndex; }
10905	VkDeviceSize GetPreferredBlockSize() const { return m_PreferredBlockSize; }
10906	VkDeviceSize GetBufferImageGranularity() const { return m_BufferImageGranularity; }
10907	uint32_t GetAlgorithm() const { return m_Algorithm; }
10908	bool HasExplicitBlockSize() const { return m_ExplicitBlockSize; }
10909	float GetPriority() const { return m_Priority; }
10910	const void* GetAllocationNextPtr() const { return m_pMemoryAllocateNext; }
10911	// To be used only while the m_Mutex is locked. Used during defragmentation.
10912	size_t GetBlockCount() const { return m_Blocks.size(); }
10913	// To be used only while the m_Mutex is locked. Used during defragmentation.
10914	VmaDeviceMemoryBlock* GetBlock(size_t index) const { return m_Blocks[index]; }
10915	VMA_RW_MUTEX &GetMutex() { return m_Mutex; }
10916
10917	VkResult CreateMinBlocks();
10918	void AddStatistics(VmaStatistics& inoutStats);
10919	void AddDetailedStatistics(VmaDetailedStatistics& inoutStats);
10920	bool IsEmpty();
10921	bool IsCorruptionDetectionEnabled() const;
10922
10923	VkResult Allocate(
10924	VkDeviceSize size,
10925	VkDeviceSize alignment,
10926	const VmaAllocationCreateInfo& createInfo,
10927	VmaSuballocationType suballocType,
10928	size_t allocationCount,
10929	VmaAllocation* pAllocations);
10930
10931	void Free(const VmaAllocation hAllocation);
10932
10933	#if VMA_STATS_STRING_ENABLED
10934	void PrintDetailedMap(class VmaJsonWriter& json);
10935	#endif
10936
10937	VkResult CheckCorruption();
10938
10939	private:
10940	const VmaAllocator m_hAllocator;
10941	const VmaPool m_hParentPool;
10942	const uint32_t m_MemoryTypeIndex;
10943	const VkDeviceSize m_PreferredBlockSize;
10944	const size_t m_MinBlockCount;
10945	const size_t m_MaxBlockCount;
10946	const VkDeviceSize m_BufferImageGranularity;
10947	const bool m_ExplicitBlockSize;
10948	const uint32_t m_Algorithm;
10949	const float m_Priority;
10950	const VkDeviceSize m_MinAllocationAlignment;
10951
10952	void* const m_pMemoryAllocateNext;
10953	VMA_RW_MUTEX m_Mutex;
10954	// Incrementally sorted by sumFreeSize, ascending.
10955	VmaVector<VmaDeviceMemoryBlock*, VmaStlAllocator<VmaDeviceMemoryBlock*>> m_Blocks;
10956	uint32_t m_NextBlockId;
10957	bool m_IncrementalSort = true;
10958
10959	void SetIncrementalSort(bool val) { m_IncrementalSort = val; }
10960
10961	VkDeviceSize CalcMaxBlockSize() const;
10962	// Finds and removes given block from vector.
10963	void Remove(VmaDeviceMemoryBlock* pBlock);
10964	// Performs single step in sorting m_Blocks. They may not be fully sorted
10965	// after this call.
10966	void IncrementallySortBlocks();
10967	void SortByFreeSize();
10968
10969	VkResult AllocatePage(
10970	VkDeviceSize size,
10971	VkDeviceSize alignment,
10972	const VmaAllocationCreateInfo& createInfo,
10973	VmaSuballocationType suballocType,
10974	VmaAllocation* pAllocation);
10975
10976	VkResult AllocateFromBlock(
10977	VmaDeviceMemoryBlock* pBlock,
10978	VkDeviceSize size,
10979	VkDeviceSize alignment,
10980	VmaAllocationCreateFlags allocFlags,
10981	void* pUserData,
10982	VmaSuballocationType suballocType,
10983	uint32_t strategy,
10984	VmaAllocation* pAllocation);
10985
10986	VkResult CommitAllocationRequest(
10987	VmaAllocationRequest& allocRequest,
10988	VmaDeviceMemoryBlock* pBlock,
10989	VkDeviceSize alignment,
10990	VmaAllocationCreateFlags allocFlags,
10991	void* pUserData,
10992	VmaSuballocationType suballocType,
10993	VmaAllocation* pAllocation);
10994
10995	VkResult CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex);
10996	bool HasEmptyBlock();
10997	};
10998	#endif // _VMA_BLOCK_VECTOR
10999
11000	#ifndef _VMA_DEFRAGMENTATION_CONTEXT
11001	struct VmaDefragmentationContext_T
11002	{
11003	VMA_CLASS_NO_COPY(VmaDefragmentationContext_T)
11004	public:
11005	VmaDefragmentationContext_T(
11006	VmaAllocator hAllocator,
11007	const VmaDefragmentationInfo& info);
11008	~VmaDefragmentationContext_T();
11009
11010	void GetStats(VmaDefragmentationStats& outStats) { outStats = m_GlobalStats; }
11011
11012	VkResult DefragmentPassBegin(VmaDefragmentationPassMoveInfo& moveInfo);
11013	VkResult DefragmentPassEnd(VmaDefragmentationPassMoveInfo& moveInfo);
11014
11015	private:
11016	// Max number of allocations to ignore due to size constraints before ending single pass
11017	static const uint8_t MAX_ALLOCS_TO_IGNORE = 16;
11018	enum class CounterStatus { Pass, Ignore, End };
11019
11020	struct FragmentedBlock
11021	{
11022	uint32_t data;
11023	VmaDeviceMemoryBlock* block;
11024	};
11025	struct StateBalanced
11026	{
11027	VkDeviceSize avgFreeSize = 0;
11028	VkDeviceSize avgAllocSize = UINT64_MAX;
11029	};
11030	struct StateExtensive
11031	{
11032	enum class Operation : uint8_t
11033	{
11034	FindFreeBlockBuffer, FindFreeBlockTexture, FindFreeBlockAll,
11035	MoveBuffers, MoveTextures, MoveAll,
11036	Cleanup, Done
11037	};
11038
11039	Operation operation = Operation::FindFreeBlockTexture;
11040	size_t firstFreeBlock = SIZE_MAX;
11041	};
11042	struct MoveAllocationData
11043	{
11044	VkDeviceSize size;
11045	VkDeviceSize alignment;
11046	VmaSuballocationType type;
11047	VmaAllocationCreateFlags flags;
11048	VmaDefragmentationMove move = {};
11049	};
11050
11051	const VkDeviceSize m_MaxPassBytes;
11052	const uint32_t m_MaxPassAllocations;
11053
11054	VmaStlAllocator<VmaDefragmentationMove> m_MoveAllocator;
11055	VmaVector<VmaDefragmentationMove, VmaStlAllocator<VmaDefragmentationMove>> m_Moves;
11056
11057	uint8_t m_IgnoredAllocs = 0;
11058	uint32_t m_Algorithm;
11059	uint32_t m_BlockVectorCount;
11060	VmaBlockVector* m_PoolBlockVector;
11061	VmaBlockVector** m_pBlockVectors;
11062	size_t m_ImmovableBlockCount = 0;
11063	VmaDefragmentationStats m_GlobalStats = { .bytesMoved: 0 };
11064	VmaDefragmentationStats m_PassStats = { .bytesMoved: 0 };
11065	void* m_AlgorithmState = VMA_NULL;
11066
11067	static MoveAllocationData GetMoveData(VmaAllocHandle handle, VmaBlockMetadata* metadata);
11068	CounterStatus CheckCounters(VkDeviceSize bytes);
11069	bool IncrementCounters(VkDeviceSize bytes);
11070	bool ReallocWithinBlock(VmaBlockVector& vector, VmaDeviceMemoryBlock* block);
11071	bool AllocInOtherBlock(size_t start, size_t end, MoveAllocationData& data, VmaBlockVector& vector);
11072
11073	bool ComputeDefragmentation(VmaBlockVector& vector, size_t index);
11074	bool ComputeDefragmentation_Fast(VmaBlockVector& vector);
11075	bool ComputeDefragmentation_Balanced(VmaBlockVector& vector, size_t index, bool update);
11076	bool ComputeDefragmentation_Full(VmaBlockVector& vector);
11077	bool ComputeDefragmentation_Extensive(VmaBlockVector& vector, size_t index);
11078
11079	void UpdateVectorStatistics(VmaBlockVector& vector, StateBalanced& state);
11080	bool MoveDataToFreeBlocks(VmaSuballocationType currentType,
11081	VmaBlockVector& vector, size_t firstFreeBlock,
11082	bool& texturePresent, bool& bufferPresent, bool& otherPresent);
11083	};
11084	#endif // _VMA_DEFRAGMENTATION_CONTEXT
11085
11086	#ifndef _VMA_POOL_T
11087	struct VmaPool_T
11088	{
11089	friend struct VmaPoolListItemTraits;
11090	VMA_CLASS_NO_COPY(VmaPool_T)
11091	public:
11092	VmaBlockVector m_BlockVector;
11093	VmaDedicatedAllocationList m_DedicatedAllocations;
11094
11095	VmaPool_T(
11096	VmaAllocator hAllocator,
11097	const VmaPoolCreateInfo& createInfo,
11098	VkDeviceSize preferredBlockSize);
11099	~VmaPool_T();
11100
11101	uint32_t GetId() const { return m_Id; }
11102	void SetId(uint32_t id) { VMA_ASSERT(m_Id == 0); m_Id = id; }
11103
11104	const char* GetName() const { return m_Name; }
11105	void SetName(const char* pName);
11106
11107	#if VMA_STATS_STRING_ENABLED
11108	//void PrintDetailedMap(class VmaStringBuilder& sb);
11109	#endif
11110
11111	private:
11112	uint32_t m_Id;
11113	char* m_Name;
11114	VmaPool_T* m_PrevPool = VMA_NULL;
11115	VmaPool_T* m_NextPool = VMA_NULL;
11116	};
11117
11118	struct VmaPoolListItemTraits
11119	{
11120	typedef VmaPool_T ItemType;
11121
11122	static ItemType* GetPrev(const ItemType* item) { return item->m_PrevPool; }
11123	static ItemType* GetNext(const ItemType* item) { return item->m_NextPool; }
11124	static ItemType*& AccessPrev(ItemType* item) { return item->m_PrevPool; }
11125	static ItemType*& AccessNext(ItemType* item) { return item->m_NextPool; }
11126	};
11127	#endif // _VMA_POOL_T
11128
11129	#ifndef _VMA_CURRENT_BUDGET_DATA
11130	struct VmaCurrentBudgetData
11131	{
11132	VMA_ATOMIC_UINT32 m_BlockCount[VK_MAX_MEMORY_HEAPS];
11133	VMA_ATOMIC_UINT32 m_AllocationCount[VK_MAX_MEMORY_HEAPS];
11134	VMA_ATOMIC_UINT64 m_BlockBytes[VK_MAX_MEMORY_HEAPS];
11135	VMA_ATOMIC_UINT64 m_AllocationBytes[VK_MAX_MEMORY_HEAPS];
11136
11137	#if VMA_MEMORY_BUDGET
11138	VMA_ATOMIC_UINT32 m_OperationsSinceBudgetFetch;
11139	VMA_RW_MUTEX m_BudgetMutex;
11140	uint64_t m_VulkanUsage[VK_MAX_MEMORY_HEAPS];
11141	uint64_t m_VulkanBudget[VK_MAX_MEMORY_HEAPS];
11142	uint64_t m_BlockBytesAtBudgetFetch[VK_MAX_MEMORY_HEAPS];
11143	#endif // VMA_MEMORY_BUDGET
11144
11145	VmaCurrentBudgetData();
11146
11147	void AddAllocation(uint32_t heapIndex, VkDeviceSize allocationSize);
11148	void RemoveAllocation(uint32_t heapIndex, VkDeviceSize allocationSize);
11149	};
11150
11151	#ifndef _VMA_CURRENT_BUDGET_DATA_FUNCTIONS
11152	VmaCurrentBudgetData::VmaCurrentBudgetData()
11153	{
11154	for (uint32_t heapIndex = 0; heapIndex < VK_MAX_MEMORY_HEAPS; ++heapIndex)
11155	{
11156	m_BlockCount[heapIndex] = 0;
11157	m_AllocationCount[heapIndex] = 0;
11158	m_BlockBytes[heapIndex] = 0;
11159	m_AllocationBytes[heapIndex] = 0;
11160	#if VMA_MEMORY_BUDGET
11161	m_VulkanUsage[heapIndex] = 0;
11162	m_VulkanBudget[heapIndex] = 0;
11163	m_BlockBytesAtBudgetFetch[heapIndex] = 0;
11164	#endif
11165	}
11166
11167	#if VMA_MEMORY_BUDGET
11168	m_OperationsSinceBudgetFetch = 0;
11169	#endif
11170	}
11171
11172	void VmaCurrentBudgetData::AddAllocation(uint32_t heapIndex, VkDeviceSize allocationSize)
11173	{
11174	m_AllocationBytes[heapIndex] += allocationSize;
11175	++m_AllocationCount[heapIndex];
11176	#if VMA_MEMORY_BUDGET
11177	++m_OperationsSinceBudgetFetch;
11178	#endif
11179	}
11180
11181	void VmaCurrentBudgetData::RemoveAllocation(uint32_t heapIndex, VkDeviceSize allocationSize)
11182	{
11183	VMA_ASSERT(m_AllocationBytes[heapIndex] >= allocationSize);
11184	m_AllocationBytes[heapIndex] -= allocationSize;
11185	VMA_ASSERT(m_AllocationCount[heapIndex] > 0);
11186	--m_AllocationCount[heapIndex];
11187	#if VMA_MEMORY_BUDGET
11188	++m_OperationsSinceBudgetFetch;
11189	#endif
11190	}
11191	#endif // _VMA_CURRENT_BUDGET_DATA_FUNCTIONS
11192	#endif // _VMA_CURRENT_BUDGET_DATA
11193
11194	#ifndef _VMA_ALLOCATION_OBJECT_ALLOCATOR
11195	/*
11196	Thread-safe wrapper over VmaPoolAllocator free list, for allocation of VmaAllocation_T objects.
11197	*/
11198	class VmaAllocationObjectAllocator
11199	{
11200	VMA_CLASS_NO_COPY(VmaAllocationObjectAllocator)
11201	public:
11202	VmaAllocationObjectAllocator(const VkAllocationCallbacks* pAllocationCallbacks)
11203	: m_Allocator(pAllocationCallbacks, 1024) {}
11204
11205	template<typename... Types> VmaAllocation Allocate(Types&&... args);
11206	void Free(VmaAllocation hAlloc);
11207
11208	private:
11209	VMA_MUTEX m_Mutex;
11210	VmaPoolAllocator<VmaAllocation_T> m_Allocator;
11211	};
11212
11213	template<typename... Types>
11214	VmaAllocation VmaAllocationObjectAllocator::Allocate(Types&&... args)
11215	{
11216	VmaMutexLock mutexLock(m_Mutex);
11217	return m_Allocator.Alloc<Types...>(std::forward<Types>(args)...);
11218	}
11219
11220	void VmaAllocationObjectAllocator::Free(VmaAllocation hAlloc)
11221	{
11222	VmaMutexLock mutexLock(m_Mutex);
11223	m_Allocator.Free(ptr: hAlloc);
11224	}
11225	#endif // _VMA_ALLOCATION_OBJECT_ALLOCATOR
11226
11227	#ifndef _VMA_VIRTUAL_BLOCK_T
11228	struct VmaVirtualBlock_T
11229	{
11230	VMA_CLASS_NO_COPY(VmaVirtualBlock_T)
11231	public:
11232	const bool m_AllocationCallbacksSpecified;
11233	const VkAllocationCallbacks m_AllocationCallbacks;
11234
11235	VmaVirtualBlock_T(const VmaVirtualBlockCreateInfo& createInfo);
11236	~VmaVirtualBlock_T();
11237
11238	VkResult Init() { return VK_SUCCESS; }
11239	bool IsEmpty() const { return m_Metadata->IsEmpty(); }
11240	void Free(VmaVirtualAllocation allocation) { m_Metadata->Free(allocHandle: (VmaAllocHandle)allocation); }
11241	void SetAllocationUserData(VmaVirtualAllocation allocation, void* userData) { m_Metadata->SetAllocationUserData(allocHandle: (VmaAllocHandle)allocation, userData); }
11242	void Clear() { m_Metadata->Clear(); }
11243
11244	const VkAllocationCallbacks* GetAllocationCallbacks() const;
11245	void GetAllocationInfo(VmaVirtualAllocation allocation, VmaVirtualAllocationInfo& outInfo);
11246	VkResult Allocate(const VmaVirtualAllocationCreateInfo& createInfo, VmaVirtualAllocation& outAllocation,
11247	VkDeviceSize* outOffset);
11248	void GetStatistics(VmaStatistics& outStats) const;
11249	void CalculateDetailedStatistics(VmaDetailedStatistics& outStats) const;
11250	#if VMA_STATS_STRING_ENABLED
11251	void BuildStatsString(bool detailedMap, VmaStringBuilder& sb) const;
11252	#endif
11253
11254	private:
11255	VmaBlockMetadata* m_Metadata;
11256	};
11257
11258	#ifndef _VMA_VIRTUAL_BLOCK_T_FUNCTIONS
11259	VmaVirtualBlock_T::VmaVirtualBlock_T(const VmaVirtualBlockCreateInfo& createInfo)
11260	: m_AllocationCallbacksSpecified(createInfo.pAllocationCallbacks != VMA_NULL),
11261	m_AllocationCallbacks(createInfo.pAllocationCallbacks != VMA_NULL ? *createInfo.pAllocationCallbacks : VmaEmptyAllocationCallbacks)
11262	{
11263	const uint32_t algorithm = createInfo.flags & VMA_VIRTUAL_BLOCK_CREATE_ALGORITHM_MASK;
11264	switch (algorithm)
11265	{
11266	default:
11267	VMA_ASSERT(0);
11268	case 0:
11269	m_Metadata = vma_new(GetAllocationCallbacks(), VmaBlockMetadata_TLSF)(VK_NULL_HANDLE, 1, true);
11270	break;
11271	case VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT:
11272	m_Metadata = vma_new(GetAllocationCallbacks(), VmaBlockMetadata_Linear)(VK_NULL_HANDLE, 1, true);
11273	break;
11274	}
11275
11276	m_Metadata->Init(size: createInfo.size);
11277	}
11278
11279	VmaVirtualBlock_T::~VmaVirtualBlock_T()
11280	{
11281	// Define macro VMA_DEBUG_LOG to receive the list of the unfreed allocations
11282	if (!m_Metadata->IsEmpty())
11283	m_Metadata->DebugLogAllAllocations();
11284	// This is the most important assert in the entire library.
11285	// Hitting it means you have some memory leak - unreleased virtual allocations.
11286	VMA_ASSERT(m_Metadata->IsEmpty() && "Some virtual allocations were not freed before destruction of this virtual block!");
11287
11288	vma_delete(pAllocationCallbacks: GetAllocationCallbacks(), ptr: m_Metadata);
11289	}
11290
11291	const VkAllocationCallbacks* VmaVirtualBlock_T::GetAllocationCallbacks() const
11292	{
11293	return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : VMA_NULL;
11294	}
11295
11296	void VmaVirtualBlock_T::GetAllocationInfo(VmaVirtualAllocation allocation, VmaVirtualAllocationInfo& outInfo)
11297	{
11298	m_Metadata->GetAllocationInfo(allocHandle: (VmaAllocHandle)allocation, outInfo);
11299	}
11300
11301	VkResult VmaVirtualBlock_T::Allocate(const VmaVirtualAllocationCreateInfo& createInfo, VmaVirtualAllocation& outAllocation,
11302	VkDeviceSize* outOffset)
11303	{
11304	VmaAllocationRequest request = {};
11305	if (m_Metadata->CreateAllocationRequest(
11306	allocSize: createInfo.size, // allocSize
11307	VMA_MAX(createInfo.alignment, (VkDeviceSize)1), // allocAlignment
11308	upperAddress: (createInfo.flags & VMA_VIRTUAL_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != 0, // upperAddress
11309	allocType: VMA_SUBALLOCATION_TYPE_UNKNOWN, // allocType - unimportant
11310	strategy: createInfo.flags & VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MASK, // strategy
11311	pAllocationRequest: &request))
11312	{
11313	m_Metadata->Alloc(request,
11314	type: VMA_SUBALLOCATION_TYPE_UNKNOWN, // type - unimportant
11315	userData: createInfo.pUserData);
11316	outAllocation = (VmaVirtualAllocation)request.allocHandle;
11317	if(outOffset)
11318	*outOffset = m_Metadata->GetAllocationOffset(allocHandle: request.allocHandle);
11319	return VK_SUCCESS;
11320	}
11321	outAllocation = (VmaVirtualAllocation)VK_NULL_HANDLE;
11322	if (outOffset)
11323	*outOffset = UINT64_MAX;
11324	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
11325	}
11326
11327	void VmaVirtualBlock_T::GetStatistics(VmaStatistics& outStats) const
11328	{
11329	VmaClearStatistics(outStats);
11330	m_Metadata->AddStatistics(inoutStats&: outStats);
11331	}
11332
11333	void VmaVirtualBlock_T::CalculateDetailedStatistics(VmaDetailedStatistics& outStats) const
11334	{
11335	VmaClearDetailedStatistics(outStats);
11336	m_Metadata->AddDetailedStatistics(inoutStats&: outStats);
11337	}
11338
11339	#if VMA_STATS_STRING_ENABLED
11340	void VmaVirtualBlock_T::BuildStatsString(bool detailedMap, VmaStringBuilder& sb) const
11341	{
11342	VmaJsonWriter json(GetAllocationCallbacks(), sb);
11343	json.BeginObject();
11344
11345	VmaDetailedStatistics stats;
11346	CalculateDetailedStatistics(outStats&: stats);
11347
11348	json.WriteString(pStr: "Stats");
11349	VmaPrintDetailedStatistics(json, stat: stats);
11350
11351	if (detailedMap)
11352	{
11353	json.WriteString(pStr: "Details");
11354	json.BeginObject();
11355	m_Metadata->PrintDetailedMap(json);
11356	json.EndObject();
11357	}
11358
11359	json.EndObject();
11360	}
11361	#endif // VMA_STATS_STRING_ENABLED
11362	#endif // _VMA_VIRTUAL_BLOCK_T_FUNCTIONS
11363	#endif // _VMA_VIRTUAL_BLOCK_T
11364
11365
11366	// Main allocator object.
11367	struct VmaAllocator_T
11368	{
11369	VMA_CLASS_NO_COPY(VmaAllocator_T)
11370	public:
11371	bool m_UseMutex;
11372	uint32_t m_VulkanApiVersion;
11373	bool m_UseKhrDedicatedAllocation; // Can be set only if m_VulkanApiVersion < VK_MAKE_VERSION(1, 1, 0).
11374	bool m_UseKhrBindMemory2; // Can be set only if m_VulkanApiVersion < VK_MAKE_VERSION(1, 1, 0).
11375	bool m_UseExtMemoryBudget;
11376	bool m_UseAmdDeviceCoherentMemory;
11377	bool m_UseKhrBufferDeviceAddress;
11378	bool m_UseExtMemoryPriority;
11379	VkDevice m_hDevice;
11380	VkInstance m_hInstance;
11381	bool m_AllocationCallbacksSpecified;
11382	VkAllocationCallbacks m_AllocationCallbacks;
11383	VmaDeviceMemoryCallbacks m_DeviceMemoryCallbacks;
11384	VmaAllocationObjectAllocator m_AllocationObjectAllocator;
11385
11386	// Each bit (1 << i) is set if HeapSizeLimit is enabled for that heap, so cannot allocate more than the heap size.
11387	uint32_t m_HeapSizeLimitMask;
11388
11389	VkPhysicalDeviceProperties m_PhysicalDeviceProperties;
11390	VkPhysicalDeviceMemoryProperties m_MemProps;
11391
11392	// Default pools.
11393	VmaBlockVector* m_pBlockVectors[VK_MAX_MEMORY_TYPES];
11394	VmaDedicatedAllocationList m_DedicatedAllocations[VK_MAX_MEMORY_TYPES];
11395
11396	VmaCurrentBudgetData m_Budget;
11397	VMA_ATOMIC_UINT32 m_DeviceMemoryCount; // Total number of VkDeviceMemory objects.
11398
11399	VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo);
11400	VkResult Init(const VmaAllocatorCreateInfo* pCreateInfo);
11401	~VmaAllocator_T();
11402
11403	const VkAllocationCallbacks* GetAllocationCallbacks() const
11404	{
11405	return m_AllocationCallbacksSpecified ? &m_AllocationCallbacks : VMA_NULL;
11406	}
11407	const VmaVulkanFunctions& GetVulkanFunctions() const
11408	{
11409	return m_VulkanFunctions;
11410	}
11411
11412	VkPhysicalDevice GetPhysicalDevice() const { return m_PhysicalDevice; }
11413
11414	VkDeviceSize GetBufferImageGranularity() const
11415	{
11416	return VMA_MAX(
11417	static_cast<VkDeviceSize>(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY),
11418	m_PhysicalDeviceProperties.limits.bufferImageGranularity);
11419	}
11420
11421	uint32_t GetMemoryHeapCount() const { return m_MemProps.memoryHeapCount; }
11422	uint32_t GetMemoryTypeCount() const { return m_MemProps.memoryTypeCount; }
11423
11424	uint32_t MemoryTypeIndexToHeapIndex(uint32_t memTypeIndex) const
11425	{
11426	VMA_ASSERT(memTypeIndex < m_MemProps.memoryTypeCount);
11427	return m_MemProps.memoryTypes[memTypeIndex].heapIndex;
11428	}
11429	// True when specific memory type is HOST_VISIBLE but not HOST_COHERENT.
11430	bool IsMemoryTypeNonCoherent(uint32_t memTypeIndex) const
11431	{
11432	return (m_MemProps.memoryTypes[memTypeIndex].propertyFlags & (VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)) ==
11433	VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
11434	}
11435	// Minimum alignment for all allocations in specific memory type.
11436	VkDeviceSize GetMemoryTypeMinAlignment(uint32_t memTypeIndex) const
11437	{
11438	return IsMemoryTypeNonCoherent(memTypeIndex) ?
11439	VMA_MAX((VkDeviceSize)VMA_MIN_ALIGNMENT, m_PhysicalDeviceProperties.limits.nonCoherentAtomSize) :
11440	(VkDeviceSize)VMA_MIN_ALIGNMENT;
11441	}
11442
11443	bool IsIntegratedGpu() const
11444	{
11445	return m_PhysicalDeviceProperties.deviceType == VK_PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU;
11446	}
11447
11448	uint32_t GetGlobalMemoryTypeBits() const { return m_GlobalMemoryTypeBits; }
11449
11450	void GetBufferMemoryRequirements(
11451	VkBuffer hBuffer,
11452	VkMemoryRequirements& memReq,
11453	bool& requiresDedicatedAllocation,
11454	bool& prefersDedicatedAllocation) const;
11455	void GetImageMemoryRequirements(
11456	VkImage hImage,
11457	VkMemoryRequirements& memReq,
11458	bool& requiresDedicatedAllocation,
11459	bool& prefersDedicatedAllocation) const;
11460	VkResult FindMemoryTypeIndex(
11461	uint32_t memoryTypeBits,
11462	const VmaAllocationCreateInfo* pAllocationCreateInfo,
11463	VkFlags bufImgUsage, // VkBufferCreateInfo::usage or VkImageCreateInfo::usage. UINT32_MAX if unknown.
11464	uint32_t* pMemoryTypeIndex) const;
11465
11466	// Main allocation function.
11467	VkResult AllocateMemory(
11468	const VkMemoryRequirements& vkMemReq,
11469	bool requiresDedicatedAllocation,
11470	bool prefersDedicatedAllocation,
11471	VkBuffer dedicatedBuffer,
11472	VkImage dedicatedImage,
11473	VkFlags dedicatedBufferImageUsage, // UINT32_MAX if unknown.
11474	const VmaAllocationCreateInfo& createInfo,
11475	VmaSuballocationType suballocType,
11476	size_t allocationCount,
11477	VmaAllocation* pAllocations);
11478
11479	// Main deallocation function.
11480	void FreeMemory(
11481	size_t allocationCount,
11482	const VmaAllocation* pAllocations);
11483
11484	void CalculateStatistics(VmaTotalStatistics* pStats);
11485
11486	void GetHeapBudgets(
11487	VmaBudget* outBudgets, uint32_t firstHeap, uint32_t heapCount);
11488
11489	#if VMA_STATS_STRING_ENABLED
11490	void PrintDetailedMap(class VmaJsonWriter& json);
11491	#endif
11492
11493	void GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo);
11494
11495	VkResult CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool);
11496	void DestroyPool(VmaPool pool);
11497	void GetPoolStatistics(VmaPool pool, VmaStatistics* pPoolStats);
11498	void CalculatePoolStatistics(VmaPool pool, VmaDetailedStatistics* pPoolStats);
11499
11500	void SetCurrentFrameIndex(uint32_t frameIndex);
11501	uint32_t GetCurrentFrameIndex() const { return m_CurrentFrameIndex.load(); }
11502
11503	VkResult CheckPoolCorruption(VmaPool hPool);
11504	VkResult CheckCorruption(uint32_t memoryTypeBits);
11505
11506	// Call to Vulkan function vkAllocateMemory with accompanying bookkeeping.
11507	VkResult AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory);
11508	// Call to Vulkan function vkFreeMemory with accompanying bookkeeping.
11509	void FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory);
11510	// Call to Vulkan function vkBindBufferMemory or vkBindBufferMemory2KHR.
11511	VkResult BindVulkanBuffer(
11512	VkDeviceMemory memory,
11513	VkDeviceSize memoryOffset,
11514	VkBuffer buffer,
11515	const void* pNext);
11516	// Call to Vulkan function vkBindImageMemory or vkBindImageMemory2KHR.
11517	VkResult BindVulkanImage(
11518	VkDeviceMemory memory,
11519	VkDeviceSize memoryOffset,
11520	VkImage image,
11521	const void* pNext);
11522
11523	VkResult Map(VmaAllocation hAllocation, void** ppData);
11524	void Unmap(VmaAllocation hAllocation);
11525
11526	VkResult BindBufferMemory(
11527	VmaAllocation hAllocation,
11528	VkDeviceSize allocationLocalOffset,
11529	VkBuffer hBuffer,
11530	const void* pNext);
11531	VkResult BindImageMemory(
11532	VmaAllocation hAllocation,
11533	VkDeviceSize allocationLocalOffset,
11534	VkImage hImage,
11535	const void* pNext);
11536
11537	VkResult FlushOrInvalidateAllocation(
11538	VmaAllocation hAllocation,
11539	VkDeviceSize offset, VkDeviceSize size,
11540	VMA_CACHE_OPERATION op);
11541	VkResult FlushOrInvalidateAllocations(
11542	uint32_t allocationCount,
11543	const VmaAllocation* allocations,
11544	const VkDeviceSize* offsets, const VkDeviceSize* sizes,
11545	VMA_CACHE_OPERATION op);
11546
11547	void FillAllocation(const VmaAllocation hAllocation, uint8_t pattern);
11548
11549	/*
11550	Returns bit mask of memory types that can support defragmentation on GPU as
11551	they support creation of required buffer for copy operations.
11552	*/
11553	uint32_t GetGpuDefragmentationMemoryTypeBits();
11554
11555	#if VMA_EXTERNAL_MEMORY
11556	VkExternalMemoryHandleTypeFlagsKHR GetExternalMemoryHandleTypeFlags(uint32_t memTypeIndex) const
11557	{
11558	return m_TypeExternalMemoryHandleTypes[memTypeIndex];
11559	}
11560	#endif // #if VMA_EXTERNAL_MEMORY
11561
11562	private:
11563	VkDeviceSize m_PreferredLargeHeapBlockSize;
11564
11565	VkPhysicalDevice m_PhysicalDevice;
11566	VMA_ATOMIC_UINT32 m_CurrentFrameIndex;
11567	VMA_ATOMIC_UINT32 m_GpuDefragmentationMemoryTypeBits; // UINT32_MAX means uninitialized.
11568	#if VMA_EXTERNAL_MEMORY
11569	VkExternalMemoryHandleTypeFlagsKHR m_TypeExternalMemoryHandleTypes[VK_MAX_MEMORY_TYPES];
11570	#endif // #if VMA_EXTERNAL_MEMORY
11571
11572	VMA_RW_MUTEX m_PoolsMutex;
11573	typedef VmaIntrusiveLinkedList<VmaPoolListItemTraits> PoolList;
11574	// Protected by m_PoolsMutex.
11575	PoolList m_Pools;
11576	uint32_t m_NextPoolId;
11577
11578	VmaVulkanFunctions m_VulkanFunctions;
11579
11580	// Global bit mask AND-ed with any memoryTypeBits to disallow certain memory types.
11581	uint32_t m_GlobalMemoryTypeBits;
11582
11583	void ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions);
11584
11585	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
11586	void ImportVulkanFunctions_Static();
11587	#endif
11588
11589	void ImportVulkanFunctions_Custom(const VmaVulkanFunctions* pVulkanFunctions);
11590
11591	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
11592	void ImportVulkanFunctions_Dynamic();
11593	#endif
11594
11595	void ValidateVulkanFunctions();
11596
11597	VkDeviceSize CalcPreferredBlockSize(uint32_t memTypeIndex);
11598
11599	VkResult AllocateMemoryOfType(
11600	VmaPool pool,
11601	VkDeviceSize size,
11602	VkDeviceSize alignment,
11603	bool dedicatedPreferred,
11604	VkBuffer dedicatedBuffer,
11605	VkImage dedicatedImage,
11606	VkFlags dedicatedBufferImageUsage,
11607	const VmaAllocationCreateInfo& createInfo,
11608	uint32_t memTypeIndex,
11609	VmaSuballocationType suballocType,
11610	VmaDedicatedAllocationList& dedicatedAllocations,
11611	VmaBlockVector& blockVector,
11612	size_t allocationCount,
11613	VmaAllocation* pAllocations);
11614
11615	// Helper function only to be used inside AllocateDedicatedMemory.
11616	VkResult AllocateDedicatedMemoryPage(
11617	VmaPool pool,
11618	VkDeviceSize size,
11619	VmaSuballocationType suballocType,
11620	uint32_t memTypeIndex,
11621	const VkMemoryAllocateInfo& allocInfo,
11622	bool map,
11623	bool isUserDataString,
11624	bool isMappingAllowed,
11625	void* pUserData,
11626	VmaAllocation* pAllocation);
11627
11628	// Allocates and registers new VkDeviceMemory specifically for dedicated allocations.
11629	VkResult AllocateDedicatedMemory(
11630	VmaPool pool,
11631	VkDeviceSize size,
11632	VmaSuballocationType suballocType,
11633	VmaDedicatedAllocationList& dedicatedAllocations,
11634	uint32_t memTypeIndex,
11635	bool map,
11636	bool isUserDataString,
11637	bool isMappingAllowed,
11638	bool canAliasMemory,
11639	void* pUserData,
11640	float priority,
11641	VkBuffer dedicatedBuffer,
11642	VkImage dedicatedImage,
11643	VkFlags dedicatedBufferImageUsage,
11644	size_t allocationCount,
11645	VmaAllocation* pAllocations,
11646	const void* pNextChain = nullptr);
11647
11648	void FreeDedicatedMemory(const VmaAllocation allocation);
11649
11650	VkResult CalcMemTypeParams(
11651	VmaAllocationCreateInfo& outCreateInfo,
11652	uint32_t memTypeIndex,
11653	VkDeviceSize size,
11654	size_t allocationCount);
11655	VkResult CalcAllocationParams(
11656	VmaAllocationCreateInfo& outCreateInfo,
11657	bool dedicatedRequired,
11658	bool dedicatedPreferred);
11659
11660	/*
11661	Calculates and returns bit mask of memory types that can support defragmentation
11662	on GPU as they support creation of required buffer for copy operations.
11663	*/
11664	uint32_t CalculateGpuDefragmentationMemoryTypeBits() const;
11665	uint32_t CalculateGlobalMemoryTypeBits() const;
11666
11667	bool GetFlushOrInvalidateRange(
11668	VmaAllocation allocation,
11669	VkDeviceSize offset, VkDeviceSize size,
11670	VkMappedMemoryRange& outRange) const;
11671
11672	#if VMA_MEMORY_BUDGET
11673	void UpdateVulkanBudget();
11674	#endif // #if VMA_MEMORY_BUDGET
11675	};
11676
11677
11678	#ifndef _VMA_MEMORY_FUNCTIONS
11679	static void* VmaMalloc(VmaAllocator hAllocator, size_t size, size_t alignment)
11680	{
11681	return VmaMalloc(pAllocationCallbacks: &hAllocator->m_AllocationCallbacks, size, alignment);
11682	}
11683
11684	static void VmaFree(VmaAllocator hAllocator, void* ptr)
11685	{
11686	VmaFree(pAllocationCallbacks: &hAllocator->m_AllocationCallbacks, ptr);
11687	}
11688
11689	template<typename T>
11690	static T* VmaAllocate(VmaAllocator hAllocator)
11691	{
11692	return (T*)VmaMalloc(hAllocator, size: sizeof(T), VMA_ALIGN_OF(T));
11693	}
11694
11695	template<typename T>
11696	static T* VmaAllocateArray(VmaAllocator hAllocator, size_t count)
11697	{
11698	return (T*)VmaMalloc(hAllocator, size: sizeof(T) * count, VMA_ALIGN_OF(T));
11699	}
11700
11701	template<typename T>
11702	static void vma_delete(VmaAllocator hAllocator, T* ptr)
11703	{
11704	if(ptr != VMA_NULL)
11705	{
11706	ptr->~T();
11707	VmaFree(hAllocator, ptr);
11708	}
11709	}
11710
11711	template<typename T>
11712	static void vma_delete_array(VmaAllocator hAllocator, T* ptr, size_t count)
11713	{
11714	if(ptr != VMA_NULL)
11715	{
11716	for(size_t i = count; i--; )
11717	ptr[i].~T();
11718	VmaFree(hAllocator, ptr);
11719	}
11720	}
11721	#endif // _VMA_MEMORY_FUNCTIONS
11722
11723	#ifndef _VMA_DEVICE_MEMORY_BLOCK_FUNCTIONS
11724	VmaDeviceMemoryBlock::VmaDeviceMemoryBlock(VmaAllocator hAllocator)
11725	: m_pMetadata(VMA_NULL),
11726	m_MemoryTypeIndex(UINT32_MAX),
11727	m_Id(0),
11728	m_hMemory(VK_NULL_HANDLE),
11729	m_MapCount(0),
11730	m_pMappedData(VMA_NULL) {}
11731
11732	VmaDeviceMemoryBlock::~VmaDeviceMemoryBlock()
11733	{
11734	VMA_ASSERT(m_MapCount == 0 && "VkDeviceMemory block is being destroyed while it is still mapped.");
11735	VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
11736	}
11737
11738	void VmaDeviceMemoryBlock::Init(
11739	VmaAllocator hAllocator,
11740	VmaPool hParentPool,
11741	uint32_t newMemoryTypeIndex,
11742	VkDeviceMemory newMemory,
11743	VkDeviceSize newSize,
11744	uint32_t id,
11745	uint32_t algorithm,
11746	VkDeviceSize bufferImageGranularity)
11747	{
11748	VMA_ASSERT(m_hMemory == VK_NULL_HANDLE);
11749
11750	m_hParentPool = hParentPool;
11751	m_MemoryTypeIndex = newMemoryTypeIndex;
11752	m_Id = id;
11753	m_hMemory = newMemory;
11754
11755	switch (algorithm)
11756	{
11757	case VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT:
11758	m_pMetadata = vma_new(hAllocator, VmaBlockMetadata_Linear)(hAllocator->GetAllocationCallbacks(),
11759	bufferImageGranularity, false); // isVirtual
11760	break;
11761	default:
11762	VMA_ASSERT(0);
11763	// Fall-through.
11764	case 0:
11765	m_pMetadata = vma_new(hAllocator, VmaBlockMetadata_TLSF)(hAllocator->GetAllocationCallbacks(),
11766	bufferImageGranularity, false); // isVirtual
11767	}
11768	m_pMetadata->Init(size: newSize);
11769	}
11770
11771	void VmaDeviceMemoryBlock::Destroy(VmaAllocator allocator)
11772	{
11773	// Define macro VMA_DEBUG_LOG to receive the list of the unfreed allocations
11774	if (!m_pMetadata->IsEmpty())
11775	m_pMetadata->DebugLogAllAllocations();
11776	// This is the most important assert in the entire library.
11777	// Hitting it means you have some memory leak - unreleased VmaAllocation objects.
11778	VMA_ASSERT(m_pMetadata->IsEmpty() && "Some allocations were not freed before destruction of this memory block!");
11779
11780	VMA_ASSERT(m_hMemory != VK_NULL_HANDLE);
11781	allocator->FreeVulkanMemory(memoryType: m_MemoryTypeIndex, size: m_pMetadata->GetSize(), hMemory: m_hMemory);
11782	m_hMemory = VK_NULL_HANDLE;
11783
11784	vma_delete(hAllocator: allocator, ptr: m_pMetadata);
11785	m_pMetadata = VMA_NULL;
11786	}
11787
11788	void VmaDeviceMemoryBlock::PostFree(VmaAllocator hAllocator)
11789	{
11790	if(m_MappingHysteresis.PostFree())
11791	{
11792	VMA_ASSERT(m_MappingHysteresis.GetExtraMapping() == 0);
11793	if (m_MapCount == 0)
11794	{
11795	m_pMappedData = VMA_NULL;
11796	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
11797	}
11798	}
11799	}
11800
11801	bool VmaDeviceMemoryBlock::Validate() const
11802	{
11803	VMA_VALIDATE((m_hMemory != VK_NULL_HANDLE) &&
11804	(m_pMetadata->GetSize() != 0));
11805
11806	return m_pMetadata->Validate();
11807	}
11808
11809	VkResult VmaDeviceMemoryBlock::CheckCorruption(VmaAllocator hAllocator)
11810	{
11811	void* pData = nullptr;
11812	VkResult res = Map(hAllocator, count: 1, ppData: &pData);
11813	if (res != VK_SUCCESS)
11814	{
11815	return res;
11816	}
11817
11818	res = m_pMetadata->CheckCorruption(pBlockData: pData);
11819
11820	Unmap(hAllocator, count: 1);
11821
11822	return res;
11823	}
11824
11825	VkResult VmaDeviceMemoryBlock::Map(VmaAllocator hAllocator, uint32_t count, void** ppData)
11826	{
11827	if (count == 0)
11828	{
11829	return VK_SUCCESS;
11830	}
11831
11832	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11833	const uint32_t oldTotalMapCount = m_MapCount + m_MappingHysteresis.GetExtraMapping();
11834	m_MappingHysteresis.PostMap();
11835	if (oldTotalMapCount != 0)
11836	{
11837	m_MapCount += count;
11838	VMA_ASSERT(m_pMappedData != VMA_NULL);
11839	if (ppData != VMA_NULL)
11840	{
11841	*ppData = m_pMappedData;
11842	}
11843	return VK_SUCCESS;
11844	}
11845	else
11846	{
11847	VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
11848	hAllocator->m_hDevice,
11849	m_hMemory,
11850	0, // offset
11851	VK_WHOLE_SIZE,
11852	0, // flags
11853	&m_pMappedData);
11854	if (result == VK_SUCCESS)
11855	{
11856	if (ppData != VMA_NULL)
11857	{
11858	*ppData = m_pMappedData;
11859	}
11860	m_MapCount = count;
11861	}
11862	return result;
11863	}
11864	}
11865
11866	void VmaDeviceMemoryBlock::Unmap(VmaAllocator hAllocator, uint32_t count)
11867	{
11868	if (count == 0)
11869	{
11870	return;
11871	}
11872
11873	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11874	if (m_MapCount >= count)
11875	{
11876	m_MapCount -= count;
11877	const uint32_t totalMapCount = m_MapCount + m_MappingHysteresis.GetExtraMapping();
11878	if (totalMapCount == 0)
11879	{
11880	m_pMappedData = VMA_NULL;
11881	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(hAllocator->m_hDevice, m_hMemory);
11882	}
11883	m_MappingHysteresis.PostUnmap();
11884	}
11885	else
11886	{
11887	VMA_ASSERT(0 && "VkDeviceMemory block is being unmapped while it was not previously mapped.");
11888	}
11889	}
11890
11891	VkResult VmaDeviceMemoryBlock::WriteMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize)
11892	{
11893	VMA_ASSERT(VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_MARGIN % 4 == 0 && VMA_DEBUG_DETECT_CORRUPTION);
11894
11895	void* pData;
11896	VkResult res = Map(hAllocator, count: 1, ppData: &pData);
11897	if (res != VK_SUCCESS)
11898	{
11899	return res;
11900	}
11901
11902	VmaWriteMagicValue(pData, offset: allocOffset + allocSize);
11903
11904	Unmap(hAllocator, count: 1);
11905	return VK_SUCCESS;
11906	}
11907
11908	VkResult VmaDeviceMemoryBlock::ValidateMagicValueAfterAllocation(VmaAllocator hAllocator, VkDeviceSize allocOffset, VkDeviceSize allocSize)
11909	{
11910	VMA_ASSERT(VMA_DEBUG_MARGIN > 0 && VMA_DEBUG_MARGIN % 4 == 0 && VMA_DEBUG_DETECT_CORRUPTION);
11911
11912	void* pData;
11913	VkResult res = Map(hAllocator, count: 1, ppData: &pData);
11914	if (res != VK_SUCCESS)
11915	{
11916	return res;
11917	}
11918
11919	if (!VmaValidateMagicValue(pData, offset: allocOffset + allocSize))
11920	{
11921	VMA_ASSERT(0 && "MEMORY CORRUPTION DETECTED AFTER FREED ALLOCATION!");
11922	}
11923
11924	Unmap(hAllocator, count: 1);
11925	return VK_SUCCESS;
11926	}
11927
11928	VkResult VmaDeviceMemoryBlock::BindBufferMemory(
11929	const VmaAllocator hAllocator,
11930	const VmaAllocation hAllocation,
11931	VkDeviceSize allocationLocalOffset,
11932	VkBuffer hBuffer,
11933	const void* pNext)
11934	{
11935	VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
11936	hAllocation->GetBlock() == this);
11937	VMA_ASSERT(allocationLocalOffset < hAllocation->GetSize() &&
11938	"Invalid allocationLocalOffset. Did you forget that this offset is relative to the beginning of the allocation, not the whole memory block?");
11939	const VkDeviceSize memoryOffset = hAllocation->GetOffset() + allocationLocalOffset;
11940	// This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
11941	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11942	return hAllocator->BindVulkanBuffer(memory: m_hMemory, memoryOffset, buffer: hBuffer, pNext);
11943	}
11944
11945	VkResult VmaDeviceMemoryBlock::BindImageMemory(
11946	const VmaAllocator hAllocator,
11947	const VmaAllocation hAllocation,
11948	VkDeviceSize allocationLocalOffset,
11949	VkImage hImage,
11950	const void* pNext)
11951	{
11952	VMA_ASSERT(hAllocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_BLOCK &&
11953	hAllocation->GetBlock() == this);
11954	VMA_ASSERT(allocationLocalOffset < hAllocation->GetSize() &&
11955	"Invalid allocationLocalOffset. Did you forget that this offset is relative to the beginning of the allocation, not the whole memory block?");
11956	const VkDeviceSize memoryOffset = hAllocation->GetOffset() + allocationLocalOffset;
11957	// This lock is important so that we don't call vkBind... and/or vkMap... simultaneously on the same VkDeviceMemory from multiple threads.
11958	VmaMutexLock lock(m_MapAndBindMutex, hAllocator->m_UseMutex);
11959	return hAllocator->BindVulkanImage(memory: m_hMemory, memoryOffset, image: hImage, pNext);
11960	}
11961	#endif // _VMA_DEVICE_MEMORY_BLOCK_FUNCTIONS
11962
11963	#ifndef _VMA_ALLOCATION_T_FUNCTIONS
11964	VmaAllocation_T::VmaAllocation_T(bool mappingAllowed)
11965	: m_Alignment{ 1 },
11966	m_Size{ 0 },
11967	m_pUserData{ VMA_NULL },
11968	m_pName{ VMA_NULL },
11969	m_MemoryTypeIndex{ 0 },
11970	m_Type{ (uint8_t)ALLOCATION_TYPE_NONE },
11971	m_SuballocationType{ (uint8_t)VMA_SUBALLOCATION_TYPE_UNKNOWN },
11972	m_MapCount{ 0 },
11973	m_Flags{ 0 }
11974	{
11975	if(mappingAllowed)
11976	m_Flags |= (uint8_t)FLAG_MAPPING_ALLOWED;
11977
11978	#if VMA_STATS_STRING_ENABLED
11979	m_BufferImageUsage = 0;
11980	#endif
11981	}
11982
11983	VmaAllocation_T::~VmaAllocation_T()
11984	{
11985	VMA_ASSERT(m_MapCount == 0 && "Allocation was not unmapped before destruction.");
11986
11987	// Check if owned string was freed.
11988	VMA_ASSERT(m_pName == VMA_NULL);
11989	}
11990
11991	void VmaAllocation_T::InitBlockAllocation(
11992	VmaDeviceMemoryBlock* block,
11993	VmaAllocHandle allocHandle,
11994	VkDeviceSize alignment,
11995	VkDeviceSize size,
11996	uint32_t memoryTypeIndex,
11997	VmaSuballocationType suballocationType,
11998	bool mapped)
11999	{
12000	VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
12001	VMA_ASSERT(block != VMA_NULL);
12002	m_Type = (uint8_t)ALLOCATION_TYPE_BLOCK;
12003	m_Alignment = alignment;
12004	m_Size = size;
12005	m_MemoryTypeIndex = memoryTypeIndex;
12006	if(mapped)
12007	{
12008	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12009	m_Flags |= (uint8_t)FLAG_PERSISTENT_MAP;
12010	}
12011	m_SuballocationType = (uint8_t)suballocationType;
12012	m_BlockAllocation.m_Block = block;
12013	m_BlockAllocation.m_AllocHandle = allocHandle;
12014	}
12015
12016	void VmaAllocation_T::InitDedicatedAllocation(
12017	VmaPool hParentPool,
12018	uint32_t memoryTypeIndex,
12019	VkDeviceMemory hMemory,
12020	VmaSuballocationType suballocationType,
12021	void* pMappedData,
12022	VkDeviceSize size)
12023	{
12024	VMA_ASSERT(m_Type == ALLOCATION_TYPE_NONE);
12025	VMA_ASSERT(hMemory != VK_NULL_HANDLE);
12026	m_Type = (uint8_t)ALLOCATION_TYPE_DEDICATED;
12027	m_Alignment = 0;
12028	m_Size = size;
12029	m_MemoryTypeIndex = memoryTypeIndex;
12030	m_SuballocationType = (uint8_t)suballocationType;
12031	if(pMappedData != VMA_NULL)
12032	{
12033	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12034	m_Flags |= (uint8_t)FLAG_PERSISTENT_MAP;
12035	}
12036	m_DedicatedAllocation.m_hParentPool = hParentPool;
12037	m_DedicatedAllocation.m_hMemory = hMemory;
12038	m_DedicatedAllocation.m_pMappedData = pMappedData;
12039	m_DedicatedAllocation.m_Prev = VMA_NULL;
12040	m_DedicatedAllocation.m_Next = VMA_NULL;
12041	}
12042
12043	void VmaAllocation_T::SetName(VmaAllocator hAllocator, const char* pName)
12044	{
12045	VMA_ASSERT(pName == VMA_NULL || pName != m_pName);
12046
12047	FreeName(hAllocator);
12048
12049	if (pName != VMA_NULL)
12050	m_pName = VmaCreateStringCopy(allocs: hAllocator->GetAllocationCallbacks(), srcStr: pName);
12051	}
12052
12053	uint8_t VmaAllocation_T::SwapBlockAllocation(VmaAllocator hAllocator, VmaAllocation allocation)
12054	{
12055	VMA_ASSERT(allocation != VMA_NULL);
12056	VMA_ASSERT(m_Type == ALLOCATION_TYPE_BLOCK);
12057	VMA_ASSERT(allocation->m_Type == ALLOCATION_TYPE_BLOCK);
12058
12059	if (m_MapCount != 0)
12060	m_BlockAllocation.m_Block->Unmap(hAllocator, count: m_MapCount);
12061
12062	m_BlockAllocation.m_Block->m_pMetadata->SetAllocationUserData(allocHandle: m_BlockAllocation.m_AllocHandle, userData: allocation);
12063	VMA_SWAP(m_BlockAllocation, allocation->m_BlockAllocation);
12064	m_BlockAllocation.m_Block->m_pMetadata->SetAllocationUserData(allocHandle: m_BlockAllocation.m_AllocHandle, userData: this);
12065
12066	#if VMA_STATS_STRING_ENABLED
12067	VMA_SWAP(m_BufferImageUsage, allocation->m_BufferImageUsage);
12068	#endif
12069	return m_MapCount;
12070	}
12071
12072	VmaAllocHandle VmaAllocation_T::GetAllocHandle() const
12073	{
12074	switch (m_Type)
12075	{
12076	case ALLOCATION_TYPE_BLOCK:
12077	return m_BlockAllocation.m_AllocHandle;
12078	case ALLOCATION_TYPE_DEDICATED:
12079	return VK_NULL_HANDLE;
12080	default:
12081	VMA_ASSERT(0);
12082	return VK_NULL_HANDLE;
12083	}
12084	}
12085
12086	VkDeviceSize VmaAllocation_T::GetOffset() const
12087	{
12088	switch (m_Type)
12089	{
12090	case ALLOCATION_TYPE_BLOCK:
12091	return m_BlockAllocation.m_Block->m_pMetadata->GetAllocationOffset(allocHandle: m_BlockAllocation.m_AllocHandle);
12092	case ALLOCATION_TYPE_DEDICATED:
12093	return 0;
12094	default:
12095	VMA_ASSERT(0);
12096	return 0;
12097	}
12098	}
12099
12100	VmaPool VmaAllocation_T::GetParentPool() const
12101	{
12102	switch (m_Type)
12103	{
12104	case ALLOCATION_TYPE_BLOCK:
12105	return m_BlockAllocation.m_Block->GetParentPool();
12106	case ALLOCATION_TYPE_DEDICATED:
12107	return m_DedicatedAllocation.m_hParentPool;
12108	default:
12109	VMA_ASSERT(0);
12110	return VK_NULL_HANDLE;
12111	}
12112	}
12113
12114	VkDeviceMemory VmaAllocation_T::GetMemory() const
12115	{
12116	switch (m_Type)
12117	{
12118	case ALLOCATION_TYPE_BLOCK:
12119	return m_BlockAllocation.m_Block->GetDeviceMemory();
12120	case ALLOCATION_TYPE_DEDICATED:
12121	return m_DedicatedAllocation.m_hMemory;
12122	default:
12123	VMA_ASSERT(0);
12124	return VK_NULL_HANDLE;
12125	}
12126	}
12127
12128	void* VmaAllocation_T::GetMappedData() const
12129	{
12130	switch (m_Type)
12131	{
12132	case ALLOCATION_TYPE_BLOCK:
12133	if (m_MapCount != 0 || IsPersistentMap())
12134	{
12135	void* pBlockData = m_BlockAllocation.m_Block->GetMappedData();
12136	VMA_ASSERT(pBlockData != VMA_NULL);
12137	return (char*)pBlockData + GetOffset();
12138	}
12139	else
12140	{
12141	return VMA_NULL;
12142	}
12143	break;
12144	case ALLOCATION_TYPE_DEDICATED:
12145	VMA_ASSERT((m_DedicatedAllocation.m_pMappedData != VMA_NULL) == (m_MapCount != 0 || IsPersistentMap()));
12146	return m_DedicatedAllocation.m_pMappedData;
12147	default:
12148	VMA_ASSERT(0);
12149	return VMA_NULL;
12150	}
12151	}
12152
12153	void VmaAllocation_T::BlockAllocMap()
12154	{
12155	VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
12156	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12157
12158	if (m_MapCount < 0xFF)
12159	{
12160	++m_MapCount;
12161	}
12162	else
12163	{
12164	VMA_ASSERT(0 && "Allocation mapped too many times simultaneously.");
12165	}
12166	}
12167
12168	void VmaAllocation_T::BlockAllocUnmap()
12169	{
12170	VMA_ASSERT(GetType() == ALLOCATION_TYPE_BLOCK);
12171
12172	if (m_MapCount > 0)
12173	{
12174	--m_MapCount;
12175	}
12176	else
12177	{
12178	VMA_ASSERT(0 && "Unmapping allocation not previously mapped.");
12179	}
12180	}
12181
12182	VkResult VmaAllocation_T::DedicatedAllocMap(VmaAllocator hAllocator, void** ppData)
12183	{
12184	VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
12185	VMA_ASSERT(IsMappingAllowed() && "Mapping is not allowed on this allocation! Please use one of the new VMA_ALLOCATION_CREATE_HOST_ACCESS_* flags when creating it.");
12186
12187	if (m_MapCount != 0 || IsPersistentMap())
12188	{
12189	if (m_MapCount < 0xFF)
12190	{
12191	VMA_ASSERT(m_DedicatedAllocation.m_pMappedData != VMA_NULL);
12192	*ppData = m_DedicatedAllocation.m_pMappedData;
12193	++m_MapCount;
12194	return VK_SUCCESS;
12195	}
12196	else
12197	{
12198	VMA_ASSERT(0 && "Dedicated allocation mapped too many times simultaneously.");
12199	return VK_ERROR_MEMORY_MAP_FAILED;
12200	}
12201	}
12202	else
12203	{
12204	VkResult result = (*hAllocator->GetVulkanFunctions().vkMapMemory)(
12205	hAllocator->m_hDevice,
12206	m_DedicatedAllocation.m_hMemory,
12207	0, // offset
12208	VK_WHOLE_SIZE,
12209	0, // flags
12210	ppData);
12211	if (result == VK_SUCCESS)
12212	{
12213	m_DedicatedAllocation.m_pMappedData = *ppData;
12214	m_MapCount = 1;
12215	}
12216	return result;
12217	}
12218	}
12219
12220	void VmaAllocation_T::DedicatedAllocUnmap(VmaAllocator hAllocator)
12221	{
12222	VMA_ASSERT(GetType() == ALLOCATION_TYPE_DEDICATED);
12223
12224	if (m_MapCount > 0)
12225	{
12226	--m_MapCount;
12227	if (m_MapCount == 0 && !IsPersistentMap())
12228	{
12229	m_DedicatedAllocation.m_pMappedData = VMA_NULL;
12230	(*hAllocator->GetVulkanFunctions().vkUnmapMemory)(
12231	hAllocator->m_hDevice,
12232	m_DedicatedAllocation.m_hMemory);
12233	}
12234	}
12235	else
12236	{
12237	VMA_ASSERT(0 && "Unmapping dedicated allocation not previously mapped.");
12238	}
12239	}
12240
12241	#if VMA_STATS_STRING_ENABLED
12242	void VmaAllocation_T::InitBufferImageUsage(uint32_t bufferImageUsage)
12243	{
12244	VMA_ASSERT(m_BufferImageUsage == 0);
12245	m_BufferImageUsage = bufferImageUsage;
12246	}
12247
12248	void VmaAllocation_T::PrintParameters(class VmaJsonWriter& json) const
12249	{
12250	json.WriteString(pStr: "Type");
12251	json.WriteString(pStr: VMA_SUBALLOCATION_TYPE_NAMES[m_SuballocationType]);
12252
12253	json.WriteString(pStr: "Size");
12254	json.WriteNumber(n: m_Size);
12255	json.WriteString(pStr: "Usage");
12256	json.WriteNumber(n: m_BufferImageUsage);
12257
12258	if (m_pUserData != VMA_NULL)
12259	{
12260	json.WriteString(pStr: "CustomData");
12261	json.BeginString();
12262	json.ContinueString_Pointer(ptr: m_pUserData);
12263	json.EndString();
12264	}
12265	if (m_pName != VMA_NULL)
12266	{
12267	json.WriteString(pStr: "Name");
12268	json.WriteString(pStr: m_pName);
12269	}
12270	}
12271	#endif // VMA_STATS_STRING_ENABLED
12272
12273	void VmaAllocation_T::FreeName(VmaAllocator hAllocator)
12274	{
12275	if(m_pName)
12276	{
12277	VmaFreeString(allocs: hAllocator->GetAllocationCallbacks(), str: m_pName);
12278	m_pName = VMA_NULL;
12279	}
12280	}
12281	#endif // _VMA_ALLOCATION_T_FUNCTIONS
12282
12283	#ifndef _VMA_BLOCK_VECTOR_FUNCTIONS
12284	VmaBlockVector::VmaBlockVector(
12285	VmaAllocator hAllocator,
12286	VmaPool hParentPool,
12287	uint32_t memoryTypeIndex,
12288	VkDeviceSize preferredBlockSize,
12289	size_t minBlockCount,
12290	size_t maxBlockCount,
12291	VkDeviceSize bufferImageGranularity,
12292	bool explicitBlockSize,
12293	uint32_t algorithm,
12294	float priority,
12295	VkDeviceSize minAllocationAlignment,
12296	void* pMemoryAllocateNext)
12297	: m_hAllocator(hAllocator),
12298	m_hParentPool(hParentPool),
12299	m_MemoryTypeIndex(memoryTypeIndex),
12300	m_PreferredBlockSize(preferredBlockSize),
12301	m_MinBlockCount(minBlockCount),
12302	m_MaxBlockCount(maxBlockCount),
12303	m_BufferImageGranularity(bufferImageGranularity),
12304	m_ExplicitBlockSize(explicitBlockSize),
12305	m_Algorithm(algorithm),
12306	m_Priority(priority),
12307	m_MinAllocationAlignment(minAllocationAlignment),
12308	m_pMemoryAllocateNext(pMemoryAllocateNext),
12309	m_Blocks(VmaStlAllocator<VmaDeviceMemoryBlock*>(hAllocator->GetAllocationCallbacks())),
12310	m_NextBlockId(0) {}
12311
12312	VmaBlockVector::~VmaBlockVector()
12313	{
12314	for (size_t i = m_Blocks.size(); i--; )
12315	{
12316	m_Blocks[i]->Destroy(allocator: m_hAllocator);
12317	vma_delete(hAllocator: m_hAllocator, ptr: m_Blocks[i]);
12318	}
12319	}
12320
12321	VkResult VmaBlockVector::CreateMinBlocks()
12322	{
12323	for (size_t i = 0; i < m_MinBlockCount; ++i)
12324	{
12325	VkResult res = CreateBlock(blockSize: m_PreferredBlockSize, VMA_NULL);
12326	if (res != VK_SUCCESS)
12327	{
12328	return res;
12329	}
12330	}
12331	return VK_SUCCESS;
12332	}
12333
12334	void VmaBlockVector::AddStatistics(VmaStatistics& inoutStats)
12335	{
12336	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12337
12338	const size_t blockCount = m_Blocks.size();
12339	for (uint32_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
12340	{
12341	const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12342	VMA_ASSERT(pBlock);
12343	VMA_HEAVY_ASSERT(pBlock->Validate());
12344	pBlock->m_pMetadata->AddStatistics(inoutStats);
12345	}
12346	}
12347
12348	void VmaBlockVector::AddDetailedStatistics(VmaDetailedStatistics& inoutStats)
12349	{
12350	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12351
12352	const size_t blockCount = m_Blocks.size();
12353	for (uint32_t blockIndex = 0; blockIndex < blockCount; ++blockIndex)
12354	{
12355	const VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12356	VMA_ASSERT(pBlock);
12357	VMA_HEAVY_ASSERT(pBlock->Validate());
12358	pBlock->m_pMetadata->AddDetailedStatistics(inoutStats);
12359	}
12360	}
12361
12362	bool VmaBlockVector::IsEmpty()
12363	{
12364	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12365	return m_Blocks.empty();
12366	}
12367
12368	bool VmaBlockVector::IsCorruptionDetectionEnabled() const
12369	{
12370	const uint32_t requiredMemFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
12371	return (VMA_DEBUG_DETECT_CORRUPTION != 0) &&
12372	(VMA_DEBUG_MARGIN > 0) &&
12373	(m_Algorithm == 0 || m_Algorithm == VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT) &&
12374	(m_hAllocator->m_MemProps.memoryTypes[m_MemoryTypeIndex].propertyFlags & requiredMemFlags) == requiredMemFlags;
12375	}
12376
12377	VkResult VmaBlockVector::Allocate(
12378	VkDeviceSize size,
12379	VkDeviceSize alignment,
12380	const VmaAllocationCreateInfo& createInfo,
12381	VmaSuballocationType suballocType,
12382	size_t allocationCount,
12383	VmaAllocation* pAllocations)
12384	{
12385	size_t allocIndex;
12386	VkResult res = VK_SUCCESS;
12387
12388	alignment = VMA_MAX(alignment, m_MinAllocationAlignment);
12389
12390	if (IsCorruptionDetectionEnabled())
12391	{
12392	size = VmaAlignUp<VkDeviceSize>(val: size, alignment: sizeof(VMA_CORRUPTION_DETECTION_MAGIC_VALUE));
12393	alignment = VmaAlignUp<VkDeviceSize>(val: alignment, alignment: sizeof(VMA_CORRUPTION_DETECTION_MAGIC_VALUE));
12394	}
12395
12396	{
12397	VmaMutexLockWrite lock(m_Mutex, m_hAllocator->m_UseMutex);
12398	for (allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
12399	{
12400	res = AllocatePage(
12401	size,
12402	alignment,
12403	createInfo,
12404	suballocType,
12405	pAllocation: pAllocations + allocIndex);
12406	if (res != VK_SUCCESS)
12407	{
12408	break;
12409	}
12410	}
12411	}
12412
12413	if (res != VK_SUCCESS)
12414	{
12415	// Free all already created allocations.
12416	while (allocIndex--)
12417	Free(hAllocation: pAllocations[allocIndex]);
12418	memset(s: pAllocations, c: 0, n: sizeof(VmaAllocation) * allocationCount);
12419	}
12420
12421	return res;
12422	}
12423
12424	VkResult VmaBlockVector::AllocatePage(
12425	VkDeviceSize size,
12426	VkDeviceSize alignment,
12427	const VmaAllocationCreateInfo& createInfo,
12428	VmaSuballocationType suballocType,
12429	VmaAllocation* pAllocation)
12430	{
12431	const bool isUpperAddress = (createInfo.flags & VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != 0;
12432
12433	VkDeviceSize freeMemory;
12434	{
12435	const uint32_t heapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex: m_MemoryTypeIndex);
12436	VmaBudget heapBudget = {};
12437	m_hAllocator->GetHeapBudgets(outBudgets: &heapBudget, firstHeap: heapIndex, heapCount: 1);
12438	freeMemory = (heapBudget.usage < heapBudget.budget) ? (heapBudget.budget - heapBudget.usage) : 0;
12439	}
12440
12441	const bool canFallbackToDedicated = !HasExplicitBlockSize() &&
12442	(createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0;
12443	const bool canCreateNewBlock =
12444	((createInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0) &&
12445	(m_Blocks.size() < m_MaxBlockCount) &&
12446	(freeMemory >= size || !canFallbackToDedicated);
12447	uint32_t strategy = createInfo.flags & VMA_ALLOCATION_CREATE_STRATEGY_MASK;
12448
12449	// Upper address can only be used with linear allocator and within single memory block.
12450	if (isUpperAddress &&
12451	(m_Algorithm != VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT || m_MaxBlockCount > 1))
12452	{
12453	return VK_ERROR_FEATURE_NOT_PRESENT;
12454	}
12455
12456	// Early reject: requested allocation size is larger that maximum block size for this block vector.
12457	if (size + VMA_DEBUG_MARGIN > m_PreferredBlockSize)
12458	{
12459	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12460	}
12461
12462	// 1. Search existing allocations. Try to allocate.
12463	if (m_Algorithm == VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
12464	{
12465	// Use only last block.
12466	if (!m_Blocks.empty())
12467	{
12468	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks.back();
12469	VMA_ASSERT(pCurrBlock);
12470	VkResult res = AllocateFromBlock(
12471	pBlock: pCurrBlock, size, alignment, allocFlags: createInfo.flags, pUserData: createInfo.pUserData, suballocType, strategy, pAllocation);
12472	if (res == VK_SUCCESS)
12473	{
12474	VMA_DEBUG_LOG(" Returned from last block #%u", pCurrBlock->GetId());
12475	IncrementallySortBlocks();
12476	return VK_SUCCESS;
12477	}
12478	}
12479	}
12480	else
12481	{
12482	if (strategy != VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT) // MIN_MEMORY or default
12483	{
12484	const bool isHostVisible =
12485	(m_hAllocator->m_MemProps.memoryTypes[m_MemoryTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0;
12486	if(isHostVisible)
12487	{
12488	const bool isMappingAllowed = (createInfo.flags &
12489	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0;
12490	/*
12491	For non-mappable allocations, check blocks that are not mapped first.
12492	For mappable allocations, check blocks that are already mapped first.
12493	This way, having many blocks, we will separate mappable and non-mappable allocations,
12494	hopefully limiting the number of blocks that are mapped, which will help tools like RenderDoc.
12495	*/
12496	for(size_t mappingI = 0; mappingI < 2; ++mappingI)
12497	{
12498	// Forward order in m_Blocks - prefer blocks with smallest amount of free space.
12499	for (size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
12500	{
12501	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12502	VMA_ASSERT(pCurrBlock);
12503	const bool isBlockMapped = pCurrBlock->GetMappedData() != VMA_NULL;
12504	if((mappingI == 0) == (isMappingAllowed == isBlockMapped))
12505	{
12506	VkResult res = AllocateFromBlock(
12507	pBlock: pCurrBlock, size, alignment, allocFlags: createInfo.flags, pUserData: createInfo.pUserData, suballocType, strategy, pAllocation);
12508	if (res == VK_SUCCESS)
12509	{
12510	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12511	IncrementallySortBlocks();
12512	return VK_SUCCESS;
12513	}
12514	}
12515	}
12516	}
12517	}
12518	else
12519	{
12520	// Forward order in m_Blocks - prefer blocks with smallest amount of free space.
12521	for (size_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
12522	{
12523	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12524	VMA_ASSERT(pCurrBlock);
12525	VkResult res = AllocateFromBlock(
12526	pBlock: pCurrBlock, size, alignment, allocFlags: createInfo.flags, pUserData: createInfo.pUserData, suballocType, strategy, pAllocation);
12527	if (res == VK_SUCCESS)
12528	{
12529	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12530	IncrementallySortBlocks();
12531	return VK_SUCCESS;
12532	}
12533	}
12534	}
12535	}
12536	else // VMA_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT
12537	{
12538	// Backward order in m_Blocks - prefer blocks with largest amount of free space.
12539	for (size_t blockIndex = m_Blocks.size(); blockIndex--; )
12540	{
12541	VmaDeviceMemoryBlock* const pCurrBlock = m_Blocks[blockIndex];
12542	VMA_ASSERT(pCurrBlock);
12543	VkResult res = AllocateFromBlock(pBlock: pCurrBlock, size, alignment, allocFlags: createInfo.flags, pUserData: createInfo.pUserData, suballocType, strategy, pAllocation);
12544	if (res == VK_SUCCESS)
12545	{
12546	VMA_DEBUG_LOG(" Returned from existing block #%u", pCurrBlock->GetId());
12547	IncrementallySortBlocks();
12548	return VK_SUCCESS;
12549	}
12550	}
12551	}
12552	}
12553
12554	// 2. Try to create new block.
12555	if (canCreateNewBlock)
12556	{
12557	// Calculate optimal size for new block.
12558	VkDeviceSize newBlockSize = m_PreferredBlockSize;
12559	uint32_t newBlockSizeShift = 0;
12560	const uint32_t NEW_BLOCK_SIZE_SHIFT_MAX = 3;
12561
12562	if (!m_ExplicitBlockSize)
12563	{
12564	// Allocate 1/8, 1/4, 1/2 as first blocks.
12565	const VkDeviceSize maxExistingBlockSize = CalcMaxBlockSize();
12566	for (uint32_t i = 0; i < NEW_BLOCK_SIZE_SHIFT_MAX; ++i)
12567	{
12568	const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
12569	if (smallerNewBlockSize > maxExistingBlockSize && smallerNewBlockSize >= size * 2)
12570	{
12571	newBlockSize = smallerNewBlockSize;
12572	++newBlockSizeShift;
12573	}
12574	else
12575	{
12576	break;
12577	}
12578	}
12579	}
12580
12581	size_t newBlockIndex = 0;
12582	VkResult res = (newBlockSize <= freeMemory || !canFallbackToDedicated) ?
12583	CreateBlock(blockSize: newBlockSize, pNewBlockIndex: &newBlockIndex) : VK_ERROR_OUT_OF_DEVICE_MEMORY;
12584	// Allocation of this size failed? Try 1/2, 1/4, 1/8 of m_PreferredBlockSize.
12585	if (!m_ExplicitBlockSize)
12586	{
12587	while (res < 0 && newBlockSizeShift < NEW_BLOCK_SIZE_SHIFT_MAX)
12588	{
12589	const VkDeviceSize smallerNewBlockSize = newBlockSize / 2;
12590	if (smallerNewBlockSize >= size)
12591	{
12592	newBlockSize = smallerNewBlockSize;
12593	++newBlockSizeShift;
12594	res = (newBlockSize <= freeMemory || !canFallbackToDedicated) ?
12595	CreateBlock(blockSize: newBlockSize, pNewBlockIndex: &newBlockIndex) : VK_ERROR_OUT_OF_DEVICE_MEMORY;
12596	}
12597	else
12598	{
12599	break;
12600	}
12601	}
12602	}
12603
12604	if (res == VK_SUCCESS)
12605	{
12606	VmaDeviceMemoryBlock* const pBlock = m_Blocks[newBlockIndex];
12607	VMA_ASSERT(pBlock->m_pMetadata->GetSize() >= size);
12608
12609	res = AllocateFromBlock(
12610	pBlock, size, alignment, allocFlags: createInfo.flags, pUserData: createInfo.pUserData, suballocType, strategy, pAllocation);
12611	if (res == VK_SUCCESS)
12612	{
12613	VMA_DEBUG_LOG(" Created new block #%u Size=%llu", pBlock->GetId(), newBlockSize);
12614	IncrementallySortBlocks();
12615	return VK_SUCCESS;
12616	}
12617	else
12618	{
12619	// Allocation from new block failed, possibly due to VMA_DEBUG_MARGIN or alignment.
12620	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12621	}
12622	}
12623	}
12624
12625	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12626	}
12627
12628	void VmaBlockVector::Free(const VmaAllocation hAllocation)
12629	{
12630	VmaDeviceMemoryBlock* pBlockToDelete = VMA_NULL;
12631
12632	bool budgetExceeded = false;
12633	{
12634	const uint32_t heapIndex = m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex: m_MemoryTypeIndex);
12635	VmaBudget heapBudget = {};
12636	m_hAllocator->GetHeapBudgets(outBudgets: &heapBudget, firstHeap: heapIndex, heapCount: 1);
12637	budgetExceeded = heapBudget.usage >= heapBudget.budget;
12638	}
12639
12640	// Scope for lock.
12641	{
12642	VmaMutexLockWrite lock(m_Mutex, m_hAllocator->m_UseMutex);
12643
12644	VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
12645
12646	if (IsCorruptionDetectionEnabled())
12647	{
12648	VkResult res = pBlock->ValidateMagicValueAfterAllocation(hAllocator: m_hAllocator, allocOffset: hAllocation->GetOffset(), allocSize: hAllocation->GetSize());
12649	VMA_ASSERT(res == VK_SUCCESS && "Couldn't map block memory to validate magic value.");
12650	}
12651
12652	if (hAllocation->IsPersistentMap())
12653	{
12654	pBlock->Unmap(hAllocator: m_hAllocator, count: 1);
12655	}
12656
12657	const bool hadEmptyBlockBeforeFree = HasEmptyBlock();
12658	pBlock->m_pMetadata->Free(allocHandle: hAllocation->GetAllocHandle());
12659	pBlock->PostFree(hAllocator: m_hAllocator);
12660	VMA_HEAVY_ASSERT(pBlock->Validate());
12661
12662	VMA_DEBUG_LOG(" Freed from MemoryTypeIndex=%u", m_MemoryTypeIndex);
12663
12664	const bool canDeleteBlock = m_Blocks.size() > m_MinBlockCount;
12665	// pBlock became empty after this deallocation.
12666	if (pBlock->m_pMetadata->IsEmpty())
12667	{
12668	// Already had empty block. We don't want to have two, so delete this one.
12669	if ((hadEmptyBlockBeforeFree || budgetExceeded) && canDeleteBlock)
12670	{
12671	pBlockToDelete = pBlock;
12672	Remove(pBlock);
12673	}
12674	// else: We now have one empty block - leave it. A hysteresis to avoid allocating whole block back and forth.
12675	}
12676	// pBlock didn't become empty, but we have another empty block - find and free that one.
12677	// (This is optional, heuristics.)
12678	else if (hadEmptyBlockBeforeFree && canDeleteBlock)
12679	{
12680	VmaDeviceMemoryBlock* pLastBlock = m_Blocks.back();
12681	if (pLastBlock->m_pMetadata->IsEmpty())
12682	{
12683	pBlockToDelete = pLastBlock;
12684	m_Blocks.pop_back();
12685	}
12686	}
12687
12688	IncrementallySortBlocks();
12689	}
12690
12691	// Destruction of a free block. Deferred until this point, outside of mutex
12692	// lock, for performance reason.
12693	if (pBlockToDelete != VMA_NULL)
12694	{
12695	VMA_DEBUG_LOG(" Deleted empty block #%u", pBlockToDelete->GetId());
12696	pBlockToDelete->Destroy(allocator: m_hAllocator);
12697	vma_delete(hAllocator: m_hAllocator, ptr: pBlockToDelete);
12698	}
12699
12700	m_hAllocator->m_Budget.RemoveAllocation(heapIndex: m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex: m_MemoryTypeIndex), allocationSize: hAllocation->GetSize());
12701	m_hAllocator->m_AllocationObjectAllocator.Free(hAlloc: hAllocation);
12702	}
12703
12704	VkDeviceSize VmaBlockVector::CalcMaxBlockSize() const
12705	{
12706	VkDeviceSize result = 0;
12707	for (size_t i = m_Blocks.size(); i--; )
12708	{
12709	result = VMA_MAX(result, m_Blocks[i]->m_pMetadata->GetSize());
12710	if (result >= m_PreferredBlockSize)
12711	{
12712	break;
12713	}
12714	}
12715	return result;
12716	}
12717
12718	void VmaBlockVector::Remove(VmaDeviceMemoryBlock* pBlock)
12719	{
12720	for (uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
12721	{
12722	if (m_Blocks[blockIndex] == pBlock)
12723	{
12724	VmaVectorRemove(vec&: m_Blocks, index: blockIndex);
12725	return;
12726	}
12727	}
12728	VMA_ASSERT(0);
12729	}
12730
12731	void VmaBlockVector::IncrementallySortBlocks()
12732	{
12733	if (!m_IncrementalSort)
12734	return;
12735	if (m_Algorithm != VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
12736	{
12737	// Bubble sort only until first swap.
12738	for (size_t i = 1; i < m_Blocks.size(); ++i)
12739	{
12740	if (m_Blocks[i - 1]->m_pMetadata->GetSumFreeSize() > m_Blocks[i]->m_pMetadata->GetSumFreeSize())
12741	{
12742	VMA_SWAP(m_Blocks[i - 1], m_Blocks[i]);
12743	return;
12744	}
12745	}
12746	}
12747	}
12748
12749	void VmaBlockVector::SortByFreeSize()
12750	{
12751	VMA_SORT(m_Blocks.begin(), m_Blocks.end(),
12752	[](VmaDeviceMemoryBlock* b1, VmaDeviceMemoryBlock* b2) -> bool
12753	{
12754	return b1->m_pMetadata->GetSumFreeSize() < b2->m_pMetadata->GetSumFreeSize();
12755	});
12756	}
12757
12758	VkResult VmaBlockVector::AllocateFromBlock(
12759	VmaDeviceMemoryBlock* pBlock,
12760	VkDeviceSize size,
12761	VkDeviceSize alignment,
12762	VmaAllocationCreateFlags allocFlags,
12763	void* pUserData,
12764	VmaSuballocationType suballocType,
12765	uint32_t strategy,
12766	VmaAllocation* pAllocation)
12767	{
12768	const bool isUpperAddress = (allocFlags & VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT) != 0;
12769
12770	VmaAllocationRequest currRequest = {};
12771	if (pBlock->m_pMetadata->CreateAllocationRequest(
12772	allocSize: size,
12773	allocAlignment: alignment,
12774	upperAddress: isUpperAddress,
12775	allocType: suballocType,
12776	strategy,
12777	pAllocationRequest: &currRequest))
12778	{
12779	return CommitAllocationRequest(allocRequest&: currRequest, pBlock, alignment, allocFlags, pUserData, suballocType, pAllocation);
12780	}
12781	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
12782	}
12783
12784	VkResult VmaBlockVector::CommitAllocationRequest(
12785	VmaAllocationRequest& allocRequest,
12786	VmaDeviceMemoryBlock* pBlock,
12787	VkDeviceSize alignment,
12788	VmaAllocationCreateFlags allocFlags,
12789	void* pUserData,
12790	VmaSuballocationType suballocType,
12791	VmaAllocation* pAllocation)
12792	{
12793	const bool mapped = (allocFlags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0;
12794	const bool isUserDataString = (allocFlags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0;
12795	const bool isMappingAllowed = (allocFlags &
12796	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0;
12797
12798	pBlock->PostAlloc();
12799	// Allocate from pCurrBlock.
12800	if (mapped)
12801	{
12802	VkResult res = pBlock->Map(hAllocator: m_hAllocator, count: 1, VMA_NULL);
12803	if (res != VK_SUCCESS)
12804	{
12805	return res;
12806	}
12807	}
12808
12809	*pAllocation = m_hAllocator->m_AllocationObjectAllocator.Allocate(args: isMappingAllowed);
12810	pBlock->m_pMetadata->Alloc(request: allocRequest, type: suballocType, userData: *pAllocation);
12811	(*pAllocation)->InitBlockAllocation(
12812	block: pBlock,
12813	allocHandle: allocRequest.allocHandle,
12814	alignment,
12815	size: allocRequest.size, // Not size, as actual allocation size may be larger than requested!
12816	memoryTypeIndex: m_MemoryTypeIndex,
12817	suballocationType: suballocType,
12818	mapped);
12819	VMA_HEAVY_ASSERT(pBlock->Validate());
12820	if (isUserDataString)
12821	(*pAllocation)->SetName(hAllocator: m_hAllocator, pName: (const char*)pUserData);
12822	else
12823	(*pAllocation)->SetUserData(hAllocator: m_hAllocator, pUserData);
12824	m_hAllocator->m_Budget.AddAllocation(heapIndex: m_hAllocator->MemoryTypeIndexToHeapIndex(memTypeIndex: m_MemoryTypeIndex), allocationSize: allocRequest.size);
12825	if (VMA_DEBUG_INITIALIZE_ALLOCATIONS)
12826	{
12827	m_hAllocator->FillAllocation(hAllocation: *pAllocation, pattern: VMA_ALLOCATION_FILL_PATTERN_CREATED);
12828	}
12829	if (IsCorruptionDetectionEnabled())
12830	{
12831	VkResult res = pBlock->WriteMagicValueAfterAllocation(hAllocator: m_hAllocator, allocOffset: (*pAllocation)->GetOffset(), allocSize: allocRequest.size);
12832	VMA_ASSERT(res == VK_SUCCESS && "Couldn't map block memory to write magic value.");
12833	}
12834	return VK_SUCCESS;
12835	}
12836
12837	VkResult VmaBlockVector::CreateBlock(VkDeviceSize blockSize, size_t* pNewBlockIndex)
12838	{
12839	VkMemoryAllocateInfo allocInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
12840	allocInfo.pNext = m_pMemoryAllocateNext;
12841	allocInfo.memoryTypeIndex = m_MemoryTypeIndex;
12842	allocInfo.allocationSize = blockSize;
12843
12844	#if VMA_BUFFER_DEVICE_ADDRESS
12845	// Every standalone block can potentially contain a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT - always enable the feature.
12846	VkMemoryAllocateFlagsInfoKHR allocFlagsInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO_KHR };
12847	if (m_hAllocator->m_UseKhrBufferDeviceAddress)
12848	{
12849	allocFlagsInfo.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR;
12850	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &allocFlagsInfo);
12851	}
12852	#endif // VMA_BUFFER_DEVICE_ADDRESS
12853
12854	#if VMA_MEMORY_PRIORITY
12855	VkMemoryPriorityAllocateInfoEXT priorityInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_PRIORITY_ALLOCATE_INFO_EXT };
12856	if (m_hAllocator->m_UseExtMemoryPriority)
12857	{
12858	VMA_ASSERT(m_Priority >= 0.f && m_Priority <= 1.f);
12859	priorityInfo.priority = m_Priority;
12860	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &priorityInfo);
12861	}
12862	#endif // VMA_MEMORY_PRIORITY
12863
12864	#if VMA_EXTERNAL_MEMORY
12865	// Attach VkExportMemoryAllocateInfoKHR if necessary.
12866	VkExportMemoryAllocateInfoKHR exportMemoryAllocInfo = { .sType: VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR };
12867	exportMemoryAllocInfo.handleTypes = m_hAllocator->GetExternalMemoryHandleTypeFlags(memTypeIndex: m_MemoryTypeIndex);
12868	if (exportMemoryAllocInfo.handleTypes != 0)
12869	{
12870	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &exportMemoryAllocInfo);
12871	}
12872	#endif // VMA_EXTERNAL_MEMORY
12873
12874	VkDeviceMemory mem = VK_NULL_HANDLE;
12875	VkResult res = m_hAllocator->AllocateVulkanMemory(pAllocateInfo: &allocInfo, pMemory: &mem);
12876	if (res < 0)
12877	{
12878	return res;
12879	}
12880
12881	// New VkDeviceMemory successfully created.
12882
12883	// Create new Allocation for it.
12884	VmaDeviceMemoryBlock* const pBlock = vma_new(m_hAllocator, VmaDeviceMemoryBlock)(m_hAllocator);
12885	pBlock->Init(
12886	hAllocator: m_hAllocator,
12887	hParentPool: m_hParentPool,
12888	newMemoryTypeIndex: m_MemoryTypeIndex,
12889	newMemory: mem,
12890	newSize: allocInfo.allocationSize,
12891	id: m_NextBlockId++,
12892	algorithm: m_Algorithm,
12893	bufferImageGranularity: m_BufferImageGranularity);
12894
12895	m_Blocks.push_back(src: pBlock);
12896	if (pNewBlockIndex != VMA_NULL)
12897	{
12898	*pNewBlockIndex = m_Blocks.size() - 1;
12899	}
12900
12901	return VK_SUCCESS;
12902	}
12903
12904	bool VmaBlockVector::HasEmptyBlock()
12905	{
12906	for (size_t index = 0, count = m_Blocks.size(); index < count; ++index)
12907	{
12908	VmaDeviceMemoryBlock* const pBlock = m_Blocks[index];
12909	if (pBlock->m_pMetadata->IsEmpty())
12910	{
12911	return true;
12912	}
12913	}
12914	return false;
12915	}
12916
12917	#if VMA_STATS_STRING_ENABLED
12918	void VmaBlockVector::PrintDetailedMap(class VmaJsonWriter& json)
12919	{
12920	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12921
12922
12923	json.BeginObject();
12924	for (size_t i = 0; i < m_Blocks.size(); ++i)
12925	{
12926	json.BeginString();
12927	json.ContinueString(n: m_Blocks[i]->GetId());
12928	json.EndString();
12929
12930	json.BeginObject();
12931	json.WriteString(pStr: "MapRefCount");
12932	json.WriteNumber(n: m_Blocks[i]->GetMapRefCount());
12933
12934	m_Blocks[i]->m_pMetadata->PrintDetailedMap(json);
12935	json.EndObject();
12936	}
12937	json.EndObject();
12938	}
12939	#endif // VMA_STATS_STRING_ENABLED
12940
12941	VkResult VmaBlockVector::CheckCorruption()
12942	{
12943	if (!IsCorruptionDetectionEnabled())
12944	{
12945	return VK_ERROR_FEATURE_NOT_PRESENT;
12946	}
12947
12948	VmaMutexLockRead lock(m_Mutex, m_hAllocator->m_UseMutex);
12949	for (uint32_t blockIndex = 0; blockIndex < m_Blocks.size(); ++blockIndex)
12950	{
12951	VmaDeviceMemoryBlock* const pBlock = m_Blocks[blockIndex];
12952	VMA_ASSERT(pBlock);
12953	VkResult res = pBlock->CheckCorruption(hAllocator: m_hAllocator);
12954	if (res != VK_SUCCESS)
12955	{
12956	return res;
12957	}
12958	}
12959	return VK_SUCCESS;
12960	}
12961
12962	#endif // _VMA_BLOCK_VECTOR_FUNCTIONS
12963
12964	#ifndef _VMA_DEFRAGMENTATION_CONTEXT_FUNCTIONS
12965	VmaDefragmentationContext_T::VmaDefragmentationContext_T(
12966	VmaAllocator hAllocator,
12967	const VmaDefragmentationInfo& info)
12968	: m_MaxPassBytes(info.maxBytesPerPass == 0 ? VK_WHOLE_SIZE : info.maxBytesPerPass),
12969	m_MaxPassAllocations(info.maxAllocationsPerPass == 0 ? UINT32_MAX : info.maxAllocationsPerPass),
12970	m_MoveAllocator(hAllocator->GetAllocationCallbacks()),
12971	m_Moves(m_MoveAllocator)
12972	{
12973	m_Algorithm = info.flags & VMA_DEFRAGMENTATION_FLAG_ALGORITHM_MASK;
12974
12975	if (info.pool != VMA_NULL)
12976	{
12977	m_BlockVectorCount = 1;
12978	m_PoolBlockVector = &info.pool->m_BlockVector;
12979	m_pBlockVectors = &m_PoolBlockVector;
12980	m_PoolBlockVector->SetIncrementalSort(false);
12981	m_PoolBlockVector->SortByFreeSize();
12982	}
12983	else
12984	{
12985	m_BlockVectorCount = hAllocator->GetMemoryTypeCount();
12986	m_PoolBlockVector = VMA_NULL;
12987	m_pBlockVectors = hAllocator->m_pBlockVectors;
12988	for (uint32_t i = 0; i < m_BlockVectorCount; ++i)
12989	{
12990	VmaBlockVector* vector = m_pBlockVectors[i];
12991	if (vector != VMA_NULL)
12992	{
12993	vector->SetIncrementalSort(false);
12994	vector->SortByFreeSize();
12995	}
12996	}
12997	}
12998
12999	switch (m_Algorithm)
13000	{
13001	case 0: // Default algorithm
13002	m_Algorithm = VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT;
13003	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
13004	{
13005	m_AlgorithmState = vma_new_array(hAllocator, StateBalanced, m_BlockVectorCount);
13006	break;
13007	}
13008	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13009	{
13010	if (hAllocator->GetBufferImageGranularity() > 1)
13011	{
13012	m_AlgorithmState = vma_new_array(hAllocator, StateExtensive, m_BlockVectorCount);
13013	}
13014	break;
13015	}
13016	}
13017	}
13018
13019	VmaDefragmentationContext_T::~VmaDefragmentationContext_T()
13020	{
13021	if (m_PoolBlockVector != VMA_NULL)
13022	{
13023	m_PoolBlockVector->SetIncrementalSort(true);
13024	}
13025	else
13026	{
13027	for (uint32_t i = 0; i < m_BlockVectorCount; ++i)
13028	{
13029	VmaBlockVector* vector = m_pBlockVectors[i];
13030	if (vector != VMA_NULL)
13031	vector->SetIncrementalSort(true);
13032	}
13033	}
13034
13035	if (m_AlgorithmState)
13036	{
13037	switch (m_Algorithm)
13038	{
13039	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
13040	vma_delete_array(pAllocationCallbacks: m_MoveAllocator.m_pCallbacks, ptr: reinterpret_cast<StateBalanced*>(m_AlgorithmState), count: m_BlockVectorCount);
13041	break;
13042	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13043	vma_delete_array(pAllocationCallbacks: m_MoveAllocator.m_pCallbacks, ptr: reinterpret_cast<StateExtensive*>(m_AlgorithmState), count: m_BlockVectorCount);
13044	break;
13045	default:
13046	VMA_ASSERT(0);
13047	}
13048	}
13049	}
13050
13051	VkResult VmaDefragmentationContext_T::DefragmentPassBegin(VmaDefragmentationPassMoveInfo& moveInfo)
13052	{
13053	if (m_PoolBlockVector != VMA_NULL)
13054	{
13055	VmaMutexLockWrite lock(m_PoolBlockVector->GetMutex(), m_PoolBlockVector->GetAllocator()->m_UseMutex);
13056
13057	if (m_PoolBlockVector->GetBlockCount() > 1)
13058	ComputeDefragmentation(vector&: *m_PoolBlockVector, index: 0);
13059	else if (m_PoolBlockVector->GetBlockCount() == 1)
13060	ReallocWithinBlock(vector&: *m_PoolBlockVector, block: m_PoolBlockVector->GetBlock(index: 0));
13061	}
13062	else
13063	{
13064	for (uint32_t i = 0; i < m_BlockVectorCount; ++i)
13065	{
13066	if (m_pBlockVectors[i] != VMA_NULL)
13067	{
13068	VmaMutexLockWrite lock(m_pBlockVectors[i]->GetMutex(), m_pBlockVectors[i]->GetAllocator()->m_UseMutex);
13069
13070	if (m_pBlockVectors[i]->GetBlockCount() > 1)
13071	{
13072	if (ComputeDefragmentation(vector&: *m_pBlockVectors[i], index: i))
13073	break;
13074	}
13075	else if (m_pBlockVectors[i]->GetBlockCount() == 1)
13076	{
13077	if (ReallocWithinBlock(vector&: *m_pBlockVectors[i], block: m_pBlockVectors[i]->GetBlock(index: 0)))
13078	break;
13079	}
13080	}
13081	}
13082	}
13083
13084	moveInfo.moveCount = static_cast<uint32_t>(m_Moves.size());
13085	if (moveInfo.moveCount > 0)
13086	{
13087	moveInfo.pMoves = m_Moves.data();
13088	return VK_INCOMPLETE;
13089	}
13090
13091	moveInfo.pMoves = VMA_NULL;
13092	return VK_SUCCESS;
13093	}
13094
13095	VkResult VmaDefragmentationContext_T::DefragmentPassEnd(VmaDefragmentationPassMoveInfo& moveInfo)
13096	{
13097	VMA_ASSERT(moveInfo.moveCount > 0 ? moveInfo.pMoves != VMA_NULL : true);
13098
13099	VkResult result = VK_SUCCESS;
13100	VmaStlAllocator<FragmentedBlock> blockAllocator(m_MoveAllocator.m_pCallbacks);
13101	VmaVector<FragmentedBlock, VmaStlAllocator<FragmentedBlock>> immovableBlocks(blockAllocator);
13102	VmaVector<FragmentedBlock, VmaStlAllocator<FragmentedBlock>> mappedBlocks(blockAllocator);
13103
13104	VmaAllocator allocator = VMA_NULL;
13105	for (uint32_t i = 0; i < moveInfo.moveCount; ++i)
13106	{
13107	VmaDefragmentationMove& move = moveInfo.pMoves[i];
13108	size_t prevCount = 0, currentCount = 0;
13109	VkDeviceSize freedBlockSize = 0;
13110
13111	uint32_t vectorIndex;
13112	VmaBlockVector* vector;
13113	if (m_PoolBlockVector != VMA_NULL)
13114	{
13115	vectorIndex = 0;
13116	vector = m_PoolBlockVector;
13117	}
13118	else
13119	{
13120	vectorIndex = move.srcAllocation->GetMemoryTypeIndex();
13121	vector = m_pBlockVectors[vectorIndex];
13122	VMA_ASSERT(vector != VMA_NULL);
13123	}
13124
13125	switch (move.operation)
13126	{
13127	case VMA_DEFRAGMENTATION_MOVE_OPERATION_COPY:
13128	{
13129	uint8_t mapCount = move.srcAllocation->SwapBlockAllocation(hAllocator: vector->m_hAllocator, allocation: move.dstTmpAllocation);
13130	if (mapCount > 0)
13131	{
13132	allocator = vector->m_hAllocator;
13133	VmaDeviceMemoryBlock* newMapBlock = move.srcAllocation->GetBlock();
13134	bool notPresent = true;
13135	for (FragmentedBlock& block : mappedBlocks)
13136	{
13137	if (block.block == newMapBlock)
13138	{
13139	notPresent = false;
13140	block.data += mapCount;
13141	break;
13142	}
13143	}
13144	if (notPresent)
13145	mappedBlocks.push_back(src: { .data: mapCount, .block: newMapBlock });
13146	}
13147
13148	// Scope for locks, Free have it's own lock
13149	{
13150	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13151	prevCount = vector->GetBlockCount();
13152	freedBlockSize = move.dstTmpAllocation->GetBlock()->m_pMetadata->GetSize();
13153	}
13154	vector->Free(hAllocation: move.dstTmpAllocation);
13155	{
13156	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13157	currentCount = vector->GetBlockCount();
13158	}
13159
13160	result = VK_INCOMPLETE;
13161	break;
13162	}
13163	case VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE:
13164	{
13165	m_PassStats.bytesMoved -= move.srcAllocation->GetSize();
13166	--m_PassStats.allocationsMoved;
13167	vector->Free(hAllocation: move.dstTmpAllocation);
13168
13169	VmaDeviceMemoryBlock* newBlock = move.srcAllocation->GetBlock();
13170	bool notPresent = true;
13171	for (const FragmentedBlock& block : immovableBlocks)
13172	{
13173	if (block.block == newBlock)
13174	{
13175	notPresent = false;
13176	break;
13177	}
13178	}
13179	if (notPresent)
13180	immovableBlocks.push_back(src: { .data: vectorIndex, .block: newBlock });
13181	break;
13182	}
13183	case VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY:
13184	{
13185	m_PassStats.bytesMoved -= move.srcAllocation->GetSize();
13186	--m_PassStats.allocationsMoved;
13187	// Scope for locks, Free have it's own lock
13188	{
13189	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13190	prevCount = vector->GetBlockCount();
13191	freedBlockSize = move.srcAllocation->GetBlock()->m_pMetadata->GetSize();
13192	}
13193	vector->Free(hAllocation: move.srcAllocation);
13194	{
13195	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13196	currentCount = vector->GetBlockCount();
13197	}
13198	freedBlockSize *= prevCount - currentCount;
13199
13200	VkDeviceSize dstBlockSize;
13201	{
13202	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13203	dstBlockSize = move.dstTmpAllocation->GetBlock()->m_pMetadata->GetSize();
13204	}
13205	vector->Free(hAllocation: move.dstTmpAllocation);
13206	{
13207	VmaMutexLockRead lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13208	freedBlockSize += dstBlockSize * (currentCount - vector->GetBlockCount());
13209	currentCount = vector->GetBlockCount();
13210	}
13211
13212	result = VK_INCOMPLETE;
13213	break;
13214	}
13215	default:
13216	VMA_ASSERT(0);
13217	}
13218
13219	if (prevCount > currentCount)
13220	{
13221	size_t freedBlocks = prevCount - currentCount;
13222	m_PassStats.deviceMemoryBlocksFreed += static_cast<uint32_t>(freedBlocks);
13223	m_PassStats.bytesFreed += freedBlockSize;
13224	}
13225
13226	switch (m_Algorithm)
13227	{
13228	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13229	{
13230	if (m_AlgorithmState != VMA_NULL)
13231	{
13232	// Avoid unnecessary tries to allocate when new free block is avaiable
13233	StateExtensive& state = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[vectorIndex];
13234	if (state.firstFreeBlock != SIZE_MAX)
13235	{
13236	const size_t diff = prevCount - currentCount;
13237	if (state.firstFreeBlock >= diff)
13238	{
13239	state.firstFreeBlock -= diff;
13240	if (state.firstFreeBlock != 0)
13241	state.firstFreeBlock -= vector->GetBlock(index: state.firstFreeBlock - 1)->m_pMetadata->IsEmpty();
13242	}
13243	else
13244	state.firstFreeBlock = 0;
13245	}
13246	}
13247	}
13248	}
13249	}
13250	moveInfo.moveCount = 0;
13251	moveInfo.pMoves = VMA_NULL;
13252	m_Moves.clear();
13253
13254	// Update stats
13255	m_GlobalStats.allocationsMoved += m_PassStats.allocationsMoved;
13256	m_GlobalStats.bytesFreed += m_PassStats.bytesFreed;
13257	m_GlobalStats.bytesMoved += m_PassStats.bytesMoved;
13258	m_GlobalStats.deviceMemoryBlocksFreed += m_PassStats.deviceMemoryBlocksFreed;
13259	m_PassStats = { .bytesMoved: 0 };
13260
13261	// Move blocks with immovable allocations according to algorithm
13262	if (immovableBlocks.size() > 0)
13263	{
13264	switch (m_Algorithm)
13265	{
13266	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13267	{
13268	if (m_AlgorithmState != VMA_NULL)
13269	{
13270	bool swapped = false;
13271	// Move to the start of free blocks range
13272	for (const FragmentedBlock& block : immovableBlocks)
13273	{
13274	StateExtensive& state = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[block.data];
13275	if (state.operation != StateExtensive::Operation::Cleanup)
13276	{
13277	VmaBlockVector* vector = m_pBlockVectors[block.data];
13278	VmaMutexLockWrite lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13279
13280	for (size_t i = 0, count = vector->GetBlockCount() - m_ImmovableBlockCount; i < count; ++i)
13281	{
13282	if (vector->GetBlock(index: i) == block.block)
13283	{
13284	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[vector->GetBlockCount() - ++m_ImmovableBlockCount]);
13285	if (state.firstFreeBlock != SIZE_MAX)
13286	{
13287	if (i + 1 < state.firstFreeBlock)
13288	{
13289	if (state.firstFreeBlock > 1)
13290	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[--state.firstFreeBlock]);
13291	else
13292	--state.firstFreeBlock;
13293	}
13294	}
13295	swapped = true;
13296	break;
13297	}
13298	}
13299	}
13300	}
13301	if (swapped)
13302	result = VK_INCOMPLETE;
13303	break;
13304	}
13305	}
13306	default:
13307	{
13308	// Move to the begining
13309	for (const FragmentedBlock& block : immovableBlocks)
13310	{
13311	VmaBlockVector* vector = m_pBlockVectors[block.data];
13312	VmaMutexLockWrite lock(vector->GetMutex(), vector->GetAllocator()->m_UseMutex);
13313
13314	for (size_t i = m_ImmovableBlockCount; i < vector->GetBlockCount(); ++i)
13315	{
13316	if (vector->GetBlock(index: i) == block.block)
13317	{
13318	VMA_SWAP(vector->m_Blocks[i], vector->m_Blocks[m_ImmovableBlockCount++]);
13319	break;
13320	}
13321	}
13322	}
13323	break;
13324	}
13325	}
13326	}
13327
13328	// Bulk-map destination blocks
13329	for (const FragmentedBlock& block : mappedBlocks)
13330	{
13331	VkResult res = block.block->Map(hAllocator: allocator, count: block.data, VMA_NULL);
13332	VMA_ASSERT(res == VK_SUCCESS);
13333	}
13334	return result;
13335	}
13336
13337	bool VmaDefragmentationContext_T::ComputeDefragmentation(VmaBlockVector& vector, size_t index)
13338	{
13339	switch (m_Algorithm)
13340	{
13341	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT:
13342	return ComputeDefragmentation_Fast(vector);
13343	default:
13344	VMA_ASSERT(0);
13345	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_BALANCED_BIT:
13346	return ComputeDefragmentation_Balanced(vector, index, update: true);
13347	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FULL_BIT:
13348	return ComputeDefragmentation_Full(vector);
13349	case VMA_DEFRAGMENTATION_FLAG_ALGORITHM_EXTENSIVE_BIT:
13350	return ComputeDefragmentation_Extensive(vector, index);
13351	}
13352	}
13353
13354	VmaDefragmentationContext_T::MoveAllocationData VmaDefragmentationContext_T::GetMoveData(
13355	VmaAllocHandle handle, VmaBlockMetadata* metadata)
13356	{
13357	MoveAllocationData moveData;
13358	moveData.move.srcAllocation = (VmaAllocation)metadata->GetAllocationUserData(allocHandle: handle);
13359	moveData.size = moveData.move.srcAllocation->GetSize();
13360	moveData.alignment = moveData.move.srcAllocation->GetAlignment();
13361	moveData.type = moveData.move.srcAllocation->GetSuballocationType();
13362	moveData.flags = 0;
13363
13364	if (moveData.move.srcAllocation->IsPersistentMap())
13365	moveData.flags |= VMA_ALLOCATION_CREATE_MAPPED_BIT;
13366	if (moveData.move.srcAllocation->IsMappingAllowed())
13367	moveData.flags |= VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT;
13368
13369	return moveData;
13370	}
13371
13372	VmaDefragmentationContext_T::CounterStatus VmaDefragmentationContext_T::CheckCounters(VkDeviceSize bytes)
13373	{
13374	// Ignore allocation if will exceed max size for copy
13375	if (m_PassStats.bytesMoved + bytes > m_MaxPassBytes)
13376	{
13377	if (++m_IgnoredAllocs < MAX_ALLOCS_TO_IGNORE)
13378	return CounterStatus::Ignore;
13379	else
13380	return CounterStatus::End;
13381	}
13382	return CounterStatus::Pass;
13383	}
13384
13385	bool VmaDefragmentationContext_T::IncrementCounters(VkDeviceSize bytes)
13386	{
13387	m_PassStats.bytesMoved += bytes;
13388	// Early return when max found
13389	if (++m_PassStats.allocationsMoved >= m_MaxPassAllocations || m_PassStats.bytesMoved >= m_MaxPassBytes)
13390	{
13391	VMA_ASSERT(m_PassStats.allocationsMoved == m_MaxPassAllocations ||
13392	m_PassStats.bytesMoved == m_MaxPassBytes && "Exceeded maximal pass threshold!");
13393	return true;
13394	}
13395	return false;
13396	}
13397
13398	bool VmaDefragmentationContext_T::ReallocWithinBlock(VmaBlockVector& vector, VmaDeviceMemoryBlock* block)
13399	{
13400	VmaBlockMetadata* metadata = block->m_pMetadata;
13401
13402	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13403	handle != VK_NULL_HANDLE;
13404	handle = metadata->GetNextAllocation(prevAlloc: handle))
13405	{
13406	MoveAllocationData moveData = GetMoveData(handle, metadata);
13407	// Ignore newly created allocations by defragmentation algorithm
13408	if (moveData.move.srcAllocation->GetUserData() == this)
13409	continue;
13410	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13411	{
13412	case CounterStatus::Ignore:
13413	continue;
13414	case CounterStatus::End:
13415	return true;
13416	default:
13417	VMA_ASSERT(0);
13418	case CounterStatus::Pass:
13419	break;
13420	}
13421
13422	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13423	if (offset != 0 && metadata->GetSumFreeSize() >= moveData.size)
13424	{
13425	VmaAllocationRequest request = {};
13426	if (metadata->CreateAllocationRequest(
13427	allocSize: moveData.size,
13428	allocAlignment: moveData.alignment,
13429	upperAddress: false,
13430	allocType: moveData.type,
13431	strategy: VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13432	pAllocationRequest: &request))
13433	{
13434	if (metadata->GetAllocationOffset(allocHandle: request.allocHandle) < offset)
13435	{
13436	if (vector.CommitAllocationRequest(
13437	allocRequest&: request,
13438	pBlock: block,
13439	alignment: moveData.alignment,
13440	allocFlags: moveData.flags,
13441	pUserData: this,
13442	suballocType: moveData.type,
13443	pAllocation: &moveData.move.dstTmpAllocation) == VK_SUCCESS)
13444	{
13445	m_Moves.push_back(src: moveData.move);
13446	if (IncrementCounters(bytes: moveData.size))
13447	return true;
13448	}
13449	}
13450	}
13451	}
13452	}
13453	return false;
13454	}
13455
13456	bool VmaDefragmentationContext_T::AllocInOtherBlock(size_t start, size_t end, MoveAllocationData& data, VmaBlockVector& vector)
13457	{
13458	for (; start < end; ++start)
13459	{
13460	VmaDeviceMemoryBlock* dstBlock = vector.GetBlock(index: start);
13461	if (dstBlock->m_pMetadata->GetSumFreeSize() >= data.size)
13462	{
13463	if (vector.AllocateFromBlock(pBlock: dstBlock,
13464	size: data.size,
13465	alignment: data.alignment,
13466	allocFlags: data.flags,
13467	pUserData: this,
13468	suballocType: data.type,
13469	strategy: 0,
13470	pAllocation: &data.move.dstTmpAllocation) == VK_SUCCESS)
13471	{
13472	m_Moves.push_back(src: data.move);
13473	if (IncrementCounters(bytes: data.size))
13474	return true;
13475	break;
13476	}
13477	}
13478	}
13479	return false;
13480	}
13481
13482	bool VmaDefragmentationContext_T::ComputeDefragmentation_Fast(VmaBlockVector& vector)
13483	{
13484	// Move only between blocks
13485
13486	// Go through allocations in last blocks and try to fit them inside first ones
13487	for (size_t i = vector.GetBlockCount() - 1; i > m_ImmovableBlockCount; --i)
13488	{
13489	VmaBlockMetadata* metadata = vector.GetBlock(index: i)->m_pMetadata;
13490
13491	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13492	handle != VK_NULL_HANDLE;
13493	handle = metadata->GetNextAllocation(prevAlloc: handle))
13494	{
13495	MoveAllocationData moveData = GetMoveData(handle, metadata);
13496	// Ignore newly created allocations by defragmentation algorithm
13497	if (moveData.move.srcAllocation->GetUserData() == this)
13498	continue;
13499	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13500	{
13501	case CounterStatus::Ignore:
13502	continue;
13503	case CounterStatus::End:
13504	return true;
13505	default:
13506	VMA_ASSERT(0);
13507	case CounterStatus::Pass:
13508	break;
13509	}
13510
13511	// Check all previous blocks for free space
13512	if (AllocInOtherBlock(start: 0, end: i, data&: moveData, vector))
13513	return true;
13514	}
13515	}
13516	return false;
13517	}
13518
13519	bool VmaDefragmentationContext_T::ComputeDefragmentation_Balanced(VmaBlockVector& vector, size_t index, bool update)
13520	{
13521	// Go over every allocation and try to fit it in previous blocks at lowest offsets,
13522	// if not possible: realloc within single block to minimize offset (exclude offset == 0),
13523	// but only if there are noticable gaps between them (some heuristic, ex. average size of allocation in block)
13524	VMA_ASSERT(m_AlgorithmState != VMA_NULL);
13525
13526	StateBalanced& vectorState = reinterpret_cast<StateBalanced*>(m_AlgorithmState)[index];
13527	if (update && vectorState.avgAllocSize == UINT64_MAX)
13528	UpdateVectorStatistics(vector, state&: vectorState);
13529
13530	const size_t startMoveCount = m_Moves.size();
13531	VkDeviceSize minimalFreeRegion = vectorState.avgFreeSize / 2;
13532	for (size_t i = vector.GetBlockCount() - 1; i > m_ImmovableBlockCount; --i)
13533	{
13534	VmaDeviceMemoryBlock* block = vector.GetBlock(index: i);
13535	VmaBlockMetadata* metadata = block->m_pMetadata;
13536	VkDeviceSize prevFreeRegionSize = 0;
13537
13538	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13539	handle != VK_NULL_HANDLE;
13540	handle = metadata->GetNextAllocation(prevAlloc: handle))
13541	{
13542	MoveAllocationData moveData = GetMoveData(handle, metadata);
13543	// Ignore newly created allocations by defragmentation algorithm
13544	if (moveData.move.srcAllocation->GetUserData() == this)
13545	continue;
13546	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13547	{
13548	case CounterStatus::Ignore:
13549	continue;
13550	case CounterStatus::End:
13551	return true;
13552	default:
13553	VMA_ASSERT(0);
13554	case CounterStatus::Pass:
13555	break;
13556	}
13557
13558	// Check all previous blocks for free space
13559	const size_t prevMoveCount = m_Moves.size();
13560	if (AllocInOtherBlock(start: 0, end: i, data&: moveData, vector))
13561	return true;
13562
13563	VkDeviceSize nextFreeRegionSize = metadata->GetNextFreeRegionSize(alloc: handle);
13564	// If no room found then realloc within block for lower offset
13565	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13566	if (prevMoveCount == m_Moves.size() && offset != 0 && metadata->GetSumFreeSize() >= moveData.size)
13567	{
13568	// Check if realloc will make sense
13569	if (prevFreeRegionSize >= minimalFreeRegion ||
13570	nextFreeRegionSize >= minimalFreeRegion ||
13571	moveData.size <= vectorState.avgFreeSize ||
13572	moveData.size <= vectorState.avgAllocSize)
13573	{
13574	VmaAllocationRequest request = {};
13575	if (metadata->CreateAllocationRequest(
13576	allocSize: moveData.size,
13577	allocAlignment: moveData.alignment,
13578	upperAddress: false,
13579	allocType: moveData.type,
13580	strategy: VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13581	pAllocationRequest: &request))
13582	{
13583	if (metadata->GetAllocationOffset(allocHandle: request.allocHandle) < offset)
13584	{
13585	if (vector.CommitAllocationRequest(
13586	allocRequest&: request,
13587	pBlock: block,
13588	alignment: moveData.alignment,
13589	allocFlags: moveData.flags,
13590	pUserData: this,
13591	suballocType: moveData.type,
13592	pAllocation: &moveData.move.dstTmpAllocation) == VK_SUCCESS)
13593	{
13594	m_Moves.push_back(src: moveData.move);
13595	if (IncrementCounters(bytes: moveData.size))
13596	return true;
13597	}
13598	}
13599	}
13600	}
13601	}
13602	prevFreeRegionSize = nextFreeRegionSize;
13603	}
13604	}
13605
13606	// No moves perfomed, update statistics to current vector state
13607	if (startMoveCount == m_Moves.size() && !update)
13608	{
13609	vectorState.avgAllocSize = UINT64_MAX;
13610	return ComputeDefragmentation_Balanced(vector, index, update: false);
13611	}
13612	return false;
13613	}
13614
13615	bool VmaDefragmentationContext_T::ComputeDefragmentation_Full(VmaBlockVector& vector)
13616	{
13617	// Go over every allocation and try to fit it in previous blocks at lowest offsets,
13618	// if not possible: realloc within single block to minimize offset (exclude offset == 0)
13619
13620	for (size_t i = vector.GetBlockCount() - 1; i > m_ImmovableBlockCount; --i)
13621	{
13622	VmaDeviceMemoryBlock* block = vector.GetBlock(index: i);
13623	VmaBlockMetadata* metadata = block->m_pMetadata;
13624
13625	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13626	handle != VK_NULL_HANDLE;
13627	handle = metadata->GetNextAllocation(prevAlloc: handle))
13628	{
13629	MoveAllocationData moveData = GetMoveData(handle, metadata);
13630	// Ignore newly created allocations by defragmentation algorithm
13631	if (moveData.move.srcAllocation->GetUserData() == this)
13632	continue;
13633	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13634	{
13635	case CounterStatus::Ignore:
13636	continue;
13637	case CounterStatus::End:
13638	return true;
13639	default:
13640	VMA_ASSERT(0);
13641	case CounterStatus::Pass:
13642	break;
13643	}
13644
13645	// Check all previous blocks for free space
13646	const size_t prevMoveCount = m_Moves.size();
13647	if (AllocInOtherBlock(start: 0, end: i, data&: moveData, vector))
13648	return true;
13649
13650	// If no room found then realloc within block for lower offset
13651	VkDeviceSize offset = moveData.move.srcAllocation->GetOffset();
13652	if (prevMoveCount == m_Moves.size() && offset != 0 && metadata->GetSumFreeSize() >= moveData.size)
13653	{
13654	VmaAllocationRequest request = {};
13655	if (metadata->CreateAllocationRequest(
13656	allocSize: moveData.size,
13657	allocAlignment: moveData.alignment,
13658	upperAddress: false,
13659	allocType: moveData.type,
13660	strategy: VMA_ALLOCATION_CREATE_STRATEGY_MIN_OFFSET_BIT,
13661	pAllocationRequest: &request))
13662	{
13663	if (metadata->GetAllocationOffset(allocHandle: request.allocHandle) < offset)
13664	{
13665	if (vector.CommitAllocationRequest(
13666	allocRequest&: request,
13667	pBlock: block,
13668	alignment: moveData.alignment,
13669	allocFlags: moveData.flags,
13670	pUserData: this,
13671	suballocType: moveData.type,
13672	pAllocation: &moveData.move.dstTmpAllocation) == VK_SUCCESS)
13673	{
13674	m_Moves.push_back(src: moveData.move);
13675	if (IncrementCounters(bytes: moveData.size))
13676	return true;
13677	}
13678	}
13679	}
13680	}
13681	}
13682	}
13683	return false;
13684	}
13685
13686	bool VmaDefragmentationContext_T::ComputeDefragmentation_Extensive(VmaBlockVector& vector, size_t index)
13687	{
13688	// First free single block, then populate it to the brim, then free another block, and so on
13689
13690	// Fallback to previous algorithm since without granularity conflicts it can achieve max packing
13691	if (vector.m_BufferImageGranularity == 1)
13692	return ComputeDefragmentation_Full(vector);
13693
13694	VMA_ASSERT(m_AlgorithmState != VMA_NULL);
13695
13696	StateExtensive& vectorState = reinterpret_cast<StateExtensive*>(m_AlgorithmState)[index];
13697
13698	bool texturePresent = false, bufferPresent = false, otherPresent = false;
13699	switch (vectorState.operation)
13700	{
13701	case StateExtensive::Operation::Done: // Vector defragmented
13702	return false;
13703	case StateExtensive::Operation::FindFreeBlockBuffer:
13704	case StateExtensive::Operation::FindFreeBlockTexture:
13705	case StateExtensive::Operation::FindFreeBlockAll:
13706	{
13707	// No more blocks to free, just perform fast realloc and move to cleanup
13708	if (vectorState.firstFreeBlock == 0)
13709	{
13710	vectorState.operation = StateExtensive::Operation::Cleanup;
13711	return ComputeDefragmentation_Fast(vector);
13712	}
13713
13714	// No free blocks, have to clear last one
13715	size_t last = (vectorState.firstFreeBlock == SIZE_MAX ? vector.GetBlockCount() : vectorState.firstFreeBlock) - 1;
13716	VmaBlockMetadata* freeMetadata = vector.GetBlock(index: last)->m_pMetadata;
13717
13718	const size_t prevMoveCount = m_Moves.size();
13719	for (VmaAllocHandle handle = freeMetadata->GetAllocationListBegin();
13720	handle != VK_NULL_HANDLE;
13721	handle = freeMetadata->GetNextAllocation(prevAlloc: handle))
13722	{
13723	MoveAllocationData moveData = GetMoveData(handle, metadata: freeMetadata);
13724	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13725	{
13726	case CounterStatus::Ignore:
13727	continue;
13728	case CounterStatus::End:
13729	return true;
13730	default:
13731	VMA_ASSERT(0);
13732	case CounterStatus::Pass:
13733	break;
13734	}
13735
13736	// Check all previous blocks for free space
13737	if (AllocInOtherBlock(start: 0, end: last, data&: moveData, vector))
13738	{
13739	// Full clear performed already
13740	if (prevMoveCount != m_Moves.size() && freeMetadata->GetNextAllocation(prevAlloc: handle) == VK_NULL_HANDLE)
13741	reinterpret_cast<size_t*>(m_AlgorithmState)[index] = last;
13742	return true;
13743	}
13744	}
13745
13746	if (prevMoveCount == m_Moves.size())
13747	{
13748	// Cannot perform full clear, have to move data in other blocks around
13749	if (last != 0)
13750	{
13751	for (size_t i = last - 1; i; --i)
13752	{
13753	if (ReallocWithinBlock(vector, block: vector.GetBlock(index: i)))
13754	return true;
13755	}
13756	}
13757
13758	if (prevMoveCount == m_Moves.size())
13759	{
13760	// No possible reallocs within blocks, try to move them around fast
13761	return ComputeDefragmentation_Fast(vector);
13762	}
13763	}
13764	else
13765	{
13766	switch (vectorState.operation)
13767	{
13768	case StateExtensive::Operation::FindFreeBlockBuffer:
13769	vectorState.operation = StateExtensive::Operation::MoveBuffers;
13770	break;
13771	default:
13772	VMA_ASSERT(0);
13773	case StateExtensive::Operation::FindFreeBlockTexture:
13774	vectorState.operation = StateExtensive::Operation::MoveTextures;
13775	break;
13776	case StateExtensive::Operation::FindFreeBlockAll:
13777	vectorState.operation = StateExtensive::Operation::MoveAll;
13778	break;
13779	}
13780	vectorState.firstFreeBlock = last;
13781	// Nothing done, block found without reallocations, can perform another reallocs in same pass
13782	return ComputeDefragmentation_Extensive(vector, index);
13783	}
13784	break;
13785	}
13786	case StateExtensive::Operation::MoveTextures:
13787	{
13788	if (MoveDataToFreeBlocks(currentType: VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL, vector,
13789	firstFreeBlock: vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13790	{
13791	if (texturePresent)
13792	{
13793	vectorState.operation = StateExtensive::Operation::FindFreeBlockTexture;
13794	return ComputeDefragmentation_Extensive(vector, index);
13795	}
13796
13797	if (!bufferPresent && !otherPresent)
13798	{
13799	vectorState.operation = StateExtensive::Operation::Cleanup;
13800	break;
13801	}
13802
13803	// No more textures to move, check buffers
13804	vectorState.operation = StateExtensive::Operation::MoveBuffers;
13805	bufferPresent = false;
13806	otherPresent = false;
13807	}
13808	else
13809	break;
13810	}
13811	case StateExtensive::Operation::MoveBuffers:
13812	{
13813	if (MoveDataToFreeBlocks(currentType: VMA_SUBALLOCATION_TYPE_BUFFER, vector,
13814	firstFreeBlock: vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13815	{
13816	if (bufferPresent)
13817	{
13818	vectorState.operation = StateExtensive::Operation::FindFreeBlockBuffer;
13819	return ComputeDefragmentation_Extensive(vector, index);
13820	}
13821
13822	if (!otherPresent)
13823	{
13824	vectorState.operation = StateExtensive::Operation::Cleanup;
13825	break;
13826	}
13827
13828	// No more buffers to move, check all others
13829	vectorState.operation = StateExtensive::Operation::MoveAll;
13830	otherPresent = false;
13831	}
13832	else
13833	break;
13834	}
13835	case StateExtensive::Operation::MoveAll:
13836	{
13837	if (MoveDataToFreeBlocks(currentType: VMA_SUBALLOCATION_TYPE_FREE, vector,
13838	firstFreeBlock: vectorState.firstFreeBlock, texturePresent, bufferPresent, otherPresent))
13839	{
13840	if (otherPresent)
13841	{
13842	vectorState.operation = StateExtensive::Operation::FindFreeBlockBuffer;
13843	return ComputeDefragmentation_Extensive(vector, index);
13844	}
13845	// Everything moved
13846	vectorState.operation = StateExtensive::Operation::Cleanup;
13847	}
13848	break;
13849	}
13850	case StateExtensive::Operation::Cleanup:
13851	// Cleanup is handled below so that other operations may reuse the cleanup code. This case is here to prevent the unhandled enum value warning (C4062).
13852	break;
13853	}
13854
13855	if (vectorState.operation == StateExtensive::Operation::Cleanup)
13856	{
13857	// All other work done, pack data in blocks even tighter if possible
13858	const size_t prevMoveCount = m_Moves.size();
13859	for (size_t i = 0; i < vector.GetBlockCount(); ++i)
13860	{
13861	if (ReallocWithinBlock(vector, block: vector.GetBlock(index: i)))
13862	return true;
13863	}
13864
13865	if (prevMoveCount == m_Moves.size())
13866	vectorState.operation = StateExtensive::Operation::Done;
13867	}
13868	return false;
13869	}
13870
13871	void VmaDefragmentationContext_T::UpdateVectorStatistics(VmaBlockVector& vector, StateBalanced& state)
13872	{
13873	size_t allocCount = 0;
13874	size_t freeCount = 0;
13875	state.avgFreeSize = 0;
13876	state.avgAllocSize = 0;
13877
13878	for (size_t i = 0; i < vector.GetBlockCount(); ++i)
13879	{
13880	VmaBlockMetadata* metadata = vector.GetBlock(index: i)->m_pMetadata;
13881
13882	allocCount += metadata->GetAllocationCount();
13883	freeCount += metadata->GetFreeRegionsCount();
13884	state.avgFreeSize += metadata->GetSumFreeSize();
13885	state.avgAllocSize += metadata->GetSize();
13886	}
13887
13888	state.avgAllocSize = (state.avgAllocSize - state.avgFreeSize) / allocCount;
13889	state.avgFreeSize /= freeCount;
13890	}
13891
13892	bool VmaDefragmentationContext_T::MoveDataToFreeBlocks(VmaSuballocationType currentType,
13893	VmaBlockVector& vector, size_t firstFreeBlock,
13894	bool& texturePresent, bool& bufferPresent, bool& otherPresent)
13895	{
13896	const size_t prevMoveCount = m_Moves.size();
13897	for (size_t i = firstFreeBlock ; i;)
13898	{
13899	VmaDeviceMemoryBlock* block = vector.GetBlock(index: --i);
13900	VmaBlockMetadata* metadata = block->m_pMetadata;
13901
13902	for (VmaAllocHandle handle = metadata->GetAllocationListBegin();
13903	handle != VK_NULL_HANDLE;
13904	handle = metadata->GetNextAllocation(prevAlloc: handle))
13905	{
13906	MoveAllocationData moveData = GetMoveData(handle, metadata);
13907	// Ignore newly created allocations by defragmentation algorithm
13908	if (moveData.move.srcAllocation->GetUserData() == this)
13909	continue;
13910	switch (CheckCounters(bytes: moveData.move.srcAllocation->GetSize()))
13911	{
13912	case CounterStatus::Ignore:
13913	continue;
13914	case CounterStatus::End:
13915	return true;
13916	default:
13917	VMA_ASSERT(0);
13918	case CounterStatus::Pass:
13919	break;
13920	}
13921
13922	// Move only single type of resources at once
13923	if (!VmaIsBufferImageGranularityConflict(suballocType1: moveData.type, suballocType2: currentType))
13924	{
13925	// Try to fit allocation into free blocks
13926	if (AllocInOtherBlock(start: firstFreeBlock, end: vector.GetBlockCount(), data&: moveData, vector))
13927	return false;
13928	}
13929
13930	if (!VmaIsBufferImageGranularityConflict(suballocType1: moveData.type, suballocType2: VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL))
13931	texturePresent = true;
13932	else if (!VmaIsBufferImageGranularityConflict(suballocType1: moveData.type, suballocType2: VMA_SUBALLOCATION_TYPE_BUFFER))
13933	bufferPresent = true;
13934	else
13935	otherPresent = true;
13936	}
13937	}
13938	return prevMoveCount == m_Moves.size();
13939	}
13940	#endif // _VMA_DEFRAGMENTATION_CONTEXT_FUNCTIONS
13941
13942	#ifndef _VMA_POOL_T_FUNCTIONS
13943	VmaPool_T::VmaPool_T(
13944	VmaAllocator hAllocator,
13945	const VmaPoolCreateInfo& createInfo,
13946	VkDeviceSize preferredBlockSize)
13947	: m_BlockVector(
13948	hAllocator,
13949	this, // hParentPool
13950	createInfo.memoryTypeIndex,
13951	createInfo.blockSize != 0 ? createInfo.blockSize : preferredBlockSize,
13952	createInfo.minBlockCount,
13953	createInfo.maxBlockCount,
13954	(createInfo.flags& VMA_POOL_CREATE_IGNORE_BUFFER_IMAGE_GRANULARITY_BIT) != 0 ? 1 : hAllocator->GetBufferImageGranularity(),
13955	createInfo.blockSize != 0, // explicitBlockSize
13956	createInfo.flags & VMA_POOL_CREATE_ALGORITHM_MASK, // algorithm
13957	createInfo.priority,
13958	VMA_MAX(hAllocator->GetMemoryTypeMinAlignment(createInfo.memoryTypeIndex), createInfo.minAllocationAlignment),
13959	createInfo.pMemoryAllocateNext),
13960	m_Id(0),
13961	m_Name(VMA_NULL) {}
13962
13963	VmaPool_T::~VmaPool_T()
13964	{
13965	VMA_ASSERT(m_PrevPool == VMA_NULL && m_NextPool == VMA_NULL);
13966	}
13967
13968	void VmaPool_T::SetName(const char* pName)
13969	{
13970	const VkAllocationCallbacks* allocs = m_BlockVector.GetAllocator()->GetAllocationCallbacks();
13971	VmaFreeString(allocs, str: m_Name);
13972
13973	if (pName != VMA_NULL)
13974	{
13975	m_Name = VmaCreateStringCopy(allocs, srcStr: pName);
13976	}
13977	else
13978	{
13979	m_Name = VMA_NULL;
13980	}
13981	}
13982	#endif // _VMA_POOL_T_FUNCTIONS
13983
13984	#ifndef _VMA_ALLOCATOR_T_FUNCTIONS
13985	VmaAllocator_T::VmaAllocator_T(const VmaAllocatorCreateInfo* pCreateInfo) :
13986	m_UseMutex((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT) == 0),
13987	m_VulkanApiVersion(pCreateInfo->vulkanApiVersion != 0 ? pCreateInfo->vulkanApiVersion : VK_API_VERSION_1_0),
13988	m_UseKhrDedicatedAllocation((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0),
13989	m_UseKhrBindMemory2((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT) != 0),
13990	m_UseExtMemoryBudget((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT) != 0),
13991	m_UseAmdDeviceCoherentMemory((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT) != 0),
13992	m_UseKhrBufferDeviceAddress((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT) != 0),
13993	m_UseExtMemoryPriority((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT) != 0),
13994	m_hDevice(pCreateInfo->device),
13995	m_hInstance(pCreateInfo->instance),
13996	m_AllocationCallbacksSpecified(pCreateInfo->pAllocationCallbacks != VMA_NULL),
13997	m_AllocationCallbacks(pCreateInfo->pAllocationCallbacks ?
13998	*pCreateInfo->pAllocationCallbacks : VmaEmptyAllocationCallbacks),
13999	m_AllocationObjectAllocator(&m_AllocationCallbacks),
14000	m_HeapSizeLimitMask(0),
14001	m_DeviceMemoryCount(0),
14002	m_PreferredLargeHeapBlockSize(0),
14003	m_PhysicalDevice(pCreateInfo->physicalDevice),
14004	m_GpuDefragmentationMemoryTypeBits(UINT32_MAX),
14005	m_NextPoolId(0),
14006	m_GlobalMemoryTypeBits(UINT32_MAX)
14007	{
14008	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14009	{
14010	m_UseKhrDedicatedAllocation = false;
14011	m_UseKhrBindMemory2 = false;
14012	}
14013
14014	if(VMA_DEBUG_DETECT_CORRUPTION)
14015	{
14016	// Needs to be multiply of uint32_t size because we are going to write VMA_CORRUPTION_DETECTION_MAGIC_VALUE to it.
14017	VMA_ASSERT(VMA_DEBUG_MARGIN % sizeof(uint32_t) == 0);
14018	}
14019
14020	VMA_ASSERT(pCreateInfo->physicalDevice && pCreateInfo->device && pCreateInfo->instance);
14021
14022	if(m_VulkanApiVersion < VK_MAKE_VERSION(1, 1, 0))
14023	{
14024	#if !(VMA_DEDICATED_ALLOCATION)
14025	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT) != 0)
14026	{
14027	VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT set but required extensions are disabled by preprocessor macros.");
14028	}
14029	#endif
14030	#if !(VMA_BIND_MEMORY2)
14031	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT) != 0)
14032	{
14033	VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_KHR_BIND_MEMORY2_BIT set but required extension is disabled by preprocessor macros.");
14034	}
14035	#endif
14036	}
14037	#if !(VMA_MEMORY_BUDGET)
14038	if((pCreateInfo->flags & VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT) != 0)
14039	{
14040	VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT set but required extension is disabled by preprocessor macros.");
14041	}
14042	#endif
14043	#if !(VMA_BUFFER_DEVICE_ADDRESS)
14044	if(m_UseKhrBufferDeviceAddress)
14045	{
14046	VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT is set but required extension or Vulkan 1.2 is not available in your Vulkan header or its support in VMA has been disabled by a preprocessor macro.");
14047	}
14048	#endif
14049	#if VMA_VULKAN_VERSION < 1002000
14050	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 2, 0))
14051	{
14052	VMA_ASSERT(0 && "vulkanApiVersion >= VK_API_VERSION_1_2 but required Vulkan version is disabled by preprocessor macros.");
14053	}
14054	#endif
14055	#if VMA_VULKAN_VERSION < 1001000
14056	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14057	{
14058	VMA_ASSERT(0 && "vulkanApiVersion >= VK_API_VERSION_1_1 but required Vulkan version is disabled by preprocessor macros.");
14059	}
14060	#endif
14061	#if !(VMA_MEMORY_PRIORITY)
14062	if(m_UseExtMemoryPriority)
14063	{
14064	VMA_ASSERT(0 && "VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT is set but required extension is not available in your Vulkan header or its support in VMA has been disabled by a preprocessor macro.");
14065	}
14066	#endif
14067
14068	memset(s: &m_DeviceMemoryCallbacks, c: 0 ,n: sizeof(m_DeviceMemoryCallbacks));
14069	memset(s: &m_PhysicalDeviceProperties, c: 0, n: sizeof(m_PhysicalDeviceProperties));
14070	memset(s: &m_MemProps, c: 0, n: sizeof(m_MemProps));
14071
14072	memset(s: &m_pBlockVectors, c: 0, n: sizeof(m_pBlockVectors));
14073	memset(s: &m_VulkanFunctions, c: 0, n: sizeof(m_VulkanFunctions));
14074
14075	#if VMA_EXTERNAL_MEMORY
14076	memset(s: &m_TypeExternalMemoryHandleTypes, c: 0, n: sizeof(m_TypeExternalMemoryHandleTypes));
14077	#endif // #if VMA_EXTERNAL_MEMORY
14078
14079	if(pCreateInfo->pDeviceMemoryCallbacks != VMA_NULL)
14080	{
14081	m_DeviceMemoryCallbacks.pUserData = pCreateInfo->pDeviceMemoryCallbacks->pUserData;
14082	m_DeviceMemoryCallbacks.pfnAllocate = pCreateInfo->pDeviceMemoryCallbacks->pfnAllocate;
14083	m_DeviceMemoryCallbacks.pfnFree = pCreateInfo->pDeviceMemoryCallbacks->pfnFree;
14084	}
14085
14086	ImportVulkanFunctions(pVulkanFunctions: pCreateInfo->pVulkanFunctions);
14087
14088	(*m_VulkanFunctions.vkGetPhysicalDeviceProperties)(m_PhysicalDevice, &m_PhysicalDeviceProperties);
14089	(*m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties)(m_PhysicalDevice, &m_MemProps);
14090
14091	VMA_ASSERT(VmaIsPow2(VMA_MIN_ALIGNMENT));
14092	VMA_ASSERT(VmaIsPow2(VMA_DEBUG_MIN_BUFFER_IMAGE_GRANULARITY));
14093	VMA_ASSERT(VmaIsPow2(m_PhysicalDeviceProperties.limits.bufferImageGranularity));
14094	VMA_ASSERT(VmaIsPow2(m_PhysicalDeviceProperties.limits.nonCoherentAtomSize));
14095
14096	m_PreferredLargeHeapBlockSize = (pCreateInfo->preferredLargeHeapBlockSize != 0) ?
14097	pCreateInfo->preferredLargeHeapBlockSize : static_cast<VkDeviceSize>(VMA_DEFAULT_LARGE_HEAP_BLOCK_SIZE);
14098
14099	m_GlobalMemoryTypeBits = CalculateGlobalMemoryTypeBits();
14100
14101	#if VMA_EXTERNAL_MEMORY
14102	if(pCreateInfo->pTypeExternalMemoryHandleTypes != VMA_NULL)
14103	{
14104	memcpy(dest: m_TypeExternalMemoryHandleTypes, src: pCreateInfo->pTypeExternalMemoryHandleTypes,
14105	n: sizeof(VkExternalMemoryHandleTypeFlagsKHR) * GetMemoryTypeCount());
14106	}
14107	#endif // #if VMA_EXTERNAL_MEMORY
14108
14109	if(pCreateInfo->pHeapSizeLimit != VMA_NULL)
14110	{
14111	for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
14112	{
14113	const VkDeviceSize limit = pCreateInfo->pHeapSizeLimit[heapIndex];
14114	if(limit != VK_WHOLE_SIZE)
14115	{
14116	m_HeapSizeLimitMask |= 1u << heapIndex;
14117	if(limit < m_MemProps.memoryHeaps[heapIndex].size)
14118	{
14119	m_MemProps.memoryHeaps[heapIndex].size = limit;
14120	}
14121	}
14122	}
14123	}
14124
14125	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
14126	{
14127	// Create only supported types
14128	if((m_GlobalMemoryTypeBits & (1u << memTypeIndex)) != 0)
14129	{
14130	const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex);
14131	m_pBlockVectors[memTypeIndex] = vma_new(this, VmaBlockVector)(
14132	this,
14133	VK_NULL_HANDLE, // hParentPool
14134	memTypeIndex,
14135	preferredBlockSize,
14136	0,
14137	SIZE_MAX,
14138	GetBufferImageGranularity(),
14139	false, // explicitBlockSize
14140	0, // algorithm
14141	0.5f, // priority (0.5 is the default per Vulkan spec)
14142	GetMemoryTypeMinAlignment(memTypeIndex), // minAllocationAlignment
14143	VMA_NULL); // // pMemoryAllocateNext
14144	// No need to call m_pBlockVectors[memTypeIndex][blockVectorTypeIndex]->CreateMinBlocks here,
14145	// becase minBlockCount is 0.
14146	}
14147	}
14148	}
14149
14150	VkResult VmaAllocator_T::Init(const VmaAllocatorCreateInfo* pCreateInfo)
14151	{
14152	VkResult res = VK_SUCCESS;
14153
14154	#if VMA_MEMORY_BUDGET
14155	if(m_UseExtMemoryBudget)
14156	{
14157	UpdateVulkanBudget();
14158	}
14159	#endif // #if VMA_MEMORY_BUDGET
14160
14161	return res;
14162	}
14163
14164	VmaAllocator_T::~VmaAllocator_T()
14165	{
14166	VMA_ASSERT(m_Pools.IsEmpty());
14167
14168	for(size_t memTypeIndex = GetMemoryTypeCount(); memTypeIndex--; )
14169	{
14170	vma_delete(hAllocator: this, ptr: m_pBlockVectors[memTypeIndex]);
14171	}
14172	}
14173
14174	void VmaAllocator_T::ImportVulkanFunctions(const VmaVulkanFunctions* pVulkanFunctions)
14175	{
14176	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
14177	ImportVulkanFunctions_Static();
14178	#endif
14179
14180	if(pVulkanFunctions != VMA_NULL)
14181	{
14182	ImportVulkanFunctions_Custom(pVulkanFunctions);
14183	}
14184
14185	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14186	ImportVulkanFunctions_Dynamic();
14187	#endif
14188
14189	ValidateVulkanFunctions();
14190	}
14191
14192	#if VMA_STATIC_VULKAN_FUNCTIONS == 1
14193
14194	void VmaAllocator_T::ImportVulkanFunctions_Static()
14195	{
14196	// Vulkan 1.0
14197	m_VulkanFunctions.vkGetInstanceProcAddr = (PFN_vkGetInstanceProcAddr)vkGetInstanceProcAddr;
14198	m_VulkanFunctions.vkGetDeviceProcAddr = (PFN_vkGetDeviceProcAddr)vkGetDeviceProcAddr;
14199	m_VulkanFunctions.vkGetPhysicalDeviceProperties = (PFN_vkGetPhysicalDeviceProperties)vkGetPhysicalDeviceProperties;
14200	m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties = (PFN_vkGetPhysicalDeviceMemoryProperties)vkGetPhysicalDeviceMemoryProperties;
14201	m_VulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkAllocateMemory;
14202	m_VulkanFunctions.vkFreeMemory = (PFN_vkFreeMemory)vkFreeMemory;
14203	m_VulkanFunctions.vkMapMemory = (PFN_vkMapMemory)vkMapMemory;
14204	m_VulkanFunctions.vkUnmapMemory = (PFN_vkUnmapMemory)vkUnmapMemory;
14205	m_VulkanFunctions.vkFlushMappedMemoryRanges = (PFN_vkFlushMappedMemoryRanges)vkFlushMappedMemoryRanges;
14206	m_VulkanFunctions.vkInvalidateMappedMemoryRanges = (PFN_vkInvalidateMappedMemoryRanges)vkInvalidateMappedMemoryRanges;
14207	m_VulkanFunctions.vkBindBufferMemory = (PFN_vkBindBufferMemory)vkBindBufferMemory;
14208	m_VulkanFunctions.vkBindImageMemory = (PFN_vkBindImageMemory)vkBindImageMemory;
14209	m_VulkanFunctions.vkGetBufferMemoryRequirements = (PFN_vkGetBufferMemoryRequirements)vkGetBufferMemoryRequirements;
14210	m_VulkanFunctions.vkGetImageMemoryRequirements = (PFN_vkGetImageMemoryRequirements)vkGetImageMemoryRequirements;
14211	m_VulkanFunctions.vkCreateBuffer = (PFN_vkCreateBuffer)vkCreateBuffer;
14212	m_VulkanFunctions.vkDestroyBuffer = (PFN_vkDestroyBuffer)vkDestroyBuffer;
14213	m_VulkanFunctions.vkCreateImage = (PFN_vkCreateImage)vkCreateImage;
14214	m_VulkanFunctions.vkDestroyImage = (PFN_vkDestroyImage)vkDestroyImage;
14215	m_VulkanFunctions.vkCmdCopyBuffer = (PFN_vkCmdCopyBuffer)vkCmdCopyBuffer;
14216
14217	// Vulkan 1.1
14218	#if VMA_VULKAN_VERSION >= 1001000
14219	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14220	{
14221	m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR = (PFN_vkGetBufferMemoryRequirements2)vkGetBufferMemoryRequirements2;
14222	m_VulkanFunctions.vkGetImageMemoryRequirements2KHR = (PFN_vkGetImageMemoryRequirements2)vkGetImageMemoryRequirements2;
14223	m_VulkanFunctions.vkBindBufferMemory2KHR = (PFN_vkBindBufferMemory2)vkBindBufferMemory2;
14224	m_VulkanFunctions.vkBindImageMemory2KHR = (PFN_vkBindImageMemory2)vkBindImageMemory2;
14225	m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties2KHR = (PFN_vkGetPhysicalDeviceMemoryProperties2)vkGetPhysicalDeviceMemoryProperties2;
14226	}
14227	#endif
14228
14229	#if VMA_VULKAN_VERSION >= 1003000
14230	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 3, 0))
14231	{
14232	m_VulkanFunctions.vkGetDeviceBufferMemoryRequirements = (PFN_vkGetDeviceBufferMemoryRequirements)vkGetDeviceBufferMemoryRequirements;
14233	m_VulkanFunctions.vkGetDeviceImageMemoryRequirements = (PFN_vkGetDeviceImageMemoryRequirements)vkGetDeviceImageMemoryRequirements;
14234	}
14235	#endif
14236	}
14237
14238	#endif // VMA_STATIC_VULKAN_FUNCTIONS == 1
14239
14240	void VmaAllocator_T::ImportVulkanFunctions_Custom(const VmaVulkanFunctions* pVulkanFunctions)
14241	{
14242	VMA_ASSERT(pVulkanFunctions != VMA_NULL);
14243
14244	#define VMA_COPY_IF_NOT_NULL(funcName) \
14245	if(pVulkanFunctions->funcName != VMA_NULL) m_VulkanFunctions.funcName = pVulkanFunctions->funcName;
14246
14247	VMA_COPY_IF_NOT_NULL(vkGetInstanceProcAddr);
14248	VMA_COPY_IF_NOT_NULL(vkGetDeviceProcAddr);
14249	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceProperties);
14250	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties);
14251	VMA_COPY_IF_NOT_NULL(vkAllocateMemory);
14252	VMA_COPY_IF_NOT_NULL(vkFreeMemory);
14253	VMA_COPY_IF_NOT_NULL(vkMapMemory);
14254	VMA_COPY_IF_NOT_NULL(vkUnmapMemory);
14255	VMA_COPY_IF_NOT_NULL(vkFlushMappedMemoryRanges);
14256	VMA_COPY_IF_NOT_NULL(vkInvalidateMappedMemoryRanges);
14257	VMA_COPY_IF_NOT_NULL(vkBindBufferMemory);
14258	VMA_COPY_IF_NOT_NULL(vkBindImageMemory);
14259	VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements);
14260	VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements);
14261	VMA_COPY_IF_NOT_NULL(vkCreateBuffer);
14262	VMA_COPY_IF_NOT_NULL(vkDestroyBuffer);
14263	VMA_COPY_IF_NOT_NULL(vkCreateImage);
14264	VMA_COPY_IF_NOT_NULL(vkDestroyImage);
14265	VMA_COPY_IF_NOT_NULL(vkCmdCopyBuffer);
14266
14267	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14268	VMA_COPY_IF_NOT_NULL(vkGetBufferMemoryRequirements2KHR);
14269	VMA_COPY_IF_NOT_NULL(vkGetImageMemoryRequirements2KHR);
14270	#endif
14271
14272	#if VMA_BIND_MEMORY2 || VMA_VULKAN_VERSION >= 1001000
14273	VMA_COPY_IF_NOT_NULL(vkBindBufferMemory2KHR);
14274	VMA_COPY_IF_NOT_NULL(vkBindImageMemory2KHR);
14275	#endif
14276
14277	#if VMA_MEMORY_BUDGET
14278	VMA_COPY_IF_NOT_NULL(vkGetPhysicalDeviceMemoryProperties2KHR);
14279	#endif
14280
14281	#if VMA_VULKAN_VERSION >= 1003000
14282	VMA_COPY_IF_NOT_NULL(vkGetDeviceBufferMemoryRequirements);
14283	VMA_COPY_IF_NOT_NULL(vkGetDeviceImageMemoryRequirements);
14284	#endif
14285
14286	#undef VMA_COPY_IF_NOT_NULL
14287	}
14288
14289	#if VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14290
14291	void VmaAllocator_T::ImportVulkanFunctions_Dynamic()
14292	{
14293	VMA_ASSERT(m_VulkanFunctions.vkGetInstanceProcAddr && m_VulkanFunctions.vkGetDeviceProcAddr &&
14294	"To use VMA_DYNAMIC_VULKAN_FUNCTIONS in new versions of VMA you now have to pass "
14295	"VmaVulkanFunctions::vkGetInstanceProcAddr and vkGetDeviceProcAddr as VmaAllocatorCreateInfo::pVulkanFunctions. "
14296	"Other members can be null.");
14297
14298	#define VMA_FETCH_INSTANCE_FUNC(memberName, functionPointerType, functionNameString) \
14299	if(m_VulkanFunctions.memberName == VMA_NULL) \
14300	m_VulkanFunctions.memberName = \
14301	(functionPointerType)m_VulkanFunctions.vkGetInstanceProcAddr(m_hInstance, functionNameString);
14302	#define VMA_FETCH_DEVICE_FUNC(memberName, functionPointerType, functionNameString) \
14303	if(m_VulkanFunctions.memberName == VMA_NULL) \
14304	m_VulkanFunctions.memberName = \
14305	(functionPointerType)m_VulkanFunctions.vkGetDeviceProcAddr(m_hDevice, functionNameString);
14306
14307	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceProperties, PFN_vkGetPhysicalDeviceProperties, "vkGetPhysicalDeviceProperties");
14308	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties, PFN_vkGetPhysicalDeviceMemoryProperties, "vkGetPhysicalDeviceMemoryProperties");
14309	VMA_FETCH_DEVICE_FUNC(vkAllocateMemory, PFN_vkAllocateMemory, "vkAllocateMemory");
14310	VMA_FETCH_DEVICE_FUNC(vkFreeMemory, PFN_vkFreeMemory, "vkFreeMemory");
14311	VMA_FETCH_DEVICE_FUNC(vkMapMemory, PFN_vkMapMemory, "vkMapMemory");
14312	VMA_FETCH_DEVICE_FUNC(vkUnmapMemory, PFN_vkUnmapMemory, "vkUnmapMemory");
14313	VMA_FETCH_DEVICE_FUNC(vkFlushMappedMemoryRanges, PFN_vkFlushMappedMemoryRanges, "vkFlushMappedMemoryRanges");
14314	VMA_FETCH_DEVICE_FUNC(vkInvalidateMappedMemoryRanges, PFN_vkInvalidateMappedMemoryRanges, "vkInvalidateMappedMemoryRanges");
14315	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory, PFN_vkBindBufferMemory, "vkBindBufferMemory");
14316	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory, PFN_vkBindImageMemory, "vkBindImageMemory");
14317	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements, PFN_vkGetBufferMemoryRequirements, "vkGetBufferMemoryRequirements");
14318	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements, PFN_vkGetImageMemoryRequirements, "vkGetImageMemoryRequirements");
14319	VMA_FETCH_DEVICE_FUNC(vkCreateBuffer, PFN_vkCreateBuffer, "vkCreateBuffer");
14320	VMA_FETCH_DEVICE_FUNC(vkDestroyBuffer, PFN_vkDestroyBuffer, "vkDestroyBuffer");
14321	VMA_FETCH_DEVICE_FUNC(vkCreateImage, PFN_vkCreateImage, "vkCreateImage");
14322	VMA_FETCH_DEVICE_FUNC(vkDestroyImage, PFN_vkDestroyImage, "vkDestroyImage");
14323	VMA_FETCH_DEVICE_FUNC(vkCmdCopyBuffer, PFN_vkCmdCopyBuffer, "vkCmdCopyBuffer");
14324
14325	#if VMA_VULKAN_VERSION >= 1001000
14326	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14327	{
14328	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements2KHR, PFN_vkGetBufferMemoryRequirements2, "vkGetBufferMemoryRequirements2");
14329	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements2KHR, PFN_vkGetImageMemoryRequirements2, "vkGetImageMemoryRequirements2");
14330	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory2KHR, PFN_vkBindBufferMemory2, "vkBindBufferMemory2");
14331	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory2KHR, PFN_vkBindImageMemory2, "vkBindImageMemory2");
14332	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties2KHR, PFN_vkGetPhysicalDeviceMemoryProperties2, "vkGetPhysicalDeviceMemoryProperties2");
14333	}
14334	#endif
14335
14336	#if VMA_DEDICATED_ALLOCATION
14337	if(m_UseKhrDedicatedAllocation)
14338	{
14339	VMA_FETCH_DEVICE_FUNC(vkGetBufferMemoryRequirements2KHR, PFN_vkGetBufferMemoryRequirements2KHR, "vkGetBufferMemoryRequirements2KHR");
14340	VMA_FETCH_DEVICE_FUNC(vkGetImageMemoryRequirements2KHR, PFN_vkGetImageMemoryRequirements2KHR, "vkGetImageMemoryRequirements2KHR");
14341	}
14342	#endif
14343
14344	#if VMA_BIND_MEMORY2
14345	if(m_UseKhrBindMemory2)
14346	{
14347	VMA_FETCH_DEVICE_FUNC(vkBindBufferMemory2KHR, PFN_vkBindBufferMemory2KHR, "vkBindBufferMemory2KHR");
14348	VMA_FETCH_DEVICE_FUNC(vkBindImageMemory2KHR, PFN_vkBindImageMemory2KHR, "vkBindImageMemory2KHR");
14349	}
14350	#endif // #if VMA_BIND_MEMORY2
14351
14352	#if VMA_MEMORY_BUDGET
14353	if(m_UseExtMemoryBudget)
14354	{
14355	VMA_FETCH_INSTANCE_FUNC(vkGetPhysicalDeviceMemoryProperties2KHR, PFN_vkGetPhysicalDeviceMemoryProperties2KHR, "vkGetPhysicalDeviceMemoryProperties2KHR");
14356	}
14357	#endif // #if VMA_MEMORY_BUDGET
14358
14359	#if VMA_VULKAN_VERSION >= 1003000
14360	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 3, 0))
14361	{
14362	VMA_FETCH_DEVICE_FUNC(vkGetDeviceBufferMemoryRequirements, PFN_vkGetDeviceBufferMemoryRequirements, "vkGetDeviceBufferMemoryRequirements");
14363	VMA_FETCH_DEVICE_FUNC(vkGetDeviceImageMemoryRequirements, PFN_vkGetDeviceImageMemoryRequirements, "vkGetDeviceImageMemoryRequirements");
14364	}
14365	#endif
14366
14367	#undef VMA_FETCH_DEVICE_FUNC
14368	#undef VMA_FETCH_INSTANCE_FUNC
14369	}
14370
14371	#endif // VMA_DYNAMIC_VULKAN_FUNCTIONS == 1
14372
14373	void VmaAllocator_T::ValidateVulkanFunctions()
14374	{
14375	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceProperties != VMA_NULL);
14376	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties != VMA_NULL);
14377	VMA_ASSERT(m_VulkanFunctions.vkAllocateMemory != VMA_NULL);
14378	VMA_ASSERT(m_VulkanFunctions.vkFreeMemory != VMA_NULL);
14379	VMA_ASSERT(m_VulkanFunctions.vkMapMemory != VMA_NULL);
14380	VMA_ASSERT(m_VulkanFunctions.vkUnmapMemory != VMA_NULL);
14381	VMA_ASSERT(m_VulkanFunctions.vkFlushMappedMemoryRanges != VMA_NULL);
14382	VMA_ASSERT(m_VulkanFunctions.vkInvalidateMappedMemoryRanges != VMA_NULL);
14383	VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory != VMA_NULL);
14384	VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory != VMA_NULL);
14385	VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements != VMA_NULL);
14386	VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements != VMA_NULL);
14387	VMA_ASSERT(m_VulkanFunctions.vkCreateBuffer != VMA_NULL);
14388	VMA_ASSERT(m_VulkanFunctions.vkDestroyBuffer != VMA_NULL);
14389	VMA_ASSERT(m_VulkanFunctions.vkCreateImage != VMA_NULL);
14390	VMA_ASSERT(m_VulkanFunctions.vkDestroyImage != VMA_NULL);
14391	VMA_ASSERT(m_VulkanFunctions.vkCmdCopyBuffer != VMA_NULL);
14392
14393	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14394	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0) || m_UseKhrDedicatedAllocation)
14395	{
14396	VMA_ASSERT(m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR != VMA_NULL);
14397	VMA_ASSERT(m_VulkanFunctions.vkGetImageMemoryRequirements2KHR != VMA_NULL);
14398	}
14399	#endif
14400
14401	#if VMA_BIND_MEMORY2 || VMA_VULKAN_VERSION >= 1001000
14402	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0) || m_UseKhrBindMemory2)
14403	{
14404	VMA_ASSERT(m_VulkanFunctions.vkBindBufferMemory2KHR != VMA_NULL);
14405	VMA_ASSERT(m_VulkanFunctions.vkBindImageMemory2KHR != VMA_NULL);
14406	}
14407	#endif
14408
14409	#if VMA_MEMORY_BUDGET || VMA_VULKAN_VERSION >= 1001000
14410	if(m_UseExtMemoryBudget || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14411	{
14412	VMA_ASSERT(m_VulkanFunctions.vkGetPhysicalDeviceMemoryProperties2KHR != VMA_NULL);
14413	}
14414	#endif
14415
14416	#if VMA_VULKAN_VERSION >= 1003000
14417	if(m_VulkanApiVersion >= VK_MAKE_VERSION(1, 3, 0))
14418	{
14419	VMA_ASSERT(m_VulkanFunctions.vkGetDeviceBufferMemoryRequirements != VMA_NULL);
14420	VMA_ASSERT(m_VulkanFunctions.vkGetDeviceImageMemoryRequirements != VMA_NULL);
14421	}
14422	#endif
14423	}
14424
14425	VkDeviceSize VmaAllocator_T::CalcPreferredBlockSize(uint32_t memTypeIndex)
14426	{
14427	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
14428	const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
14429	const bool isSmallHeap = heapSize <= VMA_SMALL_HEAP_MAX_SIZE;
14430	return VmaAlignUp(val: isSmallHeap ? (heapSize / 8) : m_PreferredLargeHeapBlockSize, alignment: (VkDeviceSize)32);
14431	}
14432
14433	VkResult VmaAllocator_T::AllocateMemoryOfType(
14434	VmaPool pool,
14435	VkDeviceSize size,
14436	VkDeviceSize alignment,
14437	bool dedicatedPreferred,
14438	VkBuffer dedicatedBuffer,
14439	VkImage dedicatedImage,
14440	VkFlags dedicatedBufferImageUsage,
14441	const VmaAllocationCreateInfo& createInfo,
14442	uint32_t memTypeIndex,
14443	VmaSuballocationType suballocType,
14444	VmaDedicatedAllocationList& dedicatedAllocations,
14445	VmaBlockVector& blockVector,
14446	size_t allocationCount,
14447	VmaAllocation* pAllocations)
14448	{
14449	VMA_ASSERT(pAllocations != VMA_NULL);
14450	VMA_DEBUG_LOG(" AllocateMemory: MemoryTypeIndex=%u, AllocationCount=%zu, Size=%llu", memTypeIndex, allocationCount, size);
14451
14452	VmaAllocationCreateInfo finalCreateInfo = createInfo;
14453	VkResult res = CalcMemTypeParams(
14454	outCreateInfo&: finalCreateInfo,
14455	memTypeIndex,
14456	size,
14457	allocationCount);
14458	if(res != VK_SUCCESS)
14459	return res;
14460
14461	if((finalCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
14462	{
14463	return AllocateDedicatedMemory(
14464	pool,
14465	size,
14466	suballocType,
14467	dedicatedAllocations,
14468	memTypeIndex,
14469	map: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
14470	isUserDataString: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
14471	isMappingAllowed: (finalCreateInfo.flags &
14472	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0,
14473	canAliasMemory: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != 0,
14474	pUserData: finalCreateInfo.pUserData,
14475	priority: finalCreateInfo.priority,
14476	dedicatedBuffer,
14477	dedicatedImage,
14478	dedicatedBufferImageUsage,
14479	allocationCount,
14480	pAllocations,
14481	pNextChain: blockVector.GetAllocationNextPtr());
14482	}
14483	else
14484	{
14485	const bool canAllocateDedicated =
14486	(finalCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) == 0 &&
14487	(pool == VK_NULL_HANDLE || !blockVector.HasExplicitBlockSize());
14488
14489	if(canAllocateDedicated)
14490	{
14491	// Heuristics: Allocate dedicated memory if requested size if greater than half of preferred block size.
14492	if(size > blockVector.GetPreferredBlockSize() / 2)
14493	{
14494	dedicatedPreferred = true;
14495	}
14496	// Protection against creating each allocation as dedicated when we reach or exceed heap size/budget,
14497	// which can quickly deplete maxMemoryAllocationCount: Don't prefer dedicated allocations when above
14498	// 3/4 of the maximum allocation count.
14499	if(m_DeviceMemoryCount.load() > m_PhysicalDeviceProperties.limits.maxMemoryAllocationCount * 3 / 4)
14500	{
14501	dedicatedPreferred = false;
14502	}
14503
14504	if(dedicatedPreferred)
14505	{
14506	res = AllocateDedicatedMemory(
14507	pool,
14508	size,
14509	suballocType,
14510	dedicatedAllocations,
14511	memTypeIndex,
14512	map: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
14513	isUserDataString: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
14514	isMappingAllowed: (finalCreateInfo.flags &
14515	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0,
14516	canAliasMemory: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != 0,
14517	pUserData: finalCreateInfo.pUserData,
14518	priority: finalCreateInfo.priority,
14519	dedicatedBuffer,
14520	dedicatedImage,
14521	dedicatedBufferImageUsage,
14522	allocationCount,
14523	pAllocations,
14524	pNextChain: blockVector.GetAllocationNextPtr());
14525	if(res == VK_SUCCESS)
14526	{
14527	// Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
14528	VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
14529	return VK_SUCCESS;
14530	}
14531	}
14532	}
14533
14534	res = blockVector.Allocate(
14535	size,
14536	alignment,
14537	createInfo: finalCreateInfo,
14538	suballocType,
14539	allocationCount,
14540	pAllocations);
14541	if(res == VK_SUCCESS)
14542	return VK_SUCCESS;
14543
14544	// Try dedicated memory.
14545	if(canAllocateDedicated && !dedicatedPreferred)
14546	{
14547	res = AllocateDedicatedMemory(
14548	pool,
14549	size,
14550	suballocType,
14551	dedicatedAllocations,
14552	memTypeIndex,
14553	map: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0,
14554	isUserDataString: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_USER_DATA_COPY_STRING_BIT) != 0,
14555	isMappingAllowed: (finalCreateInfo.flags &
14556	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0,
14557	canAliasMemory: (finalCreateInfo.flags & VMA_ALLOCATION_CREATE_CAN_ALIAS_BIT) != 0,
14558	pUserData: finalCreateInfo.pUserData,
14559	priority: finalCreateInfo.priority,
14560	dedicatedBuffer,
14561	dedicatedImage,
14562	dedicatedBufferImageUsage,
14563	allocationCount,
14564	pAllocations,
14565	pNextChain: blockVector.GetAllocationNextPtr());
14566	if(res == VK_SUCCESS)
14567	{
14568	// Succeeded: AllocateDedicatedMemory function already filld pMemory, nothing more to do here.
14569	VMA_DEBUG_LOG(" Allocated as DedicatedMemory");
14570	return VK_SUCCESS;
14571	}
14572	}
14573	// Everything failed: Return error code.
14574	VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
14575	return res;
14576	}
14577	}
14578
14579	VkResult VmaAllocator_T::AllocateDedicatedMemory(
14580	VmaPool pool,
14581	VkDeviceSize size,
14582	VmaSuballocationType suballocType,
14583	VmaDedicatedAllocationList& dedicatedAllocations,
14584	uint32_t memTypeIndex,
14585	bool map,
14586	bool isUserDataString,
14587	bool isMappingAllowed,
14588	bool canAliasMemory,
14589	void* pUserData,
14590	float priority,
14591	VkBuffer dedicatedBuffer,
14592	VkImage dedicatedImage,
14593	VkFlags dedicatedBufferImageUsage,
14594	size_t allocationCount,
14595	VmaAllocation* pAllocations,
14596	const void* pNextChain)
14597	{
14598	VMA_ASSERT(allocationCount > 0 && pAllocations);
14599
14600	VkMemoryAllocateInfo allocInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO };
14601	allocInfo.memoryTypeIndex = memTypeIndex;
14602	allocInfo.allocationSize = size;
14603	allocInfo.pNext = pNextChain;
14604
14605	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14606	VkMemoryDedicatedAllocateInfoKHR dedicatedAllocInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO_KHR };
14607	if(!canAliasMemory)
14608	{
14609	if(m_UseKhrDedicatedAllocation || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14610	{
14611	if(dedicatedBuffer != VK_NULL_HANDLE)
14612	{
14613	VMA_ASSERT(dedicatedImage == VK_NULL_HANDLE);
14614	dedicatedAllocInfo.buffer = dedicatedBuffer;
14615	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &dedicatedAllocInfo);
14616	}
14617	else if(dedicatedImage != VK_NULL_HANDLE)
14618	{
14619	dedicatedAllocInfo.image = dedicatedImage;
14620	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &dedicatedAllocInfo);
14621	}
14622	}
14623	}
14624	#endif // #if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14625
14626	#if VMA_BUFFER_DEVICE_ADDRESS
14627	VkMemoryAllocateFlagsInfoKHR allocFlagsInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_FLAGS_INFO_KHR };
14628	if(m_UseKhrBufferDeviceAddress)
14629	{
14630	bool canContainBufferWithDeviceAddress = true;
14631	if(dedicatedBuffer != VK_NULL_HANDLE)
14632	{
14633	canContainBufferWithDeviceAddress = dedicatedBufferImageUsage == UINT32_MAX || // Usage flags unknown
14634	(dedicatedBufferImageUsage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_EXT) != 0;
14635	}
14636	else if(dedicatedImage != VK_NULL_HANDLE)
14637	{
14638	canContainBufferWithDeviceAddress = false;
14639	}
14640	if(canContainBufferWithDeviceAddress)
14641	{
14642	allocFlagsInfo.flags = VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT_KHR;
14643	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &allocFlagsInfo);
14644	}
14645	}
14646	#endif // #if VMA_BUFFER_DEVICE_ADDRESS
14647
14648	#if VMA_MEMORY_PRIORITY
14649	VkMemoryPriorityAllocateInfoEXT priorityInfo = { .sType: VK_STRUCTURE_TYPE_MEMORY_PRIORITY_ALLOCATE_INFO_EXT };
14650	if(m_UseExtMemoryPriority)
14651	{
14652	VMA_ASSERT(priority >= 0.f && priority <= 1.f);
14653	priorityInfo.priority = priority;
14654	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &priorityInfo);
14655	}
14656	#endif // #if VMA_MEMORY_PRIORITY
14657
14658	#if VMA_EXTERNAL_MEMORY
14659	// Attach VkExportMemoryAllocateInfoKHR if necessary.
14660	VkExportMemoryAllocateInfoKHR exportMemoryAllocInfo = { .sType: VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO_KHR };
14661	exportMemoryAllocInfo.handleTypes = GetExternalMemoryHandleTypeFlags(memTypeIndex);
14662	if(exportMemoryAllocInfo.handleTypes != 0)
14663	{
14664	VmaPnextChainPushFront(mainStruct: &allocInfo, newStruct: &exportMemoryAllocInfo);
14665	}
14666	#endif // #if VMA_EXTERNAL_MEMORY
14667
14668	size_t allocIndex;
14669	VkResult res = VK_SUCCESS;
14670	for(allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
14671	{
14672	res = AllocateDedicatedMemoryPage(
14673	pool,
14674	size,
14675	suballocType,
14676	memTypeIndex,
14677	allocInfo,
14678	map,
14679	isUserDataString,
14680	isMappingAllowed,
14681	pUserData,
14682	pAllocation: pAllocations + allocIndex);
14683	if(res != VK_SUCCESS)
14684	{
14685	break;
14686	}
14687	}
14688
14689	if(res == VK_SUCCESS)
14690	{
14691	for (allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
14692	{
14693	dedicatedAllocations.Register(alloc: pAllocations[allocIndex]);
14694	}
14695	VMA_DEBUG_LOG(" Allocated DedicatedMemory Count=%zu, MemoryTypeIndex=#%u", allocationCount, memTypeIndex);
14696	}
14697	else
14698	{
14699	// Free all already created allocations.
14700	while(allocIndex--)
14701	{
14702	VmaAllocation currAlloc = pAllocations[allocIndex];
14703	VkDeviceMemory hMemory = currAlloc->GetMemory();
14704
14705	/*
14706	There is no need to call this, because Vulkan spec allows to skip vkUnmapMemory
14707	before vkFreeMemory.
14708
14709	if(currAlloc->GetMappedData() != VMA_NULL)
14710	{
14711	(*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
14712	}
14713	*/
14714
14715	FreeVulkanMemory(memoryType: memTypeIndex, size: currAlloc->GetSize(), hMemory);
14716	m_Budget.RemoveAllocation(heapIndex: MemoryTypeIndexToHeapIndex(memTypeIndex), allocationSize: currAlloc->GetSize());
14717	m_AllocationObjectAllocator.Free(hAlloc: currAlloc);
14718	}
14719
14720	memset(s: pAllocations, c: 0, n: sizeof(VmaAllocation) * allocationCount);
14721	}
14722
14723	return res;
14724	}
14725
14726	VkResult VmaAllocator_T::AllocateDedicatedMemoryPage(
14727	VmaPool pool,
14728	VkDeviceSize size,
14729	VmaSuballocationType suballocType,
14730	uint32_t memTypeIndex,
14731	const VkMemoryAllocateInfo& allocInfo,
14732	bool map,
14733	bool isUserDataString,
14734	bool isMappingAllowed,
14735	void* pUserData,
14736	VmaAllocation* pAllocation)
14737	{
14738	VkDeviceMemory hMemory = VK_NULL_HANDLE;
14739	VkResult res = AllocateVulkanMemory(pAllocateInfo: &allocInfo, pMemory: &hMemory);
14740	if(res < 0)
14741	{
14742	VMA_DEBUG_LOG(" vkAllocateMemory FAILED");
14743	return res;
14744	}
14745
14746	void* pMappedData = VMA_NULL;
14747	if(map)
14748	{
14749	res = (*m_VulkanFunctions.vkMapMemory)(
14750	m_hDevice,
14751	hMemory,
14752	0,
14753	VK_WHOLE_SIZE,
14754	0,
14755	&pMappedData);
14756	if(res < 0)
14757	{
14758	VMA_DEBUG_LOG(" vkMapMemory FAILED");
14759	FreeVulkanMemory(memoryType: memTypeIndex, size, hMemory);
14760	return res;
14761	}
14762	}
14763
14764	*pAllocation = m_AllocationObjectAllocator.Allocate(args&: isMappingAllowed);
14765	(*pAllocation)->InitDedicatedAllocation(hParentPool: pool, memoryTypeIndex: memTypeIndex, hMemory, suballocationType: suballocType, pMappedData, size);
14766	if (isUserDataString)
14767	(*pAllocation)->SetName(hAllocator: this, pName: (const char*)pUserData);
14768	else
14769	(*pAllocation)->SetUserData(hAllocator: this, pUserData);
14770	m_Budget.AddAllocation(heapIndex: MemoryTypeIndexToHeapIndex(memTypeIndex), allocationSize: size);
14771	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS)
14772	{
14773	FillAllocation(hAllocation: *pAllocation, pattern: VMA_ALLOCATION_FILL_PATTERN_CREATED);
14774	}
14775
14776	return VK_SUCCESS;
14777	}
14778
14779	void VmaAllocator_T::GetBufferMemoryRequirements(
14780	VkBuffer hBuffer,
14781	VkMemoryRequirements& memReq,
14782	bool& requiresDedicatedAllocation,
14783	bool& prefersDedicatedAllocation) const
14784	{
14785	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14786	if(m_UseKhrDedicatedAllocation || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14787	{
14788	VkBufferMemoryRequirementsInfo2KHR memReqInfo = { .sType: VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2_KHR };
14789	memReqInfo.buffer = hBuffer;
14790
14791	VkMemoryDedicatedRequirementsKHR memDedicatedReq = { .sType: VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
14792
14793	VkMemoryRequirements2KHR memReq2 = { .sType: VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
14794	VmaPnextChainPushFront(mainStruct: &memReq2, newStruct: &memDedicatedReq);
14795
14796	(*m_VulkanFunctions.vkGetBufferMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
14797
14798	memReq = memReq2.memoryRequirements;
14799	requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
14800	prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
14801	}
14802	else
14803	#endif // #if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14804	{
14805	(*m_VulkanFunctions.vkGetBufferMemoryRequirements)(m_hDevice, hBuffer, &memReq);
14806	requiresDedicatedAllocation = false;
14807	prefersDedicatedAllocation = false;
14808	}
14809	}
14810
14811	void VmaAllocator_T::GetImageMemoryRequirements(
14812	VkImage hImage,
14813	VkMemoryRequirements& memReq,
14814	bool& requiresDedicatedAllocation,
14815	bool& prefersDedicatedAllocation) const
14816	{
14817	#if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14818	if(m_UseKhrDedicatedAllocation || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0))
14819	{
14820	VkImageMemoryRequirementsInfo2KHR memReqInfo = { .sType: VK_STRUCTURE_TYPE_IMAGE_MEMORY_REQUIREMENTS_INFO_2_KHR };
14821	memReqInfo.image = hImage;
14822
14823	VkMemoryDedicatedRequirementsKHR memDedicatedReq = { .sType: VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS_KHR };
14824
14825	VkMemoryRequirements2KHR memReq2 = { .sType: VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2_KHR };
14826	VmaPnextChainPushFront(mainStruct: &memReq2, newStruct: &memDedicatedReq);
14827
14828	(*m_VulkanFunctions.vkGetImageMemoryRequirements2KHR)(m_hDevice, &memReqInfo, &memReq2);
14829
14830	memReq = memReq2.memoryRequirements;
14831	requiresDedicatedAllocation = (memDedicatedReq.requiresDedicatedAllocation != VK_FALSE);
14832	prefersDedicatedAllocation = (memDedicatedReq.prefersDedicatedAllocation != VK_FALSE);
14833	}
14834	else
14835	#endif // #if VMA_DEDICATED_ALLOCATION || VMA_VULKAN_VERSION >= 1001000
14836	{
14837	(*m_VulkanFunctions.vkGetImageMemoryRequirements)(m_hDevice, hImage, &memReq);
14838	requiresDedicatedAllocation = false;
14839	prefersDedicatedAllocation = false;
14840	}
14841	}
14842
14843	VkResult VmaAllocator_T::FindMemoryTypeIndex(
14844	uint32_t memoryTypeBits,
14845	const VmaAllocationCreateInfo* pAllocationCreateInfo,
14846	VkFlags bufImgUsage,
14847	uint32_t* pMemoryTypeIndex) const
14848	{
14849	memoryTypeBits &= GetGlobalMemoryTypeBits();
14850
14851	if(pAllocationCreateInfo->memoryTypeBits != 0)
14852	{
14853	memoryTypeBits &= pAllocationCreateInfo->memoryTypeBits;
14854	}
14855
14856	VkMemoryPropertyFlags requiredFlags = 0, preferredFlags = 0, notPreferredFlags = 0;
14857	if(!FindMemoryPreferences(
14858	isIntegratedGPU: IsIntegratedGpu(),
14859	allocCreateInfo: *pAllocationCreateInfo,
14860	bufImgUsage,
14861	outRequiredFlags&: requiredFlags, outPreferredFlags&: preferredFlags, outNotPreferredFlags&: notPreferredFlags))
14862	{
14863	return VK_ERROR_FEATURE_NOT_PRESENT;
14864	}
14865
14866	*pMemoryTypeIndex = UINT32_MAX;
14867	uint32_t minCost = UINT32_MAX;
14868	for(uint32_t memTypeIndex = 0, memTypeBit = 1;
14869	memTypeIndex < GetMemoryTypeCount();
14870	++memTypeIndex, memTypeBit <<= 1)
14871	{
14872	// This memory type is acceptable according to memoryTypeBits bitmask.
14873	if((memTypeBit & memoryTypeBits) != 0)
14874	{
14875	const VkMemoryPropertyFlags currFlags =
14876	m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
14877	// This memory type contains requiredFlags.
14878	if((requiredFlags & ~currFlags) == 0)
14879	{
14880	// Calculate cost as number of bits from preferredFlags not present in this memory type.
14881	uint32_t currCost = VMA_COUNT_BITS_SET(preferredFlags & ~currFlags) +
14882	VMA_COUNT_BITS_SET(currFlags & notPreferredFlags);
14883	// Remember memory type with lowest cost.
14884	if(currCost < minCost)
14885	{
14886	*pMemoryTypeIndex = memTypeIndex;
14887	if(currCost == 0)
14888	{
14889	return VK_SUCCESS;
14890	}
14891	minCost = currCost;
14892	}
14893	}
14894	}
14895	}
14896	return (*pMemoryTypeIndex != UINT32_MAX) ? VK_SUCCESS : VK_ERROR_FEATURE_NOT_PRESENT;
14897	}
14898
14899	VkResult VmaAllocator_T::CalcMemTypeParams(
14900	VmaAllocationCreateInfo& inoutCreateInfo,
14901	uint32_t memTypeIndex,
14902	VkDeviceSize size,
14903	size_t allocationCount)
14904	{
14905	// If memory type is not HOST_VISIBLE, disable MAPPED.
14906	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0 &&
14907	(m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) == 0)
14908	{
14909	inoutCreateInfo.flags &= ~VMA_ALLOCATION_CREATE_MAPPED_BIT;
14910	}
14911
14912	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
14913	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT) != 0)
14914	{
14915	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex);
14916	VmaBudget heapBudget = {};
14917	GetHeapBudgets(outBudgets: &heapBudget, firstHeap: heapIndex, heapCount: 1);
14918	if(heapBudget.usage + size * allocationCount > heapBudget.budget)
14919	{
14920	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
14921	}
14922	}
14923	return VK_SUCCESS;
14924	}
14925
14926	VkResult VmaAllocator_T::CalcAllocationParams(
14927	VmaAllocationCreateInfo& inoutCreateInfo,
14928	bool dedicatedRequired,
14929	bool dedicatedPreferred)
14930	{
14931	VMA_ASSERT((inoutCreateInfo.flags &
14932	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) !=
14933	(VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT) &&
14934	"Specifying both flags VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT and VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT is incorrect.");
14935	VMA_ASSERT((((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT) == 0 ||
14936	(inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0)) &&
14937	"Specifying VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT requires also VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.");
14938	if(inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO || inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE || inoutCreateInfo.usage == VMA_MEMORY_USAGE_AUTO_PREFER_HOST)
14939	{
14940	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_MAPPED_BIT) != 0)
14941	{
14942	VMA_ASSERT((inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) != 0 &&
14943	"When using VMA_ALLOCATION_CREATE_MAPPED_BIT and usage = VMA_MEMORY_USAGE_AUTO*, you must also specify VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.");
14944	}
14945	}
14946
14947	// If memory is lazily allocated, it should be always dedicated.
14948	if(dedicatedRequired ||
14949	inoutCreateInfo.usage == VMA_MEMORY_USAGE_GPU_LAZILY_ALLOCATED)
14950	{
14951	inoutCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
14952	}
14953
14954	if(inoutCreateInfo.pool != VK_NULL_HANDLE)
14955	{
14956	if(inoutCreateInfo.pool->m_BlockVector.HasExplicitBlockSize() &&
14957	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0)
14958	{
14959	VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT while current custom pool doesn't support dedicated allocations.");
14960	return VK_ERROR_FEATURE_NOT_PRESENT;
14961	}
14962	inoutCreateInfo.priority = inoutCreateInfo.pool->m_BlockVector.GetPriority();
14963	}
14964
14965	if((inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT) != 0 &&
14966	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
14967	{
14968	VMA_ASSERT(0 && "Specifying VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT together with VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT makes no sense.");
14969	return VK_ERROR_FEATURE_NOT_PRESENT;
14970	}
14971
14972	if(VMA_DEBUG_ALWAYS_DEDICATED_MEMORY &&
14973	(inoutCreateInfo.flags & VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT) != 0)
14974	{
14975	inoutCreateInfo.flags |= VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
14976	}
14977
14978	// Non-auto USAGE values imply HOST_ACCESS flags.
14979	// And so does VMA_MEMORY_USAGE_UNKNOWN because it is used with custom pools.
14980	// Which specific flag is used doesn't matter. They change things only when used with VMA_MEMORY_USAGE_AUTO*.
14981	// Otherwise they just protect from assert on mapping.
14982	if(inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO &&
14983	inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE &&
14984	inoutCreateInfo.usage != VMA_MEMORY_USAGE_AUTO_PREFER_HOST)
14985	{
14986	if((inoutCreateInfo.flags & (VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT | VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT)) == 0)
14987	{
14988	inoutCreateInfo.flags |= VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT;
14989	}
14990	}
14991
14992	return VK_SUCCESS;
14993	}
14994
14995	VkResult VmaAllocator_T::AllocateMemory(
14996	const VkMemoryRequirements& vkMemReq,
14997	bool requiresDedicatedAllocation,
14998	bool prefersDedicatedAllocation,
14999	VkBuffer dedicatedBuffer,
15000	VkImage dedicatedImage,
15001	VkFlags dedicatedBufferImageUsage,
15002	const VmaAllocationCreateInfo& createInfo,
15003	VmaSuballocationType suballocType,
15004	size_t allocationCount,
15005	VmaAllocation* pAllocations)
15006	{
15007	memset(s: pAllocations, c: 0, n: sizeof(VmaAllocation) * allocationCount);
15008
15009	VMA_ASSERT(VmaIsPow2(vkMemReq.alignment));
15010
15011	if(vkMemReq.size == 0)
15012	{
15013	return VK_ERROR_INITIALIZATION_FAILED;
15014	}
15015
15016	VmaAllocationCreateInfo createInfoFinal = createInfo;
15017	VkResult res = CalcAllocationParams(inoutCreateInfo&: createInfoFinal, dedicatedRequired: requiresDedicatedAllocation, dedicatedPreferred: prefersDedicatedAllocation);
15018	if(res != VK_SUCCESS)
15019	return res;
15020
15021	if(createInfoFinal.pool != VK_NULL_HANDLE)
15022	{
15023	VmaBlockVector& blockVector = createInfoFinal.pool->m_BlockVector;
15024	return AllocateMemoryOfType(
15025	pool: createInfoFinal.pool,
15026	size: vkMemReq.size,
15027	alignment: vkMemReq.alignment,
15028	dedicatedPreferred: prefersDedicatedAllocation,
15029	dedicatedBuffer,
15030	dedicatedImage,
15031	dedicatedBufferImageUsage,
15032	createInfo: createInfoFinal,
15033	memTypeIndex: blockVector.GetMemoryTypeIndex(),
15034	suballocType,
15035	dedicatedAllocations&: createInfoFinal.pool->m_DedicatedAllocations,
15036	blockVector,
15037	allocationCount,
15038	pAllocations);
15039	}
15040	else
15041	{
15042	// Bit mask of memory Vulkan types acceptable for this allocation.
15043	uint32_t memoryTypeBits = vkMemReq.memoryTypeBits;
15044	uint32_t memTypeIndex = UINT32_MAX;
15045	res = FindMemoryTypeIndex(memoryTypeBits, pAllocationCreateInfo: &createInfoFinal, bufImgUsage: dedicatedBufferImageUsage, pMemoryTypeIndex: &memTypeIndex);
15046	// Can't find any single memory type matching requirements. res is VK_ERROR_FEATURE_NOT_PRESENT.
15047	if(res != VK_SUCCESS)
15048	return res;
15049	do
15050	{
15051	VmaBlockVector* blockVector = m_pBlockVectors[memTypeIndex];
15052	VMA_ASSERT(blockVector && "Trying to use unsupported memory type!");
15053	res = AllocateMemoryOfType(
15054	VK_NULL_HANDLE,
15055	size: vkMemReq.size,
15056	alignment: vkMemReq.alignment,
15057	dedicatedPreferred: requiresDedicatedAllocation || prefersDedicatedAllocation,
15058	dedicatedBuffer,
15059	dedicatedImage,
15060	dedicatedBufferImageUsage,
15061	createInfo: createInfoFinal,
15062	memTypeIndex,
15063	suballocType,
15064	dedicatedAllocations&: m_DedicatedAllocations[memTypeIndex],
15065	blockVector&: *blockVector,
15066	allocationCount,
15067	pAllocations);
15068	// Allocation succeeded
15069	if(res == VK_SUCCESS)
15070	return VK_SUCCESS;
15071
15072	// Remove old memTypeIndex from list of possibilities.
15073	memoryTypeBits &= ~(1u << memTypeIndex);
15074	// Find alternative memTypeIndex.
15075	res = FindMemoryTypeIndex(memoryTypeBits, pAllocationCreateInfo: &createInfoFinal, bufImgUsage: dedicatedBufferImageUsage, pMemoryTypeIndex: &memTypeIndex);
15076	} while(res == VK_SUCCESS);
15077
15078	// No other matching memory type index could be found.
15079	// Not returning res, which is VK_ERROR_FEATURE_NOT_PRESENT, because we already failed to allocate once.
15080	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
15081	}
15082	}
15083
15084	void VmaAllocator_T::FreeMemory(
15085	size_t allocationCount,
15086	const VmaAllocation* pAllocations)
15087	{
15088	VMA_ASSERT(pAllocations);
15089
15090	for(size_t allocIndex = allocationCount; allocIndex--; )
15091	{
15092	VmaAllocation allocation = pAllocations[allocIndex];
15093
15094	if(allocation != VK_NULL_HANDLE)
15095	{
15096	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS)
15097	{
15098	FillAllocation(hAllocation: allocation, pattern: VMA_ALLOCATION_FILL_PATTERN_DESTROYED);
15099	}
15100
15101	allocation->FreeName(hAllocator: this);
15102
15103	switch(allocation->GetType())
15104	{
15105	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15106	{
15107	VmaBlockVector* pBlockVector = VMA_NULL;
15108	VmaPool hPool = allocation->GetParentPool();
15109	if(hPool != VK_NULL_HANDLE)
15110	{
15111	pBlockVector = &hPool->m_BlockVector;
15112	}
15113	else
15114	{
15115	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15116	pBlockVector = m_pBlockVectors[memTypeIndex];
15117	VMA_ASSERT(pBlockVector && "Trying to free memory of unsupported type!");
15118	}
15119	pBlockVector->Free(hAllocation: allocation);
15120	}
15121	break;
15122	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15123	FreeDedicatedMemory(allocation);
15124	break;
15125	default:
15126	VMA_ASSERT(0);
15127	}
15128	}
15129	}
15130	}
15131
15132	void VmaAllocator_T::CalculateStatistics(VmaTotalStatistics* pStats)
15133	{
15134	// Initialize.
15135	VmaClearDetailedStatistics(outStats&: pStats->total);
15136	for(uint32_t i = 0; i < VK_MAX_MEMORY_TYPES; ++i)
15137	VmaClearDetailedStatistics(outStats&: pStats->memoryType[i]);
15138	for(uint32_t i = 0; i < VK_MAX_MEMORY_HEAPS; ++i)
15139	VmaClearDetailedStatistics(outStats&: pStats->memoryHeap[i]);
15140
15141	// Process default pools.
15142	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15143	{
15144	VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
15145	if (pBlockVector != VMA_NULL)
15146	pBlockVector->AddDetailedStatistics(inoutStats&: pStats->memoryType[memTypeIndex]);
15147	}
15148
15149	// Process custom pools.
15150	{
15151	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15152	for(VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(item: pool))
15153	{
15154	VmaBlockVector& blockVector = pool->m_BlockVector;
15155	const uint32_t memTypeIndex = blockVector.GetMemoryTypeIndex();
15156	blockVector.AddDetailedStatistics(inoutStats&: pStats->memoryType[memTypeIndex]);
15157	pool->m_DedicatedAllocations.AddDetailedStatistics(inoutStats&: pStats->memoryType[memTypeIndex]);
15158	}
15159	}
15160
15161	// Process dedicated allocations.
15162	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15163	{
15164	m_DedicatedAllocations[memTypeIndex].AddDetailedStatistics(inoutStats&: pStats->memoryType[memTypeIndex]);
15165	}
15166
15167	// Sum from memory types to memory heaps.
15168	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15169	{
15170	const uint32_t memHeapIndex = m_MemProps.memoryTypes[memTypeIndex].heapIndex;
15171	VmaAddDetailedStatistics(inoutStats&: pStats->memoryHeap[memHeapIndex], src: pStats->memoryType[memTypeIndex]);
15172	}
15173
15174	// Sum from memory heaps to total.
15175	for(uint32_t memHeapIndex = 0; memHeapIndex < GetMemoryHeapCount(); ++memHeapIndex)
15176	VmaAddDetailedStatistics(inoutStats&: pStats->total, src: pStats->memoryHeap[memHeapIndex]);
15177
15178	VMA_ASSERT(pStats->total.statistics.allocationCount == 0 ||
15179	pStats->total.allocationSizeMax >= pStats->total.allocationSizeMin);
15180	VMA_ASSERT(pStats->total.unusedRangeCount == 0 ||
15181	pStats->total.unusedRangeSizeMax >= pStats->total.unusedRangeSizeMin);
15182	}
15183
15184	void VmaAllocator_T::GetHeapBudgets(VmaBudget* outBudgets, uint32_t firstHeap, uint32_t heapCount)
15185	{
15186	#if VMA_MEMORY_BUDGET
15187	if(m_UseExtMemoryBudget)
15188	{
15189	if(m_Budget.m_OperationsSinceBudgetFetch < 30)
15190	{
15191	VmaMutexLockRead lockRead(m_Budget.m_BudgetMutex, m_UseMutex);
15192	for(uint32_t i = 0; i < heapCount; ++i, ++outBudgets)
15193	{
15194	const uint32_t heapIndex = firstHeap + i;
15195
15196	outBudgets->statistics.blockCount = m_Budget.m_BlockCount[heapIndex];
15197	outBudgets->statistics.allocationCount = m_Budget.m_AllocationCount[heapIndex];
15198	outBudgets->statistics.blockBytes = m_Budget.m_BlockBytes[heapIndex];
15199	outBudgets->statistics.allocationBytes = m_Budget.m_AllocationBytes[heapIndex];
15200
15201	if(m_Budget.m_VulkanUsage[heapIndex] + outBudgets->statistics.blockBytes > m_Budget.m_BlockBytesAtBudgetFetch[heapIndex])
15202	{
15203	outBudgets->usage = m_Budget.m_VulkanUsage[heapIndex] +
15204	outBudgets->statistics.blockBytes - m_Budget.m_BlockBytesAtBudgetFetch[heapIndex];
15205	}
15206	else
15207	{
15208	outBudgets->usage = 0;
15209	}
15210
15211	// Have to take MIN with heap size because explicit HeapSizeLimit is included in it.
15212	outBudgets->budget = VMA_MIN(
15213	m_Budget.m_VulkanBudget[heapIndex], m_MemProps.memoryHeaps[heapIndex].size);
15214	}
15215	}
15216	else
15217	{
15218	UpdateVulkanBudget(); // Outside of mutex lock
15219	GetHeapBudgets(outBudgets, firstHeap, heapCount); // Recursion
15220	}
15221	}
15222	else
15223	#endif
15224	{
15225	for(uint32_t i = 0; i < heapCount; ++i, ++outBudgets)
15226	{
15227	const uint32_t heapIndex = firstHeap + i;
15228
15229	outBudgets->statistics.blockCount = m_Budget.m_BlockCount[heapIndex];
15230	outBudgets->statistics.allocationCount = m_Budget.m_AllocationCount[heapIndex];
15231	outBudgets->statistics.blockBytes = m_Budget.m_BlockBytes[heapIndex];
15232	outBudgets->statistics.allocationBytes = m_Budget.m_AllocationBytes[heapIndex];
15233
15234	outBudgets->usage = outBudgets->statistics.blockBytes;
15235	outBudgets->budget = m_MemProps.memoryHeaps[heapIndex].size * 8 / 10; // 80% heuristics.
15236	}
15237	}
15238	}
15239
15240	void VmaAllocator_T::GetAllocationInfo(VmaAllocation hAllocation, VmaAllocationInfo* pAllocationInfo)
15241	{
15242	pAllocationInfo->memoryType = hAllocation->GetMemoryTypeIndex();
15243	pAllocationInfo->deviceMemory = hAllocation->GetMemory();
15244	pAllocationInfo->offset = hAllocation->GetOffset();
15245	pAllocationInfo->size = hAllocation->GetSize();
15246	pAllocationInfo->pMappedData = hAllocation->GetMappedData();
15247	pAllocationInfo->pUserData = hAllocation->GetUserData();
15248	pAllocationInfo->pName = hAllocation->GetName();
15249	}
15250
15251	VkResult VmaAllocator_T::CreatePool(const VmaPoolCreateInfo* pCreateInfo, VmaPool* pPool)
15252	{
15253	VMA_DEBUG_LOG(" CreatePool: MemoryTypeIndex=%u, flags=%u", pCreateInfo->memoryTypeIndex, pCreateInfo->flags);
15254
15255	VmaPoolCreateInfo newCreateInfo = *pCreateInfo;
15256
15257	// Protection against uninitialized new structure member. If garbage data are left there, this pointer dereference would crash.
15258	if(pCreateInfo->pMemoryAllocateNext)
15259	{
15260	VMA_ASSERT(((const VkBaseInStructure*)pCreateInfo->pMemoryAllocateNext)->sType != 0);
15261	}
15262
15263	if(newCreateInfo.maxBlockCount == 0)
15264	{
15265	newCreateInfo.maxBlockCount = SIZE_MAX;
15266	}
15267	if(newCreateInfo.minBlockCount > newCreateInfo.maxBlockCount)
15268	{
15269	return VK_ERROR_INITIALIZATION_FAILED;
15270	}
15271	// Memory type index out of range or forbidden.
15272	if(pCreateInfo->memoryTypeIndex >= GetMemoryTypeCount() ||
15273	((1u << pCreateInfo->memoryTypeIndex) & m_GlobalMemoryTypeBits) == 0)
15274	{
15275	return VK_ERROR_FEATURE_NOT_PRESENT;
15276	}
15277	if(newCreateInfo.minAllocationAlignment > 0)
15278	{
15279	VMA_ASSERT(VmaIsPow2(newCreateInfo.minAllocationAlignment));
15280	}
15281
15282	const VkDeviceSize preferredBlockSize = CalcPreferredBlockSize(memTypeIndex: newCreateInfo.memoryTypeIndex);
15283
15284	*pPool = vma_new(this, VmaPool_T)(this, newCreateInfo, preferredBlockSize);
15285
15286	VkResult res = (*pPool)->m_BlockVector.CreateMinBlocks();
15287	if(res != VK_SUCCESS)
15288	{
15289	vma_delete(hAllocator: this, ptr: *pPool);
15290	*pPool = VMA_NULL;
15291	return res;
15292	}
15293
15294	// Add to m_Pools.
15295	{
15296	VmaMutexLockWrite lock(m_PoolsMutex, m_UseMutex);
15297	(*pPool)->SetId(m_NextPoolId++);
15298	m_Pools.PushBack(item: *pPool);
15299	}
15300
15301	return VK_SUCCESS;
15302	}
15303
15304	void VmaAllocator_T::DestroyPool(VmaPool pool)
15305	{
15306	// Remove from m_Pools.
15307	{
15308	VmaMutexLockWrite lock(m_PoolsMutex, m_UseMutex);
15309	m_Pools.Remove(item: pool);
15310	}
15311
15312	vma_delete(hAllocator: this, ptr: pool);
15313	}
15314
15315	void VmaAllocator_T::GetPoolStatistics(VmaPool pool, VmaStatistics* pPoolStats)
15316	{
15317	VmaClearStatistics(outStats&: *pPoolStats);
15318	pool->m_BlockVector.AddStatistics(inoutStats&: *pPoolStats);
15319	pool->m_DedicatedAllocations.AddStatistics(inoutStats&: *pPoolStats);
15320	}
15321
15322	void VmaAllocator_T::CalculatePoolStatistics(VmaPool pool, VmaDetailedStatistics* pPoolStats)
15323	{
15324	VmaClearDetailedStatistics(outStats&: *pPoolStats);
15325	pool->m_BlockVector.AddDetailedStatistics(inoutStats&: *pPoolStats);
15326	pool->m_DedicatedAllocations.AddDetailedStatistics(inoutStats&: *pPoolStats);
15327	}
15328
15329	void VmaAllocator_T::SetCurrentFrameIndex(uint32_t frameIndex)
15330	{
15331	m_CurrentFrameIndex.store(i: frameIndex);
15332
15333	#if VMA_MEMORY_BUDGET
15334	if(m_UseExtMemoryBudget)
15335	{
15336	UpdateVulkanBudget();
15337	}
15338	#endif // #if VMA_MEMORY_BUDGET
15339	}
15340
15341	VkResult VmaAllocator_T::CheckPoolCorruption(VmaPool hPool)
15342	{
15343	return hPool->m_BlockVector.CheckCorruption();
15344	}
15345
15346	VkResult VmaAllocator_T::CheckCorruption(uint32_t memoryTypeBits)
15347	{
15348	VkResult finalRes = VK_ERROR_FEATURE_NOT_PRESENT;
15349
15350	// Process default pools.
15351	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15352	{
15353	VmaBlockVector* const pBlockVector = m_pBlockVectors[memTypeIndex];
15354	if(pBlockVector != VMA_NULL)
15355	{
15356	VkResult localRes = pBlockVector->CheckCorruption();
15357	switch(localRes)
15358	{
15359	case VK_ERROR_FEATURE_NOT_PRESENT:
15360	break;
15361	case VK_SUCCESS:
15362	finalRes = VK_SUCCESS;
15363	break;
15364	default:
15365	return localRes;
15366	}
15367	}
15368	}
15369
15370	// Process custom pools.
15371	{
15372	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15373	for(VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(item: pool))
15374	{
15375	if(((1u << pool->m_BlockVector.GetMemoryTypeIndex()) & memoryTypeBits) != 0)
15376	{
15377	VkResult localRes = pool->m_BlockVector.CheckCorruption();
15378	switch(localRes)
15379	{
15380	case VK_ERROR_FEATURE_NOT_PRESENT:
15381	break;
15382	case VK_SUCCESS:
15383	finalRes = VK_SUCCESS;
15384	break;
15385	default:
15386	return localRes;
15387	}
15388	}
15389	}
15390	}
15391
15392	return finalRes;
15393	}
15394
15395	VkResult VmaAllocator_T::AllocateVulkanMemory(const VkMemoryAllocateInfo* pAllocateInfo, VkDeviceMemory* pMemory)
15396	{
15397	AtomicTransactionalIncrement<uint32_t> deviceMemoryCountIncrement;
15398	const uint64_t prevDeviceMemoryCount = deviceMemoryCountIncrement.Increment(atomic: &m_DeviceMemoryCount);
15399	#if VMA_DEBUG_DONT_EXCEED_MAX_MEMORY_ALLOCATION_COUNT
15400	if(prevDeviceMemoryCount >= m_PhysicalDeviceProperties.limits.maxMemoryAllocationCount)
15401	{
15402	return VK_ERROR_TOO_MANY_OBJECTS;
15403	}
15404	#endif
15405
15406	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex: pAllocateInfo->memoryTypeIndex);
15407
15408	// HeapSizeLimit is in effect for this heap.
15409	if((m_HeapSizeLimitMask & (1u << heapIndex)) != 0)
15410	{
15411	const VkDeviceSize heapSize = m_MemProps.memoryHeaps[heapIndex].size;
15412	VkDeviceSize blockBytes = m_Budget.m_BlockBytes[heapIndex];
15413	for(;;)
15414	{
15415	const VkDeviceSize blockBytesAfterAllocation = blockBytes + pAllocateInfo->allocationSize;
15416	if(blockBytesAfterAllocation > heapSize)
15417	{
15418	return VK_ERROR_OUT_OF_DEVICE_MEMORY;
15419	}
15420	if(m_Budget.m_BlockBytes[heapIndex].compare_exchange_strong(i1&: blockBytes, i2: blockBytesAfterAllocation))
15421	{
15422	break;
15423	}
15424	}
15425	}
15426	else
15427	{
15428	m_Budget.m_BlockBytes[heapIndex] += pAllocateInfo->allocationSize;
15429	}
15430	++m_Budget.m_BlockCount[heapIndex];
15431
15432	// VULKAN CALL vkAllocateMemory.
15433	VkResult res = (*m_VulkanFunctions.vkAllocateMemory)(m_hDevice, pAllocateInfo, GetAllocationCallbacks(), pMemory);
15434
15435	if(res == VK_SUCCESS)
15436	{
15437	#if VMA_MEMORY_BUDGET
15438	++m_Budget.m_OperationsSinceBudgetFetch;
15439	#endif
15440
15441	// Informative callback.
15442	if(m_DeviceMemoryCallbacks.pfnAllocate != VMA_NULL)
15443	{
15444	(*m_DeviceMemoryCallbacks.pfnAllocate)(this, pAllocateInfo->memoryTypeIndex, *pMemory, pAllocateInfo->allocationSize, m_DeviceMemoryCallbacks.pUserData);
15445	}
15446
15447	deviceMemoryCountIncrement.Commit();
15448	}
15449	else
15450	{
15451	--m_Budget.m_BlockCount[heapIndex];
15452	m_Budget.m_BlockBytes[heapIndex] -= pAllocateInfo->allocationSize;
15453	}
15454
15455	return res;
15456	}
15457
15458	void VmaAllocator_T::FreeVulkanMemory(uint32_t memoryType, VkDeviceSize size, VkDeviceMemory hMemory)
15459	{
15460	// Informative callback.
15461	if(m_DeviceMemoryCallbacks.pfnFree != VMA_NULL)
15462	{
15463	(*m_DeviceMemoryCallbacks.pfnFree)(this, memoryType, hMemory, size, m_DeviceMemoryCallbacks.pUserData);
15464	}
15465
15466	// VULKAN CALL vkFreeMemory.
15467	(*m_VulkanFunctions.vkFreeMemory)(m_hDevice, hMemory, GetAllocationCallbacks());
15468
15469	const uint32_t heapIndex = MemoryTypeIndexToHeapIndex(memTypeIndex: memoryType);
15470	--m_Budget.m_BlockCount[heapIndex];
15471	m_Budget.m_BlockBytes[heapIndex] -= size;
15472
15473	--m_DeviceMemoryCount;
15474	}
15475
15476	VkResult VmaAllocator_T::BindVulkanBuffer(
15477	VkDeviceMemory memory,
15478	VkDeviceSize memoryOffset,
15479	VkBuffer buffer,
15480	const void* pNext)
15481	{
15482	if(pNext != VMA_NULL)
15483	{
15484	#if VMA_VULKAN_VERSION >= 1001000 || VMA_BIND_MEMORY2
15485	if((m_UseKhrBindMemory2 || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0)) &&
15486	m_VulkanFunctions.vkBindBufferMemory2KHR != VMA_NULL)
15487	{
15488	VkBindBufferMemoryInfoKHR bindBufferMemoryInfo = { .sType: VK_STRUCTURE_TYPE_BIND_BUFFER_MEMORY_INFO_KHR };
15489	bindBufferMemoryInfo.pNext = pNext;
15490	bindBufferMemoryInfo.buffer = buffer;
15491	bindBufferMemoryInfo.memory = memory;
15492	bindBufferMemoryInfo.memoryOffset = memoryOffset;
15493	return (*m_VulkanFunctions.vkBindBufferMemory2KHR)(m_hDevice, 1, &bindBufferMemoryInfo);
15494	}
15495	else
15496	#endif // #if VMA_VULKAN_VERSION >= 1001000 || VMA_BIND_MEMORY2
15497	{
15498	return VK_ERROR_EXTENSION_NOT_PRESENT;
15499	}
15500	}
15501	else
15502	{
15503	return (*m_VulkanFunctions.vkBindBufferMemory)(m_hDevice, buffer, memory, memoryOffset);
15504	}
15505	}
15506
15507	VkResult VmaAllocator_T::BindVulkanImage(
15508	VkDeviceMemory memory,
15509	VkDeviceSize memoryOffset,
15510	VkImage image,
15511	const void* pNext)
15512	{
15513	if(pNext != VMA_NULL)
15514	{
15515	#if VMA_VULKAN_VERSION >= 1001000 || VMA_BIND_MEMORY2
15516	if((m_UseKhrBindMemory2 || m_VulkanApiVersion >= VK_MAKE_VERSION(1, 1, 0)) &&
15517	m_VulkanFunctions.vkBindImageMemory2KHR != VMA_NULL)
15518	{
15519	VkBindImageMemoryInfoKHR bindBufferMemoryInfo = { .sType: VK_STRUCTURE_TYPE_BIND_IMAGE_MEMORY_INFO_KHR };
15520	bindBufferMemoryInfo.pNext = pNext;
15521	bindBufferMemoryInfo.image = image;
15522	bindBufferMemoryInfo.memory = memory;
15523	bindBufferMemoryInfo.memoryOffset = memoryOffset;
15524	return (*m_VulkanFunctions.vkBindImageMemory2KHR)(m_hDevice, 1, &bindBufferMemoryInfo);
15525	}
15526	else
15527	#endif // #if VMA_BIND_MEMORY2
15528	{
15529	return VK_ERROR_EXTENSION_NOT_PRESENT;
15530	}
15531	}
15532	else
15533	{
15534	return (*m_VulkanFunctions.vkBindImageMemory)(m_hDevice, image, memory, memoryOffset);
15535	}
15536	}
15537
15538	VkResult VmaAllocator_T::Map(VmaAllocation hAllocation, void** ppData)
15539	{
15540	switch(hAllocation->GetType())
15541	{
15542	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15543	{
15544	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15545	char *pBytes = VMA_NULL;
15546	VkResult res = pBlock->Map(hAllocator: this, count: 1, ppData: (void**)&pBytes);
15547	if(res == VK_SUCCESS)
15548	{
15549	*ppData = pBytes + (ptrdiff_t)hAllocation->GetOffset();
15550	hAllocation->BlockAllocMap();
15551	}
15552	return res;
15553	}
15554	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15555	return hAllocation->DedicatedAllocMap(hAllocator: this, ppData);
15556	default:
15557	VMA_ASSERT(0);
15558	return VK_ERROR_MEMORY_MAP_FAILED;
15559	}
15560	}
15561
15562	void VmaAllocator_T::Unmap(VmaAllocation hAllocation)
15563	{
15564	switch(hAllocation->GetType())
15565	{
15566	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15567	{
15568	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15569	hAllocation->BlockAllocUnmap();
15570	pBlock->Unmap(hAllocator: this, count: 1);
15571	}
15572	break;
15573	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15574	hAllocation->DedicatedAllocUnmap(hAllocator: this);
15575	break;
15576	default:
15577	VMA_ASSERT(0);
15578	}
15579	}
15580
15581	VkResult VmaAllocator_T::BindBufferMemory(
15582	VmaAllocation hAllocation,
15583	VkDeviceSize allocationLocalOffset,
15584	VkBuffer hBuffer,
15585	const void* pNext)
15586	{
15587	VkResult res = VK_SUCCESS;
15588	switch(hAllocation->GetType())
15589	{
15590	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15591	res = BindVulkanBuffer(memory: hAllocation->GetMemory(), memoryOffset: allocationLocalOffset, buffer: hBuffer, pNext);
15592	break;
15593	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15594	{
15595	VmaDeviceMemoryBlock* const pBlock = hAllocation->GetBlock();
15596	VMA_ASSERT(pBlock && "Binding buffer to allocation that doesn't belong to any block.");
15597	res = pBlock->BindBufferMemory(hAllocator: this, hAllocation, allocationLocalOffset, hBuffer, pNext);
15598	break;
15599	}
15600	default:
15601	VMA_ASSERT(0);
15602	}
15603	return res;
15604	}
15605
15606	VkResult VmaAllocator_T::BindImageMemory(
15607	VmaAllocation hAllocation,
15608	VkDeviceSize allocationLocalOffset,
15609	VkImage hImage,
15610	const void* pNext)
15611	{
15612	VkResult res = VK_SUCCESS;
15613	switch(hAllocation->GetType())
15614	{
15615	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15616	res = BindVulkanImage(memory: hAllocation->GetMemory(), memoryOffset: allocationLocalOffset, image: hImage, pNext);
15617	break;
15618	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15619	{
15620	VmaDeviceMemoryBlock* pBlock = hAllocation->GetBlock();
15621	VMA_ASSERT(pBlock && "Binding image to allocation that doesn't belong to any block.");
15622	res = pBlock->BindImageMemory(hAllocator: this, hAllocation, allocationLocalOffset, hImage, pNext);
15623	break;
15624	}
15625	default:
15626	VMA_ASSERT(0);
15627	}
15628	return res;
15629	}
15630
15631	VkResult VmaAllocator_T::FlushOrInvalidateAllocation(
15632	VmaAllocation hAllocation,
15633	VkDeviceSize offset, VkDeviceSize size,
15634	VMA_CACHE_OPERATION op)
15635	{
15636	VkResult res = VK_SUCCESS;
15637
15638	VkMappedMemoryRange memRange = {};
15639	if(GetFlushOrInvalidateRange(allocation: hAllocation, offset, size, outRange&: memRange))
15640	{
15641	switch(op)
15642	{
15643	case VMA_CACHE_FLUSH:
15644	res = (*GetVulkanFunctions().vkFlushMappedMemoryRanges)(m_hDevice, 1, &memRange);
15645	break;
15646	case VMA_CACHE_INVALIDATE:
15647	res = (*GetVulkanFunctions().vkInvalidateMappedMemoryRanges)(m_hDevice, 1, &memRange);
15648	break;
15649	default:
15650	VMA_ASSERT(0);
15651	}
15652	}
15653	// else: Just ignore this call.
15654	return res;
15655	}
15656
15657	VkResult VmaAllocator_T::FlushOrInvalidateAllocations(
15658	uint32_t allocationCount,
15659	const VmaAllocation* allocations,
15660	const VkDeviceSize* offsets, const VkDeviceSize* sizes,
15661	VMA_CACHE_OPERATION op)
15662	{
15663	typedef VmaStlAllocator<VkMappedMemoryRange> RangeAllocator;
15664	typedef VmaSmallVector<VkMappedMemoryRange, RangeAllocator, 16> RangeVector;
15665	RangeVector ranges = RangeVector(RangeAllocator(GetAllocationCallbacks()));
15666
15667	for(uint32_t allocIndex = 0; allocIndex < allocationCount; ++allocIndex)
15668	{
15669	const VmaAllocation alloc = allocations[allocIndex];
15670	const VkDeviceSize offset = offsets != VMA_NULL ? offsets[allocIndex] : 0;
15671	const VkDeviceSize size = sizes != VMA_NULL ? sizes[allocIndex] : VK_WHOLE_SIZE;
15672	VkMappedMemoryRange newRange;
15673	if(GetFlushOrInvalidateRange(allocation: alloc, offset, size, outRange&: newRange))
15674	{
15675	ranges.push_back(src: newRange);
15676	}
15677	}
15678
15679	VkResult res = VK_SUCCESS;
15680	if(!ranges.empty())
15681	{
15682	switch(op)
15683	{
15684	case VMA_CACHE_FLUSH:
15685	res = (*GetVulkanFunctions().vkFlushMappedMemoryRanges)(m_hDevice, (uint32_t)ranges.size(), ranges.data());
15686	break;
15687	case VMA_CACHE_INVALIDATE:
15688	res = (*GetVulkanFunctions().vkInvalidateMappedMemoryRanges)(m_hDevice, (uint32_t)ranges.size(), ranges.data());
15689	break;
15690	default:
15691	VMA_ASSERT(0);
15692	}
15693	}
15694	// else: Just ignore this call.
15695	return res;
15696	}
15697
15698	void VmaAllocator_T::FreeDedicatedMemory(const VmaAllocation allocation)
15699	{
15700	VMA_ASSERT(allocation && allocation->GetType() == VmaAllocation_T::ALLOCATION_TYPE_DEDICATED);
15701
15702	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15703	VmaPool parentPool = allocation->GetParentPool();
15704	if(parentPool == VK_NULL_HANDLE)
15705	{
15706	// Default pool
15707	m_DedicatedAllocations[memTypeIndex].Unregister(alloc: allocation);
15708	}
15709	else
15710	{
15711	// Custom pool
15712	parentPool->m_DedicatedAllocations.Unregister(alloc: allocation);
15713	}
15714
15715	VkDeviceMemory hMemory = allocation->GetMemory();
15716
15717	/*
15718	There is no need to call this, because Vulkan spec allows to skip vkUnmapMemory
15719	before vkFreeMemory.
15720
15721	if(allocation->GetMappedData() != VMA_NULL)
15722	{
15723	(*m_VulkanFunctions.vkUnmapMemory)(m_hDevice, hMemory);
15724	}
15725	*/
15726
15727	FreeVulkanMemory(memoryType: memTypeIndex, size: allocation->GetSize(), hMemory);
15728
15729	m_Budget.RemoveAllocation(heapIndex: MemoryTypeIndexToHeapIndex(memTypeIndex: allocation->GetMemoryTypeIndex()), allocationSize: allocation->GetSize());
15730	m_AllocationObjectAllocator.Free(hAlloc: allocation);
15731
15732	VMA_DEBUG_LOG(" Freed DedicatedMemory MemoryTypeIndex=%u", memTypeIndex);
15733	}
15734
15735	uint32_t VmaAllocator_T::CalculateGpuDefragmentationMemoryTypeBits() const
15736	{
15737	VkBufferCreateInfo dummyBufCreateInfo;
15738	VmaFillGpuDefragmentationBufferCreateInfo(outBufCreateInfo&: dummyBufCreateInfo);
15739
15740	uint32_t memoryTypeBits = 0;
15741
15742	// Create buffer.
15743	VkBuffer buf = VK_NULL_HANDLE;
15744	VkResult res = (*GetVulkanFunctions().vkCreateBuffer)(
15745	m_hDevice, &dummyBufCreateInfo, GetAllocationCallbacks(), &buf);
15746	if(res == VK_SUCCESS)
15747	{
15748	// Query for supported memory types.
15749	VkMemoryRequirements memReq;
15750	(*GetVulkanFunctions().vkGetBufferMemoryRequirements)(m_hDevice, buf, &memReq);
15751	memoryTypeBits = memReq.memoryTypeBits;
15752
15753	// Destroy buffer.
15754	(*GetVulkanFunctions().vkDestroyBuffer)(m_hDevice, buf, GetAllocationCallbacks());
15755	}
15756
15757	return memoryTypeBits;
15758	}
15759
15760	uint32_t VmaAllocator_T::CalculateGlobalMemoryTypeBits() const
15761	{
15762	// Make sure memory information is already fetched.
15763	VMA_ASSERT(GetMemoryTypeCount() > 0);
15764
15765	uint32_t memoryTypeBits = UINT32_MAX;
15766
15767	if(!m_UseAmdDeviceCoherentMemory)
15768	{
15769	// Exclude memory types that have VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD.
15770	for(uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15771	{
15772	if((m_MemProps.memoryTypes[memTypeIndex].propertyFlags & VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY) != 0)
15773	{
15774	memoryTypeBits &= ~(1u << memTypeIndex);
15775	}
15776	}
15777	}
15778
15779	return memoryTypeBits;
15780	}
15781
15782	bool VmaAllocator_T::GetFlushOrInvalidateRange(
15783	VmaAllocation allocation,
15784	VkDeviceSize offset, VkDeviceSize size,
15785	VkMappedMemoryRange& outRange) const
15786	{
15787	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
15788	if(size > 0 && IsMemoryTypeNonCoherent(memTypeIndex))
15789	{
15790	const VkDeviceSize nonCoherentAtomSize = m_PhysicalDeviceProperties.limits.nonCoherentAtomSize;
15791	const VkDeviceSize allocationSize = allocation->GetSize();
15792	VMA_ASSERT(offset <= allocationSize);
15793
15794	outRange.sType = VK_STRUCTURE_TYPE_MAPPED_MEMORY_RANGE;
15795	outRange.pNext = VMA_NULL;
15796	outRange.memory = allocation->GetMemory();
15797
15798	switch(allocation->GetType())
15799	{
15800	case VmaAllocation_T::ALLOCATION_TYPE_DEDICATED:
15801	outRange.offset = VmaAlignDown(val: offset, alignment: nonCoherentAtomSize);
15802	if(size == VK_WHOLE_SIZE)
15803	{
15804	outRange.size = allocationSize - outRange.offset;
15805	}
15806	else
15807	{
15808	VMA_ASSERT(offset + size <= allocationSize);
15809	outRange.size = VMA_MIN(
15810	VmaAlignUp(size + (offset - outRange.offset), nonCoherentAtomSize),
15811	allocationSize - outRange.offset);
15812	}
15813	break;
15814	case VmaAllocation_T::ALLOCATION_TYPE_BLOCK:
15815	{
15816	// 1. Still within this allocation.
15817	outRange.offset = VmaAlignDown(val: offset, alignment: nonCoherentAtomSize);
15818	if(size == VK_WHOLE_SIZE)
15819	{
15820	size = allocationSize - offset;
15821	}
15822	else
15823	{
15824	VMA_ASSERT(offset + size <= allocationSize);
15825	}
15826	outRange.size = VmaAlignUp(val: size + (offset - outRange.offset), alignment: nonCoherentAtomSize);
15827
15828	// 2. Adjust to whole block.
15829	const VkDeviceSize allocationOffset = allocation->GetOffset();
15830	VMA_ASSERT(allocationOffset % nonCoherentAtomSize == 0);
15831	const VkDeviceSize blockSize = allocation->GetBlock()->m_pMetadata->GetSize();
15832	outRange.offset += allocationOffset;
15833	outRange.size = VMA_MIN(outRange.size, blockSize - outRange.offset);
15834
15835	break;
15836	}
15837	default:
15838	VMA_ASSERT(0);
15839	}
15840	return true;
15841	}
15842	return false;
15843	}
15844
15845	#if VMA_MEMORY_BUDGET
15846	void VmaAllocator_T::UpdateVulkanBudget()
15847	{
15848	VMA_ASSERT(m_UseExtMemoryBudget);
15849
15850	VkPhysicalDeviceMemoryProperties2KHR memProps = { .sType: VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_PROPERTIES_2_KHR };
15851
15852	VkPhysicalDeviceMemoryBudgetPropertiesEXT budgetProps = { .sType: VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT };
15853	VmaPnextChainPushFront(mainStruct: &memProps, newStruct: &budgetProps);
15854
15855	GetVulkanFunctions().vkGetPhysicalDeviceMemoryProperties2KHR(m_PhysicalDevice, &memProps);
15856
15857	{
15858	VmaMutexLockWrite lockWrite(m_Budget.m_BudgetMutex, m_UseMutex);
15859
15860	for(uint32_t heapIndex = 0; heapIndex < GetMemoryHeapCount(); ++heapIndex)
15861	{
15862	m_Budget.m_VulkanUsage[heapIndex] = budgetProps.heapUsage[heapIndex];
15863	m_Budget.m_VulkanBudget[heapIndex] = budgetProps.heapBudget[heapIndex];
15864	m_Budget.m_BlockBytesAtBudgetFetch[heapIndex] = m_Budget.m_BlockBytes[heapIndex].load();
15865
15866	// Some bugged drivers return the budget incorrectly, e.g. 0 or much bigger than heap size.
15867	if(m_Budget.m_VulkanBudget[heapIndex] == 0)
15868	{
15869	m_Budget.m_VulkanBudget[heapIndex] = m_MemProps.memoryHeaps[heapIndex].size * 8 / 10; // 80% heuristics.
15870	}
15871	else if(m_Budget.m_VulkanBudget[heapIndex] > m_MemProps.memoryHeaps[heapIndex].size)
15872	{
15873	m_Budget.m_VulkanBudget[heapIndex] = m_MemProps.memoryHeaps[heapIndex].size;
15874	}
15875	if(m_Budget.m_VulkanUsage[heapIndex] == 0 && m_Budget.m_BlockBytesAtBudgetFetch[heapIndex] > 0)
15876	{
15877	m_Budget.m_VulkanUsage[heapIndex] = m_Budget.m_BlockBytesAtBudgetFetch[heapIndex];
15878	}
15879	}
15880	m_Budget.m_OperationsSinceBudgetFetch = 0;
15881	}
15882	}
15883	#endif // VMA_MEMORY_BUDGET
15884
15885	void VmaAllocator_T::FillAllocation(const VmaAllocation hAllocation, uint8_t pattern)
15886	{
15887	if(VMA_DEBUG_INITIALIZE_ALLOCATIONS &&
15888	hAllocation->IsMappingAllowed() &&
15889	(m_MemProps.memoryTypes[hAllocation->GetMemoryTypeIndex()].propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT) != 0)
15890	{
15891	void* pData = VMA_NULL;
15892	VkResult res = Map(hAllocation, ppData: &pData);
15893	if(res == VK_SUCCESS)
15894	{
15895	memset(s: pData, c: (int)pattern, n: (size_t)hAllocation->GetSize());
15896	FlushOrInvalidateAllocation(hAllocation, offset: 0, VK_WHOLE_SIZE, op: VMA_CACHE_FLUSH);
15897	Unmap(hAllocation);
15898	}
15899	else
15900	{
15901	VMA_ASSERT(0 && "VMA_DEBUG_INITIALIZE_ALLOCATIONS is enabled, but couldn't map memory to fill allocation.");
15902	}
15903	}
15904	}
15905
15906	uint32_t VmaAllocator_T::GetGpuDefragmentationMemoryTypeBits()
15907	{
15908	uint32_t memoryTypeBits = m_GpuDefragmentationMemoryTypeBits.load();
15909	if(memoryTypeBits == UINT32_MAX)
15910	{
15911	memoryTypeBits = CalculateGpuDefragmentationMemoryTypeBits();
15912	m_GpuDefragmentationMemoryTypeBits.store(i: memoryTypeBits);
15913	}
15914	return memoryTypeBits;
15915	}
15916
15917	#if VMA_STATS_STRING_ENABLED
15918	void VmaAllocator_T::PrintDetailedMap(VmaJsonWriter& json)
15919	{
15920	json.WriteString(pStr: "DefaultPools");
15921	json.BeginObject();
15922	{
15923	for (uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15924	{
15925	VmaBlockVector* pBlockVector = m_pBlockVectors[memTypeIndex];
15926	VmaDedicatedAllocationList& dedicatedAllocList = m_DedicatedAllocations[memTypeIndex];
15927	if (pBlockVector != VMA_NULL)
15928	{
15929	json.BeginString(pStr: "Type ");
15930	json.ContinueString(n: memTypeIndex);
15931	json.EndString();
15932	json.BeginObject();
15933	{
15934	json.WriteString(pStr: "PreferredBlockSize");
15935	json.WriteNumber(n: pBlockVector->GetPreferredBlockSize());
15936
15937	json.WriteString(pStr: "Blocks");
15938	pBlockVector->PrintDetailedMap(json);
15939
15940	json.WriteString(pStr: "DedicatedAllocations");
15941	dedicatedAllocList.BuildStatsString(json);
15942	}
15943	json.EndObject();
15944	}
15945	}
15946	}
15947	json.EndObject();
15948
15949	json.WriteString(pStr: "CustomPools");
15950	json.BeginObject();
15951	{
15952	VmaMutexLockRead lock(m_PoolsMutex, m_UseMutex);
15953	if (!m_Pools.IsEmpty())
15954	{
15955	for (uint32_t memTypeIndex = 0; memTypeIndex < GetMemoryTypeCount(); ++memTypeIndex)
15956	{
15957	bool displayType = true;
15958	size_t index = 0;
15959	for (VmaPool pool = m_Pools.Front(); pool != VMA_NULL; pool = m_Pools.GetNext(item: pool))
15960	{
15961	VmaBlockVector& blockVector = pool->m_BlockVector;
15962	if (blockVector.GetMemoryTypeIndex() == memTypeIndex)
15963	{
15964	if (displayType)
15965	{
15966	json.BeginString(pStr: "Type ");
15967	json.ContinueString(n: memTypeIndex);
15968	json.EndString();
15969	json.BeginArray();
15970	displayType = false;
15971	}
15972
15973	json.BeginObject();
15974	{
15975	json.WriteString(pStr: "Name");
15976	json.BeginString();
15977	json.ContinueString_Size(n: index++);
15978	if (pool->GetName())
15979	{
15980	json.ContinueString(pStr: " - ");
15981	json.ContinueString(pStr: pool->GetName());
15982	}
15983	json.EndString();
15984
15985	json.WriteString(pStr: "PreferredBlockSize");
15986	json.WriteNumber(n: blockVector.GetPreferredBlockSize());
15987
15988	json.WriteString(pStr: "Blocks");
15989	blockVector.PrintDetailedMap(json);
15990
15991	json.WriteString(pStr: "DedicatedAllocations");
15992	pool->m_DedicatedAllocations.BuildStatsString(json);
15993	}
15994	json.EndObject();
15995	}
15996	}
15997
15998	if (!displayType)
15999	json.EndArray();
16000	}
16001	}
16002	}
16003	json.EndObject();
16004	}
16005	#endif // VMA_STATS_STRING_ENABLED
16006	#endif // _VMA_ALLOCATOR_T_FUNCTIONS
16007
16008
16009	#ifndef _VMA_PUBLIC_INTERFACE
16010	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAllocator(
16011	const VmaAllocatorCreateInfo* pCreateInfo,
16012	VmaAllocator* pAllocator)
16013	{
16014	VMA_ASSERT(pCreateInfo && pAllocator);
16015	VMA_ASSERT(pCreateInfo->vulkanApiVersion == 0 ||
16016	(VK_VERSION_MAJOR(pCreateInfo->vulkanApiVersion) == 1 && VK_VERSION_MINOR(pCreateInfo->vulkanApiVersion) <= 3));
16017	VMA_DEBUG_LOG("vmaCreateAllocator");
16018	*pAllocator = vma_new(pCreateInfo->pAllocationCallbacks, VmaAllocator_T)(pCreateInfo);
16019	VkResult result = (*pAllocator)->Init(pCreateInfo);
16020	if(result < 0)
16021	{
16022	vma_delete(pAllocationCallbacks: pCreateInfo->pAllocationCallbacks, ptr: *pAllocator);
16023	*pAllocator = VK_NULL_HANDLE;
16024	}
16025	return result;
16026	}
16027
16028	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyAllocator(
16029	VmaAllocator allocator)
16030	{
16031	if(allocator != VK_NULL_HANDLE)
16032	{
16033	VMA_DEBUG_LOG("vmaDestroyAllocator");
16034	VkAllocationCallbacks allocationCallbacks = allocator->m_AllocationCallbacks; // Have to copy the callbacks when destroying.
16035	vma_delete(pAllocationCallbacks: &allocationCallbacks, ptr: allocator);
16036	}
16037	}
16038
16039	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocatorInfo(VmaAllocator allocator, VmaAllocatorInfo* pAllocatorInfo)
16040	{
16041	VMA_ASSERT(allocator && pAllocatorInfo);
16042	pAllocatorInfo->instance = allocator->m_hInstance;
16043	pAllocatorInfo->physicalDevice = allocator->GetPhysicalDevice();
16044	pAllocatorInfo->device = allocator->m_hDevice;
16045	}
16046
16047	VMA_CALL_PRE void VMA_CALL_POST vmaGetPhysicalDeviceProperties(
16048	VmaAllocator allocator,
16049	const VkPhysicalDeviceProperties **ppPhysicalDeviceProperties)
16050	{
16051	VMA_ASSERT(allocator && ppPhysicalDeviceProperties);
16052	*ppPhysicalDeviceProperties = &allocator->m_PhysicalDeviceProperties;
16053	}
16054
16055	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryProperties(
16056	VmaAllocator allocator,
16057	const VkPhysicalDeviceMemoryProperties** ppPhysicalDeviceMemoryProperties)
16058	{
16059	VMA_ASSERT(allocator && ppPhysicalDeviceMemoryProperties);
16060	*ppPhysicalDeviceMemoryProperties = &allocator->m_MemProps;
16061	}
16062
16063	VMA_CALL_PRE void VMA_CALL_POST vmaGetMemoryTypeProperties(
16064	VmaAllocator allocator,
16065	uint32_t memoryTypeIndex,
16066	VkMemoryPropertyFlags* pFlags)
16067	{
16068	VMA_ASSERT(allocator && pFlags);
16069	VMA_ASSERT(memoryTypeIndex < allocator->GetMemoryTypeCount());
16070	*pFlags = allocator->m_MemProps.memoryTypes[memoryTypeIndex].propertyFlags;
16071	}
16072
16073	VMA_CALL_PRE void VMA_CALL_POST vmaSetCurrentFrameIndex(
16074	VmaAllocator allocator,
16075	uint32_t frameIndex)
16076	{
16077	VMA_ASSERT(allocator);
16078
16079	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16080
16081	allocator->SetCurrentFrameIndex(frameIndex);
16082	}
16083
16084	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateStatistics(
16085	VmaAllocator allocator,
16086	VmaTotalStatistics* pStats)
16087	{
16088	VMA_ASSERT(allocator && pStats);
16089	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16090	allocator->CalculateStatistics(pStats);
16091	}
16092
16093	VMA_CALL_PRE void VMA_CALL_POST vmaGetHeapBudgets(
16094	VmaAllocator allocator,
16095	VmaBudget* pBudgets)
16096	{
16097	VMA_ASSERT(allocator && pBudgets);
16098	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16099	allocator->GetHeapBudgets(outBudgets: pBudgets, firstHeap: 0, heapCount: allocator->GetMemoryHeapCount());
16100	}
16101
16102	#if VMA_STATS_STRING_ENABLED
16103
16104	VMA_CALL_PRE void VMA_CALL_POST vmaBuildStatsString(
16105	VmaAllocator allocator,
16106	char** ppStatsString,
16107	VkBool32 detailedMap)
16108	{
16109	VMA_ASSERT(allocator && ppStatsString);
16110	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16111
16112	VmaStringBuilder sb(allocator->GetAllocationCallbacks());
16113	{
16114	VmaBudget budgets[VK_MAX_MEMORY_HEAPS];
16115	allocator->GetHeapBudgets(outBudgets: budgets, firstHeap: 0, heapCount: allocator->GetMemoryHeapCount());
16116
16117	VmaTotalStatistics stats;
16118	allocator->CalculateStatistics(pStats: &stats);
16119
16120	VmaJsonWriter json(allocator->GetAllocationCallbacks(), sb);
16121	json.BeginObject();
16122	{
16123	json.WriteString(pStr: "General");
16124	json.BeginObject();
16125	{
16126	const VkPhysicalDeviceProperties& deviceProperties = allocator->m_PhysicalDeviceProperties;
16127	const VkPhysicalDeviceMemoryProperties& memoryProperties = allocator->m_MemProps;
16128
16129	json.WriteString(pStr: "API");
16130	json.WriteString(pStr: "Vulkan");
16131
16132	json.WriteString(pStr: "apiVersion");
16133	json.BeginString();
16134	json.ContinueString(VK_VERSION_MAJOR(deviceProperties.apiVersion));
16135	json.ContinueString(pStr: ".");
16136	json.ContinueString(VK_VERSION_MINOR(deviceProperties.apiVersion));
16137	json.ContinueString(pStr: ".");
16138	json.ContinueString(VK_VERSION_PATCH(deviceProperties.apiVersion));
16139	json.EndString();
16140
16141	json.WriteString(pStr: "GPU");
16142	json.WriteString(pStr: deviceProperties.deviceName);
16143	json.WriteString(pStr: "deviceType");
16144	json.WriteNumber(n: static_cast<uint32_t>(deviceProperties.deviceType));
16145
16146	json.WriteString(pStr: "maxMemoryAllocationCount");
16147	json.WriteNumber(n: deviceProperties.limits.maxMemoryAllocationCount);
16148	json.WriteString(pStr: "bufferImageGranularity");
16149	json.WriteNumber(n: deviceProperties.limits.bufferImageGranularity);
16150	json.WriteString(pStr: "nonCoherentAtomSize");
16151	json.WriteNumber(n: deviceProperties.limits.nonCoherentAtomSize);
16152
16153	json.WriteString(pStr: "memoryHeapCount");
16154	json.WriteNumber(n: memoryProperties.memoryHeapCount);
16155	json.WriteString(pStr: "memoryTypeCount");
16156	json.WriteNumber(n: memoryProperties.memoryTypeCount);
16157	}
16158	json.EndObject();
16159	}
16160	{
16161	json.WriteString(pStr: "Total");
16162	VmaPrintDetailedStatistics(json, stat: stats.total);
16163	}
16164	{
16165	json.WriteString(pStr: "MemoryInfo");
16166	json.BeginObject();
16167	{
16168	for (uint32_t heapIndex = 0; heapIndex < allocator->GetMemoryHeapCount(); ++heapIndex)
16169	{
16170	json.BeginString(pStr: "Heap ");
16171	json.ContinueString(n: heapIndex);
16172	json.EndString();
16173	json.BeginObject();
16174	{
16175	const VkMemoryHeap& heapInfo = allocator->m_MemProps.memoryHeaps[heapIndex];
16176	json.WriteString(pStr: "Flags");
16177	json.BeginArray(singleLine: true);
16178	{
16179	if (heapInfo.flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT)
16180	json.WriteString(pStr: "DEVICE_LOCAL");
16181	#if VMA_VULKAN_VERSION >= 1001000
16182	if (heapInfo.flags & VK_MEMORY_HEAP_MULTI_INSTANCE_BIT)
16183	json.WriteString(pStr: "MULTI_INSTANCE");
16184	#endif
16185
16186	VkMemoryHeapFlags flags = heapInfo.flags &
16187	~(VK_MEMORY_HEAP_DEVICE_LOCAL_BIT
16188	#if VMA_VULKAN_VERSION >= 1001000
16189	| VK_MEMORY_HEAP_MULTI_INSTANCE_BIT
16190	#endif
16191	);
16192	if (flags != 0)
16193	json.WriteNumber(n: flags);
16194	}
16195	json.EndArray();
16196
16197	json.WriteString(pStr: "Size");
16198	json.WriteNumber(n: heapInfo.size);
16199
16200	json.WriteString(pStr: "Budget");
16201	json.BeginObject();
16202	{
16203	json.WriteString(pStr: "BudgetBytes");
16204	json.WriteNumber(n: budgets[heapIndex].budget);
16205	json.WriteString(pStr: "UsageBytes");
16206	json.WriteNumber(n: budgets[heapIndex].usage);
16207	}
16208	json.EndObject();
16209
16210	json.WriteString(pStr: "Stats");
16211	VmaPrintDetailedStatistics(json, stat: stats.memoryHeap[heapIndex]);
16212
16213	json.WriteString(pStr: "MemoryPools");
16214	json.BeginObject();
16215	{
16216	for (uint32_t typeIndex = 0; typeIndex < allocator->GetMemoryTypeCount(); ++typeIndex)
16217	{
16218	if (allocator->MemoryTypeIndexToHeapIndex(memTypeIndex: typeIndex) == heapIndex)
16219	{
16220	json.BeginString(pStr: "Type ");
16221	json.ContinueString(n: typeIndex);
16222	json.EndString();
16223	json.BeginObject();
16224	{
16225	json.WriteString(pStr: "Flags");
16226	json.BeginArray(singleLine: true);
16227	{
16228	VkMemoryPropertyFlags flags = allocator->m_MemProps.memoryTypes[typeIndex].propertyFlags;
16229	if (flags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
16230	json.WriteString(pStr: "DEVICE_LOCAL");
16231	if (flags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
16232	json.WriteString(pStr: "HOST_VISIBLE");
16233	if (flags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)
16234	json.WriteString(pStr: "HOST_COHERENT");
16235	if (flags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT)
16236	json.WriteString(pStr: "HOST_CACHED");
16237	if (flags & VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT)
16238	json.WriteString(pStr: "LAZILY_ALLOCATED");
16239	#if VMA_VULKAN_VERSION >= 1001000
16240	if (flags & VK_MEMORY_PROPERTY_PROTECTED_BIT)
16241	json.WriteString(pStr: "PROTECTED");
16242	#endif
16243	#if VK_AMD_device_coherent_memory
16244	if (flags & VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY)
16245	json.WriteString(pStr: "DEVICE_COHERENT_AMD");
16246	if (flags & VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY)
16247	json.WriteString(pStr: "DEVICE_UNCACHED_AMD");
16248	#endif
16249
16250	flags &= ~(VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT
16251	#if VMA_VULKAN_VERSION >= 1001000
16252	| VK_MEMORY_PROPERTY_LAZILY_ALLOCATED_BIT
16253	#endif
16254	#if VK_AMD_device_coherent_memory
16255	| VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD_COPY
16256	| VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD_COPY
16257	#endif
16258	| VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT
16259	| VK_MEMORY_PROPERTY_HOST_COHERENT_BIT
16260	| VK_MEMORY_PROPERTY_HOST_CACHED_BIT);
16261	if (flags != 0)
16262	json.WriteNumber(n: flags);
16263	}
16264	json.EndArray();
16265
16266	json.WriteString(pStr: "Stats");
16267	VmaPrintDetailedStatistics(json, stat: stats.memoryType[typeIndex]);
16268	}
16269	json.EndObject();
16270	}
16271	}
16272
16273	}
16274	json.EndObject();
16275	}
16276	json.EndObject();
16277	}
16278	}
16279	json.EndObject();
16280	}
16281
16282	if (detailedMap == VK_TRUE)
16283	allocator->PrintDetailedMap(json);
16284
16285	json.EndObject();
16286	}
16287
16288	*ppStatsString = VmaCreateStringCopy(allocs: allocator->GetAllocationCallbacks(), srcStr: sb.GetData(), strLen: sb.GetLength());
16289	}
16290
16291	VMA_CALL_PRE void VMA_CALL_POST vmaFreeStatsString(
16292	VmaAllocator allocator,
16293	char* pStatsString)
16294	{
16295	if(pStatsString != VMA_NULL)
16296	{
16297	VMA_ASSERT(allocator);
16298	VmaFreeString(allocs: allocator->GetAllocationCallbacks(), str: pStatsString);
16299	}
16300	}
16301
16302	#endif // VMA_STATS_STRING_ENABLED
16303
16304	/*
16305	This function is not protected by any mutex because it just reads immutable data.
16306	*/
16307	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndex(
16308	VmaAllocator allocator,
16309	uint32_t memoryTypeBits,
16310	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16311	uint32_t* pMemoryTypeIndex)
16312	{
16313	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16314	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16315	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16316
16317	return allocator->FindMemoryTypeIndex(memoryTypeBits, pAllocationCreateInfo, UINT32_MAX, pMemoryTypeIndex);
16318	}
16319
16320	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForBufferInfo(
16321	VmaAllocator allocator,
16322	const VkBufferCreateInfo* pBufferCreateInfo,
16323	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16324	uint32_t* pMemoryTypeIndex)
16325	{
16326	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16327	VMA_ASSERT(pBufferCreateInfo != VMA_NULL);
16328	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16329	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16330
16331	const VkDevice hDev = allocator->m_hDevice;
16332	const VmaVulkanFunctions* funcs = &allocator->GetVulkanFunctions();
16333	VkResult res;
16334
16335	#if VMA_VULKAN_VERSION >= 1003000
16336	if(funcs->vkGetDeviceBufferMemoryRequirements)
16337	{
16338	// Can query straight from VkBufferCreateInfo :)
16339	VkDeviceBufferMemoryRequirements devBufMemReq = {.sType: VK_STRUCTURE_TYPE_DEVICE_BUFFER_MEMORY_REQUIREMENTS};
16340	devBufMemReq.pCreateInfo = pBufferCreateInfo;
16341
16342	VkMemoryRequirements2 memReq = {.sType: VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2};
16343	(*funcs->vkGetDeviceBufferMemoryRequirements)(hDev, &devBufMemReq, &memReq);
16344
16345	res = allocator->FindMemoryTypeIndex(
16346	memoryTypeBits: memReq.memoryRequirements.memoryTypeBits, pAllocationCreateInfo, bufImgUsage: pBufferCreateInfo->usage, pMemoryTypeIndex);
16347	}
16348	else
16349	#endif // #if VMA_VULKAN_VERSION >= 1003000
16350	{
16351	// Must create a dummy buffer to query :(
16352	VkBuffer hBuffer = VK_NULL_HANDLE;
16353	res = funcs->vkCreateBuffer(
16354	hDev, pBufferCreateInfo, allocator->GetAllocationCallbacks(), &hBuffer);
16355	if(res == VK_SUCCESS)
16356	{
16357	VkMemoryRequirements memReq = {};
16358	funcs->vkGetBufferMemoryRequirements(hDev, hBuffer, &memReq);
16359
16360	res = allocator->FindMemoryTypeIndex(
16361	memoryTypeBits: memReq.memoryTypeBits, pAllocationCreateInfo, bufImgUsage: pBufferCreateInfo->usage, pMemoryTypeIndex);
16362
16363	funcs->vkDestroyBuffer(
16364	hDev, hBuffer, allocator->GetAllocationCallbacks());
16365	}
16366	}
16367	return res;
16368	}
16369
16370	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFindMemoryTypeIndexForImageInfo(
16371	VmaAllocator allocator,
16372	const VkImageCreateInfo* pImageCreateInfo,
16373	const VmaAllocationCreateInfo* pAllocationCreateInfo,
16374	uint32_t* pMemoryTypeIndex)
16375	{
16376	VMA_ASSERT(allocator != VK_NULL_HANDLE);
16377	VMA_ASSERT(pImageCreateInfo != VMA_NULL);
16378	VMA_ASSERT(pAllocationCreateInfo != VMA_NULL);
16379	VMA_ASSERT(pMemoryTypeIndex != VMA_NULL);
16380
16381	const VkDevice hDev = allocator->m_hDevice;
16382	const VmaVulkanFunctions* funcs = &allocator->GetVulkanFunctions();
16383	VkResult res;
16384
16385	#if VMA_VULKAN_VERSION >= 1003000
16386	if(funcs->vkGetDeviceImageMemoryRequirements)
16387	{
16388	// Can query straight from VkImageCreateInfo :)
16389	VkDeviceImageMemoryRequirements devImgMemReq = {.sType: VK_STRUCTURE_TYPE_DEVICE_IMAGE_MEMORY_REQUIREMENTS};
16390	devImgMemReq.pCreateInfo = pImageCreateInfo;
16391	VMA_ASSERT(pImageCreateInfo->tiling != VK_IMAGE_TILING_DRM_FORMAT_MODIFIER_EXT_COPY && (pImageCreateInfo->flags & VK_IMAGE_CREATE_DISJOINT_BIT_COPY) == 0 &&
16392	"Cannot use this VkImageCreateInfo with vmaFindMemoryTypeIndexForImageInfo as I don't know what to pass as VkDeviceImageMemoryRequirements::planeAspect.");
16393
16394	VkMemoryRequirements2 memReq = {.sType: VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2};
16395	(*funcs->vkGetDeviceImageMemoryRequirements)(hDev, &devImgMemReq, &memReq);
16396
16397	res = allocator->FindMemoryTypeIndex(
16398	memoryTypeBits: memReq.memoryRequirements.memoryTypeBits, pAllocationCreateInfo, bufImgUsage: pImageCreateInfo->usage, pMemoryTypeIndex);
16399	}
16400	else
16401	#endif // #if VMA_VULKAN_VERSION >= 1003000
16402	{
16403	// Must create a dummy image to query :(
16404	VkImage hImage = VK_NULL_HANDLE;
16405	res = funcs->vkCreateImage(
16406	hDev, pImageCreateInfo, allocator->GetAllocationCallbacks(), &hImage);
16407	if(res == VK_SUCCESS)
16408	{
16409	VkMemoryRequirements memReq = {};
16410	funcs->vkGetImageMemoryRequirements(hDev, hImage, &memReq);
16411
16412	res = allocator->FindMemoryTypeIndex(
16413	memoryTypeBits: memReq.memoryTypeBits, pAllocationCreateInfo, bufImgUsage: pImageCreateInfo->usage, pMemoryTypeIndex);
16414
16415	funcs->vkDestroyImage(
16416	hDev, hImage, allocator->GetAllocationCallbacks());
16417	}
16418	}
16419	return res;
16420	}
16421
16422	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreatePool(
16423	VmaAllocator allocator,
16424	const VmaPoolCreateInfo* pCreateInfo,
16425	VmaPool* pPool)
16426	{
16427	VMA_ASSERT(allocator && pCreateInfo && pPool);
16428
16429	VMA_DEBUG_LOG("vmaCreatePool");
16430
16431	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16432
16433	return allocator->CreatePool(pCreateInfo, pPool);
16434	}
16435
16436	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyPool(
16437	VmaAllocator allocator,
16438	VmaPool pool)
16439	{
16440	VMA_ASSERT(allocator);
16441
16442	if(pool == VK_NULL_HANDLE)
16443	{
16444	return;
16445	}
16446
16447	VMA_DEBUG_LOG("vmaDestroyPool");
16448
16449	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16450
16451	allocator->DestroyPool(pool);
16452	}
16453
16454	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolStatistics(
16455	VmaAllocator allocator,
16456	VmaPool pool,
16457	VmaStatistics* pPoolStats)
16458	{
16459	VMA_ASSERT(allocator && pool && pPoolStats);
16460
16461	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16462
16463	allocator->GetPoolStatistics(pool, pPoolStats);
16464	}
16465
16466	VMA_CALL_PRE void VMA_CALL_POST vmaCalculatePoolStatistics(
16467	VmaAllocator allocator,
16468	VmaPool pool,
16469	VmaDetailedStatistics* pPoolStats)
16470	{
16471	VMA_ASSERT(allocator && pool && pPoolStats);
16472
16473	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16474
16475	allocator->CalculatePoolStatistics(pool, pPoolStats);
16476	}
16477
16478	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckPoolCorruption(VmaAllocator allocator, VmaPool pool)
16479	{
16480	VMA_ASSERT(allocator && pool);
16481
16482	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16483
16484	VMA_DEBUG_LOG("vmaCheckPoolCorruption");
16485
16486	return allocator->CheckPoolCorruption(hPool: pool);
16487	}
16488
16489	VMA_CALL_PRE void VMA_CALL_POST vmaGetPoolName(
16490	VmaAllocator allocator,
16491	VmaPool pool,
16492	const char** ppName)
16493	{
16494	VMA_ASSERT(allocator && pool && ppName);
16495
16496	VMA_DEBUG_LOG("vmaGetPoolName");
16497
16498	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16499
16500	*ppName = pool->GetName();
16501	}
16502
16503	VMA_CALL_PRE void VMA_CALL_POST vmaSetPoolName(
16504	VmaAllocator allocator,
16505	VmaPool pool,
16506	const char* pName)
16507	{
16508	VMA_ASSERT(allocator && pool);
16509
16510	VMA_DEBUG_LOG("vmaSetPoolName");
16511
16512	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16513
16514	pool->SetName(pName);
16515	}
16516
16517	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemory(
16518	VmaAllocator allocator,
16519	const VkMemoryRequirements* pVkMemoryRequirements,
16520	const VmaAllocationCreateInfo* pCreateInfo,
16521	VmaAllocation* pAllocation,
16522	VmaAllocationInfo* pAllocationInfo)
16523	{
16524	VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocation);
16525
16526	VMA_DEBUG_LOG("vmaAllocateMemory");
16527
16528	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16529
16530	VkResult result = allocator->AllocateMemory(
16531	vkMemReq: *pVkMemoryRequirements,
16532	requiresDedicatedAllocation: false, // requiresDedicatedAllocation
16533	prefersDedicatedAllocation: false, // prefersDedicatedAllocation
16534	VK_NULL_HANDLE, // dedicatedBuffer
16535	VK_NULL_HANDLE, // dedicatedImage
16536	UINT32_MAX, // dedicatedBufferImageUsage
16537	createInfo: *pCreateInfo,
16538	suballocType: VMA_SUBALLOCATION_TYPE_UNKNOWN,
16539	allocationCount: 1, // allocationCount
16540	pAllocations: pAllocation);
16541
16542	if(pAllocationInfo != VMA_NULL && result == VK_SUCCESS)
16543	{
16544	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
16545	}
16546
16547	return result;
16548	}
16549
16550	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryPages(
16551	VmaAllocator allocator,
16552	const VkMemoryRequirements* pVkMemoryRequirements,
16553	const VmaAllocationCreateInfo* pCreateInfo,
16554	size_t allocationCount,
16555	VmaAllocation* pAllocations,
16556	VmaAllocationInfo* pAllocationInfo)
16557	{
16558	if(allocationCount == 0)
16559	{
16560	return VK_SUCCESS;
16561	}
16562
16563	VMA_ASSERT(allocator && pVkMemoryRequirements && pCreateInfo && pAllocations);
16564
16565	VMA_DEBUG_LOG("vmaAllocateMemoryPages");
16566
16567	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16568
16569	VkResult result = allocator->AllocateMemory(
16570	vkMemReq: *pVkMemoryRequirements,
16571	requiresDedicatedAllocation: false, // requiresDedicatedAllocation
16572	prefersDedicatedAllocation: false, // prefersDedicatedAllocation
16573	VK_NULL_HANDLE, // dedicatedBuffer
16574	VK_NULL_HANDLE, // dedicatedImage
16575	UINT32_MAX, // dedicatedBufferImageUsage
16576	createInfo: *pCreateInfo,
16577	suballocType: VMA_SUBALLOCATION_TYPE_UNKNOWN,
16578	allocationCount,
16579	pAllocations);
16580
16581	if(pAllocationInfo != VMA_NULL && result == VK_SUCCESS)
16582	{
16583	for(size_t i = 0; i < allocationCount; ++i)
16584	{
16585	allocator->GetAllocationInfo(hAllocation: pAllocations[i], pAllocationInfo: pAllocationInfo + i);
16586	}
16587	}
16588
16589	return result;
16590	}
16591
16592	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForBuffer(
16593	VmaAllocator allocator,
16594	VkBuffer buffer,
16595	const VmaAllocationCreateInfo* pCreateInfo,
16596	VmaAllocation* pAllocation,
16597	VmaAllocationInfo* pAllocationInfo)
16598	{
16599	VMA_ASSERT(allocator && buffer != VK_NULL_HANDLE && pCreateInfo && pAllocation);
16600
16601	VMA_DEBUG_LOG("vmaAllocateMemoryForBuffer");
16602
16603	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16604
16605	VkMemoryRequirements vkMemReq = {};
16606	bool requiresDedicatedAllocation = false;
16607	bool prefersDedicatedAllocation = false;
16608	allocator->GetBufferMemoryRequirements(hBuffer: buffer, memReq&: vkMemReq,
16609	requiresDedicatedAllocation,
16610	prefersDedicatedAllocation);
16611
16612	VkResult result = allocator->AllocateMemory(
16613	vkMemReq,
16614	requiresDedicatedAllocation,
16615	prefersDedicatedAllocation,
16616	dedicatedBuffer: buffer, // dedicatedBuffer
16617	VK_NULL_HANDLE, // dedicatedImage
16618	UINT32_MAX, // dedicatedBufferImageUsage
16619	createInfo: *pCreateInfo,
16620	suballocType: VMA_SUBALLOCATION_TYPE_BUFFER,
16621	allocationCount: 1, // allocationCount
16622	pAllocations: pAllocation);
16623
16624	if(pAllocationInfo && result == VK_SUCCESS)
16625	{
16626	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
16627	}
16628
16629	return result;
16630	}
16631
16632	VMA_CALL_PRE VkResult VMA_CALL_POST vmaAllocateMemoryForImage(
16633	VmaAllocator allocator,
16634	VkImage image,
16635	const VmaAllocationCreateInfo* pCreateInfo,
16636	VmaAllocation* pAllocation,
16637	VmaAllocationInfo* pAllocationInfo)
16638	{
16639	VMA_ASSERT(allocator && image != VK_NULL_HANDLE && pCreateInfo && pAllocation);
16640
16641	VMA_DEBUG_LOG("vmaAllocateMemoryForImage");
16642
16643	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16644
16645	VkMemoryRequirements vkMemReq = {};
16646	bool requiresDedicatedAllocation = false;
16647	bool prefersDedicatedAllocation = false;
16648	allocator->GetImageMemoryRequirements(hImage: image, memReq&: vkMemReq,
16649	requiresDedicatedAllocation, prefersDedicatedAllocation);
16650
16651	VkResult result = allocator->AllocateMemory(
16652	vkMemReq,
16653	requiresDedicatedAllocation,
16654	prefersDedicatedAllocation,
16655	VK_NULL_HANDLE, // dedicatedBuffer
16656	dedicatedImage: image, // dedicatedImage
16657	UINT32_MAX, // dedicatedBufferImageUsage
16658	createInfo: *pCreateInfo,
16659	suballocType: VMA_SUBALLOCATION_TYPE_IMAGE_UNKNOWN,
16660	allocationCount: 1, // allocationCount
16661	pAllocations: pAllocation);
16662
16663	if(pAllocationInfo && result == VK_SUCCESS)
16664	{
16665	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
16666	}
16667
16668	return result;
16669	}
16670
16671	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemory(
16672	VmaAllocator allocator,
16673	VmaAllocation allocation)
16674	{
16675	VMA_ASSERT(allocator);
16676
16677	if(allocation == VK_NULL_HANDLE)
16678	{
16679	return;
16680	}
16681
16682	VMA_DEBUG_LOG("vmaFreeMemory");
16683
16684	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16685
16686	allocator->FreeMemory(
16687	allocationCount: 1, // allocationCount
16688	pAllocations: &allocation);
16689	}
16690
16691	VMA_CALL_PRE void VMA_CALL_POST vmaFreeMemoryPages(
16692	VmaAllocator allocator,
16693	size_t allocationCount,
16694	const VmaAllocation* pAllocations)
16695	{
16696	if(allocationCount == 0)
16697	{
16698	return;
16699	}
16700
16701	VMA_ASSERT(allocator);
16702
16703	VMA_DEBUG_LOG("vmaFreeMemoryPages");
16704
16705	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16706
16707	allocator->FreeMemory(allocationCount, pAllocations);
16708	}
16709
16710	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationInfo(
16711	VmaAllocator allocator,
16712	VmaAllocation allocation,
16713	VmaAllocationInfo* pAllocationInfo)
16714	{
16715	VMA_ASSERT(allocator && allocation && pAllocationInfo);
16716
16717	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16718
16719	allocator->GetAllocationInfo(hAllocation: allocation, pAllocationInfo);
16720	}
16721
16722	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationUserData(
16723	VmaAllocator allocator,
16724	VmaAllocation allocation,
16725	void* pUserData)
16726	{
16727	VMA_ASSERT(allocator && allocation);
16728
16729	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16730
16731	allocation->SetUserData(hAllocator: allocator, pUserData);
16732	}
16733
16734	VMA_CALL_PRE void VMA_CALL_POST vmaSetAllocationName(
16735	VmaAllocator VMA_NOT_NULL allocator,
16736	VmaAllocation VMA_NOT_NULL allocation,
16737	const char* VMA_NULLABLE pName)
16738	{
16739	allocation->SetName(hAllocator: allocator, pName);
16740	}
16741
16742	VMA_CALL_PRE void VMA_CALL_POST vmaGetAllocationMemoryProperties(
16743	VmaAllocator VMA_NOT_NULL allocator,
16744	VmaAllocation VMA_NOT_NULL allocation,
16745	VkMemoryPropertyFlags* VMA_NOT_NULL pFlags)
16746	{
16747	VMA_ASSERT(allocator && allocation && pFlags);
16748	const uint32_t memTypeIndex = allocation->GetMemoryTypeIndex();
16749	*pFlags = allocator->m_MemProps.memoryTypes[memTypeIndex].propertyFlags;
16750	}
16751
16752	VMA_CALL_PRE VkResult VMA_CALL_POST vmaMapMemory(
16753	VmaAllocator allocator,
16754	VmaAllocation allocation,
16755	void** ppData)
16756	{
16757	VMA_ASSERT(allocator && allocation && ppData);
16758
16759	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16760
16761	return allocator->Map(hAllocation: allocation, ppData);
16762	}
16763
16764	VMA_CALL_PRE void VMA_CALL_POST vmaUnmapMemory(
16765	VmaAllocator allocator,
16766	VmaAllocation allocation)
16767	{
16768	VMA_ASSERT(allocator && allocation);
16769
16770	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16771
16772	allocator->Unmap(hAllocation: allocation);
16773	}
16774
16775	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocation(
16776	VmaAllocator allocator,
16777	VmaAllocation allocation,
16778	VkDeviceSize offset,
16779	VkDeviceSize size)
16780	{
16781	VMA_ASSERT(allocator && allocation);
16782
16783	VMA_DEBUG_LOG("vmaFlushAllocation");
16784
16785	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16786
16787	const VkResult res = allocator->FlushOrInvalidateAllocation(hAllocation: allocation, offset, size, op: VMA_CACHE_FLUSH);
16788
16789	return res;
16790	}
16791
16792	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocation(
16793	VmaAllocator allocator,
16794	VmaAllocation allocation,
16795	VkDeviceSize offset,
16796	VkDeviceSize size)
16797	{
16798	VMA_ASSERT(allocator && allocation);
16799
16800	VMA_DEBUG_LOG("vmaInvalidateAllocation");
16801
16802	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16803
16804	const VkResult res = allocator->FlushOrInvalidateAllocation(hAllocation: allocation, offset, size, op: VMA_CACHE_INVALIDATE);
16805
16806	return res;
16807	}
16808
16809	VMA_CALL_PRE VkResult VMA_CALL_POST vmaFlushAllocations(
16810	VmaAllocator allocator,
16811	uint32_t allocationCount,
16812	const VmaAllocation* allocations,
16813	const VkDeviceSize* offsets,
16814	const VkDeviceSize* sizes)
16815	{
16816	VMA_ASSERT(allocator);
16817
16818	if(allocationCount == 0)
16819	{
16820	return VK_SUCCESS;
16821	}
16822
16823	VMA_ASSERT(allocations);
16824
16825	VMA_DEBUG_LOG("vmaFlushAllocations");
16826
16827	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16828
16829	const VkResult res = allocator->FlushOrInvalidateAllocations(allocationCount, allocations, offsets, sizes, op: VMA_CACHE_FLUSH);
16830
16831	return res;
16832	}
16833
16834	VMA_CALL_PRE VkResult VMA_CALL_POST vmaInvalidateAllocations(
16835	VmaAllocator allocator,
16836	uint32_t allocationCount,
16837	const VmaAllocation* allocations,
16838	const VkDeviceSize* offsets,
16839	const VkDeviceSize* sizes)
16840	{
16841	VMA_ASSERT(allocator);
16842
16843	if(allocationCount == 0)
16844	{
16845	return VK_SUCCESS;
16846	}
16847
16848	VMA_ASSERT(allocations);
16849
16850	VMA_DEBUG_LOG("vmaInvalidateAllocations");
16851
16852	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16853
16854	const VkResult res = allocator->FlushOrInvalidateAllocations(allocationCount, allocations, offsets, sizes, op: VMA_CACHE_INVALIDATE);
16855
16856	return res;
16857	}
16858
16859	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCheckCorruption(
16860	VmaAllocator allocator,
16861	uint32_t memoryTypeBits)
16862	{
16863	VMA_ASSERT(allocator);
16864
16865	VMA_DEBUG_LOG("vmaCheckCorruption");
16866
16867	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16868
16869	return allocator->CheckCorruption(memoryTypeBits);
16870	}
16871
16872	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentation(
16873	VmaAllocator allocator,
16874	const VmaDefragmentationInfo* pInfo,
16875	VmaDefragmentationContext* pContext)
16876	{
16877	VMA_ASSERT(allocator && pInfo && pContext);
16878
16879	VMA_DEBUG_LOG("vmaBeginDefragmentation");
16880
16881	if (pInfo->pool != VMA_NULL)
16882	{
16883	// Check if run on supported algorithms
16884	if (pInfo->pool->m_BlockVector.GetAlgorithm() & VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT)
16885	return VK_ERROR_FEATURE_NOT_PRESENT;
16886	}
16887
16888	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16889
16890	*pContext = vma_new(allocator, VmaDefragmentationContext_T)(allocator, *pInfo);
16891	return VK_SUCCESS;
16892	}
16893
16894	VMA_CALL_PRE void VMA_CALL_POST vmaEndDefragmentation(
16895	VmaAllocator allocator,
16896	VmaDefragmentationContext context,
16897	VmaDefragmentationStats* pStats)
16898	{
16899	VMA_ASSERT(allocator && context);
16900
16901	VMA_DEBUG_LOG("vmaEndDefragmentation");
16902
16903	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16904
16905	if (pStats)
16906	context->GetStats(outStats&: *pStats);
16907	vma_delete(hAllocator: allocator, ptr: context);
16908	}
16909
16910	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBeginDefragmentationPass(
16911	VmaAllocator VMA_NOT_NULL allocator,
16912	VmaDefragmentationContext VMA_NOT_NULL context,
16913	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo)
16914	{
16915	VMA_ASSERT(context && pPassInfo);
16916
16917	VMA_DEBUG_LOG("vmaBeginDefragmentationPass");
16918
16919	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16920
16921	return context->DefragmentPassBegin(moveInfo&: *pPassInfo);
16922	}
16923
16924	VMA_CALL_PRE VkResult VMA_CALL_POST vmaEndDefragmentationPass(
16925	VmaAllocator VMA_NOT_NULL allocator,
16926	VmaDefragmentationContext VMA_NOT_NULL context,
16927	VmaDefragmentationPassMoveInfo* VMA_NOT_NULL pPassInfo)
16928	{
16929	VMA_ASSERT(context && pPassInfo);
16930
16931	VMA_DEBUG_LOG("vmaEndDefragmentationPass");
16932
16933	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16934
16935	return context->DefragmentPassEnd(moveInfo&: *pPassInfo);
16936	}
16937
16938	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory(
16939	VmaAllocator allocator,
16940	VmaAllocation allocation,
16941	VkBuffer buffer)
16942	{
16943	VMA_ASSERT(allocator && allocation && buffer);
16944
16945	VMA_DEBUG_LOG("vmaBindBufferMemory");
16946
16947	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16948
16949	return allocator->BindBufferMemory(hAllocation: allocation, allocationLocalOffset: 0, hBuffer: buffer, VMA_NULL);
16950	}
16951
16952	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindBufferMemory2(
16953	VmaAllocator allocator,
16954	VmaAllocation allocation,
16955	VkDeviceSize allocationLocalOffset,
16956	VkBuffer buffer,
16957	const void* pNext)
16958	{
16959	VMA_ASSERT(allocator && allocation && buffer);
16960
16961	VMA_DEBUG_LOG("vmaBindBufferMemory2");
16962
16963	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16964
16965	return allocator->BindBufferMemory(hAllocation: allocation, allocationLocalOffset, hBuffer: buffer, pNext);
16966	}
16967
16968	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory(
16969	VmaAllocator allocator,
16970	VmaAllocation allocation,
16971	VkImage image)
16972	{
16973	VMA_ASSERT(allocator && allocation && image);
16974
16975	VMA_DEBUG_LOG("vmaBindImageMemory");
16976
16977	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16978
16979	return allocator->BindImageMemory(hAllocation: allocation, allocationLocalOffset: 0, hImage: image, VMA_NULL);
16980	}
16981
16982	VMA_CALL_PRE VkResult VMA_CALL_POST vmaBindImageMemory2(
16983	VmaAllocator allocator,
16984	VmaAllocation allocation,
16985	VkDeviceSize allocationLocalOffset,
16986	VkImage image,
16987	const void* pNext)
16988	{
16989	VMA_ASSERT(allocator && allocation && image);
16990
16991	VMA_DEBUG_LOG("vmaBindImageMemory2");
16992
16993	VMA_DEBUG_GLOBAL_MUTEX_LOCK
16994
16995	return allocator->BindImageMemory(hAllocation: allocation, allocationLocalOffset, hImage: image, pNext);
16996	}
16997
16998	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBuffer(
16999	VmaAllocator allocator,
17000	const VkBufferCreateInfo* pBufferCreateInfo,
17001	const VmaAllocationCreateInfo* pAllocationCreateInfo,
17002	VkBuffer* pBuffer,
17003	VmaAllocation* pAllocation,
17004	VmaAllocationInfo* pAllocationInfo)
17005	{
17006	VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && pBuffer && pAllocation);
17007
17008	if(pBufferCreateInfo->size == 0)
17009	{
17010	return VK_ERROR_INITIALIZATION_FAILED;
17011	}
17012	if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != 0 &&
17013	!allocator->m_UseKhrBufferDeviceAddress)
17014	{
17015	VMA_ASSERT(0 && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17016	return VK_ERROR_INITIALIZATION_FAILED;
17017	}
17018
17019	VMA_DEBUG_LOG("vmaCreateBuffer");
17020
17021	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17022
17023	*pBuffer = VK_NULL_HANDLE;
17024	*pAllocation = VK_NULL_HANDLE;
17025
17026	// 1. Create VkBuffer.
17027	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17028	allocator->m_hDevice,
17029	pBufferCreateInfo,
17030	allocator->GetAllocationCallbacks(),
17031	pBuffer);
17032	if(res >= 0)
17033	{
17034	// 2. vkGetBufferMemoryRequirements.
17035	VkMemoryRequirements vkMemReq = {};
17036	bool requiresDedicatedAllocation = false;
17037	bool prefersDedicatedAllocation = false;
17038	allocator->GetBufferMemoryRequirements(hBuffer: *pBuffer, memReq&: vkMemReq,
17039	requiresDedicatedAllocation, prefersDedicatedAllocation);
17040
17041	// 3. Allocate memory using allocator.
17042	res = allocator->AllocateMemory(
17043	vkMemReq,
17044	requiresDedicatedAllocation,
17045	prefersDedicatedAllocation,
17046	dedicatedBuffer: *pBuffer, // dedicatedBuffer
17047	VK_NULL_HANDLE, // dedicatedImage
17048	dedicatedBufferImageUsage: pBufferCreateInfo->usage, // dedicatedBufferImageUsage
17049	createInfo: *pAllocationCreateInfo,
17050	suballocType: VMA_SUBALLOCATION_TYPE_BUFFER,
17051	allocationCount: 1, // allocationCount
17052	pAllocations: pAllocation);
17053
17054	if(res >= 0)
17055	{
17056	// 3. Bind buffer with memory.
17057	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == 0)
17058	{
17059	res = allocator->BindBufferMemory(hAllocation: *pAllocation, allocationLocalOffset: 0, hBuffer: *pBuffer, VMA_NULL);
17060	}
17061	if(res >= 0)
17062	{
17063	// All steps succeeded.
17064	#if VMA_STATS_STRING_ENABLED
17065	(*pAllocation)->InitBufferImageUsage(bufferImageUsage: pBufferCreateInfo->usage);
17066	#endif
17067	if(pAllocationInfo != VMA_NULL)
17068	{
17069	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
17070	}
17071
17072	return VK_SUCCESS;
17073	}
17074	allocator->FreeMemory(
17075	allocationCount: 1, // allocationCount
17076	pAllocations: pAllocation);
17077	*pAllocation = VK_NULL_HANDLE;
17078	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
17079	*pBuffer = VK_NULL_HANDLE;
17080	return res;
17081	}
17082	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
17083	*pBuffer = VK_NULL_HANDLE;
17084	return res;
17085	}
17086	return res;
17087	}
17088
17089	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateBufferWithAlignment(
17090	VmaAllocator allocator,
17091	const VkBufferCreateInfo* pBufferCreateInfo,
17092	const VmaAllocationCreateInfo* pAllocationCreateInfo,
17093	VkDeviceSize minAlignment,
17094	VkBuffer* pBuffer,
17095	VmaAllocation* pAllocation,
17096	VmaAllocationInfo* pAllocationInfo)
17097	{
17098	VMA_ASSERT(allocator && pBufferCreateInfo && pAllocationCreateInfo && VmaIsPow2(minAlignment) && pBuffer && pAllocation);
17099
17100	if(pBufferCreateInfo->size == 0)
17101	{
17102	return VK_ERROR_INITIALIZATION_FAILED;
17103	}
17104	if((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != 0 &&
17105	!allocator->m_UseKhrBufferDeviceAddress)
17106	{
17107	VMA_ASSERT(0 && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17108	return VK_ERROR_INITIALIZATION_FAILED;
17109	}
17110
17111	VMA_DEBUG_LOG("vmaCreateBufferWithAlignment");
17112
17113	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17114
17115	*pBuffer = VK_NULL_HANDLE;
17116	*pAllocation = VK_NULL_HANDLE;
17117
17118	// 1. Create VkBuffer.
17119	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17120	allocator->m_hDevice,
17121	pBufferCreateInfo,
17122	allocator->GetAllocationCallbacks(),
17123	pBuffer);
17124	if(res >= 0)
17125	{
17126	// 2. vkGetBufferMemoryRequirements.
17127	VkMemoryRequirements vkMemReq = {};
17128	bool requiresDedicatedAllocation = false;
17129	bool prefersDedicatedAllocation = false;
17130	allocator->GetBufferMemoryRequirements(hBuffer: *pBuffer, memReq&: vkMemReq,
17131	requiresDedicatedAllocation, prefersDedicatedAllocation);
17132
17133	// 2a. Include minAlignment
17134	vkMemReq.alignment = VMA_MAX(vkMemReq.alignment, minAlignment);
17135
17136	// 3. Allocate memory using allocator.
17137	res = allocator->AllocateMemory(
17138	vkMemReq,
17139	requiresDedicatedAllocation,
17140	prefersDedicatedAllocation,
17141	dedicatedBuffer: *pBuffer, // dedicatedBuffer
17142	VK_NULL_HANDLE, // dedicatedImage
17143	dedicatedBufferImageUsage: pBufferCreateInfo->usage, // dedicatedBufferImageUsage
17144	createInfo: *pAllocationCreateInfo,
17145	suballocType: VMA_SUBALLOCATION_TYPE_BUFFER,
17146	allocationCount: 1, // allocationCount
17147	pAllocations: pAllocation);
17148
17149	if(res >= 0)
17150	{
17151	// 3. Bind buffer with memory.
17152	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == 0)
17153	{
17154	res = allocator->BindBufferMemory(hAllocation: *pAllocation, allocationLocalOffset: 0, hBuffer: *pBuffer, VMA_NULL);
17155	}
17156	if(res >= 0)
17157	{
17158	// All steps succeeded.
17159	#if VMA_STATS_STRING_ENABLED
17160	(*pAllocation)->InitBufferImageUsage(bufferImageUsage: pBufferCreateInfo->usage);
17161	#endif
17162	if(pAllocationInfo != VMA_NULL)
17163	{
17164	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
17165	}
17166
17167	return VK_SUCCESS;
17168	}
17169	allocator->FreeMemory(
17170	allocationCount: 1, // allocationCount
17171	pAllocations: pAllocation);
17172	*pAllocation = VK_NULL_HANDLE;
17173	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
17174	*pBuffer = VK_NULL_HANDLE;
17175	return res;
17176	}
17177	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
17178	*pBuffer = VK_NULL_HANDLE;
17179	return res;
17180	}
17181	return res;
17182	}
17183
17184	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingBuffer(
17185	VmaAllocator VMA_NOT_NULL allocator,
17186	VmaAllocation VMA_NOT_NULL allocation,
17187	const VkBufferCreateInfo* VMA_NOT_NULL pBufferCreateInfo,
17188	VkBuffer VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pBuffer)
17189	{
17190	VMA_ASSERT(allocator && pBufferCreateInfo && pBuffer && allocation);
17191
17192	VMA_DEBUG_LOG("vmaCreateAliasingBuffer");
17193
17194	*pBuffer = VK_NULL_HANDLE;
17195
17196	if (pBufferCreateInfo->size == 0)
17197	{
17198	return VK_ERROR_INITIALIZATION_FAILED;
17199	}
17200	if ((pBufferCreateInfo->usage & VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT_COPY) != 0 &&
17201	!allocator->m_UseKhrBufferDeviceAddress)
17202	{
17203	VMA_ASSERT(0 && "Creating a buffer with VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT is not valid if VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT was not used.");
17204	return VK_ERROR_INITIALIZATION_FAILED;
17205	}
17206
17207	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17208
17209	// 1. Create VkBuffer.
17210	VkResult res = (*allocator->GetVulkanFunctions().vkCreateBuffer)(
17211	allocator->m_hDevice,
17212	pBufferCreateInfo,
17213	allocator->GetAllocationCallbacks(),
17214	pBuffer);
17215	if (res >= 0)
17216	{
17217	// 2. Bind buffer with memory.
17218	res = allocator->BindBufferMemory(hAllocation: allocation, allocationLocalOffset: 0, hBuffer: *pBuffer, VMA_NULL);
17219	if (res >= 0)
17220	{
17221	return VK_SUCCESS;
17222	}
17223	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, *pBuffer, allocator->GetAllocationCallbacks());
17224	}
17225	return res;
17226	}
17227
17228	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyBuffer(
17229	VmaAllocator allocator,
17230	VkBuffer buffer,
17231	VmaAllocation allocation)
17232	{
17233	VMA_ASSERT(allocator);
17234
17235	if(buffer == VK_NULL_HANDLE && allocation == VK_NULL_HANDLE)
17236	{
17237	return;
17238	}
17239
17240	VMA_DEBUG_LOG("vmaDestroyBuffer");
17241
17242	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17243
17244	if(buffer != VK_NULL_HANDLE)
17245	{
17246	(*allocator->GetVulkanFunctions().vkDestroyBuffer)(allocator->m_hDevice, buffer, allocator->GetAllocationCallbacks());
17247	}
17248
17249	if(allocation != VK_NULL_HANDLE)
17250	{
17251	allocator->FreeMemory(
17252	allocationCount: 1, // allocationCount
17253	pAllocations: &allocation);
17254	}
17255	}
17256
17257	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateImage(
17258	VmaAllocator allocator,
17259	const VkImageCreateInfo* pImageCreateInfo,
17260	const VmaAllocationCreateInfo* pAllocationCreateInfo,
17261	VkImage* pImage,
17262	VmaAllocation* pAllocation,
17263	VmaAllocationInfo* pAllocationInfo)
17264	{
17265	VMA_ASSERT(allocator && pImageCreateInfo && pAllocationCreateInfo && pImage && pAllocation);
17266
17267	if(pImageCreateInfo->extent.width == 0 ||
17268	pImageCreateInfo->extent.height == 0 ||
17269	pImageCreateInfo->extent.depth == 0 ||
17270	pImageCreateInfo->mipLevels == 0 ||
17271	pImageCreateInfo->arrayLayers == 0)
17272	{
17273	return VK_ERROR_INITIALIZATION_FAILED;
17274	}
17275
17276	VMA_DEBUG_LOG("vmaCreateImage");
17277
17278	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17279
17280	*pImage = VK_NULL_HANDLE;
17281	*pAllocation = VK_NULL_HANDLE;
17282
17283	// 1. Create VkImage.
17284	VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
17285	allocator->m_hDevice,
17286	pImageCreateInfo,
17287	allocator->GetAllocationCallbacks(),
17288	pImage);
17289	if(res >= 0)
17290	{
17291	VmaSuballocationType suballocType = pImageCreateInfo->tiling == VK_IMAGE_TILING_OPTIMAL ?
17292	VMA_SUBALLOCATION_TYPE_IMAGE_OPTIMAL :
17293	VMA_SUBALLOCATION_TYPE_IMAGE_LINEAR;
17294
17295	// 2. Allocate memory using allocator.
17296	VkMemoryRequirements vkMemReq = {};
17297	bool requiresDedicatedAllocation = false;
17298	bool prefersDedicatedAllocation = false;
17299	allocator->GetImageMemoryRequirements(hImage: *pImage, memReq&: vkMemReq,
17300	requiresDedicatedAllocation, prefersDedicatedAllocation);
17301
17302	res = allocator->AllocateMemory(
17303	vkMemReq,
17304	requiresDedicatedAllocation,
17305	prefersDedicatedAllocation,
17306	VK_NULL_HANDLE, // dedicatedBuffer
17307	dedicatedImage: *pImage, // dedicatedImage
17308	dedicatedBufferImageUsage: pImageCreateInfo->usage, // dedicatedBufferImageUsage
17309	createInfo: *pAllocationCreateInfo,
17310	suballocType,
17311	allocationCount: 1, // allocationCount
17312	pAllocations: pAllocation);
17313
17314	if(res >= 0)
17315	{
17316	// 3. Bind image with memory.
17317	if((pAllocationCreateInfo->flags & VMA_ALLOCATION_CREATE_DONT_BIND_BIT) == 0)
17318	{
17319	res = allocator->BindImageMemory(hAllocation: *pAllocation, allocationLocalOffset: 0, hImage: *pImage, VMA_NULL);
17320	}
17321	if(res >= 0)
17322	{
17323	// All steps succeeded.
17324	#if VMA_STATS_STRING_ENABLED
17325	(*pAllocation)->InitBufferImageUsage(bufferImageUsage: pImageCreateInfo->usage);
17326	#endif
17327	if(pAllocationInfo != VMA_NULL)
17328	{
17329	allocator->GetAllocationInfo(hAllocation: *pAllocation, pAllocationInfo);
17330	}
17331
17332	return VK_SUCCESS;
17333	}
17334	allocator->FreeMemory(
17335	allocationCount: 1, // allocationCount
17336	pAllocations: pAllocation);
17337	*pAllocation = VK_NULL_HANDLE;
17338	(*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
17339	*pImage = VK_NULL_HANDLE;
17340	return res;
17341	}
17342	(*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
17343	*pImage = VK_NULL_HANDLE;
17344	return res;
17345	}
17346	return res;
17347	}
17348
17349	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateAliasingImage(
17350	VmaAllocator VMA_NOT_NULL allocator,
17351	VmaAllocation VMA_NOT_NULL allocation,
17352	const VkImageCreateInfo* VMA_NOT_NULL pImageCreateInfo,
17353	VkImage VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pImage)
17354	{
17355	VMA_ASSERT(allocator && pImageCreateInfo && pImage && allocation);
17356
17357	*pImage = VK_NULL_HANDLE;
17358
17359	VMA_DEBUG_LOG("vmaCreateImage");
17360
17361	if (pImageCreateInfo->extent.width == 0 ||
17362	pImageCreateInfo->extent.height == 0 ||
17363	pImageCreateInfo->extent.depth == 0 ||
17364	pImageCreateInfo->mipLevels == 0 ||
17365	pImageCreateInfo->arrayLayers == 0)
17366	{
17367	return VK_ERROR_INITIALIZATION_FAILED;
17368	}
17369
17370	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17371
17372	// 1. Create VkImage.
17373	VkResult res = (*allocator->GetVulkanFunctions().vkCreateImage)(
17374	allocator->m_hDevice,
17375	pImageCreateInfo,
17376	allocator->GetAllocationCallbacks(),
17377	pImage);
17378	if (res >= 0)
17379	{
17380	// 2. Bind image with memory.
17381	res = allocator->BindImageMemory(hAllocation: allocation, allocationLocalOffset: 0, hImage: *pImage, VMA_NULL);
17382	if (res >= 0)
17383	{
17384	return VK_SUCCESS;
17385	}
17386	(*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, *pImage, allocator->GetAllocationCallbacks());
17387	}
17388	return res;
17389	}
17390
17391	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyImage(
17392	VmaAllocator VMA_NOT_NULL allocator,
17393	VkImage VMA_NULLABLE_NON_DISPATCHABLE image,
17394	VmaAllocation VMA_NULLABLE allocation)
17395	{
17396	VMA_ASSERT(allocator);
17397
17398	if(image == VK_NULL_HANDLE && allocation == VK_NULL_HANDLE)
17399	{
17400	return;
17401	}
17402
17403	VMA_DEBUG_LOG("vmaDestroyImage");
17404
17405	VMA_DEBUG_GLOBAL_MUTEX_LOCK
17406
17407	if(image != VK_NULL_HANDLE)
17408	{
17409	(*allocator->GetVulkanFunctions().vkDestroyImage)(allocator->m_hDevice, image, allocator->GetAllocationCallbacks());
17410	}
17411	if(allocation != VK_NULL_HANDLE)
17412	{
17413	allocator->FreeMemory(
17414	allocationCount: 1, // allocationCount
17415	pAllocations: &allocation);
17416	}
17417	}
17418
17419	VMA_CALL_PRE VkResult VMA_CALL_POST vmaCreateVirtualBlock(
17420	const VmaVirtualBlockCreateInfo* VMA_NOT_NULL pCreateInfo,
17421	VmaVirtualBlock VMA_NULLABLE * VMA_NOT_NULL pVirtualBlock)
17422	{
17423	VMA_ASSERT(pCreateInfo && pVirtualBlock);
17424	VMA_ASSERT(pCreateInfo->size > 0);
17425	VMA_DEBUG_LOG("vmaCreateVirtualBlock");
17426	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17427	*pVirtualBlock = vma_new(pCreateInfo->pAllocationCallbacks, VmaVirtualBlock_T)(*pCreateInfo);
17428	VkResult res = (*pVirtualBlock)->Init();
17429	if(res < 0)
17430	{
17431	vma_delete(pAllocationCallbacks: pCreateInfo->pAllocationCallbacks, ptr: *pVirtualBlock);
17432	*pVirtualBlock = VK_NULL_HANDLE;
17433	}
17434	return res;
17435	}
17436
17437	VMA_CALL_PRE void VMA_CALL_POST vmaDestroyVirtualBlock(VmaVirtualBlock VMA_NULLABLE virtualBlock)
17438	{
17439	if(virtualBlock != VK_NULL_HANDLE)
17440	{
17441	VMA_DEBUG_LOG("vmaDestroyVirtualBlock");
17442	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17443	VkAllocationCallbacks allocationCallbacks = virtualBlock->m_AllocationCallbacks; // Have to copy the callbacks when destroying.
17444	vma_delete(pAllocationCallbacks: &allocationCallbacks, ptr: virtualBlock);
17445	}
17446	}
17447
17448	VMA_CALL_PRE VkBool32 VMA_CALL_POST vmaIsVirtualBlockEmpty(VmaVirtualBlock VMA_NOT_NULL virtualBlock)
17449	{
17450	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17451	VMA_DEBUG_LOG("vmaIsVirtualBlockEmpty");
17452	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17453	return virtualBlock->IsEmpty() ? VK_TRUE : VK_FALSE;
17454	}
17455
17456	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualAllocationInfo(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17457	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, VmaVirtualAllocationInfo* VMA_NOT_NULL pVirtualAllocInfo)
17458	{
17459	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pVirtualAllocInfo != VMA_NULL);
17460	VMA_DEBUG_LOG("vmaGetVirtualAllocationInfo");
17461	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17462	virtualBlock->GetAllocationInfo(allocation, outInfo&: *pVirtualAllocInfo);
17463	}
17464
17465	VMA_CALL_PRE VkResult VMA_CALL_POST vmaVirtualAllocate(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17466	const VmaVirtualAllocationCreateInfo* VMA_NOT_NULL pCreateInfo, VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE* VMA_NOT_NULL pAllocation,
17467	VkDeviceSize* VMA_NULLABLE pOffset)
17468	{
17469	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pCreateInfo != VMA_NULL && pAllocation != VMA_NULL);
17470	VMA_DEBUG_LOG("vmaVirtualAllocate");
17471	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17472	return virtualBlock->Allocate(createInfo: *pCreateInfo, outAllocation&: *pAllocation, outOffset: pOffset);
17473	}
17474
17475	VMA_CALL_PRE void VMA_CALL_POST vmaVirtualFree(VmaVirtualBlock VMA_NOT_NULL virtualBlock, VmaVirtualAllocation VMA_NULLABLE_NON_DISPATCHABLE allocation)
17476	{
17477	if(allocation != VK_NULL_HANDLE)
17478	{
17479	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17480	VMA_DEBUG_LOG("vmaVirtualFree");
17481	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17482	virtualBlock->Free(allocation);
17483	}
17484	}
17485
17486	VMA_CALL_PRE void VMA_CALL_POST vmaClearVirtualBlock(VmaVirtualBlock VMA_NOT_NULL virtualBlock)
17487	{
17488	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17489	VMA_DEBUG_LOG("vmaClearVirtualBlock");
17490	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17491	virtualBlock->Clear();
17492	}
17493
17494	VMA_CALL_PRE void VMA_CALL_POST vmaSetVirtualAllocationUserData(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17495	VmaVirtualAllocation VMA_NOT_NULL_NON_DISPATCHABLE allocation, void* VMA_NULLABLE pUserData)
17496	{
17497	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17498	VMA_DEBUG_LOG("vmaSetVirtualAllocationUserData");
17499	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17500	virtualBlock->SetAllocationUserData(allocation, userData: pUserData);
17501	}
17502
17503	VMA_CALL_PRE void VMA_CALL_POST vmaGetVirtualBlockStatistics(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17504	VmaStatistics* VMA_NOT_NULL pStats)
17505	{
17506	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pStats != VMA_NULL);
17507	VMA_DEBUG_LOG("vmaGetVirtualBlockStatistics");
17508	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17509	virtualBlock->GetStatistics(outStats&: *pStats);
17510	}
17511
17512	VMA_CALL_PRE void VMA_CALL_POST vmaCalculateVirtualBlockStatistics(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17513	VmaDetailedStatistics* VMA_NOT_NULL pStats)
17514	{
17515	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && pStats != VMA_NULL);
17516	VMA_DEBUG_LOG("vmaCalculateVirtualBlockStatistics");
17517	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17518	virtualBlock->CalculateDetailedStatistics(outStats&: *pStats);
17519	}
17520
17521	#if VMA_STATS_STRING_ENABLED
17522
17523	VMA_CALL_PRE void VMA_CALL_POST vmaBuildVirtualBlockStatsString(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17524	char* VMA_NULLABLE * VMA_NOT_NULL ppStatsString, VkBool32 detailedMap)
17525	{
17526	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE && ppStatsString != VMA_NULL);
17527	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17528	const VkAllocationCallbacks* allocationCallbacks = virtualBlock->GetAllocationCallbacks();
17529	VmaStringBuilder sb(allocationCallbacks);
17530	virtualBlock->BuildStatsString(detailedMap: detailedMap != VK_FALSE, sb);
17531	*ppStatsString = VmaCreateStringCopy(allocs: allocationCallbacks, srcStr: sb.GetData(), strLen: sb.GetLength());
17532	}
17533
17534	VMA_CALL_PRE void VMA_CALL_POST vmaFreeVirtualBlockStatsString(VmaVirtualBlock VMA_NOT_NULL virtualBlock,
17535	char* VMA_NULLABLE pStatsString)
17536	{
17537	if(pStatsString != VMA_NULL)
17538	{
17539	VMA_ASSERT(virtualBlock != VK_NULL_HANDLE);
17540	VMA_DEBUG_GLOBAL_MUTEX_LOCK;
17541	VmaFreeString(allocs: virtualBlock->GetAllocationCallbacks(), str: pStatsString);
17542	}
17543	}
17544	#endif // VMA_STATS_STRING_ENABLED
17545	#endif // _VMA_PUBLIC_INTERFACE
17546
17547	#if defined(__GNUC__) && !defined(__clang__)
17548	#pragma GCC diagnostic pop
17549	#elif defined(__clang__)
17550	#pragma clang diagnostic pop
17551	#endif
17552
17553	#endif // VMA_IMPLEMENTATION
17554
17555	/**
17556	\page quick_start Quick start
17557
17558	\section quick_start_project_setup Project setup
17559
17560	Vulkan Memory Allocator comes in form of a "stb-style" single header file.
17561	You don't need to build it as a separate library project.
17562	You can add this file directly to your project and submit it to code repository next to your other source files.
17563
17564	"Single header" doesn't mean that everything is contained in C/C++ declarations,
17565	like it tends to be in case of inline functions or C++ templates.
17566	It means that implementation is bundled with interface in a single file and needs to be extracted using preprocessor macro.
17567	If you don't do it properly, you will get linker errors.
17568
17569	To do it properly:
17570
17571	-# Include "vk_mem_alloc.h" file in each CPP file where you want to use the library.
17572	This includes declarations of all members of the library.
17573	-# In exactly one CPP file define following macro before this include.
17574	It enables also internal definitions.
17575
17576	\code
17577	#define VMA_IMPLEMENTATION
17578	#include "vk_mem_alloc.h"
17579	\endcode
17580
17581	It may be a good idea to create dedicated CPP file just for this purpose.
17582
17583	This library includes header `<vulkan/vulkan.h>`, which in turn
17584	includes `<windows.h>` on Windows. If you need some specific macros defined
17585	before including these headers (like `WIN32_LEAN_AND_MEAN` or
17586	`WINVER` for Windows, `VK_USE_PLATFORM_WIN32_KHR` for Vulkan), you must define
17587	them before every `#include` of this library.
17588
17589	This library is written in C++, but has C-compatible interface.
17590	Thus you can include and use vk_mem_alloc.h in C or C++ code, but full
17591	implementation with `VMA_IMPLEMENTATION` macro must be compiled as C++, NOT as C.
17592	Some features of C++14 used. STL containers, RTTI, or C++ exceptions are not used.
17593
17594
17595	\section quick_start_initialization Initialization
17596
17597	At program startup:
17598
17599	-# Initialize Vulkan to have `VkPhysicalDevice`, `VkDevice` and `VkInstance` object.
17600	-# Fill VmaAllocatorCreateInfo structure and create #VmaAllocator object by
17601	calling vmaCreateAllocator().
17602
17603	Only members `physicalDevice`, `device`, `instance` are required.
17604	However, you should inform the library which Vulkan version do you use by setting
17605	VmaAllocatorCreateInfo::vulkanApiVersion and which extensions did you enable
17606	by setting VmaAllocatorCreateInfo::flags (like #VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT for VK_KHR_buffer_device_address).
17607	Otherwise, VMA would use only features of Vulkan 1.0 core with no extensions.
17608
17609	You may need to configure importing Vulkan functions. There are 3 ways to do this:
17610
17611	-# **If you link with Vulkan static library** (e.g. "vulkan-1.lib" on Windows):
17612	- You don't need to do anything.
17613	- VMA will use these, as macro `VMA_STATIC_VULKAN_FUNCTIONS` is defined to 1 by default.
17614	-# **If you want VMA to fetch pointers to Vulkan functions dynamically** using `vkGetInstanceProcAddr`,
17615	`vkGetDeviceProcAddr` (this is the option presented in the example below):
17616	- Define `VMA_STATIC_VULKAN_FUNCTIONS` to 0, `VMA_DYNAMIC_VULKAN_FUNCTIONS` to 1.
17617	- Provide pointers to these two functions via VmaVulkanFunctions::vkGetInstanceProcAddr,
17618	VmaVulkanFunctions::vkGetDeviceProcAddr.
17619	- The library will fetch pointers to all other functions it needs internally.
17620	-# **If you fetch pointers to all Vulkan functions in a custom way**, e.g. using some loader like
17621	[Volk](https://github.com/zeux/volk):
17622	- Define `VMA_STATIC_VULKAN_FUNCTIONS` and `VMA_DYNAMIC_VULKAN_FUNCTIONS` to 0.
17623	- Pass these pointers via structure #VmaVulkanFunctions.
17624
17625	\code
17626	VmaVulkanFunctions vulkanFunctions = {};
17627	vulkanFunctions.vkGetInstanceProcAddr = &vkGetInstanceProcAddr;
17628	vulkanFunctions.vkGetDeviceProcAddr = &vkGetDeviceProcAddr;
17629
17630	VmaAllocatorCreateInfo allocatorCreateInfo = {};
17631	allocatorCreateInfo.vulkanApiVersion = VK_API_VERSION_1_2;
17632	allocatorCreateInfo.physicalDevice = physicalDevice;
17633	allocatorCreateInfo.device = device;
17634	allocatorCreateInfo.instance = instance;
17635	allocatorCreateInfo.pVulkanFunctions = &vulkanFunctions;
17636
17637	VmaAllocator allocator;
17638	vmaCreateAllocator(&allocatorCreateInfo, &allocator);
17639	\endcode
17640
17641
17642	\section quick_start_resource_allocation Resource allocation
17643
17644	When you want to create a buffer or image:
17645
17646	-# Fill `VkBufferCreateInfo` / `VkImageCreateInfo` structure.
17647	-# Fill VmaAllocationCreateInfo structure.
17648	-# Call vmaCreateBuffer() / vmaCreateImage() to get `VkBuffer`/`VkImage` with memory
17649	already allocated and bound to it, plus #VmaAllocation objects that represents its underlying memory.
17650
17651	\code
17652	VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17653	bufferInfo.size = 65536;
17654	bufferInfo.usage = VK_BUFFER_USAGE_VERTEX_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
17655
17656	VmaAllocationCreateInfo allocInfo = {};
17657	allocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17658
17659	VkBuffer buffer;
17660	VmaAllocation allocation;
17661	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17662	\endcode
17663
17664	Don't forget to destroy your objects when no longer needed:
17665
17666	\code
17667	vmaDestroyBuffer(allocator, buffer, allocation);
17668	vmaDestroyAllocator(allocator);
17669	\endcode
17670
17671
17672	\page choosing_memory_type Choosing memory type
17673
17674	Physical devices in Vulkan support various combinations of memory heaps and
17675	types. Help with choosing correct and optimal memory type for your specific
17676	resource is one of the key features of this library. You can use it by filling
17677	appropriate members of VmaAllocationCreateInfo structure, as described below.
17678	You can also combine multiple methods.
17679
17680	-# If you just want to find memory type index that meets your requirements, you
17681	can use function: vmaFindMemoryTypeIndexForBufferInfo(),
17682	vmaFindMemoryTypeIndexForImageInfo(), vmaFindMemoryTypeIndex().
17683	-# If you want to allocate a region of device memory without association with any
17684	specific image or buffer, you can use function vmaAllocateMemory(). Usage of
17685	this function is not recommended and usually not needed.
17686	vmaAllocateMemoryPages() function is also provided for creating multiple allocations at once,
17687	which may be useful for sparse binding.
17688	-# If you already have a buffer or an image created, you want to allocate memory
17689	for it and then you will bind it yourself, you can use function
17690	vmaAllocateMemoryForBuffer(), vmaAllocateMemoryForImage().
17691	For binding you should use functions: vmaBindBufferMemory(), vmaBindImageMemory()
17692	or their extended versions: vmaBindBufferMemory2(), vmaBindImageMemory2().
17693	-# **This is the easiest and recommended way to use this library:**
17694	If you want to create a buffer or an image, allocate memory for it and bind
17695	them together, all in one call, you can use function vmaCreateBuffer(),
17696	vmaCreateImage().
17697
17698	When using 3. or 4., the library internally queries Vulkan for memory types
17699	supported for that buffer or image (function `vkGetBufferMemoryRequirements()`)
17700	and uses only one of these types.
17701
17702	If no memory type can be found that meets all the requirements, these functions
17703	return `VK_ERROR_FEATURE_NOT_PRESENT`.
17704
17705	You can leave VmaAllocationCreateInfo structure completely filled with zeros.
17706	It means no requirements are specified for memory type.
17707	It is valid, although not very useful.
17708
17709	\section choosing_memory_type_usage Usage
17710
17711	The easiest way to specify memory requirements is to fill member
17712	VmaAllocationCreateInfo::usage using one of the values of enum #VmaMemoryUsage.
17713	It defines high level, common usage types.
17714	Since version 3 of the library, it is recommended to use #VMA_MEMORY_USAGE_AUTO to let it select best memory type for your resource automatically.
17715
17716	For example, if you want to create a uniform buffer that will be filled using
17717	transfer only once or infrequently and then used for rendering every frame as a uniform buffer, you can
17718	do it using following code. The buffer will most likely end up in a memory type with
17719	`VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT` to be fast to access by the GPU device.
17720
17721	\code
17722	VkBufferCreateInfo bufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17723	bufferInfo.size = 65536;
17724	bufferInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
17725
17726	VmaAllocationCreateInfo allocInfo = {};
17727	allocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17728
17729	VkBuffer buffer;
17730	VmaAllocation allocation;
17731	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17732	\endcode
17733
17734	If you have a preference for putting the resource in GPU (device) memory or CPU (host) memory
17735	on systems with discrete graphics card that have the memories separate, you can use
17736	#VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE or #VMA_MEMORY_USAGE_AUTO_PREFER_HOST.
17737
17738	When using `VMA_MEMORY_USAGE_AUTO*` while you want to map the allocated memory,
17739	you also need to specify one of the host access flags:
17740	#VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
17741	This will help the library decide about preferred memory type to ensure it has `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`
17742	so you can map it.
17743
17744	For example, a staging buffer that will be filled via mapped pointer and then
17745	used as a source of transfer to the buffer decribed previously can be created like this.
17746	It will likely and up in a memory type that is `HOST_VISIBLE` and `HOST_COHERENT`
17747	but not `HOST_CACHED` (meaning uncached, write-combined) and not `DEVICE_LOCAL` (meaning system RAM).
17748
17749	\code
17750	VkBufferCreateInfo stagingBufferInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17751	stagingBufferInfo.size = 65536;
17752	stagingBufferInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
17753
17754	VmaAllocationCreateInfo stagingAllocInfo = {};
17755	stagingAllocInfo.usage = VMA_MEMORY_USAGE_AUTO;
17756	stagingAllocInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT;
17757
17758	VkBuffer stagingBuffer;
17759	VmaAllocation stagingAllocation;
17760	vmaCreateBuffer(allocator, &stagingBufferInfo, &stagingAllocInfo, &stagingBuffer, &stagingAllocation, nullptr);
17761	\endcode
17762
17763	For more examples of creating different kinds of resources, see chapter \ref usage_patterns.
17764
17765	Usage values `VMA_MEMORY_USAGE_AUTO*` are legal to use only when the library knows
17766	about the resource being created by having `VkBufferCreateInfo` / `VkImageCreateInfo` passed,
17767	so they work with functions like: vmaCreateBuffer(), vmaCreateImage(), vmaFindMemoryTypeIndexForBufferInfo() etc.
17768	If you allocate raw memory using function vmaAllocateMemory(), you have to use other means of selecting
17769	memory type, as decribed below.
17770
17771	\note
17772	Old usage values (`VMA_MEMORY_USAGE_GPU_ONLY`, `VMA_MEMORY_USAGE_CPU_ONLY`,
17773	`VMA_MEMORY_USAGE_CPU_TO_GPU`, `VMA_MEMORY_USAGE_GPU_TO_CPU`, `VMA_MEMORY_USAGE_CPU_COPY`)
17774	are still available and work same way as in previous versions of the library
17775	for backward compatibility, but they are not recommended.
17776
17777	\section choosing_memory_type_required_preferred_flags Required and preferred flags
17778
17779	You can specify more detailed requirements by filling members
17780	VmaAllocationCreateInfo::requiredFlags and VmaAllocationCreateInfo::preferredFlags
17781	with a combination of bits from enum `VkMemoryPropertyFlags`. For example,
17782	if you want to create a buffer that will be persistently mapped on host (so it
17783	must be `HOST_VISIBLE`) and preferably will also be `HOST_COHERENT` and `HOST_CACHED`,
17784	use following code:
17785
17786	\code
17787	VmaAllocationCreateInfo allocInfo = {};
17788	allocInfo.requiredFlags = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT;
17789	allocInfo.preferredFlags = VK_MEMORY_PROPERTY_HOST_COHERENT_BIT | VK_MEMORY_PROPERTY_HOST_CACHED_BIT;
17790	allocInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT | VMA_ALLOCATION_CREATE_MAPPED_BIT;
17791
17792	VkBuffer buffer;
17793	VmaAllocation allocation;
17794	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17795	\endcode
17796
17797	A memory type is chosen that has all the required flags and as many preferred
17798	flags set as possible.
17799
17800	Value passed in VmaAllocationCreateInfo::usage is internally converted to a set of required and preferred flags,
17801	plus some extra "magic" (heuristics).
17802
17803	\section choosing_memory_type_explicit_memory_types Explicit memory types
17804
17805	If you inspected memory types available on the physical device and you have
17806	a preference for memory types that you want to use, you can fill member
17807	VmaAllocationCreateInfo::memoryTypeBits. It is a bit mask, where each bit set
17808	means that a memory type with that index is allowed to be used for the
17809	allocation. Special value 0, just like `UINT32_MAX`, means there are no
17810	restrictions to memory type index.
17811
17812	Please note that this member is NOT just a memory type index.
17813	Still you can use it to choose just one, specific memory type.
17814	For example, if you already determined that your buffer should be created in
17815	memory type 2, use following code:
17816
17817	\code
17818	uint32_t memoryTypeIndex = 2;
17819
17820	VmaAllocationCreateInfo allocInfo = {};
17821	allocInfo.memoryTypeBits = 1u << memoryTypeIndex;
17822
17823	VkBuffer buffer;
17824	VmaAllocation allocation;
17825	vmaCreateBuffer(allocator, &bufferInfo, &allocInfo, &buffer, &allocation, nullptr);
17826	\endcode
17827
17828
17829	\section choosing_memory_type_custom_memory_pools Custom memory pools
17830
17831	If you allocate from custom memory pool, all the ways of specifying memory
17832	requirements described above are not applicable and the aforementioned members
17833	of VmaAllocationCreateInfo structure are ignored. Memory type is selected
17834	explicitly when creating the pool and then used to make all the allocations from
17835	that pool. For further details, see \ref custom_memory_pools.
17836
17837	\section choosing_memory_type_dedicated_allocations Dedicated allocations
17838
17839	Memory for allocations is reserved out of larger block of `VkDeviceMemory`
17840	allocated from Vulkan internally. That is the main feature of this whole library.
17841	You can still request a separate memory block to be created for an allocation,
17842	just like you would do in a trivial solution without using any allocator.
17843	In that case, a buffer or image is always bound to that memory at offset 0.
17844	This is called a "dedicated allocation".
17845	You can explicitly request it by using flag #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
17846	The library can also internally decide to use dedicated allocation in some cases, e.g.:
17847
17848	- When the size of the allocation is large.
17849	- When [VK_KHR_dedicated_allocation](@ref vk_khr_dedicated_allocation) extension is enabled
17850	and it reports that dedicated allocation is required or recommended for the resource.
17851	- When allocation of next big memory block fails due to not enough device memory,
17852	but allocation with the exact requested size succeeds.
17853
17854
17855	\page memory_mapping Memory mapping
17856
17857	To "map memory" in Vulkan means to obtain a CPU pointer to `VkDeviceMemory`,
17858	to be able to read from it or write to it in CPU code.
17859	Mapping is possible only of memory allocated from a memory type that has
17860	`VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT` flag.
17861	Functions `vkMapMemory()`, `vkUnmapMemory()` are designed for this purpose.
17862	You can use them directly with memory allocated by this library,
17863	but it is not recommended because of following issue:
17864	Mapping the same `VkDeviceMemory` block multiple times is illegal - only one mapping at a time is allowed.
17865	This includes mapping disjoint regions. Mapping is not reference-counted internally by Vulkan.
17866	Because of this, Vulkan Memory Allocator provides following facilities:
17867
17868	\note If you want to be able to map an allocation, you need to specify one of the flags
17869	#VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT
17870	in VmaAllocationCreateInfo::flags. These flags are required for an allocation to be mappable
17871	when using #VMA_MEMORY_USAGE_AUTO or other `VMA_MEMORY_USAGE_AUTO*` enum values.
17872	For other usage values they are ignored and every such allocation made in `HOST_VISIBLE` memory type is mappable,
17873	but they can still be used for consistency.
17874
17875	\section memory_mapping_mapping_functions Mapping functions
17876
17877	The library provides following functions for mapping of a specific #VmaAllocation: vmaMapMemory(), vmaUnmapMemory().
17878	They are safer and more convenient to use than standard Vulkan functions.
17879	You can map an allocation multiple times simultaneously - mapping is reference-counted internally.
17880	You can also map different allocations simultaneously regardless of whether they use the same `VkDeviceMemory` block.
17881	The way it is implemented is that the library always maps entire memory block, not just region of the allocation.
17882	For further details, see description of vmaMapMemory() function.
17883	Example:
17884
17885	\code
17886	// Having these objects initialized:
17887	struct ConstantBuffer
17888	{
17889	...
17890	};
17891	ConstantBuffer constantBufferData = ...
17892
17893	VmaAllocator allocator = ...
17894	VkBuffer constantBuffer = ...
17895	VmaAllocation constantBufferAllocation = ...
17896
17897	// You can map and fill your buffer using following code:
17898
17899	void* mappedData;
17900	vmaMapMemory(allocator, constantBufferAllocation, &mappedData);
17901	memcpy(mappedData, &constantBufferData, sizeof(constantBufferData));
17902	vmaUnmapMemory(allocator, constantBufferAllocation);
17903	\endcode
17904
17905	When mapping, you may see a warning from Vulkan validation layer similar to this one:
17906
17907	<i>Mapping an image with layout VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL can result in undefined behavior if this memory is used by the device. Only GENERAL or PREINITIALIZED should be used.</i>
17908
17909	It happens because the library maps entire `VkDeviceMemory` block, where different
17910	types of images and buffers may end up together, especially on GPUs with unified memory like Intel.
17911	You can safely ignore it if you are sure you access only memory of the intended
17912	object that you wanted to map.
17913
17914
17915	\section memory_mapping_persistently_mapped_memory Persistently mapped memory
17916
17917	Kepping your memory persistently mapped is generally OK in Vulkan.
17918	You don't need to unmap it before using its data on the GPU.
17919	The library provides a special feature designed for that:
17920	Allocations made with #VMA_ALLOCATION_CREATE_MAPPED_BIT flag set in
17921	VmaAllocationCreateInfo::flags stay mapped all the time,
17922	so you can just access CPU pointer to it any time
17923	without a need to call any "map" or "unmap" function.
17924	Example:
17925
17926	\code
17927	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
17928	bufCreateInfo.size = sizeof(ConstantBuffer);
17929	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
17930
17931	VmaAllocationCreateInfo allocCreateInfo = {};
17932	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
17933	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
17934	VMA_ALLOCATION_CREATE_MAPPED_BIT;
17935
17936	VkBuffer buf;
17937	VmaAllocation alloc;
17938	VmaAllocationInfo allocInfo;
17939	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
17940
17941	// Buffer is already mapped. You can access its memory.
17942	memcpy(allocInfo.pMappedData, &constantBufferData, sizeof(constantBufferData));
17943	\endcode
17944
17945	\note #VMA_ALLOCATION_CREATE_MAPPED_BIT by itself doesn't guarantee that the allocation will end up
17946	in a mappable memory type.
17947	For this, you need to also specify #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT or
17948	#VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
17949	#VMA_ALLOCATION_CREATE_MAPPED_BIT only guarantees that if the memory is `HOST_VISIBLE`, the allocation will be mapped on creation.
17950	For an example of how to make use of this fact, see section \ref usage_patterns_advanced_data_uploading.
17951
17952	\section memory_mapping_cache_control Cache flush and invalidate
17953
17954	Memory in Vulkan doesn't need to be unmapped before using it on GPU,
17955	but unless a memory types has `VK_MEMORY_PROPERTY_HOST_COHERENT_BIT` flag set,
17956	you need to manually **invalidate** cache before reading of mapped pointer
17957	and **flush** cache after writing to mapped pointer.
17958	Map/unmap operations don't do that automatically.
17959	Vulkan provides following functions for this purpose `vkFlushMappedMemoryRanges()`,
17960	`vkInvalidateMappedMemoryRanges()`, but this library provides more convenient
17961	functions that refer to given allocation object: vmaFlushAllocation(),
17962	vmaInvalidateAllocation(),
17963	or multiple objects at once: vmaFlushAllocations(), vmaInvalidateAllocations().
17964
17965	Regions of memory specified for flush/invalidate must be aligned to
17966	`VkPhysicalDeviceLimits::nonCoherentAtomSize`. This is automatically ensured by the library.
17967	In any memory type that is `HOST_VISIBLE` but not `HOST_COHERENT`, all allocations
17968	within blocks are aligned to this value, so their offsets are always multiply of
17969	`nonCoherentAtomSize` and two different allocations never share same "line" of this size.
17970
17971	Also, Windows drivers from all 3 PC GPU vendors (AMD, Intel, NVIDIA)
17972	currently provide `HOST_COHERENT` flag on all memory types that are
17973	`HOST_VISIBLE`, so on PC you may not need to bother.
17974
17975
17976	\page staying_within_budget Staying within budget
17977
17978	When developing a graphics-intensive game or program, it is important to avoid allocating
17979	more GPU memory than it is physically available. When the memory is over-committed,
17980	various bad things can happen, depending on the specific GPU, graphics driver, and
17981	operating system:
17982
17983	- It may just work without any problems.
17984	- The application may slow down because some memory blocks are moved to system RAM
17985	and the GPU has to access them through PCI Express bus.
17986	- A new allocation may take very long time to complete, even few seconds, and possibly
17987	freeze entire system.
17988	- The new allocation may fail with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
17989	- It may even result in GPU crash (TDR), observed as `VK_ERROR_DEVICE_LOST`
17990	returned somewhere later.
17991
17992	\section staying_within_budget_querying_for_budget Querying for budget
17993
17994	To query for current memory usage and available budget, use function vmaGetHeapBudgets().
17995	Returned structure #VmaBudget contains quantities expressed in bytes, per Vulkan memory heap.
17996
17997	Please note that this function returns different information and works faster than
17998	vmaCalculateStatistics(). vmaGetHeapBudgets() can be called every frame or even before every
17999	allocation, while vmaCalculateStatistics() is intended to be used rarely,
18000	only to obtain statistical information, e.g. for debugging purposes.
18001
18002	It is recommended to use <b>VK_EXT_memory_budget</b> device extension to obtain information
18003	about the budget from Vulkan device. VMA is able to use this extension automatically.
18004	When not enabled, the allocator behaves same way, but then it estimates current usage
18005	and available budget based on its internal information and Vulkan memory heap sizes,
18006	which may be less precise. In order to use this extension:
18007
18008	1. Make sure extensions VK_EXT_memory_budget and VK_KHR_get_physical_device_properties2
18009	required by it are available and enable them. Please note that the first is a device
18010	extension and the second is instance extension!
18011	2. Use flag #VMA_ALLOCATOR_CREATE_EXT_MEMORY_BUDGET_BIT when creating #VmaAllocator object.
18012	3. Make sure to call vmaSetCurrentFrameIndex() every frame. Budget is queried from
18013	Vulkan inside of it to avoid overhead of querying it with every allocation.
18014
18015	\section staying_within_budget_controlling_memory_usage Controlling memory usage
18016
18017	There are many ways in which you can try to stay within the budget.
18018
18019	First, when making new allocation requires allocating a new memory block, the library
18020	tries not to exceed the budget automatically. If a block with default recommended size
18021	(e.g. 256 MB) would go over budget, a smaller block is allocated, possibly even
18022	dedicated memory for just this resource.
18023
18024	If the size of the requested resource plus current memory usage is more than the
18025	budget, by default the library still tries to create it, leaving it to the Vulkan
18026	implementation whether the allocation succeeds or fails. You can change this behavior
18027	by using #VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag. With it, the allocation is
18028	not made if it would exceed the budget or if the budget is already exceeded.
18029	VMA then tries to make the allocation from the next eligible Vulkan memory type.
18030	The all of them fail, the call then fails with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
18031	Example usage pattern may be to pass the #VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT flag
18032	when creating resources that are not essential for the application (e.g. the texture
18033	of a specific object) and not to pass it when creating critically important resources
18034	(e.g. render targets).
18035
18036	On AMD graphics cards there is a custom vendor extension available: <b>VK_AMD_memory_overallocation_behavior</b>
18037	that allows to control the behavior of the Vulkan implementation in out-of-memory cases -
18038	whether it should fail with an error code or still allow the allocation.
18039	Usage of this extension involves only passing extra structure on Vulkan device creation,
18040	so it is out of scope of this library.
18041
18042	Finally, you can also use #VMA_ALLOCATION_CREATE_NEVER_ALLOCATE_BIT flag to make sure
18043	a new allocation is created only when it fits inside one of the existing memory blocks.
18044	If it would require to allocate a new block, if fails instead with `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
18045	This also ensures that the function call is very fast because it never goes to Vulkan
18046	to obtain a new block.
18047
18048	\note Creating \ref custom_memory_pools with VmaPoolCreateInfo::minBlockCount
18049	set to more than 0 will currently try to allocate memory blocks without checking whether they
18050	fit within budget.
18051
18052
18053	\page resource_aliasing Resource aliasing (overlap)
18054
18055	New explicit graphics APIs (Vulkan and Direct3D 12), thanks to manual memory
18056	management, give an opportunity to alias (overlap) multiple resources in the
18057	same region of memory - a feature not available in the old APIs (Direct3D 11, OpenGL).
18058	It can be useful to save video memory, but it must be used with caution.
18059
18060	For example, if you know the flow of your whole render frame in advance, you
18061	are going to use some intermediate textures or buffers only during a small range of render passes,
18062	and you know these ranges don't overlap in time, you can bind these resources to
18063	the same place in memory, even if they have completely different parameters (width, height, format etc.).
18064
18065	
18066
18067	Such scenario is possible using VMA, but you need to create your images manually.
18068	Then you need to calculate parameters of an allocation to be made using formula:
18069
18070	- allocation size = max(size of each image)
18071	- allocation alignment = max(alignment of each image)
18072	- allocation memoryTypeBits = bitwise AND(memoryTypeBits of each image)
18073
18074	Following example shows two different images bound to the same place in memory,
18075	allocated to fit largest of them.
18076
18077	\code
18078	// A 512x512 texture to be sampled.
18079	VkImageCreateInfo img1CreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18080	img1CreateInfo.imageType = VK_IMAGE_TYPE_2D;
18081	img1CreateInfo.extent.width = 512;
18082	img1CreateInfo.extent.height = 512;
18083	img1CreateInfo.extent.depth = 1;
18084	img1CreateInfo.mipLevels = 10;
18085	img1CreateInfo.arrayLayers = 1;
18086	img1CreateInfo.format = VK_FORMAT_R8G8B8A8_SRGB;
18087	img1CreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18088	img1CreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18089	img1CreateInfo.usage = VK_IMAGE_USAGE_TRANSFER_DST_BIT | VK_IMAGE_USAGE_SAMPLED_BIT;
18090	img1CreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18091
18092	// A full screen texture to be used as color attachment.
18093	VkImageCreateInfo img2CreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18094	img2CreateInfo.imageType = VK_IMAGE_TYPE_2D;
18095	img2CreateInfo.extent.width = 1920;
18096	img2CreateInfo.extent.height = 1080;
18097	img2CreateInfo.extent.depth = 1;
18098	img2CreateInfo.mipLevels = 1;
18099	img2CreateInfo.arrayLayers = 1;
18100	img2CreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
18101	img2CreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18102	img2CreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18103	img2CreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
18104	img2CreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18105
18106	VkImage img1;
18107	res = vkCreateImage(device, &img1CreateInfo, nullptr, &img1);
18108	VkImage img2;
18109	res = vkCreateImage(device, &img2CreateInfo, nullptr, &img2);
18110
18111	VkMemoryRequirements img1MemReq;
18112	vkGetImageMemoryRequirements(device, img1, &img1MemReq);
18113	VkMemoryRequirements img2MemReq;
18114	vkGetImageMemoryRequirements(device, img2, &img2MemReq);
18115
18116	VkMemoryRequirements finalMemReq = {};
18117	finalMemReq.size = std::max(img1MemReq.size, img2MemReq.size);
18118	finalMemReq.alignment = std::max(img1MemReq.alignment, img2MemReq.alignment);
18119	finalMemReq.memoryTypeBits = img1MemReq.memoryTypeBits & img2MemReq.memoryTypeBits;
18120	// Validate if(finalMemReq.memoryTypeBits != 0)
18121
18122	VmaAllocationCreateInfo allocCreateInfo = {};
18123	allocCreateInfo.preferredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
18124
18125	VmaAllocation alloc;
18126	res = vmaAllocateMemory(allocator, &finalMemReq, &allocCreateInfo, &alloc, nullptr);
18127
18128	res = vmaBindImageMemory(allocator, alloc, img1);
18129	res = vmaBindImageMemory(allocator, alloc, img2);
18130
18131	// You can use img1, img2 here, but not at the same time!
18132
18133	vmaFreeMemory(allocator, alloc);
18134	vkDestroyImage(allocator, img2, nullptr);
18135	vkDestroyImage(allocator, img1, nullptr);
18136	\endcode
18137
18138	Remember that using resources that alias in memory requires proper synchronization.
18139	You need to issue a memory barrier to make sure commands that use `img1` and `img2`
18140	don't overlap on GPU timeline.
18141	You also need to treat a resource after aliasing as uninitialized - containing garbage data.
18142	For example, if you use `img1` and then want to use `img2`, you need to issue
18143	an image memory barrier for `img2` with `oldLayout` = `VK_IMAGE_LAYOUT_UNDEFINED`.
18144
18145	Additional considerations:
18146
18147	- Vulkan also allows to interpret contents of memory between aliasing resources consistently in some cases.
18148	See chapter 11.8. "Memory Aliasing" of Vulkan specification or `VK_IMAGE_CREATE_ALIAS_BIT` flag.
18149	- You can create more complex layout where different images and buffers are bound
18150	at different offsets inside one large allocation. For example, one can imagine
18151	a big texture used in some render passes, aliasing with a set of many small buffers
18152	used between in some further passes. To bind a resource at non-zero offset in an allocation,
18153	use vmaBindBufferMemory2() / vmaBindImageMemory2().
18154	- Before allocating memory for the resources you want to alias, check `memoryTypeBits`
18155	returned in memory requirements of each resource to make sure the bits overlap.
18156	Some GPUs may expose multiple memory types suitable e.g. only for buffers or
18157	images with `COLOR_ATTACHMENT` usage, so the sets of memory types supported by your
18158	resources may be disjoint. Aliasing them is not possible in that case.
18159
18160
18161	\page custom_memory_pools Custom memory pools
18162
18163	A memory pool contains a number of `VkDeviceMemory` blocks.
18164	The library automatically creates and manages default pool for each memory type available on the device.
18165	Default memory pool automatically grows in size.
18166	Size of allocated blocks is also variable and managed automatically.
18167
18168	You can create custom pool and allocate memory out of it.
18169	It can be useful if you want to:
18170
18171	- Keep certain kind of allocations separate from others.
18172	- Enforce particular, fixed size of Vulkan memory blocks.
18173	- Limit maximum amount of Vulkan memory allocated for that pool.
18174	- Reserve minimum or fixed amount of Vulkan memory always preallocated for that pool.
18175	- Use extra parameters for a set of your allocations that are available in #VmaPoolCreateInfo but not in
18176	#VmaAllocationCreateInfo - e.g., custom minimum alignment, custom `pNext` chain.
18177	- Perform defragmentation on a specific subset of your allocations.
18178
18179	To use custom memory pools:
18180
18181	-# Fill VmaPoolCreateInfo structure.
18182	-# Call vmaCreatePool() to obtain #VmaPool handle.
18183	-# When making an allocation, set VmaAllocationCreateInfo::pool to this handle.
18184	You don't need to specify any other parameters of this structure, like `usage`.
18185
18186	Example:
18187
18188	\code
18189	// Find memoryTypeIndex for the pool.
18190	VkBufferCreateInfo sampleBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18191	sampleBufCreateInfo.size = 0x10000; // Doesn't matter.
18192	sampleBufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18193
18194	VmaAllocationCreateInfo sampleAllocCreateInfo = {};
18195	sampleAllocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18196
18197	uint32_t memTypeIndex;
18198	VkResult res = vmaFindMemoryTypeIndexForBufferInfo(allocator,
18199	&sampleBufCreateInfo, &sampleAllocCreateInfo, &memTypeIndex);
18200	// Check res...
18201
18202	// Create a pool that can have at most 2 blocks, 128 MiB each.
18203	VmaPoolCreateInfo poolCreateInfo = {};
18204	poolCreateInfo.memoryTypeIndex = memTypeIndex;
18205	poolCreateInfo.blockSize = 128ull * 1024 * 1024;
18206	poolCreateInfo.maxBlockCount = 2;
18207
18208	VmaPool pool;
18209	res = vmaCreatePool(allocator, &poolCreateInfo, &pool);
18210	// Check res...
18211
18212	// Allocate a buffer out of it.
18213	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18214	bufCreateInfo.size = 1024;
18215	bufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18216
18217	VmaAllocationCreateInfo allocCreateInfo = {};
18218	allocCreateInfo.pool = pool;
18219
18220	VkBuffer buf;
18221	VmaAllocation alloc;
18222	res = vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, nullptr);
18223	// Check res...
18224	\endcode
18225
18226	You have to free all allocations made from this pool before destroying it.
18227
18228	\code
18229	vmaDestroyBuffer(allocator, buf, alloc);
18230	vmaDestroyPool(allocator, pool);
18231	\endcode
18232
18233	New versions of this library support creating dedicated allocations in custom pools.
18234	It is supported only when VmaPoolCreateInfo::blockSize = 0.
18235	To use this feature, set VmaAllocationCreateInfo::pool to the pointer to your custom pool and
18236	VmaAllocationCreateInfo::flags to #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
18237
18238	\note Excessive use of custom pools is a common mistake when using this library.
18239	Custom pools may be useful for special purposes - when you want to
18240	keep certain type of resources separate e.g. to reserve minimum amount of memory
18241	for them or limit maximum amount of memory they can occupy. For most
18242	resources this is not needed and so it is not recommended to create #VmaPool
18243	objects and allocations out of them. Allocating from the default pool is sufficient.
18244
18245
18246	\section custom_memory_pools_MemTypeIndex Choosing memory type index
18247
18248	When creating a pool, you must explicitly specify memory type index.
18249	To find the one suitable for your buffers or images, you can use helper functions
18250	vmaFindMemoryTypeIndexForBufferInfo(), vmaFindMemoryTypeIndexForImageInfo().
18251	You need to provide structures with example parameters of buffers or images
18252	that you are going to create in that pool.
18253
18254	\code
18255	VkBufferCreateInfo exampleBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18256	exampleBufCreateInfo.size = 1024; // Doesn't matter
18257	exampleBufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
18258
18259	VmaAllocationCreateInfo allocCreateInfo = {};
18260	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18261
18262	uint32_t memTypeIndex;
18263	vmaFindMemoryTypeIndexForBufferInfo(allocator, &exampleBufCreateInfo, &allocCreateInfo, &memTypeIndex);
18264
18265	VmaPoolCreateInfo poolCreateInfo = {};
18266	poolCreateInfo.memoryTypeIndex = memTypeIndex;
18267	// ...
18268	\endcode
18269
18270	When creating buffers/images allocated in that pool, provide following parameters:
18271
18272	- `VkBufferCreateInfo`: Prefer to pass same parameters as above.
18273	Otherwise you risk creating resources in a memory type that is not suitable for them, which may result in undefined behavior.
18274	Using different `VK_BUFFER_USAGE_` flags may work, but you shouldn't create images in a pool intended for buffers
18275	or the other way around.
18276	- VmaAllocationCreateInfo: You don't need to pass same parameters. Fill only `pool` member.
18277	Other members are ignored anyway.
18278
18279	\section linear_algorithm Linear allocation algorithm
18280
18281	Each Vulkan memory block managed by this library has accompanying metadata that
18282	keeps track of used and unused regions. By default, the metadata structure and
18283	algorithm tries to find best place for new allocations among free regions to
18284	optimize memory usage. This way you can allocate and free objects in any order.
18285
18286	
18287
18288	Sometimes there is a need to use simpler, linear allocation algorithm. You can
18289	create custom pool that uses such algorithm by adding flag
18290	#VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT to VmaPoolCreateInfo::flags while creating
18291	#VmaPool object. Then an alternative metadata management is used. It always
18292	creates new allocations after last one and doesn't reuse free regions after
18293	allocations freed in the middle. It results in better allocation performance and
18294	less memory consumed by metadata.
18295
18296	
18297
18298	With this one flag, you can create a custom pool that can be used in many ways:
18299	free-at-once, stack, double stack, and ring buffer. See below for details.
18300	You don't need to specify explicitly which of these options you are going to use - it is detected automatically.
18301
18302	\subsection linear_algorithm_free_at_once Free-at-once
18303
18304	In a pool that uses linear algorithm, you still need to free all the allocations
18305	individually, e.g. by using vmaFreeMemory() or vmaDestroyBuffer(). You can free
18306	them in any order. New allocations are always made after last one - free space
18307	in the middle is not reused. However, when you release all the allocation and
18308	the pool becomes empty, allocation starts from the beginning again. This way you
18309	can use linear algorithm to speed up creation of allocations that you are going
18310	to release all at once.
18311
18312	
18313
18314	This mode is also available for pools created with VmaPoolCreateInfo::maxBlockCount
18315	value that allows multiple memory blocks.
18316
18317	\subsection linear_algorithm_stack Stack
18318
18319	When you free an allocation that was created last, its space can be reused.
18320	Thanks to this, if you always release allocations in the order opposite to their
18321	creation (LIFO - Last In First Out), you can achieve behavior of a stack.
18322
18323	
18324
18325	This mode is also available for pools created with VmaPoolCreateInfo::maxBlockCount
18326	value that allows multiple memory blocks.
18327
18328	\subsection linear_algorithm_double_stack Double stack
18329
18330	The space reserved by a custom pool with linear algorithm may be used by two
18331	stacks:
18332
18333	- First, default one, growing up from offset 0.
18334	- Second, "upper" one, growing down from the end towards lower offsets.
18335
18336	To make allocation from the upper stack, add flag #VMA_ALLOCATION_CREATE_UPPER_ADDRESS_BIT
18337	to VmaAllocationCreateInfo::flags.
18338
18339	
18340
18341	Double stack is available only in pools with one memory block -
18342	VmaPoolCreateInfo::maxBlockCount must be 1. Otherwise behavior is undefined.
18343
18344	When the two stacks' ends meet so there is not enough space between them for a
18345	new allocation, such allocation fails with usual
18346	`VK_ERROR_OUT_OF_DEVICE_MEMORY` error.
18347
18348	\subsection linear_algorithm_ring_buffer Ring buffer
18349
18350	When you free some allocations from the beginning and there is not enough free space
18351	for a new one at the end of a pool, allocator's "cursor" wraps around to the
18352	beginning and starts allocation there. Thanks to this, if you always release
18353	allocations in the same order as you created them (FIFO - First In First Out),
18354	you can achieve behavior of a ring buffer / queue.
18355
18356	
18357
18358	Ring buffer is available only in pools with one memory block -
18359	VmaPoolCreateInfo::maxBlockCount must be 1. Otherwise behavior is undefined.
18360
18361	\note \ref defragmentation is not supported in custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT.
18362
18363
18364	\page defragmentation Defragmentation
18365
18366	Interleaved allocations and deallocations of many objects of varying size can
18367	cause fragmentation over time, which can lead to a situation where the library is unable
18368	to find a continuous range of free memory for a new allocation despite there is
18369	enough free space, just scattered across many small free ranges between existing
18370	allocations.
18371
18372	To mitigate this problem, you can use defragmentation feature.
18373	It doesn't happen automatically though and needs your cooperation,
18374	because VMA is a low level library that only allocates memory.
18375	It cannot recreate buffers and images in a new place as it doesn't remember the contents of `VkBufferCreateInfo` / `VkImageCreateInfo` structures.
18376	It cannot copy their contents as it doesn't record any commands to a command buffer.
18377
18378	Example:
18379
18380	\code
18381	VmaDefragmentationInfo defragInfo = {};
18382	defragInfo.pool = myPool;
18383	defragInfo.flags = VMA_DEFRAGMENTATION_FLAG_ALGORITHM_FAST_BIT;
18384
18385	VmaDefragmentationContext defragCtx;
18386	VkResult res = vmaBeginDefragmentation(allocator, &defragInfo, &defragCtx);
18387	// Check res...
18388
18389	for(;;)
18390	{
18391	VmaDefragmentationPassMoveInfo pass;
18392	res = vmaBeginDefragmentationPass(allocator, defragCtx, &pass);
18393	if(res == VK_SUCCESS)
18394	break;
18395	else if(res != VK_INCOMPLETE)
18396	// Handle error...
18397
18398	for(uint32_t i = 0; i < pass.moveCount; ++i)
18399	{
18400	// Inspect pass.pMoves[i].srcAllocation, identify what buffer/image it represents.
18401	VmaAllocationInfo allocInfo;
18402	vmaGetAllocationInfo(allocator, pMoves[i].srcAllocation, &allocInfo);
18403	MyEngineResourceData* resData = (MyEngineResourceData*)allocInfo.pUserData;
18404
18405	// Recreate and bind this buffer/image at: pass.pMoves[i].dstMemory, pass.pMoves[i].dstOffset.
18406	VkImageCreateInfo imgCreateInfo = ...
18407	VkImage newImg;
18408	res = vkCreateImage(device, &imgCreateInfo, nullptr, &newImg);
18409	// Check res...
18410	res = vmaBindImageMemory(allocator, pMoves[i].dstTmpAllocation, newImg);
18411	// Check res...
18412
18413	// Issue a vkCmdCopyBuffer/vkCmdCopyImage to copy its content to the new place.
18414	vkCmdCopyImage(cmdBuf, resData->img, ..., newImg, ...);
18415	}
18416
18417	// Make sure the copy commands finished executing.
18418	vkWaitForFences(...);
18419
18420	// Destroy old buffers/images bound with pass.pMoves[i].srcAllocation.
18421	for(uint32_t i = 0; i < pass.moveCount; ++i)
18422	{
18423	// ...
18424	vkDestroyImage(device, resData->img, nullptr);
18425	}
18426
18427	// Update appropriate descriptors to point to the new places...
18428
18429	res = vmaEndDefragmentationPass(allocator, defragCtx, &pass);
18430	if(res == VK_SUCCESS)
18431	break;
18432	else if(res != VK_INCOMPLETE)
18433	// Handle error...
18434	}
18435
18436	vmaEndDefragmentation(allocator, defragCtx, nullptr);
18437	\endcode
18438
18439	Although functions like vmaCreateBuffer(), vmaCreateImage(), vmaDestroyBuffer(), vmaDestroyImage()
18440	create/destroy an allocation and a buffer/image at once, these are just a shortcut for
18441	creating the resource, allocating memory, and binding them together.
18442	Defragmentation works on memory allocations only. You must handle the rest manually.
18443	Defragmentation is an iterative process that should repreat "passes" as long as related functions
18444	return `VK_INCOMPLETE` not `VK_SUCCESS`.
18445	In each pass:
18446
18447	1. vmaBeginDefragmentationPass() function call:
18448	- Calculates and returns the list of allocations to be moved in this pass.
18449	Note this can be a time-consuming process.
18450	- Reserves destination memory for them by creating temporary destination allocations
18451	that you can query for their `VkDeviceMemory` + offset using vmaGetAllocationInfo().
18452	2. Inside the pass, **you should**:
18453	- Inspect the returned list of allocations to be moved.
18454	- Create new buffers/images and bind them at the returned destination temporary allocations.
18455	- Copy data from source to destination resources if necessary.
18456	- Destroy the source buffers/images, but NOT their allocations.
18457	3. vmaEndDefragmentationPass() function call:
18458	- Frees the source memory reserved for the allocations that are moved.
18459	- Modifies source #VmaAllocation objects that are moved to point to the destination reserved memory.
18460	- Frees `VkDeviceMemory` blocks that became empty.
18461
18462	Unlike in previous iterations of the defragmentation API, there is no list of "movable" allocations passed as a parameter.
18463	Defragmentation algorithm tries to move all suitable allocations.
18464	You can, however, refuse to move some of them inside a defragmentation pass, by setting
18465	`pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
18466	This is not recommended and may result in suboptimal packing of the allocations after defragmentation.
18467	If you cannot ensure any allocation can be moved, it is better to keep movable allocations separate in a custom pool.
18468
18469	Inside a pass, for each allocation that should be moved:
18470
18471	- You should copy its data from the source to the destination place by calling e.g. `vkCmdCopyBuffer()`, `vkCmdCopyImage()`.
18472	- You need to make sure these commands finished executing before destroying the source buffers/images and before calling vmaEndDefragmentationPass().
18473	- If a resource doesn't contain any meaningful data, e.g. it is a transient color attachment image to be cleared,
18474	filled, and used temporarily in each rendering frame, you can just recreate this image
18475	without copying its data.
18476	- If the resource is in `HOST_VISIBLE` and `HOST_CACHED` memory, you can copy its data on the CPU
18477	using `memcpy()`.
18478	- If you cannot move the allocation, you can set `pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_IGNORE.
18479	This will cancel the move.
18480	- vmaEndDefragmentationPass() will then free the destination memory
18481	not the source memory of the allocation, leaving it unchanged.
18482	- If you decide the allocation is unimportant and can be destroyed instead of moved (e.g. it wasn't used for long time),
18483	you can set `pass.pMoves[i].operation` to #VMA_DEFRAGMENTATION_MOVE_OPERATION_DESTROY.
18484	- vmaEndDefragmentationPass() will then free both source and destination memory, and will destroy the source #VmaAllocation object.
18485
18486	You can defragment a specific custom pool by setting VmaDefragmentationInfo::pool
18487	(like in the example above) or all the default pools by setting this member to null.
18488
18489	Defragmentation is always performed in each pool separately.
18490	Allocations are never moved between different Vulkan memory types.
18491	The size of the destination memory reserved for a moved allocation is the same as the original one.
18492	Alignment of an allocation as it was determined using `vkGetBufferMemoryRequirements()` etc. is also respected after defragmentation.
18493	Buffers/images should be recreated with the same `VkBufferCreateInfo` / `VkImageCreateInfo` parameters as the original ones.
18494
18495	You can perform the defragmentation incrementally to limit the number of allocations and bytes to be moved
18496	in each pass, e.g. to call it in sync with render frames and not to experience too big hitches.
18497	See members: VmaDefragmentationInfo::maxBytesPerPass, VmaDefragmentationInfo::maxAllocationsPerPass.
18498
18499	It is also safe to perform the defragmentation asynchronously to render frames and other Vulkan and VMA
18500	usage, possibly from multiple threads, with the exception that allocations
18501	returned in VmaDefragmentationPassMoveInfo::pMoves shouldn't be destroyed until the defragmentation pass is ended.
18502
18503	<b>Mapping</b> is preserved on allocations that are moved during defragmentation.
18504	Whether through #VMA_ALLOCATION_CREATE_MAPPED_BIT or vmaMapMemory(), the allocations
18505	are mapped at their new place. Of course, pointer to the mapped data changes, so it needs to be queried
18506	using VmaAllocationInfo::pMappedData.
18507
18508	\note Defragmentation is not supported in custom pools created with #VMA_POOL_CREATE_LINEAR_ALGORITHM_BIT.
18509
18510
18511	\page statistics Statistics
18512
18513	This library contains several functions that return information about its internal state,
18514	especially the amount of memory allocated from Vulkan.
18515
18516	\section statistics_numeric_statistics Numeric statistics
18517
18518	If you need to obtain basic statistics about memory usage per heap, together with current budget,
18519	you can call function vmaGetHeapBudgets() and inspect structure #VmaBudget.
18520	This is useful to keep track of memory usage and stay withing budget
18521	(see also \ref staying_within_budget).
18522	Example:
18523
18524	\code
18525	uint32_t heapIndex = ...
18526
18527	VmaBudget budgets[VK_MAX_MEMORY_HEAPS];
18528	vmaGetHeapBudgets(allocator, budgets);
18529
18530	printf("My heap currently has %u allocations taking %llu B,\n",
18531	budgets[heapIndex].statistics.allocationCount,
18532	budgets[heapIndex].statistics.allocationBytes);
18533	printf("allocated out of %u Vulkan device memory blocks taking %llu B,\n",
18534	budgets[heapIndex].statistics.blockCount,
18535	budgets[heapIndex].statistics.blockBytes);
18536	printf("Vulkan reports total usage %llu B with budget %llu B.\n",
18537	budgets[heapIndex].usage,
18538	budgets[heapIndex].budget);
18539	\endcode
18540
18541	You can query for more detailed statistics per memory heap, type, and totals,
18542	including minimum and maximum allocation size and unused range size,
18543	by calling function vmaCalculateStatistics() and inspecting structure #VmaTotalStatistics.
18544	This function is slower though, as it has to traverse all the internal data structures,
18545	so it should be used only for debugging purposes.
18546
18547	You can query for statistics of a custom pool using function vmaGetPoolStatistics()
18548	or vmaCalculatePoolStatistics().
18549
18550	You can query for information about a specific allocation using function vmaGetAllocationInfo().
18551	It fill structure #VmaAllocationInfo.
18552
18553	\section statistics_json_dump JSON dump
18554
18555	You can dump internal state of the allocator to a string in JSON format using function vmaBuildStatsString().
18556	The result is guaranteed to be correct JSON.
18557	It uses ANSI encoding.
18558	Any strings provided by user (see [Allocation names](@ref allocation_names))
18559	are copied as-is and properly escaped for JSON, so if they use UTF-8, ISO-8859-2 or any other encoding,
18560	this JSON string can be treated as using this encoding.
18561	It must be freed using function vmaFreeStatsString().
18562
18563	The format of this JSON string is not part of official documentation of the library,
18564	but it will not change in backward-incompatible way without increasing library major version number
18565	and appropriate mention in changelog.
18566
18567	The JSON string contains all the data that can be obtained using vmaCalculateStatistics().
18568	It can also contain detailed map of allocated memory blocks and their regions -
18569	free and occupied by allocations.
18570	This allows e.g. to visualize the memory or assess fragmentation.
18571
18572
18573	\page allocation_annotation Allocation names and user data
18574
18575	\section allocation_user_data Allocation user data
18576
18577	You can annotate allocations with your own information, e.g. for debugging purposes.
18578	To do that, fill VmaAllocationCreateInfo::pUserData field when creating
18579	an allocation. It is an opaque `void*` pointer. You can use it e.g. as a pointer,
18580	some handle, index, key, ordinal number or any other value that would associate
18581	the allocation with your custom metadata.
18582	It it useful to identify appropriate data structures in your engine given #VmaAllocation,
18583	e.g. when doing \ref defragmentation.
18584
18585	\code
18586	VkBufferCreateInfo bufCreateInfo = ...
18587
18588	MyBufferMetadata* pMetadata = CreateBufferMetadata();
18589
18590	VmaAllocationCreateInfo allocCreateInfo = {};
18591	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18592	allocCreateInfo.pUserData = pMetadata;
18593
18594	VkBuffer buffer;
18595	VmaAllocation allocation;
18596	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buffer, &allocation, nullptr);
18597	\endcode
18598
18599	The pointer may be later retrieved as VmaAllocationInfo::pUserData:
18600
18601	\code
18602	VmaAllocationInfo allocInfo;
18603	vmaGetAllocationInfo(allocator, allocation, &allocInfo);
18604	MyBufferMetadata* pMetadata = (MyBufferMetadata*)allocInfo.pUserData;
18605	\endcode
18606
18607	It can also be changed using function vmaSetAllocationUserData().
18608
18609	Values of (non-zero) allocations' `pUserData` are printed in JSON report created by
18610	vmaBuildStatsString() in hexadecimal form.
18611
18612	\section allocation_names Allocation names
18613
18614	An allocation can also carry a null-terminated string, giving a name to the allocation.
18615	To set it, call vmaSetAllocationName().
18616	The library creates internal copy of the string, so the pointer you pass doesn't need
18617	to be valid for whole lifetime of the allocation. You can free it after the call.
18618
18619	\code
18620	std::string imageName = "Texture: ";
18621	imageName += fileName;
18622	vmaSetAllocationName(allocator, allocation, imageName.c_str());
18623	\endcode
18624
18625	The string can be later retrieved by inspecting VmaAllocationInfo::pName.
18626	It is also printed in JSON report created by vmaBuildStatsString().
18627
18628	\note Setting string name to VMA allocation doesn't automatically set it to the Vulkan buffer or image created with it.
18629	You must do it manually using an extension like VK_EXT_debug_utils, which is independent of this library.
18630
18631
18632	\page virtual_allocator Virtual allocator
18633
18634	As an extra feature, the core allocation algorithm of the library is exposed through a simple and convenient API of "virtual allocator".
18635	It doesn't allocate any real GPU memory. It just keeps track of used and free regions of a "virtual block".
18636	You can use it to allocate your own memory or other objects, even completely unrelated to Vulkan.
18637	A common use case is sub-allocation of pieces of one large GPU buffer.
18638
18639	\section virtual_allocator_creating_virtual_block Creating virtual block
18640
18641	To use this functionality, there is no main "allocator" object.
18642	You don't need to have #VmaAllocator object created.
18643	All you need to do is to create a separate #VmaVirtualBlock object for each block of memory you want to be managed by the allocator:
18644
18645	-# Fill in #VmaVirtualBlockCreateInfo structure.
18646	-# Call vmaCreateVirtualBlock(). Get new #VmaVirtualBlock object.
18647
18648	Example:
18649
18650	\code
18651	VmaVirtualBlockCreateInfo blockCreateInfo = {};
18652	blockCreateInfo.size = 1048576; // 1 MB
18653
18654	VmaVirtualBlock block;
18655	VkResult res = vmaCreateVirtualBlock(&blockCreateInfo, &block);
18656	\endcode
18657
18658	\section virtual_allocator_making_virtual_allocations Making virtual allocations
18659
18660	#VmaVirtualBlock object contains internal data structure that keeps track of free and occupied regions
18661	using the same code as the main Vulkan memory allocator.
18662	Similarly to #VmaAllocation for standard GPU allocations, there is #VmaVirtualAllocation type
18663	that represents an opaque handle to an allocation withing the virtual block.
18664
18665	In order to make such allocation:
18666
18667	-# Fill in #VmaVirtualAllocationCreateInfo structure.
18668	-# Call vmaVirtualAllocate(). Get new #VmaVirtualAllocation object that represents the allocation.
18669	You can also receive `VkDeviceSize offset` that was assigned to the allocation.
18670
18671	Example:
18672
18673	\code
18674	VmaVirtualAllocationCreateInfo allocCreateInfo = {};
18675	allocCreateInfo.size = 4096; // 4 KB
18676
18677	VmaVirtualAllocation alloc;
18678	VkDeviceSize offset;
18679	res = vmaVirtualAllocate(block, &allocCreateInfo, &alloc, &offset);
18680	if(res == VK_SUCCESS)
18681	{
18682	// Use the 4 KB of your memory starting at offset.
18683	}
18684	else
18685	{
18686	// Allocation failed - no space for it could be found. Handle this error!
18687	}
18688	\endcode
18689
18690	\section virtual_allocator_deallocation Deallocation
18691
18692	When no longer needed, an allocation can be freed by calling vmaVirtualFree().
18693	You can only pass to this function an allocation that was previously returned by vmaVirtualAllocate()
18694	called for the same #VmaVirtualBlock.
18695
18696	When whole block is no longer needed, the block object can be released by calling vmaDestroyVirtualBlock().
18697	All allocations must be freed before the block is destroyed, which is checked internally by an assert.
18698	However, if you don't want to call vmaVirtualFree() for each allocation, you can use vmaClearVirtualBlock() to free them all at once -
18699	a feature not available in normal Vulkan memory allocator. Example:
18700
18701	\code
18702	vmaVirtualFree(block, alloc);
18703	vmaDestroyVirtualBlock(block);
18704	\endcode
18705
18706	\section virtual_allocator_allocation_parameters Allocation parameters
18707
18708	You can attach a custom pointer to each allocation by using vmaSetVirtualAllocationUserData().
18709	Its default value is null.
18710	It can be used to store any data that needs to be associated with that allocation - e.g. an index, a handle, or a pointer to some
18711	larger data structure containing more information. Example:
18712
18713	\code
18714	struct CustomAllocData
18715	{
18716	std::string m_AllocName;
18717	};
18718	CustomAllocData* allocData = new CustomAllocData();
18719	allocData->m_AllocName = "My allocation 1";
18720	vmaSetVirtualAllocationUserData(block, alloc, allocData);
18721	\endcode
18722
18723	The pointer can later be fetched, along with allocation offset and size, by passing the allocation handle to function
18724	vmaGetVirtualAllocationInfo() and inspecting returned structure #VmaVirtualAllocationInfo.
18725	If you allocated a new object to be used as the custom pointer, don't forget to delete that object before freeing the allocation!
18726	Example:
18727
18728	\code
18729	VmaVirtualAllocationInfo allocInfo;
18730	vmaGetVirtualAllocationInfo(block, alloc, &allocInfo);
18731	delete (CustomAllocData*)allocInfo.pUserData;
18732
18733	vmaVirtualFree(block, alloc);
18734	\endcode
18735
18736	\section virtual_allocator_alignment_and_units Alignment and units
18737
18738	It feels natural to express sizes and offsets in bytes.
18739	If an offset of an allocation needs to be aligned to a multiply of some number (e.g. 4 bytes), you can fill optional member
18740	VmaVirtualAllocationCreateInfo::alignment to request it. Example:
18741
18742	\code
18743	VmaVirtualAllocationCreateInfo allocCreateInfo = {};
18744	allocCreateInfo.size = 4096; // 4 KB
18745	allocCreateInfo.alignment = 4; // Returned offset must be a multiply of 4 B
18746
18747	VmaVirtualAllocation alloc;
18748	res = vmaVirtualAllocate(block, &allocCreateInfo, &alloc, nullptr);
18749	\endcode
18750
18751	Alignments of different allocations made from one block may vary.
18752	However, if all alignments and sizes are always multiply of some size e.g. 4 B or `sizeof(MyDataStruct)`,
18753	you can express all sizes, alignments, and offsets in multiples of that size instead of individual bytes.
18754	It might be more convenient, but you need to make sure to use this new unit consistently in all the places:
18755
18756	- VmaVirtualBlockCreateInfo::size
18757	- VmaVirtualAllocationCreateInfo::size and VmaVirtualAllocationCreateInfo::alignment
18758	- Using offset returned by vmaVirtualAllocate() or in VmaVirtualAllocationInfo::offset
18759
18760	\section virtual_allocator_statistics Statistics
18761
18762	You can obtain statistics of a virtual block using vmaGetVirtualBlockStatistics()
18763	(to get brief statistics that are fast to calculate)
18764	or vmaCalculateVirtualBlockStatistics() (to get more detailed statistics, slower to calculate).
18765	The functions fill structures #VmaStatistics, #VmaDetailedStatistics respectively - same as used by the normal Vulkan memory allocator.
18766	Example:
18767
18768	\code
18769	VmaStatistics stats;
18770	vmaGetVirtualBlockStatistics(block, &stats);
18771	printf("My virtual block has %llu bytes used by %u virtual allocations\n",
18772	stats.allocationBytes, stats.allocationCount);
18773	\endcode
18774
18775	You can also request a full list of allocations and free regions as a string in JSON format by calling
18776	vmaBuildVirtualBlockStatsString().
18777	Returned string must be later freed using vmaFreeVirtualBlockStatsString().
18778	The format of this string differs from the one returned by the main Vulkan allocator, but it is similar.
18779
18780	\section virtual_allocator_additional_considerations Additional considerations
18781
18782	The "virtual allocator" functionality is implemented on a level of individual memory blocks.
18783	Keeping track of a whole collection of blocks, allocating new ones when out of free space,
18784	deleting empty ones, and deciding which one to try first for a new allocation must be implemented by the user.
18785
18786	Alternative allocation algorithms are supported, just like in custom pools of the real GPU memory.
18787	See enum #VmaVirtualBlockCreateFlagBits to learn how to specify them (e.g. #VMA_VIRTUAL_BLOCK_CREATE_LINEAR_ALGORITHM_BIT).
18788	You can find their description in chapter \ref custom_memory_pools.
18789	Allocation strategies are also supported.
18790	See enum #VmaVirtualAllocationCreateFlagBits to learn how to specify them (e.g. #VMA_VIRTUAL_ALLOCATION_CREATE_STRATEGY_MIN_TIME_BIT).
18791
18792	Following features are supported only by the allocator of the real GPU memory and not by virtual allocations:
18793	buffer-image granularity, `VMA_DEBUG_MARGIN`, `VMA_MIN_ALIGNMENT`.
18794
18795
18796	\page debugging_memory_usage Debugging incorrect memory usage
18797
18798	If you suspect a bug with memory usage, like usage of uninitialized memory or
18799	memory being overwritten out of bounds of an allocation,
18800	you can use debug features of this library to verify this.
18801
18802	\section debugging_memory_usage_initialization Memory initialization
18803
18804	If you experience a bug with incorrect and nondeterministic data in your program and you suspect uninitialized memory to be used,
18805	you can enable automatic memory initialization to verify this.
18806	To do it, define macro `VMA_DEBUG_INITIALIZE_ALLOCATIONS` to 1.
18807
18808	\code
18809	#define VMA_DEBUG_INITIALIZE_ALLOCATIONS 1
18810	#include "vk_mem_alloc.h"
18811	\endcode
18812
18813	It makes memory of new allocations initialized to bit pattern `0xDCDCDCDC`.
18814	Before an allocation is destroyed, its memory is filled with bit pattern `0xEFEFEFEF`.
18815	Memory is automatically mapped and unmapped if necessary.
18816
18817	If you find these values while debugging your program, good chances are that you incorrectly
18818	read Vulkan memory that is allocated but not initialized, or already freed, respectively.
18819
18820	Memory initialization works only with memory types that are `HOST_VISIBLE` and with allocations that can be mapped.
18821	It works also with dedicated allocations.
18822
18823	\section debugging_memory_usage_margins Margins
18824
18825	By default, allocations are laid out in memory blocks next to each other if possible
18826	(considering required alignment, `bufferImageGranularity`, and `nonCoherentAtomSize`).
18827
18828	
18829
18830	Define macro `VMA_DEBUG_MARGIN` to some non-zero value (e.g. 16) to enforce specified
18831	number of bytes as a margin after every allocation.
18832
18833	\code
18834	#define VMA_DEBUG_MARGIN 16
18835	#include "vk_mem_alloc.h"
18836	\endcode
18837
18838	
18839
18840	If your bug goes away after enabling margins, it means it may be caused by memory
18841	being overwritten outside of allocation boundaries. It is not 100% certain though.
18842	Change in application behavior may also be caused by different order and distribution
18843	of allocations across memory blocks after margins are applied.
18844
18845	Margins work with all types of memory.
18846
18847	Margin is applied only to allocations made out of memory blocks and not to dedicated
18848	allocations, which have their own memory block of specific size.
18849	It is thus not applied to allocations made using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT flag
18850	or those automatically decided to put into dedicated allocations, e.g. due to its
18851	large size or recommended by VK_KHR_dedicated_allocation extension.
18852
18853	Margins appear in [JSON dump](@ref statistics_json_dump) as part of free space.
18854
18855	Note that enabling margins increases memory usage and fragmentation.
18856
18857	Margins do not apply to \ref virtual_allocator.
18858
18859	\section debugging_memory_usage_corruption_detection Corruption detection
18860
18861	You can additionally define macro `VMA_DEBUG_DETECT_CORRUPTION` to 1 to enable validation
18862	of contents of the margins.
18863
18864	\code
18865	#define VMA_DEBUG_MARGIN 16
18866	#define VMA_DEBUG_DETECT_CORRUPTION 1
18867	#include "vk_mem_alloc.h"
18868	\endcode
18869
18870	When this feature is enabled, number of bytes specified as `VMA_DEBUG_MARGIN`
18871	(it must be multiply of 4) after every allocation is filled with a magic number.
18872	This idea is also know as "canary".
18873	Memory is automatically mapped and unmapped if necessary.
18874
18875	This number is validated automatically when the allocation is destroyed.
18876	If it is not equal to the expected value, `VMA_ASSERT()` is executed.
18877	It clearly means that either CPU or GPU overwritten the memory outside of boundaries of the allocation,
18878	which indicates a serious bug.
18879
18880	You can also explicitly request checking margins of all allocations in all memory blocks
18881	that belong to specified memory types by using function vmaCheckCorruption(),
18882	or in memory blocks that belong to specified custom pool, by using function
18883	vmaCheckPoolCorruption().
18884
18885	Margin validation (corruption detection) works only for memory types that are
18886	`HOST_VISIBLE` and `HOST_COHERENT`.
18887
18888
18889	\page opengl_interop OpenGL Interop
18890
18891	VMA provides some features that help with interoperability with OpenGL.
18892
18893	\section opengl_interop_exporting_memory Exporting memory
18894
18895	If you want to attach `VkExportMemoryAllocateInfoKHR` structure to `pNext` chain of memory allocations made by the library:
18896
18897	It is recommended to create \ref custom_memory_pools for such allocations.
18898	Define and fill in your `VkExportMemoryAllocateInfoKHR` structure and attach it to VmaPoolCreateInfo::pMemoryAllocateNext
18899	while creating the custom pool.
18900	Please note that the structure must remain alive and unchanged for the whole lifetime of the #VmaPool,
18901	not only while creating it, as no copy of the structure is made,
18902	but its original pointer is used for each allocation instead.
18903
18904	If you want to export all memory allocated by the library from certain memory types,
18905	also dedicated allocations or other allocations made from default pools,
18906	an alternative solution is to fill in VmaAllocatorCreateInfo::pTypeExternalMemoryHandleTypes.
18907	It should point to an array with `VkExternalMemoryHandleTypeFlagsKHR` to be automatically passed by the library
18908	through `VkExportMemoryAllocateInfoKHR` on each allocation made from a specific memory type.
18909	Please note that new versions of the library also support dedicated allocations created in custom pools.
18910
18911	You should not mix these two methods in a way that allows to apply both to the same memory type.
18912	Otherwise, `VkExportMemoryAllocateInfoKHR` structure would be attached twice to the `pNext` chain of `VkMemoryAllocateInfo`.
18913
18914
18915	\section opengl_interop_custom_alignment Custom alignment
18916
18917	Buffers or images exported to a different API like OpenGL may require a different alignment,
18918	higher than the one used by the library automatically, queried from functions like `vkGetBufferMemoryRequirements`.
18919	To impose such alignment:
18920
18921	It is recommended to create \ref custom_memory_pools for such allocations.
18922	Set VmaPoolCreateInfo::minAllocationAlignment member to the minimum alignment required for each allocation
18923	to be made out of this pool.
18924	The alignment actually used will be the maximum of this member and the alignment returned for the specific buffer or image
18925	from a function like `vkGetBufferMemoryRequirements`, which is called by VMA automatically.
18926
18927	If you want to create a buffer with a specific minimum alignment out of default pools,
18928	use special function vmaCreateBufferWithAlignment(), which takes additional parameter `minAlignment`.
18929
18930	Note the problem of alignment affects only resources placed inside bigger `VkDeviceMemory` blocks and not dedicated
18931	allocations, as these, by definition, always have alignment = 0 because the resource is bound to the beginning of its dedicated block.
18932	Contrary to Direct3D 12, Vulkan doesn't have a concept of alignment of the entire memory block passed on its allocation.
18933
18934
18935	\page usage_patterns Recommended usage patterns
18936
18937	Vulkan gives great flexibility in memory allocation.
18938	This chapter shows the most common patterns.
18939
18940	See also slides from talk:
18941	[Sawicki, Adam. Advanced Graphics Techniques Tutorial: Memory management in Vulkan and DX12. Game Developers Conference, 2018](https://www.gdcvault.com/play/1025458/Advanced-Graphics-Techniques-Tutorial-New)
18942
18943
18944	\section usage_patterns_gpu_only GPU-only resource
18945
18946	<b>When:</b>
18947	Any resources that you frequently write and read on GPU,
18948	e.g. images used as color attachments (aka "render targets"), depth-stencil attachments,
18949	images/buffers used as storage image/buffer (aka "Unordered Access View (UAV)").
18950
18951	<b>What to do:</b>
18952	Let the library select the optimal memory type, which will likely have `VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT`.
18953
18954	\code
18955	VkImageCreateInfo imgCreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
18956	imgCreateInfo.imageType = VK_IMAGE_TYPE_2D;
18957	imgCreateInfo.extent.width = 3840;
18958	imgCreateInfo.extent.height = 2160;
18959	imgCreateInfo.extent.depth = 1;
18960	imgCreateInfo.mipLevels = 1;
18961	imgCreateInfo.arrayLayers = 1;
18962	imgCreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
18963	imgCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
18964	imgCreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
18965	imgCreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
18966	imgCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
18967
18968	VmaAllocationCreateInfo allocCreateInfo = {};
18969	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
18970	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
18971	allocCreateInfo.priority = 1.0f;
18972
18973	VkImage img;
18974	VmaAllocation alloc;
18975	vmaCreateImage(allocator, &imgCreateInfo, &allocCreateInfo, &img, &alloc, nullptr);
18976	\endcode
18977
18978	<b>Also consider:</b>
18979	Consider creating them as dedicated allocations using #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT,
18980	especially if they are large or if you plan to destroy and recreate them with different sizes
18981	e.g. when display resolution changes.
18982	Prefer to create such resources first and all other GPU resources (like textures and vertex buffers) later.
18983	When VK_EXT_memory_priority extension is enabled, it is also worth setting high priority to such allocation
18984	to decrease chances to be evicted to system memory by the operating system.
18985
18986	\section usage_patterns_staging_copy_upload Staging copy for upload
18987
18988	<b>When:</b>
18989	A "staging" buffer than you want to map and fill from CPU code, then use as a source od transfer
18990	to some GPU resource.
18991
18992	<b>What to do:</b>
18993	Use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT.
18994	Let the library select the optimal memory type, which will always have `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`.
18995
18996	\code
18997	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
18998	bufCreateInfo.size = 65536;
18999	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
19000
19001	VmaAllocationCreateInfo allocCreateInfo = {};
19002	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19003	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
19004	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19005
19006	VkBuffer buf;
19007	VmaAllocation alloc;
19008	VmaAllocationInfo allocInfo;
19009	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
19010
19011	...
19012
19013	memcpy(allocInfo.pMappedData, myData, myDataSize);
19014	\endcode
19015
19016	<b>Also consider:</b>
19017	You can map the allocation using vmaMapMemory() or you can create it as persistenly mapped
19018	using #VMA_ALLOCATION_CREATE_MAPPED_BIT, as in the example above.
19019
19020
19021	\section usage_patterns_readback Readback
19022
19023	<b>When:</b>
19024	Buffers for data written by or transferred from the GPU that you want to read back on the CPU,
19025	e.g. results of some computations.
19026
19027	<b>What to do:</b>
19028	Use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT.
19029	Let the library select the optimal memory type, which will always have `VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT`
19030	and `VK_MEMORY_PROPERTY_HOST_CACHED_BIT`.
19031
19032	\code
19033	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19034	bufCreateInfo.size = 65536;
19035	bufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_DST_BIT;
19036
19037	VmaAllocationCreateInfo allocCreateInfo = {};
19038	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19039	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_RANDOM_BIT |
19040	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19041
19042	VkBuffer buf;
19043	VmaAllocation alloc;
19044	VmaAllocationInfo allocInfo;
19045	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
19046
19047	...
19048
19049	const float* downloadedData = (const float*)allocInfo.pMappedData;
19050	\endcode
19051
19052
19053	\section usage_patterns_advanced_data_uploading Advanced data uploading
19054
19055	For resources that you frequently write on CPU via mapped pointer and
19056	freqnently read on GPU e.g. as a uniform buffer (also called "dynamic"), multiple options are possible:
19057
19058	-# Easiest solution is to have one copy of the resource in `HOST_VISIBLE` memory,
19059	even if it means system RAM (not `DEVICE_LOCAL`) on systems with a discrete graphics card,
19060	and make the device reach out to that resource directly.
19061	- Reads performed by the device will then go through PCI Express bus.
19062	The performace of this access may be limited, but it may be fine depending on the size
19063	of this resource (whether it is small enough to quickly end up in GPU cache) and the sparsity
19064	of access.
19065	-# On systems with unified memory (e.g. AMD APU or Intel integrated graphics, mobile chips),
19066	a memory type may be available that is both `HOST_VISIBLE` (available for mapping) and `DEVICE_LOCAL`
19067	(fast to access from the GPU). Then, it is likely the best choice for such type of resource.
19068	-# Systems with a discrete graphics card and separate video memory may or may not expose
19069	a memory type that is both `HOST_VISIBLE` and `DEVICE_LOCAL`, also known as Base Address Register (BAR).
19070	If they do, it represents a piece of VRAM (or entire VRAM, if ReBAR is enabled in the motherboard BIOS)
19071	that is available to CPU for mapping.
19072	- Writes performed by the host to that memory go through PCI Express bus.
19073	The performance of these writes may be limited, but it may be fine, especially on PCIe 4.0,
19074	as long as rules of using uncached and write-combined memory are followed - only sequential writes and no reads.
19075	-# Finally, you may need or prefer to create a separate copy of the resource in `DEVICE_LOCAL` memory,
19076	a separate "staging" copy in `HOST_VISIBLE` memory and perform an explicit transfer command between them.
19077
19078	Thankfully, VMA offers an aid to create and use such resources in the the way optimal
19079	for the current Vulkan device. To help the library make the best choice,
19080	use flag #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT together with
19081	#VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT.
19082	It will then prefer a memory type that is both `DEVICE_LOCAL` and `HOST_VISIBLE` (integrated memory or BAR),
19083	but if no such memory type is available or allocation from it fails
19084	(PC graphics cards have only 256 MB of BAR by default, unless ReBAR is supported and enabled in BIOS),
19085	it will fall back to `DEVICE_LOCAL` memory for fast GPU access.
19086	It is then up to you to detect that the allocation ended up in a memory type that is not `HOST_VISIBLE`,
19087	so you need to create another "staging" allocation and perform explicit transfers.
19088
19089	\code
19090	VkBufferCreateInfo bufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19091	bufCreateInfo.size = 65536;
19092	bufCreateInfo.usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
19093
19094	VmaAllocationCreateInfo allocCreateInfo = {};
19095	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19096	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
19097	VMA_ALLOCATION_CREATE_HOST_ACCESS_ALLOW_TRANSFER_INSTEAD_BIT |
19098	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19099
19100	VkBuffer buf;
19101	VmaAllocation alloc;
19102	VmaAllocationInfo allocInfo;
19103	vmaCreateBuffer(allocator, &bufCreateInfo, &allocCreateInfo, &buf, &alloc, &allocInfo);
19104
19105	VkMemoryPropertyFlags memPropFlags;
19106	vmaGetAllocationMemoryProperties(allocator, alloc, &memPropFlags);
19107
19108	if(memPropFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
19109	{
19110	// Allocation ended up in a mappable memory and is already mapped - write to it directly.
19111
19112	// [Executed in runtime]:
19113	memcpy(allocInfo.pMappedData, myData, myDataSize);
19114	}
19115	else
19116	{
19117	// Allocation ended up in a non-mappable memory - need to transfer.
19118	VkBufferCreateInfo stagingBufCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO };
19119	stagingBufCreateInfo.size = 65536;
19120	stagingBufCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT;
19121
19122	VmaAllocationCreateInfo stagingAllocCreateInfo = {};
19123	stagingAllocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19124	stagingAllocCreateInfo.flags = VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT |
19125	VMA_ALLOCATION_CREATE_MAPPED_BIT;
19126
19127	VkBuffer stagingBuf;
19128	VmaAllocation stagingAlloc;
19129	VmaAllocationInfo stagingAllocInfo;
19130	vmaCreateBuffer(allocator, &stagingBufCreateInfo, &stagingAllocCreateInfo,
19131	&stagingBuf, &stagingAlloc, stagingAllocInfo);
19132
19133	// [Executed in runtime]:
19134	memcpy(stagingAllocInfo.pMappedData, myData, myDataSize);
19135	//vkCmdPipelineBarrier: VK_ACCESS_HOST_WRITE_BIT --> VK_ACCESS_TRANSFER_READ_BIT
19136	VkBufferCopy bufCopy = {
19137	0, // srcOffset
19138	0, // dstOffset,
19139	myDataSize); // size
19140	vkCmdCopyBuffer(cmdBuf, stagingBuf, buf, 1, &bufCopy);
19141	}
19142	\endcode
19143
19144	\section usage_patterns_other_use_cases Other use cases
19145
19146	Here are some other, less obvious use cases and their recommended settings:
19147
19148	- An image that is used only as transfer source and destination, but it should stay on the device,
19149	as it is used to temporarily store a copy of some texture, e.g. from the current to the next frame,
19150	for temporal antialiasing or other temporal effects.
19151	- Use `VkImageCreateInfo::usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT`
19152	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO
19153	- An image that is used only as transfer source and destination, but it should be placed
19154	in the system RAM despite it doesn't need to be mapped, because it serves as a "swap" copy to evict
19155	least recently used textures from VRAM.
19156	- Use `VkImageCreateInfo::usage = VK_IMAGE_USAGE_TRANSFER_SRC_BIT | VK_IMAGE_USAGE_TRANSFER_DST_BIT`
19157	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO_PREFER_HOST,
19158	as VMA needs a hint here to differentiate from the previous case.
19159	- A buffer that you want to map and write from the CPU, directly read from the GPU
19160	(e.g. as a uniform or vertex buffer), but you have a clear preference to place it in device or
19161	host memory due to its large size.
19162	- Use `VkBufferCreateInfo::usage = VK_BUFFER_USAGE_UNIFORM_BUFFER_BIT`
19163	- Use VmaAllocationCreateInfo::usage = #VMA_MEMORY_USAGE_AUTO_PREFER_DEVICE or #VMA_MEMORY_USAGE_AUTO_PREFER_HOST
19164	- Use VmaAllocationCreateInfo::flags = #VMA_ALLOCATION_CREATE_HOST_ACCESS_SEQUENTIAL_WRITE_BIT
19165
19166
19167	\page configuration Configuration
19168
19169	Please check "CONFIGURATION SECTION" in the code to find macros that you can define
19170	before each include of this file or change directly in this file to provide
19171	your own implementation of basic facilities like assert, `min()` and `max()` functions,
19172	mutex, atomic etc.
19173	The library uses its own implementation of containers by default, but you can switch to using
19174	STL containers instead.
19175
19176	For example, define `VMA_ASSERT(expr)` before including the library to provide
19177	custom implementation of the assertion, compatible with your project.
19178	By default it is defined to standard C `assert(expr)` in `_DEBUG` configuration
19179	and empty otherwise.
19180
19181	\section config_Vulkan_functions Pointers to Vulkan functions
19182
19183	There are multiple ways to import pointers to Vulkan functions in the library.
19184	In the simplest case you don't need to do anything.
19185	If the compilation or linking of your program or the initialization of the #VmaAllocator
19186	doesn't work for you, you can try to reconfigure it.
19187
19188	First, the allocator tries to fetch pointers to Vulkan functions linked statically,
19189	like this:
19190
19191	\code
19192	m_VulkanFunctions.vkAllocateMemory = (PFN_vkAllocateMemory)vkAllocateMemory;
19193	\endcode
19194
19195	If you want to disable this feature, set configuration macro: `#define VMA_STATIC_VULKAN_FUNCTIONS 0`.
19196
19197	Second, you can provide the pointers yourself by setting member VmaAllocatorCreateInfo::pVulkanFunctions.
19198	You can fetch them e.g. using functions `vkGetInstanceProcAddr` and `vkGetDeviceProcAddr` or
19199	by using a helper library like [volk](https://github.com/zeux/volk).
19200
19201	Third, VMA tries to fetch remaining pointers that are still null by calling
19202	`vkGetInstanceProcAddr` and `vkGetDeviceProcAddr` on its own.
19203	You need to only fill in VmaVulkanFunctions::vkGetInstanceProcAddr and VmaVulkanFunctions::vkGetDeviceProcAddr.
19204	Other pointers will be fetched automatically.
19205	If you want to disable this feature, set configuration macro: `#define VMA_DYNAMIC_VULKAN_FUNCTIONS 0`.
19206
19207	Finally, all the function pointers required by the library (considering selected
19208	Vulkan version and enabled extensions) are checked with `VMA_ASSERT` if they are not null.
19209
19210
19211	\section custom_memory_allocator Custom host memory allocator
19212
19213	If you use custom allocator for CPU memory rather than default operator `new`
19214	and `delete` from C++, you can make this library using your allocator as well
19215	by filling optional member VmaAllocatorCreateInfo::pAllocationCallbacks. These
19216	functions will be passed to Vulkan, as well as used by the library itself to
19217	make any CPU-side allocations.
19218
19219	\section allocation_callbacks Device memory allocation callbacks
19220
19221	The library makes calls to `vkAllocateMemory()` and `vkFreeMemory()` internally.
19222	You can setup callbacks to be informed about these calls, e.g. for the purpose
19223	of gathering some statistics. To do it, fill optional member
19224	VmaAllocatorCreateInfo::pDeviceMemoryCallbacks.
19225
19226	\section heap_memory_limit Device heap memory limit
19227
19228	When device memory of certain heap runs out of free space, new allocations may
19229	fail (returning error code) or they may succeed, silently pushing some existing_
19230	memory blocks from GPU VRAM to system RAM (which degrades performance). This
19231	behavior is implementation-dependent - it depends on GPU vendor and graphics
19232	driver.
19233
19234	On AMD cards it can be controlled while creating Vulkan device object by using
19235	VK_AMD_memory_overallocation_behavior extension, if available.
19236
19237	Alternatively, if you want to test how your program behaves with limited amount of Vulkan device
19238	memory available without switching your graphics card to one that really has
19239	smaller VRAM, you can use a feature of this library intended for this purpose.
19240	To do it, fill optional member VmaAllocatorCreateInfo::pHeapSizeLimit.
19241
19242
19243
19244	\page vk_khr_dedicated_allocation VK_KHR_dedicated_allocation
19245
19246	VK_KHR_dedicated_allocation is a Vulkan extension which can be used to improve
19247	performance on some GPUs. It augments Vulkan API with possibility to query
19248	driver whether it prefers particular buffer or image to have its own, dedicated
19249	allocation (separate `VkDeviceMemory` block) for better efficiency - to be able
19250	to do some internal optimizations. The extension is supported by this library.
19251	It will be used automatically when enabled.
19252
19253	It has been promoted to core Vulkan 1.1, so if you use eligible Vulkan version
19254	and inform VMA about it by setting VmaAllocatorCreateInfo::vulkanApiVersion,
19255	you are all set.
19256
19257	Otherwise, if you want to use it as an extension:
19258
19259	1 . When creating Vulkan device, check if following 2 device extensions are
19260	supported (call `vkEnumerateDeviceExtensionProperties()`).
19261	If yes, enable them (fill `VkDeviceCreateInfo::ppEnabledExtensionNames`).
19262
19263	- VK_KHR_get_memory_requirements2
19264	- VK_KHR_dedicated_allocation
19265
19266	If you enabled these extensions:
19267
19268	2 . Use #VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT flag when creating
19269	your #VmaAllocator to inform the library that you enabled required extensions
19270	and you want the library to use them.
19271
19272	\code
19273	allocatorInfo.flags |= VMA_ALLOCATOR_CREATE_KHR_DEDICATED_ALLOCATION_BIT;
19274
19275	vmaCreateAllocator(&allocatorInfo, &allocator);
19276	\endcode
19277
19278	That is all. The extension will be automatically used whenever you create a
19279	buffer using vmaCreateBuffer() or image using vmaCreateImage().
19280
19281	When using the extension together with Vulkan Validation Layer, you will receive
19282	warnings like this:
19283
19284	_vkBindBufferMemory(): Binding memory to buffer 0x33 but vkGetBufferMemoryRequirements() has not been called on that buffer._
19285
19286	It is OK, you should just ignore it. It happens because you use function
19287	`vkGetBufferMemoryRequirements2KHR()` instead of standard
19288	`vkGetBufferMemoryRequirements()`, while the validation layer seems to be
19289	unaware of it.
19290
19291	To learn more about this extension, see:
19292
19293	- [VK_KHR_dedicated_allocation in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap50.html#VK_KHR_dedicated_allocation)
19294	- [VK_KHR_dedicated_allocation unofficial manual](http://asawicki.info/articles/VK_KHR_dedicated_allocation.php5)
19295
19296
19297
19298	\page vk_ext_memory_priority VK_EXT_memory_priority
19299
19300	VK_EXT_memory_priority is a device extension that allows to pass additional "priority"
19301	value to Vulkan memory allocations that the implementation may use prefer certain
19302	buffers and images that are critical for performance to stay in device-local memory
19303	in cases when the memory is over-subscribed, while some others may be moved to the system memory.
19304
19305	VMA offers convenient usage of this extension.
19306	If you enable it, you can pass "priority" parameter when creating allocations or custom pools
19307	and the library automatically passes the value to Vulkan using this extension.
19308
19309	If you want to use this extension in connection with VMA, follow these steps:
19310
19311	\section vk_ext_memory_priority_initialization Initialization
19312
19313	1) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19314	Check if the extension is supported - if returned array of `VkExtensionProperties` contains "VK_EXT_memory_priority".
19315
19316	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19317	Attach additional structure `VkPhysicalDeviceMemoryPriorityFeaturesEXT` to `VkPhysicalDeviceFeatures2::pNext` to be returned.
19318	Check if the device feature is really supported - check if `VkPhysicalDeviceMemoryPriorityFeaturesEXT::memoryPriority` is true.
19319
19320	3) While creating device with `vkCreateDevice`, enable this extension - add "VK_EXT_memory_priority"
19321	to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19322
19323	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19324	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19325	Enable this device feature - attach additional structure `VkPhysicalDeviceMemoryPriorityFeaturesEXT` to
19326	`VkPhysicalDeviceFeatures2::pNext` chain and set its member `memoryPriority` to `VK_TRUE`.
19327
19328	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19329	have enabled this extension and feature - add #VMA_ALLOCATOR_CREATE_EXT_MEMORY_PRIORITY_BIT
19330	to VmaAllocatorCreateInfo::flags.
19331
19332	\section vk_ext_memory_priority_usage Usage
19333
19334	When using this extension, you should initialize following member:
19335
19336	- VmaAllocationCreateInfo::priority when creating a dedicated allocation with #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
19337	- VmaPoolCreateInfo::priority when creating a custom pool.
19338
19339	It should be a floating-point value between `0.0f` and `1.0f`, where recommended default is `0.5f`.
19340	Memory allocated with higher value can be treated by the Vulkan implementation as higher priority
19341	and so it can have lower chances of being pushed out to system memory, experiencing degraded performance.
19342
19343	It might be a good idea to create performance-critical resources like color-attachment or depth-stencil images
19344	as dedicated and set high priority to them. For example:
19345
19346	\code
19347	VkImageCreateInfo imgCreateInfo = { VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO };
19348	imgCreateInfo.imageType = VK_IMAGE_TYPE_2D;
19349	imgCreateInfo.extent.width = 3840;
19350	imgCreateInfo.extent.height = 2160;
19351	imgCreateInfo.extent.depth = 1;
19352	imgCreateInfo.mipLevels = 1;
19353	imgCreateInfo.arrayLayers = 1;
19354	imgCreateInfo.format = VK_FORMAT_R8G8B8A8_UNORM;
19355	imgCreateInfo.tiling = VK_IMAGE_TILING_OPTIMAL;
19356	imgCreateInfo.initialLayout = VK_IMAGE_LAYOUT_UNDEFINED;
19357	imgCreateInfo.usage = VK_IMAGE_USAGE_SAMPLED_BIT | VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT;
19358	imgCreateInfo.samples = VK_SAMPLE_COUNT_1_BIT;
19359
19360	VmaAllocationCreateInfo allocCreateInfo = {};
19361	allocCreateInfo.usage = VMA_MEMORY_USAGE_AUTO;
19362	allocCreateInfo.flags = VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT;
19363	allocCreateInfo.priority = 1.0f;
19364
19365	VkImage img;
19366	VmaAllocation alloc;
19367	vmaCreateImage(allocator, &imgCreateInfo, &allocCreateInfo, &img, &alloc, nullptr);
19368	\endcode
19369
19370	`priority` member is ignored in the following situations:
19371
19372	- Allocations created in custom pools: They inherit the priority, along with all other allocation parameters
19373	from the parametrs passed in #VmaPoolCreateInfo when the pool was created.
19374	- Allocations created in default pools: They inherit the priority from the parameters
19375	VMA used when creating default pools, which means `priority == 0.5f`.
19376
19377
19378	\page vk_amd_device_coherent_memory VK_AMD_device_coherent_memory
19379
19380	VK_AMD_device_coherent_memory is a device extension that enables access to
19381	additional memory types with `VK_MEMORY_PROPERTY_DEVICE_COHERENT_BIT_AMD` and
19382	`VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` flag. It is useful mostly for
19383	allocation of buffers intended for writing "breadcrumb markers" in between passes
19384	or draw calls, which in turn are useful for debugging GPU crash/hang/TDR cases.
19385
19386	When the extension is available but has not been enabled, Vulkan physical device
19387	still exposes those memory types, but their usage is forbidden. VMA automatically
19388	takes care of that - it returns `VK_ERROR_FEATURE_NOT_PRESENT` when an attempt
19389	to allocate memory of such type is made.
19390
19391	If you want to use this extension in connection with VMA, follow these steps:
19392
19393	\section vk_amd_device_coherent_memory_initialization Initialization
19394
19395	1) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19396	Check if the extension is supported - if returned array of `VkExtensionProperties` contains "VK_AMD_device_coherent_memory".
19397
19398	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19399	Attach additional structure `VkPhysicalDeviceCoherentMemoryFeaturesAMD` to `VkPhysicalDeviceFeatures2::pNext` to be returned.
19400	Check if the device feature is really supported - check if `VkPhysicalDeviceCoherentMemoryFeaturesAMD::deviceCoherentMemory` is true.
19401
19402	3) While creating device with `vkCreateDevice`, enable this extension - add "VK_AMD_device_coherent_memory"
19403	to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19404
19405	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19406	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19407	Enable this device feature - attach additional structure `VkPhysicalDeviceCoherentMemoryFeaturesAMD` to
19408	`VkPhysicalDeviceFeatures2::pNext` and set its member `deviceCoherentMemory` to `VK_TRUE`.
19409
19410	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19411	have enabled this extension and feature - add #VMA_ALLOCATOR_CREATE_AMD_DEVICE_COHERENT_MEMORY_BIT
19412	to VmaAllocatorCreateInfo::flags.
19413
19414	\section vk_amd_device_coherent_memory_usage Usage
19415
19416	After following steps described above, you can create VMA allocations and custom pools
19417	out of the special `DEVICE_COHERENT` and `DEVICE_UNCACHED` memory types on eligible
19418	devices. There are multiple ways to do it, for example:
19419
19420	- You can request or prefer to allocate out of such memory types by adding
19421	`VK_MEMORY_PROPERTY_DEVICE_UNCACHED_BIT_AMD` to VmaAllocationCreateInfo::requiredFlags
19422	or VmaAllocationCreateInfo::preferredFlags. Those flags can be freely mixed with
19423	other ways of \ref choosing_memory_type, like setting VmaAllocationCreateInfo::usage.
19424	- If you manually found memory type index to use for this purpose, force allocation
19425	from this specific index by setting VmaAllocationCreateInfo::memoryTypeBits `= 1u << index`.
19426
19427	\section vk_amd_device_coherent_memory_more_information More information
19428
19429	To learn more about this extension, see [VK_AMD_device_coherent_memory in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_AMD_device_coherent_memory.html)
19430
19431	Example use of this extension can be found in the code of the sample and test suite
19432	accompanying this library.
19433
19434
19435	\page enabling_buffer_device_address Enabling buffer device address
19436
19437	Device extension VK_KHR_buffer_device_address
19438	allow to fetch raw GPU pointer to a buffer and pass it for usage in a shader code.
19439	It has been promoted to core Vulkan 1.2.
19440
19441	If you want to use this feature in connection with VMA, follow these steps:
19442
19443	\section enabling_buffer_device_address_initialization Initialization
19444
19445	1) (For Vulkan version < 1.2) Call `vkEnumerateDeviceExtensionProperties` for the physical device.
19446	Check if the extension is supported - if returned array of `VkExtensionProperties` contains
19447	"VK_KHR_buffer_device_address".
19448
19449	2) Call `vkGetPhysicalDeviceFeatures2` for the physical device instead of old `vkGetPhysicalDeviceFeatures`.
19450	Attach additional structure `VkPhysicalDeviceBufferDeviceAddressFeatures*` to `VkPhysicalDeviceFeatures2::pNext` to be returned.
19451	Check if the device feature is really supported - check if `VkPhysicalDeviceBufferDeviceAddressFeatures::bufferDeviceAddress` is true.
19452
19453	3) (For Vulkan version < 1.2) While creating device with `vkCreateDevice`, enable this extension - add
19454	"VK_KHR_buffer_device_address" to the list passed as `VkDeviceCreateInfo::ppEnabledExtensionNames`.
19455
19456	4) While creating the device, also don't set `VkDeviceCreateInfo::pEnabledFeatures`.
19457	Fill in `VkPhysicalDeviceFeatures2` structure instead and pass it as `VkDeviceCreateInfo::pNext`.
19458	Enable this device feature - attach additional structure `VkPhysicalDeviceBufferDeviceAddressFeatures*` to
19459	`VkPhysicalDeviceFeatures2::pNext` and set its member `bufferDeviceAddress` to `VK_TRUE`.
19460
19461	5) While creating #VmaAllocator with vmaCreateAllocator() inform VMA that you
19462	have enabled this feature - add #VMA_ALLOCATOR_CREATE_BUFFER_DEVICE_ADDRESS_BIT
19463	to VmaAllocatorCreateInfo::flags.
19464
19465	\section enabling_buffer_device_address_usage Usage
19466
19467	After following steps described above, you can create buffers with `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT*` using VMA.
19468	The library automatically adds `VK_MEMORY_ALLOCATE_DEVICE_ADDRESS_BIT*` to
19469	allocated memory blocks wherever it might be needed.
19470
19471	Please note that the library supports only `VK_BUFFER_USAGE_SHADER_DEVICE_ADDRESS_BIT*`.
19472	The second part of this functionality related to "capture and replay" is not supported,
19473	as it is intended for usage in debugging tools like RenderDoc, not in everyday Vulkan usage.
19474
19475	\section enabling_buffer_device_address_more_information More information
19476
19477	To learn more about this extension, see [VK_KHR_buffer_device_address in Vulkan specification](https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/chap46.html#VK_KHR_buffer_device_address)
19478
19479	Example use of this extension can be found in the code of the sample and test suite
19480	accompanying this library.
19481
19482	\page general_considerations General considerations
19483
19484	\section general_considerations_thread_safety Thread safety
19485
19486	- The library has no global state, so separate #VmaAllocator objects can be used
19487	independently.
19488	There should be no need to create multiple such objects though - one per `VkDevice` is enough.
19489	- By default, all calls to functions that take #VmaAllocator as first parameter
19490	are safe to call from multiple threads simultaneously because they are
19491	synchronized internally when needed.
19492	This includes allocation and deallocation from default memory pool, as well as custom #VmaPool.
19493	- When the allocator is created with #VMA_ALLOCATOR_CREATE_EXTERNALLY_SYNCHRONIZED_BIT
19494	flag, calls to functions that take such #VmaAllocator object must be
19495	synchronized externally.
19496	- Access to a #VmaAllocation object must be externally synchronized. For example,
19497	you must not call vmaGetAllocationInfo() and vmaMapMemory() from different
19498	threads at the same time if you pass the same #VmaAllocation object to these
19499	functions.
19500	- #VmaVirtualBlock is not safe to be used from multiple threads simultaneously.
19501
19502	\section general_considerations_versioning_and_compatibility Versioning and compatibility
19503
19504	The library uses [**Semantic Versioning**](https://semver.org/),
19505	which means version numbers follow convention: Major.Minor.Patch (e.g. 2.3.0), where:
19506
19507	- Incremented Patch version means a release is backward- and forward-compatible,
19508	introducing only some internal improvements, bug fixes, optimizations etc.
19509	or changes that are out of scope of the official API described in this documentation.
19510	- Incremented Minor version means a release is backward-compatible,
19511	so existing code that uses the library should continue to work, while some new
19512	symbols could have been added: new structures, functions, new values in existing
19513	enums and bit flags, new structure members, but not new function parameters.
19514	- Incrementing Major version means a release could break some backward compatibility.
19515
19516	All changes between official releases are documented in file "CHANGELOG.md".
19517
19518	\warning Backward compatiblity is considered on the level of C++ source code, not binary linkage.
19519	Adding new members to existing structures is treated as backward compatible if initializing
19520	the new members to binary zero results in the old behavior.
19521	You should always fully initialize all library structures to zeros and not rely on their
19522	exact binary size.
19523
19524	\section general_considerations_validation_layer_warnings Validation layer warnings
19525
19526	When using this library, you can meet following types of warnings issued by
19527	Vulkan validation layer. They don't necessarily indicate a bug, so you may need
19528	to just ignore them.
19529
19530	- *vkBindBufferMemory(): Binding memory to buffer 0xeb8e4 but vkGetBufferMemoryRequirements() has not been called on that buffer.*
19531	- It happens when VK_KHR_dedicated_allocation extension is enabled.
19532	`vkGetBufferMemoryRequirements2KHR` function is used instead, while validation layer seems to be unaware of it.
19533	- *Mapping an image with layout VK_IMAGE_LAYOUT_DEPTH_STENCIL_ATTACHMENT_OPTIMAL can result in undefined behavior if this memory is used by the device. Only GENERAL or PREINITIALIZED should be used.*
19534	- It happens when you map a buffer or image, because the library maps entire
19535	`VkDeviceMemory` block, where different types of images and buffers may end
19536	up together, especially on GPUs with unified memory like Intel.
19537	- *Non-linear image 0xebc91 is aliased with linear buffer 0xeb8e4 which may indicate a bug.*
19538	- It may happen when you use [defragmentation](@ref defragmentation).
19539
19540	\section general_considerations_allocation_algorithm Allocation algorithm
19541
19542	The library uses following algorithm for allocation, in order:
19543
19544	-# Try to find free range of memory in existing blocks.
19545	-# If failed, try to create a new block of `VkDeviceMemory`, with preferred block size.
19546	-# If failed, try to create such block with size / 2, size / 4, size / 8.
19547	-# If failed, try to allocate separate `VkDeviceMemory` for this allocation,
19548	just like when you use #VMA_ALLOCATION_CREATE_DEDICATED_MEMORY_BIT.
19549	-# If failed, choose other memory type that meets the requirements specified in
19550	VmaAllocationCreateInfo and go to point 1.
19551	-# If failed, return `VK_ERROR_OUT_OF_DEVICE_MEMORY`.
19552
19553	\section general_considerations_features_not_supported Features not supported
19554
19555	Features deliberately excluded from the scope of this library:
19556
19557	-# **Data transfer.** Uploading (streaming) and downloading data of buffers and images
19558	between CPU and GPU memory and related synchronization is responsibility of the user.
19559	Defining some "texture" object that would automatically stream its data from a
19560	staging copy in CPU memory to GPU memory would rather be a feature of another,
19561	higher-level library implemented on top of VMA.
19562	VMA doesn't record any commands to a `VkCommandBuffer`. It just allocates memory.
19563	-# **Recreation of buffers and images.** Although the library has functions for
19564	buffer and image creation: vmaCreateBuffer(), vmaCreateImage(), you need to
19565	recreate these objects yourself after defragmentation. That is because the big
19566	structures `VkBufferCreateInfo`, `VkImageCreateInfo` are not stored in
19567	#VmaAllocation object.
19568	-# **Handling CPU memory allocation failures.** When dynamically creating small C++
19569	objects in CPU memory (not Vulkan memory), allocation failures are not checked
19570	and handled gracefully, because that would complicate code significantly and
19571	is usually not needed in desktop PC applications anyway.
19572	Success of an allocation is just checked with an assert.
19573	-# **Code free of any compiler warnings.** Maintaining the library to compile and
19574	work correctly on so many different platforms is hard enough. Being free of
19575	any warnings, on any version of any compiler, is simply not feasible.
19576	There are many preprocessor macros that make some variables unused, function parameters unreferenced,
19577	or conditional expressions constant in some configurations.
19578	The code of this library should not be bigger or more complicated just to silence these warnings.
19579	It is recommended to disable such warnings instead.
19580	-# This is a C++ library with C interface. **Bindings or ports to any other programming languages** are welcome as external projects but
19581	are not going to be included into this repository.
19582	*/
19583