From what I have read so far, commands in a single command buffer can be out of order without explicit synchronization. Here is what the vulkan spec says (https://vulkan.lunarg.com/doc/view/1.0.26.0/linux/vkspec.chunked/ch02s02.html#fundamentals-queueoperation-commandorder)
"The work involved in performing action commands is often allowed to overlap or to be reordered, but doing so must not alter the state to be used by each action command. In general, action commands are those commands that alter framebuffer attachments, read/write buffer or image memory, or write to query pools."
Edit: At first I thought that set state commands would act as some kind of barrier to ensure that draw commands are in order. I have already been explained that this is wrong. So I look at this example of bloom effect in Vulkan https://github.com/SaschaWillems/Vulkan/blob/master/examples/bloom/bloom.cpp
/*First render pass: Render glow parts of the model (separate mesh) to an offscreen frame buffer*/
vkCmdBeginRenderPass(drawCmdBuffers[i], &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);
vkCmdBindDescriptorSets(drawCmdBuffers[i], VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayouts.scene, 0, 1, &descriptorSets.scene, 0, NULL);
vkCmdBindPipeline(drawCmdBuffers[i], VK_PIPELINE_BIND_POINT_GRAPHICS, pipelines.glowPass);
VkDeviceSize offsets[1] = { 0 };
vkCmdBindVertexBuffers(drawCmdBuffers[i], 0, 1, &models.ufoGlow.vertices.buffer, offsets);
vkCmdBindIndexBuffer(drawCmdBuffers[i], models.ufoGlow.indices.buffer, 0, VK_INDEX_TYPE_UINT32);
vkCmdDrawIndexed(drawCmdBuffers[i], models.ufoGlow.indexCount, 1, 0, 0, 0);
vkCmdEndRenderPass(drawCmdBuffers[i]);
/*Second render pass: Vertical blur
Render contents of the first pass into a second framebuffer and apply a vertical blur
This is the first blur pass, the horizontal blur is applied when rendering on top of the scene*/
renderPassBeginInfo.framebuffer = offscreenPass.framebuffers[1].framebuffer;
vkCmdBeginRenderPass(drawCmdBuffers[i], &renderPassBeginInfo, VK_SUBPASS_CONTENTS_INLINE);
vkCmdBindDescriptorSets(drawCmdBuffers[i], VK_PIPELINE_BIND_POINT_GRAPHICS, pipelineLayouts.blur, 0, 1, &descriptorSets.blurVert, 0, NULL);
vkCmdBindPipeline(drawCmdBuffers[i], VK_PIPELINE_BIND_POINT_GRAPHICS, pipelines.blurVert);
vkCmdDraw(drawCmdBuffers[i], 3, 1, 0, 0);
vkCmdEndRenderPass(drawCmdBuffers[i]);
Here are the 2 subpass dependencies used by both render passes
dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
dependencies[0].dstSubpass = 0;
dependencies[0].srcStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
dependencies[0].dstStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
dependencies[0].srcAccessMask = VK_ACCESS_SHADER_READ_BIT;
dependencies[0].dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
dependencies[0].dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT;
dependencies[1].srcSubpass = 0;
dependencies[1].dstSubpass = VK_SUBPASS_EXTERNAL;
dependencies[1].srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
dependencies[1].dstStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
dependencies[1].srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
dependencies[1].dstAccessMask = VK_ACCESS_SHADER_READ_BIT;
dependencies[1].dependencyFlags = VK_DEPENDENCY_BY_REGION_BIT;
My understanding then becomes that these 2 subpass dependencies are responsible for the execution ordering of the render pass but I'm not sure how yet since I'm still fuzzy about subpass dependency. If I'm correct in my understanding can you explain to me why the subpass dependency helps order the draw command? If I'm wrong then what is ensuring the draw command order?
So what is happening is that somehing is rendered to img1
(as color attachment). Then
img1
is sampled, and stuff is written to img2
(as color attachment). Then img2
is sampled and written to a swapchain image.
dependencies[0].srcSubpass = VK_SUBPASS_EXTERNAL;
dependencies[0].srcStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
dependencies[0].srcAccessMask = VK_ACCESS_SHADER_READ_BIT;
dependencies[0].dstSubpass = 0;
dependencies[0].dstStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
dependencies[0].dstAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
For the first and second render pass instance, this possibly blocks some previous sampling of the Resource. Probably from the previous frame. Assuming there is not some other sync between subsequent frames.
dependencies[1].srcSubpass = 0;
dependencies[1].srcStageMask = VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT;
dependencies[1].srcAccessMask = VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT;
dependencies[1].dstSubpass = VK_SUBPASS_EXTERNAL;
dependencies[1].dstStageMask = VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT;
dependencies[1].dstAccessMask = VK_ACCESS_SHADER_READ_BIT;
Now the color attachment is written in VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT
and also more importantly (and conveniently), the Store Operation happens in this same stage for color attachments. It is also always VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT
irregardless whether it is STORE
or DONT_CARE
.
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT
and VK_ACCESS_SHADER_READ_BIT
is again a good match for image sampling (in the fragment shader).
So this means that img1
is fully rendered and stored from the first render pass instance, before it is sampled by the second render pass instance.
And it also means img2
is fully rendered and stored from the second render pass instance, before it is sampled by the third render pass instance.
This is an advanced sample, and you are somewhat expected to already understand synchronization.
State commands are not subject to synchronization. They only change the context of subsequent action commands as soon as they are introduced and typically last until the end of the command buffer, or until the state is changed again.
Subpass dependencies and barriers define a dependency in this way: src
synchronization scope finishes execution before dst
synchronization scope begins execution.
Subpass dependencies and barriers practically the same. Barriers are typically used outside a render pass, while a subpass dependencies inside it. Subpasses are unordered to each other, so subpass dependencies additionally have *Subpass
parameter, and synchronization scopes are limited to only that stated subpass. VK_SUBPASS_EXTERNAL
means that stuff before vkCmdBeginRenderPass
\ after vkCmdEndRenderPass
is part of the synchronization scope.
It takes time to understand the synchronization system, and I cannot properly cover it here. I have little bit more extended answer on barriers at Using pipeline barriers instead of semaphores, otherwisely there is also internet full of resources.
First, thank you for your detailed reply. I think I finally somewhat understand what happen here. First subpass dependency: You want to write to image1, but image1 could possible being read in another render pass of a previous frame (which happens in fragment shader), so you wait for the read to finish in your STAGE_COLOR_ATTACHMENT_OUTPUT. Once it finishes you begin to write.
Second subpass dependency: You are writing to image1 in STAGE_COLOR_ATTACHMENT_OUTPUT but perhaps the second renderpass is already started. The 2nd render pass wants to read/sample image1 in its fragment shader so you make it wait to read in its STAGE_FRAGMENT_SHADER_BIT until 1st render pass’ STAGE_COLOR_ATTACHMENT_OUTPUT has finished writing.
Since image2 and image3’s relationship is the same as image1 and image2’s relationship, the same thing happens in the 2nd renderpass. Is my understanding at this point correct?
There’s only 1 point of confusion left in the first subpass dependency. That is, what if instead of being read, image1 is being written to in STAGE_COLOR_ATTACHMENT_OUTPUT of another render pass? Then you possible have 2 competing writes (1 from a previous renderpass1, 1 from the current renderpass1) and the result could be incorrect, right?
@HenryWise Yea, if the usage was different, then the author would choose different parameters. Or it could result in a data hazard. The order of execution and the results of operations could be incorrect.