What's the correct way to upload the msdf generated atlas to GPU using Vulkan?

[GasimGasimzada]

I am trying to integrate msdf-atlas-gen into my engine to render texts in the scene and I have come across a weird issue when uploading the textures in Vulkan. The image is flipped vertically + there are some differences between the location of fonts.

I am using the same snippet as the one where the whole atlas is generated:

std::vector<GlyphGeometry> msdfGlyphs;
FontGeometry fontGeometry(&msdfGlyphs);
fontGeometry.loadCharset(font, 1.0, Charset::ASCII);

for (auto &glyph : msdfGlyphs) {
  glyph.edgeColoring(&msdfgen::edgeColoringInkTrap, MAX_CORNER_ANGLE, 0);
}

TightAtlasPacker packer;
packer.setDimensionsConstraint(
    TightAtlasPacker::DimensionsConstraint::SQUARE);
packer.setMinimumScale(MINIMUM_SCALE);
packer.setPixelRange(PIXEL_RANGE);
packer.setMiterLimit(1.0);
packer.pack(msdfGlyphs.data(), static_cast<int>(msdfGlyphs.size()));

int width = 0, height = 0;
packer.getDimensions(width, height);

ImmediateAtlasGenerator<float, 4, &mtsdfGenerator,
                        BitmapAtlasStorage<byte, 4>>
    generator(width, height);

GeneratorAttributes attributes;
generator.setAttributes(attributes);
generator.setThreadCount(1);
generator.generate(msdfGlyphs.data(), static_cast<int>(msdfGlyphs.size()));

std::unordered_map<uint32_t, FontGlyph> glyphs;

auto fWidth = static_cast<float>(width);
auto fHeight = static_cast<float>(height);

// my custom asset type for fonts
FontAsset fontAsset{};
fontAsset.msdfGlyphs = msdfGlyphs;
fontAsset.msdfAtlas = generator.atlasStorage();

return assetManager.addFont(fontAsset);

After MSDF fonts are loaded, I upload them to my the render device (using Vulkan):

rhi::TextureDescription description{};
msdfgen::BitmapRef<msdf_atlas::byte, 4> ref = font.data.msdfAtlas;
description.data = ref.pixels;
description.width = ref.width;
description.height = ref.height;
description.usage = rhi::TextureUsage::Color |
                    rhi::TextureUsage::TransferDestination |
                    rhi::TextureUsage::Sampled;
description.format = VK_FORMAT_R8G8B8A8_UNORM;
description.size = sizeof(msdf_atlas::byte) * 4 * ref.width * ref.height;

auto textureHandle = deviceRegistry.setTexture(description);

Expected (generated using msdf-atlas-gen application for comparison):

Actual (captured from RenderDoc):

I opened Windows Image Viewer and flipped the image vertically and there are differences between the order of characters (e.g first row last column is "W" instead of "A"):

At last, I am using MTSDF in my codebase (I am using N = 4 for the AtlasStorage and ImmediateAtlasGenerator) but I am getting results similar to MSDF instead of MTSDF (example generated using the application):

What am I doing wrong here? When I upload the texture data to the GPU, I just copy the raw texture data without any modifications (I have tested it with regular images such as PNG); so, from my understanding, this is something that I need to handle during atlas generation.

[Chlumsky]

• Your image is flipped because msdf-atlas-gen represents bitmaps as starting from the bottom row, which is the case for OpenGL, but not Vulkan.
• The characters are ordered differently because not all of the parameters are exactly the same, so the algorithm simply packs it differently.
• Seeing the same result for MTSDF as with MSDF happens most likely because you do not actually interpret the alpha channel when viewing your image data. Ignoring the alpha channel, the RGB channels will be the same. You can check if this is the case by calling msdfgen::savePng after generating your image.

[GasimGasimzada]

Thank you for the answer

Your image is flipped because msdf-atlas-gen represents bitmaps as starting from the bottom row, which is the case for OpenGL, but not Vulkan.

I am a bit new to font atlases (I know they are similar to texture atlases but I have never generated on the fly from a font file); so, I have one more question about this. Does this mean that the glyph positions are also based on the orientation of the bitmap? So, I can't just flip the font atlas and use the glyphs -- the glyphs also need to be flipped, right? (Or I don't need to flip the fonts but flip the rectangle that has the glyph texture rendered on it)

The characters are ordered differently because not all of the parameters are exactly the same, so the algorithm simply packs it differently.

Good to know as long as glyphs are aligned with the atlas!

Seeing the same result for MTSDF as with MSDF happens most likely because you do not actually interpret the alpha channel when viewing your image data. Ignoring the alpha channel, the RGB channels will be the same. You can check if this is the case by calling msdfgen::savePng after generating your image.

This was a bug on my part because I was accidentally stacking multiple MSDF images on top of each other and also I did not have color blending enabled. I fixed the bug and the textures are rendered properly.

[Chlumsky]

You can flip the whole bitmap, you can flip individual glyphs, you can keep the texture inverted and draw it upside down, and you can even probably configure the atlas generator so that it generates the images already flipped. All of the above will work, you just need to make sure to recompute the positioning data appropriately to the method you choose.

[Stanlyhalo]

Hey, I see that you put the pixels into a description.data; what is that type? I'm also using vulkan, and my data type takes in a void* but to render, I pass in a std::vector<unsigned char> type for my texture to be loaded... but if I try ref.pixels, I get buffer errors. I'm just curious about this because I can't seem to find any other person who has some code to store the data of the atlas.

[GasimGasimzada]

Hi, it has been a while but I checked my code and I am storing atlas data in memory using unsigned char (aka std::byte). I create the pixels in the following way:

std::vector<std::byte> pixels(static_cast<size_t>(bitmap.width) * bitmap.height * NumChannels);

// Copy bitmap data from bottom to up
for (int y = 0; y < bitmap.height; ++y) {
  memcpy(&pixels[NumChannels * static_cast<size_t>(bitmap.width) * y],
         bitmap(0, bitmap.height - y - 1),
         NumChannels * static_cast<size_t>(bitmap.width));
}

Then, I upload the data to the GPU by passing the reference and the size:

rhi::TextureDescription description{};
description.data = font.data.atlas.data();
description.size = font.size;
// ...other code

• Font asset: https://github.com/GasimGasimzada/liquid-engine/blob/main/engine/src/liquid/asset/FontAsset.h#L33
• MSDF loader: https://github.com/GasimGasimzada/liquid-engine/blob/main/engine/src/liquid/text/MsdfLoader.cpp#L106
• Uploading font pixels to GPU: https://github.com/GasimGasimzada/liquid-engine/blob/main/engine/src/liquid/asset/AssetRegistry.cpp#L52