DirectX11: Volume raymarching + mesh overlay — axes correspond but Y/Z sign differs (move direction opposite)

by

Summary

A scale sign mismatch in the coordinate system basis between the volume raymarching and the mesh overlay caused opposite translation directions along the Y and Z axes. The root cause is that the volume’s internal data coordinates (patient-space, LPS, or similar) use a handedness or sign convention that differs from the world coordinate system used for the PLY mesh. While the axes are parallel and aligned visually, the signs are inverted (Y and Z), causing world-space translations to move the objects in opposite directions. The robust fix is to introduce a dedicated “patient-to-world” transformation that includes the necessary sign flips, applied consistently before the View/Projection pipeline, and to keep the raymarching shader purely geometric (no UVW flipping).

Root Cause

The root cause is a sign convention disparity between the volume’s native data space and the shared world coordinate system.

  • Volume Space vs. World Space: The volume data (DICOM) is stored in a specific medical coordinate system (e.g., LPS: Left-Posterior-Superior). In a standard DirectX right-handed or left-handed world, the Y (anterior/posterior) and Z (inferior/superior) axes often have opposite directions compared to the volume’s native orientation.
  • Visual Alignment vs. Logical Alignment: The volume and mesh are visually aligned because the raymarching shader or the mesh rendering pipeline implicitly handles the inversion (or the View matrix compensates for it). However, the logical world matrices for both objects are defined in the same coordinate system.
  • The Dot Product Evidence: The dot products dot(VolY, MeshY) = -1 and dot(VolZ, MeshZ) = -1 mathematically prove that the axes are antiparallel.
  • Translation Discrepancy: When translating by a vector V in world space:
    • The mesh moves by V.
    • The volume moves by V * (scale(1, -1, -1) approx).
    • This results in the observed behavior: +Y moves the mesh forward but the volume backward.

Why This Happens in Real Systems

This discrepancy is common in medical imaging and CAD overlays due to historical data conventions and API differences.

  • DICOM/LPS Convention: Medical volume data often uses a left-handed coordinate system where Z increases towards the head (Superior). DirectX often uses a right-handed system where Z increases into the screen (or forward), or a left-handed system where Z increases away from the viewer.
  • Mesh Coordinate Systems: The PLY mesh (face scan) might be exported in a different standard (e.g., Y-up vs. Z-up) or defined relative to a specific sensor coordinate system that aligns visually but has inverted axes.
  • Implicit “Fixes”: Developers often apply “quick fixes” in the rendering loop (like flipping UVW in the shader) to make the image look correct immediately. However, this bakes the fix into the visual representation rather than the data representation, leading to misalignment when applying physical world-space logic (e.g., physics, collision, or generic transforms).

Real-World Impact

  • Physics & Collision Mismatch: If you attempt collision detection or rigid body physics in world space, the volume and mesh will “interpenetrate” or separate incorrectly because their transforms are logically different.
  • Rotation Artifacts: As noted in the input, flipping in the shader or applying local scale flips can break the pivot point of rotation. A rotation around the world origin might cause the volume to orbit differently than the mesh, or shear if the scale is non-uniform.
  • Tool/Measurement Misalignment: If you place surgical guides or measurement tools in world space based on the mesh position, they will appear at the wrong depth or height relative to the volume anatomy.
  • Maintenance Debt: “Magic constants” or inline sign flips in shaders (uvw.y = 1 - uvw.y) are opaque, hard to debug, and prone to errors when upgrading the rendering pipeline or swapping coordinate systems.

Example or Code (if necessary and relevant)

The following CPU-side pseudocode demonstrates the correct way to define the transform matrix. This matrix should be constructed once and applied as the “Model” matrix for the volume object, ensuring it aligns with the mesh’s logical world space.

// Define the necessary sign flip (e.g., Y and Z inverted)
// DirectX uses row-major matrices usually, assuming row-vector multiplication (v*M)
// If using column-major (M*v), the matrix needs to be transposed.
float signY = -1.0f;
float signZ = -1.0f;

// Construct the "Patient-to-World" matrix
// This applies the scale (sign flip) to the local axes of the volume
DirectX::XMMATRIX patientToWorld = DirectX::XMMatrixScaling(1.0f, signY, signZ);

// You likely need to combine this with your existing volume initialization transform
// (e.g., centering, initial orientation). 
// Let's assume 'initialVolumeTransform' is the matrix that places the volume 
// at the correct starting position.
DirectX::XMMATRIX finalVolumeWorldMatrix = patientToWorld * initialVolumeTransform;

// Apply this 'finalVolumeWorldMatrix' to the volume's constant buffer
// The mesh continues to use its standard World matrix (no sign flips applied here).

How Senior Engineers Fix It

Senior engineers separate data representation from visual representation. They standardize the “source of truth” for coordinates.

  1. Define a Canonical World Space: Choose one coordinate system as the “Master World” (usually the mesh’s native space or a standard engineering space, e.g., Y-up, Right-handed).
  2. Apply a Pre-View Transform: Construct a specific Volume Model Matrix that converts the volume’s internal coordinates to the Master World Space.
    • This matrix should include the sign flips (scale(1, -1, -1) or similar).
    • It should be applied before the View/Projection matrix in the vertex shader or constant buffer setup.
  3. Normalize the Raymarching Input: Pass the transformed ray origin and direction to the shader. The shader should only handle ray geometry, not coordinate correction.
    • Ray Origin: WorldPos = mul(VolumeModelMatrix, ObjectPos)
    • Ray Direction: WorldDir = mul((float3x3)VolumeModelMatrix, ObjectDir)
  4. Remove Shader Hacks: Remove any uvw.y = 1.0 - uvw.y logic from the pixel shader. The UVW coordinates (texture coordinates) should be calculated directly from the intersection of the ray (which is now in the correct world space) and the volume bounding box.

Why Juniors Miss It

Juniors often prioritize visual correctness over logical consistency, leading to “patchwork” solutions.

  • Treating the Symptom, Not the Cause: They see the volume is flipped relative to the mesh and immediately flip the texture coordinates in the shader (uvw = 1.0 - uvw). This fixes the image but breaks the math.
  • Lack of Matrix Fluency: Confusion between Row-Major vs. Column-Major matrices, or coordinate space transformations (Object -> World -> View -> Projection). They might multiply matrices in the wrong order or apply scales locally when they should be applied globally.
  • Misunderstanding Handedness: They might assume that because X matches, Y and Z must match. They overlook that medical data (LPS) and graphics APIs (RHS/LHS) often have different defaults for “Up” and “Forward”.
  • Debugging via Visuals: Relying solely on “does it look right on screen?” rather than “do the translation vectors match the mathematical basis?”.
  • Overlooking Pivot Points: Applying scale(-1, 1, 1) to a world matrix often inverts the rotation pivot or causes “mirroring” effects during animation if the hierarchy isn’t handled correctly, which is why a dedicated Model matrix is safer.