Why the reverse direction is harder
A height map has one value per pixel; a normal map has three, and they are derived from the height map's gradients. Two different height fields can share almost identical gradients, so the mapping is not one-to-one. Reconstruction therefore estimates the height field that best explains the slopes, rather than inverting an exact formula.
What you can and cannot recover
- Recoverable: smooth, continuous surfaces - cloth, skin, terrain, sculpted details.
- Deceptive: hard edges and seams, where the gradient flips and the integrator overshoots.
- Lost: absolute scale and offset, unless the map was generated with a known strength value.
- Fragile: noisy or heavily compressed normal maps - JPEG artefacts integrate into large patches of false height.
How to do it in practice
- In Blender: feed the normal map into a displace modifier's texture slot is not enough - use a normal-map-to-height node chain, or bake a height pass from a plane with the normal applied.
- In Photoshop and similar editors: dedicated 'normal to height' filters integrate the channels and normalise the result.
- In Substance Designer: the Normal to Height node is the standard route, followed by a blur to remove integration noise.
- Command-line / scripted: integrate the X and Y gradients (for example by FFT or Poisson reconstruction) and normalise to the 0-1 range.
Checks before you trust the output
- Compare silhouettes: the recovered height should produce the same edge highlights as the original.
- Look for low-frequency drift - a large light-to-dark gradient across the whole image usually means the integrator accumulated error.
- Re-generate the normal map from your new height map and diff the two - large differences mean information was lost.
- Keep the original normal map. If the height is only for a displacement pass or 3D printing, the normal map remains the better source for shading.
When to go the other way at all
The usual reasons are practical: you need a displacement map for real geometry (3D printing, parallax occlusion, tessellation), a bump map for an older engine, or a height map for terrain tools that only accept one. If none of those apply, use the normal map directly - the round trip only loses detail. And if what you actually want is the opposite conversion, generating a normal map from a height map is lossless and takes seconds with the calculator.