fix render to depth image on Apple Retina displays

[rxba]

Type

• Bug fix (non-breaking change which fixes an issue): Fixes render_to_depth_image() on macbook macos give half width and half height scene but at right resolution #6999 , Depth image saved via render_to_depth_image does not match RGB image #6270 , (probably also fixes macOS - render_to_depth_image() in loop seems error #4978 , It seems that there is a bug in the render_to_depth_image function of Open3DScene. #5980 )
• New feature (non-breaking change which adds functionality). Resolves #
• Breaking change (fix or feature that would cause existing functionality to not work as expected) Resolves #

Motivation and Context

Many users have encountered the issue that an active high resolution Retina display, e.g. an open MacBook screen, causesrender_to_depth_image to only save the lower left quarter of the depth image.
This also happens if an external monitor is connected. The only workaround is to close the MacBook display, i.e. not having an active retina display, and having an external monitor as the primary monitor.
Example:

Here is a small example to recreate the issue (credit to @jerome-godbout-lychens from #6999 ).

import open3d as o3d
import numpy as np
import threading

def normalized_depth_image(depth):
    normalized_depth = (depth - depth.min()) / (depth.max() - depth.min()) * 255
    normalized_depth = normalized_depth.astype(np.uint8)
    return normalized_depth

class Example:

    def __init__(self):
        self._app = o3d.visualization.gui.Application.instance
        self._app.initialize()
        self._window = o3d.visualization.gui.Application.instance.create_window("example", 1024, 768)
        self._scene = o3d.visualization.gui.SceneWidget()
        self._scene.scene = o3d.visualization.rendering.Open3DScene(self._window.renderer)
        self._window.add_child(self._scene)
        self._geom_mat = o3d.visualization.rendering.MaterialRecord()
        self._geom_mat.shader = 'defaultLit'
        self._geom_mat.point_size = 2.0
        self._capture_image = None
        self._capture_depth = None
        self.add_cube()

    def add_cube(self):
        mesh_box = o3d.geometry.TriangleMesh.create_box(width=3.0, height=3.0, depth=3.0)
        mesh_box.compute_vertex_normals()
        mesh_box.paint_uniform_color([0.7, 0.1, 0.1])
        self._scene.scene.add_geometry("cube", mesh_box, self._geom_mat)
        self._scene.look_at([1.5, 1.5, 1.5], [1.5, -6, 1.5], [0, 1, 0]) 

    def display(self):
        self._app.run()
    
    def imageCaptured(self, img):
        self._capture_image = np.asarray(img)
        o3d.io.write_image("demo_captured_image.png", o3d.geometry.Image(self._capture_image))
        self.check_capture_completed()
        
    def depth_captured(self, depth):
        self._capture_depth = np.asarray(depth)
        o3d.io.write_image("demo_captured_image_depth.png", o3d.geometry.Image(normalized_depth_image(self._capture_depth)))
        self.check_capture_completed()
    
    def capture_depth(self):
        self._scene.scene.scene.render_to_depth_image(self.depth_captured)

    def capture_image(self):
        self._scene.scene.scene.render_to_image(self.imageCaptured)
    
    def call_main_thread(self, fct):
        o3d.visualization.gui.Application.instance.post_to_main_thread(self._window, fct)

    def check_capture_completed(self):
        if self._capture_image is None:
            self.call_main_thread(self.capture_image)
            return
        if self._capture_depth is None:
            self.call_main_thread(self.capture_depth)
            return
        # do actual work with both image and depth here

if __name__ == "__main__":
    e = Example()
    t = threading.Timer(2, e.check_capture_completed)
    t.start()
    e.display()

Checklist:

• I have run python util/check_style.py --apply to apply Open3D code style
  to my code.
• This PR changes Open3D behavior or adds new functionality.
  • Both C++ (Doxygen) and Python (Sphinx / Google style) documentation is
    updated accordingly.
  • I have added or updated C++ and / or Python unit tests OR included test
    results (e.g. screenshots or numbers) here.
• I will follow up and update the code if CI fails.
• For fork PRs, I have selected Allow edits from maintainers.

Description

At first glance, this seems very similar to pixel coordinate scaling issues on high resolution displays on MacOS, similar to GLFW framebuffer issues. However, render_to_image() works correctly, it is only render_to_depth_image() that has this issue.

Most of the rendering process is very similar between the two functions. One of the few differences is that the depth version calls ConfigureForColorPicking.

Open3D/cpp/open3d/visualization/rendering/filament/FilamentRenderToBuffer.cpp

Lines 136 to 148 in 5f148f2

	void FilamentRenderToBuffer::CopySettings(const View* view) {
	view_ = new FilamentView(engine_, EngineInstance::GetResourceManager());
	auto* downcast = static_cast<const FilamentView*>(view);
	if (downcast) {
	view_->CopySettingsFrom(*downcast);
	}
	if (depth_image_) {
	// Disable post-processing when rendering to depth image. It's uncessary
	// overhead and the depth buffer is discarded when post-processing is
	// enabled so the returned image is all 0s.
	view_->ConfigureForColorPicking();
	}
	}

, which disables a lot of the post processing that is used for RGB rendering, as it's not required for depth rendering:

Open3D/cpp/open3d/visualization/rendering/filament/FilamentView.cpp

Lines 287 to 293 in 5f148f2

	void FilamentView::ConfigureForColorPicking() {
	view_->setSampleCount(1);
	SetPostProcessing(false);
	SetAmbientOcclusion(false, false);
	SetShadowing(false, ShadowType::kPCF);
	configured_for_picking_ = true;
	}

Surprisingly, it turns out that activating shadowing again for the depth rendering fixes the scaling issue and the depth image saves correctly. I was not able to find any obvious cause for this in Open3D code, so for now I am assuming that setShadowingEnabled in

Open3D/cpp/open3d/visualization/rendering/filament/FilamentView.cpp

Line 199 in 5f148f2

view_->setShadowingEnabled(true);

has some unknown side effect in Filament.

Results of turning shadowing on in FilamentRenderToBuffer::CopySettings if depth_image_:


As far as I can tell, this has no negative effect on the depth image result.
Additionally, turning on shadowing again in FilamentRenderToBuffer::CopySettings could be guarded by #if __APPLE__, but I omitted it for now.

Looking forward to feedback. Would also be thankful if someone else can validate that this fix works for them. Thanks!

[update-docs]

Thanks for submitting this pull request! The maintainers of this repository would appreciate if you could update the CHANGELOG.md based on your changes.

[ssheorey]

Thanks @rxba for finding and debugging this issue! Great detective work!