Use Depth Image to Occlude Virtual Objects in Unity

SOLVED
pablocael
Explorer

Use Depth Image to Occlude Virtual Objects in Unity

Hi! Im trying to use Depth Image Texture to discard pixels of virtual objects whose depth are behind real objects. For instance, if my hand passes in front of a virtual cube, it would occlude it.

 

The first naive ideia is to use a shader to directly compare depth values from virtual scene (using Depth Buffer) and real world (using HTC Depth Texture). However, those textures does not have the same proportion.

If there a way of doing this using resources from SRWorks?

 

A more depth think brought me to think that I would need to sample world points and generate a texture for each eye (since occlusion occurs differently for each eye). However I think I would need some kind of "raycast" for generating the depth values.

 

Thanks!

Ethan Lin
Regular
Solution

Re: Use Depth Image to Occlude Virtual Objects in Unity

Hi pablocael,

 

Yes, you can do this in SRWorks now. Please try the following steps:

[In Attached Pic 1]

1. Place a quad under "[ViveSR]->DaulCamera(head)->TrackedCamera(Left)->Anchor(Left)" (As a sibling to ImagePlane-left, and also, check the layer is still "Default")

2. Make the quad local position=(0,0,2), scale=(4.57, 3.42, 1). (I believe that is what you mentioned "does not have the same proportion")

3. Use CameraDepthMaskMaterial, Color Write to "None" (Set to "All" for visualization)

[In Attached Pic 2]

4. When running with enabling depth processing, check the toggle box "Update Depth Material"

5. You can try to toggle some other settings in the blue squares

6. There you go Smiley Happy !

 

Cheers

DepthMask.pngDepthProcess.png

pablocael
Explorer

Re: Use Depth Image to Occlude Virtual Objects in Unity

Thanks! It worked!!

vladstorm_
Contributor

Re: Use Depth Image to Occlude Virtual Objects in Unity

wait, is the hand tracking work already or it's just hands depth on the photo?

 

Moderator

Re: Use Depth Image to Occlude Virtual Objects in Unity

Yes and no, if you check the "Run Depth Mesh Collider" you will be able to collide the dynamic meshes (which can include your hands) with virtual objects but not just your hands exclusuvely.  We'll be sharing code to detect the hands soon enough but you can start here with this solution depending on your use case.

sjobom
Settler

Re: Use Depth Image to Occlude Virtual Objects in Unity

Is there a way to do something similar in Unreal?

Moderator

Re: Use Depth Image to Occlude Virtual Objects in Unity

There should be as the underlying native APIs are the same, so depending if you're ok with looking at and calling native code.  We will be looking into providing similar solutions for UE4 in future updates.

 

 

DaKenpachi
Pioneer

Re: Use Depth Image to Occlude Virtual Objects in Unity

@Ethan Lin, @Dario,

Is there a way to enable depth processing and it´s settings by default (meaning not during runtime), so I don´t have to enable them again everytime I start?

Moderator

Re: Use Depth Image to Occlude Virtual Objects in Unity

ViveSR_DualCameraImageCapture.EnableDepthProcess(true);

Moderator

Re: Use Depth Image to Occlude Virtual Objects in Unity

For an example that shows how to set these settings in code check this hand interaction and occlusion example:  

 

https://github.com/ViveSoftware/ViveSRWorksHand

 

Mieyona
Contributor

Re: Use Depth Image to Occlude Virtual Objects in Unity

I don't have the additional options in the "Depth"-Section.

I downloaded the 0.7.5.0 version.

Did i just missed something or do i need another version?

TKEKakashi
Contributor

Re: Use Depth Image to Occlude Virtual Objects in Unity

Same issue as @Mieyona

I don't have the options shown in the solution in my project.

@Ethan Lin:

Is there a hotfix for the "Vive SR_Dual Camera Image Renderer"-script?

 

I am relatively new to SRWorks and just downloaded the SDK again to make sure it is the newest version...

Moderator

Re: Use Depth Image to Occlude Virtual Objects in Unity

They appear at runtime, To set in a script see the example I posted.
pablocael
Explorer

Re: Use Depth Image to Occlude Virtual Objects in Unity

Hi There. Its been a month since I tried the solution, but now I came back to try the exact same steps, and its not working anymore. Im posting the setup of my DepthMaskQuad.

 

It just has no effects on virtual objects in the scene.

 

pablocael
Explorer

Re: Use Depth Image to Occlude Virtual Objects in Unity

I just notice layer is set to DualCameraLeft, but it was just a test before taking the screenshot. It doesnt work on Default either...

 

;/

Labels