Android hand tracking - starting tips ?

MDV Games
Pioneer

Re: Android hand tracking - starting tips ?

@zzy 

In this release will available Unreal Plugin for Vive Focus?

Vive Staff

Re: Android hand tracking - starting tips ?

Hi @MDV Games, yes the next version includes Android phone and Focus support in Unreal plugin

MDV Games
Pioneer

Re: Android hand tracking - starting tips ?

@zzy 

Great! Thanks!

Vive Staff

Re: Android hand tracking - starting tips ?

Re: Android hand tracking - starting tips ?

Hey,
I've tried the update and there is a progress - it actually started!
I can see the green and blue balls flying where my hand should be, thats unbeliveable.

And it recognizes gestures too! for now only pointing fist and all five works, "ok" gesture didnt trigger nothing yet.

Phone gets hot pretty quick.

And it seem to lose track of hand centre and jump to fingers but thats ok.

Sometimes hand tracking stops and doesnt want to start but after app forced close it worked again, 

Implying that Aristo service runs after app was closed or stays in memory.

Anyway im pretty excited for this technology to mature and ways I could use it!

 

Dropping logcat for good measure:

PS C:\Users\USER> adb logcat -s Unity Aristo
--------- beginning of crash
--------- beginning of main
06-01 17:30:43.501 26103 26125 I Unity   : SystemInfo CPU = ARMv7 VFPv3 NEON, Cores = 8, Memory = 3679mb
06-01 17:30:43.501 26103 26125 I Unity   : SystemInfo ARM big.LITTLE configuration: 4 big (mask: 0xf0), 4 little (mask: 0xf)
06-01 17:30:43.502 26103 26125 I Unity   : ApplicationInfo com.XRInstitute.HandTracking version 0.1 build db70123d-ea30-48e1-90d4-ca22b1ff63dc
06-01 17:30:43.502 26103 26125 I Unity   : Built from '2019.1/staging' branch, Version '2019.1.4f1 (ffa3a7a2dd7d)', Build type 'Release', Scripting Backend 'mono', CPU 'armeabi-v7a', Stripping 'Disabled'
06-01 17:30:43.843 26103 26125 E Unity   : Unable to find libaudiopluginresonanceaudio
06-01 17:30:43.908 26103 26125 D Unity   :  GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_
06-01 17:30:43.908 26103 26125 D Unity   : to_texture GL_EXT_multisampled_render_to_texture2 GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_sh
06-01 17:30:43.908 26103 26125 D Unity   : ader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_shader_non_constant_global_initializers GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_protected_textures GL_EXT_buffer_storage GL_EXT_external_buffer GL_EXT_EGL_image_array
06-01 17:30:46.089 26103 26125 I Unity   : Created eye textures with a "separate" layout.  The "multi-pass" stereo mode will be used.
06-01 17:30:46.089 26103 26125 I Unity   :
06-01 17:31:01.528 26103 26125 I Unity   : GVR SDK for Unity version: 1.200.0
06-01 17:31:01.528 26103 26125 I Unity   :
06-01 17:31:01.528 26103 26125 I Unity   : (Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 48)
06-01 17:31:01.528 26103 26125 I Unity   :
06-01 17:31:02.225 26103 26125 E Unity   : AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference
06-01 17:31:02.225 26103 26125 E Unity   : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference
06-01 17:31:02.225 26103 26125 E Unity   :      at com.google.gvr.permissionsupport.PermissionsFragment.hasPermission(PermissionsFragment.java:88)
06-01 17:31:02.225 26103 26125 E Unity   :      at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
06-01 17:31:02.225 26103 26125 E Unity   :      at com.unity3d.player.UnityPlayer.c(Unknown Source:0)
06-01 17:31:02.225 26103 26125 E Unity   :      at com.unity3d.player.UnityPlayer$e$1.handleMessage(Unknown Source:88)
06-01 17:31:02.225 26103 26125 E Unity   :      at android.os.Handler.dispatchMessage(Handler.java:102)
06-01 17:31:02.225 26103 26125 E Unity   :      at android.os.Looper.loop(Looper.java:214)
06-01 17:31:02.225 26103 26125 E Unity   :      at com.unity3d.player.UnityPlayer$e.run(Unknown Source:20)
06-01 17:31:02.225 26103 26125 E Unity   :   at UnityEngine.AndroidJNISafe.CheckException () [0x00091] in <549ed8055ed14737ac3b841d26182fec>:0
06-01 17:31:02.225 26103 26125 E Unity   :   at UnityEngine.AndroidJNISafe.CallBooleanMethod (System.IntPtr obj, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00011] in <
06-01 17:32:29.053 27650 27670 I Unity   : SystemInfo CPU = ARMv7 VFPv3 NEON, Cores = 8, Memory = 3679mb
06-01 17:32:29.053 27650 27670 I Unity   : SystemInfo ARM big.LITTLE configuration: 4 big (mask: 0xf0), 4 little (mask: 0xf)
06-01 17:32:29.054 27650 27670 I Unity   : ApplicationInfo com.XRInstitute.HandTracking version 0.1 build db70123d-ea30-48e1-90d4-ca22b1ff63dc
06-01 17:32:29.054 27650 27670 I Unity   : Built from '2019.1/staging' branch, Version '2019.1.4f1 (ffa3a7a2dd7d)', Build type 'Release', Scripting Backend 'mono', CPU 'armeabi-v7a', Stripping 'Disabled'
06-01 17:32:29.308 27650 27670 E Unity   : Unable to find libaudiopluginresonanceaudio
06-01 17:32:29.398 27650 27670 D Unity   :  GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_
06-01 17:32:29.398 27650 27670 D Unity   : to_texture GL_EXT_multisampled_render_to_texture2 GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_sh
06-01 17:32:29.398 27650 27670 D Unity   : ader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_shader_non_constant_global_initializers GL_EXT_color_buffer_half_float GL_EXT_color_buffer_float GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_protected_textures GL_EXT_buffer_storage GL_EXT_external_buffer GL_EXT_EGL_image_array
06-01 17:32:31.092 27650 27670 I Unity   : Created eye textures with a "separate" layout.  The "multi-pass" stereo mode will be used.
06-01 17:32:31.092 27650 27670 I Unity   :
06-01 17:32:49.240 27650 27670 I Unity   : GVR SDK for Unity version: 1.200.0
06-01 17:32:49.240 27650 27670 I Unity   :
06-01 17:32:49.240 27650 27670 I Unity   : (Filename: ./Runtime/Export/Debug/Debug.bindings.h Line: 48)
06-01 17:32:49.240 27650 27670 I Unity   :
06-01 17:32:49.896 27650 27670 E Unity   : AndroidJavaException: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference
06-01 17:32:49.896 27650 27670 E Unity   : java.lang.NullPointerException: Attempt to invoke virtual method 'int android.app.Activity.checkSelfPermission(java.lang.String)' on a null object reference
06-01 17:32:49.896 27650 27670 E Unity   :      at com.google.gvr.permissionsupport.PermissionsFragment.hasPermission(PermissionsFragment.java:88)
06-01 17:32:49.896 27650 27670 E Unity   :      at com.unity3d.player.UnityPlayer.nativeRender(Native Method)
06-01 17:32:49.896 27650 27670 E Unity   :      at com.unity3d.player.UnityPlayer.c(Unknown Source:0)
06-01 17:32:49.896 27650 27670 E Unity   :      at com.unity3d.player.UnityPlayer$e$1.handleMessage(Unknown Source:88)
06-01 17:32:49.896 27650 27670 E Unity   :      at android.os.Handler.dispatchMessage(Handler.java:102)
06-01 17:32:49.896 27650 27670 E Unity   :      at android.os.Looper.loop(Looper.java:214)
06-01 17:32:49.896 27650 27670 E Unity   :      at com.unity3d.player.UnityPlayer$e.run(Unknown Source:20)
06-01 17:32:49.896 27650 27670 E Unity   :   at UnityEngine.AndroidJNISafe.CheckException () [0x00091] in <549ed8055ed14737ac3b841d26182fec>:0
06-01 17:32:49.896 27650 27670 E Unity   :   at UnityEngine.AndroidJNISafe.CallBooleanMethod (System.IntPtr obj, System.IntPtr methodID, UnityEngine.jvalue[] args) [0x00011] in <
--------- beginning of system
Vive Staff

Re: Android hand tracking - starting tips ?

Hi @VirtualPierogi 

 

Thanks for trying out. For the "OK" gesture to work, please make sure:

1. the circle made by thumb and index is visible to the camera

2. the other three fingers are all visible to the camera, i.e. ring and pinky finger are not hidden by middle finger

Labels