|  |  |  |  |  |  | 

ARKit Augmented Reality mobile-app-development pantomime Uncategorized Virtual Reality

ARKit Meets Pantomime for Reach-In Augmented Reality


Apple’s new ARKit is bringing motion tracking for augmented reality to billions of devices. But as this video shows, while that’s great for viewing mixed reality scenes from a varying point of view, ARKit’s illusions break down when users get close to objects and surfaces, or try to physically interact. Pantomime’s unique approach supports close-up interaction and reaching in, while ARKit takes Pantomime from small tabletop experiences into effortless room-scale navigation and integration with real spaces.


Pantomime’s founders — alumni of the teams that invented virtual reality and created The Sims — have always seen immersive worlds a bit differently from the “VR headsets will soon be everywhere!” crowd. Long before Pokémon GO, we figured for immersive worlds to be social and scalable — a common lifestyle rather than an isolating novelty — head-mounted displays must be optional.

Pantomime’s strengths have been:

  • viewing and reaching into virtual worlds with handheld devices
  • fast, accurate, low-latency interaction using 3D models
  • device and hand presence in virtual worlds
  • no special hardware required — runs on common devices
  • networked experiences across tablets, phones, PCs and headsets


Pantomime has been betting that software-based 3D motion tracking without markers would soon be a commonplace commodity.

Now that bet is paying off: Apple’s ARKit and Google’s ARCore will provide the 3D tracking and surface recognition to make immersive augmented reality worlds work anywhere — while Pantomime makes them deeply interactive.

ARKit Brings Interactive Pantomime to Real Tables and Floors
Before ARKit, Pantomime provided portable 3D worlds where devices interact on a virtual surface, like a synthetic table or ice rink, directly under the real device. Balance the device on a real surface like a table, and the flat virtual surface lines up perfectly underneath it. Or play in mid air, on a bench or a bus, and still see the the virtual surface you’re playing on. In this kind of portable virtual world, carrying the device moves the virtual world with you. To move a device through a world, you stroke the screen, or walk the device on its edges and corners.

Now with ARKit, instead of being constrained to a portable virtual surface, users’ devices interact on various real surfaces — the floors, tables and desks wherever they’re playing — and can navigate through mid air. Now just by moving and walking around, you can move effortlessly through augmented reality worlds, indoors and out, and enjoy rich interactions with virtual objects everywhere around you and on any nearby surface.

ARKit Limitations
ARKit and ARCore are known to fail if a device is moved too quickly, if its camera is flat against a table or too close to objects, or in poor lighting. It has no model of how the device you’re holding is shaped or supported, never mind how it might collide or interact with augmented reality objects. The closer your device gets to an object, the bigger the problems. The video above shows several of the the issues and Pantomime’s patented solutions.

Pantomime Makes ARKit Reach-In Interactive
Pantomime’s approach, designed for rich close-up interaction, lets users accurately push, paddle, touch, throw, scoop, pour, and squash virtual objects in the scene!

Woven AR — Revolutionary Development Tools
We’ve been building high performance developer tools that integrate Pantomime and ARKit so developers get the best of both worlds.

Our Woven AR Developer Kit is designed to make creating, debugging and tuning AR experiences easier than ever. When developers update and tweak iOS apps, instead of having to rebuild the app from scratch on a Mac each time, developers can update and test changes directly on their iPads and iPhones! This can speed development by a factor of 10 and more.

Here’s an overview diagram of the system:

A Woven AR Developer Kit article includes more details.

ARKit Meets Pantomime for Reach-In Augmented Reality was originally published in Pantomime Corporation on Medium, where people are continuing the conversation by highlighting and responding to this story.

Powered by WPeMatico



%d bloggers like this: