My entry for Autodesk’s first Global Hackathon: a HoloLens-based tool for navigating low visibility environments

We have an internal company Hackathon being held at Autodesk locations around the world on September 6th and 7th. The theme for the event is "What's Next?" and there are variety of sub-themes under which the hacks can fall:

  1. Computer as collaborator
  2. Future of product interfaces
  3. Internet of Things
  4. Social good / Sustainability
  5. The future of anything else

I signed up some time ago to be an "evangelist" for the event, which means I'm a resource for people in the local Neuchatel office should they have questions about VR/AR and IoT (which fall under the second and third sub-themes, respectively), in particular. I wasn't sure I was going to participate in the Hackathon as part of a team, but after chatting over some ideas with colleagues in the office in Friday afternoon, I decided on one that would be pretty interesting to take on.

The inspiration for the idea was two-fold. Having played with the HoloLens over the last month or so, I've become very interested in the possible benefits of spatial sound. It's a great way for you to find a "lost" hologram in your environment, so why couldn't it be used as a way of getting unlost, yourself? The second inspiration was the excellent Pixar movie "Brave". And one scene, in particular, where the heroine is led through a forest by will-o-the-wisps.

 

 

My idea is that someone in a low visibility environment – and this could be because they're themselves blind or perhaps in an emergency response situation – could be led to an exit (or simply their destination) via spatial sound. They'd be led through a series of waypoints – with sound emanating from the closest – until they reach their destination. And as it doesn't cost anything to actually show the waypoints – which would in any case be useful for emergency responders – I've decided to display these as holograms in their own right, perhaps having a single hologram that moves from waypoint to waypoint along the projected path until it's completed. For now I'm calling this project the HoloGuide.

On the route planning… HoloLens has spatial mapping capabilities, too, but I'm not sure how effective it would be for planning a complete route from point A to point B: unless we capture and analyse a complete map of the environment beforehand, it probably makes more sense to have the route planned externally (perhaps in AutoCAD) and have the waypoints transmitted to the HoloLens (I can imagine using this approach to split complete paths into a series of waypoints). Local spatial mapping will no doubt be helpful for obstacle avoidance, at least. Although as I doubt very much HoloLens would be able to map anything at all in a burning building, so this is really just to test out the idea in a limited fashion.

In time I expect it'd be feasible to use a Google Maps-like walking directions tool to provide the broader path planning – with HoloLens adjusting for local anomalies – but for this hack we can focus on the use-case where we have external support personnel handling that.

Here's a quick display test I did with a simple glowing, rotating sphere (with some particles thrown in, for fun) as a possible waypoint marker with the spatial mapping being shown in the background, too.

HoloGuide prototype

Interestingly the video recording shows the internals of the sphere as dark, while in the headset it's very much transparent.

Anyway, that's my idea for the first Autodesk corporate Hackathon. (If you're an Autodesk employee and what to check it out or comment/vote on it, please go here.) It's shaping up to be a fun project!

5 responses to “My entry for Autodesk’s first Global Hackathon: a HoloLens-based tool for navigating low visibility environments”

  1. That's a great idea.
    I can see that, with the proliferation of BIM models for newer buildings, that data could be made more accessible to help people navigate through cities and buildings, with some tweaks to help adjust for local anomalies of course.

    1. Absolutely!

      For my own purposes I'm going to import a 2D DWF of my office into AutoCAD, draw a polyline path on top of it and send it out to HoloLens. Assuming I can get the scale to match, it should work well. So with nothing close to a full BIM (although I'm also not dealing with the issue of how I get down from the 3rd floor... :-).

      Kean

  2. Interesting, I was actually thinking the opposite, as my brother is deaf. If he could point the 'lens at someone, and see the words they were saying as closed captioning, or even just something to indicate they were talking, that might be useful. Deaf people have to analyze vibration, and they learn a lot by just holding their hand on someone's neck to feel the sound for known words. I'm always amazed that helps their speech, that must be really hard. Anyway, a sound graph under the closed caption words would be handy as they would be watching their lips as it all happens. With other tools, you have to watch a screen and its not real time.

    1. Interesting... I can imagine the speech analysis used to detect the user's voice commands could also be used to decode the speech of others.... otherwise, yes, a sound graph of some kind.

      Going somewhat off topic, a colleague showed this to me, the other day. Captions created by a neural network analysing a video, frame by frame. Things are definitely getting interesting (even if not really viable today):

      youtu.be/FmSsek5luHk

      Kean

      1. ha ha, check out frame at 0:43, "a boy is sitting on the ground with a banana", or 0:38 "a black and white photo of a microwave". I guess you have to start somewhere, now for that debugging step.

Leave a Reply

Your email address will not be published. Required fields are marked *