In this project, we explored how to add haptics to walls and other heavy objects in virtual reality. Our main idea is to prevent the user’s hands from penetrating virtual objects by means of electrical muscle stimulation (EMS). Figure 1a shows an example. As the shown user lifts a virtual cube, our system lets the user feel the weight and resistance of the cube. The heavier the cube and the harder the user presses the cube, the stronger a counterforce the system generates. Figure 1b illustrates how our system implements the physicality of the cube, i.e., by actuating the user’s opposing muscles with EMS.

Screen Shot 2017-03-16 at 6.49.46 PM

Figure 1: (a) As this user lifts a virtual cube, our system lets the user feel the weight and resistance of the cube. (b) Our system implements this by actuating the user’s opposing muscles using electrical muscle stimulation.

In fact, when the user grabs the virtual cube, the user expects the cube’s weight to create tension in the user’s biceps and the cube’s stiffness to create a tension in the user’s pectoralis. In order to create this sensation, the system actuates the respective opposition muscles. In order to put a load onto the user’s biceps, it actuates the triceps and in order to put a load onto the user’s pectoralis, it actuates the user’s shoulder muscle. This creates the desired tension in biceps and pectoralis, thereby creating the desired experience.

As illustrated in Figure 2, our system stimulates up to four different muscle groups. Through combinations of these muscle groups, our system simulates a range of effects. When pushing a button mounted to a vertical surface, for example, the system actuates biceps and wrist. In the Example Widgets section, we detail how this allows our system to simulate a wide range of objects, including walls, shelves, buttons, projectiles, etc.

Screen Shot 2017-03-16 at 6.54.05 PM

Figure 2: We use up to 8 electrode pairs, actuating (a) wrist, (b) biceps, (c) triceps, and (d) shoulders.

Our system can be worn in a small backpack, as shown in Figure 3. The backpack contains a medical grade 8‑channel muscle stimulator, which we control via USB from within our VR simulators. We use our system in the context of a typical VR system consisting of a head-worn display (using a Samsung/Oculus GearVR) and a motion capture system.

The problem of naïvely trying to render a Wall using EMS:

Screen Shot 2017-03-16 at 7.06.53 PM

Figure 3: Implementing rigid walls requires stimulating muscles with strong impulses over long periods. This draws undesired attention to the electrical stimulation.

Figure 3 illustrates the naïve approach to rendering objects using EMS: (a) From the moment the user’s fingertips reach the virtual wall, we actuate the user’s hand just strongly enough to prevent it from passing through. We achieve this with a current essentially proportional to the user’s force (further details in Implementation).

When we built this version, the results looked great. The design prevents the user’s hand from passing through the object and thus bystanders observing the scene would typically conclude that the illusion was “working”. However, during piloting, it became clear that this design did not work. Since the EMS actuation was as long and as strong as the user kept pushing, the EMS signal (a tingling in the respective muscles) could become arbitrarily strong. This would draw the user’s attention to the EMS-actuated muscles. These, however, were pointed in the wrong direction, i.e., they were pulling, when the sensation was supposed to be about pushing. One participant in our pilot said this design felt “like a magnet pulling the hand backwards”. While this design was reasonably impermeable, consistent, and definitely familiar, the strong EMS signal made this design fail with respect to our primary objective: it was not believable.

We, therefore, created two alternative designs with the objective of increasing believability:

Screen Shot 2017-03-16 at 7.30.03 PM

Figure 4: Our solutions: (a) soft design and (b) repulsive design.

The soft object design (Figure 4b): We created our first alternative design by applying a cut-off to EMS intensity. We picked a reasonably low cut-off, allowing users to penetrate objects by about 10 cm. This resulted in a design that produced the impression of soft objects. Based on this general concept, we explored various visuals, including the soft surface material shown in Figure 1, which is designed to suggest an increasingly solid inside under a soft, permeable surface. This general design became the basis for most of our object designs Figure 4b shows the same concept wrapped in visuals suggesting a magnetic field, suggesting a magnetic force that carefully pushes the user’s hand backwards. In some versions of this design, we attached a block of metal to the back of users’ hands to suggest that the magnetic field would apply there in order to affect the hand.

The repulsion object design (Figure 4c): We created our second alternative design by reducing the duration of our EMS signal. This resulted in what we call repulsion objects. This design uses a brief EMS pulse (of 200-300 ms, using the user’s calibrated maximum intensity) where the EMS propels the user’s hand backwards, removing it from the virtual object it is trying to touch. We achieve this with an EMS pulse of still reasonably low intensity, which, like all other EMS signals in our system, is pain-free at all times (for EMS pulses of similar intensity see our project Impacto).

Benefits and Contribution

Our main contribution is the concept of providing haptics to walls and other heavy objects by means of electrical muscle stimulation. We achieve this in a wearable device, suitable for real-walking virtual reality environments. We validated our prototype and our two haptic designs in two user studies.

Paper Reference

Lopes, P., Young, S., Cheng, L., Marwecki, S., and Baudisch, P.
Providing Haptics to Walls and Other Heavy Objects in Virtual Reality by Means of Electrical Muscle Stimulation.
In Proc. CHI’17.  (Download paper here.)

Press Photos

PressKit for this project (hi-res photos).

Collaborators

Sijing You, Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch