Touching Your Own Future: Haptic Tools
Date: Jan 28, 2005
I've had a bit to say in other articles about where haptic research is headed these days, about how the gurus of grasp are taking us by the hand and leading us into synthetic worlds that blur the differences between human and machine, so I thought it was time to kick back and have a look around at the goodies you can buy this afternoon to make life with your PC pretty virtual. But while I was busily investigating the potential of assorted haptic mice and force-feedback steering wheels, I also landed smack in a book by Andy Clark, called Natural Born Cyborgs: Minds, Technologies and the Future of Human Intelligence. It takes the mind-punching point of view that we humans are not just becoming one with the machine, not just tripping down hell's high road that will lead eventually to our all becoming cyborgs; we are cyborgs already. What's more, we always have been.
Because of our cortical plasticity, tools become an extension of not just the human hand but the human brain as well.
Who could resist a theory like that?
Clark is keen on what he calls "haptic touch," which, to those of us who think haptic refers to touch, makes as much sense right out of the Cracker Jack box as talking about "audio hearing."
So I decided I had better ask him what he meant. I found him at the University of Edinburgh, where he is Professor of Logic and Metaphysics. (Perhaps I should explain that I did not go to Edinburgh, but found him virtually, by reaching out through electronic tools—in the manner he discusses in his book.)
That's how I learned that in ecological psychology, "haptic touch" is the name of the system that lets us feel stuff through objects we hold, to "feel the road through the stick or cane, or even through the wheels of a car we are driving." He explained that, in this instance, "force and torque sensors buried in the arm" and other muscles are "activated the same way when we use a stick as when we touch with our hand." This is what allows humans (and a goodly number of our primate cousins—not to mention possibly raccoons and otters) to treat tools as if they were extensions of our bodies, extending our sense of touch through them.
This is a concept that power computer users can really get behind and give a hearty shove. Have you ever heard a connoisseur shopping for a new input device? They zip through a dozen of them with a disdain that would abash Goldilocks, muttering "too large," "too small," "cramps my wrist," "button hard to reach," and "really need a cordless" until that final sigh is accompanied with a "just right" and the satisfied twitch of the chosen rodent or track ball.
"Aha!" said I aloud to myself, on pondering this matter. "Those tools that offer us haptic feedback, being both input and output devices for the old noggin, become double brain-extenders." I began to see that Andy Clark had a worthwhile word to say about haptics. In fact, as I sailed around finding new touch-related I/O goodies to talk about, I began to see what he meant about our cyborg-ness. Thus, never one to fight technology or beat back cognitive science, I say we embrace the inner android and get about the business of buying the best haptic goodies we can find.
So onward—to our Virtual Bazaar in which haptic extensions of our brains are available for purchase (or soon will be).
In the consumer realm, two companies dominate the field in the creation of tactile I/O devices: Immersion Corporation and SensAble Technologies. Right now, each seems interested in consolidating a position in the marketplace.
Immersion has acquired more than 240 patents and presents products ranging from consumer electronics to medical simulations. It has defended its intellectual property fiercely, most recently and successfully in a court case against Sony. It claimed that the PlayStation DualShock's vibrating controllers constituted patent infringement and made a good enough court case to be awarded $82 million in the process.
SensAble, which has an impressive list of patents of its own, markets tools for developers who want to create haptic applications. Recognizing the need to consolidate its own market presence, it is sponsoring a contest for developers that should fatten their portfolio of projects.
Sadly, production of usable commercial haptic devices lags well behind the challenging and sometimes quirky work done in university laboratories. Academic research is by its nature open-ended. However, after a developer enters the marketplace with a device or concept, the device has to do something concrete for people (to make them pull out their pocketbooks and fork over the greens). And chief among those consumers willing to pay for any enhancement to their virtually real experience are gamers.
Climbing into a Virtual World
So I asked Andy Clark about computer gaming and how he saw it shaping our human cognitive evolution.
"It's one of the big ways we are learning to get used to new bodies operating in new environments," he told me. "A key tool for a major transition."
Ha! You knew it all along, right? Those at the gaming consoles are the ones who will be able to cope with the upcoming shifting realities. Meanwhile, as you are practicing adjusting to new worlds, it's a good idea to have a strong feel for the virtual road.
There are a slew of haptic mice, joysticks, touchpads, and steering wheels out there (and have been for years), but those who buy them have frequently been dissatisfied that so little game software supports haptic hardware. To get around this issue, some developers link the peripheral to the sound card. Devices such as the ABV Vibration Mouse by AVB Technologies, for example, plug right into the sound card.
Many force-feedback devices use a software controller such as Immersion Touchware to smooth out the relationship between the haptic peripheral and the game. The vibration is still linked to the sound card, but the additional software interface allows you to create settings for each game and save the profiles. The Immersion Web site lists more than 20 such gaming devices from several companies that you can count on to be compatible with their software.
Although these alternative manipulators do add the dimension of touch to your gaming, they do not allow you to grasp the gold ring with a reliable hand-hold. For that kind of human-computer interface, you need a sense-extender like the CyberGrasp, a haptic feedback glove that receives force information from the computer, giving you the sense that you are holding, or touching, a virtual object. The glove sends information on your hand movements to the computer as you watch the monitor, and you see these hand movements echoed there, creating a virtual reality you can see and touch. Immersion has been perfecting this product for several years and, most recently, has delivered a wireless version.
That's where the gaming market is these days; the snazzy device that is new today will be polished and fixed for years to come in versions 2.0, 3.1, and 22.17.
Assistive Haptic Tools
Any place where the edge of experimental technology pushes into the mainstream market is the occasional rough intersection. This has been true in the quest for a tactile computer interface, something useful for kinesthetic learners and essential for the visually impaired. It has been a trick to coordinate hardware and software at a cost that was not prohibitive. But developers seem to have done it.
In an excellent pairing, Immersion Desktop software has been incorporated into the Logitech iFeel MouseMan to create a haptic interface that works with Windows 98, Windows 2000, Internet Explorer, and Netscape. Users feel resistance or vibration that is triggered each time the cursor encounters buttons, windows, links, and icons. In short, the mouse-mover can navigate by feel. For those who want a bit more help, the iFeelPixel Web site offers free software downloads for audio and texture add-ins, supported by Logitech and Immersion.
Learning Something New
Okay, so let's get back to Andy Clark and his theories that we humans, as dedicated tool users, are cyborgs pretty much from the time we hatch. If each new tool becomes an extension of our minds as soon as we grok it, you might think the fields of education and training would be full of all sorts of exciting haptic applications, ready to shove our little brain waves out in all directions. In many cases, this is true.
Medical training, for example, has embraced haptic thingamajigs with enthusiasm. A suite of medical products from Immersion, for example, allows health care providers to learn in various simulated environments. These products guide practitioners in techniques from intravenous therapies to surgery on virtual human tissues that resist cutting exactly like the real thing.
One recent innovation, developed jointly by the Institut National des Sciences Appliquées and the Hôpital de la Croix Rousse (both in Lyon, France), allows students to deliver a virtual baby. BirthSIM uses a pneumatic drive controlled by a computer to simulate the contractions that occur during the journey of a baby through the mother's pelvis. Haptic feedback sensors register the amount of pressure students exert on the baby's head to train students in proper technique and to record these readings for later evaluation and correction.
Obviously, this simulator is not a mass-market item. And that is fairly common in the haptic devices used in education. For that reason, don't expect to see haptic teaching aids in your local elementary school. Although research promises some delightful devices, funding and demand haven't moved them out into the general market.
A few promising technologies in the realm of virtual learning are being developed at the University of Washington Human Interface Technology (HIT) Lab and the associated HIT Lab in New Zealand. These projects use a HIT Lab technology called ARToolkit that lets users looking through a display see an augmented reality (AR) object—a virtual component perched on a real card that can be held in the hand. One example is the Virtual Calakmul project, a simulation of the remains and artifacts of the important ancient Mayan city discovered in 1931. Another, the MagiPlanet display, allows users to see 3D planets poised above each of a set of planet cards. When these cards are arranged in the proper order to reflect their orbits around the sun, the "planets" begin to orbit.
"While this approach does not provide force feedback, it has a built-in haptic aspect," says Suzanne Weghorst, Assistant Director of Research at the UW HIT Lab. These projects allow "manipulation of ARToolkit markers with virtual objects attached, so there is a kinesthetic/proprioceptive component."
Hand-Held Haptic
Andy Clark has quite a lot to say in his book about the cell phone and how it's used as a tool. He sees it "like a prosthetic limb over which you wield full and flexible control, and on which you eventually come to automatically rely in formulating and carrying out your daily goals and projects." When I read that, it struck me as unnervingly accurate, the more so as I pondered how to get the guy sitting next to me at the Starbucks to turn the ringer down on his extra limb.
The technologies I cite here are not quite ready for purchase, but are expected soon, and I have to think they're just what Andy would have been expecting.
One snazzy technology from a Finnish company, F-Origin, really makes strides toward getting us tools that almost let us think the changes we want to see on the tiny screen. F-Origin has developed IRIS navigational software that allows you to view the content of a large display on your tiny cell phone screen. All you have to do is move your small screen around as you would a hand mirror to change the view and see different parts of a vista behind you. Instead, as you vary the angle of your hand, you see different orientations of the larger display. Reputed to be intuitive and easy to use, the display offers a touch screen with haptic feedback, giving the screen itself the feel of press-and-click responses.
Also trembling on the edge of the market is the Wings haptic technology for cell phones, from the U.S. company called Atrua. Its fingerprint protocol offers incredible opportunities for mobile commerce: A thumbprint can identify you for secure monetary transactions.
Think how convenient it will be when this brave new world allows us to have these things wired right into our temples. And, scary as that might seem, Professor Clark assures us that it won't be so different from just holding them in our hands.
Developing Haptic Interfaces and Elements
You want something to offer texture to your GUI interface? For those with vision limitations, a haptic mouse that delineates the edges of windows and icons makes sense. A number of items in the PHANTOM line of products from SensAble Technologies support these interfaces. The PHANTOM technology involves force feedback devices that are programmed with GHOST software. The end result is haptic feedback in 3D that allows users to feel the geometry of virtual objects.
The ARTools technology that creates virtual characters projected into a real world environment is also available for those who want to use it for development. This is the technology used at the HIT Lab to build augmented reality projects.
With this system, the subject peers through a viewer to see a two-dimensional black tracking pattern on a card. Computer vision technology is used to identify that pattern, using it to place a three-dimensional virtual object, say a cartoon character or an image of the planet Jupiter that overlays a live video of the real world. The software is available in a free download for non-commercial use.
Conclusion
Don't take my word on Clark's views. If you want the real skinny on his cyborg angle, you'll need to read his book—but he makes a point about purchasing that I think sums up the worth of any disembodied assistant that scans the market on high-tech products.
Clark sees a future in which "software agents" will figure out what we as individuals look at most, what we choose, and what we buy. I asked him if he thought we might eventually have a haptic link to these agents.
He agreed it was possible, following up with this faintly unnerving summary that uses Rudy Rucker's term for the human nervous system part of a cyborg. "Any technology that operates robustly and continuously," he said, "can be factored in by the rest of the mind so as to become as much a part of us as non-consciously operating wetware."
No wonder his book suggests that we could become so used to these electronic shadows that if we were to lose them suddenly, it would be like having a stroke.
I'm not sure if he's tuned in the future perfectly here, but it does suggest that periodically we should all shut off the power and use our original equipment to perceive the outside world.