Search My Blog

Tuesday, June 18, 2013

WiSee - Wi-Fi signals enable gesture recognition throughout entire Home

Now, I think this Technology has a very big future. Imagine the ease of implementation and low costs too...


What is WiSee?

WiSee is a novel interaction interface that leverages ongoing wireless transmissions in the environment (e.g., WiFi) to enable whole-home sensing and recognition of human gestures. Since wireless signals do not require line-of-sight and can traverse through walls, WiSee can enable whole-home gesture recognition using few wireless sources (e.g., a Wi-Fi router and a few mobile devices in the living room).

WiSee is the first wireless system that can identify gestures in line-of-sight, non-line-of-sight, and through-the-wall scenarios. Unlike other gesture recognition systems like Kinect, Leap Motion or MYO, WiSee requires neither an infrastructure of cameras nor user instrumentation of devices. We implement a proof-of-concept prototype of WiSee and evaluate it in both an office environment and a two-bedroom apartment. Our results show that WiSee can identify and classify a set of nine gestures with an average accuracy of 94%.

Video Link...


Qifan Pu
Sidhant Gupta
Shyam Gollakota
Shwetak Patel



Whole-Home Gesture Recognition Using Wireless Signals [paper]
Qifan Pu, Sidhant Gupta, Shyam Gollakota, Shwetak Patel
To appear, at The 19th Annual International Conference on Mobile Computing and Networking (Mobicom'13)

From the WiSee PDF...

As computing moves increasingly away from the desktop,
there is a growing need for new ways to interact with com-
puter interfaces. The Xbox Kinect is an example of a com-
mercially available input sensor that enables gesture-bas
interaction using depth sensing and computer vision. The
commercial success of these kinds of devices has spurred
interest in developing new user interfaces that remove the
need for a traditional keyboard and mouse. Gestures enable a
whole new set of interaction techniques for always-availab
computing embedded in the environment. For example, us-
ing a swipe hand motion in-air, a user could control the mu-
sic volume while showering, or change the song playing on
a music system installed in the living room while cooking,
or turn up the thermostat while in bed. However, the bur-
den of installation and cost make most vision-based sensing
devices hard to deploy at scale, for example, throughout an
entire home or building. Given these limitations, research
have explored ways to move some of the sensing onto the
body and reduce the need for environmental sensors [7, 14,
12]. However, even on-body approaches are limited to what
people are willing to constantly carry or wear, and may be
infeasible in many scenarios (e.g., in a shower).
This paper presents WiSee, the first whole-home gesture
recognition system that requires neither user instrumenta
tion nor an infrastructure of cameras. WiSee achieves this b
leveraging the WiFi signals in an environment. Since these
signals do not require line-of-sight and can traverse throu
walls, very few signal sources need to be present in the space
(e.g., a WiFi AP and a few mobile devices in the living
room). WiSee works by looking at the minute Doppler shifts
and multi-path distortions that occur with these wireless s
nals from human motion in the environment.
To this end, we address the following two challenges:
(a) How do we capture information about gestures from
wireless signals?
WiSee leverages the property of Doppler
shift [11, 2], which is the frequency change of a wave as its
source moves relative to the observer. The canonical exampl
is the change in the pitch of a train’s whistle as it approache
and departs from a listener. In the context of wireless sig-
nals, if we consider the multi-path reflections from the hu-
man body as waves from a source, then a human performing
a gesture, results in a pattern of Doppler shifts at the wire-
less receiver. Thus, a user moving her hand away from the
receiver results in a negative Doppler shift, while moving t
hand towards the receiver results in a positive Doppler shif
The challenge, however, is that human hand gestures re-
sult in very small Doppler shifts that can be hard to detect
from typical wireless transmissions (e.g., WiFi). Specific
since wireless signals are electromagnetic waves that prop
agate at the speed of light (
m/sec), a human moving at a
speed of
m/sec, results in a maximum Doppler shift of
is the frequency of the wireless transmission [2].
Thus, a 0.5 m/sec gesture results in a 17 Hz Doppler shift
on a 5 GHz WiFi transmission. Typical wireless transmis-
sions have orders of magnitude higher bandwidth (20 MHz
for WiFi). Thus, for gesture recognition, we need to detect
Doppler shifts of a few Hertz from the 20 MHz WiFi signal.
At a high level, WiSee addresses this problem by trans-
forming the received signal into a narrowband pulse with a
bandwidth of a few Hertz. The WiSee receiver (which can
be implemented on a WiFi AP) then tracks the frequency
of this narrowband pulse to detect the small Doppler shifts
resulting from human gestures. In §3, we describe our algo-
rithm in more detail and show how to make it applicable to
existing 802.11 frames.
(b) How can we deal with other humans in the environment?
A typical home may have multiple people who can affect
the wireless signals at the same time. WiSee uses the MIMO
capability that is inherent to 802.11n, to focus on gestures
from a particular user. MIMO provides throughput gains by
enabling multiple transmitters to concurrently send packe
to a MIMO receiver. If we consider the wireless reflections
from each human as signals from a wireless transmitter, then
they can be separated using a MIMO receiver.
Traditional MIMO decoding, however, relies on estimat-
ing the channel between the transmitter and receiver anten-
nas. These channels are typically estimated by sending a dis
tinct known preamble from each transmitter. Such a known
signal structure is not available in our system since the hu-
man body reflects the same 802.11 transmitter’s signals.
Our solution to this problem is inspired by the trigger
approach taken by many multi-user games that use Xbox
Kinect, in which a user gains control of the interface by
performing a specific gesture pattern. In WiSee the target
human performs a repetitive gesture, which we use as that
person’s preamble. A WiSee receiver leverages this pream-
ble to estimate the MIMO channel that maximizes the en-
ergy of the reflections from the user. Once the receiver locks
on to this channel, the user performs normal (non-repetitiv
gestures that the receiver classifies using the Doppler shif
In §3.3, we explore this idea further and show how to ex-
tract the preamble without requiring the human to perform
gestures at a pre-determined speed.
The WiSee proof-of-concept is implemented in GNURa-
dio using the USRP-N210 hardware. We classify the ges-
tures from the Doppler shifts using a simple pattern match-
ing algorithm described in §3.2. We evaluated WiSee with
a total of five users in both an office environment and a
two-bedroom apartment whose layout is shown in Fig. 7.
We performed gestures in a number of scenarios including
line-of-sight, non-line-of-sight, and through-the-wall
ios where the person is in a different room from the wireless
transmitter and the receiver. The users perform a total of 90
gestures across the locations.
Our findings are as follows:

WiSee can classify the nine whole-body gestures shown in
Fig. 1, with an average accuracy of 94%. This is promis-
ing, given that the accuracy for random guesses is 11.1%.

Using a 4-antenna receiver and a single-antenna trans-
mitter placed in the living room, WiSee can achieve the
above classification accuracy in 60% of the home loca-
tions. Adding an additional singe-antenna transmitter to
the living room achieves the above accuracy in locations
across all the rooms (with the doors closed). Thus, with
an WiFi AP acting as a receiver and a couple of mobile
devices acting as transmitters, WiSee can enable whole-
home gesture recognition.

Over a 24-hour period, WiSee’s average false positive
rate—events that detect a gesture in the absence of the tar-
get human—is 2.63 events per hour when using a pream-
ble with two gesture repetitions. This goes down to 0.07
events per hour, when the number of repetitions is in-
creased to four.

Using a 5-antenna receiver and a single-antenna transmit-
ter, WiSee can successfully perform gesture classification
in the presence of three other users performing random
gestures. However, the classification accuracy reduces as
we further increase the number of interfering users. This
is a limitation of WiSee: Given a fixed number of trans-
mitters and receiver antennas, the accuracy reduces with humans. WiSee builds on this work but significantly differs
from it in that it extracts Doppler shifts from wireless sign
to perform human gesture recognition. Further, we demon-
strate that one can perform whole-home gesture recognition
without requiring wireless devices in every room.
Finally, WiSee is related to work on through-the-wall
radar systems [3, 18, 2] that can identify objects such as
metal pins behind a wall. These systems use expensive ultra-
wideband transceivers that use bandwidths on the order of
1 GHz [3]. In contrast, WiSee focuses on gesture recogni-
tion and shows how to extract gesture information from WiFi
transmissions. The closest to our work in this domain is re-
cent work [4] that demonstrates the feasibility of using WiF
signals to detect running in through-the-wall scenarios. I
contrast, WiSee introduces mechanisms that enable it to go
beyond coarse human motion such as running and delivers
the first wireless system that can identify finer-grained hu-
man motion such as gestures in LOS, NLOS, and through-
the-wall scenarios.
(b) In-Air Gesture Recognition Systems:
The commercial
success of products like the Xbox Kinect has popularized
the use of gestures to control computer systems [21]. In-
creasingly, in-air gesture recognition is being incorpora
into consumer electronics and mobile devices, including la
tops [1], smartphones [8, 11], and GPS devices [19]. The
related work in this domain use four main techniques: com-
puter vision, ultra-sonic, electric field, and inertial sen
Vision-based systems extract gesture information using
advances in the hybrid camera technology like pixel-mixed
devices (PMDs) [21]. Likewise, ultra-sonic systems lever-
age Doppler shifts on sound waves to perform gesture recog-
nition [11]. Both these systems, however, require a line-of
sight channel between the sensing device and the human. In
contrast, WiSee leverages wireless signals that can operat
in non-line-of-sight scenarios and can go through wooden
walls and obstacles like curtains and furniture.
Electric Field sensing systems like Magic Carpet [17] in-
strument the floor with multiple sensors to perform human
localization and gesture recognition. However, this impos
heavy instrumentation of the environment and is not practi-
cal. Inertial sensing and other on-body sensing methods on
the other hand, require the users to wear multiple sensors or
carry a device such as a wrist band [7, 14, 12]. While at-
tractive, in many instances, such an approach can be incon-
venient (for instance, while showering). In contrast, WiSe
enables whole-home gesture recognition without the need to
instrument the human body.
Finally, prior work has leveraged Doppler shifts to per-
form gesture recognition in line-of-sight scenarios [11, 1
In this paper, we present algorithms that allow us to achieve
gesture recognition in line-of-sight, non-line-of-sight
through-the-wall scenarios; thus enabling gesture recogn
tion without the need for sensing devices in every room.


Read More

WiSee is a novel interaction interface that leverages ongoing wireless transmissions in the environment (e.g., WiFi) to enable whole-home sensing and recognition of human gestures

WiSee: Wi-Fi signals enable gesture recognition throughout entire home - YouTube
Watch: In-home gesture recognition via Wi-Fi signals - Cabling Install
multiuser_d.eps - wisee_paper.pdf

No comments: