Per Ola Kristensson | ShapeWriter and Gesture Keyboards

The Story of ShapeWriter and Gesture Keyboards

Note: Now often known as T9 Trace, Flext9 or Swype. In the research literature it is sometimes still referred to as "SHARK" (the name of the first system we published).

One of my research contributions that I am the most proud of so far is the work on gesture-based text entry for touch-screen keyboards (together with Dr Shumin Zhai). This technology enables users to write quickly on their mobile phones by sliding or swiping the finger over a touch-sensitive on-screen keyboard. For instance, to write the word "the" the user pushes down the finger on the T key, slides to the H and E keys, and then lifts up the finger on the E key. This input paradigm recognizes the shape of this gesture using a pattern recognizer. During practice, the shapes of words builds up in users' muscle memory which enables users to quickly recall the shapes for words without looking much at the keyboard (similar to how you remember your ATM code).

Me (left) and Shumin Zhai
naming the first system
"SHARK" in early 2002. The first publication on ShapeWriter and similar gesture keyboard technology is my master's from August 2002. Thereafter I co-authored a number of academic papers on it that were published in some of the best venues in human-computer interaction (the Association for Computing Machinery's CHI and UIST conference proceedings). The first press coverage was in San Jose Mercury News in April 2003. The first public release was available on IBM alphaWorks in 2004 (and covered by numerous press articles). I wrote all the code for that release.

Me (seated in the middle)
in our Beijing office in 2007. The idea was later commercialised by ShapeWriter, Inc, a company I co-founded in 2007. I was the Director of Engineering of this company (2007-2010) and I worked fulltime in Beijing for six months in 2007-2008 to set up and manage our engineering office. I am also the sole author of the pattern recognition algorithm that was behind all the company's products. We released ShapeWriter on multiple platforms, such as Google Android and the iPhone. We won a Google Android Developer Challenge ADC50 award in early 2008. Shortly after that we released ShapeWriter on the Apple AppStore. The iPhone version was selected by Time Magazine in 2008 as a top-11 iPhone application. Between 2003-2009 there have been more than 100 press articles on ShapeWriter. In May 2010, Nuance Communications, Inc. acquired ShapeWriter, Inc. Hence we completed a cycle of research-commercialisation-acquisition in about eight years.

• A list of international press articles referencing me or my work (2003-2010). Many of these articles are about ShapeWriter and gesture keyboards (referred to as "SHARK" or "IBM Shorthand" before 2007).
• In 2008, Time Magazine selected ShapeWriter as a Top iPhone Application and ranked it the 8^th best iPhone application in the world (in the article ShapeWriter is referred to as "WritingPad").
• In 2008, ShapeWriter won a Google Android Developer Challenge ADC50 Award.
• At CHI 2005, a premier publication venue in Human-Computer Interaction, I won the Best Doctoral Consortium Contribution Award.

ShapeWriter software

If you are looking for ShapeWriter software for your mobile phone, etc. please take a look at the ShapeWriter, Inc. website. (Since our acquisition this link does not work. If you want to see how our company website looked like, click here).

Key Publications

• The most comprehensive reference work on gesture keyboard technology available:
  • Kristensson, P.O. 2007. Discrete and Continuous Shape Writing for Text Entry and Control. Doctoral dissertation, Linköping University, Sweden. ISBN 978-91-85831-77-7. (Monograph; 200 pages). (pdf) (press coverage).
• This paper describes the command input feature in gesture keyboards:
  • Kristensson, P.O. and Zhai, S. 2007. Command strokes with and without preview: using pen gestures on keyboard for command selection. In Proceedings of the 25^th ACM Conference on Human Factors in Computing Systems (CHI 2007). ACM Press: 1137-1146. Acceptance rate: 24%. (pdf) (publisher) (youtube)
• This paper describes a complete system for gesture keyboard technology:
  • Kristensson, P.O. and Zhai, S. 2004. SHARK²: a large vocabulary shorthand writing system for pen-based computers. In Proceedings of the 17^th Annual ACM Symposium on User Interface Software and Technology (UIST 2004). ACM Press: 43-52. Acceptance rate: 20%. (pdf) (publisher)
• This paper describes the first iteration of gesture keyboard technology:
  • Zhai, S. and Kristensson, P.O. 2003. Shorthand writing on stylus keyboard. In Proceedings of the 21^st ACM Conference on Human Factors in Computing Systems (CHI 2003). ACM Press: 97-104. Acceptance rate: 16%. (pdf) (publisher).
• This is the first publication to introduce gesture keyboard technology:
  • Kristensson, P.O. 2002. Design and Evaluation of a Shorthand-Aided Soft Keyboard. Final/master's thesis (D-Uppsats), Linköping University, Sweden. ISRN LiU-KOGVET-D--02/07--SE. (Monograph; 96 pages). Available upon request.