This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at firstname.lastname@example.org.
History of Pen and Gesture Computing:
Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers
References from the approximate years 2008 to 2010.
Permission is granted to use this information in publication, including confidential reports, provided that accompanying text clearly makes reference to the URL for this page, along with the statement:
Source: Annotated Bibliography in On-line Character Recognition, Pen Computing, Gesture User Interfaces and Tablet and Touch Computers Copyright Jean Renard Ward
Most of these are in my personal collection, either as physical items or electronic files.
Copyright © Thu Oct 20 15:47:21 EDT 2016.
Mirrors of this page can be found at:
biblio07 (Prev page)
(Next page) biblio13
History of talking computers, DOS and Windows Screen readers, and other assistive technologies: Enhanced PC Talking Program by Ronald Hutchinson worked with word processors and spreadsheets 1984, outSPOKEN for Macintosh 1989, ASAP Automatic Screen Access Program "with artificial intelligence" 1990. Virtual keyboards? On-screen keyboards.
Windows Mobile smartphone with touchscreen, slide-out full keyboard on side, tilts to somewhat laptop-type configuration. Same as HTC TyTN II / HTC P4550 / HTC 8925 and XDA Stellar O2 (see). Sold under multiple other vendor and product names. Maps navigated with scroll and navigator keys, not touchscreen. Transcriber: handwriting recognition -- punctuation via keyboard, pop-up virtual keyboard.
Dual-touchscreen laptop with virtual keyboard or other elements in one side, regular display in other. Compare with Refalo, Microsoft Courier. Dual-touchscreen. Additional references in file.
Paper forms for Anoto-type optical digitizer pen: mentions 1200 dpi resolution of color printers certified by Anoto. Capturx capture software to upload electronic ink data directly into Microsoft OneNote, ArcGIS 9.2 Desktop geographical mapping software, and AutoCad. "Rite in the Rain" all-weather waterproof writing paper.
Seems to be about mouse-over/hover highlight or magnifying of targets/icons on a proximity-sensing touchscreen with a stylus. Compare with PenPoint, ZoomLens?
Web site with archived files for DOS and other screen readers and accessibility software: include Hal Lite, ScreenPower, ASAP, Tinytalk, Proxy 7, Vocal Eyes. Also Speech synthesizers such as Dectalk. Contact addresses (?) email@example.com, or David Holladay firstname.lastname@example.org
Physical Book: Development guide for iPhone (iPod, iPad) devices: Compass, Proximity, Accelerometer sensors, Objective-C, Tablet-based view controller objects, etc.
Haptic direction indicator transducer gives illusion of a force/pull in one direction. Two-dimensional tactile force display. Oscillates/vibrates with asymmetric acceleration: speed fast in one direction giving sensation of pushing/pulling, slowly in opposite direction below sensing threshold. Appliaction as direction indicator on a guide cane for the blind/visually impaired. Prototype uses OQO touchscreen hand-held computer.
Haptic direction indicator transducer gives illusion of a force/pull in one direction, used as guide display for the blind instead of audible cues (which could be problematic in noise environments, etc.). Mechanical transducer (slider/crank). Later papers have application to haptic output in hand-held smartphones.
Developer-written manual for Nintendo DS game computer, two touchscreens in Reefalo-style clamshell. Appears to read touchscreen input by polling (touchRead API function).
Capacitive-to-digital A/D for single-electrode capacitive sensors (capacitance to ground). Intended for touch buttons, sliders, wheels, and matrix keypads (compare touchscreens). Matrix keypad, each button has two electrodes (connected to row and to column lines), single key detected by row + column ID. Eight sensor inputs, so up to 4x4 keypad.
Accessibility technology: Haptic display device for Nokia haptic touch-screen with vibration (similar to iPod/iPhone) devices: six pulse/no-pulse signals via vibration corresponding to the six dots of a Braille character.
virtual/soft keyboard on touchscreen: find centroid of finger touch, and map it to (weighted distance) nearest key. Compare with stroke targeting/ink targeting in PenPoint, Tappert, and Slate PenApps?
Anoto digitizer pen with no inboard storage, all data sent in real time to hose computer via Bluetooth. Notes use of unique pattern on each printed page, allowing page to be identified, not just location.
iPhone/iPad gestures: flicking and dragging gestures does not select (press-and-hold or press gesture). Tapping halts dynamic zoom. Pinch and zoom gestures. Press to get magnified view of text while editing. Note: does not cover accessibility gestures, such as triple-flick.
MacBook multi-touch trackpad gestures on touchscreen/touchpad: Tap to click, two finger scroll (flick)/rotate/pinch/zoom, three finger swipe, four finger swipe gestures.
User Guide for iPhone: zoom/pinch, scroll, flick gestures on touchscreen, on-screen (virtual) keyboard: Japanese/Chinese keyboards show alternative characters to pick from: Pinyin and Zhuyin. Writing area for handwriting recognition of Simplified or Traditional Chinese handwriting. VoiceOver accessibility.
Apple v. Palm, and other vendors: Apple patent with Westerman (of Fingerworks) on multiple iPhone features, not just multitouch touchscreen
Apple v. Palm lawsuit: Palm Pre uses same multi-touch gestures as Apple iPad, article reviews multiple patents in litigation: telephone/address book, screen dimming, etc.
Review of VoiceOver screen reader accessibility on iPad, particular note that it is a built-in feature, unlike more expensive screen readers
"Works with iPhone" Logo-approved stylus/pens for iPhone and iPad: Pogo Stylus, Soft-Touch stylus from DayDeal. Include finger-tip stylus for Apple touchscreens. Rounded tips, plastic.
iWorks spreadsheet for iPad: refers to swipe and tap gestures
(in German) Review of VoiceOver screen reader software for MacIntosh with multi-touch trackpad: mentions "rotor" circle/rotational gesture to bring up navigation tools and command input (compare Zoom /spiral on Nokia). Supports up to 32-line braille (braillezeiler) output, 1 on Bluetooth, 31 on USB.
VoiceOver accessibility screen-reader software. Mentions voice prompting on what gesture is appropriate: Rotor gesture with two fingers to change scope of other features: flick to move by pages or lines, etc. Double-tap with two or three fingers.
VoiceOver accessibility screen-reader software. Drag of finger (or two) over item speaks the item. Gestures may be assigned new semantics: preferences may be stored removed on a server.
Web posting on Apple/HTC/Google patent litigation, and blog posting on bounce-back on scroll being a feature of MacIntosh screen enlargers (accessibility) about 1990.
Matlab/Octave tutorial: Octave is Matlab clone, interactive mathematical graphing and data calculation utility.
Microsoft TouchWall, three-laser sensor for multi touch on any flat surface, project display. Described as distinct from Microsoft Surface, which uses cameras to recognize objects placed on it (e.g. a cell phone, Surface reads photos from cell phone).
See also CrunchPad about Arrington
3D/6DOF DC magnetic position sensor, compare with Polhemus. Has Wide-Range transmitter (base station) option, senses up to 137 cm from transmitter sensor for high accuracy.
3D/6DOF DC magnetic position sensor, compare with Polhemus.
Trainable gesture recognition system by example. Includes some automation for false positive and recognition error rates. Input device is three-axis accelerometer, like a Wii.
Three-finger translate/scale/pinch/zoom/rotate/move gesture instead of two-finger pinch/zoom: more stable hand posture, improves precision and control. Touchscreen is touch glove with LED light sources on fingertips, detected optically when touch surface with diffuser glass.
User manual for Asus UMPC. Touch Pack software for Windows Vista. Press and hold gesture for right-click. Tablet PC Input Panel (SIP) launched from dragable icon. Multiple scratch-out/erase gestures: vertical, horizontal, circular; enter/backspace/space/tab editing gestures. Different space gesture in Simplified Chinese. Control panel "gestures commonly used on handheld computers". No wifi/wlan?
Tablet PC / laptop hybrid: tablet PC slate computer fits into top "cover" shell of keyboard. No wired connection to keyboard.
Negative historical review of tablet computer / pen computing, written slightly before Apple iPad, so does not mention iPad/Android tablet success. Characterizes PenPoint as "vaporware", mentions Pen Windows, but no citations to Marlin Eller history (?). Analysis from social construct theory: e.g. "truth" as socially constructed interpretation, technological success as within a socially constructed system. Based on PhD Publication. Mentions Nestor for handwriting recognition (which had no products), but not Pencept or CIC (which did have products).
Summary of publications submitted for PhD by Publication: Computer Memories: The History of Computer Form; The (In)Difference Engine: Explaining the Disappearance of Diversity in the Design of the Personal Computer; The Computer Mouse in the History of Computing: A Bitter Pill to Swallow: The Failure of the Tablet Computer
History of computers (personal computers /PCs) from social construction perspective: effect of social drivers (prestige, status, popularization in advertising and movies) on computer design evolution.
Amtel: formerly Quantum Research Group.
QTouch touchscreen. Capacitive sensing, statement about detecting two touches, refers to multi-touch. Single layer of ITO (front surface?). Discusses cross-talk from adjacent sensors, use strongest signal.
Additional material in files: Quantum acquired by Amtel: see Quantum file.
Amtel: formerly Quantum Research Group.
QTouch touchscreen. Extensive discussion of functioning of capacitive touch sensors.
Amtel: formerly Quantum Research Group.
QTouch touchscreen software library, includes hooks for software filtering of inputs (not standardized?)
Amtel: formerly Quantum Research Group.
Atmel QTwo 10-bit Touchscreen Controller AT42QT5480 Summary QT5480 AS: sliders and keys (buttons) can be defined within controller, along with "touchscreen" section, output delivered as separate input. "Channels": fixed matrix/grid of touch areas?
Amtel: formerly Quantum Research Group.
QTouch touchscreen. Capacitive sensing, statement about detecting two touches, refers to multi-touch.
QMatrix touch sensor IC devices: scan touchscreen keys consisting of passive matrix of copper or transparent ITO indium in oxide, differential detection to reject capacitive pickup on adjacent keys. States spread spectrum signals to reduce interference. Shows electrode configurations including serpentine and spiral.
QMatrix touch sensor IC devices: scan touchscreen keys consisting of passive matrix of copper or transparent ITO indium in oxide, differential detection to reject capacitive pickup on adjacent keys. States spread spectrum signals to reduce interference.
Overview of projected and mutual capacitance touchscreens. Cites to noise problems from LCD, and from floating power supplies (through ground path into touchscreen), as system issue. Calibration discussion focuses on drift in sensitivity to touch, not position.
Gesture on an inactive area (not an icon, link, scrollbar, etc.) is processed as a command if the handwritten character matches character. Compare with PenPoint.
AliasStudio 3D modeling and sketching application. Tear-off menus, marking menus. Marking menus brought up as context-sensitive menus by mouse right-button, or user-definable/customizable marking menus.
Sketching design and physical modeling package (Autodesk Alias). Uses radial marking menus (pie menus), combined with checkbox/menu below. Left mouse button "optimized" for stylus use on tablet. Marking menus customizable, four or eight radial directions with vertical list below.
Multi-pod (hexapod/pentapod) linkage mechanism for force feedback using magneto-restrictive fluid brakes. Has squeezable (pressure, but not tip-force) stylus for input.
Report on multi-touch-enabled kitchen in experimental setting: multi-touch touchscreen / surface using under-the-table optical tracking and internal reflection from finger contact. Cites to NUI Group Community for construction of multi-touch surface.
Overview of capacitive touch tablet technology. "grip suppression" for handheld smartphones, similar to palm rejection. Describes true multi-touch as possible in multi-pad self-capacitance sensor and in intersection grid mutual capacitances.
Brief overview of several multi-touch technologies: describes projected capacitive as multi-touch zones (compare with Kaplow?), multi-touch input seen by applications (in most OSs) as multiple mouse/locator devices. States only three gestures unique to multi-touch (???): pinch, expand, rotate. Asserts that first iPad / iPhone touchscreens were resistive film with "ghosting" for multi-touch.
Appears to be alternative/revised/version of 2010 SID article by same authors on projective/capacitive touchscreen. One difference is refering to glass vs. plastic substrates, plastic flexible and can be curved. (Microcracks?) Discussion of linear, diamond-chain, and other patterns for projected/mutual capacitance grids. Notes that pinch and zoom gestures do not require disambiguation of ghost points.
Spelling predictor in touchscreen user interface (or icon predictor), highlight "most likely" completions. Shows circular alphabetic arrangement of letters. Compare with IBM Simon, Quikwriting.
Comparison vibrotactile (moving parts) haptic feedback/display with electrovibration (modulated electrostatic sheet attracts/repels fingertip, changing friction). Single sheet of transparent conductor with AC signal applied. Vibration frequencies for sticky/waxy/bumpy/smooth sensations. Sensation is only for moving fingertips in contact: used to simulated multi-touch feedback approximately.
Web posting and video (see Video file of LucidTouch) of pseudo-translucency touchscreen user interface: rear-mounted second touchscreen on handheld PDA-type device, shows simulated outline of fingers on front touchscreen display. Multi-finger/multi-touch input, rear touching does not obscure/block view of display.
Small touchpad on back of display of cellphone: no fat finger problem, no blocked view (by finger) because can show precise point of touch. Prototype uses (non-transparent) capacitive trackpad. Also two thumb buttons on edges of device. No multitouch by intent. Compare with Bennett 1995 back of device.
Graphical user interface allowing users to combine information from multiple internet and other sources into single web page and share information with others. Data from multiple sources can be combined based on keywords, field names, or other attributes present in the data. Patent assigned to Virtual Agility.
Simulated usability test environment for accessibility of mobile computing mobility aids (e.g. sonar mounted on cane or wrist): multi-channel (more than stereo or 5.1) sound simulation
textbook on interactive user interface design, especially for group/CSCW and social interaction. Multi-touch and haptics consideration. iTunes as important to iPhone success as multi-touch/gesture user interface. HP iPAQ Pocket computer. Samsun i7410 Sirius projector phone.
Distinguish text and sketch/shape input by zero-order entropy context-free, separate from character and sketch recognition per se. 92% success rate claimed. Tablet PC input, not optical. Initial classification for entropy is angle with temporally adjoining points (e.g. chain codes).
Barrel-rolling gestures (rotational angle of stylus/pen barrel around own axis). Barrel rotation determined from optical flags on sides of top stylus end. Distinguish incidental barrel rolling from intentional. Unlike stylus force/pressure (which cannot be negative), rolling can be either positive or negative. Barrel roll to select mode (sketch, erase, etc.) similar to pie menus.
Microsoft Project collaborative work system: includes CPM critical path method, Gantt charts, resource and schedule allocation views, custom fields. Agile project plans. Customizing ribbon UI.
Overview of tactile sensors: touchpads, touchscreen, artificial robotic skin. Describe piezoresistive pressure (force) sensors using variable resistors in Wheatstone bridge. Interlink force sensing resistors FSR, Tactilus piezoresistive sensors, SoftPot linear touchstrip sensor. Capacitive touchscreens, optical touchscreens, Tactex taxels (large array touch sensor available as cloth)
Gesture defined as hand gesture ("a motion of the body that contain contains information", not as handwriting/shape gesture for pen-computing (cites to Kurtenbach90 for definition). Mentions Theremin as gesture-input device (non-computer). Mentions "put that there" (Bolt 84). Draft of May 27, 2009: File also contains 2002 Draft
Dual-screen digitizer tablet by Microsoft: 3 Megapixel camera, compared with One Laptop Per Child XO-2 and MoleSkin. additional references on Courier in file. Compare with Refalo? File has additional materials.
Review of Agile versus Formal development methods: refers to writing tests before implementation in Agile development. Numerous references, argues that formal proof of correctness and automatic test generation more valuable.
Speculative essay on utility and environmental impact of switching from physical paper to electronic displays: special issue of CACM on tangible user interface interaction technologies. Multi-touch 3D surface interface (e.g. globe/map).
Comprehensive graphics and image analysis processing library, image segmentation. Includes functions for alpha blending, affine operations (rotate, translate, etc.)
On a multi-touch tablet/tabletop with multiple users (but not a tablet like DiamondTouch with extra electronics to identify each user), assign touches far apart (in space, mostly) to separate users, assign touches close to another touch (in space and time) to the same user as the first touch. Hardware can stream the assigned touches to different input streams, to maintain some compatibility with existing single-stream input applications.
Accessible on-screen keyboard on touchscreen for blind/visually impaired: divide screen into large regions for groups of characters (ABC, DEF), tap on region gives sub-menu of individual characters. Cites to VoiceOver and Quikwriting, includes review of accessibility techniques for touchscreens including multi-touch.
Electrostatic digitizer proximity sensing in a picture frame: http://Synaptics.com
HP TouchSmart display, with integrated touchscreen. Comparison made to One Laptop Per Child PC design with no keyboard, instead second screen which can display a keyboard: compare with Refalo, Microsoft Courier.
Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Microsoft Tablet PC Music Composition tool in 1998(??).
Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Microsoft Tablet PC Music Composition tool in 1998(??).
Teach pen-gestures and multi-touch gestures by displaying gestures in a toolbar at the top of the application. Tapping on toolbar tab brings up practice area and explanation of the gesture. Cites to Kurtenbach Learning-while-Doing, Sketch-based user interfaces / Sketch recognition, Lineogrammer, and a simple crib sheet with tool-tips. Gestures must be discoverable/searchable.
bi-manual / multi-touch input better with pen and touch, than pen and pen our touch and touch. Use pen/stylus in dominant hand, menu selection in non-dominant hand. E.g. inking and precise input better with pen, more precise and pen obscures less of display. Draggable menus, hand postures for command/drawing mode. Uses Maxell version of Anoto pen -- multiple pens at one time, unique ID in each pen -- on DiamondTouch tablet (for touch). So overlay tablet with printed pattern.
Collection of Essays and historical notes by Dan Bricklin (Slate Corp. / Lotus 123) inventor of spreadsheet software. Chap 8: Hans On: Tablet and Gestural Computing includes essay from http://bricklin.com "Tablet Computing Old and New" (earlier "Pen Computing Old and New"). Some essays date to 2000.
Final web page for Microsoft Center for Research on Pen-Centric Computing at Brown University: three-year funding for research on applications for Microsoft Tablet PC operating system and tablet hardware: ChemPad, MathPad (mathematical notation) Music Notepad, Math Paper, Lineogrammer. Some work listed (SKETCH) pre-dates Tablet PC, dates to 1999.
Integrated in-cell LCD display and touchscreen, senses in proximity. Optical sensors in pixels detect infrared light reflected from fingertips etc.
In-cell LCD touchscreen, capacitive sensing. Stylus or finger depresses conductive surface, changes capacitance to underlying (integrated into LCD) sensing electrode by changing distance to sensor. Second style, two-electrode sensor, approaching conductor changes coupling between electrodes.
Jeff Han, Perceptive Pixel. Frustrated total internal reflection FTIR. Optical panel is both sensing surface and diffusion screen for rear projection. Also mentions Microsoft Surface, Mitsubishi Diamond Touch, and iPhone.
Using touch sensors sensing to control a flashlight/light by adding a MMI (man-machine interface: user interface). Touch sensed by microchip that adds intelligent functions to otherwise dumb device: emergency flashing, time schedule, etc. Different function depending on number of button actuations, activate function on timeout (e.g. 5 seconds). See also Bruwer 2001, 2010.
Using touch sensors with proximity sensing to control a flashlight/light by adding a MMI (man-machine interface: user interface). Touch/proximity sensed by microchip that adds intelligent functions to otherwise dumb device: emergency flashing, time schedule, etc. Different actions based on proximity vs. touch, level of proximity e.g. continue operation if user in proximity, activate function (turn on radio display) when hand is near. See also Bruwer 7,498,749, 7,778,980, 7,994,726. Compare with PenPoint proximity?
Holographic 3-D three dimensional projection on windshield of automobile. see Light Blue Optics file.
pre-distorted projected image shown on curved surface to appear undistorted
Holographic 3-D three dimensional projection on windshield of automobile
Holographic (portable) projector projects display and simulated/virtual keyboard onto table-top, senses touchscreen surface by fan of low-power infrared laster light superimposed on projected image and infrared sensors that detect scattering.
University textbook on electrical and electronic engineering and design for mechanical engineers and manufacturing engineers (in German). Has more material on power management, analog control circuits.
Book chapter in progress. Gesture recognition used broadly for natural gestures, gestures with the hands in free space.
Historical interview with Bill Buxton concerning user interfaces, multi-touch, portable computing. Oscar winner (2003). multi-touch user interface inverted in 1984. Casio wrist-watch with touch-screen in 1984: unistroke single-digits and arithmetic symbols for calculator funcs. IBM Simon in 1993, precursor to iPhone. Non-contact (touch sensors) interfaces in 1982. Apple iPhone developers knew of IBM Simon smart touchscreen phone in 1993, described by Bill Buxton as precursor to iPhone.
Online at http://www.theglobeandmail.com/news/technology/personal-tech/controller-freak/a-chat-with-microsoft-principal-researcher-bill-buxton-part-i/article1679332/
Touchscreens, like all new technologies go far back: Plato in 1960s, soft-machines, Casio PF-8000 Data bank, IBM Simon, Multidimensional (X/Y/Z force, X/Y/Z torque) touchscreen. Contact area (touchscreen) is an approximation of pressure/force.
Marking interfaces: Buxton's term for for paper-like and pen-based interfaces, gestural / gesture-based user interfaces; line recognition as synonym for sketch recognition
Describes character recognition as a "black hole" for pen-computing interfaces
Early 1969 animation system Genesys for gesture input
Review of numerous paper-like application products from 1980's and 1990's: Newton MessagePad 2000, Liveworks Liveboard, Wang Freestyle, CompuThink Paperless Office, Winfax PRO, CrossPad, etc. Alto User's Handbook (1976)
InkWriter by Aha!: recognition of handwritten paragraphs, not of characters
Quikwriting unistroke / single-stroke characters
Papyrus Allegro unistroke / single-stroke characters
Casio AT-550 character recognition watch with finger tracing 1980, Casio DBC-150-1 Databank watch, PF-8000 calculator
First shorthand / unistroke: 63 BC, Marcus Tullius, "notae Tironianae"
File also contains 1994 draft.
Transparent/overlay menus and widgets for user interface on a touchscreen: compare with earlier click-through work. Cites to ToolGlass, transparent sheets, also to "Chameleon" system detecting position of handheld touchscreen for viewing virtual scene (compare with Tagliozzi video?)
Historical article on capacitive touchscreens invented at CERN in the 1970's as simplified/better-organized user interface UI for complex computer system for SPS Super Proton Synchrotron. Capacitor sensors etched into a film of copper on a sheet of glass, each capacitor constructed so that a nearby flat conductor such as surface of a finger increases capacitance by a significant amount. Traces 80 micrometers wide, 80 micrometer spacing, insulating lacquer coating on top. First prototype produced in shop at CERN on flexible transparent Mylar sheet. Gives details of construction using ion sputtering, traces were solderable. Phase-locked oscillator, change in capacitance changed phase not frequency, drift compensation by comparison to reference capacitor in same construction.
Training manual for Tablet PCs, (physical) power button action can be changed
CMMI / Capability Maturity Model Integration: best practices for software engineering development
List of research reports and publications from MIT CSAIL group, mostly relating to sketch recognition
Transparent conductive films for touchscreens, films of conductive fibers. Cites to ITO for transparency and very low resistivity, but is brittle especially on flexible substrates. Compare to Wang/Microtouch.
IBM Simon 1994, Osborne I luggable computer 1981, Grid Compass laptop, PARCTAB hand-held ubiquitous computing unit 1989/1992, Palm PDA with Unistroke Graffiti
User interface for large touchscreen surfaces (e.g. Microsoft Surface, Rekimoto surface): use area of contact (multi-touch number of fingers, hand area, etc.) to simulate larger or smaller forces on virtual objects. Also simulate pressing, colliding (pushing on virtual edge), friction (require more area contact to slide), flicking and rotate/scale around an anchor point.
Tutorial article on use of touchscreens, including colors and contrast of image for older users, predictable (to user) organization of user interface. Citations that minimum touch area or key target size is 22..22mm across. For one-handed thumb use, 9.2mm plus minimum 1mm space around. Cites to ISO for icon design.
Study of internal Palm/US Robotics/3COM debate on whether to license PalmOS to other hardware vendors broadly (as Microsoft) instead of proprietary OS (as Apple). Cites Psion Organiser 1984 as first commercial handheld computing device. GO/Penpoint, mis-states Microsoft pen-computers as using DOS (not Pen Windows); Momenta, Grid. Describes General Magic Magic Cap as joint project of Apple (?), Sony, Monotorola, others prior to Apple Newton. Palm pursued small third-party developers, large demand.
Casio wristwatch with unistroke character input on touchscreen, circa 1984
Addition of pressure sensors, Contrast of WIMP user interfaces with other numerous non-shape-recognition-based gestures: shift-click, double-click, timed gestures hover-help and spring-loaded folders. Effort to define general basis for haptic feedback on GUI widgets, rather than ad-hoc development.
Microsoft funding of graphics research group under Andries van Dam at Brown University. Projects as of 2008 include MathPaper two-dimensional mathematics input and gesture/direct manipulation, Lineogrammer sketch recognition / diagram editing, *Pad SDK ink middleware experimental pen-computing UI kit, ChemPad for teaching chemistry and molecular models, MathPad, Music Notepad with two-dimensional gesture input, Fluid Inking for gesture editing of electronic ink (compare with Van Raamsdonk?), ReMarkable Texts digital note-taking. Devices include pen computers (Tablet PCs), PDAs, electronic whiteboards.
Dual-display computer (compare with Refalo, OLPC One Laptop Per Child) with display on one half, display/touchscreen on other half. Also reads gestures by motion using video camera, optical.
Flexible multitouch sensor touchscreen, resistive grid in two layers: avoids shadowing problem by switching individual rows/columns to high/low-impedance, and solving Kirchhof's Laws equations for measurements on all rows and columns. Small spacer dots (compare Elographics?) prevent two layers from touching during bend. Flexibility allows slight deformations, not extreme bending.
Sliding/swipe slide-to-unlock gesture on Apple iPad and iPhone. Also US 8,046,721. Compare with direct manipulation, virtual devices in LabView?
Structure-through-motion modelling of human shape, 3D model using single camera. Customer puts on patterned garment (with targets, may be irregular), multiple views from webcam, system determines 3D shape by analysis of the images. Compare with D. Holz, 2013, etc.
pen-computing user interface, annotations can be added to documents. Annotations can be recognized as gestures: recognizing a date annotation bring up a calendar application. Annotations anchored to context/text of document. Compare with Agulnick and Slate annotations and electronic ink indexing, deferred recognition.
Timeline of mobile augmented reality and non-mobile augmented reality systems, defined as overlaying computer data on real world. Sutherland head-mounted display with 6-D sensors 1968. Cites Grid Compass 1000 as first clamshell laptop: compare with Data General DG/1? IBM Simon Personal Communicator 1992. Loomis GPS system for visually impaired 1993.
Read physical physical switches on side of mouse to get a resolution/delta value to set sensitivity of the mouse. Compare with multi-button stylus?
Chinese/Japanese Kanji input on cellphone keyboard: phonetic input, displays possible Kanji characters, along with additional meanings information of the characters (for Japanese).
multi-touch multi-finger gestures on Cirque GlidePoint sensor. Additional materials in file: claim Cirque originated capacitive touchscreens in 1991
Organic user interfaces / tangible user interfaces: E-Ink / electronic ink electrophoretic display technology, electroluminescent displays on clothing fabric.
Review of of user interface systems that show more and less detail, both graphical zoom (original meaning of focus+context, and in Mac XDock) and semantic zoom. Classifies as zooming; overview plus detail; focus plus context; cue-based to highlight or suppress items selectively. Fisheye zoom (enlargement in graphical context). Mentions pressure/force of touchscreen for zooming (Zliding, etc.).
Later version of 2006 technical report, U. Maryland, same authors
Wizard-of-oz (two people) simulation of system reacting to group whiteboard utilizing both voice and real-time electronic ink/pen input streams. Compare with Freestyle at Wang.
Haptic feedback (vibration) on touch screen so that user can tell when a touch was "missed", especially in rapid input. Compare with debouncing?
Development guide for IOS for iPhone (iPad, iPod) touchscreen devices: Simulator, Objective-C, etc.
Patent application publication about Apple filing for 'swipe-gesture': article refers to left, right, up, down gestures across the (simulated) keys of a keypad -- compare with 1980's Japanese calculator with handwriting input on physical keypad?
Conductive polymer for capacitive touch screens: higher resistance than ITO (desirable for for capacitive-sheet touchscreens), and not brittle, so may be used on flexible touchscreens.
Web-only pen-computing web-tablet with no disk storage. Announced as an "open-source" hardware design, running Ubuntu Linux. File contains additional materials.
History of developments for end-user programming. Says first end-user-programmer 1980 (non-computer-person), MacIntosh. Defines end-user programming as end-user not constrained to an existing application.
Multi-touch touchscreen using projected capacitive technology (electrostatic?). Includes comparison chart for touchscreen technologies: Infrared, Surface-Acoustic-Wave (SAW), Surface Capacitance, Resistive
gestures on multi-touch/pressure-sensitive touchscreen for moving layered object: sheets of paper, stacked documents, overlapping windows. Press down on one edge to "raise" opposing edge (tiddly-winks style), "peel" gesture to see what is underneath. Haptic, visual (shading, drop-shadows), audio feedback. Cites to "Keepin' It Real".
Experimental multi-touch system (Rear Diffusion optical detection: rear DI) incorporating Z-order gestures by tunnelling and bubbling (for overlapping objects). Refers to larger target area as "iceberg tips" of object. Gestures are simple, single-stroke, and few (touch, drag, etc. except for double-tap) to avoid recognition ambiguity. Discusses dynamic gestures (e.g. pinch/zoom) which are detected and interpreted progressively, and static gestures (e.g. tap) which have no interpretation phase
File contains additional material on open-source code.
Not to be confused with Graffiti unistroke handwriting. Contains review of multi-touch hardware and software systems, some reference to Tablet PC but none to handwriting input. Multi-touch Vista, Multitouch.Framework on MacOS/iPad, Sparsh-UI. Gestures recognized by both spatial (shape?) and temporal (motion/movement) aspects of fingers moving over surface (compare with Nokia in-air optical gestures?) Tangible User Interface (TUI)/ Graspable User Interface; lasso/lazo gestures. Defines gesture as including motion. Cooperative gesturing as distinct from multi-user gesture systems for collaboration. Hovering gesture is press-and-hold, not in-air hover: uses two-hand hover on two objects to link them. iPhone generally determines target of gesture on touch-down / hitTest:withEvent / gestureStart, not finger-lift/mouse-up/gestureEnd
Partial reversal (basis for damages) in Lucent-Microsoft case: only mentions Date-Picker UI, not GO Corporation damages
Capacitive grid / mutual capacitance touchscreen on flexible substrate, to fit to curved surfaces. Comb-like interdigitated pattern between rows and columns, also three layer with intermediate coupling conductors. Mentions running traces on single layer plus vias.
Stylus with force feedback: magnetic fields acting on metal ball in tip.
Dual-screen multitouch portable computer with virtual keyboard, compare with Microsoft Courier and with Refalo, and with proposed design by Negroponte's project
File on Diamond Touch, spin-off company for DiamondTouch multi-touch digitizer table developed at Mitsubishi, comparable to Microsoft Surface
Apple iPhone patents asserted against HTC/Android and Nokia: antenna design, geographical positioning
Nokia sell Windows Hpone 7 instead of Symbian.
Review of techniques for transmitting signals through human body, some applications. Inter-body and intra-body body/device communications for body sensing networks (BSN). High speed data rate (up to 1000 Kb/s) using radio-frequency signals coupled through human body. Cites to Zimmerman capacitive-coupling signals for smart furniture at MIT.
Technical dictionary of computer terms 2009
Stylus for electromagnetic tablet, with multiple side buttons on stylus. Using stylus (optionally) in proximity only, so as not to block view of screen(?). Mentions using multiple styli (multi-touch) by having them transmit at different frequencies, or modulate signal. Also sending data from stylus by modulating signal. Compare with Wacom?
Examples of gestures for real-physics playing cards application (see "BumpTop") on surface touchscreen. Pinch/expand gesture have multiple forms: touch and swipe with second finger is photo crop, fan-out drag gesture for simulated deck of cards, etc. Two finger tap gesture to zoom, flick to move between photos in slide show.
Large whiteboard digitizer using projector, multiple small untethered electromagnetic tablets for other participants in a group: see KayeS. Interwrite DualBoard for Windows and Mac, Interwrite Mobi small wireless digitizer. See also Qomo HitVision RF wireless tablets.
Borland SideKick with phone dialer, PIM: Version 1.0 through 20.0, 97 through 99, for DOS, Windows and MAC
Version information for Lotus 123 versions: www.oldsoftwareinfo.com sells and collects old versions of software in original form. Lotus 123 for DOS only through 2.2X 1991, first Windows version 1.0 for Windows 9991, first MacIntosh version 1.0 1991, OS/2 version 2.0X, 1993.
Touchscreen / integrated display and digitizer tablet from Wacom and E-Ink.
Editorial comment on E-Ink e-paper displays, handheld devices: color display
Metal disk with fold-over/bendable prongs to attach to glove fingertip for use with capacitive touchscreens. User's finger contacts prongs inside fingertip.
Questioning methods and techniques on expert witness testimony and reports
Recognize multi-touch gestures based on shared features: move/tap/rotate/spin gestures recognized on features motion/blobcount/rotation/scale. Focuses more on direct manipulation gestures. Describes event bubbling through parent windows for gestures.
Multitouch optical touchscreen sensor: infrared LEDs in pairs behind an LCD display -- LCD displays transparent to infrared. One LED used as infrared emitter, other used as infrared detector, for infrared light reflected from objects touching the screen. Has enough proximity sensing to detect hover. Compare Fig. 7 (image of hand) with Rekimoto 2002 paper (electrostatic/capacitive).
Article on touch screens, and multi-touch gestures. Cites Elo TouchSystems and Sam Hurst as first touchscreen (resistive tablet) in 1971 (missed earlier references). Quotes Bill Buxton on haptic design. Jeff Han of Perceptive Pixel on pressure-sensitive gestures, push harder to move something "under" another window instead of double-click operation.
Extended mobile phone camera as 3D input device in addition to touchscreen: use optical flow recognition of camera to "paint with" on a physical map or other surface, coupling of two smartphones (bi-manual) via bluetooth.
Electrostatic digitizer / capacitive touchpad vendor
in German: overview article on capacitive touchscreens using ITO Indium Tin Oxide, based on Quantum Research Group (commercial company)
Extension of chorded keyboard (on multi-touch touchpad) to defining chording input as user-defined chord (multi-finger) gestures. Numerous citations to Fingerworks.
In multi-touch / multi-pen / multi-user desktop with tablet or touch input, identify user inputs by physical characteristics of writing, rather than constraining users to keep track of which pen to use: pen tilt, pressure (force), and writing speed. tablet-PC lower case, not Tablet PC. Hardware was Wacom Cintiq tablet with tilt and tip force sensing.
Anoto forms: Send non-pen data (e.g. bar codes) as if it were pen-stroke data in an unused margin of the printed form. Mentions in passing paper stylus processor/PC contacting look-up server (PLS), and application server based on response from PLS. No mention of encoding out-of-channel data (e.g. voice -- see Pen Protocol?)
Gestures: one finger to select, flick, mouse-operations; two fingers to resize/move/rotate object; five fingers to pan entire map; fist to scroll or tilt. Commercial multi-touch touchscreen tabletops PerceptivePixel, Microsoft surface, TouchTable, DiamondTouch CircleTwelve. For mouse gestures, multiple users handled first-to-touch wins. Middle finger tap for middle mouse button. "Precision-hover" when finger obscures pixel to be selected.
Two-sided (front/back) touchscreen/touchpad interfaces. Prototype built by taping two iPhones together. Citations to Chen, Hybridtouch, Lucid Touch, and Shape touch.
Anoto pattern: record handwriting information by matching snapshot images by overlaps / using sequence of images to determine motion/position of handwriting stylus. Filed 1999: WO 1999/060467
Historical review (revised from 2002 and 2008) of Tablet PC and Apple iPad. Tablet PC announced August 1999 by Bill Gates, Prototype demonstrated June 2000. Comparison with success of Apple Newton concept in the form of smart phones. Cites to IBM TransNote and Sony SlimTop Pen Tablet.
Optical mouse in a pen-shaped barrel. Newer model includes audio recording: compare with LiveScribe and Plawa.
Compiler design and development textbook.
Hand-gesture input using hand-held magnet and magnetic compass sensor in an iPhone or portable device
UI alternative to soft/simulated keyboard on touch-screen/pen computer: physical keys for separate hands on back of tablet-type computer.
Study of direct input (touchscreen) and indirect input (separate tablet) performance: suggests use of tactile feedback to deal with occlusion of visual feedback on touchscreen by hand or stylus. Without tactile feedback, performance better with larger targets to deal with occlusion. Overall, indirect input (separate tablet, no occlusion) outperforms direct input.
Virtual/on-screen keyboard on iPhone doubted at first w.r.t. BlackBerry physical keyboard (small) on smartphone, but user satisfaction higher than expected. Reasons why?
Pointing out unreliability of data from (electromagnetic) digitizer tablets, even though really only talks about stylus tilt error. One of few papers from handwriting motion analysis that shows insights into some of the problems with digitizer reliability and bad behaviors. Describes morphing of on-line and off-line traces/markings -- compare with Tappert elastic matching?
(in German) GestureID hand gestures, demonstrated at CeBIT exhibition for iPoint Presenter. Mentioned as replacement for touch-screen digitizers at infoterminals (kiosks). Uses optical pattern recognition (machine vision)
Resistive sheet touchscreen interfaces: four-wire, five-wire, eight-wire. Resistive touchscreens very susceptible to electrical noise. Coordinate median filtering (running average), reference to pen-down point and sampling delay (noise at end of stroke). Section on pen-up, pen-down debounce.
Windows 7 Vista will include multitouch, compares with Apple iPod user interface. Shown by Steve Ballmer and Bill Gates at trade show. Gestures include pinching (zoom) and two-finger rotating.
Milan code name for "surface computer", Microsoft Surface tabletop PC, using multi-touch input.
Touch/multi-touch interface in Windows 7 just for show, because touchscreen digitizer hardware adds $100 to price of PC. N-Trig dual-mode digitizer (electrostatic+electromagnetic?) has funding from Microsoft
Study of user gestures for sketch editing on electronic whiteboard: bimanual interaction/gestures not common, hypothesis users accustomed to whiteboard/paper drawing with one hand. Two-finger bring together / separate gestures. Two hand gestures for copy (hold original still, drag out with other finger)
Single-hand typing keypad layout. Shown in video implemented as movable virtual keyboard on tablet computer (TabletPC?). Audio commentary implies FrogPad has basic multi-touch patents: see file.
Product information on Lifebook T900 Tablet PC, convertible with multi-touch separate display
Video demonstration at CEATEC 2010 of prototype Fujitsu dual-touchscreen phone, screens can be oriented end-to-end or fold out side-to-side (special hinge): compare with Refalo?
Fujitsu dual-touchscreen phone, screens can be oriented end-to-end or fold out side-to-side (special hinge). May be combined to be one display, or separate apps (e.g. one keyboard, one application display) compare with Refalo?
Simulated 3-D display: each side of a cube has an LCD display showing a side/top view of something simulated to be inside
Capacitive touchscreen using micro-wires
Hand-gesture recognition using video input and motion tracking: commercial product. Described as touch-free gesture input utilizing computer vision and body tracking software to convert simple hand movements into direct mouse control.
Examples of single and multi-touch gestures supported in Gestureworks open-source library for Flash applications.
Wacom technology (?) capacitive/electrostatic digitizer in an e-book reader, but says being marketed as a "document reader", not a "book reader". See also file on "Plastic Logic".
Touch Book web tablet / tablet computer: Linux, circular-icon gesture UI (pie gestures?)
wearable/carryable devices (smartphones, etc.) communicate with each other signally via capacitive coupling through users body. Devices can respond to initial touch to configure based on user's identity. Identify user by impedance characteristics of body (biometric).
Optical touchscreen using infra-red LED diodes and light detectors. Variations include having LEDs at four corners projecting light in a full plane to the two opposite sides of the LED, and all LEDs on a side being illuminated at once and the detectors on the opposite side read in parallel. States can detect hard and soft touches by detecting position while still above surface and determining vertical height (and velocity) from how much light is reflected from the object/finger above the surface. Prior art discussion mentions resistive and capacitive touchscreen etc. but not optical.
Optical touchscreen using infra-red LED diodes and light detectors. Various geometries for reflectors, such as having LEDs at or below surface of device and low-angled mirror on the full side to direct the light across/over the touchscreen surface; LEDs projecting a plane or lines of light to detectors diagonally (rather than directly across); angled upward from points within touchscreen at an angle and detectors also at an angle to detect light reflected diagonally downward from finger or object. One figure showing light propagated across and within touchscreen glass by internal reflection. Assigned to NeoNode: see NeoNode file.
Programming tutorial for the iPad/iPhone/iPod.
E-Reader maker Spring Design (kindle-like product) sues Barnes and Nobel over using features of Alex e-reader from confidential product technical discussions
VibeTonz and vibrotactile haptic feedback project for Apple iPhone, Stephen Brewster Tacton project at U. Glasgow
Search hits highlighted in Google Chrome scrollbar UI/user-interface: references to work by David McClosy, and Raizlabs ABC and Heatmap
Brief video demonstration of Infoscroller: "thumb" on slider sized to amount of document visible, thumb shows thumbnail images of section of document as reference. Compare with highlighting on scrollbar.
Microsoft research Patrick Baudisch adding touch digitizer interaction to back of devices so that display is not obscured by finger or stylus
Three-dimensional pressure sensitive touch digitizer: IMPAD Inexpensive Multi-touch Pressure Acquisition Device, using force sensitive resistor (IFSR) ink, Ilya Rosenberg, www.touchco.com
Patrick Baudisch of Microsoft: touch pad on back of display, so that small screen remains unobstructed. Also mentions Perceptive Pixel
electromyography sensors in arm used to recognize hand/arm gestures, such as Air Guitar, etc. Recognition by neural network.
Interview with Jeff Han on Multitouch technology by perceptive pixel: states that large screen manipulation is the interesting user-interface area, criticizes capacitive sensing multi-touch input as not scalable to large surfaces or use with gloves compared to frustrated internal reflection
Blackberry keyboard for two-thumb thumb typing: rounded keys, full alphanumeric layout at bottom of display.
WIMP User interfaces predominate, but some applications better with tablet/pen computers: 2D mathematical notation MathPad), chemical structures (ChemPad), Diagramming/sketch input (Lineogrammer). Mentions Wang Freestyle, Pen Windows, GO PenPoint, Anoto photo pen, others.
Button on mouse to "reverse direction", so you can zag back from the edge of the mouse pad (e.g.) to continue drag motion. Compare with synthetic reality prototype?
Pop-up movable virtual keyboard in the form of a marking menu, using predictive work list and pause and lift (press-and-hold) to re-invoke search for text completion.
History of two-dimensional mathematics input languages as programming languages: essentially a short bibliography of papers by Mort. I Bernstein, quotes as saying that real-time / online handwriting recognition is easier than static / OCR character recognition.
File of product information on HP TouchSmart PC, Windows PC with integrated multitouch touchscreen. Used for kiosk applications.
Using BrailleDis tactile/haptic display tablet, study of best tactile representations of conventional windows control elements: command buttons (TactonButton), menus, edit boxes, ribbons, sliders, checkboxes, etc.
Music editor using mouse (vs. pen/tablet) computer. "Scrawl" / QuickSrawl gestures based on line segments in eight compass directions, composed to single stroke. Point of corner used to place note head, etc. Gestures set selected by Pencil, Crayon (dynamics) and Quill modes. Earlier version (for 68K Mac) was NoteWriter, with same Quickscrawl gestures/ routines.
Press article on Reactrix, Monster Media, and GestureTek on gesture input on interactive advertising: foot-pressure touch pad with foot input
See original 2005 paper. Detect multiple finger touches on a rear-projection surface (multi-touch), by reflected light where finger is in contact with surface: same technology used for fingerprint scanners.
Scroll bar displays highlighted/small images of text/document to be scrolled: add-on to Microsoft Visual Studio by Rocky Down's
Based on 2002 GB filing: Acoustic touchscreen using frequency profiles characteristic of different location (Lamb waves: higher frequencies disperse faster) rather than acoustic delay. Identifies touch events (start of touch), not contact. Requires profiling of construction first, to produce a table?
Techniques for smartphone as pointer on large touchscreen display. Sensing smartphone with grid of NFC tag sensors with NFC phone. Show shifted / enlarge area of what is obscured by phone next to physical pointing area. Small screen of smartphone used as joystick.
Simulated 3-D video using flat display: perform trapezoidal scaling of image as viewer moves left and right (and up and down?) as detected by Webcam camera. See also matching video file.
Gesture recognition not using a digitizer, but the acoustic sound of scratching when a finger/fingernail moves on a surface. Cannot detect spatial location, but recognizes multi-part (multi-stroke) gestures such as X, O-with-slash, lowercase-I-with-dot, Y, double tap, single tap. Cites to earlier work on passive acoustic sensing (knocks) and frustrated-internal-reflection digitizer tablets
Proximity-sensing electromagnetic digitizer for finger input on small display, using small magnet mounted on fingertip. Compare with ferrite-rod digitizer technologies? Can detect clicking "below" a virtual surface by reversal of magnetic field, can detect distance/proximity by strength of magnetic field sensed. Describes polar gestures (circular motion around a very small display)
Surface (?) acoustic wave design of touch input device comprising an arm band and propagating acoustic signals through human skin: a wearable SAW touch input digitizer. Also refers to distinctive sounds made when tapping on parts of body: arm, palm fingers and thumb.
File contains additional materials
Project description: Compare with earlier work on sketch recognition: use uses paint interface (not draw/sketch) to draw color patterns, system matches by dominant color and rough orientation, providing results in real time while user is "sketching". See also video file.
Optical-mouse type input built into portable device (basically on back of display). Thus display is not occluded by finger or touching, no digitizer on display. Gesture recognition for flick, twist, curly-cue gesture. Can also scan a larger surface, thus can do scroll-bar gestures.
File contains additional references.
Haptic/tactile vibration pixels around edges of touchscreen PDA computer
Microsoft Robotics Studio 2006, Microsoft Visual Programming Language: Review of visual programming languages and environments: cites to Sketchpad 1963, GRAIL 1967, AMBIT/G 1968, ThingLab and other early systems.
Passive stylus for resistive touchscreen made from "soft" wood with rounded tip. "No Mar" stylus to prevent scratching and damage to resistive film.
Add a haptic output actuator to an existing input devices. Generic to type of input device, and to type of haptic actuator.
T. V. Raman, visually impaired researcher at Google, accessibility technology for low-vision on touchscreen cell phones (e.g. Apple iPhone) using Gestures
Personal notes on EO/PenPoint in the UK, including examples of documentation and product.
Mid-level encyclopedia of computer science and computer technology: compare with http://Wikipedia.org
TouchSmart 2x2z ultraportable Tablet PC: multi-touch input, implies support for multi-touch gestures on laptop running Windows Vista
Tabletop computer system: slide/zoom/pinch gestures include inertia, may show additional areas of an image or of a second image when moved. Long list of references: See also TouchTable (assignee), http://appliedminds.com (IP shop). Railing around table to keep users from resting hand on tabletop touchscreen.
Dual-screen dual-display/dual-touchscreen tablet, 4x6 inch. Compare with Courier and Refalo. Software based on InkSeine note-taking with "floating tool ring" (marking-menu?). Multiple references to other current dual screen computers. Taxonomy of dual-screen "postures": landscape/portrait, book/lectern/laptop, corner-to-corner/face-to-face, battleship (opposite directions). Dual displays can be detached. Collaborative use. Dual-display use includes webpage next to notebook page.
Provide feedback using proximity data on a touchscreen digitizer. Compare with GO operating system, PenWindows.
Using pen and touch gestures simultaneously, users perform different actions with the two hands -- dominant hand holds pen/stylus for writing. Cites to earlier work on two-handed/multi-touch user interfaces.
Combining stylus/pen input and multi-touch gestures on tabletop: e.g. while writing (electronic ink) with pen on two pages of virt zooms back to full virtual desktop view.
Bidirectional LCD sensor/display: LCD display with 3D optical touchscreen and proximity (3D) sensing using synthetic apertures onto optical sensors behind each LCD pixel: no change to LCD. Permits 3D user input manipulation, proximity changes zoom.
Single layer multi-touch ITO capacitive touchscreen / touch panel sensor. Rows of separate sensing electrodes, each with own trace to side of substrate. Alternating rows separated by single sensing electrode trace across entire surface.
Organic user interfaces / tangible user interfaces: deformable user-input devices, such as foldable paper which is also a display, and responds to folding. Shows multiple-touch, multiple-user actions on a globe model, cites to Wellner.
Experimental system using large-area fingerprint scanner as touchscreen: Address fat finger problem (inaccuracy of fine touch with broad fingertip) by recognizing fingerprint to identify user, and adjusting for typical offset/posture shown by that user. Ascribes error to users having different interpretations of touch / touch location.
X/Y grid matrix of electromagnetic sensing coils used to detect ferrous / ferromagnetic objects: objects need not be tagged, if ferrous. Placing flat ferrofluid bladder on array is touch/force/pressure touchpad surface, interpolated to get higher location resolution. One use is sensing of actual objects: e.g. physical slider with ferrous handle.
Displaying lists or tables of objects (e.g. .JPGs, .MPG3, etc.) automatically sorted by different attributes, and sub-trees (e.g. folders) can be expanded or contracted in the list. Refers to sticky paths for references to objects.
Essay defining tangible interaction as distinct from direct manipulation in user interfaces: physical objects interact with users under computer control: numerous references to background of the art. See also augmented reality.
Scrolling momentum / dynamic scrolling / inertial scrolling by Synaptics in a touchpad digitizer, compared to trackball that keeps spinning after initial push
Prototype ultrasound tactile display with 3D three dimensional finger sensor. Display is array of 324 ultrasound transducers, phase and amplitude control at 40kHz, produces single pressure focal point 20 mm diameter, AM modulation for vibration up to 1 kHz. Sensation compared to feeling a rain drop strike hand. Does not discuss hand blocking/obscuring either sensor or display. Touch sensor is IR reflective tape on fingertip with WII infra-red sensor.
Capacitive multi-touch skin (flexible or solid touchscreen material) on outside of objects. Compare with MIT Capacitive sensing?). Sensing of force by embedding small slugs (metallic) in elastomeric layer on top, depression moves slugs relative to grid (compare with Rekimoto?), displacement/force (X/Y or Z) computed from changes in capacitance to slugs.
Single-layer multi-touch/Multipoint mutual capacitive touchscreen (not really a display). Compare with Cypress?
proximity-sensing hand/palm rest near keyboard: if detect user's palm/hand near keyboard, filter out touch input (palm rejection).
Multi-touch/Multipoint mutual capacitive touchscreen: compare to Fingerworks and Rekimoto?
Touchpad the full width of a laptop computer, instead of just a touchpad in the middle. Allows user to move hand less away (horizontally) from keyboard position to use touchpad. Palm discrimination by sensing fingers over keyboard, ignore touch input on touchpad (using separate proximity sensor, or visual sensor above keyboard on top edge of display screen).
jumbled silver nanowires as replacements for ITO indium tin oxide in touchscreens etc: graphene electrodes in thin film; single-walled carbon nanotubes CNT, metal/silver nanowires. Compare with other wire mesh.
Motion and/or position sensor in portable cell phone, PDA or hand-held unit to detect user activity: motion / orientation sensor on handheld tablet device? iPhone/iPad
Acoustic/sonic digitizer, single acoustic receiver that mounts on corner or top of paper (or other surface) and transmitter stylus/pen with optional ink cartridge. GPEN100C, Applications include note taking (Microsoft OneNote, IRISnotes, etc.) and handwriting recognition (MyScript). Device may also be used as stylus or in mouse-input mode with side button on stylus barrel.
File contains additional references
Miscellaneous iPad information
OCR software for IRISpen and other scanner devices. IRISnotes software for IOGEAR acoustic stylus digitizer tablet: compared with products requiring special paper (Anoto) and optical digitizer. See also IOGEAR file.
iRex Technologies e-book reader: marketed with additional applications (compare with PenPoint from GO): uses Wacom capacitive/electrostatic touch digitizer tablet component. Additional materials in file.
Exhibit-construction service for Museum exhibits, etc. (compare with Slavko?), and GestureWorks multitouch authoring framework for Adobe Flash. Gestures include touch, tap, drag etc. Support pinch gesture for zoom, distinguishes (for some reason) between one hand zoom (spread fingers of one hand on touchscreen), multi-point zoom (both hands) and pinch zoom gestures (two fingers, regardless of hand); similarly multitouch and two-finger rotate. Three-finger slide gestures for tilt (vertical) and yaw (horizontal gesture). Lists multiple (Windows?) touchscreen computer vendors.
On multi-touch touchscreen UI, user can stretch and distort figures (animations), object is triangular mesh, minimized aggregate distortion of mesh elements. Extension to 3D three-dimensional graphics.
Mechanical digitizers using multiple-link mechanical arms, articulated-arm digitizers and portable CMMs. Intended for three-dimensional / 3D digitizing of engineering objects. Applications also include medical and surgical simulators : digitizers with haptic feedback under the product name CyberForce. Several citations to studies of touchscreens with and without tactile/haptic feedback, and VibeTonz auditory/haptic components for hand-held devices (cell phones, PDAs, etc.)
Listing of papers (including research papers) from http://immersion.com for haptic/feed-back
Haptic: Tactile Feedback, Multi-modal feedback, non-visual interaction. General explanation of haptic terms.
Signature authentification / signature recognition as biometric identification: notes that paper documents are non "de-materializing", authentication of paper documents still needed.
Press releases and technology descriptions for electromagnetic tablet digitizer for Tablet PCs. Follow Microsoft model of active/passive classifications, passive is front-mounted resistive touchscreen, active is rear-mounted electromagnetic. Grid-powered and battery-powered pen/stylus. Multiple pens/stylus identifiable by ID code: (MagicPoint Digitizer writing pen) pens for colors, or identifying multiple users (also multi-touch?).
InPlay: Digitizer Technologies for Mobile Computing Applications
InPlay: Active Pen and Digitizer Human Input Devices Digital (InPlay) vs. Analog (Others)
InPlay: MP800 Overview: tablet/MagicPoint pen/stylus, custom interfacing chip set for electromagnetic digitizers.
Mostly interview with Chris Harrison and Bill Buxton regarding multi-touch input and touchscreens: mentions Casio touch-screen AT-550 wristwatch in 1980, Lemur touch-surface music controller in 2003 (see reference) quotes Bill Buxton on touchscreens being unusable by blind/visually impaired (compare with iPhone VoiceOver?) Chris Harrison latex touch-surface covering.
SID special issue on touch technology (rather than displays), especially multi-touch digitizers. Compare with 1990's article in Information Display by Ward/Tappert?
Advertisement: IRTouch multi-touch is ready for Windows 7
Editorial: The Limitless Horizon for Touch
Article: LCD In-Cell Touch
Article: Projected-Capacitive Touch Technology
Article: The State of the Touch-Screen Market in 2010
Article: Touch Screens and Touch Surfaces are Enriched by Haptic Force-Feedback: inertial actuation, piezo actuation, (electrostatic) surface actuation, lateral actuation, (contracting) electro-active polymer actuation, bending wave, capacitive electrosensor interface (CEI) in nerve endings
Journal of the SID Article: Beneath the Surface (Microsoft Surface)
Journal of the SID Article: Taking Touch to New Frontiers: Why It Makes Sense and How to Make It Happen
News: FlatFrog Laboratories optics-based multi-touch kits; touchco flexible touchscreens
Mouse with additional pressure sensors: references contrast with WIMP interfaces
"Zeroclick": Generate click/tap/up/down events (gestures) without lifting stylus, by moving to GUI element and then moving in a gesture (reversed motion) within the element area: compare with dwell in accessibility technologies, smart buttons? Generally, pen interface for medical forms: see also 8,549,443.
Special issue on organic/graspable/haptic/tangible user interfaces. Deformable/rollable input devices, such as recognizing shadows of physical stick figures or clay models.
Prototype tactile feedback for 3D three dimensional display by sound pressure on skin (of hand/fingers) from hexagonal/annular array of ultrasound/ultrasonic transducers. Resolution about 20mm -- comparable to width of fingertip. Compare with UltraHaptics.
Measure touch-typing pressure (actually force -- no, actually acceleration) using built-in accelerometer in MacBook portable computer. Applications for pressure form mode switch, emphasis/Bolding in messaging, biometric authentication using typing motion pattern. Compare to application of stylus/finger pressure/force in touchscreen/tablet applications.
SecondLight: Combined two-image display and multi-touch sensor: surface has cells (like LCD) electronically switchable between diffusing and transparent state: in transparent state, image from display behind is visible, and an optical camera behind the display can see objects above the surface. In diffusing state, rear display not visible, but reflects second image projected from above. At the same time, can sense touching fingers on surface from reflected light by FTIR frustrated total internal reflection. Mentions measuring area of contact of fingertip as indirect measure of pressure/force, and major and minor axis of oval to determine rotational orientation/angle of finger tip.
Version of 2007 and 2008 papers: Multi-touch optical touchscreen: mount array of infra-red transmitters and receivers (coarse) behind and LCD, detect reflection of infra-red reflective fingers and objects through LCD. Bi-cubic interpolation to interpolate to higher accuracy for "center" of a finger touchpoint. Cites to Rekimoto.
Handbook/dictionary reference work on biometric technologies for authentication of identity. Section on spoofing of biometric input, including spoofing of fingerprints back to 1920's.
Janam XP30 touchscreen/digitizer smart phone: Uses Graffiti 2 gestures: backspace, shift, menu (diagonal) Shortcut (bottom curlique, cut (pig-tail), paste gesture, copy (down-left), undo
Scroll/drag a web page using a stylus on a touchscreen: when move stylus to perimeter, page will auto-scroll with simulated momentum.
Simple haptic text display for deaf-blind: smartphone touchscreen divided into six parts to mimic Braille dots, vibrates when user's finger is over a part corresponding to a raised Braille dot.
MIDI controller/GUI software toolkit for touch gestures specific to controlling music and audio systems. multi-touch.
MIDI controller/GUI software toolkit for touch gestures specific to controlling music and audio systems. multi-touch.
multi-touch digitizer capable of handling an unlimited number of fingers at once: genesis of the device was that the developers designed a user interface assuming such a device was available. multi-touch.
Gesture (multi-touch gesture) recognition by heuristics, Numerous other iPad/iPhone gesture patents incorporated by reference. Specific to finger contacts / touchscreens, not other stylus digitizer input. Very large spec (362 pages), shows slide-to-unlock.
Includes description of JazzMutant/Stantum multi-touch touchscreen, overcomes masking/shadowing effect of simple switch matrix designs by using active matrix switching array (similar to active matrix LCD circuitry) to measure each coordinate in grid of perpendicular wires separately. Also refers to using conductive rubber or piezo switch elements at nodes.
See also 2007 SID article by Largillier
Historical review/survey of sketch recognition and gesture command in sketching application in pen-based (tablet) interaction/pen-computing. Cites to Sketchpad and GRAIL systems using lightpen. Ambiguity/context in traditional (pen and paper) sketching: wavy cloud becomes foliage of tree if trunk added to sketch. Whiteboards and collaborative sketching. SimuSketch for electronic circuit sketching. Sketch/handwriting recognition: Ledeen, Rubine, Kara and Stahovich, $1 recognizer for PDA. Cites to this bibliography in references.
Handheld PDA-style touchscreen for order entry (in a restaurant or any sales situation), browsing, and credit card authorization. Compare with Way Systems?
http://Madronalabs.com Madrona Soundplane: force/pressure sensing X/Y touchsurface, intended as controller for electronic musical instruments. Multi-touch sensing. Cites to earlier devices.
"Control intimacy" -- tightly-coupled haptic feedback? Capacitive grid force-sensitive touchscreen tablet: X and Y antennae separated by a rubber/elastomeric sheet, pressing brings wires closer together for stronger capacitive coupling. Cites to Radio Drum from 1980's. Used as musical instrument interface.
Multi-dimensional force sensor: multi-touch sensor used to sense dynamic pressure image over the touchpad surface. Grid of X and Y conductors, forming capacitive-sensor nodes, separated by layer of rubber dielectric. Dynamic calibration to compensate for mechanical hysteresis of rubber sheet (persistent pressure indication after touch has ended). Interpolation of values plus centroid detection, correlating to centroids detected in previous sample (tracking touches).
Accessibility multitouch touchscreen gestures for visually impaired: compare with Apple VoiceOver gestures? Second-finger tap (see earlier touchscreen gestures w/o multi-touch), "L" gestures, one-finger scan/scroll, flicks. No citation to VoiceOver.
For one-handed PDA use, user defines a ThumbSpace -- the area reachable by the thumb while holding the PDA in one hand. Show thumbnail (sic) image of whole screen in ThumbSpace for user to pick from on whole touchscreen. Refers to whole screen touching as DirectTouch.
Tutorial on touchscreen digitizers/tablets: Active (?), Resistive, Capacitive (touch). Fujitsu Tablet PC P16xx series resistive digitizer with palm rejection. States: Active digitizers only suitable for handwriting, also states resistive with a stylus or fingernail is accurate. Cites hand "vectoring" on resistive. Says only capacitive can do multitouch. Capacitive respond to skin touch. Multitouch only on small devices (phones), not on tablets (except for new iPad).
Optical (interrupted light beam) touchscreen, detects multiple touches unambiguously by projecting additional diagonal beams across surface: diagonal beams produced by same narrow-angle infrared LEDs by means of beam splitters.
noise from LCD or other flat-screen display coupled to capacitive touch screen (e.g. VCOM signal in LCD) when user touches (through user's body), address by synchronizing with LCD display electronics.
Small curved pin-cushion unit for tactile displays -- actually for wearables? Simulates texture both by static pins (4x4 grid) and by vibration (up to 20 Hz). Compare with HyperBraille / BrailleDis 7200.
Small curved pin-based unit for tactile displays -- wearables, tactile texture display on mouse. Simulates texture both by static pins (4x4 grid) and by vibration (up to 20 Hz). Compare with HyperBraille / BrailleDis 7200.
Touch-sensitive bezel around a display (not a touchscreen), display icons next to "buttons" on bezel. Compare with Wang "The Guide"? Allows for rotation of display (compare PenPoint auto-layout?)
Report on perceived versus actual performance in user interactions: mentions "kinetic scrolling" and "two-way scrolling". Compare with Apple slider/slide-to-unlock?
User selects content from a server, and it is downloaded.
Cypress Semiconductor white paper on specifications for capacitive touchscreen using XY grid for multi-touch digitizers. Accuracy, finger separation (minimum discernable distance), response time, refresh rate (a.k.a. report rate or data rate in samples/sec), power consumption. Compare with Ward paper on digitizer technology: paper does not mention stylus tilt, registration, noise, fixed or variable error, etc.
Report from Consumer Electronics Show on Microsoft digital table / surface table touch digitizer, with direct manipulation of photographs. References to iPhone interface. Digitizer touch sensing limit is 52 touch points: four players, 10 fingers plus 3 playing figures each. Compare with Wang Freestyle for graphical manipulation?
Report on Microsoft Windows 7 touch-UI interface. Also mentions digital table / surface table touch digitizer, with direct manipulation of photographs. Points out awkward positioning of touch screens at a desk, compares with keyboardless one-laptop-per-child touch system.
Nokia N8 cellphone with touch-screen interface, runs videos (Linux OS?) and GPS Ovi Maps no charge.
Overview report on E-Ink, Plastic Logic, and other vendors of electronic paper and flexible displays. E-Ink acquired by Prime View International of Taiwan, which manufactures the Kindle. Mentions color displays: no information on tablets or digitizers. Electrophoresis, electrophoretics, electrofluidic technologies.
Mentions computer input of two-arm gesture holding arms out to sides, recognized optically
Demo of marking menus versus linear menus: date of video probably earlier than posting date.
Netpage pen/stylus: piezo-electric force sensor in stylus, with spring to absorb shock and protect sensor if hit hard. Netpage is an optical position-sensor stylus using camera and infra-red illuminator/LED and a dot-like encoding of position. Cites to Sekendur. Compare with Anoto? Note: Silverbrook patent.
Volume control on (on touchscreen) when an external amplifier/speakers (headset) are plugged in. Related to LabView?
Light-sensitive touch panel with display, photo-sensor arrays integrated into cells of electronic ink paper display. Capacitive coupling to finger to avoid shadowing of light.
Capacitive touchscreen does not respond to fingernail. Resistive touchscreen does not respond well to fingertip touch (versus stylus or fingernail). Feedback (haptic/vibration or audio) improves usability of soft buttons/keys (virtual keyboard).
Pressure-sensitive piezoresistive elastomeric material in touchscreen, conductive particles in transparent elastomeric material. Particles smaller than wavelength of visible light for transparency.
Accessibility adaptation of smartphone with physical keypad for the blind: larger buttons, wider spacing, static mapping of menus to keys. No citation to VoiceOver. Electronic file includes presentation slides.
Prototyping UI design using UIMS with storyboarding, either with paper or with software tools. Prototype with Toshiba Tablet PC (Microsoft) and HP iPAQ Pocket PC with interactive maps.
Using Anoto pen, user can mark up physical print-out (stored document, Anoto pattern encodes which print out) with (dynamic) gestures for selection, cut-and-paste (into location in handwritten text), tagging, email portion. When pen synchronizes, actions performed on original electronic data. Gestures include lasso, underline, rub-out (Z erase) to cancel mark/gesture. Notes that tagging, annotation capture supports active reading, where reader underlines/marks/annotates while reading. Lack of interactive feedback a problem.
Holographic projector and infrared sensor for creating a multi-touch "touch screen" on any surface on which an image can be projected.
Scrolling using multiple taps in scrolling press zones: more taps, faster scrolling, also tap to stop scrolling. See Touchscreen Google: compare with press-hold?
Review of multi-touch gestures. Cites to 2003 for flick gestures (with hand or finger), but does not mention PenPoint/GO. Cites Buxton on multi-key/n-key rollover on a keyboard in 1970's being a form of multi-touch.
multi-touch digitizer capable of ten fingers concurrent/simultaneous input. PMatrix from Stantum. Resistive touchscreen display. Quotes Jim Meador at Stantum regarding non-proprietary nature of Apple iPad pinch gesture. Compare with JazzMutant.
Stylus for capacitive touchscreen. Small contact area would not give good performance with capacitive touchscreen.$a Refers to lost points (skipping / missing points), overflow point (points after pen up), worse when finger is wet. Flexible conductive material on tip of solid conductive material stylus.
Voice/handwriting recording using Anoto digital pen and digitizing (tablet) paper. Includes synchronized voice recording. Article / press-release attempts to make accessibility connection to being annotation device for the visually impaired.
Voice/handwriting recording using Anoto digital pen and digitizing (tablet) paper. Includes synchronized voice recording built into pen. Touch previous mark, pen plans recorded audio recorded at that time of writing: example is write "define" followed by word, get definition. Uses Anoto technology, see also EchoPen.
Collection of articles on Livescribe / Anoto pen. Tear-down photographs, applications e.g. talking dots aid for blind, labeling of physical objects.
Use Anoto Pen (Livescribe Pulse or Echopen) and printed labels to make audio notes that can be played back by visually impaired/blind user or motor impaired by touching sensor to printed dot. File contains additional references, such as LiveScribe/Anoto Post-it notes. UI for creating notes still requires use of printed paper with control functions. Compare synchronized voice and handwriting of Freestyle with LiveScribe Anoto pen.
Video demonstration of Casio PF-8000 calculator, using keys as digitizer input for recognizing handwritten numerics and math operator symbols.
Fast tutorial for practitioners on Perl programming language: method calls, inheritance, Object-Oriented Perl, polymorphism.
Microsoft found to infringe GO patent on using gestures on Tablet PC. Other patents in case concern video compression, forms input, and color management.
Exhibits (technical references) in Lucent v. Microsoft in San Diego: long list.
Press articles on Lucent v. Microsoft patent case concerning United States Patent 5,544,295 on tablet PC and gesture-based user interface
Biographical information of major personalities in Apple Newton development team: Steve Capps (Finder), Walter Smith (NewtonScript, a.k.a. WallyScript), John Sculley, Michael Tchao ("Apple had been working for years on slate computers that read handwriting"), Stepan Pachkov, Larry Yaeger (Cursive Recognizer)
See also 2002 version
Detailed specs on Apple Newton MessagePads, several models,: OS 2.0, 2.0 1.3, and 1.0 released 1993. Associated with Steve Capps.
Architectural overview of Windows 3.11 operating systems: Windows 3.11 standard mode -- 16-bit protected mode / Enhanced Mode: 32-bit virtual machine manager in 32-bit protected mode, running standard mode windows and DOS as virtual machines. Enhanced mode as new operation system with preemptive multi-tasking and virtual memory. Win32S application ran in 32-bit mode.
Scanning / virtual keyboard for accessibility for physically impaired: single key or other input, display scans through character, user selects character with key when the character comes up. One reference to touchscreens.
MacWorld Expo report: gloves (thin) with conductive wires to allow use with iPad and other capacitive touchscreens: Telefingers, iTouchGloves.
Capacitive/electrostatic tablet, single layer, with three sets of (horizontal) conductors with wavy widths. Which conductor(s) give coupling give Y ordinate, degree of coupling between adjacent conductors (of the three) gives X ordinate. Alternative "fishbone" pattern has cross hatches of different widths. Compare with charge-ratio tablets?
Virtual keyboard (transparent/translucent: or may display image of touchpad, or other image) comes up on touchscreen when proximity of user's hands is detected, removed when hands no longer near surface.
Screen shots of Windows for Pen Computing 1.0 on Windows 3.11 from 1992, German-language version. All applications (but not OS) were pen-enabled by means of pen tool "Stiftpalette". Notebook / Notizbuch application allowed free-hand electronic ink note-taking. Gestures/Gesting include check-mark to edit text, similar to PenPoint OS. Comparison with Windows for Pen Computing 2.0 on Windows 95, which is described as more of an add-on and less integrated. Shows multiple gestures accepted in text editor: check mark to edit text, space, new line, cut (pigtail), undo (loop), backspace.
Screen shots of Windows XP Tablet PC edition, German-language version. Lonestar released in 2002.
Screen shots of Windows Vista, German-language version, 2007. Includes Tablet PC functionality.
Revised version of 2008 posting on Windows for Pen computing 1.0 on Windows 3.11 from 1992. Includes video (see file) demonstration PenWindows 1.0.
Detection of events (gestures?) on a touch-pad and filtering the native coordinates based on those events. Compare with Microtouch, and with Pencept and CIC recognition Macros?
Tactile (haptic) feedback on pressure/force-sensing touchscreen, depending on multiple levels of pressure
Tactile (haptic) feedback on pressure/force-sensing touchscreen and on mechanical buttons in same device (smartphone)
Licensed model of Anoto digital pen with patterned paper for position detection. Bluetooth, multiple simultaneous pens (with unique serial number identifier for each stylus). Exploded view of hardware / stylus design. Applications include medical records review, in-the-field data collection, handwritten messages no character recognition). Note: development website in Japan.
Touchscreen absolute mode vs. relative (mouse) mode. Tapping positions cursor in absolute mode, dragging is in relative mode with fine position control. Reduces occlusion and accuracy problems. Cites to alternatives, e.g. Hybridpointing, tilt of PDA using accelerometers.
Touch typing on a multi-touch touchscreen keyboard: user first touches to establish home row position (e.g. eight fingers gesture), virtual on-screen keyboard displayed as visual feedback/confirmation at that position, user may then type at positions relative to established home row position: compare to Westerman?
Flexible tape sensor using optical fibers to detect bend and twist configuration of the tape. Discussion of limits to resolution and detecting due to spacing of points of sensing along the tape: bending or dent between two sensing points not detected.
Capacitive touchscreen/digitizer with actuator for haptic feedback, can be mounted to frame/carrier/display for ClearTek II touch sensor touchscreen
Capacitive touchscreen/digitizer that senses either a stylus or a finger touch -- but not both, stylus/pen takes precedence -- stated as palm-rejection (hand rejection). multi-touch.
Microtouch: Software toolkit MT7 for gesture recognition. Refers to "Slow Draw Anomalies" with jagged electronic ink (polling input of touch coordinates on tablet?) Stroke matches (elastic matching?), Strokes and Stroke data objects.
Electronic file includes MicroTouch Capacitive TouchSense system with haptic transducer touch feedback for Gaming, 3M MicroTouch Software MT7 User Guide.
Windows Vista / Tablet PC: defines gesture as "a glyph that defines the shape traced by the gesture", plus a hotpoint
Tutorial for Tablet PC developers on basic physical differences between stylus/pen interaction with a digitizer, and operation with a mouse. Does not refer to details of mouse/stylus message difference in programming. Tablet resolution in Tablet PC is normalized in software to 0.01 mm. Size of targets, involuntary hand movements, hand/stylus obscuring view of display, hover/proximity feedback with digitizer tablet. Involuntary hand movements (with stylus) affect accuracy. Target size 9mm for finger, 2.5 mm for stylus (electronic "active" stylus), not hand-held.
List of System Gestures for Microsoft Tablet PC/Vista. Lists Tap and DoubleTap as gestures that work for both system and application. Description for Tap refers to choosing a command from the menu or toolbar, take action if command is chosen.
Definitions of Application Gesture, Gesture, Application Gesture, Glyph (as ink data). Refers to "Microsoft gesture recognizer" as engine that specifically recognizes ink iput as either system or application commands. "Object recognizer" for recognizing non-gesture, non-handwriting shapes such as musical notation, math notation (i.e. sketch recognition would fall into this). Stroke based on pen down/up. ISF Ink serialized format (similar to JOT?)
Vista/Tablet PC SDK documentation from MSDN: defined gestures include scratch-out/erase, Triangle/insert, Check-off, Curlicue/fixed (compare with proofreader's marks), Double-Curlicue/copy, double-circle/paste, left-semicircle/undo, right-semicircle/redo, Caret gesture/paste, Inverted-caret/insert, flick left / backspace, flick right / space (compare with GO flick gestures), left/right/up/down corners with directional gestures. See also 2003 version of same reference.
General announcement of Ultra-Mobile PC (UMPC) running Windows XP Tablet PC Operating system. See also 2006 version. Ultra-Mobile PC Development Quick Start Guide. Keyboard optional, may include GPS, webcam, fingerprint reader, digital TV tuner. Microsoft Touch Pack for Ultra-Mobile PC UMPC. Brilliant Black skin with large buttons, DialKeys on-screen keyboard (for thumbs) from Fortune Fountain Ltd. UMPC display Emulator.
Microsoft Surface (optical, infrared surface reflection, rear illumination) tablet, user interface guidelines. Multi-touch, shows whole area of contact (e.g. hand). Emphasis is multi-user with 360-degree rotation. Primarily direct manipulation: No "system gestures" for commands (e.g. gestures by shape): Move, Rotate, Resize, Flick, Slide/Drag, Spin, spread (three fingers), turn (one touch stationary). Reads infra-red tags on objects on surface. Virtual keyboard (but no examples).
Windows 8 touch gestures, and mouse equivalents: tap/double-tap == click/double-click; 1 or 2 finger up/down drag -- panning with inertia / scrolling; pres-and-tap (one finger, then other) right-click; pinch to zoom (CTRL key + scroll wheel); two-finger rotate == no mouse equivalent; two-finger tap = focus is midpoint(?); press-and-hold = right-click; flick = scroll. With press-and-tap: single-sheet touchpad/touchscreen?
Tap gesture of finger represents left click of a mouse. Mouse events WM_MOUSEMOVE and WM_LBUTTONDOWN events not coordinated with GID_SELECT message for gesture. Double tap for left double click of mouse, using GID_DOUBLESELECT gesture message. Hold (press-and-hold/dwell) generates separate GID_HOLD message, used to represent right click of a mouse, GID_END messages at end of hold time threshold. Flick gesture GID_SCROLL message to scroll, GID_PAN gesture to pan (can be after hold gesture)
Microsoft Project collaborative work system: includes CPM critical path method, Gantt charts, resource and schedule views. Agile project plans.
Corporate history page on Microsoft Windows. Interface Manager code name, announced as operating system 1983, released as Windows 1.0 1985. Windows 1.0 in 256Kb Ram. Windows 2.0/2.11 1987 Control Panel. Windows 3.0 operating system. Windows NT new operating system 1993. 32-bit. Mentions nightly build/stress tests in development. Very brief mention of Tablet PC, none of Pen Windows: Windows Touch on Windows 7.
User interface design guide for Windows 7 touchscreen smartphones, Metro interface. Gestures Tap, double-tap. pan, flick, touch and hold (press-and-hold), pinch and stretch. Tap stops scrolling/panning, double tap toggles zoom state. Touch areas (not target) must be at least 7mm, with at least 3.5mm separation between multiple touches.
Standardized design templates (screen images) for Windows 7 touchscreen smartphones, Metro interface. Application bar/menu, toggle buttons, application buttons, standard icons, on-screen virtual keyboard, etc. All in portrait mode (?).
Gestures for Windows Touch (multi-touch): tap, double-tap, panning (inertial scroll), press-and-tap (two fingers) for mouse click, two finger zoom/pinch, two finger rotate, tap with two fingers (center point is hotspot), flicks, press-and-hold (right-click). Posts concerning error/discrepancy between WM_TABLET_FLICK and WM_GESTURE messages.
Using a gaze tracker / visual input to simulate mouse events: no specific reference to gestures, but to graphical user interface
Review of design principles for intuitive user-friendly interfaces: primary mode of affective communication between humans is visual, not speech. Cognitive and emotional context of interactions important. User interfaces should be designed to target population: e.g. children. ShowMe Tools: Example of automatically-retracting palettes at top/bottom of display instead of menus. One-touch resize gesture (direct manipulation) by pulling at any corner. Flick gesture to mirror image, arc with finger on corner (direction of motion) to rotate. gesture control for large-screen art museum display: large arched bar with proximity sensors for left/right/up/down hand gestures.
Educational uses of touch technology and human/computer design: Please Touch Museum (touch-screen user interface) at Phoenix Art Museum and Cleveland Museum of Art, Dallas Museum of Art: KiddyFace digital environment.
Small (1" square) electronic boards with sensors, can be attached to cover any surface. Compare with electronic artificial skin, Stiehl "Huggable" (cited), touchscreen electronic skins (Rekimoto cited). Combines light, whisker touch, audio/sound, and pressure sensors: note that most pressure stimuli also produce an incidental audio sound.
Electronic ink application using small outboard (sonic?) digitizer with a PC, and paper with embedded RFID tag. Lets writing/ink be tied to the piece of paper, similar capability to Anoto technology, however RFID can be used to locate the paper (which may be tucked into a book or folder) using RFID technology, retrieve specific notes. Demonstration includes used of deferred recognition to index/search notes.
Electronic ink application using small outboard (sonic?) digitizer with a PC, and paper with embedded RFID tag. Lets writing/ink be tied to the piece of paper, similar capability to Anoto technology, however RFID can be used to locate the paper (which may be tucked into a book or folder) using RFID technology, retrieve specific notes. Demonstration includes used of deferred recognition to index/search notes.
Display image can be projected onto any surface, including a hand: tips of fingers (more than one) identified by colored bands on fingertips allow fingers to be detected optically with respect to image, forming an optical digitizer touch surface.
Review of mathematical (2D) user interfaces. Sketch editing (free-form electronic ink) versus mathematical expression editing (semantic). MathWebSearch, SwiM, Sentido Mathematical Environment, xfy MathML, ActiveMath (web e-learning with search engine), MathType (for TeX), BrEdiMa, TeXmacs. Cites to Thimbleby.
thumb-gesture input combining motion and pressure/force sensing: light touch is move, push-touch is zoom: refers to Apple Pinch and Flick gestures for zooming and scrolling, comments that Flick may require multiple flick gestures for long list. Suggests using contact area of fingertip as substitute for pressure/force sensitive digitizer touchscreen.
Sales brochure for MobilePoint QuikQuote real-time quotation application running on Tablet PC: pen-computing-based sales/form application. See also PenPoint.
Addresses targeting accuracy (fat-finger, occlusion, parallax) on touchscreen by selecting all widgets/objects under area of touch (since finger touch contacts an area, not a single pixel), and user moving to activate a particular function: Sliding widget, multi-function widget. Gives example of moving down a scroll list for "History", then flicking a stroke left or right to move forward in history. Click sound on activation (audible highlight) and also visual highlight. Size of targets puts severe limitations on design of touchscreen UIs. Compare with Apple slide-to-unlock?
Digital watermarking, using abstract (watermark) of data object as ID of the object in a database. Abstract based specifically on perceptual qualities of the object.
Cites to 7,346,472. Digital watermarking of audio/visual signals. Compare abstract/analysis of a given signal with stored abstracts of reference signals to find a match.
Motioncomputing Tablet PC: Definition of single-letter "gestures" using side-button on stylus: appears functionally identical to "recognition macros" of Pencept PenPad 320 series product circa 1985, and similar feature from CIC HandWriter from about the same time. Press side barrel button on stylus, write one letter or symbol, release side button. Pre-defined keyboard input is invoked. LE 1700.
Motion Computing F5 Tablet PC, XFR D630 : Ruggedized Tablet PC. Also competing ruggedized tougher-tablet market from Samsung, Sony, Dell Latitude XT convertible tablet. Mentions patent infringement lawsuit from Typhoon Touch.
Tablet PC User's guide: mostly hardware features including SIM/phone card slot. Press-and-hold / tap or touch and drag touchscreen, or use Function button and tap.
User guide for Tablet PC running Windows 7 (not Windows Tablet PC/XP). For Arbeiten mit dem Stift (Stylus) has long-hold and barrel button for emulating right-mouse-click. Schreibblock (input panel) has both handwriting are input and simulated/virtual on-screen keyboard. Option to set barrel button to be erase (compare with Freestyle pen) instead of right-mouse-button click simulation.
Blog posting of testing of touchscreens and multi-touch surfaces: mentions erratic behavior at light touch, edge problems. Features bad behaviors/errors for which Moto touchscreens tend to do well. compare with other papers on digitizer effects and user Interface by Phillips and Ward.
User guide for Droid 2 Android touchscreen cell phone / smart phone with stylus and finger gestures: touch (tap), touch & hold, hold followed by drag, scrolling. Slide out physical keyboard, no handwriting recognition.
Webtop: Desktop extension to Android PDAs and Smartphones: when using external keyboard and monitor (Lapdock station), displays Firefox 3 window executes Android applications within browser: use smartphone as portable PC with external display/keyboard. Filemanager for accessing internal files of smartphone. Electronic file contains additional materials.
WII-like game controller using accelerometers and magnetic north, rather than infra-red optical digitizer for sensing hand gestures and position. Compare to accelerometer pen from Hew Crane at SRI.
Mouse gestures for Mozilla / Firefox browser, Thunderbird Mail, and Chatzilla. Gesture recognition apparently based on changes in direction. Certain gestures involve right mouse button, mouse wheel.
Anthology of tabletop/touchscreen displays, technologies, applications. Sections Under Tabletops, On and Above Tabletops, Around and Beyond Tabletops (3D)
Microsoft Project project planning tutorial
CNN using multi-touch screens from 2006 TED conference. Apple MacBook with Multi-Touch touchpad using two-finger gestures similar to Apple iPod.
Several announcements of multi-touch gestures in Apple computer products, touchpads, gestures supported in Firefox 3.1: pinch, two-finger scroll by page or window, swipe/flick, mentions fewer number of gestures in Safari web browser.
Example WPF code for multi-touch demo on Windows Vista using Multitouch.Framework.WPF.Controls.Windows from http://schema.multitouch.com
Review of use of iPad touchscreen tablet at Mercedez-Benz Financial: customers sign agreements using iPad, states that development was less difficult than iPhone because display size allowed application to be built as more ordinary web/browser application.
Proposed tablet/gesture UI user interface for a Chrome OS tablet UI: mentions dwell/hover gesture for contextual actions
Haptic/Tactile input device: Hand-held object with various ribbed textures on the surface and piezo microphone inside: rubbing or stroking a finger across the surface causes vibrations characteristic of the ribbed texture. Control of zoom, volume, position by stroking different seconds.
Additional materials in file: "M" gesture, double-tap with finger gesture to turn UI on/off (for carrying pen computer so that cursor does not move)
Capacitive method for measuring/detecting the position of the user's hand holding a mobile phone. Combines two sensors: one using the antenna, the other a dual-electrode (transcapacitance). The more of the user's hand is around the cel phone, the more the capacitance change. Purpose is to transmit stronger signal when user is covering most of the antenna, but otherwise spare the battery.
Textbook/tutorial from open-source hardware project for multi-touch (optical: rear diffusion) hardware. Discusses open-source Gesture widget and software libraries, such as X12, GDML Gesture Definition Markup Language, NUIFrame, Graffiti (not the Palm unistroke library), AME Arts, Media and Engineering Patterns Library, other gesture frameworks.
Apple selling multiple stylus products for iPhone: RadTech. Stylus made of conductive material for Apple capacitive touchscreen.
N-trig digitizer tablet, senses stylus and touch/multi-touch, electrostatic (capacitive) stylus, mentions unintentional touch / hand-rejection (palm rejection)
Additional materials in file: "M" gesture, double-tap with finger gesture to turn UI on/off (for carrying pen computer so that cursor does not move)
Multi-modal (haptic?) interface by Ken Hinckley at Microsoft, using both pen and hand. One example is using hand to hold a virtual object stationary on an active surface, and a pen/stylus to tear of a virtual page.
Re-designing of web pages for touch input, specifically for the iPad. Cites to Internation children's Digital Library re-design, multi-lingual (pictographic/iconic?) user interfaces was particularly amenable to touch input.
Notetaker application for apple iPad. Appears similar to DayTimer product by Slate in the early 1990s for PenPoint and for Pen Windows: follows paradigm of ink-as-datatype, editing of electronic ink, as in van Raamsdonk
iPhone 4 cell phone / tablet contains gyroscope in addition to accelerometer, identifies as InvenSens ITG-3200 or STMicro L3G4200D three-axis MEMS gyro.
Materials relating to two-factor authentication products (exchange of PK signatures) for iPhones, Java-enabled phones and Blackberry mobile devices, based on patent 5,511,122, licensed from Naval Research Laboratory.
Suma, by Cambridge Consultants, squeezable mouse-like input device for three-dimensional input. Article states light-sensitive sensors for pressure/touch, ability to control two points of view simultaneously with one hand.
Pseudo-transparency: on small screen, touch screen is on rear of display so that finger does not occlude view of display -- show simulated finger on display when user touches back of display for input. Earlier technology was called Lucid Touch.
software package for measuring performance of tablet digitizers: manual procedures for testing linearly by using diagonal lines, but does not mention velocity-induced errors (compare with Carau). Also Testing digitizer/Mouse: resolution, etc.
software package for measuring performance of tablet digitizers: missing coordinates, missing points, non-linearity. Part of offering to for measurements for neuromotor analysis of handwriting motion.
VibeTonz vibrotactile haptic/vibration feedback for touch interfaces and touchscreens: VibeTonz is a hardware actuator component for PDAs and devices similar to Apple iPhone
Fat fingers / touchscreen accuracy: Place touch pad on back side of hand-held touch-screen device, so that finger tip does not obstruct/obscure view of display and where user is touching
Improve accuracy of touch-screen input by sensing orientation of finger: experimental system to demonstrate used fingerprint scanner to determine orientation of finger
Review of haptics technologies for touchscreens: vibration to give the illusion that a button protrudes from the service, lateral vibration to simulate the haptic sensation of a sharp edge, high frequency vibrations to simulate a slippery surface, force-feedback systems
Nokia demonstration system of turning blocks of ice into translucent optical touch screen (based on reflected light where hands come into contact), displaying virtual flames where touched.
Microsoft LightSpace prototype: 3D optical/Kinect sensing of hand gestures including picking up objects for interactive slide show presentations on multiple was. Mentions human factors concerns (presentation technology is distracting to audience). Mentions multi-touch.
Infrared optical digitizer multi-touch flat-panel displays: notes that no optical coating is required on display. Sensors mount at corners of front surface of rectangular glass assembly.
DRM usage rights management, rights transferred from a rights supplier to a rights consumer, with some rights derived from a meta-right / meta-right is the right to receive certain rights. See ContentGuard.
Early usability study on iPad: apps work better than web pages: read-tap asymmetry -- need larger targets / touch areas, most default text (for links) is too small to touch.
Nintendo DS hand-held game machine with digitizing tablet: regular display, plus touchscreen instead of keyboard. Color displays. Built-in still camera, microphone. PictoChat collaborative drawing program over LAN. Internet connection via AOSS: refers to compatibility problems (access point settings changed).
Dedicated pen computer with MP3 and auto map software, open-source (Linux). Touchscreen digitizer.
Review of S60 No-touch / touchless electrostatic touch UI device from Nokia. Allows in-air three-dimensional 3D gestures. Compared to "Minority Report" film. Due to patent conflicts with Apple Multi-Touch (see Fingerworks), based on Active Matrix LCD with integrated optical touch screen. Gestures include clockwise and counter-clockwise circles, X mark for delete, check mark, lasso / enclosed curve, two finger pinch and expand for zoom.
Review of Nokia publication on in-air gestures, compared with film "Minority Report": clockwise and counter-clockwise circular rotations, pinch and zoom, stationary pointing, X, check mark, enclosed curve / lasso to select. See also Nokia S60
Nokia touchscreen smartphone, GPS, Symbian operating system
Nokia N8 touchscreen cellphone with network applications. Physical slide switch to unlock. Includes haptic (vibration) feedback as confirmation on some touch inputs.
Folder with information on Nokia and Microsoft Windows 7 phone OS
Nokia Smartphone, USB peripheral port (not just data port), FM radio, video, stereo microphones
Nokia touchscreen smartphone, GPS, Symbian operating system
Nokia touchscreen smartphone, GPS, Symbian operating system
User interface style/specification for Nokia Symbian touchscreen operating system, version 1.0. Includes accelerator, other sensors. Section on S^3 with touch: Finger-usable user interface (touchscreen). Does not use term "gesture". Minimum finger target size is 7x7mm plus gaps, 8x8mm for thumb. Suggest activate on life / touch release. Long key press (press-and-hold). Recommends against double-tap in some contexts, because of confusion with inadvertent touches. Tactile feedback / tacicons. Limited to 3/4 icons in touch toolbars.
Open source development system (hardware and software) for multi-touch technology, includes projection of images on reflective surface. Also mentions CUBIT multitouch system for rapid implementation of multitouch projects: ofxTouch API for C++ OpenFrameworks on various platforms.
File contains additional materials: Zaagtech multi-touch kit
NMS-5000, NMS-SUNPAD, Tablet PC, Windows Mobile, Windows CD, Windows XP Embedded. touch-screen based pen computers and PDA's. Licensee of Typhoon Technologies. Market is industrial uses, data collection. Integrated bar code, camera, magnetic stripe reader. Touch screen with palm rejection -- digitizer technology not stated, says resistive touch. Keyboard usable while user wearing heavy gloves. Compare with products by Leo Shpiz for military market.
History of barcode patterns: circular bar-code invented 1948 Silver and Woodland. Barcode to track physical items / invoices 1981.
Patent 5,825,352 (see) asserted by Elan Microelectronics, 1996 concerning multitouch, including determination of local maxima for fingers. Compare with Rekimoto?
Server receives GPS and other data from a personal device, displays location on a web homepage accessible to selected users.
Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, based on preferences in a user profile.
Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, receives a user touch input, on a point of interest, establishes communications to it.
Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back map with points of interest, computes a route based on traffic and/or weather.
for a hand-held device with a touchscreen, server requests traffic data, and computes a route to avoid traffic problems.
Server responds to GPS/routing request from a cellphone with GPS and touchscreen, sends back route with areas to avoid and points of interest.
Tool bar re-configures automatically, changing/re-arranging toolbar menu items depending on what tool user has selected and is using most frequently: file contains press reports on patent assertion against Microsoft. See also www.patenthawk.com (filed under Odom): http://patenthawk.com
computer login using user's choice of multiple input devices: mouse, keyboard etc. and multiple types of input: e.g. actual password, keystroke timing, etc.
Patent search firm (Gary Odom), see notes on Microsoft, Gary Odom correspondence
Reports and commentary Gary Odom / Patent Hawk patent infringement lawsuit vs. Microsoft, Patent Hawk firm had prior working relationship with Microsoft. "Did Patent Hawk Violate Contracts In Suing Microsoft"
computer login using user's choice of multiple input devices: mouse, keyboard etc. and multiple types of input: e.g. actual password, keystroke timing, etc.
User-interface on touchscreen (or near-touch) for document dragging: when part of document is dragged, expose another part of the document: also describes "bounce" at end of scroll: Compare with Alternate Reality Kit? Twist gesture to rotate an object or screen.
Re-exam document also in file. Bounce-list at end of scrolling on Apple iPod/iPad
Scrolling/flick gesture: faster gesture results in faster scrolling: flick-to-scroll.
multi-touch user interfaces: multi-touch gestures used to adjust parameters e.g. rotate a virtual knob.
On touch-screen user interface, target a touch to the nearest valid UI object based on position of centroid of touch: example is small keyboard.
Compare with Ink targeting, gravity? Compare with Penpoint and W4PC targeting on ink.
ITO and organic transparent conductive inks for screenprinting: intended for touchscreen electrode market
On-line/dynamic character recognition, but using visual features (OCR-like?) rather than dynamic or time-sequence recognition. Mentions problems of overlapping/touching sketch symbols and gestures. States that recognizer is rotationally invariant (so what about M and N, E and 3, S and Z, etc.?) Does not describe recognizing handwriting: electrical symbols, powerpoint symbols, etc.
File contains additional references for sketch recognition for mechanical-engineering, to 2003
OCR-type recognition, mentions down-sampling of images prior to classifier. See other Ouyang papers. States superior to Graffiti, which requires constrained writing. (Unistroke)
Recognition of clusters of squiggles and angles in a sketch as an indication that the strokes should be parsed together in one object
Trainable sketch-recognition system, using both dynamic (time sequence and stroke direction) information and static visual appearance (but not from scanner). Example is molecular-sketch and simple circuit-diagram systems. But states that input to existing system only done for molecular sketch -- no actual recognition, only prettifying?
Definitions of select terms. gesture (with motion), pointer (computer address; screen symbol), stylus (pointer or finger)
Compendium of over 100 handheld devices up to 2008 running Palm OS 5.0 on PalmPilot type models, with hardware specifications
User guide (in German) for Palm PRE hand-held touchscreen PDA
resistive touchscreen, optically bonded micro sheet of glass on top for abrasion/scratch resistance. Digitizer is two ITO sheets (coatings), spacer dots with air gap between. Activation force for touchscreen (standard) < 100 grams (but does not give diameter?)
Power-management design in computing devices. Cell-phones packing more and more functionality mid-2000s, making them increasingly similar to general purpose programmable devices, but within tight power envelope.
Center areas on handheld touchscreen smartphone targeted more accurately than at edges: half (49%) of users hold phone in one hand and use thumb. 4mm targets quite small. Numerous citations. Large target button size more accurate, greater user satisfaction (ease of use)
Organic user interfaces / tangible user interfaces: combination of physical input and haptic/physical feedback, not specific to gesture input. Reference to three-dimensional mechanical digitizers.
press reports on Quickie handwriting notes using what appears to be the Anoto pen, with an RFID tag (?) embedded in the paper. Also Girton Labs Ixp-Note, post-it note sized small digitizer, form factor like a postit note. Prototype of DPF711K, touch-screen small size with a post-it note sized display from Audiovox, along with Audiovox Digital Message Center. Sony E- Series display/touchscreen digitizer for taking notes.
touchscreens and digitizers using indium tin oxide ITO: indium is scarce resource, alternatives are antimony tin oxide, carbon nanotube, flexible polymer PEDOT
Sketch recognition implementing larger and finer set of primitives (e.g. circles and ellipses), suitable for defining sketch (picture) grammars for multiple domains/applications. Presents user with multiple interpretations, which user can correct by sketching additional lines, etc.
Study of sketch recognition based on select set of primitives, proposes a standardized test data repository "SOUSA". Motion-based recognition (gestures) may be distinct though same visual shape (e.g. single-stroke 4 drawn in opposite direction), beautifying of sketches (sketch-primitive recognition). Cites to various sketch-recognition / shape recognition experimental systems (ChemPad, biological cell recognition), 3D tracking, Paleosketch.
CityWall public multi-touch display in Helsinki: social behaviors among users coordinate use.
Electronic gesture data glove, approximately 30 gestures. "Hot-key touch points": Linear (1D position) touch sensors in glove fingers, palm, thumb for particular hand gestures with fingers, not 3D proximity sensing. Combination of touches mapped to keyboard entry for faster gaming.
Electronic gesture data glove (touch gestures of hand only), approximately 30 gestures, indented for gaming market. File contains additional materials
Accuracy of thumb-tapping one-hand use of touchscreen smartphone: edge and corners were more accurate, but users preferred targets in center of screen as more comfortable. Scatter plots showing errors for 6.8mm target size. 9.6mm target size much better.
Text file document read with scrollbar highlighting parts of text in the scrollbar
In-air mouse, using IR LED pointer tracked by camera. Test application is sketch recognition running on Tablet PC.
compare with Smart Desk / Weller: determine "touchscreen" position using webcam and edge detection to determine location of fingertip.
Durham University in UK, Dr. Liz Burd, Director of Active Learning in Computing: experimental program using multi-touch desk surfaces in education: SynergyNet project of the Technology-enhanced Learning Research Group
Durham University UK experimental SynergyNet project using multi-touch tablet desks for grade-school education
File on Plastic Logic Reader, electronic ink display with touch-sensitive touchscreen interface: swipe gesture to turn pages.
API to set parameters for scroll bounce
Optical mouse in a pen-shaped barrel. Newer model includes audio recording: compare with LiveScribe and FingerSystem. Finger Note software: ink (?) annotation of screen captures for note taking -- Compare with Wang Freestyle. User manuals in English and German.
Usability tests of pen-only (Tablet PC), whiteboard with pen (Mimio pen capture bar). Notes problem of double-click with pen on same pixel (compare with PenWindows, Tablet PC). InkKit sketching application, Penmarked annotation tool. Haptic pen (PHaNTOM pen) for tactile/haptic accessibility aid for blind to write repeatable signatures: poor single-character recognition on Tablet PC (without dictionary) made audio feedback to blind (compare: JAWS) unusable to learn writing.
Game-theory Mathematical model of overcoming Peter Principle (individuals are promoted until they reach a position they are not competent at, and then stay there: thus organizations tend to have managers who are not competent at their job) by promoting people at random.
Cites also to 2010 paper with same title, Physica A 389 (2010) 467
Variety of Seiko computer/calculator watches. RC-20, circa 1985, buttonless touchscreen input with 42x32 pixel display, graphical icons
Homemade capacitive touch stylus using anti-static film for packaging electronic components (e.g. hard drive). Roll film up into a cylinder. Mentions problems of gloves on capacitive touchscreens. Stylus for more precise pointing. www.YouTube.com comments mention using silvered gum-wrapper paper, etc.
Timelines of history of personal computers, back to 1947. Windows for Pen Computing and GO/PenPoint both announced January 1991.
POS system for DOS/Windows95 and later using modified MicroTouch unMouse touch screen to replace mouse.
File contains additional information.
Bendable computer: transparent flexible 2D touchscreen sensors combined with flexible organic display. Bending use for up/down zoom on map application, etc.
Industry-standard driver from www.pointing.com/LCS-Telegraphics to allow all (DOS) applications to use pointing devices (mouse/tablet/touchscreen/joystick/etc.) interchangeably. Wintab/Windows/Pen Windows: mentions tablet sharing, multiple application accessing tablet at once with different services: different tablet context objects for different (concurrent) applications, tablet packets with data for a high report rate. Driver supports command-input area, WT_PACKET window messages.
Also exists in version from 1993 (original), later Wacom version (after LCS/Shumer lawsuit) with minor updates
BWS Braille Window System using BrailleDis 9000 120x60 pin-matrix display. Display divided into Braille Window portions: Window title region, view type region, tactile scrollbar, Braille text output, command output, etc. Shift/function key to enable multitouch gestures on touch-sensitive display, so as not to confuse with touch reading of display. Circular zoom gestures, pinch/spread used for dividing regions. Minimap "world view". Cites to problems of screen readers only reading top-level Window, confusions of pop-up windows.
WII-like 3D gesture sensor: combines structured (patterned) IR light source with IR video sensors for depth sensing, 4 external audio sources with 2D (stereo) microphone sensors, color video.
Borland SideKick phone dialer dials a phone number found anywhere on the DOS screen: it didn't need to be in SideKick mode
Untethered electromagnetic digitizer, multiple tablets may be used wirelessly to annotate on a common shared whiteboard display
Apple iPhone touch-sensitive digitizer does not work for women with long fingernails, or with gloves. Described tongue-in-check as accessibility problem.
Nokia: In-air finger and hand gestures in front of a PDA or cell phone: describes a sonic/sonar digitizer for detecting hand gestures close to the device, but also mentions electrostatic/capacitive and optical hardware, generally relates to use of in-air hand and finger gestures, not the hardware. Mentioned in other references as multi-touch.
Additional materials in file. Vibrating touch screen puts Braille at the fingertips: rhythmic haptic encoding of braille characters similar to Morse code; haptic/tactile display of braille on a touchscreen/multi-touch surface. Spatial encoding of simulated braille as stylus moved on touchscreen. Rhythm method worked better. Prototype implemented on Nokia 770 Internet Tablet with added piezoelectric actuator.
Disassembly of Apple iPad: shows separate front-mounted digitizer with transparent screen.
SAGE system: information on light gun
Special issue on organic/graspable/haptic/tangible user interfaces. Deformable/rollable input devices, as SmartSkin, PreSense (pressure-sensing touchscreen surface -- see elastomeric touchscreens?). Examples include HoloWall -- compare with Wellner?
Optical 3D finger tracking system (multi-touch), uses shadows from synchronized multiple light sources (overhead above tablet) at known locations, shadows on known flat surface. High-speed video camera 200 fps. Away from tablet: subtract known background from image, look for bright spots of fingertips. Compare with SmartDesk.
"passive" 3D/2D tactile feedback, without direct position sensing. Instead, just an optical receiver on a finger-mounted sensor/vibrator that picks up time-modulated light (or infrared) projected by rays on a location on a surface, and vibrates in response to the light beam projected to that particular spot. 3D by using two light sources.
Internet posting of review of multi-touch technology: Perceptive Pixel, Apple, Microsoft Surface (infrared), Apple iPhone
Tactile/Haptic feedback on a touchscreen in an Automotive map user-interface: also tracks finger in proximity, or when not touching firmly. Application of Buxton 3-state model: Out-Of-Range on proximity, active tracking (a.k.a. passive tracking, and pressed.
Antiglare coating for touchscreens: silica particles in inorganic polymer matrix.
Using flexible display (polymer-dispersed imaging layer), put display in front of pressure-sensitive touchscreen (spacing dots, resistive), so that touchscreen does not reduce visibility of display. Says resistive-film touchscreens with spacer dots have 75% transparency, ITO capacitive typically 90%. Touchscreen may be opaque.
Browsing or scrolling a document (such as a map) using a virtual semi-transparent overlay over the image of the document: like moving a magnifying lens?
Convert HTML layout to resolution-independent vector display-list form, for better rendering on small (handheld/smartphone) displays.
Force/pressure sensitive touchpad/tablet using elastomeric material (conductive rubber: force decreases resistance) between perpendicular matrix layers of conductor lines. Effect is grid of force/pressure FSR force-sensitive-resister points whose sensing areas overlap and fall off with distance. Interpolation to get higher resolution of single points, or detect multi-touch points.
Multi-touch touchpad using force variable resistors with interpolation (IFSR) to get position resolution finer than minimum discrimination distance for multiple touches (like human skin). Described as touch imaging technology.
Multi-touch touchpad using force variable resistors with interpolation (IFSR) to get position resolution finer than minimum discrimination distance for multiple touches (like human skin). Described as touch imaging technology. Distinguishes true pressure (force) sensing from contact area of iPhone touchscreen. Pressure (force) readily used to adjust multiple slider values, translate/rotate/scale in 3D three dimensions. Automatic/adaptive trade-off of resolution vs. scan rate by using every other grid line: sleep mode at very low power consumption. Two layer grid with FSR force-sensing resistant paint layer between: correction matrix for point slightly off exact crossing point of layers. Distinguishes between touch sensing (surface area of contact) and actual pressure sensing, can read pressure profile / distribution.
Haptic force feedback transducer/actuator mounted on back surface of touch surface or tablet, which is mounted on springs to allow motion.
Infrared optical multi-touch touchtable using infra-red IR LEDs, low resolution camera (base on Community Core Vision Framework). Infrared-emitter signet ring generates detectable pseudo-random signal, can be use to identify user's touches, nearby touches assume to be from that user's hand (multi-user). User identification described as security feature on multi-user touch tables. Describes fingerprint authentication as easily copied, especially without "livelihood" detection.
Gesture based on rolling of thumb instead of sliding of thumb. Six microroll gestures (four directions, two circular) four slide gestures (direction) four swipe gestures (direction) two diagonal rubbing (rubout) gestures. Instead of marking menus, etc.
How to construct a multitouch FTIR optical digitizer using easily-available parts at home in your spare time. Acrylic sheets with webcam camera underneath. Recommends spraying surface with silicone lubricant to get better detection (better optical contact and reflection). Includes video showing both operation and construction.
Single camera, determine shape of an an object form multiple views. Object has photogrammetrically evaluatable marks (targets). Uses marks on floor (surface) at known locations, object at known position, so that no scale reference object needed. Example is special pants/trousers/clothing with marks on human body. Compare to Chen 2009 and structure-through-motion.
General tutorial book on gestural interfaces including motion gestures such as Wii, hand-gestures, eye-winks, body motions, multi-touch from Fingerworks and iPhone. Mentions "the Clapper" as a gestural interface with indirect manipulation. First multi-touch device in 1982. Refers to gestures as motions of the human body.
Somewhat sketchy history of touchscreen devices, does mention Mehta 1982 optical touchscreen (multi-touch), Rekimoto multi-touch.
Samsung Galaxy Tab mobile tablet device, includes Flash, android operating system. File contains additional material.
Touchscreen targeting accuracy worse while walking with smartphone, and cognitive load increased also.
Searching through electronic-ink (tablet) documents based on shape matching, not text recognition. Citations to signature verification/recognition. Compare with sketch recognition.
Use input of smart phone such as iPhone (inertial input, keys, etc.) long with input from touch surface (simulated with video detection) as user input.
Opinion piece on multi-touch having a 25 years history before Jeff Han and the iPhone from Apple, many designs not taking into account 25 years of HCI lessons learned (esp. Buxton)
eInstruction whiteboard system using wireless tablets, versus importer Qomo Hitevision of wireless tablets, contributory infringement
overview article on touchscreen products with short notes on multi-touch history, apparently taken from www.buxton.com. States that "flick" gesture first implemented in 1999: compare with PenPoint flick gestures in 1991? First multi-touch touchscreen by Boie at Bell Labs 1985, using capacitive array of touch sensors over CRT display. IBM Watson smartphone 1992.
Organic user interfaces / tangible user interfaces: Gummi flexible display, can be bent as a form of gesture input (for zoom) combined with touchpad / touchscreen sensing. Bending force determines how fast it zooms.
Duke University project: PhonePoint, use inertial sensors in smartphone to recognize writing/gestures in the air. Proposed for handwriting/text recognition input, instead of on touchscreen.
User interface with physical keyboard on rear of hand-held (two hands) display: compare with LucidTouch
Product information on hand input devices with haptic force feedback: mechanical linkages. Force/touch effects include constant/gravity, viscosity, 3D friction, spring, friction, stiffness, damping, front/back faces of virtual objects.
3D modeling software for displaying 3D models using Phantom 6DOF haptic feedback electromechanical stylus.
Electronic file contains additional materials.
Essay on multi-touch interfaces, gesture with iPhone. Jeff Han cited as multi-touch. GestureTek video gesture recognition -- GestureTek in business since late 1980s.
ShapeTape flexible bend/twist sensor used for human-body motion capture. Also mentions robotic control and robotic sensing.
Power management and power design for portable devices: tablets, smartphones. Power consumption is limiting factor for functionality of portable devices operating batteries. Significant trend is to reduce size of devices.
Multitouch touchpanel on back/rear of PDA/iphone so that display is not obscured. Multi-touch gestures (e.g. two-finger "grab"), fingertip positions displayed with small cursors on front display. Also double-side input multi-touch gestures on both sides at same time.
Review of published patent application relating to iPhone/Fingerworks touch gestures
Prototype multitouch tactile sensor skin, rectangular matrix of (distinct) RFID sensor elements in flexible insulating material between two conductive sheets, RFID sensors powered by microwaves. When enough force applied to sensor elements, mechanical switch on RFID closes, RFID sends signal (in the RFID way). Because RFID elements not affixed to sheets, no points of stress to break.
Somewhat more advanced textbook topics on computer graphics: alpha mask/channel == transparency mask (image blending). Textures, ray-tracing, etc. but not bitblt or simple masking.
University textbook on computer-human interaction - Earlier editions 1987, 1998. Discussion of direct manipulation, with Microsoft Office as an example -- direct manipulation because you have WYSIWYG behavior in the document, not because it is a tangible user interface. One chapter on input devices talks about pointing/digitizer input, Graffiti, some use of gestures. Earlier editions go back to 1986.
Anti-virus: delete file if checksum of file matches checksum of known bad file, if file has bad length, type of file (by name extension) compared with contents, set of identically-sized files. Also separate checksums for portions of files.
Upper-level course textbook on operating systems design and architecture: Unix (Mach, UNIX BSD), Windows (XT/2000, 64-bit). Historical notes on Multics protection domains / ring system, ACLs and capabilities; demand paging 1960 on B5500 and Atlas.
pressure/force-sensitive (?) resistive multi-touch touch sensor: Multiple Simultaneous Touch Activation, user presses harder to activate, stylus recognized by smaller contact area. microTILE electronics for reference design.
Electronic file contains additional information.
Application of Anoto-like optical pen with paper pattern (cites to Anoto): processor in stylus does handwriting recognition, shows text in small display in pen barrel. Includes use of stylus orientation/angle.
Touchpad as relative motion device, no isomorphism from screen to touchpad. Edge may have scrollbar area similar to wheel on mouse. Called trackball by Apple computer, introduced in 1994 on Powerbook 500. Vendors synaptics, Alps Electric, Cirque.
Multi-touch tablet smart table, rear-camera with rear projection: DViT digital vision touch. Includes SDK toolkit for Windows.
SMART board interactive whiteboard digitizers, with digital ink, multi-touch: Prospectus for public offering, educational applications
Definition Input Focus. The input focus specifies the window that receives keyboard input.
multi-touch mouse has no wheels or buttons: uses a touch surface.
Review of VoiceOver for iPhone: refers to gesture input on touchscreens has being the solution for visually-impaired/blind computer access
Mentions skeye.pad webpad tablet computer at CeBIT 2000 about the time of the Tablet PC introduction by Microsoft: Wireless connection, Webpanel, touchscreen interface with browser.
Formerly known as JazzMutant / Lemur. Resistive-grid multi-touch detection using X/Y addressable nodes in a grid pattern, notes that it works with any touch stylus, unlike capacitive digitizer. Includes gesture-recognition software for pinch, twist, rotate, close, zoom gestures. Mentions bounce on scroll action, similar to iPhone/iPad?
Resistive-grid multi-touch detection using X/Y addressable nodes in a grid pattern, notes that it works with any touch stylus, unlike capacitive digitizer.
pressure/force-sensing multi-touch resistive touchscreen digitizer.
Capacitive multi-touch that can detect both fingers and stylus (passive) at same time: apparently by size of contact/detection area?
DRM usage rights management, usage rights control what rendering/use is granted to a document or object, discrete DRM control for discrete parts of document/object, DRM rights stored separately from managed object. Compare with Benson certificates? See ContentGuard.
Filter Email for virus detection by executing program on another computer to check before sending to email recipient: send recipient converted file (e.g. PDF) instead. Virus is assumed to be executable code.
Review of 3D input and 3D display in consumer-level systems: Perspective and occlusion more important than stereoscopic display. 2D touchscreens better than existing 3D input devices, mostly because of accuracy and precision of the device. Additional references on 3D input with similar devices.
Presentation of ideas for non-visual (audio, tactile) feedback and touchscreen gestures using BrailleDis 9000 tactile display, and generally for a user interface for the blind with tactile graphical output. BrailleDis (HyperBraille) returns touch-intensity data from a coordinate matrix (capacitive?).
web tablets, open source: Neofonie WePad with Android OS, TouchBook, iFreeTablet, MSI Taiwanese tablet with Android, Asus / Asusteck Eee Pad, rumored Google Tablet.
Using feature points/characteristics of a gesture in carrying out the operation of the gesture: characterizes this as performing multiple operations. Examples include endpoints and apex point of a checkmark. Compare with using single or multiple hotpoints of a gesture?
Synaptics capacitive-grid touchpad hardware controllers: Devices support MultiFinger (multi-touch) detection (report count of fingers), Enhanced gestures (multi-finger pinch and zoom).
Synaptics transparent capacitive-grid touchpad hardware controllers: Devices support MultiFinger (multi-touch) detection (report count of fingers), Gesture recognition in controller (compare with Pencept recognition macros?). Multi-finger pinch and zoom).
Synaptics: Multi-touch/multi-finger gestures in touchpad driver for multi-touch touchpad. Compare with Pencept recognition macros?
Synaptics capacitive-grid touchpad hardware controllers: Devices support NumberOfFingers detection (report count of fingers) with additional position registers for each finger position, TouchShapes contoured special areas on tablet.
Synaptics driver/utility for Windows to enable multi-touch touchpad gestures in driver. One example: two fingers indicates scrolling operation of middle wheel on Microsoft mouse.
Additional gestures for iPhone: Pinch-in/out/mix, Rotate-(anti)clockwise, 3Tap triple-tap, etc.
History of Newton development: Walter Smith developer of NewtonScript object-oriented programming language. Original price for Newton Toolkit was $1000. Developers of Newton Toolkit: Norberto Menendez, Ben Sharpe, Peter Potrebic. Appears to be copy of Wikipedia article on Newton (platform).
Noise reduction in SAW surface acoustic wave touchscreen in cellphone. Microwave radio waves from cellphones non-negligible noise. Disable touchscreen during noise spikes.
Upper-level textbook operating system design, case studies of Linux and Windows Vista (7). ACLs access control lists stored by row, or by column. Computer security also discusses logic bombs, insider attacks, buffer overflow attacks, format string, return to libc, malware (ransomware) etc.
Personal record of expert-witness analysis concerning Xerox Unistroke handwriting recognition patent in Palm. v Vexor patent suit.
Light-pen-like device using projected barcode pattern, projector uses switchable diffuse/clear surface that can project onto the surface and also see objects above the surface. Credits Doug Englebart with first GUI graphical user interface.
Cites to this bibliography and to this collection by name "which made my related works section both more interesting and immensely more time-consuming", with thanks to Jean Renard Ward
Compendium of URL links to pen computing, lists this reference. Terminology: uses Diagram Recognition for Sketch Recognition, includes several Chinese commercial handwriting recognition / pen-computing links, multi-touch, early penpoint, etc.
Synaptics ClickPad 3 touchpad with ACM accidental contact mitigation. Shows configuration for optional multifinger (multi-touch) gesture: two-finger scrolling, pinch zoom, rotating, three-finger press / flick, four=-finger flick, scrolling (at edge?), tapping, clicking (separate?), etc.
Disassembly / tear-down of Palm Pilot PDA. Only shows touchscreen/display as a single unit.
Report on interview with Steve Jobs: Tablet PC: had pen/touchscreen, but expense, battery life, weight and OS of a PC. iPad tablet development first, then was put on iPhone. States: Finger touch not as accurate as stylus, problem on pen PCs with finger.
SpaceNavigator 3-D mouse, www.3dconnexion.com -- six-degrees-of-freedom joystick-like input device.
http://nortd.com TouchKit -- open-source multitouch table input tablet, apparently optical
In-air visual/camera tracking of hand signs / finger motion to implement in-air typing. Compare to Skinput related references, ASL gesture recognition.
Review of some touch-screen and haptic technologies (seems to confuse "haptic" with "touch-input"). Roope Takala, Haptikos project at Nokia. Christophe Ramstein, Immersion corporation. Gary Todd, Virtual Veins. Mark Williams, Warwick Manufacturing Group (WMG), UK. iPhone.
See other Thimbleby refs for handwriting recognition mathematical calculator on touchscreen/tablet. One project is 2D handwriting expression calculators, always displays results of current marks, even when - later changes to + with additional stroke, square-root radicals added or changes, etc. Second project is Lineform, vector-drawing application.
Press release for SpaceNavigator, three-dimensional mouse pointing device, using force/pressure for up and down, also rotation.
At end of drag (select) gesture, user tilts/angles stylus to select from pie menu. Combine selection and gesture/command in one input.
Virtual Keyboard and other devices simulating actual devices by generating system messages same as drivers (e.g. keyboard driver) for physical input devices. SIP Soft Input Panel. Compare with PenCept, The Guide (Wang), PenWindows, other systems with simulated keyboards, including Ken from Wang, Freestyle, Doster/Oed
Five-wire surface capacitive touchscreen: shows wire connection diagram for five-wire interface. For Tablet PCs (Microsoft)
Stantum multi-touch resistive grid touchscreen, description from 2007 patent application. Resistive grid with spacer dots, other rows and columns are set to high-impedance state.
SiMa Systems multi-touch resistive grid touchscreen, MT/MF multi-touch multi-force. Includes "Dynamic home row" virtual keyboard/typing. Comparison with Stantum PMatrix, Peratech QTC quantum tunnelling composite and FSR force sensing resistor.
Commercial company with touch/pressure sensitive digitizer using IFSR Force-sensitive resistor ink. Electronic file contains additional information on multi-touch resistive technology. Shows heat-map style image of hand on tablet surface, using actual force sensing / pressure sensing.
Detailed description of IFSR (ink force-sensing resistor) multi-touch touchscreen: Stantum / Touchco
Student project on touch-sensitive interface, using in part a Wii hardware digitizer and Flash (software) on a MAC.
Drop-down list shows values wider than the visible length of the field, width of drop-down list determined programmatically to match longest entry in list.
Drop-down list shows two-part values: does first match on user input as user types with either of the two values e.g. Code 04: Non-Warranty Service.
Linux administration manual: covers syntax and RegEx expressions in Linus ACLs access control lists.
Licensing company for two patents (see CloughWA) relating to application generators for tablet/pen computers. Company originally formed to distribute MP3 files. Press reports on settlements with Electrovaya, Motion Computing. See also Nova Mobility (product partner). EDGAR report states that original purpose of company was to sell MP3s on-line, acquired patents from Nova Mobility Systems. See file folder for patent information.
Touchscreen UI guide for soft keyboards, target size. Buttons may be smaller than a finger if spaced -- essentially larger target than displayed image. Gives pixel units for physical target size on different devices. No reference to occlusion?
Haptic/tactile display: grid of independent tactile cells (under multi-touch or single-touch touchscreen). Cells can have any kind of haptic/tactile output: piezoelectric, micro-electro-mechanical element, thermal fluid pocket, laminar flow modulation, etc. Compare to Braille displays such as BrailleDis?
Optical sensors (e.g. CCD imaging) integrated into display panel (e.g. LED). Applications include two-way video with user eye contact directed in same direction as camera, versus separate webcam camera mounted on top. Compare to in-cell optical technologies for touchscreens.
Automatic user interface behavior of scroll, move to next field/element, or enter data using cursor keys (forward, down, etc.) for small display with small keyboard (smartphone)
Report on work on ISO 9241 standards (design guidelines) by ISO TC159/SC4/WG9 for tactile and haptic interaction. Haptics subdivided into touch/tactile/cutaneous and kinasthesis/kinaesthetic (limb position, force feedback, etc.). Extends to virtual reality.
PhD Thesis on haptic touch user interfaces. Force-feedback, mechanically simulated or optically simulated (via cursor displacements) or via animation of stickiness. Changing cursor cited to 1988 for WIMP interfaces. Dynamic cursor invented 1968 by Engelbart. References to force-feedback back to 1990.
Introduction to special articles on organic/graspable/haptic user interfaces. Credits Alan Kaye (indirectly) for inspiring the Tablet PC. Deformable/rollable input devices, such as paper that can be bend or folded.
Listing of suggested multi-touch gestures: pinch, spread, press (press and hold), press and tap (second finger), press and drag (second finger), bundle (multi-select with multiple finger touches), single and multi-finger drag, rotate. Also shape gestures X (delete), check-mark, question mark (help), cut (slash -- no paste), undo (rubout / scratch out). Hand-drag, flick/scroll.
Multi-touch mouse designs: FTIR top of mouse, infra-red camera under mouse cover, flexible capacitive grid cover on mouse (cites to Rekimoto, also to Westerman but only for multi-touch keyboard). Also sensing fingertips on surface near mouse. Compare with Apple touch-mouse?
Handheld mobile (cellphone) device using GPS and other means for determining geographic position, API picks available means/method that meets accuracy requirements specified via the API
BrailleDis 9000 (BrailleDis 7200) tactile display for visually impaired. Pin-Matrix output, multi-touch input (capacitive?). Display pixels are braille dots using piezo-electric actuators.
Study of physical aspects of hand occlusion for pen/finger input on touchscreen: problems include hidden status messages, missed previews, inefficient movements, occlusion contortion, keeping hand at neutral position (e.g. right-handed off right edge of display).
Hand occlusion for pen input: points out problems of that pen / handwriting / gesture recognition is used with legacy/existing software. Pen input faster than pointing with a mouse -- compare with observations on Windows for Pen Computing and GO/PenPoint in early 1990's.
Test of Tablet PC user interface (Microsoft Windows) with conventional mouse UI. Problems are poor precision with tapping and dragging, hand occlusion, hand instability, gorilla-arm fatigue, hand fatigue, difficulty moving to keyboard, difficulty with small targets/menus/icons, and lack of additional mouse buttons or roller. Based on Vogel PhD Thesis. Various citations to Ward/Philipps 1987 paper.
Design study of pen-computing gestures, marking in interactive mapping system: iconic gestures and semantics specific to mapping (road blocked, paramedic symbol, etc.)
Wacom Bamboo tablets: pen and multi-touch. Single finger or pen for navigation, multi-touch gestures. Multi-touch gestures implemented in controller (?): Rotate and Zoom gestures.
History of Lotus 123 Release dates: First Windows (non-DOS) version in 1991, Lotus 1.0 for Windows. Via Voice speech voice recognition added in 1998.
Release and development history of Excel spreadsheet program: cites provenance from VisiCalc available at www.bricklin.com; Lotus 1-2-3
Tutorial on integrated in-cell LCD display/touchscreen technologies: Light-sensing, capacitive-sensing, switch-sensing (resistive/voltage). Optical: problems if black display / no ambient light. States Multi-touch from 1982, pinching gestures from 1983. Asserts capacitive cannot sense stylus (?).
Tutorial on touchscreen technologies: covers multi-touch, force sensing (Vissumo). Vissumo force sensors at edges of display, not at corners. Software solution to "occlusion" problem for multi-touch.
Tutorial on touchscreen technologies. Haptic feedback. Distinguishes Touch Screens (passive) from Active Digitizers. Flexible displays. Non-multi-touch: analog resistive, Surface Acoustic Wave (SAW) Surface Capacitive (electrostatic), Traditional infrared (IR), EMR Electromagnetic Resonance, APR Acoustic Pulse Recognition, DST Dispersive Signal Technology, Force sensing (Vissumo). Multi-touch: Projected capacitive, LCD In-Cell (Optical, Switch, Capacitive), Optical, digital resistive, Waveguide infrared (RPO), Vision-base optical (e.g. includes FIR/FTIR). Figures on major vendors, market share. Alternative materials to ITO.
No treatment of electromagnetic, acoustic/sonar, or Anoto pattern, as in Feb 2010 version? Covers Wacom EMR, RPO Waveguide infrared optical: waveguide layer on top of display. Circular polarizers to reduce visual reflections. Formulae for anti-glare non-smooth hardcoat surface coatings, anti-smudge hydrophobic coating, contrast ratio, anti-Newton's-rings,
Excerpt of Tutorial on touchscreen technologies.
Continuous ITO film/sheet: analog resistive, surface capacitive.
Multi-Touch. Touch technologies with patterned ITO: analog multi-touch resistive, projected capacitive, embedded on-cell and in-cell. Manufacturing processes e.g. photolithography
Without ITO: Acoustic Pulse Recognition / APR; Dispersive Signal Technology / DST; Materials and Process 5: Etch; Surface Acoustic Wave / SAW; Waveguide Infrared / RPO; Optical; Force Sensing; Vision-Based;
Electromagnetic resonance (EMR) multitouch tablets; ITO replacement materials (redacted): sub-micron metal mesh, silver (Ag) nanowires, carbon nanotubes, magnesium hydroxide, conductive polymer
No treatment of electrostatic stylus, acoustic/sonar, or Anoto pattern?
Touchscreens with touch sensor physically integrated into display: Cites to 2007 SIGGRAPH publication. In-Cell generally projected capacitive (X/Y grid matrix), analog resistive, or digital resistive: light/optical sensors in display. One use is touchpad on laptop (not touch display) with displayable icons in touchpad.
Central server distributes "digitally signed" copies of electronic document to multiple receivers, signature as specified by sender, but trust relationships are only between sender and server, and receiver(s) and server. Compare with CertCo digital signatures?
Determine shape of contact of fingertips on table-top digitizer (FTIR optical). Vertical contact and horizontal contact touch gestures, determine direction of finger from oval shape, gestures using intersection of two pointing fingers to get active point, rocking gestures. Use of pointing to reduce occlusion problem, directional sector (radial) menus.
Skinput: acoustic digitizer mounted on arm: must learn/be trained to recognized characteristic sound of tapping on different parts of the skin surface.
Personal history of RAND Corporation. States that RAND tablet was first operational digitizing tablet, 10-inch flat surface with free-pen stylus. Later printed-circuit version was 36 inches by 36 inches. Supported by JOHNNIAC project, also Videographic System to integrate characters and graphics (video) in same display, used for (electronic ink / gesture) annotation of maps with text. Saved user actions to reconstruct image rather than saving bitmap. Dictionary (stroke order) look-up to recognize handwritten Chinese characters. BIOMOD graphical user interface to construct biological models.
First version of RAND tablet was electromechanical (reversed flatbed plotter), then woven grid of Formex wires with distinct signal on each wire: 0.1" resolution. Final version used (new at the time) PCD with 0.01" resolution, still unique signal on every conductor. Notes that users quickly adjusted to separate tablet and display. 100 data points/second, with filtering, stroke identified (handwriting recognition) of features relative positions of strokes, stroke directions, corners, end-point location, and context (1 vs I? 0 vs O?)
History of GRAIL project (later BIOMOD and CLINFO). GRAIL was automatic recognition of gestures and characters for system flowcharts. BIOMOD graphical language for construction system models.
Comparison of PenPoint and iPad: similarities include Smalltalk-type desktop, instant-on, Paragraph/phatware handwriting recognition, integrated address book, calendar, email
History of CAD/CAM history. Chapter 7: Applicon. Gesture UI of Applicon CAD/CAM system known as pattern recognition command entry/ Tablet Symbol Recognition / sketching a pattern with a stylus. User could define new patterns/gestures. Circle/lasso gesture to select and zoom, dot-horizontal and dot-vertical for dimension lines. Gestures more clumsy with a mouse when implemented on Macintosh as MacBRAVO!. Ken Ledeen (Ledeen Recognizer) at ComputerVision in 1985.
Swipe gestures by moving finger across keys on (simulated) keyboard touch-screen: compare with 1980's Japanese calculator in collection?
Broad description of Fingerworks/Apple electrostatic multi-touch touchscreen sensing and multi-touch gestures: claims are to pinch gestures for cut/paste.
Review of "slab-form" tablet PCs, including Android and rumored apple iSlate. Lenovo IdeaPad U1 running Linux with multitouch tablet as display portion of Windows PC, Plastic Logic e-reader with multi-touch capacitive digitizer and annotation, dual-screen courier tablet (Microsoft?)
Physical design of large rear-projection multi-touch interactive surface (FTIR Frustrated Total Internal Reflection: see Han). Contains review of usability of multi-touch gestures over mouse/keyboard. Describes algorithm for finding "center" of finger-touch blob for pointing.
On-screen visual/virtual keyboard for accessibility: abbreviation expansion, word prediction from adaptable dictionary, expanding quadrant selection (compare with Marking menus). Click selection, dwell selection (using both a time and a distance threshold), scanning selection (hit switch when desired item is highlighted in a sequential scan of items).
Tabletop touchpad with multi-touch sensors on both top and bottom side of table. Under-the-table can use first-contact selection rather than lift-off, since cursor fully visible, not obscured by finger/hand. Bi-manual/two hands: Under-the-table can also be used at same physical position as hand on top table, but for second display, or on same display. Also useful for simultaneous-input two-person games (e.g. Diplomacy), since hands/moves not visible under table.
General reference on mouse gestures: history first mouse gesture (not using a pre-defined shape) was "drag" by Apple to replace a dedicated "move" button on the mouse. Cites lack of feedback to show that the gesture was recognized correctly (showing actual gesture: compare with GO/Agulnick), and visibility to make gestures easy to learn. Cites to Buxton for awkwardness of mouse as a gesture device compared to stylus and tablet digitizer.
Post-WIMP: defines as paper sheet user interface, contrasted to WIMP Windows Icons, Menus, Pointing device user interface. See Andries van Dam
Mentions Google chrome browser scroll bar which places highlights/marks in trough of vertical scroll bar to indicate places where term was found that user was looking for in a web page
Report on Wang Laboratories Freestyle electronic ink/voice whiteboarding and communication desktop system using Hitachi digitizer: has link to Freestyle demonstration video
WIMP user interface: window, icon, menu, pointing device: mentions post-WIMP (Andries van Dam)
Same as AT&T Tilt: Windows Mobile Pocket PC smartphone: resistive touchscreen, full slide-out / tilt physical keyboard on side. File also contains separate mobile-review article with photos of applications and features.
WebOS Palm smartphone. Capacitive touchscreen and physical QWERTY keyboard
Multitouch smartphone, first Palm PDA with webOS Linux operating system. Second "gesture area" touchpad. Automatic screen rotation.
Handwriting movement analysis: Cites to Teulings and Jean Renard Ward. Mentions early history of pen-input devices and handwriting tablets: Scripture 1895 devices for storage of pen positions, McAllister 1900
Xcode IDE and development tools superceded Project Builder IDE, which was derived from NeXT IDE and tools.
Borland SideKick PIM / Personal Information Manager for DOS: included calendar, text editor, calculator, and address book / phone dialer. Over one million copies sold in first three years (starting 1983). Later development done by Starfish Internet Sidekick.
Biography on Ken Knowlton: cites to patent 3,879,722 on automatically relabeled keys/button (soft button labels) by projection, graphical flip book from Freestyle at Wan, character/handwriting recognition with Jourjine
Review article on Multi-Touch: states that 1982 movie "Tron" shows multi-touch -- compare with Tog's movie?
Wikipedia: Gesture recognition. Describes gesture as motion of human body, does not mention pen or tablet gestures. Also on www.spiegel.de
Wikipedia: Gesture. Describes gesture as visible body actions communicate particular messages.
Wikipedia: Defines gesture recognition as recognizing human (body motion) gestures, not PenPoint-type gestures.
Wikipedia: Gesture for pen computing. States that "drag" was first gesture on apple Macintosh, involving motion while holding down mouse button.
Wikipedia: Pen computing -- contains (lengthy) historical notes going back to 19th century. direct manipulation/ handwriting recognition / gesture recognition. Absolute/relative pointing/locator input.
Wikipedia: Pie menu. Cites early references to pie menus to 1970s, as a gesture gestural user interface (similar to "drag" gesture?), makes assumptions about hierarchical menus and ability to chain multiple levels of menus.
Wikipedia: Historical timeline of Microsoft Windows operating system. DOS addressing modes replaced with Windows 95. No mention of Windows for Pen Computing, or of Tablet PC. Comparison with OS/2.
Wikipedia: Windows CE 3.0 operating system ca. 2000 for palm-top pen computers, touchscreen digitizer. Reference has no information on gesture input. See also Windows
Wikipedia: iPad iOS tablet/touchscreen computer. Cites to Newton MessagePad and PowerBook PenLite. iPad development started before iPhone. No DRM, but does prevent transfer of files.
German short-hand handwriting character recognition: segmentation based on histogram, strokes, loop-detection. Uses chain codes. Shows stenopad: interactive ink-editing GUI on Microsoft INK API.
Optical multitouch on un-instrumented surface/tabletop. Touch detected (approximately -- not perfect) by height above surface -- suggests using hand-posture information to improve. Advantages include ability to recognize movement in air/proximity, gestures in air (e.g. picking up an object and moving it), and know which of multiple touches were from the same user / same hand.
Add FSR force-sensing resistor to smartphone, study of use of pressure levels by user to select items from a menu, rather than width or volume parameters etc.. Comments on poor quality of pressure sensors affecting earlier work e.g. Nokia N810 internet tablet with non-linear pressure/force sensor. Dwell (with minimum time 1 second) less error-prone than quick release. Compare with press-and-hold. Note: On N810, says nothing about it measuring contact area and not pressure/force, though this is stated in his thesis work.
Mentions touch gestures on Windows 7/Tablet PC not working well, HP implementing alternatives
Microsoft investing in N-trig Ltd. touchscreen hardware for Windows 7 / laptops in response to Apple iPhone touch-sensing screens on mobile phones.
Responsiveness/performance of touchscreen in cell phone varies greatly with engineering to integrate (including software), for same touchscreen component. States resistive touchscreen had poor responsiveness to detect touch.
Haptic stylus to give illusion of touching objects inside a flat tablet display: when pressed on screen, stylus shortens (collapsible moving shaft) and projected tip of virtual stylus elongates in displayed image. Described as "direct touch". Note: what type of touchscreen/tablet technology -- proximity sensing?
Review of Sketch Recognition, cites to ASSIST system (and video) from MIT and to Sutherland Sketchpad. Research focus is multi-touch input for sketch recognition in programming tools.
Additional materials in electronic file
Design hand/finger gestures on touchscreen touch table by showing use the effect, and then letting user make what they think is an appropriate gesture: i.e. for pan or drag, user placed hand on surface and moved across as direct manipulation. Not so much intended for user-defined gestures, as helping designers pick gesture sets informed by user natural behavior. Refers to surface computing (two authors from Microsoft). Elicited gestures included single select: tap or lasso gesture. Group select: hold and tap (two hand), move: drag or jump gesture (one hand to the other); cut/paste: two separate taps with same hand; rubout gesture. Drags to/from off-screen. Cites to Wellner.
EdgeWrite unistroke/graffiti gesture/character recognition using joystick input, similar to approach for Trackball EdgeWrite. User moves to edge limits when making character, somewhat similar to zone-based recognition. Chart of input alphabet, similar to presentation of Grafitti.
Scrolling in mobile terminal, with feedback when the scrolling hits the last item of a menu list. Refers to flick gesture input, and to a graphical effect being displayed at the end of scrolling.
Adding electronic ink annotations for a document (browser) with ink, highlight (translucent electronic ink), erase (of electronic ink annotations): ink annotations stored separately from document. Compare with Wang Freestyle annotations, PenWindows annotation inking layer, van Raamsdonk, DotSpots?
Definition of stroke as mark on a surface, or as a single complete movement.
Definition of stylus as a pointed tool for writing or drawing.
Posting of gestures for iPhone/iPad, Windows 7 Phone, Palm webOS, Apple Mac OSX trackpad, Apple Mac OXS Magic Mouse, Microsoft Windows 7, Wacom Bamboo (tablet), GestureWorks (multi-touch: see Apple), Microsoft Surface.
See "Touch Gesture Reference Guide": Posting of gestures for iPhone/iPad, Windows 7 Phone, Palm webOS, Apple Mac OSX trackpad, Apple Mac OXS Magic Mouse, Microsoft Windows 7, Wacom Bamboo (tablet), GestureWorks (multi-touch: see Apple), Microsoft Surface.
Summary of standards for minimum touchscreen target size for Apple iPad, Microsoft, Nokia, Ubuntu: e.g. 10 mm, larger for multi-target tasks.
UK manual for Windows Mobile Pocket PC smartphone with touchscreen and fold/slant keyboard. Same model at AT\&T Tilt (see). UK manual includes presentation of apps: maps with GPS, email, web browser.
Paper file may have 2004 version: See also paper file for Higgins
direct gestures (direct manipulation) contrasted with indirect actions/gestures e.g. multi-finger swipe to clear windows.
Prototype for multi-touch LCD touch screen with proximity sensing: an infrared sensor array integrated with pixels, infrared light source into glass, sensors pick up images causes by FTIR frustrated total internal reflections. Proximity sensing by infrared sources above the display surface, sensors pick up reflected infrared light -- no comment on blocking by other fingers? Sensors at finer spacing than the LED display pixels (two between each pair of pixels). Cites to JazzMutant as multitouch: gives no date, says resistive. Cites to DiamondTouch as multi-touch.
Prettifying graphical editor for line drawings: limited use of snap-to grid, heuristics to recognize drawings or text or gestures; double-circle / lasso gesture to select, pigtail lasso to delete (compare Scriboli), a GestureBar, scribble/scratch out, undo gestures similar to W4PC, zig-zag for zoom in / out, polylines, press-and-hold/dwell for panning. Uses timeout for distinguishing characters versus drawing. Supports overdrawing to connect disjoint lines. Vertex dragging.
Multi-touch electronic ink / mathematical text editing user interface using both pen and fingers: primarily for students applying algebraic manipulation. Users learn gestures by dynamic example much more readily than by text or verbal description (duh!). Ink/text adjusts for cut/paste/move (Compare with van Raamsdonk?). Two-finger gestures for open space, swap positions, move, select. Gestures include shape of hand/palm on surface, vertical angle orientation of finger (see Rekimoto?). Instead of shape/path gestures, menu under fingertips of non-dominant hand to select tools (see Buxton bi-manual).
Touch-enabled / multi-touch mouse prototypes from Microsoft, contrasted to multi-touch touchscreen of Apple. Touch gestures on mouse -- compare with Apple touch mouse? Technologies include mechanical (two finger cups on actuator arms), optical, capacitive touch, FTIR to detect fingers on surface of mouse, tracking of finger gestures around the mouse not on surface. Includes video demonstration.
Reference to Palm Pre and Apple iPhone and unspecified patent claims concerning separate gesture area from display touchscreen in hand-held device
Acoustic touch digitizer, using bending waves within substrate/glass instead of surface waves SAW, determines center location of smearing/dispersed/reflected acoustic waves. Compare with TouchDynamics / Acoustic Pulse Recognition?
biblio07 (Prev page)
(Next page) biblio13
standing your ground is generally a bad strategy:
it just makes it easier for people to maneuver around you.
dass er immer weniger über den Sinn des eigenen Lebens weiss,
sondern dass ihn das immer weniger stört."
-- Vaclav Havel,
tschech. Hafenarbeiter und
This compilation and all annotations are copyright © Jean Renard Ward, 1993, 2004, 2011, 2013, 2016.
I can be contacted at email@example.com.