Touchscreen device first emerged in the late seventies with the Fairlight CMI, which used a light pen. In the early eighties, Hewlett Packard introduced the HP-150: an IR based touchscreen. The early 90’s saw the Apple Newton and the first touchscreen based smartphone in the shape of the IBM Simon come and go.
IBM Simon – First touchscreen Smartphone
It was US Robotics and the PalmPilot, that first cracked the mass market with a touchscreen, but these were still fairly simplistic black and white devices and used by people mostly as electronic Filofaxes. It was really Apple, with the iPod touch and iPhone that truly democratized the color touchscreen, to the point where it was device carried everywhere.
There are many touchscreen technologies on the market, the main ones are:
There are a few other esoteric technologies that use acoustics and take advantage of the fact the glass deforms slightly when touched.
This works by projecting infra-red beams across the horizontal and vertical surface of a screen, thereby producing a grid. If an object interrupts those lines, its coordinates can be determined and a touch is registered. Infra-red is useful in environments where gloves and other non-conductive objects are used to interact with the screen. As such, it is common to find it used in outdoor digital signage applications.
These are typically used on lower end products or products where a non-conductive object is used for touch (such as a gloved hand). The screen is contains electrically conductive and resistive layers. When pressure is applied, the layers are brought into contact with each other causing a contact and registering a touch.
Resistive Screen – Image courtesy of ScreenTek
Anyone who’s used an iPhone or iPod Touch will be familiar with capacitive screens. The screens are coated with a material that can carry an electrical charge (typically indium tin oxide). When touched by a conductive device (such as a finger or capacitive stylus) a charge is created and the location is sent to the device.
This is a fairly recent development. With optical touchscreen, two or more image sensors are placed around the edges (typically in the corners) of the screen. Infrared backlights are placed in the cameras field of view. A touch registers as a shadow the location (and size) of the touch can be measured by the cameras.
Camera in Pixel
This is one of the key technologies in Microsoft’s PixelSense technology. The LCD’s backlight contains IR (as well as visible) LEDs. When an object is placed on (or very close to) the surface, light reflected back from the contact is seen by the sensors. The hardware identifies these and the output is sent to the PC, which includes the sensor image and contact types. This occurs 60 times a second at a resolution of 960 x 540 (24dpi), which is half the resolution of the screen’s 1920 x 1080. Apparently, this is resolution can be increased in the future by software.
The Surface technology recognizes 52 simultaneous contacts of three types: finger, blobs and byte tags. Raw vision data can also be used by applications. One of the sample applications is Data Visualizer, which is useful for understanding what Surface recognizes. In the diagram below, two fingers, one blob and one tag are recognized and assigned unique ID’s.
Physical objects such as the palm of a hand or a business card are examples of blobs. Tags are small printed stickers. Typically they are printed in black and white, but could be made almost invisible by using IR reflective ink/coatings. There are 256 possible unique tags that are encoded as a byte and (by using custom code) groupings of tags can be used to achieve more than 256 unique combinations.
The tags are 3/4″ square and can have square or beveled corners.
Give their size and the limit of 256 unique combination, makes them unsuitable for things like customer ID’s. For such purposes, it’s likely that external magnetic card or NFC readers will be used.
About the SUR40
The Samsung SUR40 is a 40-inch 16:9 LED backlit LCD display (1920×1080) with integrated PC and PixelSense technology. The screen is considerably slimmer than the Surface 1.0 version at only 4-inches (previous model was 22-inches) and weighs in at around 80 pounds. The screen can be mounted horizontally or vertically (or any angle in between). An integrated tilt sensor records the angle. The retail price is a little over $11,000 (without legs).
There are some key installation points to consider. The main ones are space and lighting. As people can interact from any side (when mounted horizontally), adequate space needs to be provided around the product, not only for people to sit, but for onlookers too: the product will attract a lot of attention.
Lighting is critical. Being a vision based system, direct lighting is not viable. The tablet should be located away from any direct light and away from sunlight. Care should also be taken to ensure that no indirect light is reflected onto the surface.
Samsung recommends using diffused energy efficient compact fluorescent lights (CFLs) or LEDS to light the surrounding area. In the case of LEDs, cooler color temperatures should be used to avoid any emissions toward the near infra-red spectrum.
An application is provided that will check the lighting falling on the table once installed. Problem areas are shown in red and questionable areas are shown in yellow.
Samsung provides a comprehensive Venue Readiness Guide which details the precise lighting guidelines.
Most commercial displays tend to be equipped with 3-5 year warranty. The SUR40 carries only a 12 month warranty. The unit is also warranted for only 8 hours continuous use per day, which could preclude many potential applications such as retail, way finding, museums, etc. Samsung did state that they will extend this if they are involved in the selling stage and are comfortable that the type of content and usage patterns will not result in adverse “wear”. Build-a-Bear is an example where the unit is warranted for more than 8 hours use per day.
Operating System and Applications
With Windows 8 having so many touch features, one would think the SUR40 would run Windows 8. It doesn’t, and there are no plans to support it. Windows 7 Professional is supplied with the product. Windows 8 is not required, as all applications run within a Surface environment (essentially a Windows 7 application). If XP is anything to go by, Microsoft will still be supporting Windows 7 (especially in embedded products) for the next decade.
Although there are sample applications provided and a number of off-the-shelf applications are available, for the most part, sales will be full solutions sales (such as Build-a-Bear) and will include software development. Some resellers may handle this in house if they have development resources and wish to extend their portfolio to Surface development. For others, Microsoft provides a list of potential development Partners.
During the certification training a concept gift registry application from Clarity Consulting was shown
For example, an engaged couple visit a store, select prospective gifts, sort and filter them and ultimately create a gift registry. An access card is then sent their friends. Their friends visit the store, place the card on the SUR40 and interact with the list (browsing, sorting, filtering, etc.) to select the gift to purchase.
Interacting with the touch screen was a great experience. There was no lag or delay as is found on many touch screen applications and technologies. Using the sample piano application, the keys responded immediately. Probably the most impressive application was “Attract”. This contains an image of water with leaves and rocks. Touching the screen causes ripples in the water and the pressure of touch alters the size of the ripple created. As the ripples hit objects they bounce off as they would in a real pond.
The SUR40 with PixelSense represents a leap in touchscreen technology and the product is a delight to use. Care needs to be taken with installation—especially lighting. The hardware price of $11,000 isn’t excessive, but a realistic solution is likely to be in excess of $50K by the time software design and development is factored in.