Surface Computing Seminar Report 1nb6u

Abstract

The name Surface comes from "surface computing," and Microsoft envisions the coffee-table machine as the first of many such devices. Surface computing uses a blend of wireless protocols, special machine-readable tags and shape recognition to seamlessly merge the real and the virtual world — an idea the Milan team refers to as "blended reality." The table can be built with a variety of wireless transceivers, including Bluetooth, Wi-Fi and (eventually) radio frequency identification (RFID) and is designed to sync instantly with any device that touches its surface. It s multiple touch points – Microsoft says "dozens and dozens" -- as well as multiple s simultaneously, so more than one person could be using it at once, or one person could be doing multiple tasks. The term "surface" describes how it's used. There is no keyboard or mouse. All interactions with the computer are done via touching the surface of the computer's screen with hands or brushes, or via wireless interaction with devices such as smartphones, digital cameras or Microsoft's Zune music player. Because of the cameras, the device can also recognize physical objects; for instance credit cards or hotel "loyalty" cards. For instance, a could set a digital camera down on the tabletop and wirelessly transfer pictures into folders on Surface's hard drive. Or setting a music player down would let a drag songs from his or her home music collection directly into the player, or between two players, using a finger – or transfer mapping information for the location of a restaurant where you just made reservations through a Surface tabletop over to a smartphone just before you walk out the door.

 

1

In ndex 1 2 3 4 5 6 7 8 9 10 0 11 1 12 2

Sub bject Introd duction n What W Is Surfa ace Co omputing History of Surfface Computting Su urface compu c uter architecture Hard dware System m Softwa are Worrking K Atttributes Key P Percep ptive Piixel Multitou uch Appliication ns Pros & Cons Conc clusion n

Page no n 3 4 5 6 7 8 9 10 11 12 12 13

 

2

Introduction Over the past couple of years, a new class of interactive device has begun to emerge, what can best be described as “surface computing”. Two examples are illustrated in this report. They are Surface Table top  Perceptive Pixel The Surface table top typically incorporates a rear-projection display coupled with an optical system to capture touch points by detecting shadows from below. Different approaches to doing the detection have been used, but most employ some form of IR illumination coupled with IR cameras. With today’s camera and signal-processing capability, reliable responsive and accurate multitouch capabilities can be achieved. The multitouch pioneer Perceptive Pixel is building an entirely new multitouch framework from the ground up. Instead of simply mapping multitouch technology to familiar interfaces and devices, companie’s goal is far more sweeping. To use the technology as a foundation for an entirely new operating system. Because they are new to most, the tendency in seeing these systems is to assume that they are all more-or-less alike. Well, in a way that is true. But on the other hand, that is perhaps no more so than to say that all ICs are more-or-less alike, since they are black plastic things with feet like centipedes which contain a bunch of transistors and other stuff. In short, the more that you know, the more you can differentiate. But even looking at the two systems in the photo, there is evidence of really significant difference. The really significant difference is that one is vertical and the other is horizontal. Why is this significant? Well, this is one of those questions perhaps best answered by a child in kindergarten. They will tell you that if you put a glass of water on the vertical one, it will fall to the floor, leading to a bout of sitting in the corner. On the other hand, it is perfectly safe to put things on a table. They will stay there.

 

3

Wh hat Iss Surfface C Comp putingg?

Surfac ce compu uting is a new wa ay of work king with h computters that moves bey yond the e tradition nal mousse-and-ke eyboard experience. It is a natura al inte erface th hat allowss people to intera act with digital d content the e same way w they hav ve intera acted with h everyd day itemss such as photos, paintbru p ushes and d music the eir entire life: with h their ha ands, with gesture es and by putting g real-wo orld objects on the surfa ace. Surfa ace comp puting op pens up a whole new n cate egory of products p s forr s to interact with. Surrface com mputing is i a comp pletely in ntuitive and a liberating way y to interract with dig gital conttent. It blurs the liines betw ween the physical and virttual worllds. By usiing your hands orr placing g other un nique eve eryday objects o on n the surface – su uch as an item y you’re go oing to purchase at a retaiil store or a paint brush – you can eract with, share and colla aborate like l you’ve neverr done be efore. Im magine inte you u’re out at a a resta aurant with friend ds and you each place p you ur bevera age on th he tab ble – and all kindss of inform mation appears by b your glass, g suc ch as win ne pairing gs witth a resta aurant’s menu. m Th hen, with the flick of your finger, f yo ou orderr dessert and d split the e bill. We W really see s this as a broadening co ontent op pportunitiies and dellivery sy ystems. Surrface com mputing is i a powe erful mov vement. In I fact, it’s as sign nificant as a the mo ove from DOS [D Disk Ope erating System] S to GUI [G Graphic U Interrface]. Our research ows that m many pe eople are e intimida ated and isolated by today y’s techn nology. sho Ma any featurres available in mobile m ph hones, PC Cs and other elec ctronic devices lik ke dig gital cam meras aren n’t even used bec cause the e technollogy is in ntimidatin ng. Surfa ace com mputing breaks down d tho ose traditional barrriers to technolo t ogy so tha at people e can n interact with all kinds off digital content c in n a more intuitive e, engagiing and effiicient ma anner. It’s about technolog gy adaptting to the , ra ather tha an the use er ada apting to o the tech hnology. Bringing g this kind ural interface e innovation d of natu to the t comp puting sp pace is wh hat Surfa ace Comp puting is all abou ut.

 

4

Histtory o of Surrface Com mputin ng 2 2001

2 2003

2 2005

• Microso oft research hers Steve e Bathiche and Andy Wilson dev veloped idea of intera active table tha at could un nderstand the t manipu ulation of physical p pie eces.

• 1st proto otype modeel named T1 1which is baased on an IIKEA table w was born. Teeam saw the value  of the su urface comp puter beyon nd simply gaaming and b began to favvor those ap pplications tthat  took advvantage of the unique aability of Surface to reccognize phyysical objectts placed on n the  table.

• Attention n turned to its form facctor‐ Tub prroto type was designed d.

• Final stru ucture finalised,interacctive tableto op device w was built thaat seamlesslly brings bo oth the  physical and virtual worlds into o one.

2 2007

2 2011

• Microsofft Surface 2 Launches.

 

  T1 

  Tu ub Prototype e  

Microsoft Surface

   

 

5

S Surfa ce Co ompu uter A Archittecturre Te echnolog gy behin nd Surfa ace Com mputing: Micro osoft Surfface usess camerass to sense e objectss, hand gestures g a touc and ch. Thiis in nput is the en proce essed and d display yed using g rear projection.. Spe ecifically y: Micro osoft Surfface usess a rear projection p n system which displays an a image ontto the underside of a thin diff. Objects such as fingers are a visiblle throug gh the e diffr by serie es of infra ared–sen nsitive ca ameras, positione p ed undern neath the e dissplay. An n image processin p ng system m processses the camera c im mages to o detect fing gers, cusstom tagss and oth her objec cts such as a paint brushes b w when tou uching the e dissplay. The objectss recognized with h this sysstem are reported r d to appliications run nning in tthe comp puter so that t they can reac ct to obje ect shape es, 2D tag gs, mo ovement and touc ch. o the key y compon nents of surface s c computin ng is a "m multitouch h" screen n. It One of is an a idea th hat has been b floatting arou und the re esearch commun nity since e the 1980 0s and d is swiftly becom ming a hip p new prroduct in nterface — Apple''s new iPhone hass mu ultitouch scrolling g and picture man nipulation n. Multito ouch dev vices accept inputt from multip ple fingerrs and multiple ussers simu ultaneoussly, allow wing for complex c x ncluding grabbin ng, stretch hing, swiiveling and a slidin ng virtuall objects gesstures, in acrross the ttable. An nd the Surrface hass the add ded advan ntage of a horizon ntal screen, so several people p can gathe er around d and use e it togeth her. Its in nterface is i the exa act posite off the perssonal com mputer: cooperat c ive, hand dson, and d design ned for opp pub blic spac ces.

 

6

Hardware  

Surface is a 30‐inch display in a table‐like form factor that’s easy for individuals or small groups to interact with in a way that feels familiar, just like in the real world. Surface can simultaneously recognize dozens and dozens of movements such as touch, gestures and actual unique objects that have identification tags similar to bar codes. Surface is a computer embedded in a medium-sized table, with a large, flat display on top that is touch-sensitive. The software reacts to the touch of any object, including human fingers, and can track the presence and movement of many different objects at the same time. In addition to sensing touch, the surface unit can detect objects that are labeled with small "domino" stickers, and in the future, it will identify devices via radio-frequency identification (RFID) tags.

The Main Components are:

 Screen: A diff turns the Surface's acrylic tabletop into a large horizontal "multitouch" screen, capable of processing multiple inputs from multiple s. The Surface can also recognize objects by their shapes or by reading coded "domino" tags.  Infrared: Surface's "machine vision" operates in the near-infrared spectrum, using an 850-nanometer-wavelength LED light source aimed at the screen. When objects touch the tabletop, the light reflects back and is picked up by multiple infrared cameras with a net resolution of 1280 x 960.  U: Surface uses many of the same components found in everyday desktop computers — a Core 2 Duo processor, 2GB of RAM and a 256MB graphics card. Wireless communication with devices on the surface is handled using WiFi and Bluetooth antennas (future versions may incorporate RFID or Near Field Communications). The underlying operating system is a modified version of Microsoft Vista.  Projector: Microsoft's Surface uses the same DLP light engine found in many rear projection HDTVs. The footprint of the visible light screen, at 1024 x 768 pixels, is actually smaller than the invisible overlapping infrared projection to allow for better recognition at the edges of the screen.

The display screen is a 4:3 rear-projected DLP display measuring 30 inches diagonally. The screen resolution is a relatively modest 1024x768, but the touch detection system had an effective resolution of 1280x960. Unlike the screen resolution, which for the time being is constant, the touch resolution varies according to the size of the screen used—it is designed to work at a resolution of 48 dots per inch. The top layer also works as a diff, making the display clearly visible at any angle.

 

7

Five cameras c mounted d beneath h the tablle read objects o an nd touch hes on the e acrrylic surfface abov ve, which h is flood ded with near-infr n rared ligh ht to mak ke such tou uches eassier to pick out. The T came eras can read r a ne early infin nite num mber of sim multaneou us touche es and arre limited d only by y processsing pow wer. Righ ht now, Surrface is o optimized d for 52 to ouches, or o enoug gh for fou ur people e to use all a 10 fing gers at once and still have e 12 obje ects sittin ng on the table.

Sy ystem S Softwarre  

 

Surfac ce compu uters from m Microssoft uses Window ws 7 as the eir Operrating Sysstem & su urface co omputerss from oth her vend dors use their t proprietary OSs. The e varrious dem monstratiion programs are e accesse ed from a main menu, whiich scrollls leftt and righ ht in an endless e lo oop. The e mo oves the selection s n by swip ping bac ck and fo orth and selects s a applica an ation with h a single tap. Th his workss rea asonably y well and d feels qu uite naturral. When an app plication is i selecte ed, a swirly purrple ring g appearss in the center c of the scree en to ind dicate tha at the pro ogram is loa ading. Mu uch of the e softwarre was wrritten usiing Micro osoft's WPF W (Wind dows Pre esentatio on Founda ation), th hough the e XNA de evelopme ent toolk kit, a fram mework originally created c fo or writing g PC and d Xbox 36 60 gamess is also e s ed. XNA allo ows prog grammerrs to use managed m d code written w in C# to manipulate e variouss DirrectX fea atures; ma anaged code c free es the pro ogrammer from worrying w g about han ndling m memory, allocating a g and disscarding memory y automa atically. This T app proach h has allowed Micro osoft and d its partn ners to wrrite imprressive-lo ooking dem monstrattion prog grams forr Surface more qu uickly tha an would d otherwiise be posssible. emonstra ation pro ograms arre accesssed from m a main m menu, The various de wh hich scrollls left an nd right in n an endless loop p. The use er movess the sele ection by y swiiping bac ck and fo orth and selects an a application with a single tap. Th his workss rea asonably y well and d feels qu uite naturral. When an app plication is i selecte ed, a swirly purrple ring g appearss in the center c of the scree en to ind dicate tha at the pro ogram is loa ading.  

8

Working g  

Surfac ce uses cameras c t sense objects, hand ge to estures an nd touch.. This use er input is then processed p d and dissplayed using u rea ar projec ction. Surfac ce uses th his rear p projectio on system m which displays d e onto the e an image underrside of thin diffusser. Image e processsing systtem proc cesses the e camera a imagess to detec ct fingerss, custom m tags an nd other objects such s as paint p bru ushes whe en touchiing the displa ay. The objects re ecognized with th his system m are rep ported to applicattions runnin ng in the e compute er so that they ca an react to o object shapes, 2D tags, movement and d touch.

Ke ey attributes o of Surfa ace Com mputerr  

 Direc ct interac ction: Ussers can actually “grab” digital d infformation n with the eir handss and inte eract with h content by touc ch and ge esture, without w the e use of a mouse e or keyb board.

 

9

 Multii-touch c : Surface computin ng recog gnizes ma any poin nts of contac ct simultaneously y, not justt from on ne finger, as with a typicall touch screen, but up p to dozens and dozens d of items at once.

 Multii- ex xperienc ce : The horizonta h al form fa actor mak kes it eassy for severral people e to gath her aroun nd surface e compu uters toge ether, pro oviding a collab borative, face-to-fface com mputing experienc e ce.

 Objec ct recognition: s U can n place physical p o objects on the surrface to trigge er differe ent types of digita al respon nses, inclu uding the e transfe er of digittal content.

 

10 0

Pe erceptiv ve Pixe el Multiitouch

 

Comp puter scie entists se ee techno ologies such as su urface co omputing g and mu ultitouch as the ke ey to a ne ew era off ubiquito ous comp puting, where w pro ocessing pow wer is em mbedded d in almo ost every object and a every ything is interactiive. Last yea ar, New Y York Univ versity professor p r Jeff Han n launche ed a comp pany callled Perrceptive Pixel, wh hich buillds six-fig gure pluss custom multitou uch draftiing table es and d enormo ous interractive wall displa ays for la arge corp porationss and military situ uation rooms. The d display’s s surface e is a six x-millimeter-thic ck piece e of clearr acrylic c, witth infrarred LEDs s on the edges. Left L undiisturbed d, the light e es along pre edictable paths within w th he acryliic, a pro ocess kno own as total inte ernal refflection. When objects o su uch as fiingers to ouch the e surface e, the lig ght diffus ses at the t poin nt, causin ng the ac crylic’s internall-reflectiion pathw ways to cha ange. A camera below th he surface captu ures the diffusion n and se ends the infformatio on to ima age-proc cessing software s e, which can read d multip ple touch hes sim multaneo ously an nd translate them m into a comman c nd. The system s s sends infformatio on about screen touches t to applic cations via v the liightweig ght Open n Sou und Con ntrol prottocol, utilized fo or networrk-based d commu unicatio on betwe een com mputers s and mu ultimedia device es, and U Datagram Protocol P data tra ansport technology. The applicattions the en take the t appro opriate actions. a

 

11 1

Applications  Wireless! Transfer pictures from camera to Surface and cell phone. “Drag and drop virtual content to physical objects.”  Digital interactive painting  At a phone store? Place cell phone on the Surface and get information, compare different phones, select service plan, accessories, and pay at table!  At a restaurant? View menu, order drinks and meal at your table! It’s a durable surface you can eat off of (withstands spills, etc.). Need separate checks? Split bill at and pay at table.  Play games and use the Internet.  Watch television Jukebox!  Browse music, make play lists.  Billboard for advertising  Maps  

Pros & Cons of Surface Computers  

Pros

 Large surface area to view different windows and applications.  Data Manipulation - Selecting, moving, rotating and resizing (manipulating objects on the screen is similar to manipulating them in the manual world).  Quick and easy to use.  More Than One –Several people can orient themselves on different sides of the surface to interact with an application simultaneously (Max 52points of touch).  Object Recognition - Increased functionality aiding in speed and ease of use  Time saving by eliminating many processes.

 

12

Cons

 Not portable and very expensive.  Privacy - Open for many to view.  Screen Visibility - Glare, finger prints/dirt and human interaction obscuring interface.  Poor Accuracy - Fat fingers are not as accurate as a mouse or stylus.  Fatigue - Reaching across the table often can cause the arms to ache.  Objects needs to be tagged (domino tags or RFIDs)

Conclusion Some people will look at Surface and claim that it does nothing that hasn't been tried before: computers with touch screens have been around for years and have already found niches in ATMs, ticket ordering machines, and restaurant point-ofsale devices. This view largely misses the point of the product. Like most projects, Surface takes existing technology and presents it in a new way. It isn't simply a touch screen, but more of a touch-grab-move-slide-resize-and-placeobjects-on-top-of-screen, and this opens up new possibilities that weren't there before. Playing with the unit felt a bit like being in the movie Minority Report (in a good way), but it also felt like a more natural and enjoyable method of doing certain computing tasks. Sharing and looking at family photos, for example, is more fun on Surface than on any other device. The retail applications, particularly the dining application, show how businesses could use the technology to really stand out from competitors, though one wonders how diners will react when their table locks up and needs a reboot. Many people who viewed the early Xerox PARC demonstrations of the GUI came out of that experience knowing that every computer would work that way someday, and they were right. Playing with Surface, one gets the sense that although not every computer will work like this someday, many of them will. More importantly, computers running Surface-like software will end up in places that never had computers before, and the potential applications are exciting. Imagine a multiplayer real-time strategy game where you and another human opponent can move units around as quickly as you can point to them or perhaps an educational environment, where university students could assemble and disassemble anything from molecules to skyscrapers quickly and easily.

 

13

BIBLIOGRAPHY: 1. Apple iPhone Multi-touch. http://www.apple.com/iphone/ 2. www.idoub.com 3. www.whereisdoc.com 4. www.docjax.com 5. Microsoft Surface, http://www.surface.com 6. Perceptive Pixel, http://www.perceptivepixel.com

 

14