EFTA01181673.pdf

Exhibit A EFTA01181673 (12) United States Patent oto Patent No.: US 6,639,614 B1 Kosslyn et al. (45) Date of Patent: Oct. 28, 2003 (54) MULTI-VARIATE DATA PRESMTATION 0711138 PUBLICATIONS METIIOD USINO RCOLOGICAL1T VALII1 STIMULI Slephen M. Kosam, •elemends of Oraph Design'; Ulic mac, Publishing page, and pp. 267-270. (76) Imaniom Sleplten Michael Knaslyn, 28 (bantki Jako J. vno Wijk and Huub van de %Werk& "emilka SI., Cambridge. MA (US 021381 licemaps: Vistialintion of Niem maka! Information• Eind- (Baraki (Mula, 130 Brainard Rt, Apl. hoven Unlvenity of ibelmoloyy, Oogt nf Mathernatks and Compeling Science. 111abdocument,"Titeniaps for spaw eonanim:4 Viella11741- tinn of Marimbie?, wab sile: hIlp:/Avww.cs.umd.edoilwill rreemaps„ picked as of Apr. 19, 2000, fout pogoi. (s) Notko: Subject to any disclairoer, dn letni of lltis Web document, "Map koltom:nas", web sito: littpc/Awv.s- patent Is endended ur adamom, to. k r 15 martmoney....mulmarkciraanliminietiencidnil, picked u of V.S.C. 154(6) by 445 days. Apr. 19, 2000, Iwo pages. domment, "SmtdMoney lbdar Market Tbday: (20 AppL No2 09/613,605 Soenna to Uaing Mc Map of tbc Maker, wab Me: hup:// vAvwsmartmoney.comlana/ourketshaewarrndoccfm- tatory-199012313.pldred as of Apr.19,20X1, (taco pages. (22) Filet Jul. 10, 2000 (LM contirtud on trost me.) (51) IBL ...... G09G 5/00 (52) U.S. CL Mniemy lixonsina—,Inko Cabeca (58) Field of Smrek 345/836, 837, Ast/stam E.ransiner—Ildan Ddwiler 345839, 976, 977, 440 (74) Mimicy, Agent, rw flisa—Brián L. MkhaelLs; Joseph P. Gulon; Skowa Rudni& Bada lamek LLP (56) Weronce* CYM (57) ABSTRACT U.S. PATENT DOCUMENTS Ametbad h prowidni wherobymmlikuitions of eeologically 5/124976 A • 4/1991 •AreactIbleA oc al. walid images and wanda aro wsad rot vlavalkation of large 5.043,920 A M991 Mala el al motu*" of infnrmatiuo. topos and sounds oto seleclod and 5144190 A a/1992 nerwu« ei al. oembiced to acommate namul recogpition of panam laba. 5)21)00 A 611994 Desa 6111110/1 and chanem. A ubiel field of similor kans (cpn- S.347A52 A 9/1991 gay, k 5,375,201 A 17/1994 Lkonazt ants a conapoodirm fleki of dala sola, %bero tho anur- S/152.4(0 A W1995 Smith iom of emil icon illustraten tka ~Mb% valau of tle 5,555)54 A 9/1996 Wnuki. et aL tualedying dada. Wilki owa kos, elements chama in 5,584677 A 12/1996 Myca cl al stopu, siw, tambor, cek; and motku to Maskota Ile A 411997 Mick ct respeetivo cham, of data radlin the corresponding data set. 5671,381 A 9/1997 sttankle et al. Starodawnie wand ocarrence, spahi kostkot type, and 5,675,746 A 1(0097 Marshall voltamesignal (ho laser obaw Uw genami stmation, nawali as 1,750µ/0 A • 3/1998 Piat — .... 60W514 mc type and importants of data atveges, atimu- 5,774) 78 A 6/1998 Mardaall 705/35 MM" A 1/1999 tków kling Ihe U.Ste$ witenlion lori»: nenia of klang 5,974/93 A let/1999 IAgla 13 CDII% 6 Dranka Slapets EFTA01181674  US 6,639,614 B1 Ng. 2 MUM PILIBLIOVIIONS Mb document, "timattbilorway Today: Mattel Tbday: Secret to Using lbo Map of filo %tuber, web silo: Map:// 414 / 11, Jocumant, `Sou oMonq.cont: Map Station", wee site: www.smo Ortoney.comixothookuistocandiottox.efm- httpa//wwwwnatononey.contimapstaitou/, picked as of Jun. 7stotr199812313, picked as of Apr. 19,20), ate* pages. 18, 2000, dote pages. Wah docturaent,"Map Inortsdions", web eke: hop://avents- Web donsatom, "Sulariktoney.com: Map SIM %eh she: otailmonety.conohcoVniarketenaplinskudionslatenl, picked http://wwwsmartmoney.cominiapstatiOni n of Apr. 07, 2020, two pages. indox.efintstoryrapasiag, picked an of Jam 18, 2000, two Mot • cited by ccamiocc EFTA01181675 U.S. Patent Oct. 28, 2003 Sheet 1 orb US 6,639,614 Si FIG. 1 EFTA01181676 U.S. Patent Oct. 2..8, 2003 Sheet 2 of 6 Us 6,639,614 B1 FIG. 3A FIG. 2A FIG. 3B FIG. 2B FIG. 3C FIG. 2C FIG. 3D EFTA01181677 U.S. Patent Oct. 28,2003 Sheet 3 of 6 US 6,639,614 B1 Baseline FIG. 4 [green] [red] FIG. 5 0 [green] 0 [brown] FIG. 6 EFTA01181678 U.S. Patent Oct. 28, 2003 Sheet 4 of 6 US 6,639,614 B1 701 FIG. 7 EFTA01181679 U.S. Patent Oct. 211, 2003 Sheet 5 of 6 US 6,639,614 B1 FIG. 8 Plower Icon Elaueob Range Data Element Mamba ofpetals 1-12, (CULL 6 Price doge slice btaline, dollars Sim ofHead 1-7, nonnalittd for present range Absolute price ofstock Length 'Otero 1.7 Present standard Skimof price, in 0.5 SD increments, compared to avenge Coke of Stan Green/Red Price up or down Color °Yam (keen/Brown hice up/down for previous N trades Change ofIcon Motion ofFlower Stan Slow, medium, fast Standard deviation ofprice relative to baseline Color ofRead Red, Oran Price: Goan up, red down Intensity ofHead Color Bright,Dim Bright, large change; din, small change Blinking tined at once per second; duration Price changed fixed at 30 seconds Negative Sounds loudness in100inneinents proportional to volume ofbade Violin low notes Price 0.5 SD lower Plato low nota MCC 1SD lower Bass low notes Price 2 or moo SD Iowa Positive Sounds Loudness in 100 ineterneots proportional tovolume Mtn& Violin high noses Price 03 SD higher Piano high notes Price 1 SD higher trumpet high notes Prica2 a roore SD hi& Location of Sounds Right to Left, 3 secs each way; Negative let; Fixed at two stops Positive right; Background sounds Selected from negative or Average roast below or above positive sounds; baseline; Volume in 100 IIICTC11109t$ Volume of trades in the market Txneia Events Bunny nibbling near flower News item for the stock Flower ill black Trading suspended Border colors Circa, red; Overall volume up, down; Color saturation; Pruponional to amount of change Color intensity since baseline; Overall volume since baseline EFTA01181680 U.S. Patent Oct. 2,8, 2003 Sheet 6 of 6 US 6,639,614 B1 SELECT ICON ,-901 ALLOCATE DATA ITEMS ,-903 TO ICON VARIATIONS INTERPRET DATA ,-905 AND DISPLAY ICON ACCORDING TO DATA SELECTED DATA YES CHANGES NO CALCULATE SIZE AND DIRECTION OF CHANGE TRIGGER SOUND AND VISUAL CHANGES -913 OBTAIN NEXT DATA UPDATE FIG. 9 EFTA01181681  US 6,639,614 B1 1 2 MUCII-VARIATK DATA PRESENTATION display; and a lied related to memory and processing METHOD USING ECOLOGICALLY YAW) caned to roper inhatmotation. 'these same STIMULI principles also apply directly to Marna of worsts. One approach for display of lafonnadon is disclosed In s U.S. Pat. No. 5,671,381 based to Strasniuk. A three- FIELD OP 1NVENITON dImeinlonal, virtual reality display VICO is meow] to Loup lain objects that represent blocks of data as 3-1) baecbaris. 'The moml inflation rams to graphical and warthog Attributor of the data are mapped to visible or audible presentation of dale, puticularly for exploiting human pat- chniadaiSikis such as an kw having a specific sizeor min recognition. io color. The spatial tolationthip and connecting limn Wagon BACKGROUND Mons in the landscape represent structural relationships that oast in dm underlying than, with IM ground Plano mime- The field of Infomation visualization includes the wee of settling a numerical vase as a common surface plane. maps to convey informatics inn mobil manner. Apemen- Artificial perapecuive (with object compressionnear the ale information displays can bring life to otherwise then s horizon) adds to the reathrn of the view. A user an arrange matrices or sucams of numbers. Human perception and the objects according to a preferred lexical order, and then tecognilloo of data treats can be facilitated through con- "tour across the landscape to browse or much Sr pariku, strection of onmprehonsibk grapta.This trend perception is lu data bans or relationships. especially important when µWen Wµe annals of multi- There are many disadvantages of this method of vsual- mutat data from which useful information most he quickly sn For one chino, the data represeinalkiFIS aro purely derkted. Depending upon fin environment and Om intim, utificia, much Rho a two-dimensional Mischa/I with its lance of the decisions to be made, even the best information necessarily limiled information bandwidth. A sidle hietar- can become overwhelming. An example of ono such enyi. diked ken paradigm dictates the anangernenth between the foment is a fast-paced stock lead* COOT What> Itaancul icons, illustrated by cluttered linkage lines. In w.ence, the analysts are expected to quickly assess varier online 2s user can merely navigate through a Ma of bar charts laws. sources of information and make irrevocable decudoes tint (rating the are and age of files, or similarly berIgn palm. can effect their very carmine Other situations rise in civil mars. Thom is no means for indicating any data chimps, nor eminencies whore an uninformed decision couldJeopudize their magnitude, relevance, or direction. Given the lack of lives. The complier industry has generaled a number of change information, it would his impimble todeice: any look for coping with problems suck as these. 30 palletmof changes amog 'hada sets. Ilurthennore, sound On brains allow to to do many things that defy oven the is impkatented solely as a "warning tone triggered when MOM complex artificial tricorn. At the fore la ability to Ike uses moor teaches a filo icon having a predetermined organize a diverse range of informatics into relatively ittanao. There is to selection from a variety of meaningful ample patterns and to monitor them 'ultimo& for Dalliers; sounds, or any concept of spatial °depletion, intomiy, or cases that break macs with the majorily.The brainMillebeat as inherent recognition of the sash's meaning, other than its 'patina dektmor nxicts003. The couvouitheal data viou• mere OCOURCOCO. alitattion tools have failed to take advantage of the various Significantly, the system relies upon the virtual (.o., and paceplual powom of lie heath in any arm- artificial) reality canoed, n computationally complex, and load manner. visually distracting data display. Virlesi reality (VR) A graph is a visual display that iamb-ales ono or nom 4o mount to replicate physical reality, whom the bolter the relationship among numbers.Tbo heal graphs are those that VR system, the better the rendition. Visualizations of this permit n viand pattern. trend or comparison to bo quickly type accenntato tie details at the expense of data coolp(o. and acmitately compthearkd by a human wader. A poorly Mashie. As described in Roslyn's "Elements of Graph designed or constructed grape can be (Molt to decipher Design", human ears can reliably process only a limited popaly, nod coo rood lamination or etagnenisoonoln- s amount of shoal infounaiou at one lime, depending on a siom Eased upon knowledge of the wifelike audience. number of psychological kelom including relevance and coupled with art understanding of human perception and Mows. It takes additional effort for the mind to comouct (*satins, a certain crallamariship can be brought to boar a 3-1) perceptual organization of modem icon meanings and upon the task. mitigation% especially when they aro made 0044*3 loss Wel- Application of empirical findings from researchOn human SO ligibk by the artificial variations and disbitions constantly cognition sad 'sampling to creation of graphs isexplored a Introduced by "navigating." A VR display contaminated M. Rosslyn's book, "Elements of Onph Design: with irrelevant or goody detailed information nay actually (W.H. Paestum a fib., 1994), whist fa incorporated Wnt o reduce the ability to properly perceive the data patterns of by reference. The author evaluates many of the factors that most interest. ahead he otaktored in when tinkering a graph format to r A similar 44444tyrrinnl, mocalcally addressed to visual- present specific content for a specific purpose, tad derives a ization of Information woeful to money managers, is dis- set of paucities havingn basis in fie physiological psy- closed in US. Pal. Nom 5,675,746, end 5,774,878, both chology of hums cognition and perception. Exploitation of issued to Marshal. In time and related patents, so-called these principles when formulating a graphical presentatioo 3-1) "metaphors" aro used in represent data in a virtual eau daainalically fmpmvo tbo useful inforinatiuo ..eater reality setting. where diaracictialka of each metaphorical without appreciable increase in visual complexity. The object am determined by the corresponding data. The shape, author derivesn lot of principles that provide a founework on/or and rotation of each object may change according lo against which to calibrate the relative effectiveness of vaii- the data, or to highlight criteria selected by the user. The ous approaches o data vlitualhottion. In gumnal, me peiu- localism of the object may reprepent the source of rho ciples can be divided into throe sections: one regarding the m informalion (e.g., a selected market information feed), or a way in which we actively eugenics and interpret what we collective almilarky (e.g., intheary groups). lie need may sego; another regarding bow meaning is derived faun visual then -fir among the objects to observe their choracteriaks EFTA01181682  US 6,639,614 B1 3 4 more dimly. Paw example, objects 'floating' above the everyone knovoi that on sea of infection k "red," because perceptual ground plane teak( !crested data kw stocks of infiamatistion,yet Lesser would indkato such a elate with unarm "show retragmalbts respective loaniimggofmooch, a bkie icon. Sieritsrly, the dais k,xn are oomprbart of axed shapes and imminent of 3.D o ts too miedlied by lho shodthapc and loestiou,esing annihilationsof gam mints: system configuration, 'Montt no partiotbr arrange:cools $ red, gram, and yellow, to represent "deviations" from a are described. normal value Thu moonily restricts the Lesser approach to Many of the disadvantages of this system relate to the types of data that can be charactorreoll as normal or &boor- limited virolatirmion mechanisms employable. Mao arrange- mat comaliticist. It also limits the information to a Impel of moot of icons is according as a poxlelermined set of three- abstraction that relies upon the system paramelers and dimensional axes. 'Mc icons theta elves have evirospormling to tequ tea Wedge assn taPtiOns be made about the rilloni to stome,simandeolor that arepumlyarbitruy and whkh lend pcedefine the binary threshokis clench normality. Similarly, wry little to any Inherent peccoption of their respective y amer makes no reference to sound at all, mbar than "bowel valeta. The portrayal of spinning, colored or pulsating icons wand." the status of which may be reflected in a corm- merely reposinot data or data kends kg have already been sponciagly colored icon. Ignoring this multimedia caladalod.lbo totally abstract hodscape does not lend kW( is Mon thither limits the amount of status information that can to °valuating any recognizable objects,Is atoms interactiorn be derived. or patterns Thom is nothing hi the data reality arrange- Funkticrooro, a eLossergrain- as taught by lamer hag. merit to facilitate recognition of "oullices", Lc., the few mks the underlying data for one taw at a time, not largo nowt:asafoetida's data sets. Hach of these Ilmitaihms of the numbers of case& The design oftener DM to capitalize an prior art fosters a :agility's:on fora large amount of learning so any pattern relevaocv the would bo cognizable across more boron' the display begins ao (lawn. maul for Anything than ono dela col, let aloes hundreds.lhem in no -*oaken! mom than a dais browsing fool mapping" of data in which completely different objects In addition to the mbar disadvantages of virtual cagily reprenent values of different measurements. Laser tam the mentioned above, a user unfamiliar with the medium not unto reclangslar shape over and over again for dozens of only must learn to configure and navigate the data, but can r different types of data, dramatically aggravating an already easily become manybeimed or discolor:4yd by the lack of slap teaming cony. uniformity or structure. At aunt, the Marshall system is Although Lessor does waged superficially that a Lessor.. disadnalmoously grounded on depth perception, and spa- Gram could be applied to "business data" or other forms of fiat intuition, which do not recognize the limits of human information, there is no luposlion whatever that such a cognition, memory, or oumptehmilon. Without these eke sa display sold apply to anything more than partly physical, moats properly constrained, to virtual reality icons remain process.caicated data. One mini presume, given the one-to- relatively lakciphorabk, and the in fomantion ferments- ow relationshipof data and display dolma's, that a I man- tions frustratingly inscrulable.ttunbarnom, the aso of sound (ham would require a fixed physical shape of some sort, by Marshall has the disadvantage that It fa mud only for determined by the process itself. (7egorat pieces would then faunally identified aid sekctedhighlighted cases rather m palmy some binary data Mahn of the process being aced- than showim warm or any other Informailan general to tared. Again, this km nothing at all &ado with defining icons the data W. liven when sound is Implemookd, there is or data teproserthitions to exploit the advantages or !huh- nothing to indicate how the round con be used to provide lions of human memory, perceplion, or comprehension. Al multivariate umbra to the nadgals display. this very least, anyone familiar with one typo of Lesserrinut fn an effort to bring a physical Association to data vim. an for human nwdical information would have no idea how to Minion techniques, U.S. Pat. No. 5,321,800, issued as apply Misname lechnolog.v man/omit type of dma, say, air Lessen describes a graphical method lied to medical data traffic control, or weather. about a patient. A sieglo display template Is arranged ss a One other °amok, of using matrices of rectangles with represcolation of the hunum form, with binary information different shapes and colors is in use by SourtMonity.com of korai imposed upon fixed locations of the ham. Thu present as Now York, NY, to ill mnia° Mock market data. Accordiug to data values are illustrated by Iraqi's the color or intensity their information, Markel llmlaY, a user con sec "The Dodosl of a specific anatomical hication of dm display correspond- at a show." latoranstion about six hundred publicly traded ing to the data item. Once a user (i.e., on or doctor) has stocks is arranged into hadostrysegmenis. Individual market thoroughly lamed the louation and moaning of tho venous cipihtliealloa of tacit cantpany dictated the relative stet of elements, the theory is drat the information shout theresent so a rectangle representing a company, and blocks within an patient con boquiekly deolpheced.Foroximplo,arm calorOf Industry segment that have similar bimodal. of poking blue may represent the ccerespondiog location of an movements are arranged next to each oilier. During a infection, and specific Mode; of colors on the chest or trading- scission a user can request an *tidal* of the display abdomen may represent recant blood-tosting results, vital when caffein information isdesised. Theodor of each block signs, patient complaints (e.g., pain), or other diagnostic ss various in twelve swiss from bright led, to Mack, to bright observations. Data excinsionsbeyonlmeskannisted thresh- green (or onothor Wettable speetedda, blue/yellow), iodi- olds can be indicator) by blinking red icons al the plotted citing the present changein prico with =pod lo a selected location. baseline (e.g., yettlerday's close). Among who many alb:advantages of the Lessor system is Significantly, the market map demonstrates many of the that, like Muskat Maw°, the dimes and changes in the ao disadvantages of daunt information overload, without ego- irons provide precious little if any information that capital- ally highlighting the news. Rectangles dictated by the wady isms upon human perizoloo, omouny or comprolgelothet. two thousand data points are squcezed into seemingly proposed placement of rectangular blocks of data is vary location on a display screen and then six hundred data completely arbitrary, other than the °twines analonty- points of a sinµle molt* (i.e., daily change) are stmulta- oriented locationsdictated by the patient data liself.Thara the ss ncously varied at the will of the market. Often there-whim examples chnorilical in the ilk:sirloin embodiment demon- conflation of data, the average user cannot Um reasonably skate a general disavgard for Inman cognition. For example, expo:led to quickly perceive changes, Id alone patterns, EFTA01181683  US 6,639,614 B1 5 6 unless several bon, adiacent stocks happen to change too Instantly (tenpin:2d as produced by a teas ralinfackay slmnitaneoesly. The rnadvalent percentage change in a condition. Ruthermore, a flower can wave in proportion to done "small" stocks would not /yen be visible, albeit the breeze, and the fertility of the area around the base of a noteworthy, unless °iceboats., all happened within tiresome Hower may convey additional information regarding the arbitrary industry segment (i.e., adjacent nactenglos), and s local trends only if dm tees happened to ho closely monitoring that area Similarly, an auditory displayer the system selects sounds of the nap. This defeats the whole mope% of a usable that one easily recognisable as positive or negative (i.e., atortitorlog tool. good/bad). Tho sounds are then produced In loudoevo, The market map display is dhadvadageously static. location, and timing patterns according lo the underlying There Is no display motion or audible ceproentation to l O data. For example, loud and frequent birdeloirsing swank indkalo any sort of change, err whether the dingo is on the right side would indicate a generally positive reladvely good or bad, only got the uncludying data is envinmereel, ‘vhavran a roombitng of thunder to the left presently up or down from the slut. to fad, there is no would give a more negative impression. Musical sounds of Indication at all that anything in the display half has varkma histrUnienlit anal bruin amid also he used instead of, changed—the user most attempt to remember what was 5 Of in addition to animal or weather wends. previously displayed and mentally compare all 600 stocks By ming an tionlogically oriented graphical icon, a wide (or it least some of there) Wien Redisplay is next mfreshod. number of different parameters can be simultaneously Trarnitiom in the market map rectangles from red to black desolated. Motion and sound are used to highlight important to green are shown using owns than ten variations in color, changes, or to lend assurance that a status quo prevails. forcing a ism to perceive, toomnire, recall, and understand 20 In so illustrative Orthudialent, an icon of a daisy Is the sank differeoces, a damning Of not Impossible) leak con0gared to represent financial data such as relative change who under humane pressure. in Mode price from opening, absolute stock price, standard Fandoxkally, since the data Icon newt change the or deviation in peke, direction of change in price, volume of shape, the method limits the arnomil of visual information • trades, and thadderm hoods. These parameters are mapped that muWM tosehdly displayed about any emaciate, denim. 2 to the number and size of petals, the frequency of swaying Aside from the fact that SmortMoney provides illilfitiOnal motion, the color and length of rho stem, rood the color cof the information abort only a single dam metric pemenlage gram near the Mena The rumptherle of dingo is represented change since last close), it has several ether scrims draw- by the waving motion, what bigger changes maim the back. with reaped to data vipallaatix,. The market map is 30 waving faster. Similarly, when a sound occurs, the apprO- constructed using the fixed hierarchy of industry segment, Male Rower lights ape, and pulses for a predetermined market caps, and percentage chartMl, A user is helpica period with green for • higher lade, red for a lower t do when it comes to organizing the display according to other The stock symbol can also appear next to such blinking metrics that may be preferred by the user. Even if the viewer flowers for a prodetennitod limo. Hundreds of nth 'did- Wild &eke. MCC Of specific arrangenawfis or collections site can then be arranged on a display, and sorted (e.g., top for t000noting,theonly thing that ever chaeges la the mica. 35 to bottom) depending upon a user-soloctaWo choice of The map of the market oleos not take any advantage of the display parameter. Ilium Rowers can to gather sorted Into abilities of the human mind to perceive, memorize, and quad/ads of different types of Rowers, not just daisies, with comprehend limited amounts of salient information or to the type of flower representing a categorical variable, such detect charges of display motion. 40 as stoics from a particular industry. Positive and negative smarts are then superimposed on the display according to SUMMARY OF INVENTION whether stock( are being traded for prices above or below The present Singes provides a new method whereby their reaptieligro averages. combinations of "ecologically valid" (Le., occurring in Unifies the prior att, the present method includes melee- . everyday environment.) lanes and rounds ant used for 4s ally valid Icons with which a henna is likely lobe familiar. visualialim of large amosats of information. loom and This method includes a visual uniformity, salient data icons, sounds aro selected and combined to neonate natural low clutter yes higher diorautionality, and sound overlap. In recognition of pattern information. Aldasael Held of similar addition, there are alerts to clomps (both visual and icons repeserns • coinepooding fielder data sets, where the auditory), a palette of lopes, and selectable data for either a appearance of each icon illustrates the relative values of the so binary condition (red/green) or a scaled meager* of Vitia- underlying data. Within each icon, elements change in aim. tion*, Little SeperfleOUR inftitmeliall is displayed, unlike VR number, color, and motion to illustrate the respective direc- approaches, and since less detail is needed, more informa- tions and magnitude of changes of data within the eons- tion becomes recogroizeMe. There Is no 'change for change spending data set. Steneoploceic sound morrow; timing, shier siate nothing moves unless the corresponding data has spatial location, typo, and volume signal the user about 5 s changed. Lower attention is necessary since you're not general data situation, as well as the type and inmettante of "flying" anything, but rather planted dimly 011 the gawk individual data changes, enknolating the user's nuanizoa background with the flowers. This method exploits Inman during OVUM of significance. mental peewee of perception, memory Mel cognition The &Melina th an illustrative antax/iment of limeys:ken, raa dila icons can also be simply sorted (cg., lop to bottom) by icon suggesting a recognizable life form is abated as the rto wear aciacialito panoreas to give inhered onkr to the basis for virealization of a mviti-variato dal a set safe toms, depiction without the uffkaibility of peddined dimension- snub a flora and fauna, are known to grow, move and ality. change colors according to their inherent physical delemai- Outliers are really apparent from the behavior of the name. par example, a healthy flown such as • daisy has • lam, especially where large changes are indicated by green atom, and gets larger and bike, and Ins more pleb, 65 proportional changes in sound, coax, and motion. Like morally determined by the Ibex's onvinatateol. A picking out the image of a rabbit in a wheat field, dm human shorter, brown flown that has lost most of 114 petals untold mind quickly tunes out the Irrelevant visual data. The use of EFTA01181684  US 6,639,614 B1 7 8 ecologically valid visual loom (edam the learning lima and um. 8 tabelites a corremoedence between examples of operator otornion and capitalizes on the cognitive abilities display and audible elements and data characteristics in an of On human viewer. llkisintive embodiment; and Similarly, the use of natural sounds facilitates recogailiee 140. 9 diagrams a logical proems for creating au ©co. of positive or negative citanzataras or primed changes. 5 logically oriented display according to an illustrative The Mohnen, volume, solociloy, duration and other embodiment of 030 system. attributes of the sounds arc easily mapped by the brain to remand the corresponding Inflamelion regarding the mag- IMTAILED IntgettlIrflOtt nitude and typo of Mirages aimoltamously occurring la the visual date The presence of the sounds allows the met to to The present lunation provides a new and useful data lock away from the computer Suseo, while being able to visualization system whereby displayed eounda, visual liana to sounds that indicate a change in the data that may pattern changes in the sounds and patterns, and motion din requ tion selected in a manner Mal provides eadogially valid:Mina imikve int wale or other process-orioned visualization Unl for quickly identifying changes and noting trends. The rimilawk, the ecologically valid Wahl and *edible loans can recomilion of changes in data, their magnitude, type, and be easily adapted to represent numerous types of data for Is direction can be simplified by using appropriate visual and )vhkh patterns and changes axed be quickly and efficiently auditory symbols 'Jul him into steal Ili mini of cog- recognized The artalysis is (booed from the oramoue nitive alai pronoun( processes as well as their entledying correspondence for which intuition or training are brain mtlehasiiSMS. The goal a to design ir display that Imperative, as amid be required when tang a leas:stinta exploits what Is known about bruin mechanisms and Flu- The same operator, using the tame team, can apply the tool " ciplmof graph design loonier to pmvidc both a sense of the to many &Items types of data, red drag the neesseary overarching patients in way huge sets of tau and the information quickly, he it weather phenomena, Mock existence of specific cases that diverge from the overall mallow, is other dela intensive analyzes. without Matillan patient. 'Thom brain ineelanions are wad to mermeln, ity texterstseding the underlying protases that creak the remember, and understarid objects and everts in our data, and urtencambered by the analytical Nejedioes 33 enVitOnentits, both mural and anornitle. spawned by such familiarity. A use interface design incorporates three key feel/mem Displays created according to the Inventive medial me the display is etionniClat to Ms advantage of brain mecha- Interaionallylinited with respect to the anoint of informa- nisms that evolved to help our specks survive; both visual tion that can he directly determined from inspection, in an yr and auditory "ecologically valid stimuli" ate toad; *of the effort to focus upon the most relevant fedora, their changes, principles of mph design are applied. In particular, brain anti the direction and magnitudo of mach changes. Because mechanisms that under& survival skills, such as those that of the uniformity of the many iota, they can quickly be allow us to hunt for and gather food and avoid predatory, aro compared wills each other for ehrages, while sill observing of interest. CIfedive hunting requires not only the ability to general treads, if mammy. Directionof change is hinariscd, 3s monitor htformatiou representing a malic landscape, but also with the magnitude displayed only for selected changes spot and interpret a charge quickly and accurately, such as (petals, stem, speed, sound volume), thus limiting the a moving Its In a forest. enamel of refined elismimination required by the user. Thu displays rely upon images of objems (flora, firma, Menges are indicated not only wkh virtual movement cal. human artifacts and other complex NMI dyniwlo objects) added to stimulate the visual perception per so, but also =luring in the natural landscape, and everyday environ- with No simultaneous sound ocatirenee Movement speed. meats in well as munch that our specks In adapted to sound volume and location, and sound selection draw the register cosily. Our brains evolved to recognige, remember, tutees ancintke to the other ionargancous caws ISO have and mason about suds objects and crest, quickly and occurred (petals, stem length). This, the occurrences of accurately, awl the system makes use of these capacities as mukkihnetaional changes are quickly recognized, along 4s epplkd to the display of data. By further combining the with the °maim and Interactions with other data sets. principles of graph design, the displays an bo designed to 'Minn OFSCRIPTION OP 11111 PURIM*: present information clearly. Unlike other displays, those designed according to the present system are compatible These sod other features of the reseal invention will he with the perceptual, memory and reasoning brain mamba- better understood in view of the following detailed draw ip. s° mess that we use to perceive awl interpret everyday stimuli tion taken in conjunction with the drawings, in which: in our environments. MO. 1 flimgraks the dements of a viarral icon according The system disclosed provides a multi-dered cognitive to an illustrative embodiment of the splem; environment for pesentation of data: the auditory indica- #1103.2a-2e Illustrates the variation of a display ernowit tions of change, the vista display pattern, and the rola- of the visual icon of FIO. I; ss tionship between the auditory and valet displays. Misery Mae. 3t. &Iiamb arca another vadat:au Me data-display changes alert the rimer Out "something it, happening," at the eternal of the icon in FIG. 1; very least. Visual patients present very large data sets, P10.4 Muammar another variation of s display element of allowing discovery of the soarco of activity and changes the Icon in FIG. I; among data elements. Once a change has been noticed, n PIG. S illustrates the application of a color variation of an to mot immix* of the system provides further features for icon element of HO. 1; selectiag or enhancing the changed icon for display of Pf0. 6 likuurales another application of color vadatkm of Naha detailed Information. another podiost of a display icon as shown in the embodi- One illustrative embodiment of the system is a display for ment In FIG. 1; preacessaloa and tracking of financial stocks. Auditory and Pf0. 91Lustrates a portion of a system display in which 65 visual displays are than designed to signal not only the a plurality of display icons is arranged in an illustrative cursing global stale of the data, but elm lamp-scale changes, embedimeng as well as important changes of individual aides. The state EFTA01181685  US 6,639,614 131 9 10 of approx Leta tely 100 stocks of interest, sad a global stale of for its information corneal. In addition, it side-losidenaotion none aerate Vein 04 she storks can be tcpeeitonteti using the and la/irides color can be used to highlight varietal changes features of a personal computer having a color video display to other elements. and a stereophonic sound.produeing inecisinien mich as As shown in FIG. 2, the number of petals in the heed of speakers or headphones, Data can too fed to die system, the daisy can he changed In 'idlest° the incremental changes either in real time, or played back from a storage medium, In stock price PIG. 2A shows • 6-peal daisy which indicates as the user's application may require.110 auditory display that the price banal changed. F1O. 28 shows a flower head indicates the meet value of the stocks, changes in this value in which them are two petals missing, thus indicating a (e.g.. with raped to opening mks), and largo changes (up or clown) of individual stocks two-paid change down from the opening price. FIG. 2C I0 shows a flower with 12petels, indicating a six point increase Audade changes in the mixture of "positive" mul 'nags. in price. Ibis anagemeat permits the direct display of the live"sourdssigwl thestateofMedals sea, and Iced changes range from 1 to 12 petals Indicating 5 paints down or 6 signal largo nioVentOon in an individual Mock (with differed points up. Each petal can also represent a dung* of 2 sounds for up or down). Smuts ate distributed randomly dellars, or some other petals en also bo wed In over 30.second windows, with one event per appropriate is indicate percent diangt, with each petal CafteSpOndina to a measure. Each of the evens lasts 2 seconds.Id toms raised fixed percentage. in Western celestes there is atm a natural scale having negative Method to the left (slated) and positive elements MO. 3 illustrates an example of changes in size of the to the right. Negative sounds ate selected front a set of flower head to indicate absolute price of a stock F10. 3 preoonligured sounds and played with a left spatial cA show, four Piestrativo theremmor of sive, alihoogli people orientation, in, the loft speaker. Examples of negative can notmally perceptually register and memorize up to sounds ale raindrops fur negative stook-mks hada al ten seven different Sizes, sod the additional sites are not flies- than a half standard deviation (03 SD) below its adrage, inkcL FIG. 4 shows seven different lengths of a atom whooshing wind for less than I SD below average, and a element of the daisy icon, each indicating the present crack of thunder for a stock traded at less than 2 SD below student deviation of vo