Making data easy on the eye
As scientists, we are prey to a self-delusional species of hubris: we imagine that we are above the emotional and instinctual whims that drive much of human behaviour. It isn't so; our thinking is inexorably conditioned by our perceptual equipment and our biology, both of which are inherited from all those millennia of pre-scientific existence. This is nowhere more true than in the visualisation of data, where a lot of intuition, mental substitution and perceptual assumption come into play.
Psychologists have well-developed models for our responses to visual stimuli in general. The habitat theorists add a functional one for our specific response to anything resembling the outside physical world, in terms of tension between our simultaneous hunter-gatherer desires to see and to be concealed. Our reaction to either a concrete and 'real' landscape or an abstract representation of one 'stems from the spontaneous perception of landscape features which, in their shapes, colours, spatial arrangements and other visible attributes, act as sign-stimuli indicative of environmental conditions favourable to survival, whether they really are favourable or not'*. In down to earth terms: the human biochemistry responds to a well turned chloropleth or a particularly peaky or chreodic surface plot (especially one with a strongly tangential pseudolighting effect) as it would to the equivalent features of a real environment. This may not seem to have much to do with your practice of science, but don't dismiss it; it has a real and deep influence on how you and others 'read' the visual representations of your data. Designers of visualisation software are aware of all this, and address the psycho-perceptual consequences in their products. The philosophy is nowhere so relevant, though, as in the programming aimed at geographical users.
The US state of Colorado seems an appropriate place for such software to originate. The town of Golden, not far from Denver or Boulder, lies beneath the Rockies (you don't get peakier than that), surrounded by a landscape with enough chreodisation and plateaux to stir the blood of the most somnolent couch potato. The partnership at the heart of Golden Software is a familiar one in the mythology of software histories, a student and a member of faculty who set out to solve a particular problem and then went on to market the general solution. It is probably no coincidence that the partnership was sparked within the Colorado School of Mines. Their software itself is a model for construction of tools that intuitively harness environment responses to visualisation design. Four products in the Golden portfolio address different aspects of data representation; they share a strong family likeness, but split naturally into two pairs.
Most recently upgraded, at the time of writing, is Grapher 4. A mainstream package for two and three dimensional technical graphics, aimed at the midground of that market alongside such competitors as Axum, SigmaPlot, and the like, it has some particular distinguishing features of its own. Related, in many ways, is Surfer 8; this is aimed more specifically at the geographical user but is still generically applicable to any multivariate data that is susceptible to pseudo spatial modelling. It produces three dimensional surfaces or two dimensional projections of them with any combination of contours, zoning, data posting, flow vector mapping, and so on. These two represent, in many ways, opposite ends of a visualisation spectrum.
The other pair in the team are more strongly geodata oriented. MapViewer turns data into themed maps for display, while Didger facilitates the turning of maps (paper, GPS generated or electronically imaged) into data which can be used for the other three. (This seems an appropriate point at which to confess, by the way, that I am human enough to like Didger just for its name.)
All four products use worksheets in the usual worksheet mould. Grapher and Surfer treat base plane coordinate columns as rectangular (orthographic in Grapher's x,y,z plot, orthographic or by visual perspective in Surfer). Didger and MapViewer add specialist handling, modifying the reference frame to a number of surface projections (see box, 'Projecting the right image', below). In studies of small areas, this makes little difference; with nongeographic data it is, of course, irrelevant; but as the dimensions of a geodata study area increase, the quality of projection handling rapidly becomes crucial. The ability to convert displays from one projection to another is not, of course, unique; where Golden has the edge is the ease with which these conversions can be applied to a complex multidimensional representation.
Didger extends this to other representations of the (x,y) frame; in particular, 'rubber sheet' distortions (or, perhaps more accurately, corrections). This facility is valuable in a number of ways for bringing digitised data into conformity with a predefined reference grid. At one level, it allows correction of arbitrary distortions (usually compressions or stretches in either the x or y dimension) produced by the scanning of a document, or those (often 'pin cushion' in form) resulting from shrinkage of old paper documents. At another, it provides an easy way to compensate for constant or variable systemic distortions; the perspective effects in many aerial photographs, for example. With a small group of graduate and undergraduate students from various disciplines, I took all four products out into a group of wilderness habitats. The objective was to generate visual materials which would show to funding bodies and a nonspecialist public the interconnectedness of physical, chemical and biological environments in a range of conservation efforts.
In the visualisation of multidimensional or multivariate data Surfer is a joy to use. And joy is not to be sneezed at; aesthetic response to visualisations is an important factor in functional effectiveness, in generator as well as perceiver. No special preparation of data series is necessary; unlike some software, raw triads are taken directly for true surfaces or wire frames without the need for explicit intermediate gridding. A wide range of plotting options is available, most of them what you'd expect in a high end product of its type but well implemented. The rotation and 'fly by' controls are good, too, preserving the landscape illusion by staying transparently out of mind. Components from different multiple-y variable sets, generated on a common (x,z) base plate, can be overlaid directly or, for greater flexibility, generated and manipulated separately since the rendering is absolutely precise. The objects themselves can effectively be of any size subject to system limits; using Surfer to prepare an exhibition, we produced transparencies twenty thousand native image points square. In many respects, Surfer is the de facto flagship product of the group. Though strongly developed 'hands on' exploratory aspects are its great strength, a full BASIC dialect scripting language is there as well.
- Different aspects of a multivariate data set from a habitat study are successively visualised (left to right) in Surfer, before producing the final transparency (far right) which superimposes data representations including soil nitrate density (shaded green ground), susceptibility to flooding (blue blocked areas), small mammal high activity zones (pale green contour lines) and various burrow locations (red geometric symbols).
MapViewer, by contrast with Surfer, thinks of itself as purely cartographic and in a sense it is; but cartography is only the two dimensional representation of three dimensional forms to a particular set of rules. A coastline or other boundary, a river, or whatever, is a set of data points, a polygon, or a curve; and these can be defined as you wish. Your base map features do not have to represent a geographic entity; they can interpret the base plate of any three dimensional data construct which you consider appropriate. One of the transparencies prepared for the conservation funding report was constructed in MapViewer on a base map (digitised in Didger from a photograph) which modelled the side of a fish; the surface modelling represented parasite activity, antigen responses and chemical disruption. By moving the point of view it was possible to present the data in ways which synergistically combined both 'fish awareness' and landscape instincts in the viewer. Geographic material will of course be the main usage; but it pays to be creative and to think laterally when selecting tools for most effective and productive visualisation.
MapViewer's displays fall into three basic types. First come the ones which simply place small conventional plots, graphs or charts within the subsidiary regions of your base map: bar charts, line graphs, pie diagrams, scaled symbols or simply posted data values. Then there are those which show data in relation to the regional structure - density maps which randomly scatter dots or circles within each region, number of dots being proportional to regional data value; flow maps which vary line thickness to represent data values; pin maps which show point locations with associated variable class codings and labels. The final group is those which modify representation of the regions themselves to convert the whole base map into one large zonal display; this may be by colouring or hatching the regions, by raising each region to produce a prism whose height is proportional to the data value, or by a combination of the two methods. The division between second and third types is not hard and fast; the dot density distributions, for example, could arguably be shifted to group three.
Graticule lines appropriate to the reference frame (eg x,y Cartesian, latitude and longitude, etc) can be superimposed on the maps, as can automatic scale bars in various units. The units available include some which will seem strange or archaic to European eyes (I was eleven years old the last time I used my knowledge of rods or US feet) but the standard SI units from millimetre to kilometre are there as well, along with Imperial equivalents and nautical miles.
If you have used any technical or business graphing package at all, Grapher will be familiar to you; it offers a range of two dimensional plots in Cartesian or polar form, and a basic three dimensional provision in the form of an (x,y,z) data point plot. Choosing between the similar offerings in this market is a tricky business; competitors evolve towards each other, every attempt at differentiation promptly replicated elsewhere. A world ahead of spreadsheet graphing and plotting, Grapher is perhaps a lighter but more focused tool than some of its nearest market competitors; Surfer a considerably more powerful one. As always with such things, it all depends on your particular requirements; there is no 'best' choice in any absolute sense, only the best choice for your context. Grapher's claims on your decision-making lie in two directions. Golden Software would, justifiably, point to the simplicity of use in Grapher; it certainly has the edge in this respect, on several counts - especially the access to property editing, multiple plot overlays and object organisation. The other argument for one product over another is fitness for environment; Grapher's biggest asset, in that respect, is the company it keeps. Like most software families, this one has a lot in common between its members; if you go for any of the other products in the line, Grapher is its natural partner.
One very useful particular example of this 'relatedness' is the ability of grapher to import data from Golden's digitisation program, Didger, which can also be used in Surfer and MapViewer. This reusability is an important productivity consideration. Data digitised in Didger can be exported in formats specifically useful to Surfer or MapViewer as well as Grapher, in addition to a range of various generic and proprietary formats georeferenced or otherwise. All the products in this range were new to me, and to those working with me, when we took them on as the study's tools. They were, nevertheless, immediately usable and within a very short time became familiar. The reason for this (and for the discursion into habitat theory in my opening paragraphs) is the intuitive way in which the controls manage to key into inherent reflexes and perceptions which are heightened in anyone used to working with or in relation to landforms. A good map has always been one which fitted itself into the same responses as the land which it modelled; these packages pull the same psychosomatic trick.
*Appleton, J., The Experience of Landscape. 1986, Hull: Hull University Press.
New on the block
Not having used previous versions of the software, all its features were new to me at the start of this trial. I can't, though, cover everything; so a quick tour through those features identified as new in the current releases, for readers more familiar with Golden's range than I, seems in order.
Taking the most recent upgrade first, Grapher has added a 2D plot types (polar, wind chart) and extended others (rose, ternary, bubble, bar) in this release as well as adding the x,y,z type 3D type extending some 2D plots to 3D rendering. Conversion from one plot type to another without disturbing the data is new, as are broken axes and natural logarithmic scales and calculation of areas under appropriate curves or scatter plots. File types have been extended, and work sheet edits can be passed seamlessly back and forth between Grapher and Excel; there is also an Excel interface. (It is Excel's fault, not Grapher's, that this harmonious picture of coexistence falls apart on grid size limits; Grapher, subject to system limits, accommodates up to a billion cases or variables while Excel does not!)
Didger now allows import of multiple bitmaps into vector projects, overlaying them with data files or vector files and adding graticule or grids. In raster projects, the correction or warping of bitmaps is a new addition, as is the wonderful implementation of bitmap projection conversions. GPS digitisation is new as well. New spatial transformation methods have been added, Windows imposed page size limitations removed, and a number of IO options (including new file types) added. User definable datum, new projections, and a series of editing options complete the tour.
In Mapper, the changes are extensive. Some file menu conventions have been aligned with Windows norms and various improvements in detail implemented. Mapping changes include movement and rotation of pin map labels, addition of a change file button to map dialogues allowing current map settings to be applied to new data, partial map display, extension of measurement units, subdivision of the first scale bar cycle, automatically updating to reflect data file changes, file reporting of centroid coordinates and general map information, and double precision numeric. Map display types have been extended, as have detailed handling aspects such as hatching. As with the other products, file types for import and export have been expanded. Projection handling is revamped, a datum transform tool added. A number of analysis tools have appeared, including 'buffer analysis' which allows specific analysis of data within a user defined zone around given points, curves and areas, automatic summing of the interlocational distances within a list, a query structure and database browsing. Map management is similarly expanded, there are interface improvements, and symbols are in TTF format allowing creation of new user types.
Surfer now offers modeless property dialogue - that is, they can be left open while working, their content updating to reflect context. Zoom, pan, tilt, roll, yaw and field of view have a 'joystick style' real time controls. New maps created by type have default properties, which speeds up creation. Colour surfaces, so central to the quality of the application, are new in this release; most plane type maps can be overlaid onto created surfaces, as can bitmaps, and there are standard colour palettes to represent particular surface encoding. Filter grids, provided or user designed can be applied and grids can be mosaiced. Data metrics and cross validation are accessible from the display, as is Delaunay triangulation using appropriate methods such as natural neighbour. Gridding can be by moving average or local polynomial, and models now include cubic and pentaspherical variogram. Other related additions embrace volume calculations, current SDTS format digital elevation, report enhancements, import and export extensions, and others. Once again, the 32 inch page resolution has been eliminated and up to a billion worksheet cases are available subject to resource limits.
Projecting the right image
Data plots usually assume that they are rooted on a Euclidean plane - that the Earth, in effect, is flat. In nongeographical data sets, where the x and y axes simply represent experimental variables, it is obviously the correct approach. It's also OK for most geodata gathered over areas up to about 50000 hectares or so; to worry about curvature of the earth at this scale constitutes delusions of accuracy. For larger areas which relate to real ground, though, the (x,y) shape of the plot will inevitably be a distortion of reality and this may or not matter in a given case. Once distortions are deemed to have become significant in a particular context it is important, for both theoretical and psycho perceptual reasons, to choose a projection most appropriate to the purpose of the study.
Grapher, like all similar packages, assumes a rectangular data frame. Surfer, because of its particular functions, projects either orthographically or by visual perspective. Didger and MapViewer, however, both support a variety of commonly used cartographic projection conventions. This is no place for a detailed discussion of projection theory, but the table gives a quick comparison between the packages and the projections which they make available.