the_skua blog

For anyone who has been paying attention to the Fill in the gaps — bird the road less traveled thread on eBird and Paul Hurtado’s Google+, I thought I would post the two quick maps I’ve made with Geocommons that give these maps some interactivity. The first is Checklists by County for all time, as of December 10, 2011. The second is eBird checklists per Square Mile by County, as of December 10, 2011. Hopefully, these are a quick preview of things to come, as Brian Sullivan has promised a full data dump that should allow me to produce a custom tailored map experience for helping eBirders bird the road less traveled for each month of the year.

Talking tools, I will likely use the newly released Windows version of TileMill to generate tiles for the layers and then try to mash something up using ExtJs 4 and OpenLayers, a combo I’ve been intensively involved with for another project. And while ExtJS has better cross-browser capability, I find jQuery faster, so I’ll cross that bridge when I get to.

The team at eBird recently released a new mapping portal. The user interface is so, so, so much better, I cannot underestimate this point. It’s slick, well-done, clean, easy to use, modern, and many, many steps above the previous method for mapping eBird data. I love the fact that it’s in a multi-scale, interactive form and that it will zoom to the species extent. The mercator gridding is nice too. The team did a fantastic job with this upgrade.

New eBird Maps

My issue, of course, has to do with the actual representation of data on the map. Since their new mapping portal uses a REST service to serve pre-generated tiles of gridded data, it was easy to scrape an example tile, pin it to my own Google Maps and tinker with the colors a little (using Adobe Photoshop), to see if I could make something, that was in my opinion, a little bit more readable.

Old version of eBird maps.

Personally, I have a hard time extracting pattern from the new representation. For starters, I think the purple is okay, but what was wrong with the previously used green? I think it was a good color. In the new version, the “no reports” grey has been darkened and the feature color scheme has been compressed, with a darker low sighting frequency (or pale) color and a lighter high sighting frequency (or dark) color. This gives the impression of a divergent color scheme, with the brightest, saturated purple in the middle being most salient. For me, the palest shade of purple gets lost and confused among the dark grey squares of “no reports.” I can turn off the grey “no reports” grid, but then it’s a little harder to understand where there were no birds and where there were no observations.

What this problem highlights is the use of color gradients for mapping on computer screens. Darker, heavily saturated colors close to black are actually harder to notice because of the nature of additive color on digital screens, whereby as color is added, the result is lighter and tends toward white. So, pale, whitish tints actually can be more visibly salient than dark, saturated, blackish colors. I think this phenomenon helps explain the popularity of heatmap color schemes, where the highest values are the brightest white. Visually, though, a saturated bright red, or green, will stand out. Subdued darkish purple, to me, not as much.

Follows are screenshots of the original (top) with an example I’ve tinkered with (bottom), just one tile, for American Goldfinch in the northwest. Same, exact data.

Original, for comparison.

Updated version.

By using the paler white for “no reports” the image becomes less heavy and allows the observations to stand out more. “No reports” data is slightly less important, so it can be less immediately salient. For the color scheme, the palest color is still very noticeable, but since it’s closer to white (and less color saturated) it fades to the back a little and lets the ramp up to a bright, saturated green show more strongly than the compressed purple scheme.

I’ve also reduced the overall transparency, to allow the figure (the data) and the ground (the map) separate a little bit, which was another problem for the purple original. Ideally, a transparency slider would let the user adjust these transparency levels (for everything, including the map background). Often, thematic maps really need a simple, plain basemap (like ESRI’s new plain gray or rolling your own with TileMill). I understand the use of Google Maps here, because once you zoom in enough, you get individual observation points and want to see, in geographic context, where they are on the map.

So, do you think my tinkering was an improvement? Does it work? What would make reading the patterns in the data easiest?

Every now and again, I see maps trying to show an interesting point, but failing to do so because the map or graphic fails.

This morning, I came across this paint-by-numbers map (above) from Mother Jones, attempting to show how many lives will be saved every year by the EPA’s new emission standards in eastern states. Unfortunately, since the number soup is not actually a full-on thematic map, the pattern is lost and the message becomes not so clear (other than, yes, lots of lives will be saved). I took some time and made thematic maps out of them.

First, a simple makeover of Mother Jones map using the raw number of lives saved, with three classes and a simple legend. It’s interesting to see, that based on these raw numbers, the old rust belt will benefit the most.

Adjusting by state population, however, we see that these new standards will have the greatest impact on lives in the Ohio River Valley and Appalachian states.

Took me about 2 hours from start to finish (mostly data scraping and formatting, along with some initial searching for a fast web-based tool to make these – all of which failed) and it produced a clearer message, along with the revelation of some more thought provoking patterns. I think it was worth the time.

XML to Object

As a developer, I’m always learning. And I’m always looking for faster, quicker, easier, more efficient ways to do things, especially if they are awful, like parsing e4x XML with ActionScript. I don’t know how I hadn’t used this before, I but I recently found that it was possible to use the SimpleXMLDecoder to convert a tree of XML to an ActionScript object. AS3 documentation indicates that the SimpleXMLDecoder and required XMLDocument classes are outdated, suggesting the use of the e4x standard XML object. However, I couldn’t find any references on how to accomplish something similar with the XML object. Does anyone know if this method has been updated?

I find this method particularly delightful for handling GML returned from a WFS. As in:

var x:XML = new XML(event.result);

var xd:XMLDocument = new XMLDocument(x);

var decoder:SimpleXMLDecoder = new SimpleXMLDecoder(true);

var results:Object = decoder.decodeXML(xd);

var ac:ArrayCollection = results.FeatureCollection.featureMembers.reportvalues;

The event.result is the result from an HTTPService (something else I recently learned about using for transactions with GeoServer) request to the WFS. Even better than the fact that this works at all is that the object that comes after.featureMembers is already an ArrayCollection! The tricky thing is that whatever comes after .featureMembers is going to have a different name, dependent on the GeoServer layer you’re targeting. Otherwise, it could be even more automated.

Of course, if you were using OpenScales, this wouldn’t be as much of an issue. But for Google Maps users, it’s pretty slick.

Google Maps Quasi-Layers

One of my biggest critiques of the Google Maps API has always been that there is essentially no layer management. If you want some sort of layer management, you’ve either got to go OpenScales, or create objects for storing those “layers” and managing their addition and removal constantly, which is pain. If you have multiple “layers” this can get tricky. Fortunately, you can manage markers and tileoverlays separately without holding onto the instances in stored objects.

To clear just Markers:

map.getPaneManager().getPaneById(PaneId.PANE_MARKER).clear()

To clear just Polygons or TileLayerOverlays:

map.getPaneManager().getPaneById(0).clear()

While I have yet to formally introduced my open-source code project, SkuaTools, I continue to slowly add to and improve it. I will just say that it is a collection of various Flex 4 components that I have built for different projects, wanting to bring them together for potential re-use and to make them available to other developers. Before today, the project contained an AS box-plot class, classes for laying MapBox and TileMill tiles onto the Google Maps for Flash API, and an image-swipe for comparing before and after photos. Today, I’ve added a “Scented Time Slider,” which can be seen here (with source-view).

In the past, I have created many different time sliders, for navigating animated maps, but never unified the pieces into one class for re-use. Thankfully, my most recent project with the National Phenology Network, creating a visualization tool, is open-source, so I was able to extract a time slider that might see some re-use. The “scenting” involves placing a simple sparkline above the HSlider bar, representing the count of objects for the current time on the time-slider (or within a moving window). Obviously, if you’re mapping non-quantitative data, this “scenting” is less useful. I have, however, already managed to put it to good use in my Ruby-throated Hummingbird map.

I will be the first to admit that the skinning method I used to update the month and year hashes on the HSlider is far from elegant. But, it works and isn’t too clunky. The time-slider has an embedded moving-window (with relatively-sized slider handle) and adjustable start and end years, which will update the “scent” as they change. There are three simple rules for employing the ScentedTimeSlider MXML in your own project.

1. The sparkline data array needs to be sorted by date.
2. You need to set both startYear and endYear in MXML or AS declaration.
3. You need to call drawGraph() in the component anytime the sparkArray data changes.

If you’re on SVN, check it out on the source page, or snag the most recent SWC and source archive on the downloads page.

Inspired by Lanny Chambers’s “Spring 2011 Migration of Ruby-throated Hummingbirds” map (above) at hummingbirds.net, which I’ve seen in the past, but picked up again thanks to Nate Swick’s post on the ABA blog, I gathered up some pieces of code I had laying around and put together an interactive version, using the Cornell Lab of Ornithology eBird v1.1 API and a range map from NatureServe via Penn State’s Center for Environmental Informatics. I forgot how much I absolutely enjoy web-map programming when it’s about birds!

Click the screenshot above to go to the map at theskua.com/ruhu. A few notes about this map. It’s based on only the past 30 days of sightings submitted to eBird, including provisional sightings which have not yet been reviewed and/or approved by regional editors (an example of such a sighting, at the time of press, is the lone sighting in central Arkansas). Every time map is loaded, the sightings are updated from what’s been submitted to eBird, so this map will change over time.

I have learned a couple of things when comparing the map that I’ve created with the one on hummingbirds.net.

1. Data – Lanny uses a custom report form where people in his network can submit sightings as they first see the birds, he reviews them and then updates the map. This sort of focused network of observers, directed at one species and one seasonal phenomenon has produced a much more robust and thorough representation of the underlying migration in “real time”, than, apparently, the eBird data can. The emergent properties of the eBird collection mechanism are apparently not immediate enough to show the true front of real-time Ruby-throated Hummingbird arrival. I was worried that a chunk of more recent data was missing, but looking at the map generated at eBird for March 2011, the pattern appears to be the same.
2. Editing – Lanny’s map is hand-edited. He is selecting sightings to represent a “first arrival” for a general region and as a result, there are clear bands of arrival dates, whereas the eBird map is just a splattering of available sightings with some pattern in temporal distribution. I think the arrival pattern would be more evident with the careful hand of selection and summarization that Lanny does with his map. However, that takes person-time, patience in editing and vetting, and expert knowledge, characteristics raw crowd-sourced data sources often lack.

I had a great time doing this and I think it was a valuable exercise in comparing expert knowledge with the knowledge of crowds in representing a phenomenon like the arrival of Ruby-throated Hummingbirds. If there’s one practical bit of advice that can come of this, it’s that eBirders of the mid-South should get out there and submit their sightings early and often!

Update (11:00am 3/30): Looking at the comments in the ABA Blog post, I learned of Journey North’s work in tracking hummingbird migrations and that they also have a near real-time updated map of Ruby-throated Hummingbird migration that uses a similar method of data collection and presentation as hummingbirds.net.

Update (12:00pm 3/30): I’ve updated the map to use the temporal classing scheme from Journey North, since it had better temporal granularity. Also, I added a “Moving Window” function to the animation, which when used at the 5 or 7 day level really shows the northward progression nicely!

For some reason, I’ve always been attracted to MapBox, despite the fact that GeoServer handles the task of generating and serving tiles. However, the release of TileMill appears to make styling layers much easier than anything GeoServer can offer with SLDs. OpenGeo’s Styler essentially lost to TileMill in the race to create a CSS-like language for styling maps for the web. (*Edited to correct the fact that Styler is a WSYIWYG editor, not a CSS-like implementation). While GeoServer has a CSS module for converting SLDs to the more human readable CSS, TileMill created a CSS-like language and interface for styling tile layers on the web. By using Carto, TileMill brings tile layer styling from painful in GeoServer with SLDs and expensive and buggy in ArcGIS to presumably easier in TileMill. That is, of course, if you’re running Mac or Linux. As a result, I haven’t actually used TileMill, yet I have faith there will be a Windows version eventually.

With a some good-looking MapBox layers out there and the potential to easily create your own nice looking layers with TileMill, I wondered if I could pull these layers into the Google Maps API for Flash. Of course, Google Maps does offer a StyledMaps option, which uses JSON for editing the appearance of the basemap. However, you can’t add your own data to the layer, only edit what Google offers. Obviously, there are many instances where you’d want to add your own data for custom tile layers. With this in mind, I created two ActionScript classes that extend TileLayerBase for the Google Maps API for Flash, allowing the addition of MapBox and TileMill layers to the map.

A demo can be seen here, and you can download the classes as part of my newly-minted SkuaTools Google Code project (or grab the source). Maybe someone else has done this, but I couldn’t find any examples. Or maybe others have always rolled their own to do this, because it wasn’t that hard.

In one of those simple twists of internet fate, not longer after A_C_Robinson tweeted the need for redesigning the National Weather Service’s (NWS) warnings map, the NWS announced a new, experimental weather web-map, seeking comments and suggestions. I finally got around to sending them a number of comments and wanted to share those here, in an effort to stimulate discussion and encourage others to provide cartographic design comments to NWS, in an effort to improve a web map that gets a lot of use by a general audience. I cleaned up my comments a little and organized them into three categories: general cartographic design, color choice, and interaction design.

Cartographic Design

The line work on the counties layer needs to be cleaned up a little. Big heavy black lines make it hard to look at. Try a thinner, partially transparent white or pale gray line.

Not all warnings are as severe as others. I think there is yet a better organization of warnings, watches, and advisories based on overall severity concern to the potentially impacted.

Color Choice

Bright yellow is a hideous color for mapping online. With more advisories than watches or warnings, it’s on the map often and that makes the overall readability poor. It’s the most noticeable color on the map, when the color for warnings should be the most noticeable.

Combined with a better structure for severity, a better color scheme will highlight the important severities and make the map more readable.

This map has problems for the color-blind, which is almost 6% of the male population. For them, the orange and the yellow are tough to differentiate and the Flash Flood Warning is indistinguishable from the overall warnings category. Further, the Radar layer adds further color confusion, making some areas of the map unreadable. View as a three different types of color-blind with ColorOracle.

ColorBrewer2.org provides tested, perceptually-sound color schemes. While not completely color-blind safe, I might suggest using a diverging Spectral scheme (the last on the right under Diverging), with up to 5 classes, assigning the red for the most severe, the pale yellow for watches, and the blue or green for advisories.

Reserve your brightest, most intense, most saturated hues for the really important warnings, such as Tornado or Severe Thunderstorm. Right now, it’s backwards. The advisories are the brightest at yellow and the tornado warning is dullest at purple.

Interaction Design

I like the Contrast Layer, it helps bring the warning colors up and pushes the less necessary geography into the background.

It would be really nice to toggle on and off the different categories of warnings, watches, and advisories,  possibly loading the map with only warnings and watches.

It would be nice to be able to adjust transparency of some of the layers individually.

There’s no way for the user to know that clicking on the map will bring up the warnings and the forecast. Better, have a mouse-over pop up a quick warning bubble, with a link to expand to forecast in that bubble if they want.

I found the map slow to load when there were lots of warnings and watches. I’m guessing that this is due to the detailed geography that’s being loaded for colored coast-lines?

A quick update to note that I am now working independently. Currently, I’m on contract work through Penn State that’s likely to last into May. Otherwise, I’m open for business and am always interested in research and programming projects that have some geovisual element.

Aside from the Flex-based contract work that I have, I’m going to be once again striving to wrap up two publications from my master’s thesis – the first a domain-driven typology of movement patterns found in aggregated point data and the second experiments on symbolizing change with bivariate symbology in map animations.

When I was a young kid and my family visited my grandparents in Syracuse, New York for Christmas, my father and grandfather would take me downtown to see the weather beacon on the MONY Tower. We would recite the jingle to decode what the weather was forecasted to be:

Green light…weather bright. Orange on high…overcast sky. Orange flashing…raindrops splashing. Flickering white…snow in sight.

Apparently, weather beacons are neither all that common nor all that popular any more. Reasonably so, as weather forecasts are everywhere: the radio, the newspaper, the internet, your mobile device. I did always enjoy seeing something public with a color encoding for (weather) data.

I recently hung some seasonal lights on our apartment porch. These LEDs have a remote control which can be used to change the color display, making me think that they could act as a similar beacon. Of course, I would have to market a jingle that people could use to decode the colors, but it got me thinking about how weather beacons may have been one of the earliest forms of color encoding of temporal or dynamic data for public consumption. Certainly others exist, but how many of them are that public or temporally dynamic?

As augmented reality moves outside of mobile device screens into the real world, it will be important to think about how color is used to encode data on objects in the real world. Perhaps jingles such as the one used for Syracuse’s MONY tower will again become handy verbal legends to such representations.

P.S. Christmas with our entire family was neither that easy nor idyllic; it took a certain amount of Canadian Club to get my grandfather out the door.