Too much of a good thing?

Slashgeo reports yet another new Open Source GIS tool, noting "Open source GIS software is crowded with options: QGIS, uDig, gvSIG, OpenJUMP, MapWindow, GRASS, OSSIM, etc."

Yes, it is - and the question to me is: "Why so many?" - I understand the arguments for a diverse software ecosystem, and I even agree with them. But with the possible exception of GRASS, most of these packages implement the same basic functionality without implementing the advanced functionality that is necessary to seriously take on ESRI. Plus, the UIs are almost always terrible, which makes the problem even worse.

So, in that spirit I'm listing a few things that I think are missing from most, if not all, OS GIS software, in no particular order:

1. Topological data model (the only exception I can think of is GRASS)
2. Editing, especially centralized, version controlled editing.
2a. Which means you need a topology model in your database
3. UIs not designed by software engineers. (Well, at least ESRI has this problem, too.)
4. Quality cartographic tools.
5. COGO tools
6. Metadata editing tools
7. Scriptability (GRASS, being command-line based is fairly scriptable, I know of no others that are.)
8. Runs on my Macs.
9. Comprehensive interface for building extensions

Well, those are a few that come to mind right away. Make more suggestions - maybe we can turn this into a manifesto.

Open Publishing & Cartography 2.0

The new issue of Cartographic Perspectives is available for free as part of a test of going all digital and open-access. Three Cheers for NACIS on that one, let's hope they keep it up.  The lead article is by David DiBiase and makes the case for more open access. David and I organized a session at the AAG Annual Meeting in 2007 on "Envisioning a post-proprietary Geography" – sadly we were somewhat poorly attended, no doubt in part because of the free wine being given away by publishers at the same time. (The more conspiratorially minded among us might think that a bit *too* coincidental...)

The issue also features the introduction by some of my former classmates (go Anthony), of the Cartography 2.0 website – an online source book for dynamic mapping.

You guys are embarrassing me with all the awesomeness. Keep it up - time for me to get working, too.

Announcing: PDXBikemap

I'm pleased to announce the first public release of pdxbikemap.org -- it's a street routing tool for finding optimal paths while on your bike, where optimal is defined as "balancing speed and safety."

The application is still very much in beta, but please report any issues you run into, and especially send suggestions. But before you do, check out the FAQ and the known issues.

Enjoy!

Planet Money blows it

The most recent episode of Planet Money talked to an American oil worker in Angola and an Angolan about the differing ways of experiencing their country. This is a noble thing, indeed and I have no complaints about it – in theory. But I think they really dropped the ball this time.

In particular, there was a discussion about the Chinese government hiring Chinese laborers to build Angolan infrastructure as part of deal to guarantee oil prices to China, but they missed the obvious question: how many Angolans does the oil company hire vs. foreigners? Who's getting the $20,000/mo rent for the American's apartment?

But the thing that surprised me most was the way in which they seemed surprised at what the Chinese were doing.  This sort of thing has been SOP for aid-giving countries for decades (though they don't usually send labor). I once met a woman at dinner party who worked for a company that existed only because of aid tying. (The company made very large pipes for use in water/sewer systems), a friend of grew up in Egypt because his parents sold American farm-equipment around North Africa.  Obviously American farm-equipment is of high quality, but nonetheless, the people were "buying" it with American aid money and had no choice but to buy from an American manufacturer.

Similarly, the oil for infrastructure deal is as old as colonialism – think of the Reuters Concession in Iran in the late 1900s, or any number of "concessions" that were forced upon the decaying Ottoman empire in the 19th century. Similar arrangements were widely made in Africa. Chances are our American oil worker is working for a company with quite similar "arrangements".

In general, I think Planet Money has been fantastic at exploring economic issues (albeit with something of a tendency to gloss over or ignore serious critiques of the statements given by their various interviewees), but so far their work on the economics of development has really been lacking.

Come to think of it, I think maybe their thinking is just too narrow - much like economics in general, they try to oversimplify the world to fit into economistic models rather than looking at the societies and politics that economics is operating in. Like many economists they forget that economics is not physics - it's not something that operates independently of human beings. It is a creation of human beings – something we should always be careful to remember. This, of course, was so elegantly argued by Karl Polanyi decades ago, but is so easily forgotten.

Here's hoping that in the future they take a bit more care in how they report and who they choose to discuss development economics. I'd love to see them get someone like Ha-Joon Chang – but that might be hoping for too much.

Anyhow..

Buffering only the upslope areas of a DEM

OK, here's the solution to the problem proposed in my previous post.  It's not a perfect solution, as it doesn't account for all possible types of terrain (more on that below), but it's pretty close and in the data I have, works pretty well.

The first thing to do is take care of the slope and the 10' elevation change in one swoop. By adjusting the size of our pixels, we can make sure that any change in the slope that meets the 40% rule also meets the elevation change. In other words, what is the minimum Δx (i.e. the resolution) that yields a change of Δy>=10' at an angle θ?

• Because: y=x·sin(θ)
• We substitute: 10= x·sin(tan^-1(.4))
• Solving for x yields x=26.9. We round that to 27 for convenience.

• Now calculate your slope raster with an output resolution of 27ft (or equivalent, depending on your projection).
• Reclass that raster so that everything < 40% is null.
• Using your >=40% layer assign a value to each grouping (in GRASS, this is r.clump; in Arc*, this is the Region Group tool from Spatial Analyst).

This next part is hard because looping pretty much requires a programming language - I used GRASS (wrapped in a shell script), you could use AML or Python if you're an ESRI user.


For each group:

• Find minimum elevation in that group
• Buffer out 300' from the group
• Delete all pixels from the buffered region where slope is null and elevation is less than the minimum for that group.

Ta da!  You have a raster buffered out 300' upslope.

But wait....


What if a pixel is both downslope from one face and upslope from another? (Imagine a ledge or an old lake shoreline like the bench in northern Utah created by Lake Bonneville.)  Well, in this case, we will have eliminated those pixels, which we don't want to do.

We can solve this by taking an intermediary step - creating an empty raster, then instead of deleting the pixels, do a logical OR against the output of the previous iteration of our loop. This allows us to keep our pixels if they're within 300' of any upslope region.

Unfortunately, that loop can add some significant processing time, anyone have another way to do it?

Interesting challenge

Given that lately I've been doing a lot of ETL, which involves working with a lot of low-level data structures and data transformation, I don't get much chance to think about some interesting analysis problems, particularly raster based ones. But today, I got presented with one.

Using a DEM, create a 300' buffer around any slope greater than 40%, where the change in elevation from one edge of the slope to another is at least 10', but buffer only on the upslope side.

How would you do it?  I'll post my own answer soon.

Tweaking OpenLayers WFS requests to work with featureid

OpenLayers (at least as of 2.8) WFS requests always include the bbox, but that doesn't work if you specify a featureid (the two are defined as mutually exclusive).  Here's a quick work-around for this problem using apache's mod_rewrite.

I'm using apache as an AJP proxy in front of geoserver, but there's no reason this wouldn't work with other WFS servers like mapserver, etc. Put this in the appropriate config file:

RewriteEngine On

RewriteCond %{QUERY_STRING} featureid=[^&]+ [NC]
RewriteCond %{QUERY_STRING} (.*)bbox=[^&]+(.*) [NC]
RewriteRule (.*)  $1?%1%2 [P,L]

ProxyPass /geoserver/ ajp://localhost:8009/geoserver/

Introducing....

In what may be a fit of foolishness, I thought I'd create this blog to post things that I find useful in my day-to-day work, coupled with the occasional random thoughts on a topic of interest, usually related to the world as I see it (however skewed that may be).

Expect to see stuff on Economic Development, Programming, and GIS work -- especially if it's related to open source GIS.