Geo2Gov Mashup interview

“Map your address or latitude,longitude into your relationship with government”.

Adam Kennedy and Jeffrey Candiloro’s spatial engine Geo2gov won the MashupAustralia Transformation Prize for a mashup that solves a data problem for other hackers. Most hackers were spending an inordinate amount of their time dealing with locating, downloading, cleaning, formatting and otherwise overcoming problems getting access to the data they wanted to use in their application before they even got to start building it. Geo2gov was used by itsbuggeredmate and you can use it now.

What does Geo2gov do exactly?

AK: The simple summary is that geo2gov is a spatial search engine. The longer version is that geo2gov takes a location in a variety of different formats (address, postcode, suburb, place name, ip address, etc) and converts them to a GPS location, then drills that point through half a dozen spatial layers to identify federal, state, local and ward level locations, as well as your statistical location within the most recent census. Finally, it expands these basic identifiers with information on elected members and other data. [so you can link through to to find out what they are saying and doing]. For the techies, the notable part is that we use a highly tuned and somewhat unusual database schema that lets us complete the entire process from end to end in around a quarter of a second. This makes geo2gov fast and cheap enough to be added to just about any website that would like to let users search for government information based on their address.

JC: At a more prosaic level, we’re building (have built) a piece of the infrastructure necessary for Government 2.0 applications.  Knowing where a user is in relation to government services is critical for many of those applications.  We’d rather they use geo2gov and then spend their time building wonderful new tools and applications rather than trying to re-solve a difficult but solved problem.

What I would be interested to know is you experience in contacting the originators of the data you used – did you speak to them direct, were they are the hackfests? You used a phenomenal number of datasets! Were some better than others? (don’t need to point the finger but any pluses and minuses would be good).

AK: Dealing with the government has been a varied experience to say the least. Things have been generally positive at the federal level, but much more mixed at the state level. Access to data gets harder as the states and their Lands Department budgets get smaller, which is understandable. Our experience with mining-heavy states has been more problematic than it should have been, as they seem to have a mercantile attitude to their data. Fortunately, all states are required to provide several types of geo data to the Australian Bureau of Statistics, who then release the whole compilation as Creative Commons. This has allowed us to sidestep a number of problems with small or difficult state governments.

JC: The other thing we have encountered is that some government agencies don’t know what data they do have and/or who is in charge of it internally.  I’ve encountered situations where what is on the website is the province of the ‘web-people’ and completely separate to other business units.

I understand from Pamela Fox that there were some pre hackfest workshops that you attended, which was where you came up with the idea of the geo2gov so that it was ‘ready to go’ at the actual hackfests.

Can you tell me how you heard about these and how they worked.

AK: For the most part, access to these hackfests was about being on the relevant mailing lists for the larger events early enough to hear about them. The smaller mini-hackfests we went to were generally organised around a specific theme, such as Open Australia or learning how to use Google Maps. One of the main benefits was that we were able to get a feel for what types of entries other contestants were aiming for, resulting in our decision to pursue a specific niche project that was both sorely needed and which nobody else was attempting to solve.

JC: The deeper roots of geo2gov can be found in a project I started early in 2009.  I wasn’t working at the time and figured I should do something useful with my time.  A solid technical challenge that also allowed me to give something to the community was what I was looking for.  Trying to figure out how to solve the problem of determining a person’s representatives based on their street address seemed like a good fit – no-one else was doing it and I had no GIS experience!

What motivated you to participate and clearly put a huge amount of effort into it?

AK: I’m quite sure Jeffery will have a much more noble answer than me for this question. I’m a sucker for competition. From a more practical perspective, I found it easier to commit a lot of time up front because our project had a very specific scope and isn’t prone to feature creep. Now that it works, it requires very little time to maintain.

JC: We live in a representative democracy which only works if the people are engaged with their representatives.  A tool that makes it easier for people to be engaged seems like the logical next step.

Beyond that, there was the technical challenge – can this problem be solved?  Then as we progressed we found all these other potential use cases for knowing where in ‘government’ you are.

What are your backgrounds – would you describe yourselves as being part of the spatial industry or more mainstream IT? Is this your first foray into geocoding etc.

AK: This competition was the first time I’ve ever had to work with spatial data. I’m a programmer and systems designer, and for the last couple of years I’ve been working as the architect and lead developer for the web team at Corporate Express.

JC: This was also my first real foray into spatial data.  I’m a project manager/technical lead most of the time, writing code where and as needed.  Right now I’m working as the web development manager for NetComm.

If more public sector information were freely available would it present you business opportunities?

AK: Every new piece of public sector information opens more opportunities than the one before it, because it might able to be combined with every other piece to produce something novel. Right now, most of the interesting combinations we’ve seen don’t appear to have business opportunities large enough to justify doing a startup. But each new dataset means more combinations and a higher chance we find something novel and valuable enough to move on, and I think we’re getting closer to the tipping point where the new government data will spawn new businesses.

JC: Of course, if any of your readers have a great business idea built on/using our service we’d love to hear from them!

Was there any data you would have liked but couldn’t get?

AK: Our weakest area is local government. We’ve got good coverage of the councils themselves and their boundaries, but assembling a list of every councillor in every local government was just way too hard, as the data is too highly fragmented.

JC: We’d also like access to historical geo-spatial data.  Ie, the electoral boundaries for the past 10 general Federal elections in a GIS format.  Unfortunately I don’t think the data exists in those formats.

What will happen to geo2gov now? Are people continuing to use it (do you have stats) and are you continuing development?

AK: Current usage is pretty light and ad-hoc but every few weeks we hear from someone else that is using it, or would like to use it. There’s a general consensus with Open Australia to switch over to us from their current postcode search, and we expect to see strong growth in different websites using geo2gov in the lead up to the 2010 federal election. The main area of development is around the identifier scheme we created during the project, which provides a way of allocating a unique id to things like electorates and members across all levels of government without the need for a central authority to allocate them.

We’d like to provide a way to use it as a unifying translation mechanism, so you could take something like a federal APH politician id and automatically convert it into a TweetMP name, or an Open Australian URL for that member. We’ve also been doing a bit of experimentation with the 2006 Census.

JC: We’re also thinking about what other data sets we could put in and other formats that we might deliver the content in to end users.  For example, we’ve been looking at serving up pre-rendered HTML snippets that can be embedded in a web page (with a bit of JavaScript) for those users who aren’t able to process JSON.

How will (or would) you keep the data current – will the dataset providers alert you when they have an update or do you have to go looking? Can you get updates automatically e.g. by RSS feed?

AK: About 80% of the work on geo2gov was the creation of a sort of data depot, so we could download and drop in data from different sources and then run a single repeatable program that extracts, normalises, marshals and combines the data into the final database that is used by the search engine. So while we do need to manually locate and pull new copies of the data files, the process of building an updated release requires very little work.

JC: The data doesn’t change that often.  The big changes (including changes to electoral boundaries) only occur every 3 to 4 years with a general election.  There are the occasional by-elections but they aren’t hard to handle.  One of the benefits of living in a stable democracy is that the governments tend to last full terms.

Your entry said ‘geo2gov can be easily incorporated directly into other mashups as a search or address box, and can be deployed as a virtual applicance locally in addition to the regular online Amazon EC2 cloud-hosted service’.

So is it currently on Amazon?

AK: Correct, although not because we need the power. We went through the exercise of putting it on Amazon primarily as a risk mitigation exercise. As a public service we are funding out of our own pockets, we were nervous about cost control. By building geo2gov as a virtual appliance from the start, if we do come under pressure from a high volume user we can shed that load by simply giving them their own copy of the server.

And finally the dollars. I am not sure how you make your living but is this just a hobby or do you hope to make money out of it – or at least cover your costs if you do host it on Amazon?

AK: Paying for a single Amazon server is about as expensive as the cost of playing squash or cricket, so it is tolerable as a hobby in the short term. In the medium term we’re confident that the service can cover its own hosting costs, or some high-volume user will step up and take over the hosting of the service.

JC: Plus, there is the notion of service to the community.  We’re both beneficiaries of living in this wonderful country.  If we can, in some small way, give something back then I’m happy to do it.