Access to information is (in)valuable. Information, and indeed knowledge, eventually builds on data. Local governments generate and own many of the most valuable datasets. Now that the Internet provides us with the means and the tools, access to Public Sector Information (PSI) is increasingly seen to be a valuable stimulator of innovation.
To stimulate awareness, encourage development and help the release of open datasets in our locality, we recently organized an app contest, much in the vein of the "Apps for Democracy" initiative in Washington DC, called "Apps for Amsterdam". It was organized in collaboration between the City of Amsterdam, Waag Society (Media Lab), and Hack de Overheid (a coders' open data community initiative).
Recognizing the potential advantages in terms of transparency, efficiency and innovation, the political environment is often quite supportive of open data initiatives. Many issues arise, however, when actually involving local council departments in implementing an open data policy. While some of these questions are practical or technical in nature, others are financial. Privacy concerns are important and many issues revolve around the quality of data and accountability of the people responsible for that data.
In Apps for Amsterdam the general approach has been to provide "pilot" datasets -- a one-off subset of the actual information available. This is very valuable both for city officials as well as for developers and designers. The question arises how can we move from here towards a sustainable open data practice? We have learned to address some of the issues, but have yet to find answers to many others. In this note, we will discuss our learning, the questions that arose in the process and the effects on transparency, efficiency and innovation.
From February to May 2011, the city of Amsterdam hosted an Open Data App contest that promoted the wide (re)use of Public Sector Information (PSI): Apps for Amsterdam. Apps for Amsterdam was modeled after the successful Apps for Democracy contest that was held in 2008 in Washington DC, USA. Apps for Amsterdam was organized in collaboration with Waag Society -- a media lab, the Open Data activists "Hack de Overheid" ("hack the government"), and the Department of Economic Affairs of the City of Amsterdam. The Department of Economic Affairs participated in this programme because it was interested to understand the apps market and showcase the economic value of open data. During the three months that the contest ran, many data sources were made available, much publicity was gained and many applications were built. While all participants considered the contest to be a success, they also felt that this was just the first step. This field note explains the rationale of the project, gives an outline of the contest, and its practical implementation in Amsterdam. The outcomes and lessons learned follow. We conclude with recommendations for future App contests in Europe and beyond.
Apps for Amsterdam was conducted with the support of the EU programme - Open Cities, the national Service Innovation and ICT (SII) programme, the Virtual Creative Collaboration Platform (VCCP) project, and the EU Seventh Framework Programme (FP7).
URL: http://appsforamsterdam.nl/
The City of Amsterdam recognizes the potential that lies in the (re)use of the data it collects in the course of its operation. However, it comes from a tradition where data is locked in silos and even its own departments have some difficulty in getting access to others" information.
Mainly, it sees three possible benefits in the use and uptake of Open Data: (1) enhanced transparency of the local government; (2) new inventions, innovations, services and ensuing businesses; and (3) enhancing the efficiency of the bureaucracy itself. While these benefits sound attractive, until now there has only been anecdotal evidence, and the argument that these will materialize once the proper conditions are in place. To strengthen the case for Open Data, good practices are required which show and inspire those who can act to make Open Data a reality: namely politicians, civil servants, citizens and high-tech entrepreneurs. In this setting, hosting an Open Data competition was an interesting, light-weight way to motivate data owners to share their data, and programmers and entrepreneurs to apply their skills to come up with applications. The competition achieved both.
Apps for Amsterdam was modeled after Apps for Democracy, that according to its website (http://www.appsfordemocracy.org/) yielded 47 web, iPhone and Facebook Apps in just 30 days. The website boasts a $2,300,000 value to the city at a cost of $50,000 - probably an unprecedentedly wise capital investment on behalf of the local government. Based on this competition, a short but useful guide was produced (http://www.appsfordemocracy.org/guide-to-creating-your-own-apps-for-democracy/) that was drawn upon while outlining Apps for Amsterdam.
Apps for Amsterdam was divided in three time periods and had three main events: the official kick-off on the 16th of February 2011, a hackathon on the 12th of March, and a prize ceremony on the 25th of May. Planning for the first period started in December 2010. It consisted of preparing the collaboration agreement between the partners and the budget, as well as the regulations for the contest, the website, opening preliminary data sources and planning the kick-off event. Also, people were asked to take part in the jury. The second phase of this initiative consisted of chasing, unlocking and posting more data, feeding the press and the network, and preparing and hosting the hackathon. During the third phase, after the deadline for the entries was closed, the public were allowed to vote on the applications that had been created. While the jury cast their votes, preparations for the prize ceremony were underway. These preparations included finalizing the programme and inviting the participants and some inspiring speakers.
The three project partners - Waag Society, Hack de Overheid and the City of Amsterdam - set up the contest, involved their networks, and drafted the rules and regulations.
Apps for Amsterdam had the following rules and regulations:
Prizes were awarded using the following scheme:
| Prize | Number | Independent | Company | Amount |
|
Gold |
2 |
2.500 |
2.500 |
5.000 |
|
Silver |
2 |
1.000 |
1.000 |
2.000 |
|
Bronze |
4 |
500 |
500 |
2.000 |
|
Honorary Mention |
10 |
100 |
100 |
1.000 |
|
Public |
2 |
1.000 |
1.000 |
2.000 |
|
Mobility |
2 |
1.250 |
1.250 |
2.500 |
|
Totals |
20 |
7.125 |
7.125 |
14.250 |
The number of prizes was purposefully high so as to give many people the gratification of success, even if this meant that the majority of prizes were relatively low amounts of money. The prizes were determined on the basis of the suggestions in the guide that was developed from Apps For Democracy.
The jury consisted of knowledgeable representatives of the City Administration, Agentschap NL, Hack de Overheid and Waag Society and a well-known application development expert.
It turned out that finding new and relevant data was the hardest part of setting up the contest. It took many days of efforts from all organizing parties to gradually open up more datasets. Demands made about the format and quality that data should be provided in were intentionally liberal in order to make it easy for data owners to participate. Where "open data" usually requires open standards, full metadata describing the datasets, an email or telephone number for support and questions about the data, were also sought. However, it was decided early in the process, that a very strict adherence to these requirements would make it too hard for many of the participating local government departments to find qualifying datasets. In this respect we decided that "any data" was better than no data at all.
The contest itself proved to be the most effective driving force pushing the local government to "unlock" datasets. At the kick-off, there were a modest number of ten datasets. This number increased to twenty-two during the second phase. Key to making the data available were officers inside the departments who understood the rationale of open data and were able to identify the right people to address the practical issues regarding actually exporting relevant data from the various systems. These "liaison officers" proved to be an invaluable intermediary between the contest organizers on the one hand, and middle management and systems people in the various departments on the other. Some of the more interesting data that became available were incident data from the fire department, soil pollution data, traffic flow data, trash collection schedules, location of public toilets, public art works, and special needs" parking.
The hackathon was organized by Hack the Government and attracted over 100 participants, mainly coders, civil servants and data owners. The line-up of data owners, mainly the above-mentioned "liaison officers", got a huge applause when each one of them presented their datasets. Majority of the day was spent in coding. By the end of the day, over a dozen valuable demos and prototypes were presented to an enthusiastic audience.
48 apps entered the contest, and twenty-one prizes where delivered. The first prizes were won by:
About 150 people attended the kick-off, more than 100 people attended the hackathon, and nearly 200 were at the prize ceremony.
The website has been visited more than 150,000 times (February-December 2011), leading to almost 700,000 page-views and over 2.5 million hits.
While the contest was running, two parties in the Amsterdam City Council teamed up to draft an Amsterdam Open Data charter, that was later embraced by the City Council and the local government [1].
Another outcome is that the Economic Development Board of the City of Amsterdam (EDBA) embraced Open Data as one of its three "iconic" projects. Open Data has been identified as an important stimulus for innovation, transparency and efficiency in the ""city of the future"". The programme will officially start from September 2011.
The Apps for Amsterdam contest has provided the civil servants of Amsterdam with a clear communications mandate and a status to negotiate the need to release data with the city officials and policy makers, avoiding long-winded top down policy decisions.
Lastly, based on the perceived success of Apps for Amsterdam and some of its counterparts in other Dutch cities, a national Open Data contest will be launched in September 2011.
What was the contribution of Apps for Amsterdam to its primary goals: more transparency, more efficiency and more innovation?
According to the Department for Research and Statistics, the contribution of Apps for Amsterdam was mainly in the domain of "transparency" rather than efficiency or innovation. The departments involved developed a greater awareness of the extent, the vastness and richness of available data sources.
Taking advantage of open PSI involves a number of steps. The first step is to increase the awareness of this treasure. Our contest was based on datasets provided by the City of Amsterdam, but the datasets available for the competition were only a fraction of the total available in the city. The second step to make the most of data is to understand in what way it can be used. It is very important to increase this awareness, including increasing local government awareness of what can be done with its existing datasets. The next step is to find out what possibilities datasets have in combination with other datasets. For this step, engagement with users outside the government is of vital importance. Developers can (and must!) think of opportunities that civil servants cannot imagine.
Collaboration between the activists of Hack de Overheid, media lab Waag Society and the City of Amsterdam was very smooth. With these three partners, a necessary base in society was guaranteed; not quite grassroots, not exactly top-down. As the government does not necessarily have the tools, expertise and network to execute these kinds of projects on its own, this turns out to be a good working model for fertile innovation process. The contest has given the city a better insight in what datasets are the most useful to start prioritizing in order to facilitate open data efforts.
One of the interesting aspects of organizing the contest was the "crusade" through the various city departments to try and get data available for the contest. With a very good turnout of city officials during the kick-off, most of the people the organizers had spoken to were already aware of the contest and were almost always very supportive. Unfortunately this does not translate into immediate availability of useful datasets. This has a number of reasons that will be outlined here.
One thing that is very hard for people to judge is the potential for privacy infractions from sharing datasets as open data. This results in an understandable, but in some cases quite unnecessary reticence. On the other hand, we have acquired access to datasets that made us uncomfortable, being relatively sensitive in terms of privacy, and even to datasets that, through a technical blunder, contained quite serious privacy violation that we had to fix with haste.
Another issue is that enthusiasm for Open Data is just not enough. Getting the data out of the databases and into a format that is more widely useful is usually not done by the same people as those involved in policy-making. Because it is not yet implemented in the everyday working culture or standard practice of people dealing with data, policy and practice have to be invented ad-hoc, even if it is just for a one-off event. This is sometimes too much to ask, and progress stops at some indefinite point where either responsibilities are unclear or urgency is not recognized or technical hurdles prevent it.
The quality of the dataset descriptions was variable. Some were very complete, came with a fact sheet of dimensions used and the meaning of various codes. Others needed weeks of emails to get the necessary information in order for developers to be able to understand the data. In many of these cases, this was the first "experiment" of its kind, and thus a learning experience for all involved.
Moreover, a single contest does not by itself result in a sustainable open data policy. In terms of local datasets in Amsterdam, we have only two new datasets (out of about 20) that are updated regularly and that have an existence after the contest. This is not much, especially since the applications built on the other sets are doomed to fade away as the data gets stale.
The issues around privacy, accountability and licensing need legal expertise which, in turn, can enable public administrators to assess the viability of opening data. Issues around formats, access and delivery mechanisms need technical expertise to ensure that the resulting files can be reused with ease and confidence by developers. Sustainable and affordable implementation needs policy advice to prevent the efforts from stopping later on, and enable entrepreneurs to build lasting services. A case can be made for installing some kind of expert body, associated with and trusted by local government officials and social partners, that would be able to assist in resolving these issues until open data becomes standard practice.
In general, the lack of attention to the phase after the contest could be seen as a missed opportunity. The prize money was not enough, and not intended to cater to the need of taking a prototype and making it sustainable. There may have been private initiatives for further development, but if these did take place, they were not widely shared nor researched.
Therefore, the follow-up should be made a crucial part of the next contest. This makes the effort more efficient, and will have a positive effect on the sustained availability, and currency of the datasets.
Also, for the next contest, it would be an improvement if the number of datasets increased substantially. This would make the contest much more attractive and the possibilities for application developers would increase exponentially.
Another lesson is more specifically one for the Department for Research and Statistics. This Department has a lot of datasets but these are most often tables, in excel spreadsheets. These tables are not optimally useful for developers, because of the lack of proper, formal description of the contents and of the aggregation involved. Developers would need information on a different, less aggregated level. The Department for Research and Statistics can provide developers with more useful datasets, but there is always the need to be aware of privacy issues where available data becomes so detailed that the information can be singled down to individuals or households. More interaction with developers is necessary to explore the needs and possibilities.
The project was predominantly funded by the municipality Economic Affairs (and EU project Open Cities -- www.opencities.net). We need to explore the more possibilities of sponsorship by public companies.
Because of the tight budgets, we did not use local media advertising. It may be useful to see how to involve traditional (local) media. In this way, more people can be reached, and thereby demand and (re)use for datasets and applications will grow.
In conclusion, all of the organizing parties, including the local government itself, are very pleased with the results. The contest yielded some new data, a lot of applications, and a lot of attention, including spin-offs in official policy. Apps for Amsterdam was a start in this direction. It is now followed by a national contest called Apps for the Netherlands. The contest has proven that even with a few incentives, a great deal of results can be achieved by making PSI data more widely available under favorable conditions. The next steps will entail paying more attention to the sustainability of the data sources, and the business potential of the applications built using this data. In terms of the original goals of more transparency, greater (government) efficiency and stimulation of innovation, it is still too early to draw conclusions based on hard facts. But the parties involved feel that, taking into account the lessons learned and recommendations outlined above, these goals are indeed attainable.
[1] See the legislative proposal resulting from this at http://www.bestuur.centrum.amsterdam.nl/Bestuursarchief/2010/Stadsdeelraad/Voorstellen/RAVO20100928D66VVD-OpenData.pdf (in Dutch). The proposal concludes with the following short-term recommendations: 1) To start pilot projects with Open Data in Amsterdam-Center. Starting with the budget and some datasets that are relatively easy to provide as open data; 2) To cover the expenses of these pilots, find connection with initiatives in the greater Amsterdam area, and with the planned investments in modernizing the IT infrastructure in Amsterdam; 3) Where necessary, provide one-off subsidies to developers to make accessible applications based on open data; and 4) Evaluate these pilots in the fall of 2011 and, when judged successful, work towards a general policy of 'Open by Default'.