An open data story

Greetings folks!  Below is a presentation about the genealogy of open access to data and open data in Canada.  It was great to finally have the space to demonstrate the earlier work done in this space and to showcase the different open data open access communities.  The open data movement has a long history in Canada, and I believe we can thank research librarians and geomaticians for getting the ball rolling, at least if we tentatively assume that it began in the 80s and the use of the Internet.  The history of sharing data in science and geomatics does however go way back.

The other presentations which were part of the Programmable City Seminar discussed the Irish context at a County Council called Fingal level and the Irish Environmental Protection Agency.  They were great as they featured what it means for a public servant, citizen and a researcher to do this work.  Folks who follow open data in Canada would have been most impressed by their frank talks and their oratorial styles.  A video will be released in the coming weeks.

Author: Liam Currie

Title: The Role of Canadian Municipal Open Data Initiatives: A Multi-City Evaluation (2013) (Available here)

Abstract:

In this thesis I undertake a study of Canadian municipal open data initiatives in order to assess the current state of the programs and to gauge the role(s) that these initiatives may play in regards to improving public engagement in local government issues. After an initial literature review, I adopt two separate approaches. The first approach involves the creation of an inventory and evaluation of the contents of all twenty three (23) Canadian municipal open data catalogues in existence during the summer of 2012. The second approach involves asking questions of key informants in the field through the execution of nineteen (19) semi-structured interviews with open data experts from both government and civic realms in ten (10) case study cities across the country.

The results of the research illustrate the major differences and similarities between the structure, output, and roles of open data initiatives in various Canadian cities. The data provided by these programs mostly consists of politically neutral geographic data, though there are a few exceptions. I find two major program structures in Canadian cities: (1) The first type of open data program is created and operates within a specific municipal department and the (2) second type of program operates across a number of departments. Each approach has its own benefits and challenges. The open data initiatives across Canadian cities also appear to have different approaches to public engagement. Several cities have developed strong collaborative relationships with local open data advocates which are explored in some detail. Larger themes about the current state of open data, its current and future role, and the challenges faced by operators and users, are also described in this thesis. I conclude with some recommendations for improving municipal open data initiatives in the future.

M.A Thesis, August 2013
Department of Geography,
Queen’s University, Kingston, ON

Acknowledgements:

  • SSHRC – Funding
    Dr. Betsy Donald – MA Supervisor
  • Harvey Low, John Jones, Reham Gorgis-Youssef, Gina Porcarelli, Matthew Dance, Mack Male, Robert Giggey, Mary Beth Baker, Edward Ocampo-Gooding, Diane Mercier, Michael Lenczner, Andrew Durnin, Alyssa Daku, Andrew Dyck, Steve Czajka, Robert Lunn, Blair Labelle, Joey Coleman – Interview participants
  • Dr. Tracey Lauriault (Datalibre.ca, Civicaccess.ca), David Eaves (Eaves.ca), Dr. Michael Gurstein (Gurstein.wordpress.com), James Mckinney (Opennorth.ca), Keith Macdonald (City of Toronto), Cyrille Vincey (qunb.com), David Robinson (robinsonyu.com) – Additional input and support
  • The discussions and links hosted at the following blogs and listservs were also invaluable: eaves.ca, crookedtimber.org, gurstein.wordpress.com, datalibre.ca, civicaccess-discuss@civicaccess.ca, open-government@lists.okfn.org, open-data-census@lists.okfn.org

The first real open data project in Canada, is arguably, GeoGratis, launched in 1997. It disseminates geomatics legacy data, archived data or data used for independent research.  Like the DLI it used FTP in the early days of the Internet as a way to transfer data in the formats within which datasets were created and data transformation services were pointed to.  Today it disseminates data via a portal and other web services.  Geogratis is a Natural Resources Canada program, under the aegis of the Canadian Geospatial Data Infrastructure (CGDI) which is being devlivered by GeoConnections.  Shortly thereafter, as part of the CGDI, GeoBase was launched to disseminate free national scale framework data.  Both projects, unbeknownst to their creators, were revolutionary, Geogratis for launching the first free open data project at the Federal level, and Geobase as it was data being created as part of a provincial and territorial accord.  These innovations became even more significanr once a decision was made to share the data under an Unrestricted User Licence, a first for Canada. It was a way to work around Crown Copyright.  GeoConnections continues to be innovative in its data dissemination, open specifications, interoperability and standards based approach to data.  See their Operational Policies and other related documents to read more about their work. It is the gold standard in Canada, and most of the data found in the OpenData portal run by the Treasury Board Secretariat of Canada, are from Natural Resources Canada.

GeoGratis: A Canadian Geospatial Data Infrastructure Component that Visualises and Delivers Free Geospatial Data Sets (1999)

Abstract:

Many countries are in the process of setting up geospatial data infrastructures. The United States Federal Geographic Data Committee (FGDC) has assumed leadership in the evolution of the National Spatial Data Infrastructure (NSDI). Similarly, the National Geospatial Data Framework (NGDF) of the United Kingdom facilitates collaboration, standards and access to geospatial data. The Canadian Geospatial Data Initiative (CGDI) has five objectives to facilitate access, partnerships, framework data sets, supportive polices, and standards. GeoConnections is the program to build the CGDI. GeoGratis is an operational and fundamental component of Geo-Connections. The GeoGratis objective is to provide a wide range of free vector and raster geospatial data sets of the Canadian land and water mass to the public. During the early stages of GeoGratis, previously archived geospatial data sets were distributed using an Internet based File Transfer Protocol (ftp). Data sets include the Canada Land Inventory and a sample of panchromatic, multi-spectral and hyperspectral imagery. Recently implemented delivery methods are based upon the HyperText Transfer Protocol (HTTP) that permits the development of a friendlier user interface and a screen to capture a client’s profile. The client is prompted to download the file in the original format and projection or the user can change these parameters according to individual preference. GeoGratis supports the distribution of framework data sets that meet national and international standards. Free and open distribution ensures that these frameworks will be widely accepted. The National Atlas of Canada base maps are the essential framework data sets in GeoGratis. In Canada, GeoGratis reflects the philosophy that the widespread distribution of free or low cost geospatial data stimulates research and development, and promotes a more diversified user base. GeoGratis develops partnerships across government departments. Many departments are protected from the vagaries of the Internet by strong firewalls. The GeoGratis project provides a simple operational tool for these departments. Recognising the needs of a diverse and new user base, GeoGratis plans to use a variety of technologies that offer a range of interfaces to view and access the data sets. Levels of complexity range from simple bitmaps to a more complex on-line GIS with file conversion facilities. The GeoGratis project is developing database design, visualisation software and access methodologies, which may be applied to any country’s geospatial data infrastructure.

Authors:
Cameron Wilson, Canada Centre for Remote Sensing, Natural Resources Canada, Ottawa, ON, K1A 0E9 Canada
Robert. A. O’Neil, Canada Centre for Remote Sensing, Natural Resources Canada, Ottawa, ON, K1A 0E9 Canada

The DLI, is arguably, one of the first attempts by the research and university library communty to make census data accessible to Canadians.  The DLI cannot be considered an Open Data project since data are behind a paywall, irrespective, it was one of the first real lobby efforts to make data available, to work around StatCan’s (back in the day when we had a census) regressive cost recovery and data pricing policies.  The paper provides an excellent review of the history of data advocacy in this community of practice, introduces the associations and also discusses early Internet FTP data transfer, standards, and the data consortium or group purchase model.  Today the DLI serves thousands of faculty and students in research libraries, and its boot camps continue to provide ongoing education and builds capacity in Canadian Research Libraries.  Many of the organizations mentioned remain actively involved in Research Data Canada, the creation of trusted digital repositories and work on the preservation of research data.

The Canadian Data Liberation Initiative: An Idea worth Considering? an International Household Survey Network Paper

Abstract

The Data Liberation Initiative (DLI) is a Canadian program aimed at providing Canadian post secondary institutions affordable access to Statistics Canada data resources. It is a partnership between Statistics Canada and the academic sector. While it initially focused on the dissemination of public use microdata files it now encompasses all publically-available data. This paper describes the background of this project and some of the key success factors so that other agencies may be able to determine its applicability for their own situations. It was written in the hopes that other agencies may find the Data Liberation project as a useful model to consider. It was also written for a Canadian audience that is interested in the history of this project which has now been in operation for over 16 years.

Authors:

Ernie Boyko is a former staff member of Statistics Canada where he held a number of Directorships, including Agriculture, Corporate Planning, Electronic Publishing, and Operations for the 1991 Census. It was during his time at Statistics Canada that Wendy Watkins and he co-founded the Canadian Data Liberation Initiative. He is an active member of the Canadian Association of Public Data Users and the International Association for Social Science Information Services and Technology. He is currently an Adjunct Data Librarian at the Carleton University Data Centre and occasionally works on projects with the International Household Survey Network. .

Wendy Watkins is the Manager of the Carleton University Library Data Centre. Her inspiration for Data Liberation came while working at Statistics Canada for a two year period. She is a founding member of the Canadian Association of Public Data Users and active in the International Association for Social Science Information Services and Technology.

Steve Reitano conducted a survey and shared the results of his organizational management research project on the benefits of Open Data which he did as part of his Master’s Degree in Executive Management at Royal Roads University.

The Benefits of Open Data (Canada)

Abstract:

The purpose of this research project is to examine the benefits and the challenges of publishing Open Data for government organizations. It is presumed that open and accessible data offers multiple benefits, including improved openness and accountability, as well as an increase in innovation and economic growth. This paper aims to help public organizations make sound and informed decisions for extending their Open Data initiatives by determining the social, economic and environmental benefits of publishing Open Data, thereby creating a more cost-effective, transparent, efficient and responsive government.

I am posting these here, as open data is maturing in Canada, and we are begining to see studies outside of government and done in Universities.  There are a few others and there are some older ones which are about the topic but a little older.  Open data is not new in Canada, it goes back to the 1980′s, and I will endeavour to post what I find here so that we may have a new and an historical perspective on the issue.  There are a few timelines floating around as well, and will point to these once I consolidate them.

geo Hackathons in Montreal

2 geo Hackathons are taking place in Montreal very soon!
  1. Défi GeoHack: (le 2 octobre)
  2. EcoHack:  (séance de travail le 21 aout - https://www.eventbrite.ca/event/7881011317 , événement principal mi-octobre, date exacte à préciser)

Montrealers are taking open data, apps development and hackathons to new heights and a more focussed level than anywhere else in Canada right now.  There has been Hacking Health in a few cities, and Apps for Climate Change in BC, and Random Hack of Kindness, which have brought great people together.  The folks in Quebec, however, really ground their efforts into the development socio-political-technologies for social good and tie their work into the political process.  For instance:

  • Logement Insalubre an open data movement which brings together apps developers, open data advocates and housing and homelessness activists and organizations to address the dearth of data on these topics,
  • Montreal Accessible which matched people with disabilities, apps developers and volunteers to conduct accessibility surveys and map results, à la participatory geography,
  • Hackons la corruption, where more than 100 developers and hacktivists came together to mine contract data
  • ZoneCone, which is an open source, interoperable, standards based à la GeoConnections app to navigate Canada’s construction season by integrating transportation road network data and road constructions zones
  • OpenNorth, while a national organization, was created by a groups of anglo, international and franco Montrealers,  is the first open data non profit developing tools for a better democracy
  • and Ajah.ca one of the first companies in Canada to merge open data, web aggregation data/content algorithms and data about the charitable sector in Canada and then create products to improve and aid the sector in becoming more evidence-based while also pushing for national scale research.

Montreal Ouvert also created a Table De Concertation to advise the city on its open data strategy.  They have taken great pains to ensure that it has broad  representation.  They also have pic nic‘s,  how great is that!

This is great stuff!
Geomatics, Cartography and geography students can contribute to and learn from these activities, and class projects can be hackathon like prototypes dedicated to resolving real world issues in an interactive, collaborative and multidisciplinary way.  Data specialist can help by improving the social science of these works in some instances, study these effort, and info specialists can contribute their semantic ontology, portal design, keyword searches, data management and preservation as well as cataloguing skills to the processes which are normalizing across the country.  There are open data initiatives across the country at city, provincial and federal scales  and they all need a multiplicity of skills.
I love this stuff, and hope you do too!

Moving to Ireland!

That is right! I am leaving the Geomatics and Cartographic Research Centre (GCRC) in Ottawa for a postdoctoral research position at the National Institute for Regional and Spatial Analysis (NIRSA) in Maynooth Ireland

I leave the GCRC  on Aug. 23rd, my academic home of 11 years, at Carleton University in Ottawa under the directorship of D. R. Fraser Taylor to work with Rob Kitchin on the Programmable City Project  at NIRSA, National University of Ireland at Maynooth, funded by the European Research Council (ERC). I officially start September 1st and leave Canada on the 25th.

My primary focus will on the following research question:

  • How are digital data generated and processed about cities and their citizens? with some crossover on
  • How are discourses and practices of city governance translated into code? 

The particulars will be worked out once the international team gathers for the first time in Maynooth in September. Research activities, reflections and results will be shared on a website or blog once the project gets started. They will of course be disseminated in the usual academic fora.

I will continue to be involved with the GCRC as a member and do watch for the Developments in the Theory and Practice of Cybercartography: Applications and Indigenous Mapping book ed. by Prof. Taylor and myself as associate editor.  In addition I will continue to be:

  • a member of the Canadian Geomatics Round Table – Legal and Policy Dimension Task Team,
  • the Chair of the Canadian Cartographic Association (CCA)Mapping Technology & Spatial Data Special Interest Group,
  • and a member of the Research Data Alliance -CODATA legal interoperability of research data working group.

Also, I will continue to monitor issues related to open data and open government in Canada but at a much reduced capacity as I begin to shift my focus on Dublin and Boston, Ireland, the EU and global scales. The Datalibre.ca blog will continue to be co-authored by Hugh McGuire and I and the focus may become more global. Of course, I will continue to post on the CivicAccess list I co-founded with Michael Lenczner, Hugh McGuire, Daniel Haran, Stéphane Guidon, Gabe Sawhney and others.  Open data has come a long way in Canada since we founded this first national list in 2005 and I urge you  to register, keep up to date and to take it to the next level.  It has been a great ride!

I have worked with many community based organizations in Canada on the topic of bringing data &mapping and evidence-based decision making to the social sector. I will not be able to help as much as I did, but will continue to keep my eyes open and share information on issues pertaining to civil society and open data as much as possible.  I will of course continue to respond to information requests.  My final talk in Canada will be done remotely for the Ontario Nonprofit Network (ONN) Sector Snapshots: NFP Trends & Tactics for a Changing Ontario on September 19.

The work on the SSHRC Partnership Development Grant at the GCRC with Teresa Scassa at the Centre for Law, Technology and Society (CLTS) and the Canadian Internet Public Policy Interest Clinic (CIPPIC) will be coming to a close and final reporting for that is underway, and as discussed work on the book is nearly complete.

I will not be in Canada, but will keep linkages open.  I will sorely miss my colleagues at the GCRC (Esp. Peter Pulsifer, Glenn Brauen, Amos Hayes and JP Fiset, and others) as we really grew and flourished together, and I am incredibly thankful to Fraser for being such a great academic advisor and for providing us all with incredible research project experience.  I am also sad to say goodbye to the great folks at the Carleton University MADGIC, and the Dept. of Geography, but will not be abandoning the ongoing work on the preservation of data as NIRSA has two data archives!  Also, keep your eyes open for the GeoConnections Geospatial Data Preservation Primer that will soon be released in September of 2013 on the GeoConnections operational policies website.

I am looking forward to the move to Dublin, working with Prof. Kitchin and a fantastic new group of colleagues and starting the next leg of this academic career and bid you all aurevoir, and not adieu, as I expect we will all keep in touch, you might visit and we may even conjure some interesting International research collaborations and projects.

I will share new coordinates once I have them, but you can also follow me here:

@traceylauriault (twitter)
tlauriau at gmail dot com

A Fed open data ask!

The local community group and a representative of hundred of municipalities, has made the request for the following data at the Ottawa open data round-table consultation and also in a letter.  So far these data have not materialized and there has not been a response.  These groups require these data to do the following:

The data focus of these initiatives is collect and report on data that captures social, economic, and environmental trends at the municipal or neighbourhood levels in Canadian cities and communities.

Your feedback on the following request would be greatly appreciated:

What we would be most interested in, as a starting point, is a series of open discussions with the Federal government about community data: data that measures trends and conditions in Canadian cities & communities. Community Data Canada (cdc-dcc.info) was established in 2009 as a  forum for these discussions.

A series of “Community Data Roundtables” would address two fundamental limitations of data access at the level of municipal and community geographies:

1.    We don’t know exactly what to request of Federal departments. HSRDC is a good example. That agency has unknown quantities of administrative data related to several topics:

o   Employment Insurance

o   People with disabilities

o   Homelessness data (HIFIS)

o   Other data sets dealing with seniors, First nations, students, and much else

2.    Much of the community data we require reside with provincial government departments. In these cases, there is a role for the Federal government to act as a convener of provincial governments. We lack the capacity and influence to reach 10-13 jurisdictions (in the case of the 3 territories). Federal government-convened Community Data Roundtables would tackle the following topics:

o   Education data: Meeting with officials of provincial ministries of Education.

o   Health data: Meeting with officials of ministries of health. This includes compelling CIHI to share their data. They are were created as a private non-profit operating at arms length from the government, but must be accountable to governments at some level.

o   Environment Data: Air quality index databases

A final note: we are already making good progress with several Federal government agencies:

·         Industry Canada – bankruptcies data

·         Citizenship & Immigration Canada – permanent residents data cube, and on track to develop an IMDB data cube (linking immigrant and income databases)

·         Environment Canada – air quality monitoring station data

·         CRA – Income data via Statistics Canada

·         CMHC – Housing data

Compelling these agencies to make their data available via a portal will ultimately be beneficial to us, but we are currently able to access these data sets through direct relationships with these agencies.

As a follow-up to last week’s note, there is one specific data product we would like to add to the Federal Government’s Open Data portal:

  • Statistics Canada’s Business Register listings without contact information, but with 6-digit postal codes and/or street addresses

Statistics Canada’s response to the request made by the Community Data Program is para-phrased as follows

Statistics Canada almost never releases individual business listings unless they meet three criteria at the same time:

1. They are needed for research purposes

2. They are needed by a federal government department or agency

3. Their release is approved by the chief statistician

This was part of a letter that was sent as a follow-up.

Today the federal government announced that it would make Canadian public sector information (PSI), or administrative data (admin data) or government data more accessible and useable.  A new portal was launched, data are now being disseminated under a more open licence and some additional services are being provided and there is an apps gallery and a developers corner.

This announcement coincides with today’s endorsement of the G8 Open Data Charter.

The licence gets high marks!  It is more open and is less restrictive.  The Government consulted on this file and listened to experts.  Teresa Scassa, Canada’s Research Chair in Information Law provides an excellent overview of the new licence here. Interestingly, BC, Ontario and Alberta will also be adopting this licence, which demonstrates legal interoperability between jurisdictions.  This is excellent.

The Portal is still a little buggy, and folks on the Civicaccess.ca list are reviewing its features and commenting on it.  See the archived threads here.

In terms of data, the issues I am concerned about do not yield any results, homelessness for example yields National Household Survey, clearly there are some tagging issues.  I also searched charity, charities, business, business registries, poverty and the results were either null or sub-par.  Cities yielded close to 800 hits while health outbreaks yielded none and brown fields yielded 4.  When data are accessed the interface, description, and location map are very good, as seen here.

The new Search Summaries of Completed ATI Requests is also very useful and this should simplify the access to information (FOI requests) process and reduce costs.  I was happy to find homelessness information in it, and it is interesting that these data are not linked with those in the portal.  Some work to be done here.

I will need to engage with it more, however, overall it is an improvement over the Pilot.  Please send me your comments here at tlauriau at gmail dot com

In terms of a more open government and greater transparency!  This is one good step in that direction, but we have a few other issues, such as a cancelled census, a lack of a data archive, the bullying of NGOs who speak out against the government, the muzzling of scientists, the closing of monitoring stations, a lack of evidence based policy and the decimation of the Library and Archives Canada.  Soo…..

 

Canada has signed on to the G8 Open Data Charter. The official UK G8 Presidency site includes the Charter and its associated technical Annex.  This Charter falls under one of this year’s G8 agenda items, which is to promote greater transparency. The main points of the charter are:

  1. Principle 1: Open Data by Default
  2. Principle 2: Quality and Quantity
  3. Principle 3: Usable by All
  4. Principle 4: Releasing Data for Improved Governance
  5. Principle 5: Releasing Data for Innovation

The G8 countries have committed to the following 3 actions:

  • Action 1: G8 National Action Plans
  • Action 2: Release of high value data (List is pasted below)
  • Action 3: Metadata mapping

These are all good things.  The devil will be in the details and implementation in Canada will depend on collaboration and interoperability between provinces, territories, cities and municipalities and the federal governments.  It will be interesting to see how crown corporations such as Canada Post, Canada Housing and Mortgage and Corporation (CMHC), CBC/Radio Canada and others fare.  At the moment, it there is uncertain if these are to follow the same rules.

The list of high value data that should be released somewhat overlap with information collected by the Open Knowledge Foundation Open Data Census.  Note the postal code data requested under geospatial, this is a big ask for Canada especially in light of the Geocoder copryright lawsuit instigated by Canada Post. Digital Copyright Canada has also done some good work on the postal code file.

It will be very interesting to see if greater access to data will mean an increase in evidenced based policy making and greater participatory democracy. The government will need to be more receptive to citizen input, and so far, if the census and issues around science are any indication, this does not look promising.  Releasing data is one thing, acting on the evidence and having the mechanisms in place and willingness to hear from citizens is another.

Data Category (alphabetical order) Example datasets
Companies Company/business register
Crime and Justice Crime statistics, safety
Earth observation Meteorological/weather, agriculture, forestry, fishing, and hunting
Education List of schools; performance of schools, digital skills
Energy and Environment Pollution levels, energy consumption
Finance and contracts Transaction spend, contracts let, call for tender, future tenders, local budget, national budget (planned and spent)
Geospatial Topography, postcodes, national maps, local maps
Global Development Aid, food security, extractives, land
Government Accountability and Democracy Government contact points, election results, legislation and statutes, salaries (pay scales), hospitality/gifts
Health Prescription data, performance data
Science and Research Genome data, research and educational activity, experiment results
Statistics National Statistics, Census, infrastructure, wealth, skills
Social mobility and welfare Housing, health insurance and unemployment benefits
Transport and Infrastructure Public transport timetables, access points broadband penetration

« Older entries § Newer entries »