If you can, please help the Canadian Council on Social Development (CCSD) with a donation to cover the legal fees. Champs & Associates did much pro-bono work on this file, however, $25 000 are required and any help would be most appreciated.

B E T W E E N:
CANADIAN COUNCIL ON SOCIAL DEVELOPMENT,
COMMUNITY SOCIAL PLANNING COUNCIL OF TORONTO,
COMMUNITY DEVELOPMENT HALTON,
CANADIAN ARAB FEDERATION,
ONTARIO COUNCIL OF AGENCIES SERVING IMMIGRANTS,
COUNCIL OF AGENCIES SERVING SOUTH ASIANS,
CANADIAN MENTAL HEALTH ASSOCIATION – TORONTO,
AFRICAN CANADIAN LEGAL CLINIC,
NATIONAL ABORIGINAL HOUSING ASSOCIATION,
SOUTH ASIAN LEGAL CLINIC OF ONTARIO,
SOCIAL PLANNING COUNCIL OF WINNIPEG,
METRO TORONTO CHINESE & SOUTHEST ASIAN LEGAL CLINIC
Applicants

-and-
ATTORNEY GENERAL OF CANADA
Respondent
_____________________________________________________________________
APPLICANTS’ MEMORANDUM OF FACT AND LAW
(Vol. V of V)
Memorandum of Fact and Law
_____________________________________________________________________
Paul Champ/Anne Levesque
Champ and Associates
43 Florence Street
Ottawa, Ontario
K2P 0W6
T: (613) 237-4740
F: (613) 232-2680

Bravo.

The ACA had the following to say in their letter:

The ACA is concerned that Bill C-19 contains provisions that explicitly override existing legislated
information management processes. The current disposition of records in the Firearms Registry is for destruction ten years after last administrative action. Further, Sections 12 and 13 of the Library and Archives of Canada Act (LAC Act) require written permission from the Librarian and Archivist of Canada to destroy records. However, sections 29 (1) (2) and (3) of the Ending the Long-gun Registry Act call for the destruction of records “as soon as possible” and that the LAC Act be ignored.

The ACA urges the government to trust the records management processes already enshrined in current legislation. The Firearms Registry records were scheduled according to business needs and to potential historical value by the creating department and professional archivists at Library and Archives Canada. These processes ensure accountability and transparency with regards to the destruction of government records. Destroying records for political expediency and disregarding the LAC Act sets a very dangerous precedent for future legislation and record keeping practices.

The ACA is an academic & professional association that has taken a stance on this important issue which is exactly in their mandate to do.  Excellent.

While it might seem obvious that they are records, yet this has been an ongoing debate among archivists and copyright advocates.  Here are some thoughts at least as it pertains to geospatial data:

The following definitions in the Library and Archives of Canada Act are relevant for geospatial information and databases (LAC Act, Definitions, pp. 1-2, and also p. 6 for recording):

  • Documentary heritage: publications and records of interest to Canada.
  • Government record: a record that is under the control of a government institution.
  • Ministerial record: a record of a member of the Queen’s Privy Council for Canada who holds the office of a minister and that pertains to that office, other than a record that is of a personal or political nature or that is a government record.
  • Publication: any library matter that is made available in multiple copies or at multiple locations, whether without charge or otherwise, to the public generally or to qualifying members of the public by subscription or otherwise. Publications may be made available through any medium and may be in any form, including printed material, on-line items or recordings.
    • Record: any documentary material other than a publication, regardless of medium or form.
  • Recording: anything that requires a machine in order to use its content, whether sounds, images or other information.

The LAC Act makes no specific reference to form, format or media, which means that geospatial data, databases and maps are considered to be government records, that may have documentary heritage qualities.  Record creators are therefore responsible for managing this content in the event that these records are to be accessioned by LAC. LAC’s responsibility is to advise, provide leadership, technical expertise, professional expertise and financial support to assist record creators and other federal geospatial information producers with preserving their records.

When archiving geospatial data, databases and maps, the Copyright Act must also be taken into consideration, as would licenses and access rights.  Geospatial data, databases and maps may fall within the following definitions in the Copyright Act:

  • Artistic work: includes paintings, drawings, maps, charts, plans, photographs, engravings, sculptures, works of artistic craftsmanship, architectural works, and compilations of artistic works;
  • Book: which is a volume or a part or division of a volume, in printed form, but does not include (a) a pamphlet, (b) a newspaper, review, magazine or other periodical, (c) a map, chart, plan or sheet music where the map, chart, plan or sheet music is separately published, and (d) an instruction or repair manual that accompanies a product or that is supplied as an accessory to a service;
  • Collective Work:  which is  (a) an encyclopedia, dictionary, year book or similar work, (b) a newspaper, review, magazine or similar periodical, and (c) any work written in distinct parts by different authors, or in which works or parts of works of different authors are incorporated;
  • Compilation: (a) a work resulting from the selection or arrangement of literary, dramatic, musical or artistic works or of parts thereof, or (b) a work resulting from the selection or arrangement of data but not the geospatial data itself…;
  • Computer Program: which is a set of instructions or statements, expressed, fixed, embodied or stored in any manner, that is to be used directly or indirectly in a computer in order to bring about a specific result;
  • Every original literary, dramatic, musical and artistic work: includes every original production in the literary, scientific or artistic domain, whatever may be the mode or form of its expression, such as compilations, books, pamphlets and other writings, lectures, dramatic or dramatico-musical works, musical works, translations, illustrations, sketches and plastic works relative to geography, topography, architecture or science;
  • Literary work: includes tables, computer programs, and compilations of literary works;
  • Definition of “publication”: 2.2 (1) For the purposes of this Act, “publication” means (a) in relation to works, (i) making copies of a work available to the public

If any data, databases or maps are produced by an employee of the Federal Government, then copyright belongs to Her Majesty. Works created by the Federal Government of Canada can be licensed and  federal producers’ geospatial data, databases and maps are covered by many licenses.  However, there are no universal licenses for geospatial or any other data being used within the Government of Canada.

A digital map, a digital mapping dataset and a digital mapping database can be a record, a recording, a publication, part of a ministerial record and also be of documentary heritage as defined in the Act.  How these are classified is contingent on how it fits in a department or agency’s mandate and other digital mapping data producers’ business practices and how these are valued by the institution.  For instance the Atlas of Canada may be a publication, which is also a record that could also be considered to be of documentary heritage.  The host institutions, the record creator, would appraise[1] it as such, as would Atlas Stakeholders (e.g., school teachers) or potentially the Librarian and Archivist of Canada might do so.  How and when to capture snapshots of the Atlas, whether or not that is the best method to adopt, and when to accession[2] it in the Archive is not yet determined, and would have to be discussed between LAC and the Atlas.  A host institution and LAC would then collaborate in deciding how the Atlas or any other geospatial dataset is to be made available to future generations of Canadians and how those future generations will view the them. The processes of appraising, how to manage, and how to preserve would be repeated for other datasets, databases and maps.

The TBS Standard on Geospatial Data is the only TB document that relates specifically to the geospatial information domain. Coming into effect in June 2009, the objective of this standard is “to support stewardship and interoperability of information by ensuring that departments access, use and share geospatial data efficiently and effectively to support program and service delivery”. While this standard does not explicitly reference digital information archiving and preservation, there is a linkage between it and LAC’s File Format Guidelines for Preservation and Long-term Access.  These guidelines recommend the same format as included in the TB standard (ISO TC 211 ISO 19115 Geographic Information – Metadata (NAP – Metadata) (North American Profile)) for preservation of and long term access to digital geospatial information held by government organizations.

A digital map, dataset or database could be a ministerial record if it formed part of a particular decision in that office. These digital objects, irrespective of their form, would then be a part of that particular ministerial record set.

For Archivists, specifically InterPARES I stated that five characteristics are required for a digital entity to be a record:

  1. stable content and fixed form;
  2. embedded action;
  3. archival bond;
  4. three persons (i.e. author, addressee, writer);
  5. and an identifiable administrative and documentary context.

For some case studies, conducted by InterPARES II, particularly Cybercartographic Atlas of Antarctica (CS06)[3] and VanMap (CS24)[4], which are explicitly designed to allow for data to change and information to be added, this means that they are not or do not contain records in archival terms.  To become records, they must be fixed in time and space.  InterPARES 2, VanMap and San Diego Centre for Supercomputing have collaboratively designed a research study to determine whether it might be feasible to introduce fixity into the system by changing the system’s architecture so that each time a layer is updated the layer is saved and set aside. This would allow composite views of VanMap to be assembled for any given date, consisting of layers that had been saved on that date or most recently prior to that date.[5]  This is however a far from perfect solution and is both expensive and beyond the capacity of most institutions creating and using these dynamic products.

One of the most serious problems in this respect is that for most geospatial data creators the term “record” means data, databases, and related information.  For many archivists, these are not considered records except in very special and limited circumstances, where the concept of “bounded variability”[6] may be applied.  This is not simply a matter of semantics.  It is a fundamental difference in perspective between creators and preservers, compounded by the emergence in all disciplines of ephemeral interactive information which exists only in cyberspace.  This is particularly the case for many of the InterPARES 2 Case Studies (like the Cybercartographic Atlas and VanMap) which are interactive, experiential and dynamic.  Duranti and Thibodeau argue that

interactions between humans and computer systems, experiences enabled or mediated by experiential systems, and processes which are carried out with at least some degree of spontaneity by dynamic systems are not the residue of action. They are not means of remembering either what was done or what is to be done. In short, they are not records.[7]

This archival position is entirely defensible from the perspective of the theory of diplomatics[8] but is problematic in many scientific situations, such as for computational data where a model or a simulation is the primary result.  The nature of the “record” is changing dramatically and traditional archival science will have to adapt to these changes in both theoretical and practical terms if they are to preserve this new information environment in the archives of the twenty first century.

Who manages and preserves these then?

Although the problems and challenges of archiving these dynamic data sets and digital artifacts are being identified, the institutional environment is often not conducive to the systematic action required to address the problem.  For example Library and Archives Canada (LAC) is not ready to systematically archive born digital geospatial data, databases and maps let alone complex artifacts such as the Atlas of Canada.

Currently, there are a few LAC guidelines for cartographic material, but these are general in nature and primarily address paper maps.  The 2006 LAC  Managing Cartographic, Architectural and Engineering Records in the Government of Canada[8] made only passing reference to digital maps such as “the National Archives acquires geomatic systems” and “geomatic records include geomatic systems, discs, CD-ROMs and other cartographic material in electronic formats” (LAC 2006a).  This has since been updated[9]; however, the focus remains primarily on paper content and digitized paper content. The 2011 document still refers the reader to the 2001 Canadian Committee on Archival Description (CCAD) Rules for Archival Description Chapter 5[10] for information pertaining to standards and practices for cartographic records.  The Rules primarily address paper maps while general issues pertaining to digital databases and programs are covered in the 2003 Chapter 9: Records in Electronic Form[11]While both of the CCAD documents are designed for archivists who manage these records, these documents could be useful references for government record managers.

The LAC Local Digital Format Registry (LDFR) File Format Guidelines for Preservation and Long-term Access Version 1.0[12] includes Geospatial guidelinesThe Geospatial[13] section recommends TC 211 ISO 19115 Geographic Information – Metadata (NAP– Metadata) (North American Profile). This is a dynamic document that will improve as knowledge grows and as digital geospatial data record creators and managers contribute to it. While these documents are lauded as a good start, they fall short of adequate guidelines for the kind of digital geospatial artifacts discussed in the InterPARES 2 Case Studies, the portals in the InterPARES 2 General Study on Data Portals.

[1]     Defined as n. ~ The study of the creation, form, and transmission of records, and their relationship to the facts represented in them and to their creator, in order to identify, evaluate, and communicate their nature and authenticity. http://www.archivists.org/glossary/term_details.asp?DefinitionKey=21

[2]     n., The process of assessing the value of records for the purpose of determining the length and conditions of their preservation. (InterPARES 2 Terminology Database – http://www.interpares.org/ip2/ip2_terminology_db.cfm )

[3]     v., To take legal and physical custody of a body of records and to document it in a register. (InterPARES 2 Terminology Database – http://www.interpares.org/ip2/ip2_terminology_db.cfm)

[4]     Sherry Xie, Diplomatic Analysis CS06 Cybercartographic Atlas of Antarctica (revised), (Vancouver, 2006).

[5]     Jennifer Douglas, CS24 Diplomatic Analysis Template Preservation of the City of Vancouver GIS database (VanMap) (Vancouver, 2006).

[6]     Evelyn McLellan, CS24 City of Vancouver Geographic Information System (VanMap), (Vancouver, 2005). See also the article “From Data to Records:  Preserving the Geographic Information System of the City of Vancouver,” by Glenn Dingwall, Richard Marciano, Regan Moore and Evelyn Peters McLellan in this issue of Archivaria.

[7]     Evelyn McLellan, CS24 City of Vancouver Geographic Information System (VanMap) CS24.

[8]     Luciana Duranti and Kenneth Thibodeau, 2006, “The Concept of Record in Interactive, Experiential and Dynamic Environments: the View of InterPARES,” Archival Science vol. 6 no.1, pp.13-68  p. 59.

[9]     Library and Archives Canada (LAC), File Format Guidelines for Preservation and Long-term Access Version 1.0, accessed February 2011. http://www.collectionscanada.gc.ca/digital-initiatives/012018-2200-e.html.

[10]     Library and Archives Canada (LAC), File Format Guidelines for Preservation and Long-term Access Version 1.0 2.9 Content Category: Geospatial, accessed February 2011. http://www.collectionscanada.gc.ca/digital-initiatives/012018-2220.09-e.html

[11]     Library and Archives Canada (LAC), accessed 2006, Managing Cartographic, Architectural and Engineering Records in the Government of Canada,  Ottawa: Government of Canada, http://www.collectionscanada.ca/information-management/002/007002-2050-e.html (NOTE – this link is no longer live)

[12]     Library and Archives Canada (LAC), accessed 2011, Managing Cartographic, Architectural and Engineering Records in the Government of Canada,  Ottawa: Government of Canada, http://www.collectionscanada.gc.ca/government/002/007002-2050-e.html.

[13]     Canadian Committee on Archival Description, 2001, Rules for Archival Description Chapter 5 Cartographic Materials, accessed February 2011. http://www.cdncouncilarchives.ca/rad_ch5.pdf

[14]     Canadian Committee on Archival Description, 2003, Rules for Archival Description Chapter 9 Records in Electronic Form, accessed February 2011. http://www.cdncouncilarchives.ca/RAD_chap9_revised_Aug2003.pdf

My understanding is, that the Gun Registry, as it is familiarly known, or the Canadian Firearms Information System is an official government record that cannot arbitrarily be destroyed.  Below I provide references to Acts, Regulations, Directives and Policies that support that perspective.  However, at the bottom you will find reference to proposed Bill C-19, and you will discover that this Bill aims to override all of these.  Is it constitutional for a government to create an Act to go against its own Acts, Regulations and policies?

Normally, Library and Archives Canada (LAC) has the final say on whether or not a database or record can be destroyed (see slide #3) as LAC is:

accountable for the final decisions about what gets kept and what gets deleted

And that statement is legally grounded in the  Library and Archives Act:

Destruction and disposal

12. (1) No government or ministerial record, whether or not it is surplus property of a government institution, shall be disposed of, including by being destroyed, without the written consent of the Librarian and Archivist or of a person to whom the Librarian and Archivist has, in writing, delegated the power to give such consents.

To implement the law and follow the regulations the Multi-Institutional Disposition Authorities (MIDA), were created to direct when and how records are destroyed, and it is not arbitrary:

Under the Library and Archives of Canada Act (2004), Library and Archives Canada (LAC) Canada is charged with various responsibilities regarding the disposal of this information, including the authorization of records destruction by government institutions (Section 12) and the preservation of records for their archival or historical importance (Section 13).

The Multi-Institutional Disposition Authorities (MIDA) in this collection are issued by the Librarian and Archivist to provide direction to government institutions subject to the Library and Archives of Canada Act regarding the disposal of records managed by all or a multiple number of government institutions. They are designed to eliminate the need for government institutions individually to prepare submissions for and negotiate agreements with the Librarian and Archivist for records which have similar administrative or operational status.

The Canadian Firearms Registry, which includes the Canadian Firearms Information System, is part of Canadian Firearms Program of the Royal Canadian Mounted Police (RCMP) which is a Schedule I government organization according to the Access Information Act which means it would fall under Disposition rules and LAC ACT which states that:

The disposal of records managed by government institutions occurs under the Library and Archives of Canada Act. A government institution is subject to the Library and Archives of Canada Act if it is listed in Schedule I of the Access to Information Act or in the Schedule to the Privacy Act. (see MIDA)

The Treasury Board Policy on Information Management states, keeping in mind this is a policy not a directive, a regulation nor an act, but does say some nice things, not necessarily accompanied with budgets on the management of records:

3.1 Information is an essential component of effective management across departments. The availability of high-quality, authoritative information to decision makers supports the delivery of programs and services, thus enabling departments to be more responsive and accountable to Canadians.

Managing information and records using a whole-of-government approach where legislation permits, supports managers’ ability to transform organizations, programs and services in response to the evolving needs of Canadians. While information management encompasses records, as well as documents, data, library services, information architecture, etc., records and their management are mentioned at key points in the policy for the purpose of emphasis. Integrating information management considerations into all aspects of government business enables information to be used and recognized as a valuable asset. All these activities are indicative of a culture that values information.

The Treasury Board Directive on Information Management Roles and Responsibilities. refers back to the LAC Act regarding disposition:

9.2.5 issues records disposition authorities, pursuant to section 12 of the Library and Archives of Canada Act, to enable departments to carry out their records retention and disposition plans;

It would seem that there are Acts, Regulation, Policies and Directives that lean toward not being able to destroy the Gun Registry.

There are however some items in the current  Firearms Act that allow for the registrars to destroy some data in the registry under some prescribed conditions:

84. The Registrar may destroy records kept in the Canadian Firearms Registry at such times and in such circumstances as may be prescribed.

Not to be a stickler, but it does not say all of the records.   And under Other Records of the Registrar the Firearms Act states under 85. that:

Destruction of records

(3) The Registrar may destroy any record referred to in subsection (1) at such times and in such circumstances as may be prescribed.

In addition, Chief Firearms Officers 87. (1) A chief firearms officer shall keep a record of

(a) every licence and authorization that is issued or revoked by the chief firearms officer;

(b) every application for a licence or authorization that is refused by the chief firearms officer;

(c) every prohibition order of which the chief firearms officer is informed under section 89; and

(d) such other matters as may be prescribed.

Destruction of records

(2) A chief firearms officer may destroy any record referred to in subsection (1) at such times and in such circumstances as may be prescribed.

However, According to the Firearms Records Regulations, there are some pretty explicit instructions as to when a record can be destroyed:

DESTRUCTION OF RECORDS

4. (1) Subject to subsection (2), for the purpose of section 84 of the Act, a record kept in the Canadian Firearms Registry shall not be destroyed until after the expiration of 10 years after the date of the last administrative action taken regarding the information in the record.

(2) A record, kept in the Canadian Firearms Registry under paragraph 83(1)(a) of the Act, of a registration certificate that is issued or revoked shall not be destroyed.

5. For the purpose of subsection 87(2) of the Act, a record kept by a chief firearms officer shall not be destroyed until after the expiration of 10 years after the date of the last administrative action taken regarding the information in the record.

6. (1) Despite section 5, the following records shall not be destroyed until after the death of the individual in respect of whom they are kept:

(a) the record attesting to the fact that the individual has met the requirements of section 7 of the Act, including the date and place of any course or test; and

(b) records of every certification issued under paragraph 7(4)(a) of the Act.

(2) Despite sections 4 and 5, records of prohibition orders kept under paragraph 87(1)(c) of the Act and records of information concerning prohibition orders made under section 147.1 of the National Defence Act shall not be destroyed until after the death of the individual in respect of whom the record is kept unless the individual meets the requirements of subsection 7(3) of the Act.

To summarize, the Firearms Act permits the destruction of some records in the Canadian Firearms Information System under certain circumstances and these are stipulated in the act.  Data can be destroyed but not arbitrarily, not the entire database,  and it should be deposited at Library and Archives Canada.

Bill C-19 An Act to amend the Criminal Code and the Firearms Act is however proposing to override all those good laws, regulations, directives, and policies with the following:

29. (1) The Commissioner of Firearms shall ensure the destruction as soon as feasible of all records in the Canadian Firearms Registry related to the registration of firearms that are neither prohibited firearms nor restricted firearms and all copies of those records under the Commissioner’s control.

(2) Each chief firearms officer shall ensure the destruction as soon as feasible of all records under their control related to the registration of firearms that are neither prohibited firearms nor restricted firearms and all copies of those records under their control.

(3) Sections 12 and 13 of the Library and Archives of Canada Act and subsections 6(1) and (3) of the Privacy Act do not apply with respect to the destruction of the records and copies referred to in subsections (1) and (2).

It is in this fine print that we need to be watchful of this Government, as it is making rules to abolish a program and destroy records and it is trying to do so legally or at least using legal means to do so.  DATA seems to really be a FOUR LETTER word for this Government, they do not like the census, they do not use data to support their policies such as building jails, and they do not really like accountability, since in the end data help citizens assess and evaluate how their governments are doing.

No record, no way of evaluating!

Thanks to those who pointed to resources from CARTA, CivicAccess list, Twitter and CAPDU List.

The fight to Save the Census Continues Print E-mail
2011 Census

The fight to save the Census continues as CCSD et al vs The Government of Canada will be heard in the Federal Court on November 23, 2011 at 09:30AM.  CCSD and 12 other partners are fighting for Canada’s equal right to be counted in the Mandatory Short Form, the only mandatory tool left in the group of census surveys that reaches every Canadian.

The Short form (little more than a head count) only asks 10 questions of Canadians,  none of which determines one’s ethnicity and cultural heritage, aboriginal status or disability. The exclusion of the important groups of Canadians is a clear breach of our Charter of Rights and Freedoms. Gaps in data of this measure will ensure that decision makers at every level, will not have the necessary information to serve these important groups of Canadians. Perhaps that’s the point, is this marginalization by design?

We deserve better, these groups of Canadians deserver better, join us in this important moment in Canadian history.

If you would like more information or want to get involved, contact our President, Peggy Taillon at taillon@ccsd.ca.

Make a donation to the data defence fund and become of a member of CCSD.

Partners in the challenge include:

SOCIAL PLANNING TORONTO,
COMMUNITY DEVELOPMENT HALTON,
SOCIAL PLANNING COUNCIL OF WINNIPEG
CANADIAN ARAB FEDERATION,
ONTARIO COUNCIL OF AGENCIES SERVING IMMIGRANTS,
COUNCIL OF AGENCIES SERVING SOUTH ASIANS,
CANADIAN MENTAL HEALTH ASSOCIATION TORONTO,
AFRICAN CANADIAN LEGAL CLINIC,
NATIONAL ABORIGINAL HOUSING ASSOCIATION,
SOUTH ASIAN LEGAL CLINIC OF ONTARIO,
AFRICAN CANADIAN LEGAL CLINIC,
METRO TORONTO CHINESE SOUTHEAST ASIAN LEGAL CLINIC

Via: Peggy Taillon, President and CEO of CCSD

I entered into the discourse on open data to facilitate the production of these types of reports.

Social Justice in the OECD – How Do the Member States Compare?  Sustainable Governance Indicators 2011

I am really interested in public policy issues such as social justice, health inequality and the environment and hope that open data and open government policies will lead to being able to access these types of data, especially at the neighbourhood scales. I hope that apps will open the door to access, but that eventually we will work toward comprehensive access to data for this type analysis and develop new ways to dialogue between citizen and government using data for evidence-based decision-making.

Currently in Canada, it is incredibly difficult to put one of these reports together. The way data are aggregated differ and because one has to try and pry data from multiple federal agencies, multiple agencies in each province and territory and from a number of municipal agencies. Because of staff changes in government offices, contacts are lost and numerous cold calls have to be remade and data renegotiated.

Page 14 of this report shows the model used to create the indices in this OECD report. At a glance there are 29 variables, each consisting between one to 5 data sets suggesting that potentially these data may need to be accessed from more than 50 different public officials at different levels of government, divisions, departments, etc. Then there is the negotiating of use, licenses, costs, aggregation, accuracy, timeliness and formats since no two agencies even within one government department follow the same rule book and in fact, access is often determined by the mood of the public official or what they think the rules are. Doing a time series is even more complex as data are not collected at the same intervals. A follow up report to track trends requires almost the same amount of work since the data gathering process often has to start nearly from scratch. This is a highly inefficient and cost prohibitive process.

To make matters worse, in Canada, we have lost our think tanks and national social policy research organizations who used to do this kind of work as their funding was cut, and of course we have lost the census.

I hope we can think of open data and open government to include apps to get the bus, find a skating rink or remember to take out the garbage, but more importantly, to inform public policy on transit, public health, and the environment. Also, with open data we need the resources to produce information products such as this report. Many things can be crowdsourced, a census and this type of analysis cannot and there is a role for government and non profit organizations to translate the data into meaningful information and then for us to use that knowledge to improve, track and critique or develop new programs to address what the data tell us.

Apps rely on one or two datasets, these reports rely on hundreds. I want the hundreds which requires a broader open data policy in Canada at all levels of government and I would go further to suggest that open data needs to move beyond the institutional boundaries of IT and CIO divisions and into thematic areas, as that is where data for these indicators are produced and owned.

I met Alex at the Cybera Summit at the Banff Centre in October and that is where I was  introduced to the WEHUB. There are many interesting ways to do open data, science and to use the cloud to do so.  I invited Alex to prepare the following guest post about how WEHUB  does it.
********************************

Water and Environmental Hub…aggregating water data from across North America and making it available through an API

by:

Alex Joseph, Executive Director – Water and Environmental Hub 

As anyone searching for water data from multiple sources knows…there isn’t really a Google for water data. 

A search for water data often results in a web page with a phone number to call someone, or an anonymous info request form. The water datasets that are available are often embedded as graphs in .pdf files obscuring the raw data or available in real time but embedded in html code on web pages. In the best cases, raw water data is available in large .zip files where you get the whole dataset or the opposite, you are faced with downloading hundreds of individual observation stations and then try and sew together hundreds of spreadsheet files, hoping that the columns all line up!

It gets even more time consuming and expensive when one tries to find water data that crosses political boundaries. Imagine the effort required to find data on the “Lake Winnipeg Watershed”? A search involves multiple provinces, states, 3 levels of government, multiple departments within those governments etc. etc. with a high probability that each of those datasets is in a different format.

Besides the challenges with access to water data, the few water datasets that are accessible on the web are unlikely to be provided through an API. Thus, those generous web developers that attended the World Bank sponsored Water Hackathons last week likely found that very little water data is available through an API allowing them to build dynamic water apps….

…but this is changing.

The Water and Environmental Hub (WEHUB) project is an open cloud-based web platform that aggregates, federates, and connects water data and information with users looking to search, discover, download, analyze, model and interpret water and environmental-based information. By combining water expertise with an open web development approach and an entrepreneurial foundation, the project hopes to spur economic diversification and benefit both public users and the private sector by improving the access to water data and tools for academia, government, industry, NGOs and the general public.

The WEHUB also enables organizations and users to develop customized applications on top of the WEHUB platform using our (RESTful) API, so that the data can be easily shared, integrated, leveraged, and customized.

The web platform is structured as a three-tiered system with a Client, Server and Database.  Each tier in the system is divided into components that address the catalogue, spatial and non-spatial data, and the social network requirements.  The catalogue acts as the index for the data and allows for easy search, download and upload of the data. The spatial data is shown on the client – as a map – making it easy for the user to visualize the data.  The social network allows for commenting, flagging and sharing of data. The WEHUB employs a Representational State Transfer (REST) software architecture. Open standards (e.g. OGC standards such as WMS, WFS, SOS, WaterML, GroundwaterML) are used whenever practical, efficient and economical to meet the needs of users.

In terms of geographical scope, the project began with Alberta and Western Canadian water data and information, a region to which the partners have relevant expertise and networks. As development successes are achieved, the project has extended across North America, with scalability a key design thrust.

The University of Toronto Map and Data Librarians put together a really fun panel for Open Access Week with the City of Toronto Open Data team and Jury Konga on the topic of Open Data. As promised here are my slides. There were some great questions from the audience and it was a very well attended session.

There is also an honourable mention to the Toronto Wellbeing initiative.

This year I thought I would honour Wendy Watkins a founder with Ernie Boyko of the Data Liberation Initiative (DLI) in Canada on for Ada Lovelace Day.

Wendy Watkins

In Canada our census data is sold back to us under a cost recovery program initiated by the Brian Mulroney Conservative Government in the early 1980s.  In fact, the Conservatives of that day also tried to Cancel the census but alas the constitution prevented them for doing so and instead they cut Statistics Canada’s budget severely which instituted a very regressive cost recovery practice.  The prices were so high that not only could citizens not afford to use their own data, universities encouraged students to use free US census data since they did not have the resources to pay for Canadian census data.  During those years, Canadians became experts on the US and not on Canada.

It is through the hard work of Wendy Watkins, her collaborators, data & map & research librarians that Canadian universities now have Census data for faculty and students along with associated census geographic files.  I had the good fortune as a student to benefit from the DLI.  Here is an excerpt from one of Wendy’s papers about the history of the DLI:

In April 1993, after receipt of the “Liberation Paper,” the Social Science Federation of Canada (SSFC) hosted a meeting with representatives from the Social Sciences and Humanities Research Council (SSHRC), the Association of Universities and Colleges of Canada (AUCC), the Canadian Association of Research Libraries (CARL), the Canadian Association of Public Data Users (CAPDU) and other interested parties to devise a strategy to make Canadian data more readily available to the education and research communities. The meeting resulted in the striking of a smaller working group, under the aegis of the SSFC, to devise a plan that would be acceptable to all parties. Statistics Canada and the DSP [Depository Service Program] played advisory roles in this process. While the initiative has involved government in an advisory role, it is unique in that it was conceived and developed by members of the Canadian research community.

The working group, consisting of researchers, representatives from CARL and CAPDU, as well as members of the SSFC, held a series of meetings over the next months. Advice from both Statistics Canada and the Depository Services Program was invited and found to be invaluable. When the group had formulated a working document to which both Statistics Canada and the DSP agreed, meetings were arranged with senior management in several government departments. The SSFC also met with Ministers and their executive assistants in order to move the proposal forward. Finally, in December 1995, the DLI had received a strong enough informal blessing that the project was deemed to be a go. Letters of agreement were distributed and data began to be released.

More officially, the DLI received approval by the Treasury Board Ministers in a February 1996 decision. It was subsequently included as part of the federal government’s Science and Technology Strategy in March. Most recently, in October 1996, it was officially announced by Dr. John Gerard, Minister of State for Science and Technology at a press conference held in conjunction with National Science and Technology Week and the 30th anniversary of Carleton University’s Data Centre. ( Data Liberation and Academic Freedom, 1996).

The DLI not only fueled Canadian research, it promoted academic freedom, advanced data driven informed decision-making and created a new class of librarian called data librarians and also data centres in libraries.  Data also became artifacts to be collected in libraries, which added a new practice of adding digital material in a catalog along with hard copy books on shelves, the DLI spurred the early adoption of the Internet with the use of basic FTP protocols to transfer data from Statistics Canada and university libraries, and it was the forerunner in the acquisition of digital materials.  The DLI also promoted collaboration between universities and government via a consortium agreement that has been embraced by other organizations such as the Community Social Data Strategy.  Finally, the DLI also accelerated a new type of expertise in data metadata, data catalogs, data citation and data preservation (b).

Today there is a very vibrant DLI community of practice that shares knowledge on a yearly basis at DLI Bootcamps, maintains a repository of training materials, an active blog Data Interests Group for Reference Services and actively exchanges expertise on a DLILIST listserv.

The DLI also politicized access to data very early on and in a sense they began the discourse on data access in Canada.  The cancellation of the 2011 Census being one of the big issues DLI supporters took on.  Further Wendy Watkins and her colleagues participate in key roundtable discussions on access to research data, the preservation of data and develop important infrastructures that disseminate Canadian Data.

Data users and Canadians can thank Wendy for being on the vanguard of open data, open government and data liberation in Canada and for building an incredible cadre of data literate librarians, faculty and students.  Open Data initiatives in Canada can benefit from her work and should recognize Wendy as one of their data access pioneers.  Now we just need to have a census and for those data to be cost free to the public.

Some Watkins’ Papers:

« Older entries § Newer entries »