Public Consultation Summary

INTRODUCTION

From June to September 2018, the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Social Sciences and Humanities Research Council of Canada (SSHRC) (the agencies) held an online consultation on the draft Tri-Agency Research Data Management Policy (the policy).

The agencies received 130 written submissions from a broad range of stakeholders, including universities, colleges and polytechnics; libraries; government agencies and departments; organizations that raise awareness and build supports for research data management; and academic associations and individual researchers from various health, natural sciences, engineering, social sciences and humanities research disciplines. Some stakeholders posted their submissions online (e.g. Research Data Canada and the Portage Network).

This report summarizes the consultation feedback and, where warranted, clarifies the framing, scope and aims of the policy. It is structured around the following themes:

  1. Policy objectives, scope and implementation timeline
  2. Research data management in the context of Indigenous research
  3. Institutional strategies
  4. Data management plans
  5. Data deposit
  6. Monitoring and compliance

Annex A provides a detailed breakdown of respondents and feedback.

Stakeholder feedback is essential to developing a policy that advances data management in Canada and, as a result, Canadian research excellence. The agencies wish to thank everyone who responded to our call for comments on the draft policy.

THEME 1: POLICY OBJECTIVES, SCOPE AND IMPLEMENTATION TIMELINE

Many respondents commented on the principle and rationale of the policy, its scope and the agencies’ implementation timeframe.

Principle and rationale

Overall, the agencies found that there is broad support for the underlying principle of the policy: that research data collected with the use of public funds should be responsibly managed and, where appropriate, available for reuse by others. The agencies found support for the policy across a variety of stakeholder groups, including researchers, research institutions and organizations operating in the Canadian research data management (RDM) community.

Many of the stakeholders who expressed approval of the policy said the text could be stronger and more compelling. In particular, they suggested that data management plans (DMPs) should be required for all publicly funded research projects and that the data deposit requirement should be more specific (see themes 4 and 5, respectively). Some respondents also suggested that the policy should more explicitly promote principles of open data.

Meanwhile, respondents who voiced reservations about the policy were mostly concerned about the resources and capacity that would be needed to comply. This issue is discussed in themes 3, 4 and 5.

What we heard

After reading the proposed policy I just wanted to share my strong support for it and hope that the official implementation date will be sooner rather than later. This proposed policy is excellent.

- (University researcher)

 

We believe that not only is it important for the three agencies to provide a data management policy, but that such action is required since public funds are involved in generating these data.

- (College, translated from French)

 

In all of [our] conversations regarding the draft policy, there was general agreement that the direction of the policy is good…. There was also recognition that funder policies like this are critical to changing research culture, as demonstrated by similar policy contexts in other jurisdictions.

- (National research data organization)

 

Although I feel that there is great progress toward publishing and sharing data, I think that the policy does not go far enough (or is not detailed enough).

- (University researcher)

Policy scope and definitions

Numerous respondents asked about the scope of the policy and, in particular, the policy’s requirements for storing and sharing data (see theme 5). The policy does not require all research data to be shared. Although the agencies strongly encourage researchers to provide access to data where ethical, legal and commercial requirements allow and in accordance with the standards of their disciplines, the policy recognizes that some data should not be shared.

Some respondents asked why the draft policy excludes scholarship and fellowship holders and research chairs, writing that the policy should apply to these groups. Some respondents suggested that the policy should also apply to other government-funded research.

Respondents asked how the policy would apply to international research collaborations or other partnerships, where several (possibly conflicting) RDM policies could overlap. As well, data ownership in international research is often unclear and agency-funded researchers may not be in complete control of how data are managed and preserved.

Some respondents suggested the agencies promote the use of the FAIR principles (findable, accessible, interoperable and reusable). In this way, the policy would capture international best practices and advance their adoption in Canadian research. Promoting the FAIR principles would also help clarify the agencies’ expectations, facilitate compliance (specifically with the data deposit requirement) and lead to a more effective policy.

Researchers and institutions expressed interest in developing and sharing best practices for institutional strategies, DMPs and data deposits.

Respondents also asked about the draft policy’s definition of “data”, raising the following considerations:

  • The document does not explicitly acknowledge the wide range of different and equally valid models of scientific inquiry, including qualitative methodological approaches;
  • The definition of data used is not consistent with the definition in the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (2018) “TCPS2 (2018)”;
  • The distinction between digital and non-digital data and how they should be treated should be clarified;
  • The draft does not sufficiently draw the distinction between “research materials” and “research data.”

What we heard

While the existing draft indicates that the ethical and legal obligations must be complied with, we believe that further clarification is needed to effectively regulate the accessibility of data in repositories and to ensure some degree of consistency in practices across the country.

- (University research ethics committee, translated from French)

 

Many researchers will not be able to share their data and to deposit the data in any repository.

- (University researcher)

 

It is unclear why Chairholders are exempt from the policy. It is understood that the policy would be hard to apply to scholarship and fellowship holders due to the transient nature of their positions. However, Chairholders often derive significant funding from the tri-agencies and would be held to different standards based on the tri-agency program. This would be difficult to manage at an institutional level and at the Chairholder level. In addition, it is unclear if Chairholders are exempt completely or only as it relates to their specific chair funding.

- (University)

 

For many large, international collaborative projects, investigators combine individual-level data (e.g., genetics, imaging, personal health information) from multiple cohorts, each acquired in different countries. Our site-specific investigators would not be able to deposit such data as they do not own these data. Hence, the policy will need to include some guidance in such multicohort studies.

- (Research institute)

Implementation timeline

Respondents asked for clarification on the agencies’ proposed timeline for the policy’s implementation. There was broad agreement that the policy should be phased in. Many respondents recommended an implementation period spanning several years. Similarly, some respondents suggested that the agencies should pilot certain policy elements (e.g., DMPs) before they are implemented across the agencies’ funding opportunities.

What we heard

Once the agencies have completed consultation and begin the proposed incremental implementation of the policy, establishing clear, reasonable deadlines for each aspect of the policy (e.g., establishing institutional strategies, requiring DMPs as part of grant proposals and mandatory data deposits) will be critically important. In setting these timelines, the tri-agencies will need to allow sufficient time for institutions to reconcile strategies with existing policies, as well as increase awareness of the new requirements and build DMP writing capacity among researchers.

- (Non-governmental organization)

THEME 2: RESEARCH DATA MANAGEMENT IN THE CONTEXT OF INDIGENOUS RESEARCH

Respondents highlighted concerns related to the treatment of data in the context of Indigenous research. This theme was mentioned in submissions from Indigenous groups, individual researchers, research institutions, government agencies and non-governmental organizations. While the draft policy itself does not establish specific standards for RDM, the agencies expect the policy to stress the importance of following accepted principles of RDM, particularly in the context of Indigenous research.

Policy treatment of data in the context of Indigenous research

Indigenous organizations and leaders stressed the importance of acknowledging a need for ensuring Indigenous ownership and control of data (e.g. the OCAP®Footnote 1 principles of ownership, control, access and possession). They highlighted that Indigenous communities often have difficulty accessing research data and, as a result, are unable to use and benefit from them. They also stressed the need to remove power, control and benefit away from individual researchers and toward communities. Indigenous peoples and communities want to be empowered in the research process to decide how data are used, published, stored and shared. This would include a process through which data governance is co-developed and/or community-approved and Indigenous data repositories are owned and controlled by Indigenous communities.

Respondents suggested the policy should specifically acknowledge and reflect accepted principles of Indigenous RDM to support Indigenous ownership and co-creation of research. In addition to the OCAP® principles, the principles most often referenced were data sovereignty, the protection of collective identity, and free, prior and informed consent.

Stakeholders recommended several ways to establish Indigenous ownership and control of research data. Some suggested the policy should specifically reference existing principles of RDM in the context of Indigenous research, such as OCAP®. Respondents also suggested that the policy should explicitly state its alignment with Chapter 9 of TCPS 2 (2018) (“Research Involving the First Nations, Inuit and Métis Peoples of Canada”) and that adherence to TCPS 2 (2018) supersede the implementation of any element of the policy. However, some stakeholders had reservations about agency policies being out of touch with Indigenous reality. Finally, some respondents suggested that section 3.2 of the policy (Data Management Plans) include additional, specific requirements for the management of data in the context of Indigenous research. These requirements would include, among other things, that research proposals include plans that address data governance and are community approved.

What we heard

In order to truly “enable excellence in data management,” this policy must integrate the right of Indigenous Peoples in Canada to self-determination—including data governance—through inclusion of the principles of ownership, control, access and possession (OCAP®).

- (Non-governmental organization)

 

Given the specificities of research data with Indigenous communities and given the historical context surrounding this type of research, [we recommend] that this topic be addressed explicitly (e.g. it could take the form of a reference to [Chapter] 9 of the TCPS 2) in the policy after proper consultation with relevant stakeholders, including Indigenous researchers and communities.

- (University)

Approach to policy engagement

Respondents raised concerns about the agencies’ approach to discussions with Indigenous communities, groups and leaders. Some thought the agencies did not provide sufficient time to engage in discussions and provide meaningful feedback. Respondents recommended the agencies focus on developing sustained, meaningful relationships with Indigenous partners to contribute to the development and implementation of the policy, as well as ongoing matters related to RDM.

THEME 3: INSTITUTIONAL STRATEGIES

Section 3.1 of the draft policy requires each institution administering tri-agency funds to create an institutional RDM strategy. Many submissions, especially from research institutions, commented on this requirement.

Costs and capacity

The primary concern for many institutions was that they were unequipped to develop and implement an institutional RDM strategy. Respondents pointed to the specialized knowledge and skills needed to account for the various ethical, legal and logistical considerations in developing and implementing an effective strategy. Some stakeholders also commented that institutional strategies should describe how institutions would build expertise to support RDM through a comprehensive training and development program, but such programs require significant time and resources.

Some institutions felt there would be considerable costs in developing the infrastructure and implementing an RDM strategy, especially providing—or supporting access to—recognized repositories or other platforms that securely preserve, curate and provide continued access to the research data (see theme 5). They suggested that additional resources would be needed to implement institutional strategies.

These concerns were particularly acute for smaller and less research-intensive institutions. Moreover, the college, cégep and polytechnic community highlighted that they do not receive the same funding for indirect costs as universities. As a result, they operate with a shorter-term vision than universities, without as many resources to invest in institutional strategy development and RDM capacity building.

The agencies would expect strategies to reflect each institution’s strengths, challenges, goals and circumstances. The strategy requirement’s aim would not be for a common standard across institutions. Some institutions have already developed significant RDM capacity. This would be captured in their strategies. For other institutions, the strategies would be largely aspirational in nature. However, the agencies would expect all strategies to reflect a genuine commitment to implementing world-class RDM practices to the best of each institution’s ability.

In the draft policy, the agencies proposed several items that research institutions could include in their strategies. Respondents requested that the agencies share a more detailed set of guidelines and provide more information on RDM best practices, particularly as they pertain to research institutions.

A handful of institutions referenced the agencies’ proposal that institutional strategies include ensuring researchers create DMPs. These respondents suggested this responsibility should fall to the agencies because the agencies will ultimately receive the DMPs from researchers submitting funding applications (see theme 4).

What we heard

The tri-agencies should seek to develop their own standards and expectations for both institutional strategies and data management plans, so both institutions and researchers have a clearly defined set of criteria to fulfil.

- (University researcher)

 

The importance of data management and open access is supported; however, this will present significant costs and challenges for small universities.

- (University)

 

Given the costs involved in digital data management, it would be important to take into account the fact that colleges, unlike universities, do not have access to the Federal Indirect Costs Program (now called the Research Support Fund) or to such provincial contributions. Thus, the financial resources available to colleges for the implementation of a digital data management system are significantly lower than those of universities.

- (College, translated from French)

Other concerns

Respondents also expressed concerns over the possibility of a patchwork of institutional strategies with varying levels of strength and researcher requirements. One of the consequences of uneven institutional commitments could be researchers spending unequal amounts of time and resources on RDM.

Several researchers highlighted the importance for institutions to design strategies that allow researchers to continue to use their preferred research software. They were concerned that the policy could lead to the need for closed-source, proprietary software that is not compatible with their research programs.

Some respondents stated that, given the rapidly evolving nature of RDM, institutional strategies should be reviewed regularly to ensure they remain consistent with current best practices. The need for review and revision could be made explicit in the policy.

What we heard

The variation between institutional policies has the potential to create considerable inconsistencies. If an institution took a minimalist approach, it would result in inequities in time [and] resources that grant [applicants] are expending between institutions to fulfil the same objectives.

- (Non-governmental organization)

THEME 4: DATA MANAGEMENT PLANS

Section 3.2 of the draft policy encourages researchers to create DMPs. It states that, for specific funding opportunities, the agencies may require DMPs to be submitted to the appropriate agency at time of application. In these cases, the DMPs may be considered in the adjudication process.

Capacity

Researchers and institutions commented that the DMP requirement could have resource implications, and that they would need support to meet the time, skills and capacity demands of complying. Researchers and their disciplines have varying levels of readiness and some will require more time than others to raise awareness of RDM best practices.

What we heard

We feel that it would be appropriate to add logistics to the sequence of specific accountabilities in the data management plan.

- (Research association, translated from French)

 

Implementing good RDM practices and complying with the policy represent a significant investment [of] time and resources: how will the tri-agencies support the implementation costs of the policy? Researchers are concerned that allowing RDM expenses to be budgeted as allowable expenses without a concomitant increase in the value of the awards will simply displace resources away from other parts of their research.

- (National research data organization)

 

If DMPs are not a standard requirement, then providing examples of the "specific opportunities" by agency would help researchers determine if they are required to complete a DMP.

- (University)

 

We encourage the tri-agencies to use more direct and clearer language throughout the policy, particularly by making requirements for data management plans (DMPs), data sharing and other key elements more explicit. In so doing, the policy would provide a stronger mandate for the development of RDM infrastructure and services as an institutional priority, supporting the implementation of the policy and the agency’s vision for research excellence.

- (University)

 

As the text identifies, “data management plans as an essential step in research project design” the best time to do this is at the application stage, not as a separate and independent step.

- (College)

Strength and clarity of language

Many respondents thought the language in Section 3.2 is too vague, specifically in the first paragraph, where the agencies encourage the use of DMPs but do not require them, and then say that DMPs “may” be required in certain funding opportunities. Some felt that if the agencies consider DMPs to be “an essential step in research project design,” then the policy should do more than encourage researchers to create DMPs.

Administration and review of DMPs

Many respondents asked for more details about the administration of DMPs. For example, the draft policy states that, for specific funding opportunities, the agencies may require DMPs to be submitted to the appropriate agency at the time of application. This raised questions about which funding opportunities would be included. Some respondents felt additional details should be provided about how and when DMPs would be submitted and reviewed.

THEME 5: DATA DEPOSIT

Section 3.3 of the draft policy requires grant recipients to deposit data and other research outputs into a recognized digital repository. Specifically, it requires deposit of digital research data, metadata and code that directly supported the research conclusions in journal articles, pre-prints and other research outputs that arise from agency-supported research.

Ownership and access

Respondents commented that research partnerships with private-sector organizations or international colleagues could pose challenges to the data deposit requirement. This can happen when agency-funded researchers do not have full ownership over the research data. In such cases, the partners could object to depositing data, whether because of business interests or because they are bound by different funder policies. Respondents also noted that the deposit requirement could pose capacity challenges. In projects working with especially large datasets, it could be difficult to find repositories that can securely store, preserve and curate all of the data.

Data sharing

The draft policy does not include a data sharing requirement. The policy encourages researchers to provide access to data where ethical, legal and commercial obligations are met. However, it does not require them to do so. Nevertheless, many respondents interpreted the data deposit requirement not only as an obligation to securely preserve research data but also to share it.

Some researchers also had questions relating to data sharing in the context of their disciplines. The agencies encourage disciplinary communities to develop their own expectations for data sharing and recognize that these standards will differ across disciplines and models of research.

What we heard

The policy would require researchers to deposit data into a “recognized repository” (or slight variant on this). Although the FAQs do provide some examples of repositories, neither the policy nor the FAQs define what would qualify as a recognized repository. Mindful of the great variety of existing repositories across fields, the policy should provide more specific guidance on how institutions and researchers could gauge what would be acceptable and “recognized” repositories.

- (National association)

Recognized repositories

Some respondents expressed concerns regarding the language used to refer to repositories. Specifically, they asked for more guidance about what constitutes a “recognized” digital repository. Other stakeholders requested a more detailed definition of “research data” that support conclusions in research outputs and need to be deposited. These respondents suggested that language from the FAQs on this subject could be included in the text of the policy itself.

Capacity

Some respondents raised the issue of a lack of repository capacity in their disciplines. This issue appears to be particularly acute for French-language repositories. Respondents also raised questions about the storage of research data over time and the ownership of research data repositories.

What we heard

Canada needs to invest in developing repositories and platforms for collection and curation of data collected and provide ongoing financial support for these repositories and platforms from funding agencies.

- (University researcher)

 

Since the digital data repositories currently available do not cover all the standards specific to a variety of research disciplines, the time required to develop them must be factored into the policy’s implementation schedule.

- (Provincial association, translated from French)

THEME 6: MONITORING AND COMPLIANCE

Section 5 of the draft policy states that, by accepting agency funds, institutions and researchers accept the terms and conditions set out in the agencies’ policies, agreements and guidelines. In the event of an alleged breach of agency policy, agreement or guideline, the agencies may take steps outlined in the Tri-Agency Framework: Responsible Conduct of Research.

The draft policy does not detail how the agencies would monitor compliance. Many respondents said they would like to know more about how the policy will be monitored and enforced. For instance, institutions asked what their responsibilities would be in monitoring compliance. Some respondents noted that sound monitoring and compliance is crucial for the RDM and data sharing policies to have this effect. One suggestion was for the agencies to mandate the use of a digital object identifier (DOI) or another identifier for DMPs and/or deposited data to aid compliance.

What we heard

There is no language around enforcement, which might mean the same situation as open access requirements. These are not checked and therefore not followed.

- (University librarian)

 

There is a funding agency role in both policing access and in providing resources for maintaining access. There is no better way to ensure compliance than through requirements for further funding. This is a problem when some agencies/funders provide zero indirect cost support and expressly forbid charging for network access and data storage on grants.

- (University researcher)

Capacity

Several respondents noted that enforcing compliance will require a commitment of resources. If institutions were to have this responsibility, they felt they would need support to build this capacity.

 

ANNEX A

 

Who responded

The agencies received 130 written submissions from a broad range of stakeholders, including universities, colleges and polytechnics; libraries; government agencies and departments; organizations that raise awareness and build supports for research data management; and academic associations and individual researchers from various social sciences and humanities (SSH), natural sciences and engineering (NSE) and health research disciplines.

Institutional affiliation
Postsecondary institutions 91
Other organizations and associations 18
Government 12
Academic associations 9
Collective or individual
Collective 78
Individual 52
Discipline group
Multiple 65
SSH 23
NSE 23
Health 19
Language
English 105
French 25

What they said

N/130 % Issue raised
78 60% Concerns about costs/capacity
57 44% Concerns about aspects of the policy not being appropriate for some fields (particularly private or anonymous research data)
51 39% More clarification needed about implementation timeline
39 30% More clarification needed about data repositories
37 28% Interest in plans for monitoring and ensuring compliance
37 28% Proposal to include more precise definitions, especially what research data includes
27 21% Questions about policy not applying to scholarships, fellowships, research chairs
27 21% Interest in plans for administrative flow of DMPs, especially about who will review DMPs
26 20% Concerns about policy not being strong or compelling enough (particularly DMP)
18 14% Questions about academic IP and competitive advantage
16 12% Proposal to include timing guidelines for depositing data
15 12% Proposal to include details about Indigenous research in the policy itself and not in FAQ
15 12% Concerns about Indigenous consultation
13 10% More clarification needed about best practices
13 10% Confusion about whether the institution or researcher is responsible for the researcher having a DMP
13 10% Questions about how the policy will apply to international and other partnerships
13 10% Concern about institutions implementing institutional strategies of varying strength