View source for Journal:Compliance culture or culture change? The role of funders in improving data management and sharing practice amongst researchers

{{Infobox journal article
|name         = 
|image        = 
|alt          = <!-- Alternative text for images -->
|caption      = 
|title_full   = Compliance culture or culture change? The role of funders in improving data management and sharing practice amongst researchers
|journal      = ''Research Ideas and Outcomes''
|authors      = Neylon, Cameron
|affiliations = Curtin University
|contact      = Email: cn at cameronneylon dot net
|editors      = 
|pub_year     = 2017
|vol_iss      = '''3'''
|pages        = e21705
|doi          = [http://doi.org/10.3897/rio.3.e21705 10.3897/rio.3.e21705]
|issn         = 2367-7163
|license      = [http://creativecommons.org/licenses/by/4.0/ Creative Commons Attribution 4.0 International]
|website      = [https://riojournal.com/articles.php?id=21705 https://riojournal.com/articles.php?id=21705]
|download     = [https://riojournal.com/article/21705/download/pdf/ https://riojournal.com/article/21705/download/pdf/] (PDF)
}}
{{ombox	
| type      = content	
| style    = width: 500px;	
| text      = This article should not be considered complete until this message box has been removed. This is a work in progress.	
}}
==Abstract==
There is a wide and growing interest in promoting research data management (RDM) and research data sharing (RDS) from many stakeholders in the research enterprise. Funders are under pressure from activists, from government, and from the wider public agenda towards greater transparency and access to encourage, require, and deliver improved data practices from the researchers they fund.

Funders are responding to this, and to their own interest in improved practice, by developing and implementing policies on RDM and RDS. In this review we examine the state of funder policies, the process of implementation and available guidance to identify the challenges and opportunities for funders in developing policy and delivering on the aspirations for improved community practice, greater transparency and engagement, and enhanced impact.

The review is divided into three parts. The first two components are based on desk research: a survey of existing policy statements drawing in part on existing surveys and a brief review of available guidance on policy development for funders. The third part addresses the experience of policy implementation through interviews with funders, policy developers, and infrastructure providers.

In our review we identify, in common with other surveys, that RDM and RDS policies are increasingly common. The most developed are found among funders in the United States, United Kingdom, Australia, and European Union. However many other funders and nations have inspirational statements or are developing policy. There is a broad pattern of policy development moving from aspiration to recommendations, requirements, and finally reporting and auditing of data management practice.

There are strong similarities across policies: a requirement for data management planning, often in grant submissions; expectations that data supporting published articles will be made available; and, in many cases, requirements for data archiving and availability over extended periods beyond grants. However there are also important differences in implementation.

There is essentially no [[information]] available on the uptake and success of different policies in terms of compliance rates, or degrees of data availability. Many policies require a data management plan as part of grant submission. This requirement can be enforced, but there is disagreement on the value of this. One view is that requirements such as DMPs are the only way to force researchers to pay attention to these issues. The other is that such requirements lead to a culture of compliance in which the minimal effort is made and planning is seen as a “tick-box” exercise that has no further value. In this view, requirements such as DMPs may actually be damaging the effort to affect culture change towards improved community practice.

One way to bring these two views together is to see DMPs as living documents that form the basis of collaboration between researchers, funders, and data managers throughout the life of a research project. This approach is reflected in guidance on policy development that emphasizes the importance of clarifying responsibilities of various stakeholders and ensuring that researchers are recognized for good practice and see tangible benefits.

More broadly, this points to the need for a program of improving RDM and RDS to be shared projects, with the incentives for funders and researchers aligned as far as is possible. In the interviews, successful policy implementation was often seen to be dependent on funders providing the required support, both in the form of infrastructure and resourcing, and via the provision of internal expertise among program managers. Where resources are limited, leveraging other support — especially from institutional sources — was seen as being as important as was ensuring the scope of policy requirements were commensurate with the support available and readiness of research communities.

Throughout the desk research and interviews, a consistent theme is the desire for cultural change, where data management and sharing practices are embedded within the norms of behavior for research communities. There is general agreement that progress from inspirational policies to actually achieving compliance is challenging and that broad cultural change, with the exception of specific communities, is a long way off. It is interesting to note that discussion of cultural change is largely externalized. There is little engagement with the concept of culture as an issue to consider or work with and very little engagement with models of how cultural change could be enabled. The disagreement over the value of DMPs is one example of how a lack of active engagement with culture and how it changes is leading to problems.

'''Key findings''': 

* Policies on RDM and RDS are being developed by a number of agencies, primarily in the Global North. These policies are broadly consistent in inspiration and outlines but differ significantly in details of implementation.

* Policies generally develop along a path starting with inspirational statements, followed by recommendations, then requirements, and finally auditing and compliance measures.

* Measurement of policy adoption and compliance in terms of the overarching goals of increased availability and re-use of data is not tracked and is likely unmeasurable currently.

* Data management plans are a central requirement for many policies, in part because they can be made compulsory and act as a general focus for raising awareness.

* There are significant differences in the views of stakeholders on the value of data management planning in its current form.

* Some stakeholders regard them as successful in raising awareness, albeit with some limitations.

* Some regard them as actively damaging progress towards real change in practice by making RDM appear as one administrative activity among the many required for grant submission.

* Successful policy implementation is coupled with funder support for infrastructure and training. Seeing RDM as an area for collaboration between funders and researchers may be valuable.

* Internal expertise and support from a funder often leads to a gap, which becomes a problem with monitoring and implementation.

* DMPs can be a helpful part of the process, but it will be important to make them useful documents throughout and beyond the project.

* If the objective of RDM and RDS policy is cultural change in research communities, then direct engagement with and understanding of the various cultures of researchers and other stakeholder communities, alongside frameworks of how they change, is an important area for future focus.

'''Keywords''': review, data management planning, data sharing, policy, implementation, culture change, policy design

==Introduction: Aims and scope==
This review was written in support of the International Development Research Center (Canada) program ''Exploring the opportunities and challenges of implementing open research strategies within development institutions''.<ref name="NeylonExploring16">{{cite journal |title=Exploring the opportunities and challenges of implementing open research strategies within development institutions |journal=Research Ideas and Outcomes |author=Neylon, C.; Chan, L. |volume=2 |pages=e8880 |year=2016 |doi=10.3897/rio.2.e8880}}</ref> The project was constructed as a pilot in which the proposed IDRC data sharing policy is tested in the context of eight funded research projects. The intent of the review is to support the project and its participants by examining the existing literature on funder policies on research data management and sharing, and, using interviews with relevant experts (see Neylon's project data package for recordings and transcripts<ref name="NeylonDataset17">{{cite journal |title=Dataset for IDRC Project: Exploring the opportunities and challenges of implementing open research strategies within development institutions. International Development Research Center |journal=Zenodo |author=Neylon, C. |year=2017 |doi=10.5281/zenodo.844394}}</ref>), to develop an understanding of the current state of policy implementation and its challenges.

The review is focused on funder policy, as opposed to policies of research institutions or disciplinary communities. It aims to develop an overview of the funder policy landscape as a whole, examine existing guidance for funders on policy development, and to probe the issues that funders, and the researchers and institutions that they fund, are facing in the implementation and adoption of policies.

As the development of data sharing and data management policy and practice among funders of development research has been limited to date, the review does not focus specifically on data sharing in a development research context. However it does draw out issues that are likely to be relevant in this context, including the provision of infrastructure; (mis)alignment among the culture of research disciplines, institutions, and stakeholders; and motivations of researchers and funders. The surfacing of these issues is intended to guide the research program in its examination of data sharing in the real world of the projects that are taking part.

==Research funder policies on data sharing and management==
Research funders are increasingly developing policies on research data management (RDM) and research data sharing (RDS) or open data. The oldest and most developed policies are in the United States, Australia, and the United Kingdom. The European Union has imposed new requirements for the Horizon 2020 program, and various member states are developing policies. Canada’s Tri-Councils have been developing policy and infrastructure over the past several years.

A range of development research funders have also developed RDM and RDS policies. The U.K. Department for International Development included requirements for data management and sharing within the DFID Research Open and Enhanced Access Policy that came into effect in November 2012. The requirements included the provision of a data management plan when proposals were submitted, and datasets were to be made available through institutional and subject repositories within 12 months of final data collection.

The World Bank Open Access Policy also includes requirements for datasets associated with formal publications to be made available through the Bank’s Open Knowledge Repository. The World Bank also has a strong record of making general economic and research data available through its data portal and has a very strong disclosure policy in general, relative to other similar organizations.

===Review methodology===
Information on funder policies was sourced from the ''Comprehensive Brief on Research Data Management''<ref name="ShearerComp15">{{cite web |url=http://science.gc.ca/default.asp?lang=En&n=1E116DB8-1 |archiveurl=http://web.archive.org/web/20151001135755/http://science.gc.ca/default.asp?lang=En&n=1E116DB8-1 |title=Comprehensive Brief on Research Data Management Policies |author=Shearer, K. |work=Science.gc.ca |publisher=Government of Canada |date=07 April 2015 |archivedate=01 October 2015 |accessdate=04 December 2017}}</ref>, ''Current Best Practice for Research Data Management Policies''<ref name="HodsonCurrent14">{{cite journal |title=Current Best Practice for Research Data Management Policies |journal=Zenodo |author=Hodson, S.; Molloy, L. |year=2015 |doi=10.5281/zenodo.27872}}</ref>, as well as two online collations of funder policies with a United States federal agency<ref name="WhitmireATable16">{{cite journal |title=A table summarizing the Federal public access policies resulting from the US Office of Science and Technology Policy memorandum of February 2013 |journal=Figshare |author=Whitmire, A.; Briney, K.; Nurnberger, A. et al. |year=2016 |doi=10.6084/m9.figshare.1372041.v5}}</ref> and United Kingdom focus.<ref name="DCCOver12">{{cite web |url=http://www.dcc.ac.uk/resources/policy-and-legal/overview-funders-data-policies |title=Overview of funders' data policies |publisher=Digital Curation Center |date=2012 |accessdate=16 August 2015}}</ref> Specific policy documents for the U.K. Department for International Development<ref name="DFIDResearch13">{{cite web |url=https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/181176/DFIDResearch-Open-and-Enhanced-Access-Policy.pdf |format=PDF |title=DFID Research Open and Enhanced Access Policy v1.1 |author=Department for International Development |date=January 2013 |accessdate=18 January 2016}}</ref>, the World Bank<ref name="WBWorld12">{{cite web |url=http://documents.worldbank.org/curated/en/992881468337274796/World-Bank-Open-Access-Policy-for-Formal-Publications |title=World Bank Open Access Policy for Formal Publications (English) |author=World Bank |date=01 April 2012 |accessdate=28 January 2016}}</ref>, the Australian Research Council, and National Health and Medical Research Council were also examined.

A focus was retained on funder policies as opposed to those of research institutions. National policies (with the exception of the United States OSTP Memo) and statements from trans-national bodies are high-level and generally inspirational rather than specific. As the review focuses on the impact of implementation details, these were not considered, except where they had a direct effect on the details of research funder requirements.

===Funder motivations===
The motivations for funders to develop data management and sharing policies are generally focused around two broad issues. The first of these is maximizing the impact and reach of research, and the second is concerns around citizen access and engagement with research. Fecher and Friesike<ref name="FecherOpen13">{{cite journal |title=Open Science: One Term, Five Schools of Thought |journal=SSRN |author=Fecher, B.; Friesike, S. |year=2013 |doi=10.2139/ssrn.2272036}}</ref> identified five categories of framing within the broader open science movement: democratic, pragmatic, infrastructure, public, and measurement. While issues might be raised with this classification, these framings are a useful lens for examining funder motivations to support data management and sharing. Funder policies and the documents around them often refer to the motivations for developing policy and guidance, and in the area of data, these public motivations fall largely into the "democratic" and "pragmatic" categories, with gestures towards the "public" category. These motivations tend to also align with public statements of governments addressing broader open data agendas.

Other categorizations of drivers have been described<ref name="BorgmanTheCon12">{{cite journal |title=The conundrum of sharing research data |journal=Journal of the Association for Information Science and Technology |author=Borgman, C.L. |volume=63 |issue=6 |pages=1059–1078 |year=2012 |doi=10.1002/asi.22634}}</ref><ref name="LeonelliWhyThe13">{{cite journal |title=Why the Current Insistence on Open Access to Scientific Data? Big Data, Knowledge Production, and the Political Economy of Contemporary Biology |journal=Bulletin of Science, Technology & Society |author=Leonelli, S. |volume=33 |issue=1–2 |pages=6–11 |year=2013 |doi=10.1177/0270467613496768}}</ref>, and policy documents enumerate a range of positive outcomes for RDS.<ref name="OECDOpen">{{cite web |url=http://www.oecd.org/sti/outlook/e-outlook/stipolicyprofiles/interactionsforinnovation/openscience.htm |title=Open Science |author=OECD |accessdate=17 September 2015}}</ref><ref name="RSScience12">{{cite web |url=https://royalsociety.org/~/media/policy/projects/sape/2012-06-20-saoe.pdf |format=PDF |title=Science as an open enterprise |author=The Royal Society Science Policy Centre |publisher=The Royal Society |date=June 2012 |accessdate=27 January 2016}}</ref><ref name="ShearerComp15" /> The value of the categorization of Fecher and Friesike is its basis in the analysis of the discourse of these documents as a means of interrogating the underlying motivations.

The "democratic" framing is described as being concerned that access to knowledge, and the ability to reuse it, should be equitably distributed. It is distinguished from the "public" framing, which is concerned with the public, or rather non-professional engagement in the consumption and production of knowledge, rather than equity per se. Reference to these framings in policy statements is generally political, aligning with government language on democratization and engagement.

For instance the U.K. Department for International Development in its Research Open and Enhanced Access Policy<ref name="DFIDResearch13" /> states that “DFID is committed to greater transparency in its activities and spending, and is working to make data more accessible to the public” and that “[the U.K. g]overnment is also committed to expanding access to publicly-funded research.” Similarly the World Bank policy on access to formal publications states that the bank “supports the free online communication and exchange of knowledge as the most effective way of ensuring that the fruits of research, economic and sector work, and development practice are made widely available, read, and built upon” and that it “is therefore committed to open access, which, for authors, enables the widest possible dissemination of their findings and, for readers, increases their ability to discover pertinent information.”<ref name="WBWorld12" />

In most cases, the focus of policy language aligns strongly with the "pragmatic" framing, and elements of this are also seen in the examples above. This is often also described as “the impact agenda”; the goal of maximizing value creation through funder investment in research. In the ''Concordat on Open Research Data''<ref name="RCUKDraft15">{{cite web |url=http://www.rcuk.ac.uk/research/opendata/ |title=Draft Concordat on Open Research Data |author=RCUK |date=17 July 2015 |accessdate=29 January 2016}}</ref>, a U.K. document developed by a multi-stakeholder group and endorsed by the U.K. Research Councils, the first motivation for data sharing is that “[t]he societal benefits from making research data open are potentially very significant; including economic growth, increased resource efficiency, securing public support for research funding and increasing public trust in research.”

The Office of Science and Technology Policy of the U.S. Executive Administration starts its memorandum ''Increasing Access to the Results of Federally Funded Scientific Research''<ref name="HoldrenIncreasing13">{{cite web |url=https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/ostp_public_access_memo_2013.pdf |format=PDF |title=Increasing Access to the Results of Federally Funded Scientific Research |author=Holdren, J.P. |publisher=Office of Science and Technology Policy |date=22 February 2013 |accessdate=29 January 2016}}</ref> with the statement that the “Administration is committed to ensuring that, to the greatest extent and with the fewest constraints possible and consistent with law and the objectives set out below, the direct results of federally funded scientific research are made available to ''and useful'' for the public, industry, and the scientific community” [emphasis added]. In the European Union the focus on “big data” as an economic good, in particular Vice President Kroes’ description of it as “the new oil,” is also an element of this "pragmatic" framing.

Funder policy statements less frequently involve language that invokes the "infrastructure" or "measurement" framing identified by Fecher and Friesike. This raises questions, as infrastructure and technology are clearly required to deliver on the goals of the "democratic" and "pragmatic" agendas, and measurement of policy compliance is a key tool for driving uptake, or at least monitoring adoption.

A useful analysis is therefore to consider how, for any given policy, the language reflects the discourse associated with each of the five sets of discourse, the extent to which the substance of the policy addresses each school, and whether a funder is tracking success against the goals of each framing. A well-balanced policy would require actions that align with the stated goals and measure success against those same goals. In practice, most policies invoke the core of the "democratic" and "pragmatic" discourse, require actions more aligned with the "infrastructure" discourse, and have weak reporting requirements. Where reporting requirements are stronger, they rarely provide data that would support assessment of progress towards the goals of the discourse, describing the "democratic" and "pragmatic" framing.

==References==
{{Reflist|colwidth=30em}}

==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. In some cases important information was missing from the references, and that information was added. The original article lists references alphabetically, but this version — by design — lists them in order of appearance.

<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2017)‎]]
[[Category:LIMSwiki journal articles (all)‎]]
[[Category:LIMSwiki journal articles on big data‎‎]]
[[Category:LIMSwiki journal articles on informatics‎‎]]