Previous Contents Next
Issues in Science and Technology Librarianship
Summer 2017
DOI:10.5062/F4QC01RP

[Refereed]

Historical Development and Key Issues of Data Management Plan Requirements for National Science Foundation Grants: A Review

Judith E. Pasek
STEM Liaison Librarian
University of Wyoming
Laramie, Wyoming
jpasek@uwyo.edu

Abstract

Sharing scientific research data has become increasingly important for knowledge advancement in today's networked, digital world. This article describes the evolution of access to United States government information in relation to scientific research funded by federal grants. It analyzes the data sharing policy of the National Science Foundation, which requires inclusion of a Data Management Plan in research proposals seeking agency funding. This policy is compared to a similar policy of the National Institutes of Health. Eight key issues limiting the success of the National Science Foundation policy are identified. These issues derive from instances of ambiguity, contradiction, inconsistency, lack of clarity, and gaps in guidance. Academic librarians can help fill the gaps in data sharing policy guidance by assisting researchers in the development of Data Management Plans and facilitating implementation of data curation practices.

Introduction

The United States government has a long history of promoting advancements in science and technology. Information access and sharing policies have been key factors in this effort. These information policies have evolved through time in concert with overarching scientific and political issues and developments, as well as with technological improvements affecting information dissemination. Many federal agencies award funding to scientific research institutions, including universities, to advance initiatives of importance to the American public. As such, federal science and information policies have a significant impact upon academic institutions that receive government funding, including associated research support services such as academic libraries.

One of the largest sources of science research funding in the U.S. government is the National Science Foundation (NSF). The NSF was established in 1950 as a U.S. federal agency "to promote the progress of science; [and] to advance the national health, prosperity, and welfare by supporting research and education in all fields of science and engineering" (NSF 2016b, 2016c, section A). The agency currently funds about 11,000 research and education proposals each year covering most disciplines within science and engineering. However, NSF typically does not fund areas targeted by other federal agencies such as defense research, human and animal medical research, and drug testing. NSF provides funding to approximately 2,000 research organizations, including about one-quarter of the federal support for basic research at academic institutions (NSF 2016b, 2016c).

The National Science Board develops, recommends, and promotes policies for NSF, considering the related laws and policies established by the U.S. Congress and the President of the United States (NSF 2016b, 2016c). Numerous federal policies set the framework for NSF policy development including those addressing intellectual property, publication and distribution, copyrights, privacy rights, treatment of human and animal subjects, non-discrimination, national security, and international relations.

The purpose of this paper is to review the policy environment related to sharing of scientific information. It centers on the NSF policy requirement to include a Data Management Plan (DMP) with each research proposal submitted on or after January 18, 2011. Policy analysis employs a descriptive approach that "relies on existing policy statements and related literature" (McClure et al. 1999, p. 328). Policy analysis techniques used herein include documentation of the historical context, examination of the key policy instrument, literature review (of policy discourse and related policies), side by side comparison (to a similar policy), and key issues identification. Eight key issues are identified based upon critical examination of the NSF policy guidance for Data Management Plans for instances of ambiguity, contradiction, inconsistency, lack of clarity, and gaps in guidance.

Historical Framework of Government Science Information Access

Public policy environments are dynamic, evolving to address changing needs of society. Examining the historical background associated with a policy establishes the context and political influences that led to the policy creation (McClure et al. 1999). Identifying preceding and associated policies of the policy of primary interest, in this case the NSF DMP requirement, also promotes understanding. Information policies that apply to all federal agencies establish the foundation upon which more specific science information policies rest. Information policies of federal science agencies in turn influence activities of outside entities through program delivery and associated requirements. This historical review begins with broad government-wide information policies. It then traces the development of federal science programs especially relating to the NSF and associated science information policies. The U.S. laws and policies, events, and influencing reports covered herein that shape information access and sharing of federally funded science research have evolved from the mid-1940s following World War II through the end of President Obama's administration, which concluded in January 2017 (Table 1).

Table 1. Timeline of U.S. laws and policies, events, and key reports relating to science information access and sharing

Year Category Focus1 Description
1945 Report S "Science--The Endless Frontier" (Vannevar Bush to President Roosevelt)
1950 Event S NSF founded
Early 1960s Event S Committee on Scientific and Technical information (COSATI) placed in Executive Office of the President
1962 Law G Depository Library Act
1963 Report S "Science, Government, and Information" (Weinberg Report; President's Science Advisory Committee)
1971 Event S COSATI moved to NSF
1972 Event S COSATI, Science Advisor position, and Office of Science and Technology abolished (President Nixon)
1980 Law G Paperwork Reduction Act (later amended)
1985 Policy G OMB Circular A-130: "Management of Federal Information Resources" (since revised; replaced in 2016 as "Managing Information as a Strategic Resource")
1990 Report S "Helping America Compete" (OTA)
1993 Law G Government Performance Results Act
1993 Law G Government Printing Office Electronic Information Access Enhancement Act
1997 Report S "Bits of Power: (NRC Committee on Issues in the Transborder Flow of Scientific Data)
1999 Policy G OMB Circular A-110 revised and amended; definition of "research data" added
2002 Law G E-Government Act
2003 Policy S "NIH Data Sharing Policy and Implementation Guidance" established
2005 Policy G OMB Memorandum M-06-02 (on making government information searchable)
2005 Report S "Long-Lived Digital Data Collections" (National Science Board)
2005 Policy S NIH Public Access Policy issued
2007 Report S "Rising Above the Gathering Storm" (CPGE)
2007 Law S America COMPETES Act
2008 Law S Consolidated Appropriations Act mandated the NIH Public Access Policy
2009 Policy G Open Government Initiative (President Obama)
2009 Policy G Open Government Directive (OMB)
2009 Report S "Harnessing the Power of Digital Data for Science and Society" (IWGDD)
2010 Law G GPRA Modernization Act
2010 Law S America COMPETES Reauthorization Act
2011 Policy S NSF Data Management Plan requirement added (NSF grants guidance)
2012 Policy G "Building a 21st Century Digital Government" memorandum (President Obama)
2013 Policy G Executive Order: "Making Open and Machine Readable the New Default for Government Information" (President Obama)
2013 Policy G OMB Memorandum M-13-13: Open Data Policy
2013 Event G Data.gov launched
2013 Policy S Executive Order: "Increasing Access to the Results of Federally Funded Scientific Research" [Open Access Policy] (OSTP)
2016 Policy S NSF post-award publication requirement added
2016 Policy G OMB Circular A-130 substantially revised as "Managing Information as a Strategic Resource"
1 G = U.S. Government-wide focus; S = Science focus

Government-wide Information Sharing Policies

The United States government generally promotes public accessibility of non-classified information in the interest of developing an informed citizenry and supporting democratic processes. The Federal Depository Library Program, authorized by the Depository Library Act of 1962 (P.L. 85-579) established a means to provide free access to government documents (Jaeger et al. 2010).

The 1980s saw increasing focus on development of information policies, with the production of numerous Office of Management and Budget (OMB) circulars (Smith 1998). A government circular is a widely-distributed written policy statement that typically interprets law and provides guidance on implementation of procedures or rules. In producing Circular A-130 (since revised), "Management of Federal Information Resources," OMB used its authority under the Paperwork Reduction Act of 1980 (44 U.S.C. §§ 3501-3521), later amended, to place emphasis on cost-savings, privatization of information dissemination, value-added restrictions for government agencies, and collection of user fees for dissemination, with little regard for needs of the scientific research community (OTA 1990).

The Government Performance Results Act of 1993 (GPRA) (P.L. 103-62) ushered in an era of greater government accountability, and transparency requirements were added with the GPRA Modernization Act of 2010 (P.L. 111-352) (OMB n.d.a). In the mid-1990s, many federal agencies began making government documents accessible on their web sites following passage of the Government Printing Office Electronic Information Access Enhancement Act of 1993 (P.L. 103-40) (Jaeger et al. 2010). Internet access to government information was further promoted by the E-Government Act of 2002 (P.L. 107-347). OMB memorandum M-06-02 issued in 2005 specified that government information should not only be published on the Internet, but that it should be "searchable across agencies" via search functions or by use of formal information models, such as taxonomies, ontologies, controlled vocabularies, or metadata schemas (Johnson 2005, pp. 1-2). President Obama's Open Government Initiative emphasized that transparency of information, participation, and collaboration are essential to accountability, public involvement, and effectiveness of government programs (Obama 2009). The Open Government Directive, issued by OMB on December 8, 2009, established requirements for government agencies to "publish government information online [in open formats]...improve the quality of government information...create and institutionalize a culture of open government...[and] create an enabling policy framework for open government" (Orszag 2009, pp. 2-5). In 2012, President Obama issued a memorandum entitled "Building a 21st Century Digital Government" that encouraged federal agencies to use mobile and web-based technologies to improve digital services including information delivery.

A related Executive Order (Obama 2013) directed OMB to issue an Open Data Policy that would "whenever possible and legally permissible,...ensure that [government] data are released to the public in ways that make the data easy to find, accessible, and usable ("Section 1"). The Executive Order set a default for new government information to be managed throughout the life cycle to be open, machine readable, and interoperable, while safeguarding "individual privacy, confidentiality, and national security" ("Section 1"). The Open Data Policy (Burwell et al. 2013) stated an underlying purpose to "help fuel entrepreneurship, innovation, and scientific discovery -- all of which improve American's lives and contribute significantly to job creation" (p. 1). The online platform, Data.gov, was launched to improve access to government data sets including "datasets that were produced through agency-funded grants, contracts, and cooperative agreements" (Burwell et al. 2013, p. 8). In 2016, the principles of open data were incorporated into a substantially revised OMB Circular A-130, which provides guidance to federal agencies on "Managing Information as a Strategic Resource."

Science as a Government Concern

In addition to government-wide information sharing policies, advancements in science and technology programs and economic and political factors influenced the evolution of science information policies. The idea that "science was a proper concern of government" and "that scientific progress was essential for the good of the country" (Smith 1998, para. 5) arose out of political concerns. Transformative events included World War II, the launch of the Sputnik satellite into space by the former Union of Soviet Socialist Republics (USSR), and a report by Vannevar Bush (1945) to President Roosevelt entitled "Science--The Endless Frontier." Among the recommendations, Bush advocated for public funding of basic research by scientific research organizations including academic institutions, and proposed that a new agency be established for that purpose. The NSF was created in 1950 as a result.

In 1958, Senator Hubert Humphrey pushed for greater access to scientific information (Smith 1998). Advisory committees were established, which led to the 1963 Weinberg Report, entitled "Science, Government, and Information," which declared that the "transfer of information is an inseparable part of research and development" (President's Science Advisory Committee 1963, p. 1). A Committee on Scientific and Technical Information (COSATI) became a focal point for coordination of scientific programs with its placement within the Executive Office of the President in the 1960s (OTA 1990; Smith 1998). However, it lost prominence when it was moved to NSF in 1971 and then abolished in 1972, a time when President Nixon also abolished his Science Advisor position and the Office of Science and Technology (Smith 1998).

During the 1980s, science programs entered an era of protectionism as concerns about foreign competition, especially with Japan, mounted and efforts to restrict the export of U.S. high technology equipment and expertise ensued (Smith 1998). By the late 1980s, recognition of the need for more technology research, especially to develop better electronic communication networks increased. The U.S. Congress, Office of Technology Assessment (OTA 1990) published a report "Helping America Compete," which emphasized the necessity of greater investment in science and technology initiatives. The "Gathering Storm" report (CPGE 2007), resulting from a 2005 National Academies study of U.S. competitiveness in relation to national investments in science and technology, outlined 20 recommendations for improvement, including greater investments in science and technology education and research.

Open Science Policy Development

These concerns about competitiveness in science and technology beginning in the 1990s renewed interest in information sharing, which developed into policies of the early 21st century. Concomitantly, efforts to increase government transparency extended information policies to include federally funded scientific research conducted by non-federal organizations. In 1997, the National Research Council (NRC), Committee on Issues in the Transborder Flow of Scientific Data, published a report, Bits of Power, which made recommendations regarding global access to natural science research data. The committee advocated for "full and open exchange of scientific data," by which it meant "that the data and information derived from publically funded research [be] made available with as few restrictions as possible, on a nondiscriminatory basis, for no more than the cost of reproduction and distribution" (NRC 1997, p. 10).

A similarly worded data sharing policy has been a part of the NSF grant proposal guidance for many years (NSF-CISE 2015) and includes a statement that, "Investigators are expected to share with other researchers, at no more than incremental cost and within reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants" (NSF 2016a, Chapter VI.D.4.b; NSF 2016c, Chapter XI.D.4.b).

The America COMPETES Act (America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act) (P.L. 110-69) was enacted in 2007, and extended as the America COMPETES Reauthorization Act of 2010 (P.L. 111-358) (Holdren 2011). The Office of Science and Technology Policy (OSTP), under authorization of the America COMPETES Reauthorization Act of 2010 coordinated efforts to develop policies and approaches for long-term management and dissemination of unclassified federally funded research results (ACRL 2012).

In 2009, the Interagency Working Group on Digital Data (IWGDD) issued a report entitled "Harnessing the Power of Digital Data for Science and Society" that addressed access and preservation of federally-funded digital data derived from science and technology research and development. Among the recommendations and goals were the ideas of "promot[ing] a data management planning process," "maximizing digital data access and utility," and "enabl[ing] communities of practice" (IWGDD 2009, pp. 15-16, 20). This echoed recommendation 4 of a 2005 report of the National Science Board on "Long-Lived Digital Data Collections," from which the NSF Data Management Plan (DMP) requirement evolved (NSF-GEO n.d.). An NFS policy change, effective for all grant proposals submitted on or after January 18, 2011, added the requirement to include a Data Management Plan as a supplemental document of no more than two pages (NSF 2016b, 2016c, Chapter II.C.2.j; NSF n.d.b).

Efforts to encode research information sharing policies into law in alignment with the Obama Administration's Open Government Initiative met with limited success. Passage of the Consolidated Appropriations Act of 2008 (P.L. 110-161) made the National Institutes of Health (NIH) Public Access Policy mandatory, which stipulates that published results of NIH-funded research must be deposited in the freely accessible PubMed Central digital repository within 12 months of publication (ACRL 2012). An attempt to roll back the NIH policy in the form of the Research Works Act (H.R. 3699) was withdrawn in early 2012 by sponsoring legislators when a major publisher withdrew its support (ACRL 2012).

A similar bill to the NIH policy, entitled the Federal Research Public Access Act (FRPAA) was introduced to U.S. Congress in 2006, and 2009, and again in 2012 (as S. 2096 and H.R. 4004) (ACRL 2012; ALA c1996-2016; ARL n.d.). The purpose was to require deposit of peer-reviewed journal articles, in free and stable archives or repositories within six months of publication, that result from federally funded research supported by any of 11 agencies with annual research budgets of $100 million or more. The unenacted FRPAA was replaced by the Fair Access to Science and Technology Research Act (FASTR), introduced in 2013 and 2015 (as S. 779 and H.R. 1477) (ALA c1996-2016). Although not enacted, a petition regarding the 2013 FASTR bills prompted an Executive Order issued by OSTP for agencies to develop plans to make the results of scientific research freely available within one year of publication and to better manage digital data (Holdren 2013; Stebbins 2013; ALA c1996-2016). The Committee on Homeland Security and Governmental Affairs of the United States Senate issued a report in March 2016 in support of the FASTR bill as amended, which "would codify and expand current policies and practices of federal agencies" (S. Rep. No. 114-224 2016).

Meanwhile, the NSF added a post-award requirement that copyrighted material published in peer-reviewed journal articles and juried conference papers be deposited and made publicly accessible in a compliant repository within 12 months of publication, along with metadata and a persistent identifier (NSF 2016a, 2016c). This requirement became effective for funded proposals that were submitted on or after January 25, 2016. Additionally, regulations as specified in OMB Circular A-110 as revised and amended (Uniform Administration Requirements for Grants and Agreements with Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations) allow for access, use, and publication by federal agencies of research data produced under an award, and public disclosure of research data under the Freedom of Information Act (FOIA) [(5 U.S.C. § 552)] when published research results were used in developing federal actions with the force of law (OMB n.d.b).

NSF Data Management Plan Requirement Analysis

The NSF grant proposal guidance (Chapter II.C.2.j) describes, in general terms, what may be included in a DMP (NSF 2016b, 2016c). The list includes:

  1. the types of data, samples, physical collections, software, curriculum materials, and other materials to be produced in the course of the project;
  2. the standards to be used for data and metadata format and content...;
  3. policies for access and sharing including provisions for appropriate protection of privacy, confidentiality, security, intellectual property, and other rights or requirements;
  4. policies and provisions for re-use, re-distribution, and the production of derivative; and
  5. plans for archiving data, samples, and other research products, and for preservation of access to them. (NSF 2016b, 2016c, Chapter II.C.2.j)

The NSF policy references further guidance specific to directorates and divisions, corresponding to disciplinary areas (Table 2). A few units have recently revised and improved their guidance relative to what was provided in early 2011.

Table 2. Web sites of NSF Directorates/Divisions policy guidance regarding Data Management Plans1

NSF Unit Publication Date Web Site
[Directorate for Social, Behavioral & Economic Sciences] (SBE) 2010 Oct 12 https://www.nsf.gov/sbe/SBE_DataMgmtPlanPolicy.pdf
[Directorate for Education & Human Resources] (EHR) 2011 Mar https://www.nsf.gov/bfa/dias/policy/dmpdocs/ehr.pdf
[Directorate for Engineering] (ENG) n.d. https://www.nsf.gov/eng/general/ENG_DMP_Policy.pdf
NSF n.d.b https://www.nsf.gov/bfa/dias/policy/dmp.jsp
Division of Atmospheric and Geospace Sciences (AGS) n.d. https://www.nsf.gov/geo/geo-data-policies/ags/index.jsp
Directorate for Mathematical and Physical Sciences, Division of Astronomical Sciences (AST) n.d. https://www.nsf.gov/bfa/dias/policy/dmpdocs/ast.pdf
Directorate for Biological Sciences (BIO) updated 2015 Oct 1 https://www.nsf.gov/bio/biodmp.jsp
Directorate for Mathematical and Physical Sciences, Division of Chemistry (CHE) n.d. https://www.nsf.gov/bfa/dias/policy/dmpdocs/che.pdf
Directorate for Computer & Information Science & Engineering (CISE) 2015 Mar 15 https://www.nsf.gov/cise/cise_dmp.jsp
Directorate for Mathematical and Physical Sciences, Division of Materials Research (DMR) n.d. https://www.nsf.gov/bfa/dias/policy/dmpdocs/dmr.pdf
Directorate for Mathematical and Physical Sciences, Division of Mathematical Sciences (DMS) n.d. https://www.nsf.gov/bfa/dias/policy/dmpdocs/dms.pdf
Division of Earth Sciences (EAR) 2010 Sep https://www.nsf.gov/geo/ear/2010EAR_data_policy_9_28_10.pdf
Directorate for Geosciences (GEO) n.d. https://www.nsf.gov/geo/geo-data-policies/index.jsp
Division of Ocean Sciences (OCE) posted 2016 Dec 15 https://www.nsf.gov/pubs/2017/nsf17037/nsf17037.jsp
Directorate for Mathematical and Physical Sciences, Division of Physics (PHY) n.d. https://www.nsf.gov/bfa/dias/policy/dmpdocs/phy.pdf
Division of Polar Programs (PLR) 2016 https://www.nsf.gov/pubs/2016/nsf16055/nsf16055.jsp?WT.mc_id=USNSF_25&WT.mc_ev=click
1 See References section for full citations.

A closer review of the NSF policy requirement that a DMP be included in research funding proposals (effective 2011 Jan 18) is in order, especially considering its effect on expansion of research support services. "A detailed examination of the key [policy] instrument focuses on identifying and describing its goals, objectives, explicating definitions, determining applicability, and specifying implementation requirements" (McClure et al. 1999, p. 317). Policy analysis often includes investigating questions of clarity, consistency, ambiguity, contradiction, gaps, and other factors that may present challenges to policy implementation (McClure & Jaeger 2008). This analysis of the NSF policy focuses on four main concepts: policy goals, data definitions, flexibility, and data access and sharing concerns.

Policy Goals

While the NSF DMP policy itself does not provide any goal or purpose statements, a press release dated May 10, 2010, includes quotes from several agency managers that provide justifications for the added requirement:

The changes are designed to address trends and needs in the modern era of data-driven science...Researchers from numerous disciplines need to work together to attack complex problems; openly sharing data will pave the way for researchers to communicate and collaborate more effectively...It will address the need for data from publicly-funded research to be made public...The change reflects a move to the Digital Age, where scientific breakthroughs will be powered by advanced computing techniques that help researchers explore and mine datasets...It is imperative that data be made not only as widely available as possible but also accessible to the broad scientific communities...[and] requiring the data management plans [is] consistent with NSF's mission and the growing interest from U.S. policymakers in making sure that any data obtained with federal funds be accessible to the general public (NSF 2010a).

Supplemental guidance for the NSF Division of Social and Economic Sciences provides additional purposes of contributing "to improved training for graduate and undergraduate students, and [making] possible significant economies of scale through the secondary analysis of extant data" (NSF-SBE n.d.). DMP guidance for the NSF Engineering Directorate identifies as purposes "to stimulate new advances as quickly as possible and to allow prompt evaluation of the results by the scientific community" (NSF n.d.a). The NSF DMP policy evidently has multiple goals of increasing scientific collaboration and innovation, sharing and reuse of data among researchers, facilitating peer review, training scientists, informing the public, accountability to the public, and transparency of government operations. That is a broad charge representing potentially conflicting interests. The DMP press release mentioned the Open Government Directive of the Obama Administration, but did not specify any related or authorizing policies or legislation.

Data Definitions and Ambiguities

Ambiguities and inconsistencies of the NSF DMP policy are most evident in supplemental guidance offered by various Directorates and Divisions. The unit guidance documents largely repeat the general NSF policy data sharing statements; however, definitions of data and what is excluded (or included) varies somewhat.

Identifying what may be encompassed by the term data is problematic for interpreting policy. Borgman (2012) noted that "NSF sidesteps the definition of data" (p. 1061). Part of the difficulty, as recognized by NSF, is that data can take many forms and can vary widely depending upon discipline and line of inquiry. "Data may exist only in the eye of the beholder: The recognition that an observation, artifact, or record constitutes data is itself a scholarly act" (Borgman 2012, p. 1061). Furthermore, data can vary depending upon the source, with "physical and life sciences...[depending mostly] on observations, experiments, or models...[while researchers] in the social sciences...may gather or produce their own data, or they may obtain data from other sources such as public records" (Borgman 2012, p. 1061). Borgman further reviewed how data can be described by categories, or purposes, or the means by which it is handled. Categories can include observational, computational, experimental, and records data, with the appropriate choice dependent upon the purpose of research study and the methods or approaches used. Three dimensions can define purpose: (1) Specificity, ranging from exploratory investigation to observatories capturing repetitive data sets; (2) Scope, ranging from studies of single phenomena to modeling entire systems; and (3) Goal, from empirical to theoretical. Approaches to handling data will vary by the number of people involved, the amount of time and labor needed, and the amount and type of processing needed for interpretation. "Generally, the more handcrafted the data collection and the more labor-intensive the postprocessing for interpretation, the less likely that researchers will share their data" (Borgman 2012, p. 1066).

A federal definition of research data was added to the OMB A-110 Circular during the 1999 amendment. This amendment extended public access to include federally funded research data requested under FOIA when the requested data was used in developing regulations of the funding agency. The research data definition was developed in response to commenters on the proposed rule change who wanted more clarification on several concepts, including what was meant by data in the context of the policy. Some commenters were concerned about difficulty in recruiting participation for research studies if personal privacy and proprietary information could not be assured of remaining confidential. The final definition, incorporated into the policy, excludes private and proprietary information and intellectual property prior to publication (which includes citation in regulations). OMB indicated that their intent in writing the new section was not to require researchers to reveal their research data publicly while studies were still ongoing.

OMB defines research data as:

the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues. This 'recorded' material excludes physical objects (e.g., laboratory samples). Research data also do not include:
  1. Trade secrets, commercial information, materials necessary to be held confidential by a researcher until they are published, or similar information which is protected under law; and
  2. Personnel and medical information and similar information the disclosure of which would constitute a clearly unwarranted invasion of privacy, such as information that could be used to identify a particular person in a research study. (OMB n.d.b, section __.36 on intangible property)

The exclusion of physical objects in the OMB definition (which applies across all federal agencies) seems to contradict the inclusion of samples and physical collections within the primary NSF DMP policy direction. This could lead to inconsistent policy interpretations for some disciplines. The OMB definition is reiterated (in full or in part) by the data sharing policy guidance for the NSF Directorate for Education & Human Resources and the NSF Directorate for Social, Behavioral and Economic Sciences (NSF 2010b, 2011; OMB n.d.b).

The NSF Directorate for Computer & Information Science & Engineering guidance also cites the OMB definition but clarifies further that "this definition includes not only original data but also 'metadata' (e.g., experimental protocols, software code written for statistical or experimental analyses or for proofs-of-concept, etc.)" (NSF-CISE 2015; "Definition and Policy"). The OMB definition does appear to include raw data, such as measurements, other than those protected from disclosure by laws, but excludes processed or interpreted data other than that contained within published works. The OSTP public access memo (Holdren 2013) that provides direction to large research agencies also cited the OMB research data definition, adding an exclusion for digital laboratory notebooks.

In contrast, the NSF Directorate for Engineering guidance quotes the first part of the OMB definition, and provides some differing guidance regarding exclusions (NSF n.d.a). Raw data are specifically excluded from the interpretation of the OMB definition as being included within "preliminary analyses." The NSF Directorate for Engineering guidance includes analyzed data, defining it as "(but...not restricted to) digital information that would be published, including digital images, published tables, and tables of the numbers used for making published graphs" (NSF n.d.a, "Data to be Managed"). In other words, the research results would not have to be published before being defined as data, but there would be an expectation of publication. This appears to be contrary to the OMB exclusions for "preliminary analyses [and] drafts of scientific papers" (OMB n.d.b, section __.36 on intangible property). When government-wide policy definitions and specific agency policy definitions are inconsistent or contradictory it can be difficult for a researcher to determine which takes precedence in any given instance, and whether they are in compliance.

The data management policies for the remaining directorates are silent on issues of data definition; however, two divisions within the NSF Directorate of Mathematical and Physical Sciences provide examples of types of data typical of their disciplinary areas. The NSF Division of Astronomical Sciences lists "images of astronomical objects, spectra, data tables, time series, theoretical formalisms, computational strategies, software, and curriculum materials" (NSF-AST n.d., "1. Products of the Research"). The NSF Division of Chemistry identifies "numerical data on chemical systems such as spectra, diffraction patterns, physical properties, time-dependent information on chemical and physical processes, theoretical formalisms, computational strategies, final and intermediate numerical results from theoretical calculations, software, and curriculum materials" (NSF-CHE n.d., "1. Products of the Research"). The NSF Division of Social and Economic Sciences within the Directorate of Social, Behavioral & Economic Sciences states that "the kinds of qualitative information collected...can range from microfilms and other copies of very old documents to oral interviews and video tapes about historical events in science or about contemporary technological controversies" (NSF-SBE n.d., "Qualitative Information"). This additional, yet non-contradictory, information about what constitutes data for particular disciplines can help clarify for researchers what may need to be addressed when drafting a DMP.

Rather than identifying types of data, the OMB Open Data Policy memorandum broadly defined data as "all structured information" and described open data as "publically available data structured in a way that enables the data to be fully discoverable and usable by end users" (Burwell et al. 2013, pp. 4-5). Open data was further identified with principles of being public, accessible, described, reusable, complete, timely, and managed post-release. This government-wide direction reinforces the intent of the NSF DMP policy, specifically in relation to making digital information publicly accessible.

Flexibility and Peer Review

The apparent intent of establishing separate policy guidance for each NSF directorate or division was to "acknowledge that each discipline has its own culture about data-sharing, and...to avoid a one-size-fits-all approach...But for all disciplines, the data management plans [are] subject to peer review, and the new approach [allows] flexibility at the directorate and division levels to tailor implementation as appropriate" (NSF 2010a). Indeed, the responses provided to most questions about the DMP policy within a Q&A document state that answers "will be determined by the community of interest through the process of peer review and program management" (NSF 2010c). The term community of interest, which is used to describe how DMPs will be evaluated by peer review, is associated with digital archiving practice (Borgman 2012). However, communities of interest are neither clearly defined nor static, may overlap multiple areas of study, and may have differing concepts of what constitutes data and data practices. "It is left to the investigator--or to the data archive--to designate the appropriate community of interest" (Borgman 2012, p. 1061).

The standard NSF (2010c) response regarding determination by a community of interest was provided for questions about:

NSF has placed the burden of determining what specifically should be covered in a DMP onto the researchers who are seeking funding, rather than providing more detailed guidance. Academic librarians are attempting to fill this guidance gap by providing consultation services and templates for writing DMPs to improve competitiveness for funding approval. NSF does recognize that researchers will need to address any institutional strategies for data access and retention upon their departure from employment, as well as any special conditions that apply to collaborative international projects (NSF 2010c). These aspects are likely to require availability of additional institutional support services, such as consultation services for data documentation and metadata standards, and assistance in transfer, storage, and access to research data sets.

Data Access and Sharing

Guidance for the NSF Division of Astronomical Sciences makes a distinction between access to data and data sharing. "'Access to data' refers to data made accessible without explicit request from the interested party, for example those posted on a website or made available to a public database" (NSF-AST n.d., "3. Access to Data"). "'Data sharing' refers to the release of data in response to a specific request from an interested party" (NSF-AST n.d., "3. Access to Data"). By these definitions, librarians are more likely to assist researchers with providing access to data rather than with data sharing. However, NSF guidance for other directorates and divisions do not make this distinction, and appear to apply the term data sharing to both instances.

In addition to describing plans for data access and sharing, astronomers are encouraged to describe within their proposals any disclaimers or conditions for use, re-use, re-distribution, or production of derivatives (NSF-AST n.d.). Although NSF data sharing policies provide minimal guidance on protecting data confidentiality, a report by the Council on Governmental Relations (CGR) provides summaries of restrictions placed on research data access and retention in relation to laws governing privacy, intellectual property, export controls and other concerns (CGR 2006).

Many uncertainties and issues are associated with sharing research data, such as its potential for future use or reuse, accessibility of formats, compatibility of data types, completeness, availability of documentation of collection and processing procedures, metadata standards, potential for misuse or misinterpretation, scientific competitiveness and cultures, intellectual property considerations, and ethical and legal concerns (Tenopir et al. 2011, 2015; Borgman 2012). Anderson (2004) provided a comprehensive review of numerous challenges to implementing access, archiving, and preservation of digital scientific data, and organized them into four categories: science-based issues, data management issues, policy issues, and technical issues.

Correspondingly, there are many disincentives for researchers to sharing their data. In a survey of researchers' attitudes about sharing data, conducted in 2009-2010 prior to the NSF DMP requirement, nearly half of respondents indicated that their organization did not provide a formal process for storing data beyond the life of a project, and 59% indicated there was inadequate funding for that purpose (Tenopir et al. 2011). Insufficient time was the primary reason given for not sharing research data (54%) followed by lack of funding (40%). However, respondents to a follow-up survey conducted in 2013-2014, (i.e., after the NSF DMP requirement was instituted), were less concerned about funding support and time needed (Tenopir et al. 2015). The primary barrier shifted to concern about being able to publish before sharing data. In the span of a few years, researchers became more accepting of the notion that sharing data is important to scientific progress, and in general became more willing to share and reuse research data. However, impediments to greater sharing of research data continue to include reluctance on the part of researchers as well as lack of incentives. Factors contributing to reluctance by researchers to sharing of their data include "perceived ownership of data (reflected in the right to publish first) and a need for control (reflected in the fear of data misuse)" as well as "lack of sufficient formal recognition (e.g., citations, co-authorship)" (Fecher et al. 2015, p. 19).

A researcher's motivations and willingness to share research data will influence how their DMPs are developed and what communities of interest they might identify. The researcher's perspectives will also be affected by how they conceive their intended goals for sharing data. Four rationales for sharing data, as described and modeled by Borgman (2012), "are to: (1) reproduce or to verify research, (2) make results of publicly funded research available to the public, (3) enable others to ask new questions of extant data, and (4) advance the state of research and innovation" (p. 1067). These four rationales can be arrayed on a public to private (researcher) continuum, as well as on a beneficiaries dimension from producers to users. Motivations of various interest groups and individuals for sharing or accessing data do not necessarily coincide, as each may place differing relative values on each rationale. Conflicts can even arise within an individual, such a researcher who wants to be able to access data of other researchers, while simultaneously trying to restrict access to their own data. Researchers who also have substantial teaching responsibilities are less likely to share their data compared to those dedicated solely to research activities (Tenopir et al. 2011). Age and subject discipline also contribute to differing levels of data sharing practice (Tenopir et al. 2015).

Comparison of NSF and NIH Data Management Policies

Given the similarities of scientific research support to academic institutions and the purposes of data sharing policies for NSF and NIH, a side-by-side comparison of the policy features may be instructive. Table 3 summarizes key aspects of the contents of the NSF and NIH data sharing and publication policies (NIH 2003, 2004, 2007, 2016; NSF 2016a, 2016c; NSF n.d.b). In response to the 2013 OSTP memorandum, NIH also developed a plan to establish policies and approaches to enhance sharing of research data including: the preparation and peer review of data management plans; specific guidance on data management, repositories, and data standards; and infrastructure support to facilitate discoverability (NIH 2015).

Table 3. Side-by-side comparison of NSF and NIH data sharing policies

Policy NSF-"Dissemination and Sharing of Research Results" and guidance of NSF units; "Public Access to Copyrighted Material" "NIH Data Sharing Policy and Implementation Guidance" and related documents; "Public Access Policy;" "Human Data Sharing" policy
Web Site https://www.nsf.gov/bfa/dias/policy/dmp.jsp https://grants.nih.gov/policy/sharing.htm
Effective Date January 18, 2011 (except where otherwise noted) October 1, 2003 (except where otherwise noted)
Types of Research Research and education in all fields of science and engineering (except areas funded by other agencies). "Basic research, clinical studies, surveys, and other types of research supported by NIH."
Applicability of DMP All research funding proposals. Research funding applications "requesting $500,000 or more of direct costs in any single year to NIH." "Intramural research involving human data undergoing scientific review after October 1, 2015."
Goals No goals stated within policy; however, many goals aimed at both researcher and public access are presented in related documents and reports. Many goals stated relating to advancement of research, although some mention of public access is also included. Emphasizes preservation of "unique data."
DMP Requirement Mandatory 2-page supplemental documentation or a statement that no data is to be collected. A brief paragraph immediately following the Research Plan Section. References to data-sharing plans may also be added to budget and budget justification, background and significance, or human subjects sections, as appropriate.
DMP Instructions Broad statements of types of information to include. Little actual guidance provided as to approaches that should be considered. Extensive discussion of human subjects, privacy issues, and anonymizing data; proprietary data concerns; statements about inappropriate limitations for use or re-use; methods of data sharing; data-sharing agreements; and indication of planned workshops. Examples of DMPs and types of data sets provided.
Additional DMP Guidance Brief and general direction provided in guidance documents of directorates and divisions. Minimal guidance provided in Q&As. Several reference sources provided on data management and sharing. Q&As provide helpful, additional guidance.
Review Process Proposal peer reviewers and NSF program managers to determine appropriateness of DMP in relation to communities of interest. Proposal peer reviewers are not to consider DMP in evaluations. Appropriateness of DMP to be decided by NIH program managers.
Timeliness "Reasonable time" not defined. Some NSF units provide data submission timelines for specific programs. Expected "not later than the acceptance for publication of the main findings from the final dataset." "Data sharing prior to the publication of major results is encouraged in many instances." For "large epidemiological and longitudinal studies...the data would be released in waves as data become available." "Intellectual property rights may justify...a delay of 30 to 60 days." "Small Business Innovation Research...grantees may withhold their data for 4 years after the end of the award."
Data Definitions Definitions not provided in policy; however, some directorate guidance documents reference the definition of "research data" contained in OMB Circular A-110. "Final research data" to exclude summary statistics or tables, but to include underlying data. Excludes pathology reports, clinical source documents, laboratory notebooks, partial data sets, preliminary analyses, and physical objects--including audio and video tapes and laboratory specimens.
Funding for Data Sharing May be included in proposal. May be included in proposal. May request no-cost extension if providing access to final data set involves considerable delays.
Reports to Funding Agency Not specified in data-sharing policy, but related directions indicate that implementing the DMP is a condition of the award. DMP progress to be included in reports. May make DMP a condition of award. DMP is expected to be enacted, and progress noted in reports. Data sharing is required for all NIH grant awards even when a DMP is not required in a proposal.
Enforcement Not specified in data-sharing policy, but generally covered by other funding guidance. DMP becomes part of award contract. "NIH can take various actions to protect the Federal Government's interests...NIH may make data sharing an explicit term and condition of subsequent awards."
Open Access Publishing Deposit of copyrighted peer-reviewed journal articles and juried conference papers in a publically accessible and compliant repository within 12 months of publication along with metadata and a persistent identifier. Effective for funded proposals submitted on or after January 25, 2016. NIH Public Access Policy stipulates mandatory deposit of final peer-reviewed manuscripts or published results, within 12 months of publication, in the PubMed Central digital repository--a free full-text archive maintained by NIH's National Library of Medicine. Policy initially issued in 2005; made a legal requirement as of April 7, 2008.

Borgman (2012) observed that the NSF DMP requirement is more comprehensive than that of the NIH because it applies to all proposals, not just the largest ones, and it relies upon peer review rather than solely on negotiation with a program manager. However, the brevity of the NSF data sharing policy statements, and the types of questions found in the NSF DMP Q&As, indicate vagueness and ambiguity. This can make it difficult for grant applicants to know whether they are compliant with the DMP requirement or not.

Differences in the approaches of NSF and NIH in addressing criteria relating to data sharing likely reflect the environments under which the policies were developed. The stated goals of the NIH policy emphasize sharing between researchers to advance science. Public access appears to be a secondary result rather than a primary driver of the policy. The NIH policy developed during a time of advancing digital technologies and networking capabilities that allowed not only for greater sharing and accessibility of information, but also facilitated analyses of "big data." Combining of data into large sets in this new era is particularly important for medical disciplines that emphasize reproducibility and validation, and often use techniques of meta-analysis and systematic literature reviews. A centralized and open access repository (i.e., PubMed Central) for agency-funded and disciplinary literature facilitates these research methodologies. Furthermore, detailed guidance regarding what information should be included in DMPs (including examples), how to treat human subjects data in terms of privacy concerns, the timeliness of sharing data, and acceptable data repositories (and related resources) directly supports the needs of collaborating health scientists, especially those engaged in large studies.

Interestingly, the Open Government and Open Data initiatives of the Obama administration apparently resulted in a push by NSF for data management planning (to enforce data sharing and archiving) prior to establishment of a policy of open access publishing, just the opposite of the sequence followed by NIH. In contrast to NIH, the NSF DMP policy developed during a time of emphasis on greater government transparency and accountability. Policy development in this case may have been driven more by political needs than by specific needs of scientific research. The general public is not often the primary audience for research outputs even when the underlying purpose of study may be to improve society. The varied array and sometimes conflicting purposes of the NSF DMP policy as stated by various NSF managers likely reflects attempts to justify a government mandate for public access in terms that science researchers might accept. This apparent conflict of purposes may at least partially explain why goals were not stated within the NSF DMP policy itself, and also the reluctance of many researchers to change their data management practices. Where data sharing has been most prevalent has been in disciplines, such as physics and astronomy, that rely upon automated measurements or observatories to amass large data sets that can be reused by multiple researchers. These sharing practices evolved to meet needs of researchers rather than being driven by public policy. However, the NSF DMP policy attempts to bring that culture to all science disciplines and at all scales of research ranging from "small science" to "big data." The weaknesses of the NSF DMP policy derive primarily from limited guidance and lack of examples and stated resources to help science researchers comply with the intent of the policy.

Key Issues

"An important component of any descriptive assessment of information policy initiatives is to identify and describe key issues that summarize the current status and areas of contention for a particular initiative" (McClure et al. 1999, p. 315). Key issues of the NSF DMP policy were identified through the preceding critical review of policy documents, supplemented by related literature critiques, to look for instances of ambiguity, contradiction, inconsistency, lack of clarity, and gaps in guidance.

Weaknesses in the NSF data sharing policy and DMP requirement include a general lack of stated policy goals and definitions for data and other terms, minimal guidance and examples of DMPs, some inconsistencies in DMP guidance between NSF units, and potential gaps in funding support for long-term preservation. Policy short-comings are identified as issues in the following paragraphs and compared to other studies.

Issue 1: Lack of stated NSF DMP goals and purposes. A lack of stated goals and purposes within the NSF data sharing policy leads to ambiguity, as various people may interpret the intent of the policy differently. Indeed, the policy appears to be derived from a diversity of goals, reflected in sometimes conflicting statements by NSF managers. Consequently, different data sharing practices may be needed depending upon whether data are to be shared within limited scientific communities or more broadly to the public. "Making data available to the users beyond one's specialty requires much more documentation effort" (Borgman 2012, p. 1070). Interpretation of the NSF policy intent is largely left to the discretion of researchers applying for NSF funding. This leads to uncertainty, as DPMs are being written, about the expectations of NSF and even the appropriateness and adequacy of stated data management practices. Recognizing that differing goals may apply to various research projects, developing examples of appropriate data sharing practices in relation to differing goals could improve clarity.

Issue 2: Ambiguous definitions of "data" and data-related terms. The OMB Circular A-110 definition of research data frames it in terms of a validation purpose (CGR 2006), one of only four main purposes of sharing data described by Borgman (2012). Furthermore, the OMB definition was created to indicate what may be excluded from public disclosure under FOIA requests. The intent of the NSF policy is broader, encouraging sharing between researchers, not just with public individuals. While the protection of confidential data is paramount for public disclosures of research results, limited sharing of such information with other researchers may be appropriate under certain circumstances and with stipulations of how the data may be used. The differences between expectations for public versus private sharing, use, and re-use should be delineated in data sharing policies.

The application of selective parts of the OMB definition of research data, ambiguous meanings for some words, and contradictory interpretation of words to suit particular viewpoints by different NSF directorates leads to inconsistent policy guidance. A standardized definition of data and related terms that are applicable across all NSF units would improve clarity. A unified NSF data definition could incorporate exceptions and additions to the OMB data definition such as inclusion of physical samples and metadata, or exclusion of digital laboratory notebooks. Agreement among NSF units as to the distinctions between "raw data" and "preliminary analyses" is needed to minimize ambiguity. Terms that could be clarified may include, but not be limited to, raw data, analyzed data, preliminary analyses, published data, access to data, and data sharing. Better definition of data-related terms applicable to the NSF DMP requirement could add clarity to the appropriateness of or considerations for pre-publication release, particularly for sharing with other researchers to advance science. Specific examples of data types and formats could be developed to address unique aspects by disciplinary areas, as has been done by only a few NSF units to date, while still adhering to a unified definition of data for NSF DMP purposes.

Issue 3: Undefined terms: "incremental cost" and "reasonable time." The term incremental cost has specific meaning to economists that might not be familiar to many scientists. Economic aspects of scientific data dissemination are discussed in Bits of Power (NRC 1997). Incremental cost pricing allows for cost recovery associated with data maintenance, recompiling, production of copies, shipping, and customer support, but not of the core service involving the original data collection and analysis. OMB Circular A-130 operates under a marginal cost model, which is actually lower than an incremental cost model (NRC 1997; OMB 2000). Provision of a definition of incremental cost within the NSF policy could clarify what charges may be attached to dissemination of research data that were collected using government funding.

Parameters for what constitutes a reasonable time for dissemination of research data in relation to different types of investigation and products should be provided within NSF policy guidance. The NIH data sharing policy direction could serve as a model for improving clarity on general guidance for timeliness. Most NSF guidance documents are silent on this aspect; however, the NSF Division of Ocean Sciences guidance specifies that data and data products must be "made accessible within two (2) years of collection" (NSF-OCE 2016, "Data and Sample Archiving"). The NSF Division of Earth Sciences similarly specifies that data should usually be made available within two years of collection, but further indicates that "for continuing observations or for long-term (multi-year) projects, data are to be made public annually" (NSF-EAR 2010). NSF program solicitations do sometimes give more specific direction; for example, data collected from the EarthScope Facility is expected to be "openly available in near-real time" (NSF-EAR n.d.). The NSF Division of Polar Programs direction generally specifies that data should be made available in a data center within the earlier of two years of collection or the end of the award, but provides different timeline requirements for specific programs, i.e., within six months of collection for Arctic Observing Network data, and within five years for Arctic Social Science Program research data (NSF-PLR 2016). Some publishers and journals (e.g., Nature, Science) now require data deposit along with manuscript submissions, with the intent of encouraging greater access to data for verification (especially during peer review) and to further the advancement of scientific research.

Issue 4: Minimal NSF guidance and absence of DMP examples. NSF has avoided providing much guidance on how to complete DMPs or on data sharing standards or protocols, ostensibly in the interests of allowing flexibility for a wide variety of research and the evolving nature of electronic technologies. Some organizations and academic libraries are attempting to address that gap by providing a variety of resources to their user communities, including the DMPTool. Although these resources are much needed, without more guidance from NSF they may result in divergent and conflicting advice. The NIH data sharing policy serves as an example of how greater direction can be provided while still allowing flexibility of application. Additionally, the NIH policy Q&As reference the guidelines for preparing DMPs compiled by the Inter-university Consortium for Political and Social Research (ICPSR) (NIH 2004; ICPSR n.d.). The guidelines for the NSF Directorate of Social, Behavioral & Economic Sciences mentions ICPSR as a possible archiving service, but do not provide a reference to their guidelines (NSF-SBE n.d.). The ICPSR guidelines are mostly generic, and most examples given are not exclusive to social sciences, such that they may provide direction to science disciplines as well. Resources, such as the ICPSR guidelines and similar references, could be identified by NSF to aid implementation of their policy. The NSF also could develop (internally or through outsourcing) sets of additional examples for specific disciplinary areas. Providing additional NSF guidance would likely improve the quality of DMPs beyond what might transpire through peer review feedback mechanisms. Some movement in that direction is evident in the 2015 revision of guidance for the NSF Directorate for Biological Sciences, which added a list of some resources that provide information on data management practices (NSF-BIO 2015).

Issue 5: Inconsistent guidance by directorates, divisions, and other NSF units. Although the apparent intent of providing some additional guidance by NSF unit was to address differences inherent in varied scientific disciplines, most of the guidance merely restates broad direction within existing policies. Where specific types of data and data formats are identified, most often they ae not exclusive to a particular disciplinary area. Some differences in emphasis by discipline are apparent, though. Stated differences have resulted in guidance inconsistencies between NSF units, especially in the interpretation of definitions and types of data relevant to the NSF Engineering Directorate. Inconsistencies in direction could be addressed through harmonization of guidance for directorates, divisions, and other NSF units. Differences in disciplinary practices could more readily be addressed through development of specific examples of types of data and formats common to different areas of investigation and types of studies. Harmonized information might better be presented in a single document, with specific examples organized by disciplinary areas associated with the NSF units. "The needs of research must drive the determination of specific policies; however, they need to be harmonized, removing any contradictions to better support the interdisciplinary world of today" (NSB 2005, p. 31).

Issue 6: Ambiguity of "communities of interest." Borgman (2012) discussed the shortcomings of designating communities of interest by which to interpret the appropriateness of DMP plans through peer reviews. She stated that, "The boundaries of communities of interest are neither clear nor stable....Communities of interest, as used by NSF, appear to be narrower than disciplines or research specialties...Common to both communities of practice and epistemic cultures is the idea that knowledge is situated and local" (Borgman 2012, p. 1061-2). While researchers may identify communities of interest in their DMPs, peer reviewers may or may not represent or be familiar with those communities of interest. Given the nebulous nature of communities of interest, selection of peer reviewers becomes an important variable in the determination of the adequacy and appropriateness of DMPs. The DMP review process could be improved by placing more emphasis on the development or adoption of standards for scientific data management, documentation, archiving, and preservation applicable to various types of data collection and recording. Examples of existing, applicable standards include metadata standards listed by the Digital Curation Centre, geospatial standards managed or endorsed by the Federal Geographic Data Committee, and the Open Archival Information System (OAIS; ISO 14721:2012).

Issue 7: Lack of centralized access to shared data. Locating useful research data can be challenging even when researchers make their data publicly accessible. "Only about a third (36%) of the [2009-2010 survey] respondents agree[d] that others can access their data easily, although three-quarters share[d] their data with others" (Tenopir et al. 2011, p. 9). "One stark difference [noted] between NSF and many other agencies [is that the] vast majority of long-lived data collections supported by NSF are managed by external research organizations, while other agencies...focus more heavily on archiving and curating many such data collections themselves (NSB 2005, p. 14). Given the wide diversity of research funded by NSF, it would be impractical to require data to be deposited into a centralized archive. Some disciplines already support digital repositories that provide a centralized location for research results in their fields. A National Science Board policy analysis concluded that "the weakness of NSF strategies and policies governing long-term data collections is that they have been developed incrementally and have not been considered collectively" (NSB 2005, p. 12).

Development of better indexes, databases, or portals that catalog or consolidate access to research data sets and publications resulting from NSF support, could improve findability. NSFs funding for development of the DataOne network for environmental data is a start in that direction. The Registry of Research Data Repositories provides a portal to thousands of data repository web sites, independently of NSF. The NSF Division of Ocean Sciences recently improved their guidance by specifying required Data Centers and repositories for data collected under specific programs (NSF-OCE 2016). NIH also maintains a list of available repositories applicable primarily to medical fields.

Encouraging use of standards or schemas for descriptive metadata and assignment of a persistent identifier (i.e., DOI = digital object identifier) also would make research data more readily discoverable and accessible. While NSF data sharing policy documents generally specify that metadata is to be provided along with data, NSF typically does not offer directorate guidance on what metadata schemas might be appropriate for various disciplines, or what minimum information should be included. Some disciplinary data repositories do have requirements for use of specific metadata schemas, though. "More than half of the [2009-2010 survey] respondents (56%) reported that they did not use any metadata standard and about 22% of respondents indicated that they used their own lab metadata standard" (Tenopir et al. 2011, p. 9). The 2015 version of guidance from the NSF Directorate of Computer & Information Science & Engineering does state that NSF encourages use of persistent identifiers (NSF-CISE 2015) but that guidance is generally lacking in similar NSF documents by other NSF units. The updated guidance for the NSF Division of Ocean Sciences specifies that unique sample identifiers (IGSNs) must be assigned to all physical geological samples and referred to in publications, in addition to archiving of voucher and type specimens (NSF-OCE 2016).

Issue 8: Uncertainty of long-term funding for data curation and preservation. "Nearly half (48%) of the [2009-2010 survey] respondents reported that their organization or project does not provide the necessary funds to support data management during the life of a research project ...Also, 59% of the respondents replied that their organization or project does not provide the necessary funds to support data management beyond the life of the project" (Tenopir et al. 2011, p. 7). NSF allows researchers to include budget requests for support of data management activities and access in their research proposals. However, grants normally do not extend much beyond the collection and analysis phases of research. Academic institutions, which often have tight administrative budgets, will likely need to plan for more extended support of the data products of research, especially to support individual researchers and small projects. This may involve the establishment and management of digital data commons or repositories by university libraries and/or centralized IT facilities along with providing personnel having data science expertise (NSB 2005). The National Science Board recognized three categories of research data collections based upon scope and complexity:

"The distinction between centralized and distributed collections can have important implications for developing policy for funding and ensuring their persistence and longevity" (NSB 2005, p. 21). Funding needs, especially over long periods, can be difficult to estimate. Costs will vary depending upon whether local or outsourced services are needed. Costs will also vary depending upon the intended audiences (e.g., public or research community) and level of documentation required. While data repository user fees can help defray costs of data maintenance, such practices may discourage access, use, and reuse. Finding sustainable sources of funding will likely remain a challenge.

Ensuring data maintenance can be even more problematic when researchers leave an institution. "In its role as the grantee, the research institution is required to hold title to or own the data through its contractual obligations...By tradition and for practical reasons, the creators of the data retain possession of the data on behalf of the institution" (CGR 2006, p. 10). "It is common for institutions to indicate in policy that the principal investigator serves as the custodian of data for their projects and as responsible agent for data preservation and retention" (CGR 2006, p. 12). Although an investigator may leave an institution, "institutions are obligated to assure access to and retention of data, and possibly to defend the value of associated intellectual property" (CGR 2006, p. 16). Given these legal obligations, it is often not appropriate for a researcher to take all of their research records with them. DMPs should include plans for such eventualities.

Opportunities for Librarians

Academic librarians are in a position to provide consulting and instruction services to help fill gaps in agency guidance, given librarians' expertise in information access, archiving and preservation. More than half (59%) of the respondents to the 2009-2010 survey indicated that their organization did not provide data management training on best practices (Tenopir et al. 2011), although significantly more respondents agreed in the 2013-2014 follow-up survey that their organizations provided such support (Tenopir et al. 2015). An assessment conducted at Georgia Tech in 2009 to identify data curation service needs found that a majority of respondents did not have a DMP, and of those without a DMP, 47% said they lacked knowledge about them (Parham et al. 2012). However, there was high interest in obtaining data curation services from library personnel. Tenopir et al. (2012) also reported on an assessment of academic libraries regarding their plans to provide research data services. "More libraries are offering or planning to offer informational/consultative-type services...rather than technical assistance services" (p. 17); however, a majority of libraries had no plans to provide consultation on DMPs or data and metadata standards, or outreach and collaboration with other data service providers (Tenopir et al. 2012). This suggests that there is room for academic librarians to increase assistance to researchers in meeting the DMP mandate of NSF and other governmental agencies.

Librarians who have or acquire data management expertise can provide guidance that is otherwise lacking from agency policies and guidance documents, and can explore technical assistance options for storage in research data repositories. Through researcher interviews, University of Houston librarians identified opportunities to provide consultative assistance or training for grant proposal processes and funder data management requirements, DMP preparation, data management practices, data visualization, manuscript preparation, data sharing systems and repositories, and making connections with campus data-related services (Peters and Dryden 2011). Bracke (2017) noted that many of the skills needed to provide an array of data management services are extensions of skills that liaison librarians and informationists already possess, but that collaboration with others having complementary skills is important. Additional examples where librarians can play key roles are in promoting and educating researchers about data organization and documentation practices for eventual curation and sharing, use of digital object identifiers and good data citation practices, understanding and use of open access publishing, and selection of appropriate data repositories.

The Association of Research Libraries (ARL) recognizes that there are leadership opportunities for academic librarians in helping researchers meet the NSF DMP requirements (Hswe and Holt 2012). They advise librarians to reach out to researchers and university administrators to encourage collaboration on data curation planning and metadata standards before data are generated (Hswe and Holt 2010). In their guide for librarians, they list a number of academic library web sites that provide data management tools and resources. Another resource is the ICPSR, which has compiled detailed guidelines for preparing DMPs, including a list of elements to consider with examples of text for each element (ICPSR n.d.). Data sharing requirements increase the need for services to support curation of data sets to move them from project level into data repositories designed for long-term storage. Respondents surveyed in 2013-2014 indicated they were significantly less satisfied with long-term data storage and metadata tools compared to respondents in 2009-2010 (Tenopir et al. 2015), perhaps reflecting a greater awareness of the need for technical support combined with a lag in the development of organizational infrastructure support for research data management services.

As researchers plan to prepare their data sets for sharing, they need to be aware that licensing and conditions for use and re-use may differ from those forms used in textual publication. The Open Knowledge Foundation supports the Panton Principles, which promote use of Open Data in Science by recommending the explicit placement of data sets into the public domain (Murray-Rust et al. 2010). They provide information about appropriate licensing options for sharing digital data. SPARC (Scholarly Publishing & Academic Resources Coalition) advocates for Open Data and provides some educational resources on the topic (SPARC c2007-2016). Librarians may want to explore these resources as well as others, and develop approaches to promoting and sharing these types of information sources within their research institutions.

Conclusion

The federal government has a long history of providing information to the public about programs and operations, facilitating democratic processes. This applies as well to scientific research funded by the federal government, although some periods of protectionism and secrecy have occurred during times of war and global competitiveness. An amendment to OMB Circular A-110 in 1999 placed more emphasis on sharing scientific research with the public, including research conducted at universities with federal funding. In recent years, an emphasis on Open Government has promoted even greater access and sharing of data and information.

Much of the recent emphasis on sharing data and information coincides with advancements in electronic technologies that allow for ease of digital collection and transmission of vast amounts of information via the Internet. In many cases, scientists are now able to amass large databases of information. Research has become more data intensive and governance is increasingly data driven. Combining of data sets has become more common as a means to uncover new relationships and understandings. Innovation and knowledge acquisition advances through a cumulative process, where sharing of research data plays a key role. Open sharing can encourage interdisciplinary pursuits and reveal unexpected new uses for data and information. The NSF policy on data sharing, and particularly the DMP requirement were developed out of recognition of these trends, and serves as further impetus to promote scientific research sharing. Such sharing now involves not just the publication of results within the scientific community, but also access to the underlying data. Scientific data and results can also be more broadly disseminated beyond closely associated groups of scientists, to researchers around the world, and to public audiences for purposes of information, transparency, and accountability.

These purposes are not without resistance and controversy, especially in relation to protection of confidential information and intellectual property. Greater guidance, including better definition of what is meant by research data, can help to navigate these concerns. Eight key issues were identified through critical examination of the NSF DMP requirement: lack of stated NSF DMP goals and purposes; ambiguous definitions of "data" and data-related terms; undefined terms of "incremental cost" and "reasonable time;" minimal NSF guidance and absence of DMP examples; inconsistent guidance by directorates, divisions, and other NSF units; ambiguity of "communities of interest;" lack of centralized access to shared data; and uncertainty of long-term funding for data curation and preservation. Given the shortcomings of the brief NSF data sharing policy, researchers are likely to need assistance in developing acceptable DMPs. Additional agency policy guidance and clarification is needed. Encouraging policy discussions between NSF unit managers, researchers, librarians, and other stakeholders to refine guidance, and looking to NIH policies for examples could be helpful in addressing identified issues.

Academic librarians are in a position to provide consulting and instruction services to help fill gaps in funding agency guidance on data sharing, given librarians' expertise in information access, archiving and preservation. There are many challenges to effectively implementing data sharing practices. Understanding of the unique qualities and needs of scientific research as well as the ever-evolving technical aspects of handling and managing digital data are necessary. As supporters and facilitators of research endeavors, academic librarians have a broad array of opportunities to become involved in promoting and supporting data management needs. Such opportunities include increasing awareness of data management tools and best practices and funder requirements for data sharing, assisting in creation of quality data management plans, providing technical support for data curation and promoting archiving in data repositories, involvement in development of metadata and data management standards and policies, and encouraging proper data use and data citation practices. More encouragement for Open Data initiatives and open access publication would also complement current policy focus.

Librarians interested in providing data management services typically acquire specialized knowledge and skills on the job and through targeted professional development to build a "broad understanding of data types, metadata, and legal and regulatory frameworks," (Kennan 2016, p. 1) combined with people and advocacy skills, basic understanding of information technology, and contextual knowledge about the research environment, processes, and funder policies. Data librarians may work collaboratively with IT specialists, research offices, and other subject and functional specialists to round out needed expertise to support researcher needs. Institutional support, particularly for infrastructure and related data services, is required to carry out the long-term goals of scientific data sharing and comply with funder requirements. Librarians are encouraged to continue the conversation with various stakeholders, including researchers and instructors, funders and policy-makers, research offices, IT specialists, program administrators, and vendors of research tools to improve provision and coordination of data management services.

References

Anderson, W.L. 2004. Some challenges and issues in managing, and preserving access to, long-lived collections of digital scientific and technical data. Data Science Journal 3:191-202. DOI: 10.2481/dsj.3.191

America COMPETES Act of 2007, Pub. L. No. 110-69, 121 STAT. 572 (Aug. 9, 2007).

America COMPETES Reauthorization Act of 2010, Pub. L. No. 111-358, 124 STAT. 3982 (Jan. 4, 2011).

[ALA] American Library Association (US). c1996-2016. The Fair Access to Science and Technology Research Act (FASTR) [Internet]. Chicago (IL): American Library Association. [cited 2016 Jun 21]. Available from: http://www.ala.org/advocacy/advleg/federallegislation/access/fastr

[ACRL] Association of College & Research Libraries, Government Relations Committee (US). 2012. ACRL Legislative Agenda 2012 [Internet]. Chicago (IL): American Library Association. [2012 Mar 22; cited 2017 Sep 13]. Available from: https://web.archive.org/web/20170322224358/http://www.ala.org/acrl/issues/washingtonwatch/12agenda

[ARL] Association of Research Libraries (US). [n.d.]. Federal Research Public Access Act (FRPAA) [Internet]. Washington (DC): Association of Research Libraries. [cited 2016 May 23]. Available from: http://www.arl.org/focus-areas/public-access-policies/federally-funded-research/2429-frpaa#

Borgman, C.L. 2012. The conundrum of sharing research data. Journal of the American Society for Information Science and Technology 63:1059-1078. DOI: 10.1002/asi.22634

Bracke, M.S. 2017. Agricultural librarians becoming informationists: A paradigm shift. Library Trends 65(3):448-466. DOI: 10.1353/lib.2017.0011

Burwell, S.M., VanRoekel, S., Park, T., & Mancini, D.J. 2013. Open Data Policy--Managing Information as an Asset [Internet]. Washington (DC): Office of Management and Budget. Memorandum No. M-13-13. [2013 May 9; cited 2017 Apr 20]. Available from http://obamawhitehouse.archives.gov/sites/default/files/omb/memoranda/2013/m-13-13.pdf

Bush, V. 1945. Science The Endless Frontier: A Report to the President by Vannevar Bush, Director of the Office of Scientific Research and Development, July 1945 [Internet]. Washington (DC): U.S. Government Printing Office. [1945 Jul 25; cited 2016 May 18]. Available from: https://www.nsf.gov/about/history/nsf50/vbush1945.jsp

Consolidated Appropriations Act of 2008, Pub. L. No. 110-161, 121 STAT. 1844 (Dec. 26, 2007).

[CPGE] Committee on Prospering in the Global Economy of the 21st Century (US), and Committee on Science, Engineering, and Public Policy (US). 2007. Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington (DC): National Academies Press. 564 p.

[CGR] Council on Government Relations (US). 2006. Access to and Retention of Research Data: Rights and Responsibilities [Internet]. Washington (DC): Council on Government Relations. [2006 Mar 1; cited 2016 Jun 7]. Available from: https://www.umassd.edu/media/umassdartmouth/officeofresearchadministration/Access_to_and_Retention_of_Research_Data_Rights_and_Responsibilities.pdf

Depository Library Act of 1962, Pub. L. No. 87-579, 76 STAT. 352 (Aug. 9, 1962).

E-Government Act of 2002, Pub. L. No. 107-347, 116 STAT. 2899 (Dec. 17, 2002).

Fair Access to Science and Technology Research Act of 2015, H.R. 1477, 114th Cong. (2015).

Fair Access to Science and Technology Research Act of 2015, S. 779, 114th Cong. (2015).

Fecher, B., Friesike, S., & Hebing, M. 2015. What drives academic data sharing? PLoS ONE 10(2):e0118053. DOI: 10.1371/journal.pone.0118053

Federal Research Public Access Act of 2012, H.R. 4004, 112th Cong. (2012).

Federal Research Public Access Act of 2012, S. 2096, 112th Cong. (2012).

Freedom of Information Act, 5 U.S.C. § 552.

Government Performance and Results Act of 1993, Pub. L. No. 103-62, 107 STAT. 285 (Aug. 3, 1993).

Government Printing Office Electronic Information Access Enhancement Act of 1993, Pub. L. No. 103-40, 107 STAT. 112 (Jun. 8, 1993).

GPRA Modernization Act of 2010, Pub. L. No. 111-352, 124 STAT. 3866 (Jan. 4, 2011).

Holdren, J.P. 2011. America COMPETES Act Keeps America's Leadership on Target [Internet]. [Washington (DC)]: White House. [2011 Jan 6; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/blog/2011/01/06/america-competes-act-keeps-americas-leadership-target

Holdren, J.P. 2013. Increasing Access to the Results of Federally Funded Scientific Research [Internet]. Washington (DC): Office of Science and Technology Policy. [2013 Feb 22; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/sites/default/files/microsites/ostp/ostp_public_access_memo_2013.pdf

Hswe, P. & Holt A. 2010. Guide for Research Libraries: The NSF Data Sharing Policy [Internet]. Washington (DC): Association of Research Libraries. [modified 2010 Dec 22; cited 2016 Jun 7]. Available from: http://old.arl.org/rtl/eresearch/escien/nsf/index.shtml

Hswe, P. & Holt A. 2012. A New Leadership Role for Libraries [Internet]. Washington (DC): Association of Research Libraries. [modified 2012 Mar 30; cited 2016 Jun 7]. Available from: http://old.arl.org/rtl/eresearch/escien/nsf/leadershiproles.shtml

[IWGDD] Interagency Working Group on Digital Data (US). 2009. Harnessing the Power of Digital Data for Science and Society: Report of the Interagency Working Group on Digital Data to the Committee on Science of the National Science and Technology Council [Internet]. [Washington (DC): National Science and Technology Council]. [2009 Jan; cited 2016 Dec 19]. Available from: https://www.nitrd.gov/About/Harnessing_Power_Web.pdf

ICPSR [Inter-university Consortium for Political and Social Research] (US). [n.d.]. Guidelines for Effective Data Management Plans [Internet]. Ann Arbor, MI: ICPSR. [cited 2016 Jun 8]. Available from: http://www.icpsr.umich.edu/files/datamanagement/DataManagementPlans-All.pdf

Jaeger, P.T., Bertot, J.C., & Shuler, J.A. 2010. The Federal Depository Library Program (FDLP), academic libraries, and access to government information. The Journal of Academic Librarianship 36:469-478. DOI: 10.1016/j.acalib.2010.08.002

Johnson, C. 3rd. 2005. Improving Public Access to and Dissemination of Government Information and Using the Federal Enterprise Architecture Data Reference Model [Internet]. Memorandum M-06-02. Washington (DC): Office of Management and Budget. [2005 Dec 16; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/omb/memoranda/fy2006/m06-02.pdf

Kennan, M.A. 2016. Data management: Knowledge and skills required in research, scientific and technical organisations. In: IFLA WLIC 2016; 2016 August 18; Columbus, OH. p. 1-10. Available from: http://library.ifla.org/id/eprint/1466

McClure, C.R. & Jaeger, P.T. 2008. Government information policy research: Importance, approaches, and realities. Library & Information Science Research 30(4):257-264. DOI: 10.1016/j.lisr.2008.05.004

McClure, C.R., Moen, W.E. & Bertot, J.C. 1999. Descriptive assessment of information policy initiatives: The government information locator service (GILS) as an example. Journal of the American Society for Information Science 50(4):314-330. DOI: 10.1002/(SICI)1097-4571(1999)50:43.0.CO;2-T

Murray-Rust, P., Neylon C., Pollock R. & Wilbanks, J. 2010. Panton Principles: Principles for Open Data in Science [Internet]. [2010 Feb 19; cited 2016 Jun 8]. Available from: http://pantonprinciples.org/

[NIH] National Institutes of Health (US). 2003. NIH Data Sharing Policy and Implementation Guidance [Internet]. Bethesda (MD): National Institutes of Health. [updated 2003 Mar 5; cited 2016 Jun 7]. Available from: https://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm

[NIH] National Institutes of Health (US). 2004. Frequently Asked Questions: Data Sharing [Internet]. Bethesda (MD): National Institutes of Health. [revised 2004 Feb 16; cited 2016 Jun 7]. Available from: https://grants.nih.gov/grants/policy/data_sharing/data_sharing_faqs.htm

[NIH] National Institutes of Health (US). 2007. NIH Data Sharing Policy [Internet]. Bethesda (MD): National Institutes of Health. [updated 2007 Apr 17; cited 2016 Jun 7]. Available from: https://grants.nih.gov/grants/policy/data_sharing/

[NIH] National Institutes of Health (US). 2015. National Institutes of Health Plan for Increasing Access to Scientific Publications and Digital Scientific Data from NIH Funded Scientific Research [Internet]. [Bethesda (MD)]: National Institutes of Health. [2015 Feb; cited 2016 Dec 20]. Available from: https://grants.nih.gov/grants/NIH-Public-Access-Plan.pdf

[NIH] National Institutes of Health (US). 2016. NIH Public Access Policy Details [Internet]. Bethesda (MD): National Institutes of Health. [updated 2016 Mar 25; cited 2016 May 23]. Available from: https://publicaccess.nih.gov/policy.htm

[NRC] National Research Council (US). 1997. Bits of power: Issues in global access to scientific data. Washington (DC): National Academies Press. 250 p. [cited 2016 May 23]. Available from: https://www.nap.edu/catalog/5504/bits-of-power-issues-in-global-access-to-scientific-data

[NSB] National Science Board (US). 2005. Long-lived digital data collections: Enabling research and education in the 21st century [Internet]. Arlington (VA): National Science Foundation. 87 p. Report No.: NSB-05-40. [cited 2016 May 23]. Available from: https://www.nsf.gov/pubs/2005/nsb0540/

[NSF] National Science Foundation (US). 2010a. Scientists Seeking NSF Funding Will Soon Be Required to Submit Data Management Plans [Internet]. Arlington (VA): National Science Foundation. Press Release 10-077. [2010 May 10; cited 2016 May 23]. Available from: https://www.nsf.gov/news/news_summ.jsp?cntn_id=116928

[NSF] National Science Foundation (US). 2010b. Data Management for NSF SBE Directorate Proposals and Awards [Internet]. [Arlington (VA): National Science Foundation.] [2010 Oct 12; cited 2016 Jun 7]. Available from: https://www.nsf.gov/sbe/SBE_DataMgmtPlanPolicy.pdf

[NSF] National Science Foundation (US). 2010c. Data Management & Sharing Frequently Asked Questions (FAQs) [Internet]. Arlington (VA): National Science Foundation. [updated 2010 Nov 30; cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpfaqs.jsp

[NSF] National Science Foundation (US). 2011. Data Management for NSF EHR Directorate Proposals and Awards [Internet]. [Arlington (VA): National Science Foundation.] [2011 Mar; cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/ehr.pdf

[NSF] National Science Foundation (US). 2016a. Award and Administration Guide: Chapter VI - Other Post Award Requirements and Considerations [Internet]. Arlington (VA): National Science Foundation. NSF 16-1. [2016 Jan 25; cited 2016 May 18]. Available from: https://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/aag_6.jsp

[NSF] National Science Foundation (US). 2016b. Proposal & Award Policies & Procedures Guide [Internet]. Arlington (VA): National Science Foundation. NSF 16-1. [2016 Jan 25; cited 2016 May 18]. Available from: https://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/

[NSF] National Science Foundation (US). 2016c. Proposal and Award Policies and Procedures Guide [Internet]. Arlington (VA): National Science Foundation. NSF 17-1. [posted 2016 Oct 25; effective 2017 Jan 30; cited 2016 Dec 5]. Available from: https://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf17001

[NSF] National Science Foundation (US). [n.d.a] Data Management for NSF Engineering Directorate Proposals and Awards [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 3]. Available from: https://www.nsf.gov/eng/general/ENG_DMP_Policy.pdf

[NSF] National Science Foundation (US). [n.d.b]. Dissemination and Sharing of Research Results [Internet]. Arlington (VA): National Science Foundation. [cited 2016 May 23]. Available from: https://www.nsf.gov/bfa/dias/policy/dmp.jsp

[NSF-BIO] National Science Foundation, Directorate for Biological Sciences (US). 2015. Updated Information About the Data Management Plan Required for Full Proposals [Internet]. [Arlington (VA): National Science Foundation.] [updated 2015 Oct 1; cited 2016 Jul 21]. Available from: https://www.nsf.gov/bio/biodmp.jsp

[NSF-CISE] National Science Foundation, Directorate for Computer & Information Science & Engineering (US). 2015. Data Management Guidance for CISE Proposals and Awards [Internet]. Arlington (VA): National Science Foundation. [2015 Mar 15; cited 2016 May 23]. Available from: https://www.nsf.gov/cise/cise_dmp.jsp

[NSF-GEO] National Science Foundation, Directorate for Geosciences (US). [n.d.]. Directorate for Geosciences-Data Policies [Internet]. Arlington (VA): National Science Foundation. [cited 2016 Jun 6]. Available from: https://www.nsf.gov/geo/geo-data-policies/index.jsp

[NSF-AST] National Science Foundation, Directorate of Mathematical and Physical Sciences, Division of Astronomical Sciences (AST) (US). [n.d.]. Advice to PIs on Data Management Plans [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/ast.pdf

[NSF-CHE] National Science Foundation, Directorate of Mathematical and Physical Sciences, Division of Chemistry (CHE) (US). [n.d.]. Advice to PIs on Data Management Plans [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/che.pdf

[NSF-DMR] National Science Foundation, Directorate of Mathematical and Physical Sciences, Division of Materials Research (DMR) (US). [n.d.]. Advice to PIs on Data Management Plans [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/dmr.pdf

[NSF-DMS] National Science Foundation, Directorate of Mathematical and Physical Sciences, Division of Mathematical Sciences (DMS) (US). [n.d.]. Advice to PIs on Data Management Plans [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/dms.pdf

[NSF-PHY] National Science Foundation, Directorate of Mathematical and Physical Sciences, Division of Physics (PHY) (US). [n.d.]. Advice to PIs on Data Management Plans [Internet]. [Arlington (VA): National Science Foundation.] [cited 2016 Jun 7]. Available from: https://www.nsf.gov/bfa/dias/policy/dmpdocs/phy.pdf

[NSF-SBE] National Science Foundation, Directorate for Social, Behavioral & Economic Sciences (US). [n.d.]. Data Archiving Policy [Internet]. Arlington (VA): National Science Foundation. [cited 2016 Jun 3]. Available from: https://www.nsf.gov/sbe/ses/common/archive.jsp

[NSF-AGS]. National Science Foundation, Division of Atmospheric and Geospace Sciences (US). [n.d.]. Division of Atmospheric and Geospace Sciences Advice to PIs on Data Management Plans [Internet]. Arlington (VA): National Science Foundation. [cited 2016 Dec 19]. Available from: https://www.nsf.gov/geo/geo-data-policies/ags/index.jsp

[NSF-EAR]. National Science Foundation, Division of Earth Sciences (US). 2010. [EAR Division Data Sharing Policy] [Internet]. Arlington (VA): National Science Foundation. [2010 Sep; cited 2016 Dec 19]. Available from: https://www.nsf.gov/geo/ear/2010EAR_data_policy_9_28_10.pdf

[NSF-EAR]. National Science Foundation, [Division of Earth Sciences] (US). [n.d.]. EarthScope Data and Sample Policies [Internet]. Arlington (VA): National Science Foundation. [cited 2016 Dec 19]. Available from: https://www.nsf.gov/geo/geo-data-policies/ear/es-data-policy.pdf

[NSF-OCE]. National Science Foundation, Division of Ocean Sciences (US). 2016. Division of Ocean Sciences (OCE) Sample and Data Policy [Internet]. Arlington (VA): National Science Foundation. NSF 17-037. [posted 2016 Dec 15; cited 2016 Dec 19]. Available from: https://www.nsf.gov/pubs/2017/nsf17037/nsf17037.jsp

[NSF-PLR]. National Science Foundation, Division of Polar Programs (US). 2016. Dear Colleague Letter: Data Management and Data Reporting Requirements for Research Awards Supported by the Division of Polar Programs [Internet]. Arlington (VA): National Science Foundation. NSF 16-055. [2016 May 9; cited 2016 Dec 19]. Available from: https://www.nsf.gov/pubs/2016/nsf16055/nsf16055.jsp?WT.mc_id=USNSF_25&WT.mc_ev=click

Obama, B. 2009. Transparency and Open Government: Memorandum for the Heads of Executive Departments and Agencies [Internet]. [Washington (DC)]: White House. [2009 Jan 21; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/the-press-office/transparency-and-open-government

Obama, B. 2012. Building a 21st Century Digital Government [Internet]. [Washington (DC)]: White House. [2012 May 23; cited 2016 Dec 6]. Available from: https://obamawhitehouse.archives.gov/the-press-office/2012/05/23/presidential-memorandum-building-21st-century-digital-government

Obama, B. 2013. Executive Order -- Making Open and Machine Readable the New Default for Government Information [Internet]. [Washington (DC)]: White House. [2013 May 9; cited 2017 Apr 20]. Available from https://obamawhitehouse.archives.gov/the-press-office/2013/05/09/executive-order-making-open-and-machine-readable-new-default-government-

[OMB] Office of Management and Budget (US). 1999. Federal Register Notice re OMB Circular A-110 [Internet]. [Washington (DC)]: White House. [1999 Sep 30; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/omb/fedreg_a110-finalnotice

[OMB] Office of Management and Budget (US). 2000. Circular No. A-130 Revised: Transmittal Memorandum No. 4: Management of Federal Information Resources [Internet]. [Washington (DC)]: White House. [2000 Nov 28; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/omb/circulars_a130_a130trans4

[OMB] Office of Management and Budget (US). 2016. Circular No. A-130: Managing Information as a Strategic Resource [Internet]. [Washington (DC)]: White House. [2016 Jul 7; cited 2017 Apr 25]. Available from: https://www.whitehouse.gov/sites/whitehouse.gov/files/omb/circulars/A130/a130revised.pdf

[OMB] Office of Management and Budget (US). [n.d.a]. Government Performance and Results Act (GPRA) Related Materials [Internet]. [Washington (DC)]: White House. [cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/omb/mgmt-gpra/index-gpra

[OMB] Office of Management and Budget (US). [n.d.b]. CIRCULAR A-110 REVISED 11/19/93 As Further Amended 9/30/99 [Internet]. [Washington (DC)]: White House. [cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/omb/circulars_a110/

Orszag, P.R. 2009. Open Government Directive [Internet]. Memorandum M-10-06. [Washington (DC): White House.] [2009 Dec 8; cited 2017 Apr 20]. Available from: https://obamawhitehouse.archives.gov/open/documents/open-government-directive

Paperwork Reduction Act of 1980, 44 U.S.C. §§ 3501-3521 (1980).

Parham, S.W., Bodnar, J., & Fuchs S. 2012. Supporting tomorrow's research: Assessing faculty data curation needs at Georgia Tech. College & Research Libraries News [Internet]. [cited 2016 Jun 7];73:10-13. DOI: 10.5860/crln.73.1.8686

Peters, C. & Dryden, A.R. 2011. Assessing the academic library's role in campus-wide research data management: A first step at the University of Houston. Science & Technology Libraries 30(4):387-403. DOI: 10.1080/0194262X.2011.626340

President's Science Advisory Committee (US). 1963. Science, government, and information: The responsibilities of the technical community and the government in the transfer of information [Internet]. Washington (DC): White House. [cited 2016 Jun 13]. Available from: http://garfield.library.upenn.edu/papers/weinbergreport1963.pdf

Research Works Act, H.R. 3699, 112th Cong. (2011).

S. REP. NO. 114-224, at 5 (2016) (Conf. Rep).

Smith, K. 1998. Federal information policy: Putting it all together: 1998 Miles Conrad Memorial Lecture, February 24, 1998 [Internet]. [cited 2016 Jun 14]. Available from: https://nfais.memberclicks.net/assets/docs/MilesConradLectures/smith1998.pdf

SPARC. c2007-2016. Setting the Default to Open [Internet]. Washington (DC): SPARC. [2016 Jul 25]. Available from http://sparcopen.org/

Stebbins, M. 2013. Expanding Public Access to the Results of Federally Funded Research [Internet]. [Washington (DC)]: White House. [2013 Feb 22; cited 2016 Jun 21]. Available from: https://obamawhitehouse.archives.gov/blog/2013/02/22/expanding-public-access-results-federally-funded-research

Tenopir, C., Allard, S., Douglass, K., Aydinoglu, A.U., Wu, L., Read, E., Manoff, M., & Frame, M. 2011. Data Sharing by Scientists: Practices and Perceptions. PLoS ONE 6(6):e21101. DOI: 10.1371/journal.pone.0021101

Tenopir, C., Birch, B., & Allard, S. 2012. Academic Libraries and Research Data Services: Current Practices for the Future [Internet]. Association of College & Research Libraries. [cited 2016 Jun 7]. Available from: http://www.ala.org/acrl/sites/ala.org.acrl/files/content/publications/whitepapers/Tenopir_Birch_Allard.pdf

Tenopir, C., Dalton, E.D., Allard, S., Frame, M., Pjesivac, I., Birch, B., Pollock, D., & Dorsett, K. 2015. Changes in data sharing and data reuse practices and perceptions among scientists worldwide. PloS ONE 10(8):e0134826. DOI: 10.1371/journal.pone.0134826

[OTA] U.S. Congress, Office of Technology Assessment (US). 1990. Helping America compete: The role of federal scientific and technical information [Internet]. Washington (DC): U.S. Government Printing Office Report No.: OTA-CIT-454. [cited 2017 Apr 20]. Available from: http://ota.fas.org/reports/9023.pdf

Previous Contents Next

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License. W3C 4.0 Checked!