IG for Surveying Open Data Practices rev-002

Name of Proposed Interest Group:

Interest Group on an Open Questionnaire for Research Data Sharing Survey

Introduction (A brief articulation of what issues the IG will address, how this IG is aligned with the RDA mission, and how this IG would be a value-added contribution to the RDA community):

The open data landscape is changing rapidly and we are only beginning to understand the impact of policies and changes in researchers’ practice.  As open data policies are implemented and data sharing practices evolve, comprehensive benchmarking and tracking of open data practices can serve to illuminate advances in data sharing (where and by whom) and help to understand the reasons for different data sharing practices. Various survey reports have examined the gap between policy and daily research practices and how they might be bridged but the granularity and coverage of surveys needs addressing as open data practices vary widely between scientific disciplines and regions. In addition existing surveys differ widely by their questions and by respondent groups. The RDA community, inclusive of researchers, practitioners, and decision-makers would benefit from a coordinated, common open survey approach that could be adopted and implemented to track changes in practice and policy overtime.

Following a successful BoF session at the RDA 10^th Plenary Meeting, we would like to initiate an Interest Group to 1. develop a community-designed modular and interoperable open survey(s) questionnaire(s); 2. determine how such open survey(s) can be implemented; and 3. Explore how the open survey(s) results could be analyzed globally.

User scenario(s) or use case(s) the IG wishes to address (what triggered the desire for this IG in the first place):

Several stakeholders would be interested to use a survey questionnaire made freely available to investigate the progress, evolution and developments in data sharing among their constituents. Such stakeholders include but are not limited to policy makers, researchers and their research institutions, funding bodies and companies. The potential use cases for such stakeholders are:

Use case 1: The G7 of Science Ministers have organized a working party for Open Science since 2016. The latest meeting communiqué in 2017 (Turin, Italy www.g7italy.it/sites/default/files/documents/G7%20Science%20Communiqu%C3%A9.pdf) stated the importance of research metrics and indicators for Open Science. This IG and the survey(s) that it will generate would contribute directly and indirectly to this call for action.

Use case 2: As a result of the publication of the report ‘Open Data: a researcher perspective’, report partners CWTS and Elsevier were approached by the European Commission to inquire if the survey questionnaire used to perform the survey in their report could be used to run a survey among Horizon 2020 participants. Running the survey could have informed the European Commission on the practice of researchers funded through Horizon 2020, thus providing useful data to inform the European Commission both as a policy maker and a funder.

In 2017, the European Commission launched a call for tender for the development of the  next generation of their Open Science Monitor. This call specification included the requirement of running a survey on data sharing. The tender was won by a consortium including both CWTS and Elsevier which will lead in 2018 to a new revised version of the survey implemented for the report mentioned above. This first use case highlights the potential impact an open survey on data sharing could have for such players as the European Commission.

User case 3: The National Institute of Science and Technology Policy in Japan conducted “A Survey on Open Research Data and Open Access” to investigate Japan’s current status and challenges in Open Science. Authors conducted a survey of Japanese researchers at the Science and Technology Experts Network of NISTEP in Nov-Dec 2016. They were asked about their experience on sharing and using their article and data, their recognition of open research data, the sufficiency of resources, and the needs to support researchers. The response rate was very high (70.5%) and results were already used in a discussion on Open Science in the government’s Cabinet Office.

User case 4: Publishers and related companies are conducting similar surveys. For example, Digital Science published their report “The State of Open Data” with the results of a global survey of 2,000 researchers in 2016. This survey assessed the global landscape around open data and sharing practices. It  highlighted the extent of awareness around open data, the incentives around its use, and perspectives researchers have about making their own research data open. Such reports allow publishers and related companies activities to evaluate trends of data sharing in the research community to develop and offer best-fitted solutions. They also help researchers understand the potential of data sharing and enhances their practices.

User case 5: Science Granting Councils Initiative – The national research funding organizations involved in this peer-learning network (https://sgciafrica.org/en-za), which seeks to enhance the capacity of research planning and management capabilities, are interested in the potential of open data to support their research communities. The survey instrument proposed here would create an opportunity for this network to contribute to and benefit from its application. As research councils in this region formulate their strategies, comparable survey data would provide useful guidance to orient their policies and practices. Work in this direction would also provide an opportunity for SGCI members to contribute to a wider Global Research Council call for action on open data ‘to compare and learn from their emerging practices, and collaborate on training and outreach activities.’

Objectives (A specific set of focus areas for discussion, including use cases that pointed to the need for the IG in the first place.   Articulate how this group is different from other current activities inside or outside of RDA.):

Through this Interest Group, our objectives are three-fold:

1. Develop the User Community:

• Participation and perspectives: to support application of proposed activities, initial efforts will focus on involving and understanding the perspectives of users who have commissioned and are interested in commissioning open data surveys. What did they seek to understand through their survey? What data proved useful and for what purpose?
• Promote dialogue among questionnaire designers and survey users

1. Develop a community-designed modular and interoperable open survey(s):

• Horizon scanning: identify and analyze existing surveys to compare similarities and differences in topics addressed and indicators used; identify relevant stakeholders.
• Engagement & recruitment: engage with stakeholders (those who have commissioned and developed such surveys and other relevant ones identified in previous step) to raise awareness of this IG, assess their interest in collaborating, and identify skills and expertise available.
• Develop survey(s) and/or survey modules taking advantage of existing ones, available and willing expertise, focusing on communities with high interest and willingness to participate. Survey(s) could, for example, be developed in a step-wise approach developing specific pilots for specific communities, geographies, etc. (e.g. initial interest from funding bodies community could lead to a first Working Group).
• Address language and cultural differences to identify common grounds that can be applied globally.

1. Determine how such open survey(s) could be implemented and results analyzed globally:

• Assess available existing technical tools to choose the solution(s) best fitted for our purpose.
• Identify multiplying networks to deploy survey(s) such as societies or associations that can help achieve a deep reach out effect to run survey(s).
• Run survey(s) using above tool(s) and networks.

• To avoid duplication of efforts, we will investigate opportunities to perform survey(s) analysis in a coordinated fashion to reduce costs by avoiding duplication.
• This coordinated approach also seeks to promote reliability in the analysis of surveys and comparability of findings, which should allow for better benchmarking.

Participation (Address which communities will be involved, what skills or knowledge should they have, and how will you engage these communities.  Also address how this group proposes to coordinate its activity with relevant related groups.):

We anticipate this IG will attract a broad-based audience. During the BoF session at the 10^th RDA Plenary Meeting, we engaged scholars, research administrators, research funders, government representatives, publishers and research data service providers. Given the interest in the topic and the potential benefits flowing from comparable and harmonized survey data, we are confident this IG will be able to engage representatives from these sectors on an ongoing basis.

In order to be successful, this IG will need to count on skills and knowledge from experts who have experience in developing survey questionnaire development as well as in analysing and interpreting the results of these survey. Accordingly we will aim to engage and recruit in the IG the authors of existing surveys. Experts of research policies in local setting and specific disciplines will also be essential for the IG. This is why we plan to involve experts from various scientific communities.

We will engage stakeholders through our personal network and by direct interaction when names are available (e.g. authors of existing survey reports; list of 70+ participants to the BoF session we organized at the 10^th Plenary in Montreal). The IG will also run an engagement exercise through RDA plenaries and side events, or additional events of relevance which will be identified (e.g. submitting abstract to SciDataCon-IDW 2018).

Coordination with other RDA groups will take two forms:

1. Some IGs and WGs have activities that could have common interests with our IG as they deal policy, rewards, metrics, etc. which are key to development of data practices and policies. The two relevant IGs are Data policy standardization and implementation IG and Sharing rewards and credit IG. There might additionally be overlapping interests with the following groups: Data usage metrics WG, Exposing data management plans WG and Mapping the landscape IG.
2. There are many RDA IGs that represent specific scientific communities that we will want to engage as they might have interest to explore data sharing practices through a survey for their own community. Example of scientific communities IG: Agricultural Data IG, Biodiversity Data Integration IG, Chemistry Research Data IG, Digital Practices in History and Ethnography IG, Linguistics Data IG, Health Data IG, etc.

Similarly there are non-scientific communities also represented as IGs that we will want to engage such as Early Career and Engagement IG.

We will contact and communicate with co-chairs of those IGs in the first months of the IG and meet those interested at the two RDA Plenaries in year 1.

Outcomes (Discuss what the IG intends to accomplish.  Include examples of WG topics or supporting IG-level outputs that might lead to WGs later on.):

The first outcome of the IG will be the development of community-designed modular and interoperable open survey(s) questionnaire(s). These questionnaires will be made open and freely available. We anticipate this contribution will promote the use of surveys by organizations who would like to better understand research data sharing practices and/or policy effects. Being open and freely available, will reduce barriers  that may prevent organizations that have the interest but  would otherwise not be able to perform such an undertaking for lack of expertise and/or resources.

The second outcome of the IG will be a consequence of the first as survey(s) will provide results to track changes in practice and policy overtime. These results will help articulate better policies, identify existing gaps, prioritize research funding, initiate projects and initiatives such as for example research infrastructure.

The third outcome is dependent on the success of the third objective described above (analyze survey(s) globally) which could achieve if fruitful an aggregated survey result analysis. This last outcome will help improve reliability in the analysis of surveys and comparability of findings. It would also allow for better benchmarking, leading to global or regional views on research data sharing but also global approaches to data sharing in specific communities (e.g. universities).

While each of these outcomes could be driven through dedicated Working Groups, it is more likely that WGs emerge for specific time-limited tasks. The first outcome might, for example, lead to pilots for the development of a survey module targeting a specific stakeholder community (e.g. funding bodies). The second outcome could lead to a WG looking into the policy dimensions of data sharing. And the third outcome might in itself become a WG to explore a centralized survey(s) analysis scheme.

Finally, this survey might become a good example of RDA-based survey in the context of Open Science and Data Sharing as community-based development involving various stakeholders, developing a de-facto standard to conduct global and comprehensive surveys.

Mechanism (Describe how often your group will meet and how will you maintain momentum between Plenaries.):

An IG ‘Executive Committee’, composed of a limited group (max. 12-15), will meet virtually by video conference and/or phone on a monthly basis during the first year. The Executive Committee will focus its efforts on generating awareness, engagement, and recruitment of stakeholders into an IG community. The frequency of calls will be assessed for the following years depending on coordination needs and support generated by emerging sub-groups and WGs, who might lead on IG activities for limited periods of time. In addition to the IG co-chairs, the executive committee will take on board a representative from each sub-group and/or WG who will be the main contact point and who will report progress and decisions.

Sub-groups and/or Working Groups are indeed likely to emerge as the activities of the IG develop. Each will have a specific tasks and/or community focus. Their work will be self-organized but the co-chairs will create a structure to coordinate and support their contributions. Their interactions could take place virtually or physically (e.g. regionally localized group).

A calendar of events (e.g. conferences, workshops, etc.) will be created in order to collect and use opportunities for physical meetings beyond the RDA Plenaries.

More broadly, should the IG be approved and collaborations initiated to develop common survey instruments, we are aware that the issues of governance and infrastructure will need to be specifically addresses by the IG. Such support could be obtained for example through international organisations like the OECD who have an interest in developing policy indicators.

Timeline (Describe draft milestones and goals for the first 12 months):

The Gantt chart below shows how objectives 1 and 2 described above will start in year 1 and will overlap with an earlier start for objective 1 as its initial outcomes are needed to move forward objective 2. Objective 3 will start during phase 3 as we start exploring how survey(s) can be analyzed globally.

The modular approach suggested below will allow each survey/module developed in subsequent phases to benefit from expertise, experience and networks developed in previous surveys/modules. Process will be streamlined thus increasing efficiency and allowing use of Standard Operating Procedures developed and improved in previous phases.

RDA20QuestionaireResearchDataSurvey_v4_20180129.docx