Special Sessions

Abstract 
While a lot of STI research focuses on innovations in schol arly publishing as a research object, our own scholarly communication practices often rely on more traditional (closed) journal - based pre - publication peer review. In this special session, we aim to discuss to what extent this is an intentional choice, and in what ways we might benefit from the various affordances of open peer review - in particular the prospect of reusing open review reports across various outlets or contexts. The discussion will be prepared by four short pitches. First, some of the organiz ers of the preceding STI conference will report about experiences with open peer review in the 2023 STI iteration in Leiden (Ludo Waltman). A second pitch will collect the existing evidence on implications of open peer review for diversity and participatio n (Serge Horbach). A third pitch will reflect what open and portable review reports mean for the workings of the peer review economy in a publishing context (Wolfgang Kaltenbrunner). Finally, there will be a pitch on the possibilities of automation in peer review and its implications for openness (Felicitas Hesselmann). 

The subsequent discussion will focus on the following questions: 

• What are STI attendants’ motivations to (not) engage in open review formats?
• What are the most important opportunities and challenges related to implementing open and portable peer reviews?
• How do open and portable reviews alter the development of research arguments and the engagement with audiences?
• How compatible are various publication cultures with open reviews?

Abstract
Standardisation and standards are not yet established in the context of science, technology and innovation indicators. Starting from framing standardisation in the context of transdisciplinary research, we look first at the science base of standardization by analyzing the citation patterns of scientific publications they reference. Then, we present for the case of the bioeconomy standardization both on the co ntext of science and the regulatory framework. In the second half of the session, we use web mining to look at the implementation of standards, particularly international management system standards. After giving an overview of their implementation in Germ any and their link to companies’ innovativeness , we focus first on ISO 50001, a standard for energy efficient management, before we look at the spatial distribution of management system standards in German districts and their linkages to established scienc e and technology indicators.

The session will be based on the presentation of five papers generated in the context of the BMBF funded project StaK aWi by Fraunhofer ISI and ZEW – Leibniz Centre for European Economic Research. 

Abstract
Over the past five years , the reform movements for open science and research assessment have expanded significantly. Increasingly, their agendas overlap , with champions of open science seeing current assessment practices as incentivizing behaviors that hinder wider accommodation of open science activities among researchers (UNESCO, 2021, Nosek, 2019, Spector, 2022) . Reform of academic career assessment systems is often earmarked as a site through which to incentivize academics to more fully embrace open ways of working (e.g. EC, 2017, Mustajoki et al., 2021, Pietilä et al., 2023, Méndez and Sánchez - Núñez, 2023) . It is of course easy to say that researchers in general should be ‘recognized’ and  ‘rewarded’ for open science (in its various incarnations) . But what might this look like in practice, what has actually been tried out so far, how might researchers in their roles as evaluators and producers of knowledge respond to such steering efforts, and what might be the consequences for career evaluations and research practices in different research fields and academic institutional settings? 

This special session brings together a range of science studies researchers and practitioners working on the emerging intersections between open science and academic career research assessment reform s , to present, discuss, and debate current and future practices . Each speaker will be invited to reflect upon the promises, achievements, and challenges faced in turning this broad promise into workable practices.

Abstract
There is a growing interest in the scientometric community in open research information (ORI), that is, scientometric data sources that are open. Data in these sources can be freely accessed and shared. Examples of open data sources include global multidisciplinary infrastructures (e.g. Crossref, OpenAlex, OpenAIRE, OpenCitations, an ORCID), discipline - specific infrastructures (e.g., PubMed), and regionally focused infrastructu res (e.g., SciELO and Redalyc). 

The aim of this special session is to provide an overview of the current state and potential futures of the landscape of open research information. This will include the quality (e.g., accuracy and completeness) of the data available from open scientometric data sources, the use and adoption of open scientometric data sources, and developments regarding governance, cooperation, and sustainability of open scientometric data source. To get started, there will be four five - minut e talks aimed at offering a diversity of perspectives on the state of the open research information landscape. These talks will be given by the organizers of the special session, highlighting the various perspectives on openness of research information in the organizing team. One of the talks will cover the Barcelona Declaration on Open Research Information. The special session then explores potential futures of ORI through participatory foresight methods engaging present stakeholders, such as bibliometrici ans, policymakers, research evaluators, and the academic community.

After the conference, the scenarios and results will be consolidated, discussed with experts, and disseminated through explainer videos to engage a wider audience. The findings will inform the development of a research agenda for Open Research Information.

Abstract
This special session will serve as an exploratory forum to critically examine and envision the future of research evaluation in the context of sustainability, technological integration, and societal impact. Recognizing the evolving and somewhat elusive concept of sustainable scholarship, this session is designed to open up a multifaceted dialogue and brainstorm potential research direc tions that can lead to actionable changes. We will commence with two brief, thought - provoking presentations that set the stage by highlighting key challenges and opportunities within sustainable scholarship. The focus will be particularly on the integrati on of Artificial Intelligence (AI) and the dynamics of peer review systems. 

More specifically we will address the following challenges:

1. Design and Modelling of Evaluation Panels: Addressing biases related to gender, ethnicity, ageism, and other forms o f discrimination.
2. Biases in Evaluative Processes by Experts and Metrics: Critically examining the inherent biases present in the processes conducted by experts and research metrics.
3. Efficiency and Responsibility in AI Utilization: Discussing the role of AI in maximizing research evaluation efforts and the ethical considerations tied to its use.
4. Diversity, Creativity, and Academic Freedom: Encouraging environments that foster diversity and creativity, ensuring that academic freedom remains at the fo refront of scientific inquiry. 

Following the presentations, the floor will open for a discussion, inviting all participants to brainstorm and contribute ideas and perspectives on shaping a sustainable research evaluation culture. Participants will engage in a hands - on experiment involving expert judgment. They will be presented with a scenario that requires the selection of candidates for a hypothetical academic position, aiming to expose and address inherent biases and decision - making processes. This sess ion will serve not only as a discussion platform but also as an incubator for innovative ideas that can drive a more equitable and impactful research evaluation ecosystem.

Abstract
Over the past decade, there has been a significant increase in the number of bibliometric analyses published in the peer - reviewed literature, particularly outside of library and information science where bibliometric s is traditionally situated. While diversity has always been a strength, it also poses challenges in ensuring quality. The variability in reporting quality we currently see may, at least partially, be grounded in the lack of guidance on how to conduct and report bibliometric studies by experts from within the field. This special session will engage the community in a discussion about quality and reporting of bibliometric analyses. Our session relates to the conference theme of “Into the great wide open?” in that the level of openness in the bibliometrics community – our methods and analytical tools, the application of our work, our disciplinary backgrounds, and increasingly more often our data sources – have given rise to a challenge in ensuring the quality of our work and the integrity of the field. The session will combine an introductory panel and an open fishbowl conversation. Five chairs will be arranged in the center of the room, four of which will be occupied initially by our panellists . A fifth chair will be left empty for audience members to participate in the conversation , forcing a panellist to leave the fishbowl and invite a dynamic discussion. This session will be an opportunity for participants to provide their input and discuss current issues in the context of open science with a focus on transparency and reproducibility, rigor, robustness and reliability of bibliometric analyses

The trans-national reform movement for responsible research assessment is growing considerably. As part of this, there is increasing interest in monitoring and evaluating research cultures and environments (RCE) within national, program and institutional assessment processes. Moving from broad aspirations to workable, transparent and legitimate evaluation practices is not however straightforward. Meeting these demands will necessitate relevant, trusted and reliable indicators, that can provide the robustness of evidence needed for monitoring and evaluation. Rapidly unfolding developments around RCE indicators thus merit urgent attention from experts within the STI community. The aim of this special session is to bring together researchers and practitioners to debate urgent questions over emerging RCE indicators, draw out important lessons, and compare experiences a cross research systems. 

The format will consist of short presentations from invited experts from the Research on Research Institute’s AgoRRA RCE workstream and audience Q&A. Speakers will share insider accounts of the latest policy developments, debates and controversies in their respective national research systems, and to draw out technical and social issues RCE developments are throwing up. These include methodological questions over indicator design and data needs, potential disagreements over what to evaluate, resourcing, and opposition by some research system stakeholders.

Background
In its proposal for a new strategy, "Towards responsible publishing" , cOAlition S summarizes: “In the five years that have elapsed since the publication of the Plan S principles, the move toward full and immediate Open Access (OA) has become global and irreversible.” However, “this has been delivered through business models – such as Read and Publish agreements and APCs – which are highly inequitable”. The coalition now envisages a “scholar-led” and “community-based” scholarly communication system “without author-facing charges”. Furthermore, the general OA ambition of Plan S is “extended to include all scholarly outputs, such as preprints and peer review reports” because “the current practice of pre-publication peer review needlessly delays the sharing of research outputs”.

Session Goals
The aim of the session is to discuss the new focuses and priorities of Coalition S based on experiences so far – including evidence provided by the areas of research contributing to the STI conferences. Three topics will be prepared for short introductions to start discussions with the audience:

1) Empirical examination of the OA transformation so far: The discussion will focus on the empirical evidence pertaining to the transition into Open Access publishing in recent years. A core question will be how the transition aligns with the traditional research policy criteria of research quality and affordability.

2) Challenges with changing the market of scientific publishing: The discussion will focus on new initiatives for scholarly publishing, their foundation and implementation issues, in particular the vision of a “scholar-led” and “community-based” scholarly communication system “without author-facing charges”. To what degree are the trends in the present market of scientific publishing aligned with these principles, and how can they be fully aligned?

3) Exploration of change trajectories for editorial practices: What are the experiences so far with alternatives to pre-publication peer review, and how do they align with the plurality of existing community-based efforts, including disciplinary differences?

Abstract
Doctoral training is vital to the knowledge economy and increasingly central to STI policy. In many OECD countries, current science policy concerns about 'PhD production' are increasingly running into broader questions in STI policy. There is a pronounced need to understand where trained doctorates across countries e nd up working, how the careers of these graduates are changing, what factors affect these changes, and what these changes mean for individuals, organisations and policy .

This session explores recent empirical approaches using information from Electronic repositories of Theses and Dissertations (ETDs) to monitor these changes both from a policy and an academic standpoint. What do STI policy makers need to know about post - degree outcomes of docto rate holders? How can new empirical approaches address the significant empirical challenges as trained doctorate holders move into, around, and out of the domestic labor - market? What sources can be effectively leveraged, and h ow can potentially sensitive d ata , such as theses and dissertation repositories, be legally retrieved and used?

Abstract
In today's rapidly evolving social landscape, which encompasses critical issues such as climate change, poverty, and technological advancements, the intersection of policy and science holds paramount significance. Understanding how research informs and shapes policy decisions has become increasingly crucial, sparking discussions on methodologies for assessing impact. While tools like Overton and Altmetric.com have gained popularity, they do not provide neither enough coverture, nor open data, presenting two significant challenges: establishing open - access sources for policy documents and bridging the coverage gaps between the Global South and the Global North. The special session aims to address two specific interests: (i) exploring the differences and synergies between open and proprietary data sources of policy documents, and (ii) examining the disp arities in coverage between the Global South and Global North regarding policy documents and related scholarly outputs within these data sources. The proposal aims to foster debate on potential methodologies for leveraging open sources of policy documents and for identifying and analyzing inequalities in accessing research impact on policy. Furthermore, the session seeks to explore strategies for overcoming these inequalities and enhancing the significance of open data sources in the process.

The session's dynamics will involve brief speeches from invited guests, approximately 10 minutes each, forming a debate panel. Audience participation will be encouraged through commenting or making questions on the topic

This Special Session will explore the measurement challenges related to understanding and managing increasingly diverse research and innovation systems, with a focus on the questions facing policy makers. It is widely recognized that too many R&I environments fall well short of a desired level of di- versity, which leads to under - performance in terms of creativity and innovation, perpetuation of une- qual relations, reproduction of stereotypes, and other adverse effects. Moving forward, STI indicators must adapt and expand to provide policy makers (amongst other stakeholders) with the information they need to craft effective interventions. The presentations in the special sessions will explore the current ways in which diversity dimensions are measured, complexities associated with compounded inters ectional effects and evolving societal framings of diversity, and constructive ways to take this dis- cussion forward. Four dimensions of diversity will be covered in detail: gender, ethnicity, socio - eco- nomic background and physical ability.