Global metascience seminar monitor

Overview Towards more transparent & replicable attributions Charlotte Rulkens

Charlotte Rulkens is an art historian and researcher working at the intersection of art history, epistemology, metascience, and Open Science. Her research focuses on attribution, expert judgement, replication, and transparency in the humanities. Before entering academia, she worked as a curator at the Mauritshuis in The Hague, where she specialised in seventeenth-century Dutch painting.

Abstract

Can replication contribute to better research in the humanities? This talk explores that question through art historical attribution studies of Rembrandt and Rubens. Focusing on the role of expert judgement in knowledge production, it shows how adapting concepts of replication can support methodological refinement, reduce bias, preserve expert reasoning, and foster more collaborative research practice. (https://www.eventbrite.com/e/rembrandt-rubens-revisited-tickets-1999788884386?aff=ebdsoporgprofile)

Good infrastructure is patient work. This year's annual meeting is built around exactly that idea — Crossref as precision infrastructure that is interconnected, long-term, producing something of real and lasting value only through sustained collective effort.

On the agenda: a new service provider program, our latest POSI audit, metadata quality across the network, a continued focus on research integrity, updates from across our diverse community, support for adopting rich metadata with new helper tools, and the results of our board election.

We hope you'll join us on 22 October. We’re running two sessions to make it easier for our global community to take part: Session 1 from 07:00–11:00 UTC and Session 2 from 14:00–19:00 UTC. (https://www.crossref.org/crossref-annual-meeting/)

(https://unesco-org.zoom.us/meeting/register/huknf3lgT-WABq3QC0PIEg#/registration)

This session of the AFIRE Experimental Funders Group will dive into Desk Rejection, the practice whereby staff at a funding agency screen applications early in the process and remove those that are unlikely to succeed, without sending them out for external peer review.

The AFIRE Experimental Funders Group series provides a dedicated space for research funders to delve into current issues, share experimental approaches, and discuss the future of research and innovation (R&I) funding.

We are pleased to invite Journals to participate in a focused training session on using the Harvard Dataverse Repository to manage and share research data associated with publications.

This training is specifically intended for journals that: Publish research data (e.g., quantitative/qualitative datasets, code, documentation) related to publications and are interested in enabling or enhancing data sharing by their authors. (https://support.dataverse.harvard.edu/event/getting-started-harvard-dataverse-repository-journals-0)

(https://sortee.org/upcoming/)

ReBound brings together researchers and key actors from research governance and policy to foster dialogue and build a shared understanding of research culture. Participants will work collaboratively to identify practical, actionable ways to strengthen research culture in selected areas, drawing on examples of both best and worst practice. The conference engages with pressing debates facing the scientific community today, including challenges in scholarly communication, the development of sustainable research assessment models, and growing constraints on international collaboration and academic freedom. It also explores the roles and options available to individual researchers and higher education institutions in shaping the future of research culture. (https://www.berlin-university-alliance.de/commitments/research-quality/austausch-vernetzung/Konferenz_Research-Culture/index.html)

Despite impressive performance in publications, many AI models fail when deployed in real-world settings. A key reason is poor or misleading validation. This talk explores common pitfalls in validating AI models, especially the misuse of performance metrics, and shows how they can create false confidence. Practical recommendations will be offered to guide more robust and trustworthy validation practices, aimed at supporting the safe and effective integration of AI into real-world workflows. With Annika Reinke

Dr. Annika Reinke is Deputy Head of the Department of Intelligent Medical Systems at the German Cancer Research Center, where she leads the Validation of Intelligent Systems group. She is an internationally recognized expert in the validation and benchmarking of biomedical AI and image analysis algorithms, with a particular focus on improving the quality, reliability, and methodological rigor of validation pipelines. (https://www.eventbrite.com/e/the-long-road-to-trust-why-ai-needs-rigorous-validation-tickets-1999788077974?aff=ebdsoporgprofile)

(https://us02web.zoom.us/meeting/register/Cn_XcxgVT7q7gTqFaCL4hw#/registration)

Lancaster University researchers will host a Cuppa Conundrum session in response to their work exploring creative methods for research integrity interventions. Cuppa Conundrums are informal, discussion-based sessions designed to prompt reflection and dialogue around contemporary issues in research and academia.

This session will build upon the previous cuppa conundrum session that identified current issues and barriers in the research integrity process, focusing on the potential changes we would like to see in the future of research integrity. The discussions could consider research culture change, policy and procedure and examining further case studies to problem solve together.

Best suited for: - those working in research integrity, ethics, governance, and research culture - researchers seeking to increase their own and collective accountability of a positive research culture - those committed to protecting and upholding the integrity of research

For further information about the Reimagining Research Practices project that developed the cuppa conundrum series please find further information here: Ethics - Lancaster University https://www.lancaster.ac.uk/research/research-culture/reimagining-research-practices/ethics/ If you are interested in hosting your own event of this style, please refer to our cuppa conundrum facilitation pack here: OSF | Cuppa Conundrums Facilitator Information - https://osf.io/agjt3/files/uwr3e

Note that these sessions form part of an IAA AHRC grant that is exploring creative methods for research integrity interventions. Your contributions to the session will be anonymised in adherence to Chatham House rules. (https://us02web.zoom.us/meeting/register/RwrG0HlrSQedBjkj-MbCYg#/registration)

Join usonline on 28–30 September 2026 for a free conference with 37 talks and workshops on how AI is changing research. Monday explores AI in the research pipeline, including evidence synthesis, research evaluation, and ethical adoption. Tuesday focuses on replications and reproductions, with a hands-on session on agent-assisted reproducibility checks. Wednesday combines workshops on knowledge management, evidence synthesis, and preregistration review with talks on analyst variability. Everyone is welcome: attend the full programme or dip into individual sessions. (https://forrt.org/ai-metascience/)

SPILLING THE REPRODUCIBILITEA: A DECADE ON & DECADES AHEAD

🗓️ September 25, 2026 💻 Online via Zoom | Free & open to all 🔗 Learn more: https://reproducibilitea.org/conference2026

Grab your favourite cup as we're pouring out what the reproducibility community has been brewing for ten years: - The problems in research reproducibility - How the community has been tackling them - Grassroots initiatives still driving change - Live journal club sessions to make sure every voice is heard 🎤 Speakers include Monya Baker (author of the landmark Nature piece "1,500 scientists lift the lid on reproducibility"), representatives from Retraction Watch, the African Reproducibility Network, and more.

Spots are first-come, first-served. 👉 Register here: https://forms.gle/KgemQ3gM2fEezYLv7 (https://reproducibilitea.org/conference2026)

Lancaster University researchers will host a Cuppa Conundrum session in response to their work exploring creative methods for research integrity interventions. Cuppa Conundrums are informal, discussion-based sessions designed to prompt reflection and dialogue around contemporary issues in research and academia.

This session will explore dilemmas of research integrity inspired by a series of case studies from across UK HEI's, considering support processes, potential barriers to reporting issues and the possibilities of earlier interventions in cases of research misconduct.

Best suited for: - those working in research integrity, ethics, governance, and research culture - researchers seeking to increase their own and collective accountability of a positive research culture - those committed to protecting and upholding the integrity of research (https://us02web.zoom.us/meeting/register/1Ki6WtyXTfm4UNRuxTKCYA#/registration)

The LIS Bibliometrics Conference 2026 invites researchers, practitioners, librarians, information professionals, and policymakers to explore the evolving intersections between AI, bibliometrics, scholarly communication, and research evaluation.

As AI technologies increasingly influence scholarly communication, research assessment, and knowledge production, the conference aims to critically examine both the opportunities and challenges AI presents for bibliometric research and practice. (https://lisbibliometrics.wordpress.com/c-2026/) (https://lisbibliometrics.wordpress.com/c-2026/)

On Tuesday 15th September 2026, UKRIO will host a subscriber exclusive training webinar providing an introduction to Research Integrity.

This interactive one-hour online webinar is aimed at researchers and research contributors from any discipline who are relatively new to research integrity. The session provides an introduction to the key principles and practices that underpin trustworthy, ethical, and high-quality research.

Drawing on the principles of the UK Concordat to Support Research Integrity, the webinar will explore the importance of honesty, rigour, transparency and open communication, care and respect, and accountability throughout the research process. Participants will have the opportunity to discuss real-life case scenarios in an interactive and supportive environment, helping them to consider how research integrity principles can be applied in practice.

Only registrations from email addresses affiliated with UKRIO subscriber organisations will be approved. To check if your organisation is a UKRIO subscriber, visit: https://ukrio.org/our-work/our-subscribers/ (https://us02web.zoom.us/webinar/register/WN_ffDw4xpmSCaq-CWhWuVIwg#/registration)

The 30th Annual STI-ENID Conference centres on "Indicators for a Responsible Research Future" — how science and technology indicators can promote responsible research and innovation, with attention to diversity, inclusivity, openness, and sustainability in research assessment.

Hosted by the ECOOM teams at KU Leuven, Vrije Universiteit Brussel, and the University of Antwerp.

This event is an ONLINE satellite event of RSECon26, organised by DRA and UKRN.

RSEs operate in a research ecosystem where the research process is rapidly becoming more open and reproducible.

This event will deliver three components:

1. A practical introduction to ongoing and emerging innovations (e.g., open publishing, data sharing, reproducible computation, code check), drawing on The Carpentries’ "7 Steps towards Reproducible Research" materials; 2. Pedagogical guidance on how to effectively train colleagues and support adoption of these practices, plus a discussion session on challenges and opportunities for convincing peers to engage; 3. A co-creation session to create a customised version of these materials focusing on the priorities most relevant to RSEs.

The training will be led by Danny Garside, DRA trainer and community manager, who brings experience presenting this material in train-the-trainer contexts with UKRN. The session will also draw lessons learnt from DRA's delivery of the OSPARK Bootcamp - a program which equips Open Science advocates with communication and outreach skills.

The registration fee for this training is 130 GBP.

Spaces are limited and available on a first come, first served basis. (https://events.digital-research.academy/event/134/)

In this presentation, I question some common assumptions in modern metascience. For example, I question whether exploratory research is more “tentative” than confirmatory research and whether questionable research practices such as HARKing (hypothesizing after the results are known) and undisclosed multiple testing are problematic. I discuss these and other metascience assumptions and argue that they may not be as problematic as commonly assumed.

In this talk, I introduce a concept of measurement validity with three key features that I argue can facilitate improved validation practices. First, it is restricted to the question of what a test measures. This contrasts with influential guidelines from the American Psychological Association, which state that validity should be ascribed to interpretations of test scores for different test uses. Second, it is based on a necessary and, in principle, sufficient causal condition for valid measurement; and third, it makes an explicit distinction between the validity potential of measurement procedures (e.g., tests) and the realised validity of concrete measurement outcomes (e.g., specific sets of test scores).

Anne-Sophie Schaltegger (ETH Zürich) — "A social and cultural perspective on interdisciplinary and transdisciplinary research assessment: learning from three Swiss funding schemes"

Research assessment mechanisms are still considered one of the main barriers to interdisciplinary and transdisciplinary research (IDR/TDR). Particularly in the funding domain, criteria and processes used to evaluate the quality of IDR/TDR proposals, or to monitor the progress of funded IDR/TDR projects, create biases against collaborative, boundary-spanning research. To be able to develop adequate solutions, more knowledge is needed about the current lived realities and experiences of actors involved in IDR/TDR assessment: panel members, funding officers, and the researchers themselves. In this seminar, I present insights from my ethnographic study of ten assessment panels and implicated funding officers and researchers in three Swiss IDR/TDR funding schemes. I will outline three main findings: 1) the need to acknowledge and leverage – rather than to reduce – the conceptual and epistemic diversity in assessment panels, 2) the potential of scaffolding IDR/TDR assessment as a relational, social process, and 3) the opportunity of using assessment itself as a resource for all involved actors in the challenging endeavour of making IDR/TDR happen successfully.

Hussein Zeidan (Vrije Universiteit Amsterdam) — "From slogan to practice: Restoring transdisciplinarity as a serious way of working"

Transdisciplinarity is increasingly celebrated as a pathway to better science, yet its growing visibility masks persistent conceptual ambiguity and a widening gap between rhetoric and practice. While publications often present it in optimistic terms, the real frustrations surface informally, in the hallway conversations where scholars acknowledge the tensions, contradictions, and institutional pressures shaping their work. In this contribution, I surface these underlying tensions and question how Pop-Transdisciplinarity have taken hold. I invite us to reconsider what we expect transdisciplinarity to achieve, how it operates in practice, and how it intersects with other approaches to societal and epistemic challenges. I aim to move beyond superficial claims toward a more honest, ethically grounded, and rigorous engagement, one that confronts transdisciplinarity's blind spots and situates it within broader debates on knowledge, power, and the moral responsibilities of science.

UK Reproducibility Network's inaugural conference! Convening the research community, drive broader collaboration, and to solidify UKRN’s long-term sustainability plan, the UKRN conference will showcase impact of its core initiatives and engage the research sector community as change agents to improve UK research #UKRN2026JoinUs (https://www.conference.ukrn.org/)

Research assessment shapes not only careers and funding decisions but also the culture of research itself. However, many current assessment systems, based on publication metrics or the expectations of linear career trajectories, were never intentionally designed, and therefore often fail to value what truly contributes to rigorous, innovative, and socially valuable research. This webinar shows how research on research can inform better funding decision-making. I will share real-world examples from my work as a statistician at the Swiss National Science Foundation (SNSF) and my current research on funding and evaluation.

(https://www.aimos2026.org/)

ICSSI 2026 is an interdisciplinary event that brings together experts from various fields to explore the dynamics of scientific research and innovation. The gathering facilitates dialogue between researchers from academia and industry alongside policymakers, publishers, funders, and administrators. The conference emphasises science and technology policy environments, opportunities, and challenges around the world, aiming to strengthen the worldwide significance and influence of research in this field.

The 2026 themes are: 1. The Future of Science Policy — how research can support a stronger social framework for science. 2. The Future Infrastructure of the Scientific Ecosystem — new technologies and models reshaping how scientists collaborate, publish, and make discoveries. 3. The Scientific Workforce — shifting skill requirements, representation, and professional development in science's international talent ecosystem.

Institutionally organized research funding emerged in the early 20th century, with significant expansion during and after World War II in the US. This sparked the adoption of project-based peer-reviewed research funding across Western systems, though at varying rates. Current criticisms focus on fairness issues, high costs, and questionable validity.

The seminar discusses emerging alternatives, including lottery-based approaches. While random funding distribution faces skepticism, tiebreaker lotteries among peer-reviewed proposals have gained more acceptance. The speaker proposes a lottery-first approach, in which a lottery determines who is eligible to submit a full proposal, presenting empirical data on fairness, costs, and satisfaction.

The talk concludes by examining why funding systems resist change despite criticisms and what functions beyond merit selection may explain this persistence.

Taking inspiration from guerrilla warfare, I will introduce a framework for managing scientific research that addresses the complexity of research while advancing goals of transparency and reliability. Real science is complex, and there is no single scientific method. It is a chaotic network of theories, data, models, graphs, comparisons, summaries, and narratives. A flaw in any part can ruin everything.

How can small teams of researchers challenge this complexity and emerge victorious? More statistical training is not enough. More resources is not enough. Lawrence of Arabia once said, "Guerrilla warfare is more intellectual than a bayonet charge." And this is how we too will proceed.

Instead of confronting the complexity directly, with more money and more people and more data and more meta-analyses, we must develop a SCIENTIFIC WORKFLOW that subdivides, analyses, and transparently justifies our individual research projects. The tools to do this already exist, and individual researchers can become scientific guerrillas today to improve their chances of victory.

The project Examining the Reproducibility and Robustness of Research (ERЯOR) is a comprehensive program to systematically detect, report, and prevent errors in scientific publications modelled after bug bounty programs in the technology industry with two major goals: 1) Estimating a benefit-cost ratio of funding error detection, and 2) Obtaining robust empirical estimates of types of errors and their prevalence in the literature.

discussing the interesting paper of Felipe Romero on philosophy of science and the replicability crisis.

discussing the 2021 paper by Hoekstra and Vazire, Aspiring to greater intellectual humility in science.

The gathering brought together researchers from across disciplines to explore barriers to collaboration and foster new connections. The afternoon was followed by a morning workshop designed to strengthen interdisciplinary collaborations across policy, research, and commercialisation.