Interesting Reads

AI Readiness

• AI-Ready Open Data: This Bipartisan Policy Center article discusses what it means for data to be "AI-ready" and how to implement this data in a strategical and ethical way. The article creates a clear path for policymakers to move the agenda forward in three actionable ways.

AI Reproducibility

• The Reproducibility Issues that Haunt Healthcare AI: This Nature article asks the question: Health-care systems are rolling out artificial-intelligence tools for diagnosis and monitoring. But how reliable are the models?

• The Reproducibility Crisis in ML-based Science: Hosted by the Center for Statistics and Machine Learning at Princeton University, this piece highlights the scale and scope of the crisis, identifies root causes of the observed reproducibility failures, and suggests progress towards solutions.

• The replication crisis has led to positive structural, procedural, and community changes

• Wanted: standards for automatic reproducibility of computational experiments 

• Is AI leading to a reproducibility crisis in science? by Philip Ball - This article discusses the ways in which scientists worry that ill-informed use of artificial intelligence is driving a deluge of unreliable or useless research. 

• Detailed Implementation of a Reproducible Machine Learning-Enabled Workflow by Kenneth E. Schackart lll, Heidi J. Imker, Charles E. Cook 

• Reproducibility vs. Replicability: A Brief History of a Confused Terminology by Hans E. Plesser

Ethics

• GRAIMATTER Green Paper: Recommendations for disclosure control of trained Machine Learning (ML) models from Trusted Research Environments (TREs)

• International Supercomputing Conference Session Explores Ethics in AI and HPC

• Open Geo AI - PODCAST  This podcast series, 'Open Geo AI', is an innovative series that aims to educate and inspire listeners about the powerful synergy between geospatial artificial intelligence, open data policies, and the utilization of satellite data to foster informed decision-making and societal progress.

• ISC Session Explores Ethics in AI and HPC in the Era of LLMs by Eric Hunter - This article provides an overview of the SC23 session on Ethics in AI and HPC led by Jay Lofstead and Jakob Luettgau. This session discussed the potential legal liabilities and other societal impacts of the adoption of generative AI. 

FAIR Data

• ORCID & GO FAIR US: Collaborators Working to Realize a FAIR Data Ecosystem - recent blog post by Nancy Hoebelheinrich and Shawna Sadler titled, “ORCID & GO FAIR US: Collaborators Working to Realize a FAIR Data Ecosystem.” This blog takes a deep dive into the FAIR Data Ecosystem and highlights how partnership between the two organizations can provide collaborative support to researchers.

• Are the FAIR Data Principles fair? - This practice paper describes an ongoing research project to test the effectiveness and relevance of the FAIR Data Principles.

• Making poorly described data FAIR-er using GenAI written by Mark Hahnel

• Quality in FAIR: is your data 'Barely FAIR', 'Human FAIR', 'Machine-Actionable FAIR' and 'WorldFAIR' presentation by Lesley Wyborn 

• FAIR Enough: Develop and Assess a FAIR-Compliant Dataset for Large Language Model Training?

FAIR in AI/ML

• FAIR AI Models in High Energy Physics: From our colleagues at UC San Diego, NCSA, and others, this article studies the robustness of FAIR AI models and their portability across hardware architectures and software frameworks, and reports new insights on the interpretability of AI predictions by studying the interplay between FAIR datasets and AI models. It could offer an approach for other scientific domains on how to make their AI models FAIR.

• Perspective: Unlocking ML requires an ecosystem approach This article calls out the differences between traditional software development and the ML lifecycle where "data is the new code".

• FAIR for AI: An interdisciplinary and international community building perspective

• New Croissant Metadata Format helps Standardize ML Datasets

• The way we measure progress in AI is terrible

LLMs

• Local climate services for all, courtesy of large language models

• OLMo: Accelerating the Science of Language Models

• Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research 

• Paloma: A Benchmark for Evaluating Language Model Fit 

• A Critique of Pure LLM Reason 

• On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?

Policy

• Strengthening and Democratizing the U.S. Artificial Intelligence Innovation Ecosystem is a document that describes "An Implementation Plan for A National Artificial Intelligence Research Resource."

• Are Institutional Research Data Policies in the US Supporting the FAIR Principles? A Content Analysis - We discuss ways in which these institutional policies represent a missed opportunity to implement the FAIR principles and suggest ways policies could be modified to encourage researchers to follow them. We also discuss future research opportunities to examine how policy implementation may affect what institutional support researchers receive.

• Does history rhyme? Supercomputing, AI, and the US government's support for a research data infrastructure written by George Strawn

Software

• ​Policy recommendations to ensure that research software is openly accessible and reusable

• Sustainable research software for high-quality computational research in the Earth System Sciences: Recommendations for universities, funders and the scientific community in Germany

• Research Software, Citation, and FAIR at FORCE2024

• The O3 guidelines: open data, open code, and open infrastructure for sustainable curated scientific resources

• Fatiando a Terra: a journey into open-source software for Geophysics

Other

• The People's Speech Dataset just got better discusses dynamic data in action and how updating datasets improves speech for everyone.

• The Environmental Impact of Digital Research

• Ten simple rules for recognizing data and software contributions in hiring, promotion and tenure