Introduction

Important Dates

• Artifact submission deadline: July 16, 2025
• Author notification: August 8, 2025

• Artifact submission deadline: August 7, 2025
• Author notification: August 23, 2025

Types of Artifacts

• Data repositories, with data used for or produced through the study;
• Tools and frameworks that are implementations of systems and services that can be used and potentially extended;
• Qualitative artifacts, such as interview scripts and survey templates. Interview transcripts and survey results are also very valuable, provided the authors can share them (for example, interviews may contain sensitive information about a company);
• Software engineering-specific machine learning models, for example, pre-trained models that can be used to solve software engineering problems;
• Replication packages, which can be a mix of the above depending on the research type, help other researchers replicate the study presented in the accepted paper.

Artifact badging

• The artifact is relevant to the paper;
• The artifact is placed in a publicly accessible archival repository, and a DOI or link to this persistent repository is provided;
• The artifact is properly documented, with a README file explaining, at a minimum, the repository`s content.

• The artifact is relevant to the paper;
• The artifact is properly documented, with a README file that provides a sufficient description to enable the artifact to be exercised;
• The artifact is complete (with all data and components) for full executions;
• The artifact is exercisable, i.e., other researchers can successfully execute it.

• The first page of the authors’ paper will be badged with the awarded badge(s)
• The paper will be marked with the badge(s) in the list of accepted papers on the webpage of the respective symposium and on the CBSoft program;
• The authors will be invited to present their artifacts in a special session during CBSoft 2025;
• The artifact will compete for a Best Artifact Award, which will be given during the CBSoft 2025 opening to reward authors` efforts in creating and sharing outstanding research artifacts.

Submission instructions

• The README file should describe the artifact and explain how the repository is organized. This information is the minimum for other people to get interested in using the artifact.
• The README file should contain a link to the accepted paper. The paper PDF can be within the artifact’s repository or in an external service (e.g., ArXiv).
• For artifacts that focus on data, the README file must cover storage requirements and ethical and legal statements if relevant.
• For artifacts that focus on code, the README file must cover aspects relevant to how to install and execute it. The README file should contain two special sections: one for requirements and one for installation. The requirements section must describe the requirements for executing the software system. Basic requirements, e.g., Java version, should be described. A requirements.txt file with explicit versioning information (e.g., for Python-only environments) should be provided if relevant. For completeness and fully functional software systems, the requirements section should cover aspects of hardware environment requirements (e.g., performance, storage, or non-commodity peripherals) and software environments (e.g., Docker, VM, and operating system). The installation section must include notes illustrating a very basic usage example or a method to test the installation. This could be, for instance, on what output to expect that confirms that the code is installed and working.

• The authors must publicly make the artifact available to the Evaluation Committee (and potential users). The authors must place their artifacts in a publicly accessible archival repository (and have a DOI).
• Note that links to individual websites, temporary drives (e.g., Google Drive), and GitHub are non-persistent. Thus, artifacts placed in such locations will not be considered for evaluation. Examples of persistent storage that offer DOIs are Zenodo, Figshare, and Open Science Framework. Therefore, if your artifact is placed in a non-persistent repository, you should create a copy of it in a publicly accessible archival repository and use a link to it to submit your artifact.

• Artifacts do not necessarily have to be publicly available for the evaluation process if the goal is only the “Functional” badge. This is the case of artifacts that authors will share with potential users upon request. In this case, for the evaluation process, the authors are asked to provide a private link or a password-protected link.

• Title and authors of the accepted paper;
• A structured abstract containing:
  - Description: A simple description of the artifact to be evaluated (1 paragraph);
  - Repository: The link to the artifact to be evaluated (with instructions on how to access it, if private);
  - Badges: The claimed badge(s), i.e., “Available” or “Functional”, with a brief explanation of why the artifact is eligible for such badges;
  - Required skills: Skills and knowledge required by a reviewer to properly review and execute the artifacts (e.g., programming languages, pieces of technology, etc.);
  - Required resources: Requirements to run the artifact (RAM, disk, packages, specific devices, operating system, etc). As we have explained before, such requirements should be reasonable for software engineering researchers.

Review Process

Organization