MMSys was the first ACM SIGMM conference to implement the ACM’s artifact reviewing and badging policy . Authors are invited to formally submit their supporting materials following the process below.
The reviewers for the Reproducibility Track will attempt to reproduce the experiments/simulations and assess if the submitted artifacts support the claims made in the paper through the clearly defined artifacts and instructions submitted with the paper including traces, source code, tools, original datasets, etc.
The Reproducibility Track awards badge(s) according to ACM’s artifact reviewing and badging policy to the accepted papers that publicly release the artifacts used in the paper. The badge(s) appears as a highlight along with the paper in the ACM DL.
Note that applying for a reproducibility badge is optional and does not influence the final decision regarding paper acceptance.
To make the reviewing task easier, we strongly recommend you to use a format for the artifacts that ensure easy reviewing and reproducibility. Note the following:
If you have an unusual experimental set up that requires specific hardware or proprietary software, contact the Reproducibility Track Chairs before submission
The artifacts go through a revision process, during which e-mail exchanges can occur between the authors and the Reproducibility Track chairs on behalf of the reviewers. The evaluators are asked to evaluate the artifacts based on the criteria defined by ACM.