2024-01-23-simko24a.md

title	abstract	layout	series	publisher	issn	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	container-title	volume	genre	issued	pdf	extras
Reproducibility of the Methods in Medical Imaging with Deep Learning.	Concerns about the reproducibility of deep learning research are more prominent than ever, with no clear solution in sight. The Medical Imaging with Deep Learning (MIDL) conference has made advancements in employing empirical rigor with regards to reproducibility by advocating open access, and recently also recommending authors to make their code public—both aspects being adopted by the majority of the conference submissions. We have evaluated all accepted full paper submissions to MIDL between 2018 and 2022 using established, but adjusted guidelines addressing the reproducibility and quality of the public repositories. The evaluations show that publishing repositories and using public datasets are becoming more popular, which helps traceability, but the quality of the repositories shows room for improvement in every aspect. Merely 22% of all submissions contain a repository that was deemed repeatable using our evaluations. From the commonly encountered issues during the evaluations, we propose a set of guidelines for machine learning-related research for medical imaging applications, adjusted specifically for future submissions to MIDL. We presented our results to future MIDL authors who were eager to continue an open discussion on the topic of code reproducibility.	inproceedings	Proceedings of Machine Learning Research	PMLR	2640-3498	simko24a	0	Reproducibility of the Methods in Medical Imaging with Deep Learning.	95	106	95-106	95	false	Simk\'o, Attila and Garpebring, Anders and Jonsson, Joakim and Nyholm, Tufve and L\"ofstedt, Tommy	given family Attila Simkó given family Anders Garpebring given family Joakim Jonsson given family Tufve Nyholm given family Tommy Löfstedt	2024-01-23		Medical Imaging with Deep Learning	227	inproceedings	date-parts 2024 1 23	https://proceedings.mlr.press/v227/simko24a/simko24a.pdf