response-to-reviewers.tex

\documentclass[11pt]{mhletter}

\newcommand{\ui}[1]{\emph{\textbf{#1}}}

\begin{document}

\clearpage

\begin{center}
\textbf{\large Summary of changes}
\end{center}

The following is a point-by-point explanation of our responses to the reviewers' comments.  Appended after this is a version of the manuscript showing the detailed differences between the manuscript of July 2019 and the new version.  (Red indicates deleted text, and blue indicates added text.)  The manuscript differences include additional minor corrections and updates beyond the changes made for reviewer feedback, which we have done in an effort to improve the manuscript further (for example, by reducing the overall number of citations and improving overall text cohesion).

\vspace*{2ex}

\textbf{\underline{Reviewer \#1}}

\textbf{\textit{1. When reading the title, the introduction of a newly developed format is expected. The abstract also gives the impression as if the manuscript would describe a recent development. However, on page 3 (lines 38-39) it is stated that SBML Level 3 was released in 2010 (and published in 2015?). To prevent such a misunderstanding, it would be better to rephrase title and abstract.}}

Thank you for pointing out the potential for misunderstanding.  In the revised version of the manuscript, we have edited the abstract to clarify that SBML Level~3 has already been in some use, and that this is not an entirely new development.  With respect to the title, we have tried to find an alternative, but could not find one that did not leave open the possibility of \emph{other} misinterpretations or misunderstandings.  We feel the title, while imperfect, is still the best we can do for a review article like this, and we hope the changes to the abstract are enough to avoid potential misunderstanding.


\textbf{\textit{2. The only new features of SBML Level 3 compared to previous levels that are discussed are the modularity comprising packages. Is this the only relevant feature that has been introduced?}}

We have updated the manuscript to emphasize that some other smaller changes have been made, though the modularity is the main change.  These points are made in a new paragraph at the end of the section titled ``The structure of SBML''.


\textbf{\textit{3. While the development of SBML is indeed a success story, the manuscript's tone tends to be overenthusiastic and to gloss over difficulties or problems. Are there alternative standards? How frequently refuse researchers to provide SBML files for their models? How is the support by journals (probably favorable, but this should be discussed)?}}

The goal of the manuscript is to describe the possibilities that SBML offers, but we did not intend to gloss over difficulties or problems.  The manuscript already mentioned some related standards, and we have now tried to clarify the relationships better in the edited second paragraph of the section titled ``Impact of SBML''.  Since a comparison to other formats is not the focus of the paper, and the journal's paper guidelines have length limitations, we trust that this compromise is good enough.  With respect to journals, we have added text to the manuscript mentioning journal and funding agency guidelines.  We have also added more discussions of difficulties involving SBML.  These changes are in the second-to-last paragraph of the section titled ``Impact of SBML'' and the last three paragraphs of the section titled ``Forthcoming challenges''.


\textbf{\textit{Minor point: the individual parts of the manuscript, while well written, could be better connected.}}

To help connect the sections better, we have expanded the final paragraph in the introduction to provide a preview of the remaining sections, and in other parts of the text, we have added phrases referencing different sections in order to try to connect the sections better.


\clearpage
\textbf{\underline{Reviewer \#2}}

\textbf{\textit{When/why is standardization beneficial? Interchange standards such as SBML become necessary only when i) there are already multiple competing formats/tools and ii) interoperation between these tools is desirable. A new tool defining its own format in a niche area may itself become the de facto standard for that subfield, independent of the HARMONY/COMBINE/SBML community and process. This is probably the norm in computational biology and is, in and of itself, not a bad thing. Challenges arise only when multiple groups develop tools independently, they cannot interoperate, and interoperability is necessary or desirable. It would help with the framing of the article if the authors addressed the criteria for language standardization explicitly early on. For example, in the context of the descriptions of the Level 3 extensions, it is not immediately clear to the reader why the qualitative modeling field required an SBML standard (instead of just using tool-specific formats); later this becomes (implicitly) apparent on page 13 line 40 when the authors note the need for CellNOpt to interoperate with GINsim (the example of model exchange between Simmune and BioNetGen also makes this point)}}

Thank you for raising this issue.  This is, unfortunately, a difficult point to address with SBML because its needs have been largely driven on demand in a bottom-up fashion, with proposals for features and packages coming when people felt the need or desire.  Consequently, we are not sure how to give explicit ``criteria for language standardization'' in the sense described above.  Nevertheless, we have added text to try to help address the issue: (a) in the introduction, we added text that mentions how SBML's popularity led to communities of modelers asking whether it could be expanded or adapted to more cases; and (b) in the section titled ``SBML Level 3's modularity and breadth'', the new fourth paragraph discusses benefits of building on SBML rather than creating a new format.  As part of these and other edits, we also reiterated that standard formats are important for reproducibility of scientific results, and thus are another reason to standardize around tool-agnostic formats like SBML.


\textbf{\textit{2. Costs/benefits of the standardization process. Related to the above, it is important to note that the standards development process has both benefits and costs. The benefits are well-described in the paper. However, costs include that i) developing a standard requires time and resources that could be used for further advances to tools and methods; ii) standard formats inevitably lag behind the cutting edge; iii) standard formats generally have greater complexity than ad hoc or fit-to-purpose solutions because they represent the superset of features required to support multiple use cases, or iv) they may represent only the lowest-common denominator of functionality among multiple representations. The authors could likely identify several others.}}

These are important issues indeed.  In the revised manuscript, we have added text in the section titled ``Forthcoming challenges'' to acknowledge and touch upon some of the costs and benefits.  We feel that a more extended discussion of these topics has to be considered beyond the scope of this paper, but we trust that the new changes are enough to address the point to a reasonable extent.


\textbf{\textit{3. The value of SBML as a technical platform vs.\ as a governing body.  In the article the authors describe the value/impact of SBML both as a language and as a framework for community self-governance and standard setting. In the context of the Level 3 extensions it is difficult to disentangle the benefits of these two aspects. For example, the authors of CellNOpt and GINsim could have participated in the SBML governance structure and HARMONY/COMBINE events, but created a language independent of SBML itself. In what way is it an advantage to embed a qualitative modeling language within a format originally developed for reaction-based mathematical modeling? It is not made clear how the new packages connect with the core structure of SBML and thereby derive technical benefits, vs.\ simply representing nearly independent languages developed through the standard-setting process.}}

Thank you for this question. It points out that this issue was not addressed sufficiently in the original manuscript.  We have expanded a paragraph in the section titled ``SBML Level 3's modularity and breadth'' to discuss how building on top of an existing format offers advantages over developing a separate format.


\textbf{\textit{4. Human readability/usability of SBML. Using languages such as XML and RDF to represent linked data allows maximal expressiveness but comes at the expense of human interpretability and usability. For a model of any reasonable complexity, both SBML (XML-based) and BioPax (RDF-based) are nearly unreadable, which inevitably introduces the requirement of additional tools/viewers/editors or custom code to use and process them. It is perhaps for this reason that a recent publication by Kirouac et al. on reproducibility in quantitative pharmacology modeling  called for an "open-source, standardized format" (which SBML most certainly is) but also noted that SBML requires "extra effort" and is in practice "rarely" used, suggesting instead Excel or text files. This view is of course heretical but reflects a real challenge for the SBML community. This problem should be noted and ideally the article should include some reflection on approaches to addressing it (greater outreach, investment in tool development, perhaps a simplified text-based SBML-compatible format covering 80\% of typical modeling use cases, etc.)}}

This is a fair point.  We agree that XML- and RDF-based formats are not suitable for human use, and it is certainly true that spreadsheets are often tightly integrated in researchers' workflows.  We have added a new paragraph to the section on ``Forthcoming challenges'' that addresses this topic.


\textbf{\textit{5) SBML as a knowledge representation vs. a model implementation.  A final comment regards the notion of biological modeling languages as knowledge representations vs. model implementations. The SBML authors note rightly that the reaction-based format of SBML represents an abstraction of the underlying mathematical dynamical system and thus allows models with the same reaction structure to be simulated with different parameters, rate laws, etc.\ (which a model encoded as MATLAB equations would not easily allow). The authors might consider how this concept can be extended further up the abstraction hierarchy. For example, a signaling model could be encoded both as a qualitative model in sbml.qual or a classical reaction-based model in core SBML. These models are different from an implementation perspective but actually derive from the same knowledge. With this in mind, claims that SBML serves as a "knowledge integrator" (page 16, line 42) or a "knowledge base" (page 17, line 23) should not be offered with some caveats. From one point of view, an SBML model may represent a knowledge abstraction of a more detailed process; from another it may represent only one particular implementation of a yet more abstract knowledge representation.}}

Thank you for this perspective.  We struggled with how to address this better, but ultimately we concluded that we cannot adequately expand on these issues in this article, both due to space limitations and time limitations.  We have removed the problematic statements to avoid potentially misleading claims.  We hope that a future publication can explore this and related topics more deeply.


\textbf{\textit{Minor comments.}}

We have updated the text to address the minor comments.

\end{document}


%%% Local Variables: 
%%% mode: latex
%%% TeX-master: t
%%% End: