Henning Teickner, Klaus-Holger Knorr

 

 

Q1: “Several previous studies have used mid infra-red spectra to assess peat organic matter chemistry, particularly holocellulose and lignin content. One in particular had calibrated the spectral data with a set of chemical analyses using various (non-peat) organic materials. This paper aimed to re-examine the calibration using more refined models and gauge the suitability for applying them to peat and peat vegetation. The authors do a thorough job in applying some sophisticated statistical techniques to the data used by Hodgkins et al. (2018) and show how the calibration models can be improved and what the limitations of the current dataset are. From this point of view, the paper is useful and adds to the precision and accuracy of using infra-red spectra to rapidly assess peat chemistry, with possible application to other organic matter types. A weakness is that the authors have restricted themselves to the training chemical data from the Hodgkins et al. paper which are a curious set of paper, wood and plant leaves. In fact, a prior referee of the Hodgkins et al. paper described the set as being “bizarre” and actually it is surprizing that such reasonable results are obtained from such an unlikely dataset. One would normally expect the training set for peat analysis to be based upon samples of actual peat, from as wide a set of sources as possible. Clearly, the authors have limited themselves to what was available from the previous study and have not engaged in any further chemical analysis. To be fair, this weakness is acknowledged (L462) and the need for further representative training data expressed as the next step. Additionally, given the quite detailed statistics, which use a Bayesian approach, some of the methodology might be made clearer to the more general reader as several concepts are not explained and taken for granted. Also, in places, the English is a little unclear or awkwardly expressed; I have tried to indicate some of the more difficult passages below. Finally, there are quite a few errors in the supplementary information."

A1: We thank you for your critical appraisal of the scope of our manuscript. It is completely correct that it would have been nice if we would provide already in this study improved models which use peat material as training samples. As you point out, we do not hide this fact throughout the manuscript and fully acknowledge that this issue was discussed by a reviewer of the study of Hodgkins et al. (2018) (old: l. 53 to 54).

There are three pragmatic reasons and one scientific reason why the scope of the manuscript is limited to what we have analyzed and presented and we take the chance to describe these in more detail here, but also to justify the current scope of the study:

1. This manuscript is a by-product of a PhD project which was conducted with the aim to assess if the prediction models can be reliably used during this project. For this reason, there are limited resources to conduct additional measurements.

2. The required holocellulose and Klason lignin measurements would have been outside our area of expertise.

3. Considering the growing number of studies which use the original models of Hodgkins et al. (2018), we considered it important to publish our analysis as soon as possible to avoid future misinterpretation of the models’ predictions. An alternative option would have been to find collaborating partners who conduct (or conduct ourselves) the required measurements which would have cost additional time, but we would of course be open to that.

4. We think — as you do — that the manuscript in its present form makes genuine contributions to the scientific discourse, especially of researchers computing spectral prediction models for peat.

We build on the critical comment of a reviewer of the original study of Hodgkins et al. (2018) by giving these critical comments a quantitative, causal, and testable foundation.

The quantitative foundation is that our critical appraisal of the original models builds on an actual analysis of the models with data and thus provides more than what critical comments in a peer review process can and should reasonably do.

The causal foundation is that we provide — based on our quantitative analysis — a detailed causal explanation how and why the original training data are not representative for peat. It is easy to state a reasonable assumption that training data acquired from samples of a different type of material may not be representative for peat. We argue that it is a genuine contribution to show that this is actually the case and to explain why. From this analysis, we learn something about the chemistry of the original training data, the peat samples, and about the relation between this chemistry and the performance of prediction models (and spectral data). We think that this knowledge will be important also when peat samples are available as training samples, because there is no standard stating what is a representative dataset or stating how to define a set of sources as wide as necessary to compute models with a certain predictive accuracy. Peat itself can be quite diverse. We show here — as a hypothesis to be tested — what aspects probably are especially important when computing and validating improved models.

The testable foundation is that we provide hypotheses which can be tested. Our first hypothesis is that the normalized “carbohydrate” peak height (carb) is a good indicator for holocellulose content only when there are no (large) differences in OH bond types and abundances between the samples. We assume that relevant differences in OH bond types and abundances can also exist for peat samples because the chemistry and structure of cell walls of the peat forming vegetation can vary (both the amount and type of OH bonds and the holocellulose content) (e.g. comparing tropical peat with large fractions of wood, peat from (temperate) forest mires, or peat formed by Sphagnum). This hypothesis can be tested e.g. by comparing predictive performances of models fitted to either type of data. Testing this hypothesis would go beyond computing improved models: we can for example also learn something about how to interpret MIRS in terms of the decomposability of peat.

Our second hypothesis is that the normalized “aromatic” peak height (arom15arom16) is a good indicator for Klason lignin only when samples do not differ in protein content. In direct analogy to the “carbohydrate” peak height, this hypothesis is relevant (because also peat samples can differ in their protein content), it can be tested in the same way as our hypothesis for the normalized “carbohydrate” peak height, and testing it can make us learn something about peat chemistry and MIRS interpretation: we can for example learn whether arom15arom16 (or intensities of similar parts of MIRS as are widely used in the interpretation or computation of humification indices (e.g. Broder et al. 2012)) are good indicators for some concept of decomposability. This may ultimately lead to a similar discussion as whether the C/N ratio is a good indicator for decomposability (or degree of decompostion), a discussion which to our knoweldge has not been considered relevant yet.

In addition, we apply tools to validate models (particularly the analysis of the prediction domain, the analysis of model coefficients, the analysis of model residuals) which have to our knowledge only seldomly been applied on spectral prediction models for peat. In particular where sample sizes are small, such analyses could in analogy to ours here show how small peat datasets are also not representative. We hope that our study helps to circulate such model validation steps by showing in detail how they can be useful and how to apply them.

To summarize: We agree that it would be ideal if we had already today models which use peat training data to predict peat holocellulose and Klason lignin contents. However, we argue that our study makes genuine contributions which go beyond this aim and which merit publication.

We would also like to thank you for the detailed recommendations to improve language and method descriptions. We addressed these points in the specific comments below.

 

Q2: “Abstract (L1) I feel there is a need to distinguish peat from soil organic matter. True, peat is a type of soil (a Histosol) and the organic matter of peat is a type of ‘soil organic matter’. However, the general expression of ‘soil organic matter’ is usually associated with mineral soils. Thus, in L3, Hodgkins et al. set out to predict peat holocellulose and Klason lignin contents, not organic matter holocellulose and Klason lignin contents; indeed, they specifically removed the training samples containing silicates.”

 

A2: We assume that you suggest to include the word “peat” (old: l. 3 to 4; new: l. 3 to 4): “Recently, Hodgkins et al. (2018) developed models to quantitatively predict peat holocellulose and Klason lignin contents”. We think that this is a good suggestion to avoid confusion and we changed the line as suggested.

There are several reasons why we framed the manuscript in terms of SOM and not exclusively peat and we list them here to explain our rationale:

1. We suggest that our study builds a bridge to studies computing prediction models for mineral soils because we show as a proof-of-concept that it is possible to predict holocellulose content also for the mineral-rich training samples. We therefore think that our study is also of interest for colleagues working with mineral soils.

2. There are studies computing prediction models which predominantly include mineral soil data, but nevertheless also include peat samples or even focus on peat samples (e.g. Helfenstein et al. 2021). These studies refer e.g. to soil organic carbon (SOC) which we also see as a term mostly used in the mineral soil community. However, as this study shows, these distinctions in terminology are less and less useful when soil C content is considered as continuum. We suggest that our study is of relevance also for mineral soils (because of our proof-of-concept mentioned above, because of the exemplary application of model validation approaches, and because of factors which may complicate prediciton of holocellulose and Klason lignin contents which may also be relevant in mineral soils).

Q3: “L13 “mineral-rich samples” – it is unclear whether this refers to peat or to mineral soil. Of course, it could be both. As mentioned later (L96), some peats can become contaminated with mineral material to varying levels, though these are relatively rare occurrences."

A3: We assume that you suggest to clarify already in l. 13 (old) what “mineral-rich samples” should refer to — mineral soil or peat. In fact, we wanted to communicate that this is a proof-of-concept applying to both cases. One problem is that we have no information on the mineral content for the particular training samples and another problem is that the training data probably are not representative for either case. However, the mechanism by which minerals create peaks in the spectra should be the same for either soil type.

We hope that the following change makes more clear that we refer to both soil types (old: l. 12 to 13; new: l. 12 to 14): “Finally, we provide a proof-of-concept that holocellulose contents can also be predicted for mineral-rich samples (e.g. peat with mineral admixtures or potentially mineral soils).”

Q4: “L29 “OM derived from plants and SOM” – better to reword as “SOM and OM derived from plants” (OM derived from SOM doesn’t make sense)."

A4: Thank you for pointing out this inconsistency. We actually wanted to write: “Plant and soil OM” (i.e. that the fractionation procedure is applied to both types of OM, not only SOM and not only plant OM). We changed the text accordingly (old: l. 29; new: l. 30).

Q5: “L37 “few species” – unclear if this is of wood or some other plants."

A5: Thank you for pointing out that this is not clearly written. We changed the text to make this clearer (old: l. 37; new: l. 38): “Most of these models consider only wood, material from few woody and non-woody species, or only specific vegetation organs (Elle et al. 2019).”

Q6: “L46 “a later study” – need to specify which."

A6: We here refer to all studies which were cited in the previous sentence. These are the studies which have cited Hodgkins et al. (2018) and which have used the models developed in Hodgkins et al. (2018) (and thus are the studies we are aware of which could have potentially validated the original models).

We changed the sentence to state this explicitly and hope that this was what you had in mind (old: l. 46; new: l. 48): “A major problem is that neither Hodgkins et al. (2018), nor any later study which was cited above and which used the models provided a thorough validation of the models.”

Q7: “L48 Replace “are” with “were”."

A7: We are not entirely sure whether you refer to l. 48 or actually wanted to refer to l. 49. In l. 48 (old), the sentence “The peaks are baseline corrected, their maximum height computed and divided by the sum of absorbance values in the spectra.” refers to the procedure in general, and thus present tense should be correct. However, it is true that the next sentence (old: l. 49) “These values are used to calibrate the prediction models.” refers to what was done in Hodgkins et al. (2018) and thus past tense should be used (as mentioned in Q8).

We therefore changed the two sentences to (old: l.48 to 49; new: l. 50 to 52): “In the procedure, the peaks are baseline corrected, their maximum height computed and divided by the sum of absorbance values in the spectra. In Hodgkins et al. (2018), these values were used to calibrate the prediction models.”

Q8: “L49 Replace “are” with “were”."

A8: Please see our reply to Q7.

Q9: “L51 Replace “for example” with “, for example,”."

A9: Thank you. We changed the text as suggested (new: l. 54).

Q10: “L56 Replace “of” with “in”."

A10: Thank you. We changed the text as suggested (new: l. 59). We made analogous changes in: old: l. 110, new: l. 117.

Q11: “L66 Replace “which” with “in which”."

A11: Thank you for mentioning this issue. We think that “for which” is more appropriate here since we refer to assumptions of a distribution and not in a distribution (please correct us if you see this differently).

We changed the sentence to (old: l. 66; new: l. 68): “In contrast, if unrealistic predictions occur, it is more reasonable to use a distribution for which assumptions are consistent with knowledge on the data generating process.”

Q12: “L67 I suspect most readers of ‘SOIL’ will not be so familiar with the term “beta distribution”; some explanation here is warranted."

A12: Thank you for pointing this out. We provide a brief explanation in the subsequent sentence (old: l. 67): “For example, the beta distribution assumes a lower and upper bound which can be mapped to the interval [0, 100] mass-%.” This sentence mentions the values the Beta distribution is valid for and this is in our opinion the most relevant characteristic here.

To make clearer what the role of the beta distribution in a model is, we elaborated this to (correcting the interval the beta distribution is valid for) (old: l. 67; new: l. 70): “For example, the beta distribution assumes a lower and upper bound which can be mapped to the interval (0, 100) mass-%. It can be used as replacement for the normal sampling distribution used in ordinary linear regression models, such as the original models.”

We hope that this is an explanation you had in mind, but we are open for detailed suggestions on how to improve the text here.

Q13: “L77 The meaning of “prediction domain” should be given."

A13: The term “prediction domain” is described in more detail in section 2.3. However, we agree that it is a good idea to explain the term already here: In principle, the prediction domain is the range of predictor variable values in the training data set.

To make this clear, we changed the sentences in l. 77 to 78 (old) to (new: l. 81 to 83): “Is the prediction domain of the training data representative for peat samples? If a model extrapolates outside the prediction domain — the range of predictor variable values in the training data set (Wadoux et al. 2021) — it is unclear if the predictions are valid.”

Q14: “L79 Replace “on” with “to”, replace “e.g.” with “, for example,”."

A14: Thank you for pointing this out. We changed the text as suggested (new: l. 84). We made analogous changes in: (1) old: l. 197, new: l. 214; (2) old: l. 423, new: l. 446; (3) old: l. 426, new: 450.

Q15: “L80 The meaning of “no problem-specific correlations” is unclear."

A15: Thank you. We agree that this term is not clear enough. With “problem-specific correlations”, we wanted to describe the dependence of a prediction model on spectral properties obviously specific for one type of material, only.

We changed the sentence to make clearer what we mean (new: l. 84): “An advantage of the independent reference sample set used by Hodgkins et al. (2018) is that the model is applicable to various OM types and, for example, does not depend on obviously material-specific spectral characteristics (Hodgkins et al. 2018).”

Q16: “L82 Generally, it is not good to start a sentence with “But”; “However,” might be better here."

A16: Thank you. We changed the sentence as suggested (new: l. 87).

Q17: “L83 The “generality” of the training data set is not so obvious – it is quite wood based, since paper itself is derived from wood. Secondly, it could be argued to be a disadvantage since it overlooks any potential diagnostic peculiarities found only in peat."

A17: We partly agree here. We agree that we should stress more that the generality is a potential advantage since it has not been tested so far. We tried to stress this aspect more as suggested (new: l. 88): “A potential advantage is therefore the potential generality of the training data set Hodgkins et al. (2018) used.”

However, we argue that the training data set is (at least one of the) most general training data sets available (based on the references cited in the introduction), and that the non-applicability to peat samples is also not so obvious without a detailed analysis.

It is true that the data set contains many wood samples, but it also contains not a small amount of grass, leave, and needle samples (22 % of the samples) which could potentially have similar spectral properties as peat samples. For this reason, we argue that without knowing our analysis, there are not many reliable arguments neither for, nor against the applicability of a model fitted to the data to peat samples.

Of course we agree with you that the original study should have validated the applicability of the training data, but we do not think that without any analysis it is possible to say beforehand that the models could not work. This is the only point we want to make here.

Q18: “L97 Replace “predictions” with “prediction”?"

A18: Thank you for pointing this error out. We assume that this refers to l. 87 and corrected this as suggested (new: l. 91).

Q19: “L98 Replace “also for” with “that includes”."

A19: Thank you. We changed the text as suggested (new: l. 103).

Q20: “L99 Of course, as mentioned above, it would have been better to have a set of genuine mineral soil samples.”

A20: We agree that this important limitation should be stressed again here. We therefore added the following sentence (new: l. 105): “This is no replacement for a model using more training data, but it is a test whether it is at least in principle possible for MIRS to contain sufficient information to predict holocellulose contents of relatively diverse samples.”

However, we stress that it is not clear what a “set of genuine mineral soil samples” is unless one defines it via its prediction domain (or the molecular structures which eventually cause the prediction domain). This is probably not that easy since different minerals also differ in their spectral properties (e.g. Stuart 2004), meaning that this requires detailed research. Our study exemplarily shows approaches to analyze such difficulties.

Q21: “L114 “The data are available…” – delete this; it is obvious."

A21: We agree that the formulation is not clear. We wanted to write that the data are accessible directly from the published paper (as supplementary information). It certainly is obvious that the data are used in Hodgkins et al. (2018), but we think that it is important to point to the exact data source used. Unfortunately, the supporting information has no own DOI.

To stress that we refer here to the access to the supplementary information, we changed the sentence to (new: l. 121): “The data are accessible from the supplementary information of Hodgkins et al. (2018).”

Q22: “L117 Replace “SOM” with “peat OM”."

A22: We changed the text as suggested (new: l. 125).

Q23: “L122 “informative priors” – again this term may not be readily understood and should be made clearer. Secondly, what exactly these priors are should be made more specific."

A23: On the one hand, we agree that it would be good to communicate this to readers unfamiliar with Bayesian analysis. On the other hand, we have to assume a certain familiarity with Bayesian statistics to avoid repeating too much content from textbooks on Bayesian statistics.

The term “informative” is no fixed term, but is a qualitative and problem-specific descriptor for the information content of a prior (Gelman, Simpson, and Betancourt 2017). Here, we mean with a “weakly informative prior” a non-flat proper (meaning that the prior probability distribution integrates to 1) prior which has no strong effect on the predictions in comparison to the respective frequentist model. That this was indeed the case was tested in supplementary information 1.

In particular, we assumed for both the intercept and regression coefficient a normal(0, 2.5) prior for z-transformed independent and dependent variable. This means that a one σ (denoting the standard deviation of z-transformed variables) change in the independent variable (e.g. the z-transformed normalized carb peak height) can result in the same change in the dependent variable (e.g. the z-transformed holocellulose content) when the absolute coefficient is ≥1, which has a prior probability of approximately 0.69. The relevant information is that in comparison to the frequentist model, the Bayesian models produce the same prediction intervals.

To describe this better in the text, we changed the text to (old: l. 121 to 123, new: l. 128): “For this reason, to facilitate model comparison, we also recomputed the original models as Bayesian models with weakly informative priors. With weakly informative priors, we mean here priors which result in approximately the same prediction intervals as the original frequentist models (supplementary Fig. 1).”

We think that giving detailed parameter values would not support understanding unless you have more detailed knowledge on Bayesian statistics (and perhaps the software implementation) and we stress that these are available in full detail already via the reproducible research compendium (file analysis/paper/002-paper-m-original-models.Rmd, l. 40 to 41).

 

Broder, T., C. Blodau, H. Biester, and K. H. Knorr. 2012. “Peat Decomposition Records in Three Pristine Ombrotrophic Bogs in Southern Patagonia.” Biogeosciences 9 (4): 1479–91. https://doi.org/10.5194/bg-9-1479-2012.