Table  summarizes the peat properties (target variables) for which we computed models. The variables were selected because they can be used to understand and quantify important processes in peatlands and because many of them cannot be predicted with existing models e.g.,.

The prediction models were computed with a subset of the pmird database . The pmird database is a heterogeneous collection of infrared spectra and other chemical and physical properties of peat, peat forming vegetation, and dissolved organic matter (DOM) compiled from previous studies. For many peat variables, the database has data from several different sites and covers global gradients of conditions under which peat is formed. This makes the models more representative for many peat properties than existing models . The spectra that we used are mid-infrared spectra measured in transmission mode on dried and ground peat pressed to pellets together with potassium bromide. Details on the measurement devices and other measurement metadata are summarized, for each model, in Table S1 in the Supplement. C and N, O, H, and some of the S contents were measured with combustion elemental analyzers. Contents of all other chemical elements and of the remaining S measurements were measured using wavelength-dispersive X-ray spectrometry on peat pellets pressed from dried and milled material. Isotope values were measured with isotope ratio mass spectrometry coupled to the elemental analyzers used in the respective studies. Bulk density was measured from known dimensions of peat layers cut from cores and dry masses. Loss on ignition was measured by combustion. Details on analytical methods for individual studies may be found in the metadata stored in the pmird database.

We did not use spectra that were classified as already baseline corrected in the pmird database , except for those from dataset 13 where we checked that the corrected spectra are similar to the result of our procedure here, to avoid that differences in preprocessing would decrease the predictive accuracy of the models. The pmird database contains a comparatively small number of vegetation and dissolved organic matter (DOM) samples and we therefore do not consider our models applicable to DOM or vegetation in general, except for Sphagnum which forms the bulk undecomposed peat material in bogs. Since DOM can have spectral properties different from peat, we did not include DOM samples in our models. Except for some cores, the peat is from ombrotrophic bogs, and peat with larger mineral contents and fen peat is thus underrepresented. An overview on the geographical distribution of the peat cores used to train and test each model is given in Table S2.

The pmird database does not contain data for saturated hydraulic conductivity, total porosity, macroporosity, volume fraction of solids, specific heat capacity, and dry thermal conductivity for samples with MIRS . For these variables, we used modified versions of models from , , , and to predict these variables from bulk density or N content. These models are described in Sect. S1. As suggested in previous studies e.g.,, these models (also known as pedotransfer functions) can be used with bulk density and N contents predicted from MIRS to predict these physical peat properties, whereby all relevant errors are propagated.

The pmird database also does not contain ΔGf0 estimates. We predicted ΔGf0 from element contents (at least C, H, N, O) using modified versions of the models from , , and for the enthalpy of combustion and the entropy of formation, as described in . These models are described in Sect. S2.

All computations for this manuscript were made in R 4.3.0 . For each of the target variables, except the physical peat properties mentioned in the previous section, we computed three spectral prediction models. The three models use differently preprocessed spectra (no derivative, first derivative, second derivative spectra), but otherwise were computed in the same way. All spectral preprocessing was done with the ir package . To harmonize the spectra, we interpolated them to unit wavenumber resolution and clipped them to the range 650 to 4000 cm⁻¹. Next, we conducted an atmospheric correction of water vapor and CO₂ artifacts using the approach suggested in . First, we subtracted a baseline created from a Savitzky-Golay smoothed version of the spectra where regions with strong CO₂ peaks (645 to 695 cm⁻¹ and 2230 to 2410 cm⁻¹) were linearly interpolated and then we used CO₂ and water vapor spectra from the pmird R package see also to perform the atmospheric correction as described in . Due to differences in devices and measurement conditions, this procedure attenuated CO₂ artifacts, but did not remove them completely. Thereafter, the corrected spectra were baseline corrected using a convex hull procedure , normalized using the signal normal variate (SNV), the three versions of derivative spectra were computed, and all spectra were binned with a bin width of 10 cm⁻¹ to reduce the number of redundant predictor variables and reduce possible wavenumber shifts between measurements from different devices. Finally, we excluded intensities from 2250 to 2400 cm⁻¹, to avoid that remaining CO₂ peaks confound predictions.

We used normal, gamma, and beta distributions as likelihoods (Table ) and used Bayesian statistics to compute all prediction models. All models were computed with brms , using a logit (beta regression), log (gamma regression) or identity link function (normal regression), assuming a constant shape parameter (beta, gamma) or standard deviation (normal), using a normal prior for the intercept, gamma priors for the shape parameter or standard deviation, and regularized horseshoe priors for the slopes (for each predictor variable). The regularized horseshoe prior shrinks coefficients to zero except where they are strongly related to the response variable, conditional on other predictors. To reduce overfitting, we defined a large amount of shrinkage, by assuming that 5 of the 321 predictor variables have non-zero coefficients . The regularized horseshoe prior can lead to a complex posterior geometry that is difficult to sample from even with efficient sampling algorithms and to run most of the models without divergent transitions, we had to increase the degrees of freedom of the student-t distribution of the horseshoe prior from 1 to 3 or 4. This reduced the number of divergent transitions, but also leads to less regularization (deviation from the horseshoe shape) which may lead to overfitting and less interpretable model coefficients. In our case, less regularization was not a critical limitation because our aim was not to interpret model coefficients, but to optimize predictive accuracy, and because our model validation did not indicate overfitting (Table ).

The posterior distributions were estimated with Markov Chain Monte Carlo (MCMC) sampling with Stan , using 4 chains, 3000 warmup iterations and 2000 sampling iterations per chain. Chains were initialized with pathfinder . Maximum Monte Carlo standard errors for predictions of the target variables by the best models (see the next section) for each target variable are shown in Table . As mentioned above, some models had divergent transitions, but at least one model per target variable did not and we only evaluate and interpret models without divergent transitions. The largest rank-normalized R^ for model parameters was 1.01, indicating convergence of the chains .

We used the Kennard–Stone algorithm as implemented in the prospectr package to split the observations for each target variable into a training and a testing dataset, using the euclidean distance between the underived preprocessed spectra. The number of observations assigned to the training dataset was defined as min⁡(0.8n,nmax), where n is the number of available observations for a target variable and nmax=200. Observations that were not part of the training data were used for model testing. All models for the same target variables use the same observations for training and testing such that the models are comparable.

This procedure was chosen because our aim is to develop prediction models that are applicable to as diverse peat samples as possible, that is, to compute one prediction model with maximum prediction domain (the value ranges covered by all predictor variables) and smallest possible prediction error across this prediction domain. The Kennard–Stone algorithm maximizes the distance between spectra covered by the training data and therefore selects a diverse training data set.

An ideal test of the prediction models would use test data that covers the whole spectral range of the training data and is independent of the training data, which in the case of peat samples are samples from different peatland sites. We had to deviate from this ideal because of the heterogeneity of the pmird data. In particular, because there are only few samples from few peatland sites with large carbonate or silicate contents, it would have been possible with independent observations either only to test the models over a much smaller range of spectral variation if only independent test data would have been used, or to test the predictive accuracy for spectral conditions the model was not trained on. In the first case, we would risk overfitting in the untested spectral range, and in the second case, the predictive accuracy would be underestimated due to extrapolation. Therefore, as an alternative, we did not separate observations from the same cores or sites when defining training and test data. This allowed us to test the models across a much larger spectral range within their prediction domains. For many target variables, the overfitting risk should be small because both the training and the test data have samples from many different sites.

To compare models, we used the expected log predictive density (ELPD) e.g., computed on the test data. Model evaluation was performed with the loo package . Following rules of thumb , we assumed models to have equivalent predictive performance (according to the capability of our evaluation) when the difference of their ELPD (ΔELPD) is smaller than 4, and otherwise when ΔELPD is larger than two times its standard error (using normal approximation for ΔELPD). Models with divergent transitions were not considered during model evaluation. To give an easier to interpret performance metric, we also computed the root mean square error (RMSE).

We do not interpret model coefficients and how this may reflect causal links between molecular structures and target variables (1) because our model coefficients are not intended to estimate causal effects, (2) because it is very likely that they do not represent causal effects, and (3) because specific wavenumbers cannot be assigned unambiguously to molecular structures e.g.,. For those interested in model coefficients, we show a plot of the model coefficients with the best model for each target variable and a table listing possible assignments to molecular structures for coefficients with a posterior probability of being larger than 0 of at least 90 % or a posterior probability of being smaller than 0 of at least 90 % in Sect. S5.

A regression model interpolates a target variable within the range of predictor values – the prediction domain . If such a model is used for prediction with new data that are outside the prediction domain, it is unclear how large prediction errors are, particularly for models with high dimensional prediction domain, such as spectral prediction models. Consequently, it should be checked that new data are within the prediction domain of the model e.g.,, even though this is no guaranty for accurate predictions.

For this reason, we computed the training prediction domain for each model as the range of the predictor variable values across all training samples (training prediction domain), and a prediction domain for the test samples (test prediction domain) for each model as the range of predictor variable values across all testing samples. When samples are outside the prediction domain, predictions may be less reliable than estimated by the model validation.

The difference between testing and training prediction domain shows where the models need further testing. The difference between training (or testing) prediction domain and the prediction domain formed by all relevant spectra in the pmird database indicates whether the model covers the spectral variability in the pmird database, as approximation of the spectral variability of peat in general, and therefore indicates where additional data can improve the models.

When making predictions with the models, irpeat checks whether the input data are within the testing or training prediction domain. This is a safety device to avoid a misuse of models and it provides information for those who want to improve our models.

This test of MIRS against prediction domains is only a first test because even spectra within the prediction domain may have spectral properties and values for the predicted peat property that are different from the training or test data. With a large enough training and test dataset, such edge cases become more and more unlikely. Additional checks to be provided by future studies are based on a list of error sources identified through targeted tests of the models against such possible edge cases.

Based on previous experience in the interpretation of peat MIRS and on the peaks caused by silicates, carbonates, amides, carbohydrates, aromatics, and lipids , we suggest that the main gradients in peat chemistry that control spectral variation are (1) the content of silicates, (2) the content of carbonates, (3) the initial vegetation composition that controls differences in the initial content of amides, carbohydrates, and aromatics, and (4) the degree of decomposition, which increases the relative contents of amides, lipids, and aromatics, and decreases the overall content of carbohydrates e.g.,. Previous studies suggest that differences in amide contents and silicates can bias predictions and similar effects are likely for carbonates, because carbonates cause dominant peaks that overlap with peaks caused by aromatics and amides and because large carbonate contents usually indicate higher pH values and therefore shifts in carboxyl peaks due to deprotonation . To test for such confounding factors, we plotted model residuals versus Ca, Si, and N measured for the same samples (residuals were not plotted for samples where Ca, Si, and N, respectively, were not measured).

To fill data gaps in the pmird database, we used the best models for each target variable (Table ) to predict missing values, including prediction errors as samples from the posterior predictive distribution, for the target variables where samples have MIRS. These predictions are stored in a published data table. Moreover, we created two additional data tables that indicate, for each prediction, whether the MIRS is in the training or testing prediction domain for the respective model. We restricted gap filling to peat and litter samples with absorbance-FT-MIR spectra. In contrast to the model development, we included spectra that may have already been baseline corrected, since the prediction domains can be used to screen spectra that are not similar to the data used to train and test the models. For variables that can be predicted without MIRS if other data are available (e.g., C, H, O, N, bulk density) with the additional models developed here (ΔGf0, saturated hydraulic conductivity, total porosity, macroporosity, volume fraction of solids, specific heat capacity, and dry thermal conductivity) or previously published models (NOSC, C/N, H/C, O/C), we created an additional data table with predictions without MIRS. For ΔGf0, we required C, H, O, and N contents to be measured for this; contents of other elements were included if available and otherwise the contents were set to 0 g g⁻¹ when computing ΔGf0.

The predictive accuracy for the best models for each target variable is summarized in Table  and plots of measured versus predicted values are shown in Fig. . Estimates for the predictive accuracy are both worse and better than that of previously published models using spectra in the visible, near infrared or mid-infrared range (Fig. ), but these estimates are not directly comparable because of different modeling approaches and differences in the variability of data used to train and test the models. Studies that use large databases can use modeling approaches that require more training data but may outperform linear models, such as cubist . Another reason for differences in the predictive accuracy is the chemical diversity of the training and testing data. For example, one reason for the better predictive accuracy for C contents, N contents, and C/N reported in certainly is that samples are from one site only which leads to less confounding between predictors and C content. Similarly, data from , , and also cover smaller gradients in peat properties than the pmird database, in particular no peat with large mineral fractions (Fig. ).

Not only differences in the range of chemical properties, but also the distribution of observations along chemical gradients can lead to differences in estimated predictive accuracies. For example, models from and for C and N contents have a better predictive accuracy and were computed with many peat samples, but the majority of observations is from mineral soils with small C and N contents. It has been repeatedly observed that prediction errors are larger for larger C contents than for smaller C contents . These heterogeneous prediction errors are probably caused by two factors: Firstly, spectra of mineral soil have prominent mineral peaks which allow a more accurate estimation of small C contents, whereas at large OM contents there is a much more complex and diverse pattern of peaks caused by organic matter molecular structures. Secondly, whenever a variable is positive and the majority of values is small, prediction errors are smaller due to the positivity constraint e.g.,. This does not only explain worse estimated predictive accuracy of our models for C and N, but can also explain why our model for bulk density has a better estimated predictive accuracy than models from and because the majority of the peat samples have a small bulk density and the positivity constraint therefore implies smaller prediction errors. A last reason for a better predictive accuracy is outlier removal in previous studies not based on specific theoretical considerations . We did not remove outliers here because we wanted to develop prediction models that are applicable to a diverse range of peat samples, while outlier removal may lead to better predictive accuracy for peat samples with specific chemical properties. Overall, the modeling approach and data properties of our studies are most directly comparable to , who focused on C contents, and here, our model performs similarly well.

In summary, our models have a roughly similar or better predictive performance for some variables as have previous studies focusing exclusively on peat samples. Albeit direct comparison of prediction errors to high-quality models computed with large spectral libraries is not possible in terms of peat, it is very likely that the predictive accuracy for peat properties could be improved with more flexible modeling approaches, such as localization e.g., or different machine learning algorithms, which would however, in many cases, require more balanced data and, for some variables, more data in general.

There are some interesting patterns in the plots of measured versus predicted values (Fig. ): For C, there are two outliers, one with a measured C content >0.6 g g⁻¹ and one with a measured C content <0.1 g g⁻¹ (Fig. S20). A peat sample with >0.6 g_C g⁻¹ should be decomposed because litter initially has large O and H contents that cause small relative C contents and because preferential decomposition of organic matter fractions with large O and H contents (carbohydrates, phenols) leads to a relative accumulation of C , but the spectrum does not have pronounced aromatic or lipid peaks one would expect for a peat sample with such high C content (Fig. S20). A peat sample with <0.1 g_C g⁻¹ must have a comparatively large mineral content because undecomposed peat forming litter has much larger C contents, yet the spectrum does not have typical silicate peaks (Fig. S20). We therefore assume that either the C measurements in the pmird database are not correct for these two measurements or that spectra were incorrectly assigned to these samples. For H, the plot indicates overestimation for smaller H contents and underestimation for larger H contents. Samples with smaller H contents have larger Si contents (Fig. S18) which suggests that predictions are confounded by silicate peaks.

For P and K, several samples with the largest measured P and K contents have relatively large prediction errors. These samples are Juncus effusus samples from a short-term NPK fertilization experiment with high nutrient loads . While it appears that the models can predict P and K contents reasonably well for these samples, more samples would be required to evaluate whether the model overfits to spectral characteristics of J. effusus litter (Fig. S20) rather than spectral characteristics for high P contents in vegetation.

For Si, there are several samples with larger Si content for which the model overestimates Si contents, besides the four samples with maximum Si content for which the model underestimates Si contents. The overestimated samples have a large Ca content and the underestimated do not (Fig. S16). Presence of carbonates therefore biases predictions of Si contents, even though this bias seems to be small for the training data. The one observation marked as outlier in Fig.  may be a measurement error of Si contents or erroneously assigned spectrum to this sample because the sample does not have typical characteristics for silicate rich peat, such as a pronounced peak around 1100 cm⁻¹ and peaks around 1900 cm⁻¹ (Fig. S20) .

For S, larger Ca contents do not bias predictions, but they increase the residual variance indicating, similarly to Si, that peaks caused by carbonates confound predictions (Figs. , S16). One reason for the difficulty to predict S contents here is that samples in the data with large S contents also contain large Ca contents, but only some samples with large Ca content have carbonate peaks. Samples with large carbonate peaks probably are influenced by groundwater with relative high pH value under which calcite can precipitate with sufficiently high Ca²⁺ concentrations. In contrast, samples with large Ca contents, but without carbonate peaks probably have Ca²⁺ bound to carboxyl groups, which leads to a small carboxyl peak (around 1730 cm⁻¹) and a more pronounced peak around 1640 cm⁻¹ which has contributions by carboxylates (Fig. S19).

It is interesting that prediction of other variables (e.g., C, N, K, P, Ti, bulk density) is possible without such bias. Since the majority of samples with large Ca contents (ca. >15000 µg g⁻¹) are from cores from one permafrost peatland site, this may be due to overfitting, indicating that more peat MIRS from carbonate rich samples need to be published to improve development and testing of transmission-MIR prediction models for carbonate-rich peat.

Ti is the only target variable for which the training RMSE is significantly larger than the testing RMSE (Table ). This indicates that Ti contents were more variable in the training data than in the testing data. Since Ti is unlikely to cause detectable peaks in peat MIRS, prediction of Ti concentrations probably relies mainly on a similar atmospheric deposition across the analyzed peatlands and residual enrichment as peat is lost due to decomposition or fires. That even small Ti concentrations can be estimated from MIRS therefore supports application of Ti concentrations as decomposition indicator, even though the model also suggests that there are other sources of variation. In fact, the most useful application of the models may not be the accurate prediction of Ti contents, but the detection of conditions where Ti concentrations are controlled by other factors than residual enrichment from decomposition. Such conditions can be detected by comparing predictions of the model to Ti measurements.

The models for δ13C and δ15N have prediction errors much too large for most current applications of such isotope measurements. C and N isotope values are controlled by many different processes that can easily lead to a large variation in δ13C and δ15N values despite similar spectral properties. For δ13C, these factors are differences in δ13C signatures of assimilated CO₂ due to isotope fractionation , the Suess effect , and differences in the fraction of CO₂ assimilated from methanotrophy . For δ15N, these factors are different pathways via which N is assimilated by peat forming vegetation and the opposite effects of isotope fractionation and enrichment of OM fractions with negative δ13C values during aerobic decomposition , which agrees with weak correlations of δ15N values with peat decompostion indicators . It may therefore be the case that δ13C and δ15N measurements cannot be estimated accurately with MIRS prediction models. However, many of the δ13C and δ15N measurements in the pmird database do not correct for blank effects , which biases δ13C and δ15N values proportionally to the C and N mass . According to blank corrections for other projects, this bias has a magnitude of ca. 0.2 ‰ to 0.5 ‰ (depending on the sample C mass) for δ13C and a magnitude of ca. 0.2 ‰ to 1 ‰ (depending on the sample N mass) for δ15N. This is only a fraction of the estimated RMSE (Table ) and we therefore currently assume that unbiased measurements would still result in large prediction errors relative to measurements of δ13C and δ15N. For δ15N, the contribution of this bias is certainly larger than for δ13C. A factor that contributes to the large prediction errors therefore are biased measurements due to uncorrected blank effects.

For C/N, prediction errors are larger for samples with larger C/N. Two factors probably contribute to this pattern: Firstly, large C/N values imply small N contents and at large C/N values, very small changes in N contents cause large changes in C/N values. Such slight changes in N contents probably cause only small changes in peak intensities that are not easy to detect in MIRS and hence not easy to predict. Secondly, some of the samples with large C/N values have large silicate contents (Fig. S18). The large differences in spectra between undecomposed Sphagnum peat and mineral-rich peat very likely confounds linear relations present in peat without large mineral contents.

To summarize, for many variables, accurate predictions are possible with the models developed here, whereby the accuracy needed will of course depend on the specific purpose of the analysis. The models for δ13C and δ15N are probably not accurate enough for any analysis of isotope values, even if blank effects are corrected. Besides silicates and nitrogen, calcium – either in the form of carbonates or in the form of Ca²⁺ bound to carboxylates – is an important contributor to spectral variability in peat samples and makes it difficult to predict S, Si, and Ca contents, at least with the modeling approach used here. Since the pmird database contains Ca-rich samples only from few sites, future tests of the models with additional Ca-rich peat samples would be useful.

A comparison of training and testing prediction domains for our models shows that the testing prediction domains covers only a small fraction of the training prediction domain for H, O, NOSC, ΔGf0, H/C, O/C, C/N, and LOI (Fig. ). For these peat properties, only few samples were left for model testing and we could therefore not test the predictive accuracy of the models for some parts of the training prediction domain. Future studies should test these blind spots. The difference between testing and training prediction domain can be used to identify samples useful to test the models in the future. Similarly, the difference between the training prediction domain and the domain formed from all spectra identifies samples that would be useful additions to the training data if the target variable would be measured for these samples. For example, for H, O, NOSC, ΔGf0, H/C, O/C, and C/N there is a lack of mineral-rich samples in both the training and testing data, whereas for LOI there are mineral-rich samples in the training data, but not enough mineral-rich samples were left for model testing (Fig. ). The prediction domains are available from the irpeatmodels package.

Table  summarizes the results of our gap filling. Even though we filled all gaps with MIRS predictions, predictions that are outside the training and testing prediction domain may be unreliable and therefore we consider all observations for which the spectra is neither inside the training nor the testing prediction domain as unfilled gaps for our evaluation here.

Since the pmird database contains many bulk density and N measurements, but no measurements for porosity, hydraulic conductivity, specific heat capacity, and thermal conductivity, many missing values could be estimated with the pedotransfer functions and the remaining gaps filled with MIRS-predicted bulk density and N. Similarly, a large fraction of fillable gaps for element contents could be imputed, except for H and O for which training and testing prediction domains cover a smaller fraction of the spectral variability. Much fewer gaps could be filled for O, NOSC and ΔGf0 than for H because of differences in the preprocessing for these models: the models for H uses underived spectra, whereas the models for O, NOSC and ΔGf0 use first or second derivative spectra, where high frequency features are more emphasized and therefore many spectra are not within the prediction domains (Table ). For observations, where spectra are outside the prediction domain only because a few of the variables exceed the prediction domain boundaries by small values are still reliable; the amount of useful predictions is therefore probably underestimated. Compared with existing databases , this makes the gap-filled pmird database one of the largest available data sources for contents of many elements, hydraulic and thermal properties, and peat chemistry, in particular NOSC, and ΔGf0. The gap filling predictions are available from Zenodo .

The models can be used with R by installing the irpeat and irpeatmodels packages. The irpeatmodels package contains the models itself and irpeat contains functions to interact with the models. This design was chosen to account for faster development cycles for code to interact with the models and code for other functions of the irpeat package and also to account for size limitations for software packages in online repositories.