Soil samples were collected from past projects in the Congo Basin and along the Albertine Rift, the western branch of the East African Rift System. Table  gives an overview of corresponding data sources and data contributors to the different sample sets and denotes the origin, the number of samples, and sampling layers used for our CSSL. The sample locations of the entire library are clustered over a large geographical area of central Africa, from a latitude of 2.8 to -11.6∘ and a longitude of 12.9 to 30.4∘. From our entire library, six clustered regions were identified, which contained at least 80 samples to allow for reliable analysis. Therefore, this subset will be further presented in the paper (see Tables  and in the Appendix for information on the entire library). Four of the selected regions are located in the Democratic Republic of the Congo (Haut-Katanga, South Kivu, Tshopo, Tshuapa), while the other two are located in Rwanda (Iburengerazuba) and in Uganda (Kabarole), respectively (Figs.  and ). Site-specific characteristics, coordinate ranges, altitudes, average climate, and dominant soil types are summarized in Table . Annual precipitation ranges from about 1200 mm in Haut-Katanga to over 2000 mm in the tropical forest of Tshuapa. Mean annual temperature varies from 17.6 ∘C in the high altitudes of Iburengerazuba and South Kivu to 24.9 ∘C in Tshopo . The study elevations range from 380 ma.s.l. in Tshuapa and Tshopo to high altitudes of 2300 ma.s.l. in South Kivu along the rift valley . Soil types are primarily Ferralsols, Acrisols, or Nitisols . The different regions contain multiple Köppen–Geiger climatic zones: The three regions located close to the Equator (Tshuapa, Tshopo, Kabarole) are classified as Af (tropical rainforest), while the eastern DRC and western Rwanda are classified as a mixture of climate zones Cfb (temperate, without dry season, warm summer), Csb (temperate, dry summer, warm summer), Aw (tropical savannah), and Cwb (temperate, dry winter, warm summer). The regions along the rift valley (South Kivu, Iburengerazuba, Kabarole) are partially also classified as Am (tropical monsoon). Finally, the southeast of the DRC is classified as Cwa (temperate, dry winter, hot summer) .

In preparation for total carbon (TC) and total nitrogen (TN) analyses, all soil samples were sieved through a 2 mm mesh and either air-dried or oven-dried at temperatures of 50 or 60 ∘C. After sieving and drying, soil samples were ground to a powder (<50 µm) using a ball mill. TC and TN were analyzed via dry combustion using either a LECO 628 elemental analyzer (LECO Corporation, USA), an ANCA-SL automated nitrogen carbon analyzer (SerCon, UK), or a vario EL cube CNS elemental analyzer (Elementar, Germany). In order to ensure data quality and facilitate the harmonization of all TC and TN data, a subset of these samples was remeasured on the LECO. This performance comparison demonstrated high comparability of TC and TN data across all three instruments (R2=0.99 for TC and TN, results not shown). The large majority of the soil samples originate from highly weathered and acidic soils and do not contain any carbonates, and therefore, TC contents correspond to total organic carbon contents. Only in a few samples from termite mounds in the subtropical Haut-Katanga province has calcium carbonate been detected where pH values are >8 . Moreover, the widely used slash-and-burn practices could additionally have influenced soil TC contents, even when visible charcoal pieces were removed prior to any measurement. Additionally, soil pH (either in H2O, KCl or CaCl2, depending on the study), texture (laser diffraction particle size analyzer), and aqua-regia-extractable macro-/micronutrients (Al, Fe, Ca, Mg, Mn, Na, P, and K; inductively coupled plasma-optical emission spectroscopy) were analyzed for a subset of samples. The chemical and MIR prediction results for these soil characteristics are not presented in this paper but were carried out using the same methods and are available on our GitHub repository at https://doi.org/10.5281/zenodo.4351254 (last access: 20 December 2020).

All CSSL and AfSIS spectra were processed using the R packages “prospectr” , “simplerspec” , and “resemble” in the R statistical computing environment . Replicates of spectral measurements were aggregated to one average spectrum per sample. The spectra were then resampled to a resolution of 16 cm-1 and trimmed to a spectral range of 4000–600 cm-1. Both spectral libraries were scanned on two FT-IR Bruker spectrometers (Bruker Optics GmbH, Germany), which use the same settings and the same internal standards. The scanning methods of the CSSL were adapted to the standard operating procedures of the Soil Plant Spectral Diagnostics Laboratory at ICRAF. For these reasons, no instrument standardization was necessary.

As spectral pretreatments have a marked impact on the performance of quantitative infrared models , the preprocessing procedure was specifically optimized for the MIR spectra of the central African samples. This procedure was based on the PLS method , which is also known as projection to latent structures and is widely used for regression analysis in infrared spectroscopy. However, it is also useful for projecting the spectral data onto a low-dimensional (and therefore less complex) subspace containing all the meaningful information of the original data. The projection model can be expressed as 1X=SP′+E, where X is the original spectral matrix of n×d dimensions, S is the PLS score matrix of n×l dimensions (where l≤min⁡(n,d)) which contains the extracted variables, and P is the matrix of loadings of d×l dimensions which captures the spectral variability across observations. E is an error term. For spectral data with high collinearity, the optimal l (or the number of PLS factors) is usually small, which means that only a few PLS factors or latent variables are enough to properly represent the original variability of X. An important aspect of this type of projection is that it is obtained in such a way that the covariance between S and an external set of one or more variables is maximized. For a detailed description on PLS we refer the reader to . In PLS, P can be used on new spectral observations to project them onto the lower dimensional subspace: 2Snew=XnewP-1.

The spectral reconstruction residuals of the projection model can then be computed by back-transforming the matrix of scores to a spectral matrix and comparing it against the original spectral matrix as follows: 3Enew=Xnew-SnewP′.

Finally, the spectral reconstruction error (also known as the Q statistic) is computed as the sum of squares of Enew: 4Qnew=EnewE′new.

The Q statistic indicates how well a given new sample is represented by the PLS model . This statistic is widely used in chemometrics for outlier identification and uncertainty assessment .

In summary, our approach offers a data-driven solution to the selection of the spectral preprocessing steps which are optimized for the target/prediction set. The optimal set of steps is defined as the one that minimizes the Q statistic. This approach does not require prior knowledge of the response values of the target set and therefore is well suited for preprocessing optimization. It assumes that PLS models that cannot account for the spectral variability in the target set may also fail at producing accurate predictions of the response variable. In other words, as suggested by , large Q values can be used as proxies for large prediction errors, and therefore Q values can be used to judge the suitability of a set of preprocessing steps. To find an optimal combination of spectral pretreatments, we defined a set of different pretreatments {h1,h2,…,hz}, where hi represents one pretreatment or a sequence of pretreatments (with unique parameter values) to be applied on the spectral data. For this purpose, a projection model was built with the AfSIS spectra (using TC and TN as external variables) for each combination of spectral pretreatments: 5hi(Xa)=Sa(i)Pa′(i). This model was then used on the CSSL pretreated spectra with reconstruction residuals (Ec) computed as follows: 6Ec=hi(Xc)-[hi(Xc)Pa-1ScPa′(i)], where Pa denotes the loadings corresponding to the PLS model built with the AfSIS library and Sc the projected scores of the central African Library.

For this analysis we fixed the number of PLS factors to 20 because projected variables beyond this dimension did not capture a sufficient amount of the original spectral variance. For example, PLS variable 21 amounted to less than 0.01 % of the original variance in all the cases. The mean Q value (Q‾) for the ith set of pretreatments was obtained by 7Q‾(i)=1md∑j=1mEcjEcj′, where m and d are the number of samples and the number of spectral variables in the CSSL respectively. To allow for comparisons across the reconstruction errors obtained for the different pretreatments, Q‾ was standardized as follows: 8sQ‾(i)=Q‾(i)max(hi(Xc))-min(hi(Xc)).

Tested pretreatments included different combinations of standard normal variate, multiplicative scatter correction, spectral detrending, first and second derivatives, and window sizes from 3 to 35 points in increments of 2. Minimal spectral reconstruction error was achieved with a Savitzky–Golay filter with a second-order derivative using a second-order polynomial approximation with a window size of 17 cm-1 , and a subsequent multiplicative scatter correction. This pretreatment was then applied to the spectra prior to MBL.

To analyze the difference between the two spectral libraries and to visualize the similarities between soil samples, a principal component analysis (PCA) was conducted on the preprocessed spectra of both libraries. The PCA was performed with centering but without scaling of the absorbance values.

In the following we describe the method we used to assess the performance of MBL for predicting TC and TN for six distinct regions at different scenarios of regional soil extrapolation. Three specific modeling strategies were tested on the selected regional sets which we call validation sets (see Sect. ). With the regional analysis we demonstrate how predictions of soil properties within new sites from distinct regions – which are compositionally less variable than the available SSLs – perform and profit from knowledge present in the AfSIS SSL. The analysis also demonstrates the added value of our new CSSL in addition to the AfSIS SSL alone. Doing so, the aims of the modeling scenarios were (1) to minimize the costs and time for traditional methods by optimizing the transfer of stored spectral information to the new region of interest and (2) to test different levels of geographical extrapolations for new regions, when no chemical analyses of local samples are available.

We used MBL as our predictive modeling approach. In the chemometrics literature, MBL is also known as local modeling, which describes a family of (nonlinear) machine learning methods designed to handle complex spectral data sets . This type of learning method does not attempt to fit a general (global) predictive function using all available data. Instead, a new and unique function (f^i) is built on demand, every time a new prediction for a given response variable is required. This new function is built using only a subset of relevant observations from a reference set that are queried through a k-nearest neighbor search. The MBL method implemented for this study uses a spectral nearest neighbor search based on a moving window correlation dissimilarity. To measure the dissimilarity (r) between two spectra (Xi and Xj), the following equation was used: 11r(Xi,Xj;w)=12w∑k=1d-w1-ρ(Xi,{k:k+w},Xj,{k:k+w}), where d is the number of spectral variables, ρ represents the Pearson's correlation function, and w is the size of a moving window. This window size was optimized based on a spectral nearest-neighbor search within the AfSIS library. For every sample in the AfSIS library, its closest sample (in the spectral space) was identified. Then, samples were compared against their closest neighbors in terms of TC and TN and root mean squared differences (RMSDs), computed according to the following equations: 12j(i)=NN(xai,Xa-i)

13RMSD=12m∑i=1n∑h=12(yai,h-yaj(i),h)2, where NN(xai,Xa-i) represents a function to obtain the index of the nearest neighbor of the ith observation found in Xa (excluding the ith observation), and yci,h is the value of the ith observation for the hth property variable (either TC or TN). In total 10 window sizes were evaluated using this approach (from 31 up to 121 in steps of 10), and according to the RMSD, an optimal window size w of 71 was chosen.

After nearest neighbor retrieval, our MBL method fits a local model using the weighted average partial least squares (WA-PLS) regression algorithm proposed by . In this WA-PLS, the final prediction is a weighted average of multiple predictions generated by PLS models built from different PLS factors. A range of latent variables from 5 to 30 in increments of 1 was used for the WA-PLS calculations. The weight for each component is calculated as follows: 14wj=1s1:j×gj, where s1:j is the root mean square of the spectral residuals of the new observation when a total of j PLS components are used (i.e., all the components from the first one to the jth one), and gj is the root mean square of the regression coefficients corresponding to the jth PLS component (see , for more details).

The number of neighbors that needed to be retrieved was optimized using nearest neighbor (NN) cross-validation . Using this method, for each observation to be predicted, its nearest neighbor was excluded from the group of neighbors, and then a WA-PLS model is fitted using the remaining ones. This model is then used to predict the value of the response variable of the nearest observation. Predicted values are finally cross-validated with the actual values (see , for additional details). For the optimization of the nearest neighbor search, i.e., the nearest neighbor cross-validation, a grouping factor was used to avoid overfitting: keeping the nearest neighbor out, the model was trained with the remaining neighbors which were not from the same region as the hold-out neighbor (region corresponds to the sentinel sites within the AfSIS SSL). The minimum number of available neighbors was tested for each region prior to training the respective final models, which were then trained with neighborhood sizes varying from 150 to 500 neighbors in increments of 10. The best model and the optimal number of neighbors were determined by the minimal RMSE (Eq. ) of the nearest neighbor cross-validation, where n is the number of neighbors used for the model, yi is the measured value of the hold-out neighbor, and y^i is the value predicted by the remaining neighbors.

Subsequently, independent from their distances to the validation set, 1 to 20 spiking samples were added from the target region and forced into the neighborhood of every observation and thus used in the predictive models. Our approach differs from previous studies using local modeling methods in combination with spiking, where the samples were not forced into the neighborhoods (e.g. ). Our approach guarantees that the spiking set (which is assumed to carry important information) is fully used.

Stepwise spiking was applied to test the effect of spiking in general and to find the smallest number of samples required for satisfying model performances. This was necessary since soil samples from the same geographical region are usually governed by very similar formation processes (spatial autocorrelation; ), and MIR spectra partially reflect the compositional characteristics of these samples. Moreover, it is widely accepted that the most accurate predictions can be achieved by models built with samples originating from the same region because large nonlinear complexity is avoided e.g.,.

For model validation, the RMSE statistics of the nearest neighbor cross-validation described in the previous section were used. Prediction accuracy of the predicted vs. the measured values was also calculated using RMSE (Eq. ), where in this case yi is the actual measured reference value and y^i the prediction of the final model. 15RMSE=1n∑i=1n(yi-y^i)2

Model validation and prediction performance were additionally evaluated using the mean error (ME; mean of the absolute difference between predicted and observed values) and the ratio of performance to the interquartile distance (RPIQ; ). For calculating RPIQ, the interquartile range of the observed reference data is divided by the RMSE of the nearest neighbor validation or by the RMSE of the prediction (RMSEpred). This is particularly useful since RPIQ does not make any assumptions about the distribution of the reference data.

The first three principal components accounted for 85 % of the spectral variability (Fig. ). These components indicate that the majority of CSSL samples lie within the spectral domains of the AfSIS SSL as their PCA scores overlap. This overlapping is, however, less evident for the spectra of the South Kivu region and, to a lesser extent, for the samples of the Iburengerazuba and Tshuapa regions, which suggests that the type of soils in these regions may not be well represented by the AfSIS SSL compared to the other regions.

In general, MBL retrieved accurate TC and TN predictions for all the strategies (with RMSEpred values below 9 gkg-1 for TC and below 1.7 gkg-1 for TN). South Kivu and Iburengerazuba regions showed the highest RMSEpred, which was mainly due to the high TC and TN ranges (Fig. ). Prediction errors for Haut-Katanga, Tshopo, and Tshuapa were comparably smaller; however, the RPIQpred values were among the smallest across regions as well (RPIQpred 0.59–2.72). Relative to their TC and TN ranges, predictions for these three regions were less accurate than for South Kivu and Iburengerazuba (PRIQpred 1.10–4.45). The TC and TN predictions in Kabarole were the most accurate compared with the other five regions (RPIQpred 2.65–5.48 for TC and TN and all strategies; Table ).

Our analysis shows that TC and TN in six regions of our CSSL can be reasonably well predicted through the use of existing SSLs comprised of soils from completely different geographical areas and without any local samples using MBL methods (RMSEpred<9 gkg-1 TC and <0.17 gkg-1 TN, Table ). The resulting prediction errors were comparable to other large-scale MIR prediction studies e.g., and also to other soil infrared studies, which analyzed geographical extrapolation possibilities e.g.,. The advantage of using MBL as the method to build prediction models is that it finds similar spectral observations for every new observation to fit suitable models. This approach works efficiently since spectral similarity is in fact reflecting the similarity between observations in terms of soil composition, information which is largely contained in the MIR features of a sample. This means that the predictive success of MBL models largely depends on the quality of the spectra dissimilarity methods used to find spectral neighbors. In other words, MBL can be described as a method driven by compositional similarity search. The improved prediction accuracy (lower RMSEpred and higher RPIQpred) when reducing extrapolation (strategy 2) can be explained by the addition of more proximal central African soil samples to the library that are more similar to each predicted region. The continental AfSIS SSL is missing data for most of central Africa (Fig. ); none of the tropical forest soils with high contents of organic carbon or with distinctive mineral–organic composition are covered by this large-scale SSL. Naturally, this variability impacts the generalization ability of any predictive model or modeling strategy. Moreover, variance arising from instrument and reference laboratory differences was avoided through the use of local models. However, it is not clear why Kabarole exhibited higher prediction errors in strategy 2. A possible reason could be random variance (Fig. ) or nonlinearity. Two regions (South Kivu and Tshuapa) did not show any substantial changes on RMSEpred and RPIQpred values for TC when comparing strategy 1 and strategy 2. Note that both South Kivu and to some extent also Tshuapa cover a distinct score space in Fig.  and therefore are not well represented by the remaining central African regions, nor by the AfSIS SSL.

All central African regions from the CSSL show large variability in TC and TN contents (Fig. ) and contain samples from various land cover (forest/croplands), altitudes (Table ), and parent materials. These differences suggest that soils have developed and been transformed under a variety of environmental conditions. For example, high diversity in organic compounds and their stabilization in soils (i.e., organo–mineral association, complexation, aggregation) can introduce nonlinear relationships that are difficult to predict with locally linear calibration methods (i.e., memory-based learning in combination with PLS regression). Thus, we conclude that the particularly high soil diversity in these two regions, in terms of biogeochemical and physical properties, introduces additional complexity into the soil spectral prediction workflow. Similarly high RMSEs have been shown in other studies for samples with organic carbon higher than 150 gkg-1 . As in our study, these high errors were attributed to high TC contents. To improve prediction accuracies for these diverse regions, more data are needed. The creation of subsets from large spectral libraries via spectral similarities, for example, has been shown to be effective to train calibration models e.g.,. Hence, in order to reduce uncertainties for regions in central Africa that are diverse in terms of soil chemical composition, in particular for the Great Lakes region, there is a pressing need to fill the existing gaps in the continental library by gathering more data on the ground.

The spiking of the calibration models with local target samples had a positive effect for all included regions (Fig.  and Table ). Kabarole, Iburengerazuba, and South Kivu, which showed the most substantial reductions of RMSEpred for TC and TN by spiking, cover different land uses, high altitudes along the Albertine Rift, and larger climatic ranges (Table ). These soils are not adequately represented by the continental AfSIS SSL, nor by the remaining central African regions, and therefore exhibited a strong effect when spiked with local soil data. Although the effect of spiking on RMSEpred for TC and TN was somewhat smaller for the other included regions (Haut-Katanga, Tshopo, and Tshuapa), it still produced noticeable improvements compared to strategy 1 and strategy 2 (smaller RMSEpred and larger RPIQpred values). The TC and TN ranges of Haut-Katanga, Tshopo, and Tshuapa were narrower, and they also seem to be better represented by each other and by the AfSIS SSL (with the exception of a few samples of Tshuapa; Fig. ). In these three regions, sufficiently similar spectra were available, and the MBL found the required neighbors to build accurate models and predict TC and TN, thus lowering the positive effect of spiking. Additionally, the weaker influence of spiking on soils of Tshopo (RPIQpred TC: 1.43 and RPIQpred TN: 1.62) can be explained by an outlier in the predictions (Fig. ) and a slightly uneven distribution of the reference data between the validation and spiking sets (Fig. ). In summary, spiking has already been shown to improve performance e.g., and also proved its value in our study. However, a threshold of 20 samples poses non-negligible additional costs for laboratory reference analysis, and the benefit in terms of gain of accuracy by spiking depends on the region and is not always guaranteed. In some cases, however, a smaller number of spiking samples can substantially reduce the RMSEpred (e.g. Iburengerazuba and Kabarole). The required prediction accuracy and additional investments depend hereby on the field of application. The achieved predictions and their errors from this study are more than satisfactory for the study of TC and TN dynamics and will improve the availability of high-resolution soil data of central Africa. Thus, spiking is recommended when soils are highly variable and show large distances to existing spectral libraries.

Our regional predictions of TC and TN show promising results when analyzing soils from geographically distinct areas in central Africa that are not covered by the continental AfSIS SSL (Fig. ). Six central African regions were predicted for soil TC and TN with sufficient accuracy using the large-scale AfSIS soil spectral library only. The general positive effect of adding geographically closer samples to the AfSIS SSL (strategy 2) underlines the usability of spectral libraries for new regions. The generally positive effect of strategy 3, spiking of all regional predictions for TC and TN with samples from the target area, encourages the future amendment of currently existing libraries to improve prediction accuracy. To improve future soil analyses and to extend the geographical area covered by an SSL, we suggest the following workflow:

Preprocessing. Different spectral preprocessing methods influence model and prediction performance. We suggest selecting the best preprocessing strategies using spectral projections and minimizing the reconstruction error (see Sect. ).