Effective accounting of changes in soil organic C stocks requires the measurement of the stocks and their uncertainty for a defined baseline and over the monitoring period. Internationally, the default method to determine soil organic C stock (Cs) for the accounting of change is to multiply measurements of soil organic C concentration, bulk density, and gravel content at a fixed depth of 0–30 cm and to report the stock as a mass of carbon per unit area in tonnes of organic C per hectare : Cs=Cm×ρ×1-g100×d, where Cm is the mass of soil organic C in the soil (%), ρ is the soil bulk density (g cm-3), g is the gravel content (%), and d is the thickness of the layer (cm). Our definition of soil organic C used here extends that of the IPCC Guidelines , which address the measurement of soil organic C in mineral soil in the 0–30 cm layer, by extending to deeper soil layers. Based on typical rooting depths found in agricultural crops and pastures and the capacity of deeper soil horizons to sequester relatively large amounts of soil organic C, there is evidence to suggest that measurements should extend to deeper layers e.g..

Conventionally, the measurement of soil organic C stocks involves soil sampling (see Sect. 3.2), followed by sample preparation and laboratory analysis. For the analytical determination of soil C concentration, sample preparation typically entails drying, crushing, grinding, sieving, sub-sampling, quantification of the sample's water content, and further fine grinding for dry-combustion analysis e.g.. Conventional measurements of soil bulk density typically involve using the volumetric ring method, where a pit is dug and a metal core of known volume is driven into the soil at the fixed depth. The bulk density is then determined by dividing the oven-dry soil mass of the sample by the volume of the core . Alternatively, the clod method e.g. has been used for soil with abundant rock fragments, where clods of soil are sampled and sealed (e.g. with paraffin) and the volume of the sample is determined by its displacement of water in a vessel. Gravel content is conventionally measured by breaking the soil cores into specific depth intervals, drying them in an oven, crushing the soil with a mortar and pestle, and then sieving it to separate the fine-earth (≤ 2 mm) fraction from the gravel.

Equation () can be used to quantify and report the change in soil organic C stocks at fixed depth intervals. However, this method can systematically overestimate or underestimate C stocks if bulk densities increase or decrease, respectively, from changes in land use or land management practices (e.g. changes in cultivation). Where bulk densities differ between management practices or across time periods, more accurate estimates of the C stock and its change can be derived using measures of cumulative or equivalent soil masses per unit area . Various studies have recognized the importance of this approach, which also reduces the effect of depth of sampling errors . Both measurement-based methods under the Australian ERF Table 1; use an equivalent soil mass (ESM) approach to quantify soil organic C stock change.

Before measuring the soil organic C stocks (Eq. ), a sampling design must be derived to determine the sampling locations. Methods to select sampling locations include probability sampling and non-probability sampling, which result in two widely used sampling philosophies: design- and model-based sampling . In design-based sampling, the randomness of an observation originates from the random selection of sampling sites, whereas in model-based sampling, randomness comes from a random term in the model of the spatial variation, which is added to the model because our knowledge of the spatial variation is imperfect. Probability sampling is, therefore, a requirement for design-based sampling but not for model-based.

Choosing which approach to use depends mostly on purpose . For instance, if one needs estimates of the mean or total soil organic C stock and their accuracy over a given area, whose quality is not dependent on the correctness of modelling assumptions, then design-based sampling might be most suitable. If the aim is to produce a map of the soil organic C stock over the area, then model-based sampling will be preferable. However, because design-based sampling can also be used for mapping and model-based sampling for the estimation of means or total C stocks, the choice of which approach to use can be difficult. demonstrated the use of probability sampling, which allowed design-based, model-assisted, and model-based estimation of the total soil organic C stock across 2837 ha of grazing land in Australia. Spectroscopic and active gamma attenuation sensors were used for estimating soil organic C stocks and their accuracy in the 0–10, 0–30, and 0–100 cm layers and for mapping the stocks in each layer across the study area. Although the design-based, model-assisted, and model-based estimates of the total soil organic C stocks were similar, the variances of the model-based estimates were shown to be smaller than those of the design-based methods. The authors noted that the advantage of the design-based and model-assisted methods, unlike the model-based approach, was that their estimates of the baseline soil organic C stocks and their variances did not rely on the assumptions of a model. Further, they noted that although the model-based approach produced the smallest variance of the predicted total soil organic C stocks, the results cannot be generalized to other sample sizes and types of sampling designs. We suggest that whatever the method used, careful consideration of the sampling design is needed for the estimation of the baseline soil organic C stocks and for monitoring. Further discussion on the advantages and disadvantages of the sampling approaches can be found in .

Gravels are defined as coarse fragments with particles that are coarser than 2 mm . The presence of gravel has a significant effect on the mechanical and hydraulic properties of soil . If gravel is present but not accounted for, it could bias the measurements of soil organic C stocks . For example, the presence of abundant coarse fragments (> 20 %) adversely affected the measurement of soil bulk density by both conventional and AGA using backscatter methods . Sensing of gravel is difficult, and so it is typically measured manually by drying, crushing, sieving, and weighing of the soil and gravel. This method is time-consuming.

Based on our review and assessment of the available sensor technologies above, currently, the most suitable proximal sensing techniques for measuring soil organic C and for monitoring its change are vis–NIR and mid-IR spectroscopy for estimating soil organic C concentration and AGA for measuring bulk density. There are no practical or efficient sensors available for measuring gravel. Presently, a possible best option might be wet sieving to separate the gravel fraction and quantification by weighing or image analysis . Table  provides a summary of the benefits and limitations of each of these technologies, and Table  assesses their cost and accuracy.

The cost of spectrometers can vary widely (Table ). Portable vis–NIR spectrometers (350–2500 nm) can be purchased from different manufacturers from approximately less than AUD 10 000 to more than AUD 100 000 (Table ), although their cost is continually decreasing as technologies develop. Smaller, cheaper vis–NIR spectrometers that use micro-electromechanical systems (MEMSs) technologies are emerging and are less expensive, but not many have been thoroughly tested. Prices for mid-IR portable spectrometers (2500–20 000 nm) are approximately AUD 50 000–70 000, and for mid-IR benchtop spectrometers, they are approximately AUD 25 000–90 000, depending on the size, type of detector, sensitivity, and amount of automation. The cost of spectroscopic measurements of soil organic C concentration in the laboratory is larger than measurements under field conditions because of the need for sieving, drying, and grinding. mid-IR measurements are expensive because samples need to be finely ground. Both vis–NIR and mid-IR techniques can accurately predict the soil organic C content of dry soils (Table ). The accuracy of vis–NIR predictions on wet soil, that is under field conditions, is generally less than that for dry soil, although the difference can be relatively small (Table ).

AGA sensors for measuring bulk density are also quite readily available (Table ). Measurement costs are significantly smaller for AGA using transmission than for AGA using backscatter because of the additional soil preparation required for the latter. There are few reports on the use of AGA for the measurement of bulk density, but they report good accuracy for AGA using transmission and variable accuracy for AGA using backscatter. There is no information on the cost and accuracy of wet sieving and image analysis for quantifying gravel (Table ). suggested that a system could be easily developed and that measurement costs would be small.

We need an integrated multi-sensor approach for measuring and monitoring soil organic C stocks because simultaneous measurements of soil organic C concentration, bulk density, and gravel are needed. There are two currently available, field-deployable, proximal multi-sensor systems to measure soil organic C stocks. One involves inserting the sensors into the soil profile and making measurements in situ, while the other requires sampling undisturbed soil cores and measuring the soil ex situ.

Veris^® Technologies produces commercial sensors for precision agriculture (http://www.veristech.com, last access: 6 December 2017), including several field-deployable systems which can measure electrical conductivity, pH, and penetration resistance and also record the vis–NIR spectra of soil. The system that is particularly relevant to soil organic C accounting is the P4000, which uses a hydraulic probe system to insert four sensors into the soil to characterize the profile. The sensors are a vis–NIR spectrometer (350–2200 nm), an electrical conductivity (EC) sensor, and an insertion force sensor. The system does not measure bulk density. Using insertion force as a surrogate for bulk density might be possible but would be prone to errors. tested the accuracy of predictions of soil organic matter (SOM) using the P4000 system and evaluated whether the predictions were improved when the sensors were combined. They found that the accuracy of predictions of soil organic matter content with the vis–NIR alone was good, but the inclusion of insertion force only improved prediction accuracy by about 10 %. They concluded that these small improvements did not provide strong support for combining vis–NIR sensor measurements with measurements of insertion force. However, there was no testing of bulk density.

The Soil Condition Analysis System (SCANS) uses a combination of proximal sensing technologies, smart engineering, and data analytics to characterize soil laterally across the landscape and vertically down the profile. The SCANS has an automated soil core sensing system, which can be used in the laboratory or the field. The system has a vis–NIR (350–2500 nm) spectrometer, an AGA densitometer, and digital cameras that measure intact soil core samples that are either (wet) under field condition or dry, at user-defined intervals over the length of 1.2 m soil cores. The system can measure soil organic C content and composition (particulate, humus, and resistant C), bulk density, clay content, cation exchange capacity, volumetric water content, available water capacity, pH, iron, and clay mineralogy . Each measurement with the sensors takes approximately 35 s so that measuring a 1 m core at 2 cm intervals (i.e. 50 measurements along the core) takes about 30 min. showed that the sensing system could be used to accurately baseline soil C stocks for accounting purposes, and assessed its cost-efficiency and reliability for soil C accounting. They found that compared to more conventional methods that use composite sampling and laboratory analysis, sensing with the SCANS is more cost-efficient in that it provides a good balance between accuracy and cost.

The measurement of soil organic C using spectroscopy (e.g. vis–NIR, mid-IR) requires the calibration of the spectra to soil organic C content using multivariate statistics or machine-learning algorithms. The calibrations can be derived using existing large spectral libraries (ESLs) e.g. or using new site-specific libraries developed with local soil samples (LSLs). Using an ESL to predict soil organic C incurs no immediate cost, but it is likely that the predictions at the local site (farm or field scales) will be biased . Using an LSL will produce more accurate (unbiased) predictions but will incur a cost because soil needs to be analysed in the laboratory to derive the local model.

Significant investment has been made in developing large regional, country, and global spectral libraries , and there will be value in using these for developing site-specific calibrations. These ESLs could reduce the need for site-specific data. Various approaches have been proposed to make better use of ESLs for local predictions of soil properties. They are based on either constraining the ESL with spectral or soil sample similarities or augmenting the ESL with site-specific samples.

Memory-based learning (MBL) methods aim to constrain the ESL with spectral information and derive calibrations for each unknown sample on a case-by-case basis. By selecting a subset of the ESL to predict each unknown sample, these methods effectively derive site-specific (i.e. local) calibrations. Methods include the LOCAL and locally weighted regression (LWR) algorithms and their variants. Essentially, the methods select calibration samples from the ESL with a distance metric (e.g. Mahalanobis distance) in the multivariate space between the calibration and the unknown samples. In LWR weighting the calibration samples are also weighted according to their spectral dissimilarity (distance) to the unknown samples.

proposed spectrum-based learning (SBL), which is a type of MBL. The spectrum-based learner selects nearest neighbours from an ESL using distance metrics calculated in the principal component space and optimizing the number of components used to identify the nearest neighbours in the selection. Spectroscopic modelling is then carried out with both the selected neighbours and the matrix of distances to the unknown samples as the training data set.

The ESL can also be constrained with other information such as soil order, type, texture, and parent material e.g.. proposed the use of both spectral similarities and geographically constrained local calibrations to predict soil organic C content. They reported improvements in the accuracy of predictions when the ESL was restricted to the geographic region from which the unknown samples originated. developed general ESL calibrations for Australian soil using the machine learning algorithm Cubist. The authors showed that the algorithm makes inherently local predictions because Cubist partitions the spectra into local subsets that are each modelled separately.

Two techniques use the augmenting approach. They are “spiking”, which uses several local spectra to augment the calibration made with an ESL e.g., and spiking with extra-weighting , which uses multiple copies of the local samples to improve their leverage in the calibrations. showed that the latter approach improved on spiking and suggested it might be more appropriate with larger spectral libraries.

developed a new approach, which they call rs-local, which makes the best use of ESLs and minimizes the number of site-specific, local samples for deriving calibrations. The method is data-driven and makes no assumptions on spectral or sample similarities. Using data from farms in Australia and New Zealand, they showed that by combining 12–20 local samples with a well-selected set of samples from an ESL, the robustness and accuracy of the predictions was improved compared to predictions made using a “general” calibration and other methods tested. The authors suggested that rs-local can reduce analytical cost and improve the financial viability of soil spectroscopy.

As described above, to measure and monitor soil organic C with spectra, a local spectroscopic model needs to be developed to ensure that the estimates of organic C are unbiased. Therefore, once the soil in a study area has been sampled, according to an appropriate soil sampling design (see Sect. 3.2), the spectra of the sampling units in the sample should be recorded using standardized protocols and guidelines, such as those described by the Global Spectral Library in . Following a spectral outlier analysis to identify erroneous spectra (due to rocks, roots, and other non-soil materials), the spectra of the sample, which characterizes the variability in the study area, are used to guide the selection of data for the spectroscopic modelling and prediction, i.e. the training, validation, and prediction sets.

A method that ensures that the training spectra adequately represent the sample must be used to select the training set e.g. the Kennard–Stone or Duplex algorithms. The validation set should be selected by random sampling to ensure an unbiased assessment of the spectroscopic model predictions. How many spectra to use for training and validation depends on the available budget because the selected sampling units will need to be analysed with conventional laboratory methods and on the heterogeneity of the sample. Spectroscopic models for the prediction of soil organic C that are developed and validated with too few data can lead to unstable and erroneous results . Once the total number of sampling units are selected for the spectroscopic modelling, a general rule of thumb is to use two-thirds for training and the remaining third for validation, although this is not a hard rule and the choice might depend on the total number of sampling units that are selected. The prediction set is made up of the data that remain in the sample after the training and validation sets have been selected.

Once the spectra for the modelling have been selected, soil aliquots of the respective sampling units need to be prepared for the analysis of soil organic C concentrations in the laboratory by dry combustion analysis (e.g. LECO – Laboratory Equipment Corporation). It is important to note that the inaccuracy and imprecision of analytical results are directly related to the sampling, handling, and analytical procedure . Therefore, soil sample preparation (drying, crushing, grinding, sub-sampling) and analytical measurements should be made with certified methods and in an accredited laboratory that conducts regular technical and inter-laboratory proficiency programs. For example in Australia accreditation is though the National Association of Testing Authorities (NATA) and the Australian Soil and Plant Analysis Council (ASPAC). We recommend that an independent assessment of the analytical accuracy is performed by including a small but representative proportion of “blind” duplicates (e.g. 15 %) in the analysis. If the blind duplicate samples exceed a predetermined threshold value (e.g. 0.05 % soil organic C), then the samples should be re-analysed by the laboratory. As with any modelling, the dictum when developing spectroscopic calibrations is “garbage in = garbage out” and conversely “quality in = quality out” .

The spectra and analytical data in the training set should then be analysed and if necessary, transformed, pre-processed, and pre-treated. For example, if the algorithm for the modelling assumes that the response variable is normally distributed, then the analytical data will need to be checked and if necessary transformed (e.g. with logarithmic transforms) to approximate a normal distribution. Similarly, the spectra may need a transformation to apparent absorbance; it may need smoothing and baselining (e.g. using a Savitzky–Golay filter with a first derivative; ). The spectra may also need to be mean-centred, and if recorded at field conditions, it may need corrections to remove the effects of water on the spectra (e.g. with either the external parameter orthogonalization – EPO, , or direct standardization – DS, ). See for a comparison of EPO and DS. It is important to note that whatever transformations, pre-processing, and pre-treatments are applied to the training set, they must also be applied to the validation and prediction sets.

Before embarking on the modelling, it is sensible to check for outliers and influential data in the training set objectively. This can be done by calculating Studentized residuals , to check for observations with unusually large residuals or data that deviate significantly from their mean, i.e. those with high leverage . If outliers are detected, then further checks for data entry and other errors should be made. One should not remove data unless there is reasonable evidence to suggest that the data are in error.

Spectroscopic models should be developed by cross-validation to obtain optimal parameterization of the models and to minimize or prevent problems with under- or over-fitting. Once a model is developed, model diagnostics should be performed to interpret the model and also to check that the statistical assumptions of the particular algorithm being used are not violated. For example, this could simply be done by calculating the residuals of the data in the training set and plotting these against the estimated soil organic C concentrations. This plot can help to diagnose dependence of the predicted value, non-constant (or heteroscedastic) variances, and non-linear trends that indicate the need for data transformations or alternate curvilinear modelling methods (see for example, ).

If all assumptions about the model are correct and the model has a good diagnosis, then the optimized model should be validated with the independent validation set, which was selected at random and which was not used in the training process. The type of algorithm used (e.g. partial least squares regression (PLSR), support vector machines (SVMs), regression trees) is not critical as long as the optimization and validation are done well. Modelling uncertainties could be derived with Monte Carlo e.g. or Bayesian methods.

The predictions on the validation set should be assessed with statistics that completely describe the errors in the same units as the analyte (i.e. soil organic C content). For this, we recommend the use of the RMSE, which measures the inaccuracy of the model predictions, the mean error (ME), which measures their bias, and the standard deviation of the error (SDE), which measures their imprecision. Inaccuracy may be defined as combining both bias and imprecision, so that RMSE2 = ME2 + SDE2 . Other indices that are commonly reported are the coefficient of determination (R2), the ratio of performance to deviation (RPD) , or the ratio of performance to interquartile range (RPIQ) and the concordance correlation coefficient . We do not recommend the use of the R2 alone because it does not account for bias in the model predictions or the RPD or RPIQ alone because their categories are subjective and variable .

If the independent model validation statistics are not too dissimilar to those reported in the literature for soil organic C predictions at the field and farm scales, e.g. for vis–NIR spectroscopy 0.1–1.0 % organic C, with a median value of 0.3 % organic C , we recommend that additional sampling units be selected to augment and potentially extend the range of the training set. These values should serve only as guidelines, and if the validation statistics fall outside of these ranges after the augmentation of the training set, then it might not be sensible to proceed with spectroscopy for the estimation of organic C.

The optimized and validated spectroscopic model may then be applied to all of the spectra: the training, validation, and prediction sets, to estimate consistently the soil organic C concentration of the entire sample set and their uncertainty. If the model was developed on a transformed organic C scale (e.g. square root or logarithmic), then the estimates need to be back-transformed to the original units.

Figure  summarizes the procedures for the spectroscopic measurement, modelling, and prediction of soil organic C.

Standardization of the sensing methods and their procedural guidelines are needed if sensing is to underpin methodologies that help to account for soil organic C stock change. This is particularly important for international initiatives like “4 pour 1000”, which aim to demonstrate that soil can play an important role in mitigating climate change. Standards and guidelines are also essential in schemes that use financial incentives for landholders to adopt C sequestration practices (see Table ). In this case, standards will help to ensure that only authentic abatement is credited. In the Australian ERF, methods need to comply with offsets integrity standards, which require that abatement is additional, eligible, measurable, verifiable, evidence-based, statistically defensible, supported by relevant peer-reviewed scientific results, permanent, with no leakage, and conservative (see Sect. 2).

reported that some standards exist for the analysis of soil properties, including organic C, but suggested that new standards are needed for the measurement of soil C stocks and the verification of C change. In Table , we propose a list of data and information that need to be reported when developing a sensing methodology for soil C accounting. These include, for instance, the type of sensor used, the requirements for calibration, the number of reference analyses to use, the requirements for validation, the statistics to report, and the information that must be recorded for auditing and verification. Of course, the list in Table , is additional to data on the project area, the sampling design used, the soil sampling method, the method for estimation, and the preparation of the samples for analysis.