Proximal sensing for soil carbon accounting

Maintaining or increasing soil organic carbon (C) is vital for securing food production and for mitigating greenhouse gas (GHG) emissions, climate change, and land degradation. Some land management practices in cropping, grazing, horticultural, and mixed farming systems can be used to increase organic C in soil, but to assess their effectiveness, we need accurate and cost-efficient methods for measuring and monitoring the change. To determine the stock of organic C in soil, one requires measurements of soil organic C concentration, bulk density, and gravel content, but using conventional laboratory-based analytical methods is expensive. Our aim here is to review the current state of proximal sensing for the development of new soil C accounting methods for emissions reporting and in emissions reduction schemes. We evaluated sensing techniques in terms of their rapidity, cost, accuracy, safety, readiness, and their state of development. The most suitable method for measuring soil organic C concentrations appears to be visible–near-infrared (vis–NIR) spectroscopy and, for bulk density, active gamma-ray attenuation. Sensors for measuring gravel have not been developed, but an interim solution with rapid wet sieving and automated measurement appears useful. Field-deployable, multi-sensor systems are needed for cost-efficient soil C accounting. Proximal sensing can be used for soil organic C accounting, but the methods need to be standardized and procedural guidelines need to be developed to ensure proficient measurement and accurate reporting and verification. These are particularly important if the schemes use financial incentives for landholders to adopt management practices to sequester soil organic C. We list and discuss requirements for developing new soil C accounting methods based on proximal sensing, including requirements for recording, verification, and auditing.

1 Introduction

Soil is the most abundant terrestrial store of organic carbon (C) (Batjes, 1996). Soil organic C is an indicator of soil quality because it affects nutrient cycling, aggregate stability, structure, water infiltration, and vulnerability to erosion (Lal, 2013; Tiessen et al., 1994). Management of soil organic C is central to maintaining soil health, ensuring food security, and mitigating climate change (Lal, 2004, 2016b). This is why stabilizing or increasing the stocks of organic C in soil, through the identification and implementation of locally appropriate agronomic and environmental management practices, has become a political imperative (ITPS, 2015).

Maintaining or increasing soil organic C stocks is essential for addressing land degradation, which was recognized under the United Nations Sustainable Development Goal 15, i.e. the sustainable development of life on land (United Nations, 2015). Used in concert with data on land cover and land productivity, soil organic C stock will be the first metric used to quantify Indicator 15.3.1, the “Proportion of land that is degraded over total land area” (Decision 22/COP.11; UNCCD, 2013). The United Nations Convention to Combat Desertification (UNCCD) will use soil organic C stock as one of its indicators to monitor progress towards achieving land degradation neutrality (Orr et al., 2017).

Sequestration of soil organic C has been considered as a possible solution to mitigate climate change (Lal, 2016a). Both policy makers and scientists expect there is significant potential for sequestration in agricultural soils due to the large historical losses experienced by agroecosystems (Cole et al., 1997). Studies over several decades have demonstrated that improved land use and management can help to sequester organic C in soil and to reduce greenhouse gas (GHG) emissions (Ogle et al., 2005; Poeplau and Don, 2015; Smith et al., 2008). This opportunity was the motivation behind the “4 pour 1000” initiative, launched at the COP21 meeting in Paris, which aims to increase global soil organic C stocks by 4 parts per 1000 (or 0.4 %) per year as compensation for global GHG emissions from anthropogenic sources. Engagement in “4 pour 1000” involves stakeholders voluntarily committing to implementing farming practices that maintain or enhance organic C stocks in agricultural soil and to preserving C-rich soil (Lal, 2016a). In addition to increasing soil organic C, the initiative provides an opportunity to implement transparent and credible protocols and methods for C accounting, monitoring, reporting, and verification that are compatible with national GHG inventory procedures (Chambers et al., 2016).

Although there is good potential for changes in agricultural land use and management to sequester soil organic C (Machmuller et al., 2015), their implementation requires efficient new methods for measurement and monitoring of C stocks. To determine baselines and to assess the success of management practices for sequestering soil organic C, the variability in soil organic C stock needs to be quantified in both space (laterally across the landscape and vertically down the soil profile) and time. Changes in soil organic C stock that are due to changes in land use and management or climate occur slowly (e.g. compared to changes in biomass C) and must be measured over periods longer than 5 to 10 years (Smith, 2004). Changes are also likely to be small relative to the C stock that is present in the soil, which is variable. New measurement methods are also needed to delineate the potential benefits and liabilities of establishing soil C projects and C accounting activities (IPCC, 2007). Methods for soil organic C sampling and analysis must be accurate, practical, inexpensive, and cost-efficient and when used for monitoring, they must consider the minimum detectable difference (Batjes and van Wesemael, 2015). Additionally, the new methods should accurately quantify the magnitude and uncertainty of soil C change that could be attained by introducing defined agricultural management practices (Paustian et al., 2016).

The quantification of soil organic C stock change requires the measurement of soil organic C concentration, bulk density, and gravel content over time. Conventional methods to measure soil organic C stocks involve field soil sampling, followed by sample preparation and laboratory analysis. For the determination of soil C concentration, dry combustion (Nelson and Sommers, 1996) is the benchmark method because of the vast experience acquired in using it and because of its accuracy. For bulk density, conventional measurements are made using the volumetric ring method or the clod method for soil containing a lot of rock fragments (Blake and Hartge, 1986). For gravel content, the conventional procedure involves manual separation of gravel from the fine-earth fraction (McKenzie et al., 2002). These methods are time-consuming and expensive, particularly for measuring at depth.

Sensors can provide rapid, accurate, inexpensive, and non-destructive measurements of soil organic C stocks and other soil properties. Their measurements are accurate and cost-efficient (Viscarra Rossel and Brus, 2018). While there are several reviews on the use of sensors for measuring soil organic C concentration (Bellon-Maurel and McBratney, 2011; Izaurralde et al., 2013; Reeves et al., 2012; Stenberg et al., 2010; Viscarra Rossel et al., 2011), few studies report sensors for measuring bulk density or gravel (Fouinat et al., 2017; Lobsey and Viscarra Rossel, 2016) or the integration of sensing for soil organic C accounting. Our objective here is to review the current state of proximal sensing for soil C accounting. Specifically, our aims were to review: (1) soil C accounting for emissions reporting and in emissions reduction schemes, (2) the current state of proximal sensing for measuring soil organic C stocks and monitoring its change and (3) the use of proximal sensors in the development of new soil organic C accounting methodologies.

2 Soil organic carbon accounting

The development of new technologies for soil organic C accounting concerns two areas of national GHG policy and reporting: national emissions reporting obligations under the United Nations Framework Convention on Climate Change (UNFCCC) and the Kyoto Protocol and domestic schemes that seek to reduce or offset emissions through a range of activities, including improved land management practices.

(Australian Government, 2014)(Australian Government, 2015)(Australian Government, 2018)

Table 1 Examples of methodologies for estimating change in soil organic C for mitigation actions related to changed management of agricultural land. Emissions Reduction Fund (ERF); Climate Change and Emissions Management Act (CCEMA); Alberta Offset System (AOS); Climate Policy Framework (CPF).

¹ http://aep.alberta.ca/climate-change/guidelines-legislation/specified-gas-emitters-regulation/offset-credit-system-protocols.aspx (last access: 6 December 2017). ² http://www.usask.ca/soilsncrops/conference-proceedings/previous_years/Files/cc2000/docs/posters/018_post.PDF (last access: 6 December 2017). ³ http://ssca.ca/images/new/PSCB.pdf (last access: 6 December 2017). ⁴ http://www.climateactionreserve.org/how/protocols/grassland/ (last access: 6 December 2017).

The UNFCCC, and later the Kyoto Protocol, set up a system of national communications and national inventory reporting to be compiled by parties and published by the UNFCCC. To estimate GHG emissions and to monitor changes in C stocks, including soil organic C, the International Panel on Climate Change (IPCC) developed a tiered methodology that relates data on land use and management activities to emissions and storage factors to estimate fluxes from the activities (IPCC, 2006). The three-tiered approach depends on the scale, capability, and availability of data. Where country-specific data are currently lacking, a global default (Tier 1) approach can be used. Tier 1 methods use default equations with data from globally available land cover classes and global defaults for reference soil organic C stocks, change factors, and emission factors. Tier 2 methods include nationally derived land cover classes and data for reference soil organic C stocks, change factors, and emission factors specific to local conditions. Tier 3 methods might include national data from the integration of ongoing ground-measurement programs, earth observation, and mechanistic models. Tier 2 and Tier 3 approaches are thought to produce estimates with reduced uncertainty.

National-scale soil monitoring networks can produce information on changes in soil organic C stocks relative to a baseline through repeated measurements across a defined network of sites over time. They can provide a set of observations that represent the variation in climate, soil, or land use management at a national scale (Batjes and van Wesemael, 2015). However, there are trade-offs between the ability to detect a change and the size of the network and the number of measurements required, which is directly related to cost (Conant and Paustian, 2002a). Conventional analytical methods for soil monitoring are likely to be cost-inefficient. Sensing, on the other hand, can be used to cost-efficiently measure soil organic C stocks and to estimate baselines for national inventory reporting and monitoring (Viscarra Rossel et al., 2014).

Accurate and cost-efficient methods to quantify changes in C stocks are also needed for a growing number of national and sub-national emissions reduction and C accounting and trading schemes that incorporate mitigation from soil organic C sequestration following changes in land management (ICAP, 2017). In this context, how to measure, report, and verify the impacts of mitigation actions is important for decision makers because access to financial payments depends on the ability to demonstrate the sequestration or emissions reductions that might be attained. To date, however, relatively few methodologies have been developed for quantifying a change in soil organic C stock from changes in land management. Those that have been developed are based on approaches that use either direct measurement or mechanistic modelling (Table 1). Under such schemes, endorsed methodologies set out the rules for estimating emissions reductions or C offsets from different activities. Proponents that change some permissible aspect of their land management, which leads to increases in net C stocks or reductions in emissions, can use these methods to earn payments. Information from these activities can then also contribute to national inventories.

For example, the Australian Government established the Emissions Reduction Fund (ERF) to encourage the adoption of management strategies that result in either the reduction of GHG emissions or the sequestration of atmospheric CO₂. The ERF is enacted through the Carbon Credits (Carbon Farming Initiative) Act 2011 (CFI Act), and under it, carbon credits can be earned by anyone (e.g. landholders, businesses, and community groups) undertaking a project that aims to reduce emissions or sequester C (Australian Government, 2011). Projects must comply with approved methods (which are legislative instruments) that define the activities that are eligible to earn C credits and how the abatement is measured, verified, and reported. These methods¹ must comply with the Offsets Integrity Standards described in the CFI Act, which require that any C abatement generated through the implementation of a method can be used to meet Australia's climate change targets under the Kyoto Protocol or other international agreements. The legislation ensures that only genuine emissions reductions are credited; that the methods used in the ERF are eligible, evidence-based (supported by relevant peer-reviewed science), measurable, verifiable, conservative, additional, and permanent (25 or 100 years); and that there is no leakage. Thamo and Pannell (2016) discuss these requirements and the challenges they pose for development of policy for soil C sequestration. Once a method is implemented by a proponent, it can be used to produce Australian Carbon Credit Units (ACCUs) and an offsets report that is then submitted to the Clean Energy Regulator. Proponents receive one ACCU for every tonne of emissions reduction. One ACCU corresponds to the sequestration or emission avoidance of 1 t of CO₂ equivalent, which proponents can sell to generate income.

The Australian ERF currently has three soil C sequestration methods:

• i.

  “Sequestering carbon in soils in grazing systems” (Australian Government, 2014) that aims to quantify changes in soil organic C stocks over time using conventional soil composite sampling and laboratory analysis;

• ii.

  “Estimating sequestration of carbon in soil using default values” (Australian Government, 2015) that uses default values for the rates of soil C change from different activities, predicted with the Full Carbon Accounting Model (FullCAM), which is used in the Australian National Greenhouse Gas Inventory;

• iii.

  “Measurement of Soil Carbon Sequestration in Agricultural Systems”, a new soil carbon method (Australian Government, 2018) legislated and made available on 25 January 2018. It increases the range of eligible activities to allow not only grazing businesses to participate but also cropping and horticultural businesses, it allows choice between different sampling designs to reduce sampling error (including the use of covariates and prior information to inform the design), and it allows the use of soil sensors to improve the accuracy and reduce the cost of measuring and monitoring soil carbon stocks. This is the first methodology in the world to legislate sensing for soil organic C accounting.

In Canada, the Government of Alberta amended the Climate Change and Emissions Management Act (CCEMA) in 2007 to require industries with substantial emissions to report and reduce their emissions to established targets. There are three options to meet the targets, one of which is emission offsets. The Alberta Offset System operates under a set of standards, known as the “Offset Quantification Protocols and Guidance Documents”. The development of protocols involves the inclusion of expert engagement, rigorous peer review, defensible scientific methodologies, and documented transparency. Under the Alberta Offset Scheme, the Conservation Cropping Protocol quantifies soil organic C change following changed land management from conventional cropping to conservation cropping (reduced tillage and summer fallow). The protocol uses Canada's National Emissions Tier 2 methodology, which developed C sequestration coefficients based on measuring and modelling local crop rotations, soil/landscape types, and inter-annual climate variation for geo-specific polygons in the national eco-stratification system. A second Canadian scheme in Saskatchewan, the Prairie Soil Carbon Balance (PSCB) Project was developed for quantifying and verifying the direct measurement of soil organic C changes in response to a shift from conventional tillage to no-till, direct-seeded cropping systems (McConkey et al., 2013). It was not designed to monetize soil carbon offsets but has been supported by farm groups with an interest in securing financial recognition for GHG mitigation. A regional monitoring project was established to measure the temporal change in soil C storage under agricultural cropland on the Canadian Prairies (Ellert et al., 2001). The sampling scheme reported in Ellert et al. (2001) for monitoring was designed to maximize the ability to detect changes in soil C over time by ensuring that exact sample locations can be relocated, limiting horizontal variability of soil C. This PSCB approach was subsequently used to assess sampling designs at contrasting scales in the USA (Conant and Paustian, 2002b).

3 Soil sampling, measurement, and the estimation of soil organic C stocks

3.1 Measuring soil organic C stocks

Effective accounting of changes in soil organic C stocks requires the measurement of the stocks and their uncertainty for a defined baseline and over the monitoring period. Internationally, the default method to determine soil organic C stock (C_s) for the accounting of change is to multiply measurements of soil organic C concentration, bulk density, and gravel content at a fixed depth of 0–30 cm and to report the stock as a mass of carbon per unit area in tonnes of organic C per hectare (IPCC, 1997):

$$\begin{array}{}\text{(1)}& {\displaystyle }{\displaystyle }{C}_{\mathrm{s}}=C_{\mathrm{m}}\times \mathit{\rho }\times \left(\mathrm{1}-{\displaystyle \frac{g}{\mathrm{100}}}\right)\times d,\end{array}$$

where C_m is the mass of soil organic C in the soil (%), ρ is the soil bulk density (g cm⁻³), g is the gravel content (%), and d is the thickness of the layer (cm). Our definition of soil organic C used here extends that of the IPCC Guidelines (IPCC, 2006), which address the measurement of soil organic C in mineral soil in the 0–30 cm layer, by extending to deeper soil layers. Based on typical rooting depths found in agricultural crops and pastures and the capacity of deeper soil horizons to sequester relatively large amounts of soil organic C, there is evidence to suggest that measurements should extend to deeper layers (Fan et al., 2016; Lorenz and Lal, 2005; Viscarra Rossel et al., 2016b).

Conventionally, the measurement of soil organic C stocks involves soil sampling (see Sect. 3.2), followed by sample preparation and laboratory analysis. For the analytical determination of soil C concentration, sample preparation typically entails drying, crushing, grinding, sieving, sub-sampling, quantification of the sample's water content, and further fine grinding for dry-combustion analysis (Nelson and Sommers, 1996; Rayment and Lyons, 2011). Conventional measurements of soil bulk density typically involve using the volumetric ring method, where a pit is dug and a metal core of known volume is driven into the soil at the fixed depth. The bulk density is then determined by dividing the oven-dry soil mass of the sample by the volume of the core (Blake and Hartge, 1986). Alternatively, the clod method (Cunningham and Matelski, 1968; Hirmas and Furquim, 2006; Muller and Hamilton, 1992) has been used for soil with abundant rock fragments, where clods of soil are sampled and sealed (e.g. with paraffin) and the volume of the sample is determined by its displacement of water in a vessel. Gravel content is conventionally measured by breaking the soil cores into specific depth intervals, drying them in an oven, crushing the soil with a mortar and pestle, and then sieving it to separate the fine-earth (≤ 2 mm) fraction from the gravel.

Equation (1) can be used to quantify and report the change in soil organic C stocks at fixed depth intervals. However, this method can systematically overestimate or underestimate C stocks if bulk densities increase or decrease, respectively, from changes in land use or land management practices (e.g. changes in cultivation). Where bulk densities differ between management practices or across time periods, more accurate estimates of the C stock and its change can be derived using measures of cumulative or equivalent soil masses per unit area (Wendt and Hauser, 2013). Various studies have recognized the importance of this approach, which also reduces the effect of depth of sampling errors (Ellert et al., 2001; Gifford and Roderick, 2003; Lee et al., 2009; VandenBygaart and Angers, 2006; Wendt and Hauser, 2013). Both measurement-based methods under the Australian ERF (Australian Government, 2014, 2018) use an equivalent soil mass (ESM) approach to quantify soil organic C stock change.

3.2 Soil sampling and estimation

Before measuring the soil organic C stocks (Eq. 1), a sampling design must be derived to determine the sampling locations. Methods to select sampling locations include probability sampling and non-probability sampling, which result in two widely used sampling philosophies: design- and model-based sampling (Brus and DeGruijter, 1993; Papritz and Webster, 1995; de Gruijter et al., 2006). In design-based sampling, the randomness of an observation originates from the random selection of sampling sites, whereas in model-based sampling, randomness comes from a random term in the model of the spatial variation, which is added to the model because our knowledge of the spatial variation is imperfect. Probability sampling is, therefore, a requirement for design-based sampling but not for model-based.

Choosing which approach to use depends mostly on purpose (Brus and de Gruijter, 1997). For instance, if one needs estimates of the mean or total soil organic C stock and their accuracy over a given area, whose quality is not dependent on the correctness of modelling assumptions, then design-based sampling might be most suitable. If the aim is to produce a map of the soil organic C stock over the area, then model-based sampling will be preferable. However, because design-based sampling can also be used for mapping and model-based sampling for the estimation of means or total C stocks, the choice of which approach to use can be difficult. Viscarra Rossel et al. (2016b) demonstrated the use of probability sampling, which allowed design-based, model-assisted, and model-based estimation of the total soil organic C stock across 2837 ha of grazing land in Australia. Spectroscopic and active gamma attenuation sensors were used for estimating soil organic C stocks and their accuracy in the 0–10, 0–30, and 0–100 cm layers and for mapping the stocks in each layer across the study area. Although the design-based, model-assisted, and model-based estimates of the total soil organic C stocks were similar, the variances of the model-based estimates were shown to be smaller than those of the design-based methods. The authors noted that the advantage of the design-based and model-assisted methods, unlike the model-based approach, was that their estimates of the baseline soil organic C stocks and their variances did not rely on the assumptions of a model. Further, they noted that although the model-based approach produced the smallest variance of the predicted total soil organic C stocks, the results cannot be generalized to other sample sizes and types of sampling designs. We suggest that whatever the method used, careful consideration of the sampling design is needed for the estimation of the baseline soil organic C stocks and for monitoring. Further discussion on the advantages and disadvantages of the sampling approaches can be found in de Gruijter et al. (2006).

4 Sensors for soil organic C accounting

We reviewed the literature on proximal soil sensing (see Viscarra Rossel et al., 2011 for a definition), to find the sensors that can be used to determine the soil organic C stock, that is, sensors to measure organic C concentration, bulk density, and gravel content (Eq. 1). Table 2 summarizes the assessment of each sensor technology in terms of their rapidity, accuracy, cost, safety, readiness for field-deployment, and stage of research and development. Below we also evaluate and report their suitability for measuring soil organic C and for monitoring its change.

We did not consider mobile spectroscopic sensing (Christy, 2008) in this review. Although these systems can be used to produce maps of soil organic C concentrations, which could be used in the sampling design and for the estimation of the C stocks, they are insufficient for soil organic C accounting because they do not measure C stocks; they do not measure the same depth consistently, and measurements are often made only within the 0–20 cm layer, which is shallower than the recommended 0–30 cm minimum depth of measurement for soil C accounting.

Table 2 Assessment of proximal sensing technologies regarding their readiness to underpin carbon accounting methodologies. The sensing methods are visible–near-infrared (vis–NIR) and mid-infrared (mid-IR) diffuse reflectance spectroscopy; laser-induced breakdown spectroscopy (LIBS); inelastic neutron scattering (INS); active gamma-ray attenuation (AGA); gamma- and X-ray computed tomography (CT). A “?” indicates “unknown” or “not sufficiently developed”; accuracy is relative to the conventional dry-combustion method for soil organic C concentration, volumetric ring method for bulk density, and manual processing for gravel content; cost: $ – AUD 0–40 000; $$  – AUD 40 000–100 000; $$$ – AUD 100 000+; active source refers to whether the source of energy used by the sensor is radioactive.

4.1 Sensing of soil organic C concentrations

4.1.1 Soil colour

Because organic C is known to affect soil colour, it is possible to use colour to estimate the organic C content of the soil (Ibañez Asensio et al., 2013; Liles et al., 2013; Viscarra Rossel et al., 2006a), for example, using digital cameras (Viscarra Rossel et al., 2008a) or mobile phone applications (Aitkenhead et al., 2016; Stiglitz et al., 2017). Although soil colour has been used to accurately predict soil organic matter content at regional and larger scales (Ibañez Asensio et al., 2013; Liles et al., 2013; Viscarra Rossel et al., 2008a), predictions are often inaccurate at field–farm scales or with soil that has inherently small C concentrations.

4.1.2 Soil visible, near-, and mid-infrared spectroscopy

Spectroscopic methods characterize soil organic C according to absorptions at specific wavelengths in the given spectral region. Visible and infrared spectroscopic techniques are highly sensitive to both the organic and inorganic components of soil, making their use in the agricultural and environmental sciences particularly relevant. Absorptions in the visible (vis: 400–700 nm) portion of the electromagnetic spectrum are due to electronic transitions and are useful for characterizing organic matter in soil as well as iron-oxide mineralogy (Sherman and Waite, 1985). Absorptions in the near infrared (NIR: 700–2500 nm) correspond to overtones and combinations of fundamental absorptions that occur in the mid-infrared region (mid-IR: 2500 and 25 000 nm) (Williams and Norris, 2001). As a consequence, absorptions in the NIR range are weaker and less distinctive compared to those in the mid-IR. It is useful to combine the vis and NIR ranges as each provides complementary information on soil. Instrument manufacturers have recognized this, and many offer spectrometers that measure the vis–NIR range.

Visible–NIR spectroscopy has been used successfully to predict soil organic C concentration, even under field conditions, but in the latter case using a method for correcting or removing the effects of soil water on the vis–NIR spectra (Ji et al., 2015; Minasny et al., 2011). Mid-IR can also accurately predict soil organic C concentrations. However, mid-IR spectroscopy is used mainly in the laboratory with measurements on oven- or air-dried and finely ground (typically 80–500 µm) soil samples (Le Guillou et al., 2015; Reeves, 2010; Reeves et al., 2012). There are strong water absorptions in the mid-IR, which tend to either mask or deform absorptions due to other soil constituents, thus degrading the calibrations and predictions of soil organic C with mid-IR spectra. There are published reviews on vis–NIR and mid-IR spectroscopy for predicting soil properties, including soil organic C concentrations (Bellon-Maurel and McBratney, 2011; Kuang et al., 2012; Soriano-Disla et al., 2014; Stenberg et al., 2010; Viscarra Rossel et al., 2006b, 2016a). We direct the reader to those reviews for further details.

To predict soil organic C, the spectroscopic techniques described above require the development of an empirical model (or calibration) that relates the spectra to corresponding soil data analysed with a reference analytical method such as dry-combustion analysis. This data set, which holds the spectra, soil analytical data, and metadata is referred to as a spectral library. To be useful for site-specific predictions of soil organic C, the spectral library should contain data that represent the local variability of soil organic C concentration. In Sect. 5.2 below, we review the methods that can be used to derive spectroscopic calibrations for predictions of soil organic C concentrations.

4.1.3 Laser-induced breakdown spectroscopy

Laser-induced breakdown spectroscopy (LIBS) uses atomic emission spectroscopy. A focused laser pulse heats the surface of the soil sample to break the chemical bonds and vaporise it, generating a high-temperature plasma on the surface of the sample. The resulting emission spectrum is then analysed using a spectrometer covering a spectral range from 190 to 1000 nm. The different LIBS peaks from the analysed samples can be used to identify the elemental composition of the soil. Information on peak intensities can then be used to quantify the concentration of elements in the sample. Cremers and Radziemski (2006) and Senesi and Senesi (2016) provide detailed descriptions of the method.

Reports that use LIBS for measuring soil organic C mostly use large benchtop instruments with prepared samples and calibrations to predict soil organic C from the measured elemental C (Bel'kov et al., 2008; Cremers et al., 2001; Ebinger et al., 2003; Knadel et al., 2017). These studies have reported good correlations between LIBS measurements and those from dry combustion, particularly for soil with similar morphology (Cremers et al., 2001; Ebinger et al., 2003). They also reported that LIBS measurements are rapid (less than a minute per sample). LIBS can provide rapid and accurate soil elemental analysis on prepared soil samples, and evidently, LIBS spectra can be calibrated to estimate soil organic C.

Currently, the primary constraints of LIBS for measuring soil organic C are sample preparation, sample representativeness because only a tiny volume of soil is ablated (Izaurralde et al., 2013), and our limited understanding of the accuracy of its measurements on soil that is wet under field conditions. We are aware of only one study that reports the use of LIBS for measuring the organic C content of intact, field-moist soil core samples in the laboratory (Bricklemyer et al., 2011). In that study, Bricklemyer et al. (2011) evaluated the accuracy of a field-scale LIBS calibrations for estimating total, inorganic, and organic C concentrations. The authors showed that LIBS spectra recorded from intact soil cores could be calibrated to quite accurately estimate total C and inorganic C concentrations. However, estimates of soil organic C were poor (r² = 0.22). The authors suggested that the poor predictability might be due to the low variance of organic C in their soil samples (Bricklemyer et al., 2011). Another factor that may have contributed to the poor predictability may be the narrow spectral range, which did not capture emissions from several other elements associated with organic C (Senesi and Senesi, 2016).

Although there are some commercially available portable LIBS systems (Harmon et al., 2005), there are few reports on their use for measuring soil organic C. Da Silva et al. (2008) used a portable LIBS system to measure a small number of pre-processed tropical soil samples in the laboratory. Izaurralde et al. (2013) used an “SUV-portable” LIBS system for field measurements of soil organic C, but significant sample processing was needed before measurements could be made, including breaking up soil cores and pelleting sub-samples in a hydraulic press. The authors speculated that there is potential for using LIBS to measure intact soil cores under field conditions (wet), albeit less accurately. More research and development is needed to assess the potential for LIBS to accurately estimate soil organic C concentrations in soil that is under field conditions.

4.1.4 Inelastic neutron scattering

Inelastic neutron scattering (INS) involves spectroscopy of gamma rays induced by fast and thermal neutrons interacting with the nuclei of the elements in soil. Fast neutrons, generated by a neutron generator, penetrate the soil and stimulate gamma rays that are then detected by an array of scintillation detectors such as sodium iodide (NaI) detectors. Peak areas in the measured spectra are proportional to the elemental composition of the soil, and peak intensities (counts) together with established calibrations can be used to determine soil organic C in units of g C m⁻² (Wielopolski et al., 2008).

The suggested benefits of INS include the ability to interrogate large volumes of soil over a relatively large footprint and the ability to measure to a depth of approximately 30–50 cm (Chatterjee et al., 2009). Thus, there is good potential for using it to non-invasively measure soil organic C (volumetrically) to a depth of around 30 cm. There is also no sample preparation required. Wielopolski et al. (2011) reported the feasibility of an INS instrument for measuring soil organic C. The sensor used high-energy neutrons and photons to sample soil volumes up to about 0.3 m³ and from approximately 20 to 30 cm deep, with a 150 cm diameter footprint. Although this technology appears useful, it is not yet sufficiently developed, the equipment is expensive, and there are concerns around the safe use of fast neutron generators in farms (Izaurralde et al., 2013).

4.2 Sensing of soil bulk density

4.2.1 Active gamma-ray attenuation

Active gamma-ray attenuation (AGA) measures the attenuation of the radiation by the soil, as defined by Beer–Lambert's Law, and provides a direct measure of the soil density. Because the mass attenuation coefficient of soil is a function of both photon energy and its elemental composition, attenuation is affected by texture and mineralogy. The measurement of bulk density by AGA can be made using either the scattering method or the transmission method. The former is applied to mostly surface determinations, mainly using gamma/neutron surface gauges, while the latter is used for measurements at depth, which can be made in the laboratory or the field (Pires et al., 2009).

AGA with a gamma (or neutron) surface gauge and a source of radiation that is lowered into the soil to the effective measurement depth can be used to measure soil density. The backscattered gamma radiation that originates from the source loses some of its energy on the way back to the scintillation detector at the surface, and the energy of the detected radiation is proportional to the density of the soil. The technique requires considerable soil preparation and correction for soil water to derive soil bulk density. Soil surface preparation requires that there are no gaps between the soil and the sensor, and, for measurements at depth, a pit needs to be dug to the effective measurement depth into which the active gamma source is lowered. Reports on the accuracy of these measurements are variable (Holmes et al., 2011; Timm et al., 2005) and are possibly due to problems with uneven soil surfaces and soil preparation issues. Relationships between bulk density measured with a neutron density meter and those on paired soil samples made with the conventional ring method were not strong, and they were variable (R² = 0.14–0.47, N = 75; Holmes et al., 2011). Others needed new calibrations for different soil types or bulk densities < 1.4 g cm⁻³ (C'assaro et al., 2000; Rousseva et al., 1988). Other studies have shown that bulk density measured with a neutron–gamma surface gauge tended to be less dense than the conventional ring method, although differences were not statistically significant (Bertuzzi et al., 1987; Rawitz et al., 1982; Timm et al., 2005).

AGA using measurements of transmission can be used to measure soil density. In this case, the measurements are made axially through a soil core and the attenuation of gamma radiation passing through it to the scintillation detector is proportional to the density of the soil. Pires et al. (2009) and Lobsey and Viscarra Rossel (2016) provide descriptions of the measurement principles. The method requires sampling of intact soil cores and when measurements are made on soil under field condition, corrections for water, θ, are needed. No other sample preparation is required.

The bulk density of the soil cores, ρ_b, can be derived with (Lobsey and Viscarra Rossel, 2016):

$$\begin{array}{}\text{(2)}& {\displaystyle }{\displaystyle }{\mathit{\rho }}_{\mathrm{b}}={\displaystyle \frac{\mathrm{1}}{x{\mathit{\mu }}_{\mathrm{s}}}}\ln \left({\displaystyle \frac{I_{\mathrm{0}}}{I}}\right)-{\displaystyle \frac{{\mathit{\mu }}_{\mathrm{w}}}{{\mathit{\mu }}_{\mathrm{s}}}}{\mathit{\rho }}_{\mathrm{w}}\mathit{\theta },\end{array}$$

where I is the incident radiation at the detector, I₀ is the un-attenuated radiation emitted from the source, x is the sample thickness in cm, μ_s is the mass attenuation coefficient of dry soil in cm² g⁻¹, μ_w is the mass attenuation coefficient of soil water at 0.662 MeV, ρ_w is the density of water (taken as 1 g cm⁻³), and θ is the volumetric water content of the soil in cm⁻³ cm⁻³.

Good agreement between measures of bulk density with an AGA transmission sensor and the conventional volumetric ring method has been found for dry samples in the laboratory (Pires et al., 2009). More recently, Lobsey and Viscarra Rossel (2016) showed that this method with vis–NIR corrections for water could accurately and rapidly (on average 35 s per measurement) measure, ex situ, the bulk density of soil cores sampled (wet) under field condition. The method facilitates the analysis of soil bulk density at fine depth resolution enabling the characterization of the spatial variability of soil bulk density in lateral and vertical directions. Lobsey and Viscarra Rossel (2016) report that the accuracy of measurements was similar to that obtained using the conventional single-ring method on the same samples (RMSE = 0.06 g cm⁻³; R² = 0.90; N = 32). Further, the authors show that the method can be used to determine organic C stocks on a fixed-depth or equivalent soil mass basis (Lobsey and Viscarra Rossel, 2016).

4.2.2 Computed tomography

Computed tomography (CT) was introduced in soil science several decades ago (Petrovic et al., 1982). Since then, it has been used to assess porosity and pore size distribution (inversely related to bulk density), tortuosity, soil structure, and compaction (Lopes et al., 1999; Pires et al., 2010). CT is based on the principle that electromagnetic radiation (commonly X- or gamma rays), is attenuated by matter. Similar to the AGA described above, attenuation follows the Beer–Lambert Law. CT is used to convert the attenuation of the radiation by matter into CT numbers called tomographic units (TUs), and the soil mass attenuation coefficient is used to derive soil density. The techniques can produce cross-sectional images to create a three-dimensional model, and hence they have good potential for measuring soil bulk density (and gravel content; see Sect. 4.3.1 below) of intact soil core samples.

Only a few studies have demonstrated that gamma-ray CT can be used to measure soil bulk density (Pedrotti et al., 2005; Timm et al., 2005). Timm et al. (2005) compared measurements of bulk density with gamma-ray CT to several other methods, including the conventional volumetric single-ring method, and found the CT technique to be more accurate. In an evaluation of the potential for X-ray microtomography for measuring the bulk density of soil with different textures and at different depths, Segnini et al. (2014) found only moderate linear agreement with the conventional volumetric ring method (R² = 0.58, N = 12). They concluded that factors such as the “beam hardening” effect (see Cnudde and Boone, 2013) and the polychromatic nature of X-ray microtomography make it difficult to measure soil bulk density directly. However, more research and evaluation is needed. An advantage of the CT methods is that they can provide a detailed analysis of soil bulk density profiles at a fine spatial resolution. Cnudde and Boone (2013) provide a review of the applications and limitation of X-ray CT.

4.2.3 Spectroscopic and pedotransfer functions

The bulk density of soil is a measure of the amount of pore space in a volume of soil. Thus, spectroscopy, being a surface measurement, cannot physically measure density, particularly if the soil has been ground and sieved. Nonetheless, it is often suggested that predictions of bulk density using vis–NIR or mid-IR spectra are possible. The reason is that under certain conditions, these spectroscopic models rely on second- or higher-order correlations to other soil constituents that are spectroscopically active (e.g. minerals, organic matter, water). However, because the predictions are “indirect”, they can be biased. Moreira et al. (2009) found that using vis–NIR spectroscopy on dry soils to predict soil bulk density produced inaccurate results and concluded that further research was needed to assess the limits and specificity of the method. A more recent study by Roudier et al. (2015), using vis–NIR on wet intact cores, found that predictions of bulk density were relatively accurate (in soil containing no gravel), but calibration was at a very local scale and thus for very specific conditions. Pedotransfer functions (PTFs) are commonly used to estimate bulk density (Tranter et al., 2007). However, they are often biased and imprecise and therefore unsuitable for the determination of soil organic C stocks, even when developed with soil from the same study area (Don et al., 2007). A further disadvantage of using PTFs is that they use other soil properties, which need to be measured, as input variables.

4.3 Sensing of gravel

Gravels are defined as coarse fragments with particles that are coarser than 2 mm (McKenzie et al., 2002). The presence of gravel has a significant effect on the mechanical and hydraulic properties of soil (Brakensiek and Rawls, 1994; Sauer and Logsdon, 2002). If gravel is present but not accounted for, it could bias the measurements of soil organic C stocks (Lobsey and Viscarra Rossel, 2016; Poeplau et al., 2017). For example, the presence of abundant coarse fragments (> 20 %) adversely affected the measurement of soil bulk density by both conventional and AGA using backscatter methods (Holmes et al., 2011). Sensing of gravel is difficult, and so it is typically measured manually by drying, crushing, sieving, and weighing of the soil and gravel. This method is time-consuming.

4.3.1 Wet sieving and image analysis

Viscarra Rossel and Lobsey (2017) developed a wet-sieving system combined with image analysis to more efficiently measure the gravel content of soil core samples in the field. The system enables rapid wet sieving of the core samples in 10 cm increments. The system is modular and can accommodate soil cores of various lengths. The authors tested the system using four soil types with varying textures and gravel contents. They showed that for a 1 m soil core, gravel could be separated from the soil, at 10 cm intervals, in 10–20 min, which is considerably faster (by a factor of more than 10) than the conventional method. By imaging the resulting gravel and measuring the pixel area or pixel volume occupied by the gravel in the images, they could accurately estimate the gravel content of the different soil types compared with manual weighing (R² = 0.79–0.90, average R² = 0.85 over the four soil types; Viscarra Rossel and Lobsey, 2017). The authors suggested that the method showed good promise for use in a soil C stock measuring system and that further testing and improvements of the wet sieving might include using a dispersing agent (e.g. (NaPO₃)₆). A further advantage of the method is that as well as gravel, un-decomposed plant materials and roots are also separated (Viscarra Rossel and Lobsey, 2017).

4.3.2 Computed tomography

There is good potential for the development of CT methods to measure gravel (and bulk density). However, there is little research on the topic. Fouinat et al. (2017) tested a novel X-ray CT method to analyse distinct deposits in lake sediment cores. The analysis highlighted the presence of denser > 2 mm mineralogical particles (i.e. gravel) in the silty sedimentary matrix. When compared to conventional manual measurements that involved sieving and water displacement to measure volume, they found that CT measurements overestimated the volume of the gravel by 11.6 %. The authors suggested that the overestimation might be due to pixel resolution issues. Nonetheless, the authors obtained a strong positive correlation (ρ = 0.81) between the CT measurements and the more conventional method. More research and development is needed, and the discussion above for sensing bulk density also applies here for gravel.

The development of CT for quantifying both gravel (and bulk density) appears promising. Portable CT scanners exist for medical and other applications, including soil, but further research and development or significant modification of existing systems are needed to measure bulk density and gravel content for C accounting. The system would need to have (1) the ability to rotate the soil core around fixed sensors or rotate the sensors around the core; (2) short measurement times to produce images of appropriate resolution; (3) adequate emission energies and appropriate shielding; (4) specialized software for the reconstruction of images and measurement of bulk density and gravel; and (5) appropriate size and weight to allow routine field deployment. Of course, a vis–NIR sensor for measuring soil organic C concentration would need to be used with it. In the meantime, however, the separation of gravel by rapid wet sieving and quantification by automated weighing or by image analysis might be an efficient interim.

4.4 Evaluation of sensing for soil organic C accounting

Based on our review and assessment of the available sensor technologies above, currently, the most suitable proximal sensing techniques for measuring soil organic C and for monitoring its change are vis–NIR and mid-IR spectroscopy for estimating soil organic C concentration and AGA for measuring bulk density. There are no practical or efficient sensors available for measuring gravel. Presently, a possible best option might be wet sieving to separate the gravel fraction and quantification by weighing or image analysis (Viscarra Rossel and Lobsey, 2017). Table 3 provides a summary of the benefits and limitations of each of these technologies, and Table 4 assesses their cost and accuracy.

Table 3 Benefits and limitations of sensing technologies for soil C accounting. vis–NIR – visible and near infrared diffuse reflectance spectroscopy; mid-IR – mid-infrared spectroscopy; AGA – active gamma-ray attenuation; ESM – equivalent soil mass; SOP – standard operating procedure.

Table 4 Assessment of the accuracy and cost of sensing technologies for soil C accounting. The methods are visible–near infrared (vis–NIR) and mid-infrared (mid-IR) diffuse reflectance spectroscopy; active gamma-ray attenuation (AGA) and wet sieving and image analysis. The accuracy at local scale (i) for soil organic C concentrations is based on comparison with dry-combustion analysis on the same samples and represented by median values for dry soil samples (vis–NIR and mid-IR) and wet samples (vis–NIR only); (ii) for bulk density is based on a comparison with the volumetric ring method and represented by median values for wet soil corrected for water with vis–NIR measurements on the same samples (transmission) or for wet soil corrected for gravimetric water content on paired samples (backscatter); and (iii) for gravel content is based on a comparison of image analysis with weighing on the same wet-sieved samples and represented by median values. The statistics reported are the root mean square error of validation (RMSE_v), the coefficient of determination ($R_{\mathrm{v}}^{\mathrm{2}}$), and the ratio of performance to deviation (RPD_v). N are the number of local sites at which accuracy was assessed.

^a Sourced from Sankey et al. (2008b), Roudier et al. (2017), Chodak et al. (2007), Conforti et al. (2017), Fontan et al. (2010), Martin et al. (2002), McCarty and Reeves (2006), Nduwamungu et al. (2009), Peng et al. (2013, 2014), Araujo et al. (2015), Rodionov et al. (2014), Sarkhot et al. (2011), Shi et al. (2014), Udelhoven et al. (2003), Vagen et al. (2006), Vohland and Emmerling (2011), and Yang et al. (2012). ^b Sourced from Viscarra Rossel et al. (2017), Roudier et al. (2015), Wetterlind and Stenberg (2010), and Rodionov et al. (2014). ^c Sourced from Viscarra Rossel et al. (2006b), Araujo et al. (2015), McCarty and Reeves (2006), Bornemann et al. (2010), Peng et al. (2014), and Yang et al. (2012). ^d Sourced from Lobsey and Viscarra Rossel (2016). ^e Sourced from Holmes et al. (2011). ^f Sourced from Viscarra Rossel and Lobsey (2017).

The cost of spectrometers can vary widely (Table 4). Portable vis–NIR spectrometers (350–2500 nm) can be purchased from different manufacturers from approximately less than AUD 10 000 to more than AUD 100 000 (Table 4), although their cost is continually decreasing as technologies develop. Smaller, cheaper vis–NIR spectrometers that use micro-electromechanical systems (MEMSs) technologies are emerging and are less expensive, but not many have been thoroughly tested. Prices for mid-IR portable spectrometers (2500–20 000 nm) are approximately AUD 50 000–70 000, and for mid-IR benchtop spectrometers, they are approximately AUD 25 000–90 000, depending on the size, type of detector, sensitivity, and amount of automation. The cost of spectroscopic measurements of soil organic C concentration in the laboratory is larger than measurements under field conditions because of the need for sieving, drying, and grinding. mid-IR measurements are expensive because samples need to be finely ground. Both vis–NIR and mid-IR techniques can accurately predict the soil organic C content of dry soils (Table 4). The accuracy of vis–NIR predictions on wet soil, that is under field conditions, is generally less than that for dry soil, although the difference can be relatively small (Table 4).

AGA sensors for measuring bulk density are also quite readily available (Table 4). Measurement costs are significantly smaller for AGA using transmission than for AGA using backscatter because of the additional soil preparation required for the latter. There are few reports on the use of AGA for the measurement of bulk density, but they report good accuracy for AGA using transmission and variable accuracy for AGA using backscatter. There is no information on the cost and accuracy of wet sieving and image analysis for quantifying gravel (Table 4). Viscarra Rossel and Lobsey (2017) suggested that a system could be easily developed and that measurement costs would be small.

4.5 Integrated multi-sensor systems for soil organic C accounting

We need an integrated multi-sensor approach for measuring and monitoring soil organic C stocks because simultaneous measurements of soil organic C concentration, bulk density, and gravel are needed. There are two currently available, field-deployable, proximal multi-sensor systems to measure soil organic C stocks. One involves inserting the sensors into the soil profile and making measurements in situ, while the other requires sampling undisturbed soil cores and measuring the soil ex situ.

Veris^® Technologies produces commercial sensors for precision agriculture (http://www.veristech.com, last access: 6 December 2017), including several field-deployable systems which can measure electrical conductivity, pH, and penetration resistance and also record the vis–NIR spectra of soil. The system that is particularly relevant to soil organic C accounting is the P4000, which uses a hydraulic probe system to insert four sensors into the soil to characterize the profile. The sensors are a vis–NIR spectrometer (350–2200 nm), an electrical conductivity (EC) sensor, and an insertion force sensor. The system does not measure bulk density. Using insertion force as a surrogate for bulk density might be possible but would be prone to errors. Wetterlind et al. (2015) tested the accuracy of predictions of soil organic matter (SOM) using the P4000 system and evaluated whether the predictions were improved when the sensors were combined. They found that the accuracy of predictions of soil organic matter content with the vis–NIR alone was good, but the inclusion of insertion force only improved prediction accuracy by about 10 %. They concluded that these small improvements did not provide strong support for combining vis–NIR sensor measurements with measurements of insertion force. However, there was no testing of bulk density.

The Soil Condition Analysis System (SCANS) (Viscarra Rossel et al., 2017) uses a combination of proximal sensing technologies, smart engineering, and data analytics to characterize soil laterally across the landscape and vertically down the profile. The SCANS has an automated soil core sensing system, which can be used in the laboratory or the field. The system has a vis–NIR (350–2500 nm) spectrometer, an AGA densitometer, and digital cameras that measure intact soil core samples that are either (wet) under field condition or dry, at user-defined intervals over the length of 1.2 m soil cores. The system can measure soil organic C content and composition (particulate, humus, and resistant C), bulk density, clay content, cation exchange capacity, volumetric water content, available water capacity, pH, iron, and clay mineralogy (Viscarra Rossel et al., 2017). Each measurement with the sensors takes approximately 35 s so that measuring a 1 m core at 2 cm intervals (i.e. 50 measurements along the core) takes about 30 min. Viscarra Rossel et al. (2016b) showed that the sensing system could be used to accurately baseline soil C stocks for accounting purposes, and Viscarra Rossel and Brus (2018) assessed its cost-efficiency and reliability for soil C accounting. They found that compared to more conventional methods that use composite sampling and laboratory analysis, sensing with the SCANS is more cost-efficient in that it provides a good balance between accuracy and cost.

5 Developing a soil organic C accounting methodology with proximal sensing

The rationale for using sensing in a method for soil organic C accounting is that although sensing may not be as precise per individual measurement compared to laboratory analysis, sensing is more cost-efficient; that is, sensing provides a balance between accuracy and cost. Because sensing is cheaper, simpler, and more practical to use, many more measurements can be made across space (laterally and vertically) and time, so that as an ensemble, the data are more informative. Sensing can also be non-destructive, allowing the soil samples to be stored in archives for future measurement should auditing and verification be required. The archived soil samples can then also ensure that there is consistent temporal data for use in dynamic models or for the testing of new technologies and approaches as they become available. Below we describe considerations needed for developing soil organic C accounting methodologies with proximal sensing.

5.1 Development of spectral libraries

The measurement of soil organic C using spectroscopy (e.g. vis–NIR, mid-IR) requires the calibration of the spectra to soil organic C content using multivariate statistics or machine-learning algorithms. The calibrations can be derived using existing large spectral libraries (ESLs) (Shepherd and Walsh, 2002; Stevens et al., 2013; Viscarra Rossel and Webster, 2012) or using new site-specific libraries developed with local soil samples (LSLs). Using an ESL to predict soil organic C incurs no immediate cost, but it is likely that the predictions at the local site (farm or field scales) will be biased (Clairotte et al., 2016; Guerrero et al., 2014b). Using an LSL will produce more accurate (unbiased) predictions but will incur a cost because soil needs to be analysed in the laboratory to derive the local model.

Significant investment has been made in developing large regional, country, and global spectral libraries (Brown et al., 2006; Shepherd and Walsh, 2002; Viscarra Rossel et al., 2016a), and there will be value in using these for developing site-specific calibrations. These ESLs could reduce the need for site-specific data. Various approaches have been proposed to make better use of ESLs for local predictions of soil properties. They are based on either constraining the ESL with spectral or soil sample similarities or augmenting the ESL with site-specific samples.

Memory-based learning (MBL) methods aim to constrain the ESL with spectral information and derive calibrations for each unknown sample on a case-by-case basis. By selecting a subset of the ESL to predict each unknown sample, these methods effectively derive site-specific (i.e. local) calibrations. Methods include the LOCAL (Shenk et al., 1997) and locally weighted regression (LWR) (Naes et al., 1990) algorithms and their variants. Essentially, the methods select calibration samples from the ESL with a distance metric (e.g. Mahalanobis distance) in the multivariate space between the calibration and the unknown samples. In LWR weighting the calibration samples are also weighted according to their spectral dissimilarity (distance) to the unknown samples.

Ramirez-Lopez et al. (2013) proposed spectrum-based learning (SBL), which is a type of MBL. The spectrum-based learner selects nearest neighbours from an ESL using distance metrics calculated in the principal component space and optimizing the number of components used to identify the nearest neighbours in the selection. Spectroscopic modelling is then carried out with both the selected neighbours and the matrix of distances to the unknown samples as the training data set.

The ESL can also be constrained with other information such as soil order, type, texture, and parent material (Sankey et al., 2008a; Vasques et al., 2010). Shi et al. (2015) proposed the use of both spectral similarities and geographically constrained local calibrations to predict soil organic C content. They reported improvements in the accuracy of predictions when the ESL was restricted to the geographic region from which the unknown samples originated. Viscarra Rossel and Webster (2012) developed general ESL calibrations for Australian soil using the machine learning algorithm Cubist. The authors showed that the algorithm makes inherently local predictions because Cubist partitions the spectra into local subsets that are each modelled separately.

Two techniques use the augmenting approach. They are “spiking”, which uses several local spectra to augment the calibration made with an ESL (Guerrero et al., 2010; Sankey et al., 2008b; Viscarra Rossel et al., 2009), and spiking with extra-weighting (Guerrero et al., 2014a), which uses multiple copies of the local samples to improve their leverage in the calibrations. Guerrero et al. (2014a) showed that the latter approach improved on spiking and suggested it might be more appropriate with larger spectral libraries.

Lobsey et al. (2017) developed a new approach, which they call rs-local, which makes the best use of ESLs and minimizes the number of site-specific, local samples for deriving calibrations. The method is data-driven and makes no assumptions on spectral or sample similarities. Using data from farms in Australia and New Zealand, they showed that by combining 12–20 local samples with a well-selected set of samples from an ESL, the robustness and accuracy of the predictions was improved compared to predictions made using a “general” calibration and other methods tested. The authors suggested that rs-local can reduce analytical cost and improve the financial viability of soil spectroscopy.

5.2 Spectroscopic modelling: training, validation, and prediction

As described above, to measure and monitor soil organic C with spectra, a local spectroscopic model needs to be developed to ensure that the estimates of organic C are unbiased. Therefore, once the soil in a study area has been sampled, according to an appropriate soil sampling design (see Sect. 3.2), the spectra of the sampling units in the sample should be recorded using standardized protocols and guidelines, such as those described by the Global Spectral Library in Viscarra Rossel et al. (2016a). Following a spectral outlier analysis to identify erroneous spectra (due to rocks, roots, and other non-soil materials), the spectra of the sample, which characterizes the variability in the study area, are used to guide the selection of data for the spectroscopic modelling and prediction, i.e. the training, validation, and prediction sets.

A method that ensures that the training spectra adequately represent the sample must be used to select the training set e.g. the Kennard–Stone (Kennard and Stone, 1969) or Duplex (Snee, 1977) algorithms. The validation set should be selected by random sampling to ensure an unbiased assessment of the spectroscopic model predictions. How many spectra to use for training and validation depends on the available budget because the selected sampling units will need to be analysed with conventional laboratory methods and on the heterogeneity of the sample. Spectroscopic models for the prediction of soil organic C that are developed and validated with too few data can lead to unstable and erroneous results (Reeves et al., 2012). Once the total number of sampling units are selected for the spectroscopic modelling, a general rule of thumb is to use two-thirds for training and the remaining third for validation, although this is not a hard rule and the choice might depend on the total number of sampling units that are selected. The prediction set is made up of the data that remain in the sample after the training and validation sets have been selected.

Figure 1 Framework for spectroscopic measurement, modelling, and prediction of soil organic C.

Once the spectra for the modelling have been selected, soil aliquots of the respective sampling units need to be prepared for the analysis of soil organic C concentrations in the laboratory by dry combustion analysis (e.g. LECO – Laboratory Equipment Corporation). It is important to note that the inaccuracy and imprecision of analytical results are directly related to the sampling, handling, and analytical procedure (Viscarra Rossel and McBratney, 1998). Therefore, soil sample preparation (drying, crushing, grinding, sub-sampling) and analytical measurements should be made with certified methods and in an accredited laboratory that conducts regular technical and inter-laboratory proficiency programs. For example in Australia accreditation is though the National Association of Testing Authorities (NATA) and the Australian Soil and Plant Analysis Council (ASPAC). We recommend that an independent assessment of the analytical accuracy is performed by including a small but representative proportion of “blind” duplicates (e.g. 15 %) in the analysis. If the blind duplicate samples exceed a predetermined threshold value (e.g. 0.05 % soil organic C), then the samples should be re-analysed by the laboratory. As with any modelling, the dictum when developing spectroscopic calibrations is “garbage in = garbage out” and conversely “quality in = quality out” (Viscarra Rossel et al., 2008b).

The spectra and analytical data in the training set should then be analysed and if necessary, transformed, pre-processed, and pre-treated. For example, if the algorithm for the modelling assumes that the response variable is normally distributed, then the analytical data will need to be checked and if necessary transformed (e.g. with logarithmic transforms) to approximate a normal distribution. Similarly, the spectra may need a transformation to apparent absorbance; it may need smoothing and baselining (e.g. using a Savitzky–Golay filter with a first derivative; Savitzky and Golay, 1964). The spectra may also need to be mean-centred, and if recorded at field conditions, it may need corrections to remove the effects of water on the spectra (e.g. with either the external parameter orthogonalization – EPO, Minasny et al., 2011, or direct standardization – DS, Ji et al., 2015). See Roudier et al. (2017) for a comparison of EPO and DS. It is important to note that whatever transformations, pre-processing, and pre-treatments are applied to the training set, they must also be applied to the validation and prediction sets.

Before embarking on the modelling, it is sensible to check for outliers and influential data in the training set objectively. This can be done by calculating Studentized residuals (Cook and Weisberg, 1982), to check for observations with unusually large residuals or data that deviate significantly from their mean, i.e. those with high leverage (Martens and Naes, 1989). If outliers are detected, then further checks for data entry and other errors should be made. One should not remove data unless there is reasonable evidence to suggest that the data are in error.

Spectroscopic models should be developed by cross-validation to obtain optimal parameterization of the models and to minimize or prevent problems with under- or over-fitting. Once a model is developed, model diagnostics should be performed to interpret the model and also to check that the statistical assumptions of the particular algorithm being used are not violated. For example, this could simply be done by calculating the residuals of the data in the training set and plotting these against the estimated soil organic C concentrations. This plot can help to diagnose dependence of the predicted value, non-constant (or heteroscedastic) variances, and non-linear trends that indicate the need for data transformations or alternate curvilinear modelling methods (see for example, Martens and Naes, 1989).

If all assumptions about the model are correct and the model has a good diagnosis, then the optimized model should be validated with the independent validation set, which was selected at random and which was not used in the training process. The type of algorithm used (e.g. partial least squares regression (PLSR), support vector machines (SVMs), regression trees) is not critical as long as the optimization and validation are done well. Modelling uncertainties could be derived with Monte Carlo (Viscarra Rossel, 2007) or Bayesian methods.

The predictions on the validation set should be assessed with statistics that completely describe the errors in the same units as the analyte (i.e. soil organic C content). For this, we recommend the use of the RMSE, which measures the inaccuracy of the model predictions, the mean error (ME), which measures their bias, and the standard deviation of the error (SDE), which measures their imprecision. Inaccuracy may be defined as combining both bias and imprecision, so that RMSE² = ME² + SDE² (Viscarra Rossel and Webster, 2012). Other indices that are commonly reported are the coefficient of determination (R²), the ratio of performance to deviation (RPD) (Williams and Norris, 2001), or the ratio of performance to interquartile range (RPIQ) (Bellon-Maurel et al., 2010) and the concordance correlation coefficient (Lin, 1989). We do not recommend the use of the R² alone because it does not account for bias in the model predictions or the RPD or RPIQ alone because their categories are subjective and variable (Reeves and Smith, 2009).

If the independent model validation statistics are not too dissimilar to those reported in the literature for soil organic C predictions at the field and farm scales, e.g. for vis–NIR spectroscopy 0.1–1.0 % organic C, with a median value of 0.3 % organic C (Viscarra Rossel et al., 2016a), we recommend that additional sampling units be selected to augment and potentially extend the range of the training set. These values should serve only as guidelines, and if the validation statistics fall outside of these ranges after the augmentation of the training set, then it might not be sensible to proceed with spectroscopy for the estimation of organic C.

The optimized and validated spectroscopic model may then be applied to all of the spectra: the training, validation, and prediction sets, to estimate consistently the soil organic C concentration of the entire sample set and their uncertainty. If the model was developed on a transformed organic C scale (e.g. square root or logarithmic), then the estimates need to be back-transformed to the original units.

Figure 1 summarizes the procedures for the spectroscopic measurement, modelling, and prediction of soil organic C.

Table 5 Data to be recorded for auditing and verification in a soil organic C accounting method.

5.3 Standards for auditing and verification

Standardization of the sensing methods and their procedural guidelines are needed if sensing is to underpin methodologies that help to account for soil organic C stock change. This is particularly important for international initiatives like “4 pour 1000”, which aim to demonstrate that soil can play an important role in mitigating climate change. Standards and guidelines are also essential in schemes that use financial incentives for landholders to adopt C sequestration practices (see Table 1). In this case, standards will help to ensure that only authentic abatement is credited. In the Australian ERF, methods need to comply with offsets integrity standards, which require that abatement is additional, eligible, measurable, verifiable, evidence-based, statistically defensible, supported by relevant peer-reviewed scientific results, permanent, with no leakage, and conservative (see Sect. 2).

Bispo et al. (2017) reported that some standards exist for the analysis of soil properties, including organic C, but suggested that new standards are needed for the measurement of soil C stocks and the verification of C change. In Table 5, we propose a list of data and information that need to be reported when developing a sensing methodology for soil C accounting. These include, for instance, the type of sensor used, the requirements for calibration, the number of reference analyses to use, the requirements for validation, the statistics to report, and the information that must be recorded for auditing and verification. Of course, the list in Table 5, is additional to data on the project area, the sampling design used, the soil sampling method, the method for estimation, and the preparation of the samples for analysis.

6 Final remarks

Currently, the most suitable proximal sensing techniques for soil C (stocks) accounting are vis–NIR spectroscopy for estimating organic C concentration and active gamma attenuation for measuring bulk density. There are no practical or well-developed sensors for measuring gravel content in the soil. CT appears promising for measuring both bulk density and gravel, but there are no systems available for this specific purpose, and so the approach requires significant research and development. A useful interim for measuring gravel might be its separation by rapid wet sieving and quantification by automated weighing or image analysis.

The use of mid-IR spectrometers for measuring organic C under field condition needs further research and development. Laboratory measurements of soil with mid-IR are possible, but additional transport, sample preparation by drying, and fine grinding will incur more labour costs. Although other technologies, such as LIBS and INS have some advantages, they also have significant disadvantages and are currently not sufficiently developed for making cost-efficient measurements.

Sensing of soil organic C with spectrometers requires a multivariate calibration. There has been significant investment made to develop large regional, country, and global spectral libraries, and we believe that there will be value in using them to improve the accuracy and cost-efficiency of soil spectroscopy. Statistical data-driven methods are being developed that use such libraries and reduce the need for local samples for deriving site-specific calibrations. Nevertheless, there is a need to develop sensors that are more direct (i.e. less reliant on empirical calibrations), accurate, safe, and inexpensive for measuring soil organic C.

Cost-efficient organic C accounting requires that the individual sensing techniques, above, be combined in a field-deployable, integrated multi-sensor data-analytics system, to derive estimates of the C stocks in soil profiles from at least the top 30 cm, but preferably deeper. Such systems are currently being developed and are showing that sensing with vis–NIR and active gamma attenuation sensors can provide accurate, rapid, and cost-efficient estimates of C stocks on either a fixed-depth or an equivalent soil mass basis.

Sensing can be used to underpin soil C accounting methodologies and to evaluate land use and soil management practices that aim to increase soil organic C stocks, improve soil health, increase agricultural production, and mitigate GHG emissions. But to ensure proficient measurement and accurate reporting and verification, the sensing methods should be standardized, supported by peer-reviewed science, and covered by robust procedural guidelines. This is particularly important in schemes that use financial incentives for landholders to adopt management practices to sequester soil organic C.

The new legislated soil C accounting method under the Australian Emissions Reduction Fund allows practitioners to use sensors for C accounting. The method is the first in the world to do so, and it might provide a template for other countries to follow.