On the risks of good intentions and poor evidence – comment on “Back to the future? Conservative grassland management can preserve soil health in the changing landscapes of Uruguay” by Säumel et al. (2023)

In this article, we make comments on some methodological issues and on the general approach of the paper “Back to the future? Conservative grassland management can preserve soil health in the changing landscapes of Uruguay” (Säumel et al., 2023). We have identified several design and methodological problems in the aforementioned article that may induce potential misinterpretations. First, our concerns are related to aspects of the study design and methodology that, in our opinion, introduce biases and critical errors. Second, we are concerned about the possible interpretations of the study with respect to the design of policies and the development of non-tariff barriers for South American countries.

1 Introduction

The article “Back to the future? Conservative grassland management can preserve soil health in the changing landscapes of Uruguay”, written by Säumel et al. (2023), analysed a set of soil parameters that describe the chemical conditions of the first 10 cm of 101 sampling areas under different land uses and land covers in Uruguay. Upon thorough examination, several deficiencies and considerations were discerned within the article that warrant attention, as they have the potential to give rise to erroneous or misleading interpretations.

2 Our main criticisms

Given the sensitive nature of soil degradation, the potential ramifications of drawing conclusions based on insufficient evidence could lead to misguided interpretations and subsequent actions. First, our concerns with respect to the article are related to aspects of the study design and methodology that, in our opinion, introduce biases and critical errors. Second, we are concerned about the possible interpretations of the study with respect to the design of policies and the development of non-tariff barriers for South American countries.

2.1 Design and methodological issues

2.1.1 About the sampling scheme

The authors of Säumel et al. (2023) indicate that they “randomly selected monitoring sites across the country”. Having a randomized design is certainly an advantage. However, no details are given on how this randomization was carried out. This process requires stratification, the definition of a grid, a criterion to discard sites that cannot be accessed, etc. A detailed example of a possible country-wide stratification alternative is presented in Altesor et al. (2019), in which the authors used land cover maps and a 10 km × 10 km grid in which 20 cells were drawn. Within each cell, five 1 km × 1 km squares were randomly chosen, and two patches belonging to two different natural grassland communities within that area were sampled by Lezama et al. (2019). Those areas corresponded to a MODIS grid pixel (231 m × 231 m), allowing for a clear localization of the sample. Random sampling implies complex logistics of displacements in the field, especially in areas with a low road density, such as the north of Uruguay. The absence of a description of the design and the coincidence of the sampling area locations with the distribution of roads (particularly national highway no. 5) does not allow one to dispel doubts about possible biases in the collection of samples. The coordinates of each of the sampling sites are also not indicated in the study, although it is stated that they were used to locate the soil groups in the Soil Map of Uruguay at a scale of 1:1 000 000. Aside from the general design, the authors indicated that “We sampled topsoil three times at each land use at the edges of the plot” (Säumel et al., 2023, p. 427)”. Does this mean 3 samples per plot (so the 280 samples are full of pseudo-replications) or three different times? Did they use composite samples?

2.1.2 Representativeness of land use types

It is striking that the proportions of land use types sampled in Säumel et al. (2023) differ strongly from those present in Uruguay, particularly if the sample sites were randomly chosen. According to the latest cartographies, both forestry and native forests are overrepresented. Both occupy 12.5 % (2 204 060 ha) of the country according to Baeza et al. (2022), whereas the samples of these land covers corresponded to 54 % of the total in Säumel et al. (2023). It is also difficult to make inferences about croplands in Uruguay with no samples in the southwestern region of the country (i.e. the main cropland area) (Baeza and Paruelo, 2020). This lack of coverage of the main cropland zone of the country is evident from the low number of samples under annual crop use (see Table 1 in the original paper). As an example, the authors only had 15 samples of cropland sites (5 %), while agricultural land use covered more 31 % of Uruguay in 2015–2016 (when the sampling was conducted) (https://uruguay.mapbiomas.org/, last access: 28 July 2024, Collection 1).

2.1.3 Design of the study

Any study that intends to establish differences associated with land use types from spatial sampling must minimize the sources of variation, excluding the factor to be compared (e.g. soil depth, texture, slope, rockiness, and water availability). Two widely used approaches in observational studies are paired sites or block sampling (e.g. Perelman et al., 2019). Säumel et al. (2023) compared all different land uses against each other, implicitly assuming that the observed differences were only due to land use types, without controlling for other factors that also co-varied in space with land use (e.g. all riverine native forest are located in lowlands). In addition, in a paired design aimed to compare land use effects on ecosystems, it is normally necessary to document that the paired sites sampled are located at an equivalent topographic position, on equivalent soil types, etc. There is no evidence that this was done in the aforementioned study. How did the authors control these types of effects? Did the differences in soil characteristics between tree plantations and native forest (or grasslands) result from the effect of the land cover or were they the consequence of planting trees on soils defined a priori for this use? The design of the study precludes an answer to these questions. This is not a trivial point because the soils defined as “Afforestation priority” in Uruguay have an inherently low fertility and pH. Moreover, the authors actually recognized the importance of soil heterogeneity – “In addition, the lateral heterogeneity of Pampean soils over short distances makes separating geochemical and anthropic signatures difficult (Roca, 2015)” (Säumel et al., 2023, p. 434) – although it is a key point that we consider they did not properly contemplate.

In addition, the land use trajectories proposed in Säumel et al. (2023) are oversimplified into four categories. The authors ignored well-known land use sequences in the region, such as annual crop–grassland returns, rotations with annual crops and perennial pastures, and the cropping history prior to 1986 (the agricultural peak of the 1950s).

2.1.4 Soil type characterization

The only approach used to characterize the site is the soil map of Uruguay at a scale of 1:1 000 000. This does not allow one to perceive critical edaphic and topographic differences. The soil group on such a soil map is defined by the dominant soil type at a scale of 1:1 000 000. It is well known that the fact that two sites belong to the same soil group does not mean that they have the same soil type (large differences in texture and other soil properties are common between soils in the same soil group). The assignment of a soil group without any field evidence is, at least, striking given the coarse resolution of the map used. It is surprising that the authors did not evaluate texture to characterize soils for (at least) the following two reasons: (i) it is key to give evidence about the comparability between pairs and (ii) it is a property that correlates/explains all other soil properties measured in Säumel et al. (2023), such as soil organic carbon (SOC) and the cation exchange capacity (CEC). In addition, there is a strong emphasis on CONEAT units in the article. This is a conceptual error, as CONEAT units are not a soil type per se: “CONEAT groups are not strictly basic soil mapping units, but constitute homogeneous areas, defined by their productive capacity in terms of beef, sheep and wool” (Art. 65 of Law 13695; https://www.gub.uy/ministerio-ganaderia-agricultura-pesca/politicas-y-gestion/coneat, last access: 28 July 2024). Again, there are normally large variations in soil types and properties inside a single CONEAT unit.

2.1.5 SOC data

One of the major shortcomings of the Säumel et al. (2023) publication is the lack of details on the way soil organic carbon (SOC) is reported. First, characterizing SOC changes using only the first 10 cm is, at least, incomplete and risky. This is even more of an issue if the particulate and mineral-associated fractions of the organic C are not differentiated. Land cover, management, or changes in the relative abundance of plant functional types may change the vertical distribution of SOC. In fact, within the same land use (native grasslands), paired grazed–ungrazed areas significantly differ with respect to the upper layer distribution of SOC and belowground C inputs (Piñeiro et al., 2009; López-Mársico et al., 2015). Such effects are evident way below 10 cm. Second, the authors reported SOC as a percentage or concentration without indicating if data were on a gravimetric or a volumetric basis. Reporting SOC without considering bulk density precludes any reasonable comparison on an equivalent soil mass (Gifford and Roderick, 2003). No data on bulk density, which is well-known to be affected by the land use types evaluated in this work, such as afforestation (Hernández et al., 2016) or crop production and crop–pasture rotations (Rubio et al., 2021), were reported. This is particularly critical if only data for the first few centimetres of the soils are reported. SOC stocks would differ dramatically between soils with different levels of compaction and, hence, differing bulk density. Moreover, soils under native forests and tree plantations have an upper layer with mixed soil and plant residues (“litter layer”). Were litter layers excluded or included in the sampled soils? Furthermore, the comparisons of C stocks between riparian forests, tree plantations, and grasslands made by Säumel et al. (2023) need to be standardized for two the key factors in determining SOC – C inputs (net primary production) and soil texture (Parton et al., 1994; Schimel et al., 1994; Krull et al., 2001). Riverine forests have a completely different water regime than grasslands or tree plantations and, consequently, differences in net primary production. Alluvial soils are expected to have profound differences in soil texture compared with upland areas, and this will impact the SOC saturation level of the soil (Chung et al., 2008; Stewart et al., 2007; Mayzelle et al., 2014; Pravia et al., 2019). The saturation level is largely associated with texture, particularly with the fine-soil-particle fraction (Hassink, 1997; Feng et al., 2013). However, the aforementioned study ignores the well-known effects of texture on SOC.

2.1.6 Grassland (GL) categories

Säumel et al. (2023) stated that they “...subdivided GL plots according to the intensity of use: (i) undisturbed GLs (without grazing), (ii) partially grazed GLs (with sporadic grazing and low animal charge), and (iii) highly grazed GLs (with high animal charge)”. (Note that we assume that “animal charge” means stocking rate.) Some of the authors of this reply have been working on grassland ecology in Argentina, Brazil, and Uruguay for more than 35 years. We were particularly interested in identifying different grazing situations. Actually, we have compiled a set of ungrazed situations based on an extensive search (Lezama et al., 2014). There were very few sites available. Except for very particular situations, we found it extremely difficult to define the level of grazing intensity on commercial ranches, as such information is seldom recorded (see Lezama and Paruelo, 2022). Aside from how they were defined, it is not clear how the different “categories” of grassland entered into the analysis. However, in the results, the authors said that no differences were detected “among different GL subtypes”. Several local studies on paired grazed–ungrazed native grasslands have previously shown important changes in SOC stocks that varied according to soil types (Piñeiro et al., 2009, 2010). More recent studies have shown that belowground C inputs are heavily impacted by the grazing condition (grazed and ungrazed comparisons) (López-Mársico et al., 2023). Säumel et al. (2023) ignored the well-documented differences among native grazed or ungrazed grasslands with respect to species composition and vegetation structure. Furthermore, the grassland communities of Uruguay have been thoroughly described (Lezama et al., 2019) and mapped (Baeza et al., 2019) in detail, showing that the phytosociological units defined for the country are quite stable under different levels of grazing intensity and degradation (Altesor et al., 2019).

2.1.7 Some additional issues

Some other issues related to the analysis and conclusions preclude clear comparisons with previous studies and/or generalizations. Some examples of such issues are given in the following:

• a.

  According to Säumel et al. (2023), “Total P concentration was determined calorimetrically after microwave-assisted digestion with a Unicam spectrometer at a wavelength of 660 nm.” (Säumel et al., 2023, p. 429). The problem with this is that total soil P is not a fertility parameter, as it has a low correlation with P availability.

• b.

  Säumel et al. (2023) also states that “The pH of our topsoil samples are [sic] mainly in the category of very strongly to extremely acidic and is lowest in TPs (Fig. 6), below the means reported so far (Jobbágy and Jackson, 2003; Céspedes-Payret et al., 2012).” (Säumel et al., 2023, p. 433). The issue here is that the authors measured pH in CaCl₂ (“Acidity was measured by adding calcium chloride (0.01 M) to the samples at a 2.5:1 proportion, and after shaking and 2 h rest, read with a pH meter (HI2550 meter, Hanna Instruments, USA).” (Säumel et al., 2023, p. 427)), whereas Céspedes-Payret et al. (2012) and Jobbágy and Jackson (2003) measured it in water; therefore, the results are not comparable. The pH being measured in water extractions is more common or standard lab analysis in Uruguay (Hernández et al., 2016; Beretta-Blanco et al., 2019; Grahmann et al., 2020) and results in higher values than pH measured with CaCl₂.

• c.

  We found the conclusions related to the role of riverine forest soils as a sink for trace metals extremely speculative. This kind of analysis must be performed at the catchment level. No evidence is provided on the location of the data reported. Do they correspond to the same basin? Are they physically connected?

2.2 Misleading interpretations and their consequences

2.2.1 Recommendations that go against grassland conservation

Two recommendations that the authors made in the discussion go against grassland conservation:

• a.

  the conversion of grasslands into silvopastoral systems;

• b.

  the expansion of native forests and the use of native species in tree plantations.

There is profuse evidence that planting trees in open ecosystems, such as Uruguayan grasslands, is not a solution with respect to ecosystem restoration or conservation (Veldman et al., 2015, 2019; among others), although this evidence goes against popular beliefs, particularly originating from countries originally covered by native forests.

2.2.2 Generalizations that may lead to some serious misinterpretations

The article included some generalizations that may lead to some serious misinterpretations:

• a.

  Säumel et al. (2023) state that “Our topsoil data indicate that carbon sequestration occurs mainly in the topsoils of native riverine forests that cover less than 5 % of Uruguayan territory.” (Säumel et al., 2023, p. 435). We think that the aforementioned authors cannot state that SOC sequestration occurs mainly in the topsoil because (i) they did not measure SOC stocks or bulk density; (ii) they cannot relate a non-paired, observational study to cause–effect processes (no checking of same soil type besides CONEAT, which includes several soil types); and (iii) they did not sample below 10 cm. Moreover, SOC accumulation in riverine areas may result from erosion (natural or anthropic) of SOC formed in upland soils and, therefore, correspond to a spatial reallocation of SOC.

• b.

  Säumel et al. (2023) also report that “Organic carbon content and the exchangeable cations are strongly reduced in the topsoils of GLs, TPs and AC compared to NFs (Figs. 4b, d–h and 5b, d–h).” (Säumel et al., 2023, p. 433). As we stated before, the experimental design does not allow one to evaluate reductions or changes in soil cations, as it is an observational study without any explicit control of the other forming factors (in particular, parental material and topography that widely differ among Uruguayan soils) that would allow one to use a space-for-time substitution approach necessary to relate observed differences to land use changes.

2.2.3 The discussion links the agricultural production system with widespread soil degradation in Uruguay

The discussion of Säumel et al. (2023) starts linking the agricultural sector of Uruguay with “Socioeconomic and conventional management practices that drive soil degradation” and the generation of “inputs trap” and “credit or poverty trap”. Even though the characteristics, practices, and structure of the agricultural sector of Uruguay are open to criticism and debate, the article presents no data or evidence to start a discussion about this issue. Aside from the intention of the authors, such a comment at the beginning of the discussion may be interpreted as the characterization of the agricultural sector of a country of the Global South by the developed Global North. The general impression of an independent reader is that soil degradation is widespread in Uruguay, which is not the case; in fact, Uruguay is the country in the region with the highest area under natural grasslands (Baeza et al., 2022). Moreover, this type of “scientific evidence” on the poor environmental performance of South American countries, spread by scientists of European countries (see i.e. Kehoe et al., 2020), helps to build non-tariff barriers for primary products and provides excuses to set conditions in international trade agreements. Nevertheless, we strongly agree that Uruguay and other South American countries have major environmental problems. Most of the authors of this reply have been and still are involved in documenting, modifying, proposing solutions, and/or generating policies in our region, including Uruguay, Argentina, and Brazil (Staiano et al., 2021; Baeza et al., 2022; Overbeck et al., 2022; Paruelo et al., 2022; Gallego et al., 2023; Baldassini et al., 2023; among others). We are also involved in identifying the underlying causes of the environmental problems in the Global South. National debts, the lack of commitment of developed countries with environmental agreements, nature commodification, land grabbing, and the role of multinational financial markets in the agricultural sector are some of the factors promoting land use and land cover changes and degradation as well as setting limits on country-level policies. Considering all of the above, we want to stress the risks of simplifying a complex problem that involves a myriad of actors and factors, based on what we believe is not solid scientific evidence.