Over the past years, several remote sensing maps of land cover have been produced, but they still exhibit certain differences compared to the real land use that reduce their value for climate and carbon cycle modelling as well as for national estimates of forest carbon stocks and their change. This paper outlines a straightforward framework for evaluating map accuracy and estimating uncertainty in land cover area, specifically for forest-related land cover maps in Poland for the year 2018. The study compares stratified field-based data from the National Forest Inventory (NFI) with remote sensing data on forest variables, at the pixel level, in order to identify suitable methods for accuracy and area uncertainty estimation. Additionally, the paper introduces and presents a variety of accuracy metrics applicable to assess overall uncertainties in GHG inventories. The results indicate that the High-Resolution Layer Forest Type (HRL FTY) product (part of the broader Copernicus Land Monitoring Service [CLMS] portfolio), assessed using NFI field-based information, achieved an overall accuracy (OA) of 69.2%. This metric varies among particular nature protection forms, with the highest observed ones in Natura 2000 sites of 70.45%. The primary source of map errors was associated with distinguishing between broad-leaved and coniferous forest areas. Improving future maps necessitates more precise differentiation between species to better support national forest monitoring systems for the purpose of greenhouse gas (GHG) inventories where information on the spatial distribution and variability of forests sources, biodiversity assessment, threat prevention, estimation of carbon content is becoming an important part of the associated reporting system.
Keywords: remote sensing; Copernicus Land Monitoring Service; broad-leaved; coniferous; National Forest Inventory; GHG inventory; land use; land use change and forestry; accuracy metrics; uncertainty
The reporting on national greenhouse gas (GHG) inventories, covering among other aspects emissions and removals from the category of land use, land use change and forestry (LULUCF) for the European Union's Member States is based on requirements resulting from the EU mechanism for the monitoring and reporting of GHG emissions [[
With the adoption of the so called "LULUCF regulation" in 2018 [[
Generally, most of the LULUCF GHG inventories are based on national forest and land statistics. In order to improve the quality of GHG emission and removal data, the updated LULUCF regulation [[
The comprehensive analysis of spatial and thematic content of the Copernicus Land Monitoring Service (CLMS) products in respect to current and future obligations towards national accounting and reporting of GHG emissions and removals becomes more and more important. The CLMS, particularly concerning the pan-European products, are recently under the scope of verification and assessment on whether and how they can support the future reporting of the category LULUCF at the country level. However, some discrepancy may occur when identifying objects on maps and arranging them in a three-dimensional space. In turn, accuracy refers to the correct interpretation of the reported value to the value accepted as real for a given phenomenon. Similar attempts consisting of an application of CLMS products for the purpose of GHG inventories in the LULUCF sector could be found at [[
The pan-European and the local CLMS components are coordinated by the European Environment Agency (EEA). They offer information services based on satellite Earth observation and in situ (non-space) data. These information services are freely and openly accessible to users through six thematic Copernicus services: atmosphere monitoring, marine environment monitoring, land monitoring, climate change, emergency management and security. Broader information on about the thematic scope of CLMS products can be found in the [CLMS portfolio].
Furthermore, several key European Union (EU) policies and regulations are aimed at protecting forests (e.g., Forest Strategy, Timber Regulation, Nature Directives) and the High-Resolution Layer Forest Type (HRL FTY) product could be considered as a key tool in the completing of these missions. This product uses the Food and Agriculture Organization's definition of forests to filter out non-forested but tree-covered areas from the High-resolution layer dominant leaf type (HRL DLT) product. Due to the fact that the HRL FTY assumes participation in a number of tasks, the quality of data contained therein is considered vital.
This section captures an overview in terms of definitions and product specifications for the high-resolution layer (HRL) Forest products as part of the CLMS, coordinated by the EEA. The HRLs are currently produced in regular 3-year intervals at a 10–20 meter spatial resolution for 39 European countries (EEA 39). Evolving scientific developments and user requirements are continuously analysed in a close stakeholder interaction process with the European Entrusted Entities (EEE), targeting a future pan-European roll-out of new/improved CLMS products and assessing transferability to global applications [[
For example, according to product documentation [[
Of note, the HRLs were designed for use by a broad user community as a basis for environmental analyses and for supporting political decision-making. It is noted that they can serve as a support in the reporting on LULUCF from 2021 onwards for more frequent monitoring of land areas and land cover changes. Nevertheless, LULUCF reporting requires the use of IPCC compliant LC/LU class definitions, which at the moment, are not fully met through the HRLs [[
Within a wide group of a forest-related HRL portfolio for the year 2018, a number of secondary layers have also been prepared. One of them, namely the HRL FTY product (in 10 m and aggregated 100 m resolution), was prepared taking into account elements of the FAO's forest definition. It should be underlined that TCD and DLT do not consider any of the forest definitions, so the application of filtering enables users to adapt the existing TCD and DLT products to their forest definition (if different from the FAO definition). Furthermore, based on the TCD and DLT, two change products exist, providing information on the gain and loss of tree cover and its associated leaf type between the reference years of HRL 2015 (2016) and HRL 2018 in 20 m resolution. Those are the Copernicus High-Resolution Forest Layer Tree Cover Change Mask (TCCM20m) 2015–2018 and DLTC (DLTC20m). The TCCM 20m raster product provides information on the change between the reference years 2015 and 2018 and consists of four thematic classes (unchanged areas with no tree cover; new tree cover; loss of tree cover; unchanged areas with tree cover) at a 20 m spatial resolution. DLTC20m raster product provides information on the change between the reference years 2015 and 2018 and consists of 9 thematic classes, of which there are 5 change classes (new broadleaved cover; new coniferous cover; loss of broadleaved cover; loss of coniferous cover; and potential change among DLTs). Taking into account all of the above, the portfolio of commonly recognised primary and secondary status layers and change forests related CLMS products consists of the following:
- – Dominant Leaf Type 10 m (DLT 10m);
- – Tree Cover Density 10 m (TCD 10m);
- – Forest Type 10 m (FTY 10m);
- – Tree Cover Change Mask 20 m (TCCM 20m); and
- – Dominant Leaf Type Change 20 m (DLTC 20m).
In regard to HRL FTY product data application, its use allows one to get as close as possible to the FAO forest definition. However, in many cases (when MMU differs from the minimum area of forest definition) potential application will result in a significant underestimation in terms of the domestic area representativeness of the forest class within the area from 0.1 to 0.5 ha. HRL FTY with its original (10 m (2018) / 20 m (2012, 2015)) resolution is only including two products: 1) a product that has a MMU of 0.5 ha, as well as has a 10% TCD threshold applied, and 2) a support layer that maps, based on the DLT product, trees under agricultural use and in an urban context (derived from Corine Land Cover and imperviousness 2009 data). For the final 100 m, product trees under agricultural use and urban context from the support layer were removed. A recently available 10 m 2018 reference year FTY product has the agricultural/urban trees removed. The Forest Type product is also available as an aggregated version in 100 m spatial resolution, fully aligned to the EEA 100 m reference grid.
Contrary to the TCD and DLT products non-forest trees are excluded in HRL FTY following the forest definition of the FAO [[
- Includes: forest nurseries and seed orchards that constitute an integral part of the forest; forest roads, cleared tracts, firebreaks and other small open areas < 0.5 ha and/or < 20 m width. Forest in national parks, nature reserves and other protected areas such as those of specific scientific, historical, cultural or spiritual interest; windbreaks and shelterbelts of trees with an area of more than 0.5 ha and a width of more or equal than 20 m; plantations primarily used for forestry purposes, including cork oak stands.
- Excludes: land predominantly used for agricultural practices. In this sense, fruit trees and olive groves are also excluded. Gardens and urban parks are also not considered as part of the forest.
The 10 m Forest Type product has been produced externally by applying a "forest" definition, mostly following the FAO definition, whereas tree cover in traditional agroforestry systems such as Dehesa/Montado is explicitly included for EEA purposes. The product is derived through a spatial intersection of the two primary status layers TCD and DLT and excludes areas under agricultural use and in urban contexts as provided by the FADSL (Forest Additional Support Layer). The Mediterranean wooded pasturelands known as "DEHESA" in Spain and "MONTADO" in Portugal are agroforestry systems of high natural and cultural value (HNCV) that cover around 3.5 million hectares of the south-western Iberian Peninsula, where they are the main land use systems [[
- 10 m spatial resolution.
- TCD range of ≥10–100%.
- MMU (minimum number of pixels to form a patch) of 0.5 ha (50 pixels); applicable both for tree-covered areas and for non-tree-covered areas in a 4-pixel connectivity mode, but not for the distinction of DLT within the tree-covered area for which no such minimum is set. The potentially available leaf type information for areas below 10% density within non-forest patches smaller than the MMU is explicitly kept from the pixel-based DLT products to ensure consistency.
- MMW of 10 m.
Study area for the thematic accuracy assessment encompasses of land classified in Poland as a "forest" according to Art. 3 of the [[
The spatial arrangement of forest habitats is largely mirrored in the spatial composition of prevailing tree species. With the exception of mountainous regions, where stand composition is characterized by either spruce in the west or a mix of spruce and beech in the east, and a handful of other areas featuring diversified species structures, the predominant species in most of the country's stands is pine. The diversity of growing conditions for forests in Poland is associated with the allocation of natural-forest habitats, as illustrated in Figure 1.
Graph: Figure 1. Species structure of dominant tree species (as of 1 January 2018)Source: based on Forestry in Poland 2019
In terms of forest area, coniferous species dominate Polish forests, encompassing 68.6% of the total forested area. Poland's favourable climatic and site conditions within the Euro-Asiatic natural range have led to the development of several important ecotypes of pine. Pine forests account for 58.1% of the total forest area across all ownership categories, with 60.8% in State Forests and 55.3% in privately-owned forests. It's worth noting that since 1945, there have been significant changes in the species composition of forests, including a notable increase in the proportion of broad-leaved tree stands. When considering state forests, where this trend can be tracked through annual updates of forest land area and timber resources, the total area covered by broad-leaved stands has risen from 13% to 31.4% [[
Graph: Figure 2. Structural changes of tree species by aggregated groups of dominant tree species for selected years between 2010 and 2021 (as of 1 January)
Recent observations (Table 1; Table 2) indicate a decrease in the area of the youngest stands (age classes I and II) noticed for several decades, which may raise concerns about the desired structure of age classes [[
Table 1. General characteristics of forest area by forms of forest ownership in 2019
Total (of which): 9242.4 96.9 11.9 14.8 23.8 20.0 23.1 3.3 1. Public forests (of which) 7460.1 97.3 12.2 14.0 22.2 19.4 25.9 3.6 i) owned by the State Treasury 7375.9 97.3 12.2 14.0 22.2 19.4 25.9 3.6 ii) managed by the State Forests 7110.0 97.4 12.5 14.2 22.2 19.4 25.5 3.6 iii) national parks 185.6 94.9 3.0 8.2 19.6 17.7 40.2 6.2 iv) stock of Treasury Agricultural Property 27.6 89.1 3.7 14.5 24.5 28.7 17.7 - v) municipally owned 84.2 96.5 6.5 10.7 21.2 20.9 31.9 5.3 2. private forests 1782.3 95.1 10.7 18.2 31.3 22.3 10.9 1.7
1 Source: Forestry in Poland 2019
Table 2. General characteristic of forest area by forms of forest ownership in 2022
Total (of which): 9260.3 97.2 11.5 14.2 21.5 21.6 24.4 4.0 1. Public forests (of which) 7474.6 97.5 12.0 13.6 19.9 20.5 26.9 4.6 i) owned by the State Treasury 7390.6 97.5 12.1 13.6 20.0 20.3 26.9 4.6 ii) managed by the State Forests 7121.4 97.6 12.4 13.8 20.1 20.5 26.3 4.5 iii) national parks 183.5 95.0 1.9 7.3 15.2 16.0 45.9 8.7 iv) stock of Treasury Agricultural Property 27.6 91.5 4.1 11.6 23.7 29.5 22.6 - v) municipally owned 84.0 97.6 4.1 9.7 18.0 25.1 34.5 6.2 2. private forests 1785.7 95.8 9.4 16.5 28.1 26.4 13.6 1.8
2 Source: Forestry in Poland 2022
The consequence of the reduction in the level of felling is an increase in the area of older stands (Table 2). Keeping mature stands for felling for too long may cause depreciation of the wood raw material and increase the risk of damage caused by abiotic factors. Observed changes in the age structure of forests, beyond their quantitative implications in terms of volume changes, can significantly impact the accuracy of remote assessments of forest resources, particularly in the qualitative dimension.
The imperative for large-scale forest inventories is notably derived from [[
The National Forest Inventories (NFIs) are conducted by The Bureau of Forest Management and Forest Geodesy based on a contract with the General Directorate of State Forests, marking the 4th cycle of the State Forest Inventory, covering the period from 2016 to 2020. This initiative builds upon the efforts of previous cycles spanning from 2005 to 2019.
The main objective of the historical NFI (carried out in forests of all forms of ownership) was to assess the overall forest condition and its evolution on a large scale. The inventory was designed to provide reliable information on the forest, in particular on species' structure, age, health status and the presence of damage. Measurements and observations were made on permanent sample plots in a grid pattern. The basis for determining the gridding pattern of these plots was the system of the ICP Forest utilised for the assessment of damage in forests, consistent with the system in force in the European Union [Commission Regulation]. General information collected for each sample plot consists of, among other aspects: geographic coordinates, ownership type, land use category, topographic plot location, forest stand origin, stand dominant species and age, quality class, forest cover and density index, stand vertical structure, forest site type, felling and silviculture activities, general health condition, damage types and intensity. Additionally, the following information is also recorded: forest function, nature protection form, protection category and forest management type [[
Since nature protection forms have been considered an element in the data collection process, an assessment conducted considered an accuracy metrics calculation for particular forms of nature protection, including Natura 200 sites, national parks and nature reserves. Within the Natura 2000 sites, the following areas were considered collectively in line with the list of protection sites as stipulated by updated LULUCF Regulation [[
- Sites of Community importance adopted and special areas of conservation designated in accordance with Article 4 of Council Directive 92/43/EEC and land units outside of those which are subject to protection and conservation measures under Article 6(
1 ) and (2 ) of that Directive in order to meet site conservation objectives; - breeding sites and resting places of the species listed in Annex IV to Directive 92/43/EEC which are subject to protection measures under Article 12 of that Directive;
- the natural habitats listed in Annex I to Directive 92/43/EEC and the habitats of species listed in Annex II to Directive 92/43/EEC which are found outside sites of Community importance or special areas of conservation, and which contribute to those habitats and species reaching favourable conservation status under Article 2 of that Directive or which can be made subject to preventive and remedial measures under Directive 2004/35/EC of the European Parliament and of the Council;
- special protection areas classified under Article 4 of Directive 2009/147/EC of the European Parliament and of the Council and the land units outside of those areas which are subject to protection and conservation measures under Article 4 of Directive 2009/147/EC and Article 6(
2 ) of Directive 92/43/EEC in order to meet site conservation objectives; - land units which are subject to measures for the preservation of birds reported as not being in secure status under Article 12 of Directive 2009/147/EC in order to fulfil the requirement under Article 4(
4 ), second sentence, of that Directive to strive to avoid pollution or deterioration of habitats or fulfil the requirement under Article 3 of that Directive to preserve and maintain a sufficient diversity and area of habitats for bird species; and - any other habitats which the Member State designates for equivalent purposes to those laid down in Directives 92/43/EEC and 2009/147/EC.
A distinct assessment was carried out specifically for national parks and nature reserves. The potential overlap of various forms of nature protection could introduce a bias in the evaluation of remote sensing data. Nevertheless, considering the diverse functions of national parks and nature reserves, along with the various protective activities associated with them that result in variability in the species and age structure of protected forests, an analysis of the accuracy of the associated imagery in these areas was conducted. The objective was to potentially identify the impact of activities that differ between these forms of nature protection.
National parks were established with the primary goal of preserving biodiversity, resources, inanimate nature and landscape values. Their aim is to restore the proper condition of natural resources and components, and to recreate distorted natural habitats, plant habitats, animal habitats or fungi's habitats, in accordance with the Act on nature protection [Act on nature... 2004].
In contrast, nature reserves encompass areas preserved in their natural or slightly changed state. They include ecosystems, refuges and natural habitats, along with plant habitats, animal habitats and fungi habitats. Nature reserves also cover creations and components of inanimate nature distinguished by special natural, scientific, cultural or landscape values, as stipulated in the Act on nature protection [[
The ideal remote observation is supposed to be a representation of everything that exists in the real world. The term "accuracy" must always be taken as a relative term because some observations are more accurate than others. They can also be more accurate in certain aspects and not in others. In general, large-scale remote observations are expected to have a higher accuracy than those performed on a small-scale. Making a map to be as accurate as its parameter dictates is of vital importance. This section is intended to provide only some of the more common examples of remote sensing data accuracy statistics by comparing between plot and remote sensing data, sharing common spatial and temporal characteristics. Since the accuracy of particular remote collected data is unknown, no guide would be possible to formulate for reducing the uncertainty created when they disagree.
The only homogenous reference data set is the NFI. The NFI is a terrestrial random inventory at permanent sample plots that is conducted every five years (in 2019, the third NFI was completed). A total of 45,973 sample plots spread over Poland were considered in the assessment. Each sample plot is defined by a circle with a diameter of 150 m, where at each area stand and tree characteristics are recorded. The exact locations of the permanent NFI plots are not made public, to preserve the samples' anonymity. This spatially explicit dataset is valuable as it serves as testing data for high-resolution satellite data classification. The spatial linkage between the satellite data and plot information allows for establishing connections. An accuracy analysis was conducted on the dataset under study, ensuring the full alignment of HRL FTY data (including raster coding representing thematic pixel values for all non-forest areas, broadleaved forests, and coniferous forests) with NFI plot information characterizing NFI coding of the forest types (typ_dstan), namely for all temporally non-forested areas, broadleaved forests, and coniferous forests (Figures 3–10). Data interpretation was based on NFI plots, and aggregation was carried out according to a simplified rule used for layer evaluation, which is based on aggregation including valid pixels, but without taking into account SWF objects (in some cases considered forests), because the study focuses specifically on areas already designated as forests.
Graph: Figure 3. HRL Forest Type layer at the national scale.
Graph: Figure 4. Random overview of intersecting NFI plots and HRL FTY imaginary
Graph: Figure 5. Overview of intersecting NFI plots and HRL FTY imaginary (example 1)
Graph: Figure 6. Overview of intersecting NFI plots and HRL FTY imaginary (example 2)
Graph: Figure 7. Overview of intersecting NFI plots and HRL FTY imaginary (example 3)
Graph: Figure 8. Overview of intersecting NFI plots and HRL FTY imaginary (example 4)
Graph: Figure 9. Overview of intersecting NFI plots and HRL FTY imaginary (example 5)
Graph: Figure 10. Overview of intersecting NFI plots and HRL FTY imaginary (example 6)
The accuracy of the HRL FTY 2018 data was evaluated by comparing (intersecting) information collected through measurements of sampling plots within the 3
A statistical validation was elaborated on based on a stratified systematic sampling approach with an area-weighted accuracy calculation. The uneven sampling resulting from the systematic stratified sampling method was reproduced by applying a weighting factor to each sample based on the ratio of the number of samples to the size of the stratum considered. It should be noted that inclusion probabilities of all sample units within a geographic stratum are equal. Importantly, a weighted error matrix estimator is required to account for the different inclusion probabilities among strata when combining sample data over several strata. What is important is that the estimation weight is the inverse of each sample unit's inclusion probability, and the proportion of area for each cell of the error matrix is estimated. Otherwise, the actual accuracy of the maps might be overestimated or underestimated. Overall accuracy (OA) and class specific accuracies (user and producer accuracy) are computed for all thematic classes from the weighted sampling probability-corrected confusion matrices (Table 3; Table 4; Table 5; and Table 6) for points classified into class 0–2 in HRL FTY and validated in NFI classes 0–2, and 95% confidence intervals are calculated for each accuracy. The summary of the results of validation of HRL FTY 2018 with the NFI data is given below.
Table 3. General confusion matrix of HRL FTY with the NFI plots (0 = non-tree covered areas, 1 = broadleaf dominated forest, 2 = coniferous dominated forest)
2006 3287 6437 1178 3707 15939 1270 2701 23597 3592 15761 26620
Table 4. Confusion matrix of HRL FTY with the NFI plots – Natura 2000 areas only (0 = non-tree covered areas, 1 = broadleaf dominated forest, 2 = coniferous dominated forest)
245 358 720 88 461 1944 108 316 2670 313 1956 3065
Table 5. Confusion matrix of HRL FTY with the NFI plots—nature reserves only (0 = non-tree covered areas, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
27 24 55 6 201 55 262 4 33 149 186 14 261 228
Table 6. Confusion matrix of HRL FTY with the NFI plots—national parks only (0 = non-tree covered areas, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
43 60 124 26 75 335 22 69 424 69 346 468
The diagonal elements highlighted in bold of the confusion matrix represent observations that were correctly classified in HRL FTY. These are indicative of the accuracy of our classification. In the conducted analysis, a comprehensive comparison was made using data from all 45,973 NFI plots. The results of the comparison at the country level showed full compliance for 1,144 NFI plots identified as areas without tree cover, 11,054 NFI plots with a predominance of broad-leaved species, and 19,626 NFI plots with a predominance of coniferous species.
Within the Natura 2000 sites analysed in detail, a comprehensive comparison was made using data from 5,334 NFI plots exclusively. In this case, both HRL FTY areas and NFI sample plots intersecting with Natura 2000 sites were the only ones considered. The intersection exercise utilises an archived data set of Natura 2000 sites boundaries – version 2018 [[
Within the nature reserves' areas, a comprehensive comparison was made using data from 503 NFI plots exclusively. In this case, both HRL FTY areas and NFI sample plots intersecting with nature reserves' sites were the only ones considered. The intersection exercise utilises an archived data set of nature reserves' sites boundaries [Nature Reserves reference data]. The results of the comparison showed full compliance of HRL FTY with 4 NFI plots identified as areas without tree cover, 201 NFI plots with a predominance of broad-leaved species, and 149 NFI plots with a predominance of coniferous species.
Within the national parks' areas, a comprehensive comparison was made using data from 883 NFI plots exclusively. In this case, both HRL FTY areas and NFI sample plots intersecting with national parks sites were the only ones considered. The intersection exercise utilises an archived data set of national parks boundaries [National Parks reference data]. The results of the comparison showed full compliance of HRL FTY with 21 NFI plots identified as areas without tree cover, 234 NFI plots with a predominance of broad-leaved species, and 333 NFI plots with a predominance of coniferous species. Overview of accuracy statistics (overall and per age class of broad-leaved and coniferous fractions) of the comparisons are provided in the Table 7 and the following tables.
Table 7. OA of HRL FTY based on assessment of NFI plots and by age classes (n = number of sample plots, OA = overall accuracy, 0 = non-forest, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
2883 279 6 44 3970 308 19 68 3615 323 25 62 3359 364 25 55 3647 408 34 72 5160 576 47 99 4561 457 30 92 2954 337 34 54 3075 381 33 66 2148 302 41 30 1477 248 36 43 980 193 32 18 1315 349 79 43 565 NA NA NA 17.69 16.31 7.27 15.57 48.70 45.16 50.00 43.18 69.60 74.68 89.47 58.82 75.30 77.40 72.00 75.81 79.19 83.24 84.00 74.55 81.71 80.64 85.29 83.33 81.47 79.86 61.70 77.78 82.92 80.74 70.00 80.43 84.09 85.16 79.41 85.19 84.98 88.71 75.76 84.85 82.27 81.79 41.67 73.33 78.74 79.03 75.00 72.09 79.49 82.90 84.38 77.78 59.47 56.73 28.57 60.00 59.47 56.73 28.57 60.00
The OA is a measure informing the proportion of correctly mapped sites out of all the reference sites [[
OA is straightforward to calculate and understand, offering basic accuracy information to both the map user and producer [[
Table 8. UA of HRL FTY based on the Polish NFI sample plots (UA = user's accuracy, 0 = non-forest, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
31.85 37.38 28.57 30.43 70.14 71.32 77.01 67.63 73.73 73.28 65.35 71.15
The UA (Table 8) and PA (Table 9) for a given class are typically not the same. In the example above, the PA for the class of broad-leaved forests is 69.35%, indicating that only 69.35% of the reference for broad-leaved forests have been correctly identified as such by the classification. On the other hand, the UA for the same class is 70.14%, meaning that of all the areas classified as "broad-leaved forests," only 70.14% actually correspond to areas with the predominance of broad-leaved species. In other words, the classification system has a higher tendency to correctly identify non-tree covered areas among all the reference sites, but when it identifies an area as "non-tree covered," it is not as accurate, and there is a higher chance that the identified area may contain trees. These metrics are valuable in assessing the performance and reliability of a classification model for specific classes within the dataset. They provide more nuanced information than the OA, allowing users to understand the strengths and weaknesses of the classification results for individual classes.
Table 9. Producer's accuracy (PA) of HRL FTY based on the Polish NFI sample plots (PA = producer's accuracy, 0 = non-forest, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
17.77 16.25 7.27 16.94 69.35 71.76 76.72 69.85 83.17 84.12 80.11 78.54
In an accuracy assessment, the EO as provided in Table 10, indicates a quantitative measure of how well a classification system detects relevant classes. Specifically, EO is expressed as a percentage and represents instances where the system fails to identify or include certain elements that should have been identified. This metric is crucial in assessing the performance of classification systems, especially in fields like remote sensing, where accurate identification of features is essential for reliable data interpretation and decision-making.
Table 10. HRL FTY EO based on the Polish NFI sample plots (EO = error of omission, 0 = non-forest, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
82.23 83.75 92.73 83.06 30.65 28.24 23.28 30.15 16.83 15.88 19.89 21.46
Table 11. HRL FTY EO based on the Polish NFI sample plots (EC = error of commission, 0 = non-forest, 1 = broadleaf dominated forest and 2 = coniferous dominated forest)
68.15 62.62 71.43 69.57 29.86 28.68 22.99 32.37 26.27 26.72 34.65 28.85
The reference data applied in the accuracy assessment could not have been considered comprehensively. Although the HRL Forest with the Forest Type product already provides one type of forest product following a forest definition that could be applied in this case, the MMU (minimum number of pixels to form a patch) for the HRL FTY of 0.5 ha (50 pixels) derived through a spatial intersection of the two primary status layers TCD and DLT, as already mentioned in the assessment, is resulting in a significant underestimation in terms of the area representativeness of the forest class within the area from 0.1 to 0.5 ha. To fully represent the forest class in Poland, taking into account that the HRL FTY does not provide relevant forest products strictly following Polish domestic forest definition, supplementation by the information as contained in the HRL SWF should be further explored. The issue of domestic forest class area representativeness could be fixed by consideration of harmonized information on linear structures such as hedgerows, as well as patches (100 m
The accuracy requirements of each HRL product were subject to an internal validation. The accuracies achieved were 90% for PA (commission) and 90% for UA (omission). We have noticed significant differences in overall areas addressing particular classes within HRL Forest of the Forest Type. The OA assessed for the HRL FTY with the NFI data is equal to 69.22%. There are no significant differences between the OA statistics associated to analysed nature protection sites. Within the following nature protection forms of Natura 2000, nature reserves and national parks' OA were calculated to 70.45, 70.38 and 66.59%, respectively.
By applying simple Pearson's correlation [[
Graph: Figure 11. The Pearson's correlation between number of NFI samples ((n) NFI plots) validated and OA of HRL FTY
In terms of the classification system, particularly in the context of estimating forest types, some discrepancies have been identified between remotely sensed data and ground observations. The OA of the HRL classification ranges from 17.69% (for non-tree covered areas at the general level) to 84.09% (for the age class as of the ages between 71–80) as indicated in Table 7. This suggests variability in the accuracy of the system with the age dependency appearing to be influenced by the age of the forest. Older forests (especially at the age of over 40 years), which are typically more widely represented (Table 1; Table 2), tend to have a higher OA compared to younger forests. This metric is slightly reduced at the age of over 100 years (Table 7). Furthermore, Table 7 contains detailed numerical data supporting the observed tendencies for different forest age classes and nature protection forms where similar observation has been noticed.
Of note, the precision standards for each HRL product underwent internal validation conducted by the data providers, aiming for 90% accuracy in PA (commission) and 90% in UA (omission). An observation revealed substantial discrepancies in specific class areas within the HRL FTY. The OA appraised for the HRL FTY using the NFI data stands at 69.22%.
An in-depth analysis of the accuracy of the national HRL FTY layer indicated that on a national scale, broad-leaved stands exhibit higher commission and omission errors than expected within internal validation (10% in both cases), reaching 29.9% and 30.6%, respectively. Similar patterns emerge when considering Natura 2000 sites and National Parks only. For Natura 2000 sites, broad-leaved stands are linked to larger commission and omission errors, ranging from 28.9% to 32.4% and 28.4% to 30.1%, respectively. Notably, while nature reserve sites exhibit a higher EO (20.3%), the EC for broad-leaved stands is lower than for coniferous stands (23.0% and 34.6%, respectively).
The elevated omission errors concerning broad-leaved stands primarily stem from underestimating forest plantations and nurseries (correctly reflected in NFI measurements). Conversely, the relatively higher error in overestimating broad-leaved stands is attributed to an inclination to overrate broad-leaved forests at the expense of coniferous forests. These observations align with similar findings presented by [Mirończuk A et al. 2020] while assessing the primary layer of HRL DLT 2018. The EC related to broad-leaved forests predominantly results from misclassifying small coniferous forest patches as broad-leaved. Generally, ECs occur more frequently in broad-leaved forests than in coniferous ones, where in many instances, coniferous forest patches were misidentified as broad-leaved forests.
In general, the area covered by broad-leaved forests is exaggerated, and in the author's opinion, is affected by the visibility and composition of the forest underneath layers (second floor of forests and understory affecting general species composition of tree stands) on remote observations. Considering the expected significant changes in species structure between coniferous and deciduous species (as indicated by the stable trend in Fig. 2) and acknowledging the substantial predominance of coniferous species in Poland's total forest area (as depicted in Fig. 1), investigation of the issue of overestimating deciduous species requires further scrutiny to enhance accuracies in subsequent iterations of CLMS products.
- CLMS Copernicus Land Monitoring Service
- DLT Dominant leaf type
- DLTC Dominant leaf type change
- EC Error of commission
- EEA European Environmental Agency
- EO Error of omission
- ESA European Space Agency
- ETRS European Terrestrial Reference System
- EU European Union
- FAO Food and Agriculture Organization
- FTY Forest type
- GHG Greenhouse gas
- HRL High-resolution layer
- ICP International Co-operative Programme on assessment and monitoring of air pollution effects on forests
- LAEA Lambert azimuthal equal area
- LC Land cover
- LU Land use
- LULUCF Land use, land use change and forestry
- MMW Minimum mapping width
- MMU Minimum mapping unit
- MS Member state
- NFIs National Forest Inventories
- OA Overall accuracy
- PA Producer's accuracy
- TCD Tree cover density
- TCCM Tree cover change mask
- UA User's accuracy
- UNECE United Nations Economic Commission for Europe
- UNFCCC United Nations Framework Convention on Climate Change
By Marcin Żaczek; Mariusz Walęzak; Anna Olecka; Sylwia Waśniewska and Anna Paczosa
Reported by Author; Author; Author; Author; Author
