Abstract
In partial fulfillment of a grant to assess the potential impact of emerald ash borer on Minnesota, U.S., community forests, six communities were selected in 2009, and eight communities were selected in 2011, to complete tree surveys or inventories. Trained volunteers in each community were used to identify, measure, and assess their community trees. Training methods, technical assistance, and measurement tools utilized were updated between 2009 and 2011 based on input from community volunteers and university training staff, allowing for a post hoc study of volunteer efficacy to be conducted. To assess volunteer efficacy and the effect of updated training protocols on data quality, comparisons between volunteer-collected data and university-collected data were analyzed for agreement in genus and species identification, tree measurements, and condition rating for a subsample of trees in each community. Agreement was the greatest for tree identification at the genus level (>90%) and the lowest overall for condition rating (<70%) for all communities. Statistically differences between the 2009 and 2011 communities were detected with 2011 communities having higher levels of agreement on average. The increased probability of agreement with university researchers is likely attributable to increased focus on field-instruction, technical assistance, and more sophisticated tools used by the 2011 communities. However, detailed volunteer demographic data for each community was not available for analysis and could provide further insight into differences detected. Decisions to use volunteer collected data should incorporate appropriate levels of training and tool sophistication for the level of specificity required for a project.
- Citizen Science
- Community Involvement
- Emerald Ash Borer
- Minnesota
- Tree Inventory
- Urban Forest Inventory
- Urban Forest Volunteers
- Volunteer Data Accuracy
- Volunteer Data Quality
Effective urban forest management is dependent on a community’s ability to develop, implement, and sustain the activities necessary for realizing the social, environmental, and economic benefits of the urban forest. Information pertaining to the size, condition, diversity, and stocking level of city trees should form the basis for rational decision making related to the management of urban forests. However, the costs of collecting data fundamental to management can be prohibitive for communities lacking the necessary financial and staffing resources to carry out urban forest inventories (Kenney et al. 2011). Increasingly, urban and community forestry agencies and local governments are relying on volunteers to carry out programmatic goals. And while using volunteers can add much to a program, the mismanagement of a volunteer resource can lead to problems (Ball 1986). Appropriately managed community volunteers and organizations can support and increase the ability of municipalities to plan and implement activities that maintain the function of urban forests (Elmendorf et al. 2003; Jack-Scott et al. 2013). Volunteer-driven inventory or survey initiatives can support management of the urban forests, while also providing indirect benefits, such as increased community engagement and empowerment, advocacy, knowledge, and skill development (Bloniarz and Ryan 1996; Cozad et al. 2006).
The use of volunteers or citizen scientists to aid in monitoring, maintenance, and stewardship programs has increased in areas of natural resource management. Government programs in Minnesota, North Carolina, Ohio, Connecticut, Virginia, and Washington, U.S., have taken advantage of volunteer networks to assess and track water-quality trends in lakes, rivers, and streams (Heiskary et al. 1994; Penrose and Call 1995; Fore et al. 2001; Engel and Voshell 2002). Similar efforts drawing upon volunteers have been used to monitor for the presence of invasive species as part of early detection and response initiatives across the U.S. (Brown et al. 2001; Brandon et al. 2003; Delaney et al. 2007; Crall et al. 2011).
Other volunteer programs have focused on increasingly complex tasks, such as biodiversity assessment or species dynamics in ecosystems (Engel and Voshell 2002; Foster-Smith and Evans 2003; Leslie et al. 2004; Gillet et al. 2012). Volunteer programs have also been used to inventory, assess, and monitor forest systems at both a state and local level (Rock and Lauten 1996; Brandon et al. 2003; Galloway et al. 2006; Crall et al. 2011).
Although the use of volunteers is a widely accepted practice for the management of natural resources, it is not without skepticism as to data quality. Organizations and communities seeking to gather information on the state of a resource rely on high-quality data that is both accurate and useful. Increasingly, evidence supports the notion that volunteers can collect data of similar quality to that of professional scientists (Penrose and Call 1995; Rock and Lauten 1996; McLaren and Cadman 1999; Brown et al. 2001; Fore et al. 2001; Nicholson et al. 2002; Engel and Voshell 2002; Delaney et al. 2007; Crall et al. 2011; Gillet et al. 2012).
Despite the wealth of research on volunteer efforts in the broader field of natural resource management (Rock and Lauten 1996; Brown et al. 2001; Brandon et al. 2003; Galloway et al. 2006; Crall et al. 2011), there are few studies examining the use of volunteers in urban forests. Bloniarz and Ryan (1996) assessed volunteer efficacy and reliability as part of an urban forest inventory initiative across two communities of suburban Boston, Massachusetts, U.S. More recently, efforts were made to assess the quality of data collected by volunteers as part of urban tree inventories in several cities across the upper Midwest, U.S., as well as Malmö, Sweden (Roman et al. 2017). Volunteers demonstrated a high level of aptitude in their ability to identify trees to genus and assess mortality status of trees (Roman et al. 2017). However, volunteer assessment of other parameters, such as crown transparency, wood condition, or maintenance needs displayed lower levels of agreement with assessments of the same trees collected by professional and more experienced scientists (Cozad et al. 2006; Roman et al. 2017).
The Fore et al. (2001) assessment of volunteer accuracy was based on the direct comparison of volunteer-collected data to data collected by professionals following the same sampling protocols and locations. Differences in levels of agreement between volunteers and professionals have been attributed to one or a combination of factors, including volunteer age (Delaney et al. 2007), duration and complexity of task (Penrose and Call 1995; Darwall and Dulvy 1996; Brandon et al. 2003; Foster-Smith and Evans 2003; Newman et al. 2003), and amount of training received (Fitzpatrick et al. 2009). Complex and varied environments can also affect the accuracy of volunteer-collected data (Foster-Smith and Evans 2003).
Perception of comfort or competency to reliably collect data has been linked to the complexity of tasks and training provided to volunteers (Brandon et al. 2003; Foster-Smith and Evans 2003). Appropriate levels of volunteer training and support can affect the level of data quality. When assessing ability to collect reliable data pertaining to bird species identification and count, training and assessment of volunteers with low to moderate skill levels proved critical (McLaren and Cadman 1999). Volunteers without adequate training generally could not complete complex and arduous tasks (Newman et al. 2003). However, provided sufficient amount of instruction and technical support, volunteers produced data comparable to that collected by experienced professional scientists even when the difficulty of the task was increased (Fore et al. 2001; Foster-Smith and Evans 2003; Gillet et al. 2012).
Data collection that requires the volunteer to assess subjective measures, as opposed to measurable and objective parameters, can contribute to unreliable results (Galloway et al. 2006); however, comparisons of levels of agreement between professionals can also differ widely (Bloniarz and Ryan 1996; Foster-Smith and Evans 2003). Limitations of equipment used by volunteer groups to collect certain parameters may also contribute to statistical differences between volunteer-collected and professionally collected data. However, when using comparable equipment, relatively small differences in levels of agreement occurred, which suggests volunteers can reliably collect accurate and useful data when provided with adequate tools (Nicholson et al. 2002).
Data quality, based on practical levels of agreement over statistical significance, is an important consideration when including community volunteers in the resource management process. Previous studies have shown that even certified, professional arborists can differ significantly from one another in their assessment of taxonomic identification, tree condition (Bloniarz and Ryan 1996), and tree appraisals (Ponce-Donoso et al. 2017). Statistically significant differences in measurement of tree crown metrics and trunk diameter have been reported for professionals, although the differences were typically under 2.54 cm for DBH measurements (McRoberts et al. 1994; Elzinga et al. 2005). In an examination of volunteer accuracy for street tree inventories across two different communities in suburban Boston, an 80% agreement level between street-tree data collected by volunteers and certified arborists was used as a reasonable threshold for agreement (Bloniarz and Ryan 1996). As variations in measurements among professionals is not uncommon, determining an acceptable value of variation or agreement in measurement will depend on management objectives.
Historically, the validity and usefulness of volunteer-collected data for use in natural resource management has been a common concern and source of scrutiny. As government agencies increasingly look to volunteer-generated data to support monitoring and management programs, quality of data is cited as a source of major concern among regulatory bodies and scientists (Penrose and Call 1995). This critique is primarily attributable to a lack of understanding regarding the potential for error, or bias, of volunteer-collected data (Dickinson et al. 2010).
While the existing literature provides evidence that trained volunteers can effectively collect monitoring and assessment data across a range of natural resource disciplines, research on the use of volunteers, as part of urban forest inventory or survey initiatives, is minimal. The first objective of this study was to determine the level of agreement between trained volunteers and experienced university researchers for different urban forest inventory tasks (i.e., species identification, tree measurements, and condition-rating). The second objective of this study was to determine the effect training method had on the probability of agreement between volunteer-collected and researcher-collected data.
MATERIALS AND METHODS
In 2009, six Minnesota communities were selected to participate in an emerald ash borer Rapid Response and Community Preparedness grant to assess the vulnerability of their urban forests to emerald ash borer infestations. Eight additional communities were selected in participate in 2011 and 2012. Communities were selected based on population, capacity to manage their urban forest, and location in the four primary ecological provinces as defined in the Minnesota Department of Natural Resources and the U.S. Forest Service Ecological Classification System (Minnesota Department of Natural Resources 2016). The first group of communities included: Crookston, Hendricks, Hibbing, Hutchinson, Morris, and Rochester. The second group included: Brainerd, Bemidji, Ely, Mankato, Mora, Royalton, Saint Cloud, and Starbuck.
The fourteen Minnesota communities took part in volunteer-led tree surveys or inventories to count, identify, and measure both publicly- and privately-owned community trees, and condition rate trees in public spaces. In each community, areas were selected using an urban street tree rapid-sampling protocol (Jaenson et al. 1992). Surveys of the sample areas were completed by community volunteers. The communities of Starbuck and Hendricks each completed a full inventory due to their small size. Volunteers were recruited through press releases in community newspapers, and solicitation of Minnesota Master Naturalists, Minnesota Master Gardeners, and Minnesota Tree Care Advocates through email lists. Volunteers ranged in age from 18 to 75, in level of education completed from some high school to doctorate, and in experience working with natural resources from no experience to professional experience. Volunteer demographics reflect all 14 communities and were not recorded separately for each community. No analysis of volunteer performance based on demographic information was possible.
Volunteer Training
Experienced research and outreach staff from the University of Minnesota, Department of Forest Resources, trained volunteers in each community. All volunteers were provided with training manuals written by University of Minnesota researchers specifically for the project, as well as custom-made field identification cards for the tree species most prevalent in the volunteers’ community. Volunteers in each community were trained to perform the same tasks: tree identification, measurement of trunk DBH in inches (to the nearest inch), and crown width (CRW) in feet (to the nearest foot), a quantitative condition rating of trees, proper use of inventory maps, completion of survey data sheets, and data entry in Microsoft® Excel®. Volunteers were also provided with safety instructions for working in their communities.
Volunteers were trained and instructed to identify trees to species when possible or to genus when they felt unable to properly identify species. Trunk DBH was measured using a measuring tape at 1.37 meters above ground level. Crown width was determined by measuring two crown radii from the tree drip line to the trunk, separated by 90 degrees (U.S. Forest Service 2017). The two radii were summed to provide an average crown width.
Condition-rating instruction focused on identification and assessment of nine different characteristics when evaluating the condition of a tree—four were exclusive to canopy condition, and five were trunk condition assessments. Volunteers were provided photo identification cards and manuals with images and descriptions of tree defects to aid in identification and assessment of condition. The condition-rating criteria were assessed and given a numeric score in quarter-point increments. Crown condition criteria included: stag-heading, tip dieback, crown symmetry, and live crown ratio. Trunk condition criteria included: cambium loss, presence of exposed and/or decayed wood, sprouts or suckers, stem cracks, and included branch unions. Scores were grouped into categories to provide a quantitative ranking of the tree’s condition.
Training in all communities began with classroom instruction focused on the basics of tree identification, concentrating on identifying tree species by their leaves, bark, fruit, and buds. Classroom instruction also covered measurement of DBH and CRW, tree condition rating, and how to properly record data and tree location on paper datasheets and maps. The classroom instruction was followed by field training and practice of the tasks covered during classroom training. Volunteers were required to collect data in groups of two or three. Training methods varied between the two groups of communities, but were consistent within the two community groups. Completed inventory datasheets and maps were mailed to the University of Minnesota and the spreadsheets were emailed to researchers for analysis.
Method One
Volunteers from the first group of six communities were trained over the spring and summer of 2010. For these communities, training consisted of approximately a one:one ratio of classroom instruction to field instruction. The training manual was 62 pages long, covering all training topics, and included few images.
Trunk DBH was measured using either a diameter tape and recorded as diameter or measured using a linear tape and recorded as circumference. Circumference measurements were converted to diameter when the data were entered into the spreadsheet.
Volunteers were taught to convert their walking pace to a linear measurement in feet during the field-training sessions. Researchers used a 15.25-meter tape, placed on the ground, and volunteers would walk along the tape to assess the average length of their individual pace. Crown width was then determined when a volunteer would walk from the trunk to the drip line, for the two crown radii, counting their paces as they walked. Volunteers recorded CRW as the sum of their pace converted to feet, based on their average pace determined during training.
At the end of training volunteers practiced field data collection with university researchers present to address issues and answer questions. After each community training session, volunteers scheduled their own survey times and teams, with little additional onsite involvement from university researchers during the data collection. Occasional requests for technical assistance were handled by researchers via phone or email.
Method Two
Training for the remaining eight communities was altered from the training of the first group of communities. Training methods, technical assistance, and training manuals were updated and refined based upon feedback and informal assessment from volunteers in the first group of communities. Training of the volunteers in the second group of communities took place during the summers of 2011 (Brainerd, Bemidji, Mora, Royalton, Saint Cloud, Starbuck) and 2012 (Bemidji, Ely, Mankato). The training manual was reduced to 25 pages and included more images and less text. The ratio of time spent in classroom instruction to field instruction remained the same, as did the training content and sequence. Key differences in training from the first group of communities included: updated DBH measurement tools, updated CRW measurement technique and tools, and an additional two-weeks of post-training technical assistance.
The second group of communities were provided with diameter tapes for the measurement of DBH. The CRW measurement technique was also updated. Volunteers in the second group were provided a 15.25-meter linear measuring tape to measure and sum two crown radii, from the trunk to the drip line, at 90 degrees from each other, to obtain an average CRW.
Post-training technical assistance consisted of trainers from the university research and outreach team joining volunteers for the first two weeks of field data collection to answer questions and provide support. At no time during the technical assistance did the university researchers collect data for the communities.
Agreement Assessment
Of the 14 communities that participated in the emerald ash borer Rapid Response Community Preparedness project, nine communities were selected for assessment of volunteer data. Two communities in first group, Crookston and Morris, were excluded due to high volunteer attrition and substantial portions of the survey completed by city personnel or University of Minnesota researchers.
Two communities in the second group, Mora and Royalton, were also excluded, because two full growing seasons had occurred since the initial volunteer survey. A final community, Bemidji, was excluded, as a large portion of the city’s urban forest suffered damage due to high winds after the volunteer survey had been completed, and a meaningful assessment of agreement was deemed unlikely.
Assessments occurred during the summer months of 2011, 2012, and 2013 (Table 1). For those communities where assessment was not possible in the same growing season, an increment borer was used to obtain a core sample. The core samples were measured and used to verify less than 2.54 cm of DBH growth had occurred since the volunteer measurements were recorded.
Details for each community: number of assessed trees, mean DBH and CRW of assessed trees, training method used, and the growing season year of the volunteer and researchers’ data collection. DBH and CRW were measured in centimeters and meters, respectively. Standard deviation is shown in parentheses.
Assessment Protocol
To assess and validate the data collected by community volunteers, data comparisons were completed by revisiting participating communities, identifying, measuring and condition-rating trees that were initially measured by volunteers as part of the community tree surveys. Measurements for DBH and CRW used diameter tapes and linear tapes following the protocol employed in Method Two for all nine communities.
A subsample of trees surveyed by volunteers in each community was randomly selected for assessment by university researchers. The community surveys were designed utilizing a stratified random sampling technique, and it was vital to maintain a similar sampling technique when determining which public trees to select for accuracy validation. Proportional sampling was used in conjunction with stratified, random sampling to provide statistically significant and desired representativeness of a population (Van Dalen 1979). Sample size for the assessment was determined using the following equation:
[1]
where s = required sample size, χ2 = the table value of χ2 for one degree of freedom, N = the population size, P = the population proportion, d = the degree of accuracy expressed as a proportion (Krejcie and Morgan 1970). The resulting sample size was approximately 90 trees in each of the nine communities.
Agreement between volunteer and researcher data was determined for each tree and each recorded metric. Agreement for tree identification was analyzed at genus and species level, except for the community of Starbuck, which identified trees only to the genus level and so no agreement data was available at the species level. Agreement thresholds were pre-defined for the DBH and CRW measurements with DBH ±2.54 cm and CRW ±1.5 m considered to be in agreement. Crown width thresholds were derived from predicted growth curves based on previous research for Minnesota urban trees (Frelich 1992; North 2013). Volunteer and researcher agreement for DBH and CRW measurements were analyzed using both the recorded field measurement and the pre-defined thresholds. The numerical values for condition-rating were binned into categorical groups and agreement was analyzed at the categorical value.
Agreement analysis included comparison of frequency counts and χ2 analysis for the comparisons between volunteers and university researchers by community and training method for each metric (Galloway et al. 2006; Crall et al. 2011). The χ2 analysis was performed using the gmodels (Warnes et al. 2015) package in R (R Core Team 2017). Logistic regression models were used to examine the effect of training method on agreement between volunteers and university researchers for metrics identified by χ2 analysis as significant. Statistical significance was set at a P-value of 0.05. All statistical analysis was completed using R version 3.3.3 (R Core Team 2017).
RESULTS
In total, 779 trees were assessed by university researchers in 9 Minnesota communities (Table 1). Table 2 illustrates the frequency of agreement between volunteers and university researchers for each inventory metric. For tree identification, agreement at the genus level was substantially greater than agreement at the species level for all communities. At the genus level, there was a 97% agreement compared to 64% agreement at the species level between volunteers and researchers regardless of community. Agreement between volunteers and researchers for DBH and CRW was 51% and 8%, respectively, regardless of community of training method (Table 2). The greatest frequency of agreement between volunteers and university researchers for DBH was Saint Cloud at 62%, and for CRW was Mankato at 21% (Table 2). Agreement in all communities increased for both DBH (86%) and CRW (62%) when agreement criteria were relaxed to include DBH ±2.54 cm and CRW ±1.5 m (Table 2). Condition-rating agreement, regardless of community, was 58%, with Saint Cloud having the highest frequency of agreement at 68% and Hendricks having the lowest at 41% (Table 2).
Frequency of volunteer and researcher agreement by community for the seven assessed inventory metrics. DBH and CRW were measured in centimeters and meters, respectively.
Tree identification agreement for all communities by genus and for species are detailed in Table 3. Acer and Fraxinus were the most abundant genera assessed, with genus-level agreement at 99% and 98%, respectively. Species-level identification agreement was lower for both Acer and Fraxinus than genus, 81% and 44 % respectively (Table 3). Agreement frequency for species identification was 67% (χ2 = 48.474, P < 0.001).
Frequency of volunteer and researcher agreement for tree identification of genera and of species within a genera. Species agreement with n/a indicates trees within that genus were only identified to the genus level. Where species and genus agreement percentages are identical, only one species for the given genus was observed in the communities.
Comparison of agreement frequency between volunteers and researchers based on training method are shown in Table 4. Training Method Two had a greater frequency of agreement for all assessment metrics, with the highest increased agreement frequency for CRW ±1.5 (18%) and condition (14%), and the lowest increase of agreement for genus (1%). Method Two had significantly higher frequency of agreement between volunteers and researchers than Method One for the DBH (P = 0.021), DBH ±2.54 (P = 0.013), CRW ±1.5 (P < 0.001), and condition (P < 0.001; Table 4).
Frequency of volunteer and researcher agreement by training method with χ2 and P-values. DBH and CRW were measured in centimeters and meters, respectively.
The probability of agreement for measurement DBH and DBH ±2.54 cm by approximately 1.4 and 1.7 times respectively, increased significantly for Method Two compared to Method One (Table 5). The probability of agreement for measurement of CRW ±1.5 m was increased by approximately 2.1 times, and condition-rating probability of agreement was approximately 1.8 times higher for Method Two compared to Method One (Table 5).
Logistic regression parameter estimates of agreement between volunteer and researcher for inventory metrics based on training method. Training Method One was set as the base level. Coefficient and odds ratio show the increase in agreement for volunteers trained using Method Two. DBH and CRW were measured in centimeters and meters, respectively. Standard errors are shown in parentheses.
DISCUSSION
Adequacy of data quality is a key distinction in assessing levels of acceptable accuracy of collected data. The U.S. Forest Service has measurement quality objectives (MQO) for genus, species, and DBH consistency rates; however, these are intended for professional field inventory crews who collect forest inventory field data as part of Forest Inventory and Analysis (FIA) projects (U.S. Forest Service 2017). Bloniarz and Ryan (1996) established both statistically significant and pragmatically acceptable levels of agreement between data collected by volunteers and professionals for urban forest inventories, with volunteer-collected data comparing favorably to experienced professionals, considering practical levels of agreement (Bloniarz and Ryan 1996). A similar differentiation between statistical significance and practical levels of agreement was considered in an analysis of volunteer accuracy in the Illinois ForestWatch program (Brandon et al. 2003). Mattson et al. (1994) supported the notion that volunteers could reliably and accurately produce data that was sufficient for purposes of detecting trends in environmental conditions. Despite falling short of the MQO set forth by the U.S. Forest Service FIA thresholds for urban forests (U.S. Forest Service 2017), the volunteers in the current study still met accuracy or exceeded thresholds set forth by earlier studies (Bloniarz and Ryan, 1996; Cozad et al. 2006; Roman et al. 2017).
Volunteers in communities who underwent training in Method Two had higher frequency of agreement than their volunteer counterparts who went through Method One, suggesting training method and measurement tools used influence volunteer data quality. In all nine communities, volunteers had the greatest levels of agreement for tree identification to genus (94% to 98%; Table 2). The results for genus identification were similar to or had exceeded the results of past studies (Bloniarz and Ryan 1996; Roman et al. 2017). While volunteers trained using Method Two performed better on average than volunteers trained using Method One, no statistical or practical significant difference was found. The high level of agreement for both groups is likely attributable the similarity of training methods for tree identification and the use of the photo-based tree identification cards created for each community. The results in Table 3 suggest that genus identification was relatively easy for volunteers to master to a high level of proficiency with a few exceptions. Abies, Crateagus, and Maackia were the most likely genera to be miss identified. While sample size for these genera are too small for detailed analysis, their misidentification suggests genera or species that are less common in an area are more likely to be misidentified. Abies, for example, was not a common street tree or yard tree in the 14 communities (data not shown), and university training staff anecdotally noticed that initially many volunteers identified any tree with evergreen foliage as pines. Training and identification cards appear to have helped separate out the true pines (Pinus) from Picea and Abies, yet Abies was still misidentified (as either Pinus or Picea). Picea was more abundant in most communities and may simply be more familiar to the volunteers, allowing for easier identification. Crataegus and Maackia were also misidentified, which is likely attributable to the small, ornamental trees being confused with the more prevalent Malus. Through recognition of uncommon genera or species and the common genera or species they are likely to be confused with, volunteer managers and trainers could increase volunteer training to recognize the differences and allow for more accurate identification. However, there are uncommon yet unique species, like Ginkgo biloba, where identification appears to be reliable, possibly given the uniqueness of Ginkgo biloba. Truly unique species without a common look-a-like may require less attention during training. Training methods that involve a test or quiz component—to isolate species or genera likely to be misidentified—could help to focus training on problem species, thereby increasing data quality.
Species identification was problematic even for genera identified correct with high frequency. The genus Fraxinus was correctly identified 98% of the time, yet species within the genus were only correctly identified 44% of the time (Table 3). While managers requested identification of trees to species level in most communities, the use of species information may not always have practical value. Invasive insect species that currently pose threats to the forests and woodlands of Minnesota include emerald ash borer (Agrilus planipennis) and gypsy moth (Lymantria dispar). Emerald ash borer infests all three native species of the Fraxinus genus (i.e., white ash, green ash, and black ash) predominantly found in Minnesota’s urban forests, while gypsy moth will defoliate hundreds of species of plants, albeit oaks and aspen tend to be more common tree hosts (U.S. Forest Service 2017). Asian longhorned beetle (Anoplophora glabripennis), not yet present in Minnesota, primarily uses trees in the Acer genus as its preferred host (Minnesota Department of Agriculture 2017). Volunteer ability to accurately identify tree genera is encouraging and warrants considerations for resource managers as knowledge of genus-level diversity will likely prove adequate to assess potential canopy losses, or insecticide treatment costs needed to formulate management objectives based on projections from tree survey or inventory data.
For the nine communities, measurement of DBH had an agreement frequency of 51% (Table 2); however, when allowing for a margin of error of 2.54 cm, the percentage agreement increased to 85% (Table 2). All communities except three (Hutchinson, Rochester, and Starbuck) were above the 80% agreement threshold proposed by Bloniarz and Ryan (1996), with volunteers in St Cloud attaining a 98% agreement frequency. For both DBH and DBH ±2.54 cm, volunteers trained under Method Two had statistically greater frequency of agreement (Table 3). The greater agreement for volunteers trained by Method Two may be attributable to several differences and possible introduced bias regarding the assessment of increased agreement in DBH and CRW measurements.
The first difference was the increased technical assistance in which researchers accompanied volunteers for their first few inventory outings, which allowed researchers to identify and correct errors in measurement techniques, such as measuring too low or high along the trunk, crooked or twisted tapes, and errors reading the tape. An additional source of improvement is likely attributable to the use of diameter tapes provided by the university research team to communities trained by Method Two. The use of diameter tapes helped with the consistency of measurement and data recording versus linear tapes (circumference measurements), but the diameter tapes used by volunteers trained by Method One, where some of the measurements needed to be converted from circumference to diameter by entering data in the appropriate spreadsheet column, likely increased error. The combination of increased technical assistance and improved measurement devices for volunteers trained by Method Two increased the overall agreement by 6% (Table 3), with volunteers approximately 1.6 times more likely to agree with university researchers (Table 5).
Additional sources of error in agreement may have been introduced by temporal differences in measurement between volunteers and researchers. An increment core was taken from a small sample of trees in each community and measured in the field (data not shown) to verify that less than 2.54 cm of growth had occurred between the time of volunteer and researcher measurements. Even though trees were believed to have added less than 2.54 cm in diameter between measurements, it is plausible that some error was introduced due to tree growth. A source of bias was unintentionally introduced as researchers measured DBH in all nine communities using only the measurement techniques and tools employed by the communities in training Method Two. The higher level of agreement seen in communities trained with Method Two likely reflects, at least partially, the introduced bias. An unbiased assessment of agreement should have compared volunteer and researcher measurements with researchers using the same tools and techniques used by volunteers. However, the results still provide valuable insight for managers seeking to use volunteers to collect urban-forest data.
In communities trained by Method One, length of stride (pace) was used to determine CRW. The volunteer frequency of agreement (7%; Table 2) with the university researchers was lower than that of volunteers who underwent training in Method Two (9%; Table 2) and who used a 15.25 m measuring tape to determine CRW. As with DBH, when a margin of error in CRW was allowed (±1.5 m), agreement between volunteers and researchers increased with training Method Two; volunteers had an 18% increase in frequency of agreement and were more than twice as likely to agree with researchers (Table 4). However, only Saint Cloud (82%; Table 2) had a frequency equal to or greater than the 80% threshold. While training Method Two did have statistical increases in agreement, the level of agreement, for most communities, may still be too low for any detailed management purposes. Source of errors and bias in CRW measures were similar to DBH (e.g., temporal, improper measurement technique, inadequate measurement tool). An example of improper measurement technique was observed when volunteers would walk from the trunk to the drip line, looking up at the canopy directly overhead and stopping when they reached the drip line. This resulted in volunteers stopping either a meter or more short or long of the actual drip line when viewed perpendicular to the crown radius being measured. Even with the linear tape, volunteers often held the tape slack as they negotiated obstacles (e.g., curbs, parked cars) while looking up, further reducing the precision of the measurement. The increase in technical assistance and improved measurement tools are plausible explanations for increased frequency of agreement, yet these cannot be separated from the unintentional bias introduced by researchers employing the same measuring tools and techniques for all communities, as those used by the communities trained using Method Two.
Further improvement for assessment of CRW measurements by volunteers might include standardized radii measurement locations (e.g., the cardinal directions), use of more accurate measurement tools (e.g., laser measurers, linear tapes), increased field practice, and technical assistance. To assess the efficacy of volunteer measurements of CRW, measurement protocols and objectives should be standardized and well-defined at the beginning of the project. For example, understanding which genera contribute the most to a community’s canopy cover would require less rigorous measurements, as high data quality would be assessed based only on accurate ranking of genera by relative canopy cover amount and not by a precise measurement. Training volunteers to measure canopy by counting their paces may be adequate for the ranking top contributors to canopy cover. Where more precise CRW measurements are required, for estimates of ecosystem services, then more precise tools (e.g., linear tapes or laser measurement devices and compasses) coupled with strict measure protocols, such as measuring and reporting four crown radii in in the cardinal directions, should be used. The additional measurements with more precise tools and protocols should help increase data quality. Increased training may incorporate simple steps to correct improper measurement techniques, such as having the data recorder locate and stand at the drip-line for the measurer to have two static points (trunk and data recorder) to measure between, removing the need to look up while walking to locate the drip line. Practice in the field could be augmented with trainers pre-measuring crown radii on a set of training trees, to help volunteers calibrate their measurements during training.
Agreement for tree condition rating was approximately 1.8 times more likely (Table 5) for volunteers trained with Method Two compared to Method One. Here, the only differences between the volunteer groups was the training and technical assistance. Condition rating was believed to be the most complex and subjective of the inventory tasks carried out by the volunteers and this might explain the low overall agreement frequency, with none of the nine communities achieving the 80% agreement threshold. However, given that tree condition can be highly variable from year to year or season to season (Hursh and Haasis 1931; Beedlow et al. 2013), the effect of temporal differences between volunteer rating and researcher rating likely had a substantial impact on frequency of agreement. A more practical approach to assessing volunteer condition-rating performance would be to assess volunteer-rated trees during the same season and year.
A final factor not controlled by this study that might have influenced results can be attributed to data fatigue. Complex, detailed, or arduous tasks can contribute to frustration and boredom on the part of the volunteer, which have negative ramifications for the quality of data collection (Darwall and Dulvy 1996; Newman et al. 2003). If volunteers became unengaged in the task at hand, or lost their sense of objectivity during data collection, bias and/or imprecision could have been introduced into the data and influenced results.
CONCLUSION
The results of this research indicate that trained volunteers can collect urban forest survey data at a higher frequency of agreement with university researchers when provided with appropriate tools and technical assistance. The implications of these findings carry substantial weight, especially for communities lacking the financial, administrative, or technical resources that are otherwise necessary to provide quality information about their urban forests. Although not directly addressed by this research, evidence indicates dynamic training that is responsive to expressed needs and concerns improves the experience of the volunteer participant (Hager and Brudney 2004; Leslie et al. 2004; Simes 2006; Fernandez-Gimenez et al. 2008).
Smaller communities often have difficulty implementing urban forest management practices. Most often, this is due to challenges in finding support and time, a lack of resources, and uncertainty regarding responsibility and authority pertaining to management. Engagement and empowerment of volunteers within communities can also be harnessed by municipal foresters, resource managers, and decision makers to leverage funding or impact policy that benefits the urban forest (Bloniarz and Ryan 1996). Effective urban forestry programs promote the importance and value of urban forests, seek to include a range of involvement across the community, and acknowledge the shared responsibility of both public and private interests in resource management (Elmendorf et al. 2003).
Volunteers can play a vital role in collecting quality data necessary for management of urban forests. A large body of work has helped to establish and support the use of volunteers to effectively assess and monitor natural resources (Penrose and Call 1995; Rock and Lauten 1996; McLaren and Cadman 1999; Brown et al. 2001; Fore et al. 2001; Engel and Voshell 2002; Nicholson et al. 2002; Delaney et al. 2007; Crall et al. 2011; Gillet et al. 2012). Beyond providing useful data for management and monitoring programs, volunteers can also have profound positive impacts for communities through increased civic engagement or momentum-building toward future management efforts (Westphal 1993; Bloniarz and Ryan 1996; Nicholson et al. 2002; Foster-Smith and Evans 2003; Galloway et al. 2006), leveraging of limited budgets (Mattson et al. 1994; Brown et al. 2001), and expanded data collection on large temporal and spatial magnitudes that would otherwise be beyond the capability of most scientific endeavors (Dickinson et al. 2010). Further examination of volunteer ability to collect high-quality data in urban forests will help refine and improve the use of volunteers, as well as volunteer experience and engagement.
Acknowledgments
We would like to thank all of the communities and volunteers who worked on the surveys and inventories. Support was provided by the University of Minnesota, Department of Forest Resources, University of Minnesota Extension. Funding was provided through the Minnesota Department of Natural Resources Division of Forestry and the U.S. Forest Service.
- © 2018, International Society of Arboriculture. All rights reserved.
LITERATURE CITED
Jump to section
Related Articles
Cited By...
- No citing articles found.