1 Introduction

The leaves constitute the organ of the vegetable body capable of converting solar energy into chemical energy through photosynthesis [7]. Vegetable growth and development are determined by the leaf morphology, especially its area, the largest component of biomass accumulation and productivity [17, 34]. The leaf size and morphology undergo an intraspecific and interspecific variation, and many physiological parameters (e.g., nitrogen content, photosynthesis and leaf respiration) are normalized by leaf area in order to study the relationship between them [23, 41].

The leaf morphology (e.g. area, perimeter, width and length) is strongly used to investigate vegetable responses to different environmental conditions, such as the soil fertility [4, 13], soil water/dry availability [29, 38], light availability [25], effect of pesticides application [20], climate changes [15, 27, 31], phytopathogens [18], herbivory [9], screening of genotypes for breeding [39], pollution [12]. In fact, measuring the area, length, width, perimeter and their derivations (e.g. specific leaf area, perimeter/leaf area ratio, length/ width ratio) is a powerful tool used by professionals of several areas of knowledge such as vegetable improvers, botanists, phytopathologists, ecologists and agronomists [14, 36, 37, 40].

There is equipment that measures the leaf dimensions with great accuracy, but it is quite expensive, quotes around US$3,500.00 (quotation performed in January 2021). In addition, it is possible to estimate leaf area using allometric models (equations) using leaf morphology variables (e.g. length and width), adjusted for each species and/or crop. However, a big leaves sample is required to develop these models. For example, Antunes et al. [1] analyzed 1563 coffee leaves. It is important to note that these models estimate the leaf area of a single leaf at a time. In tests with many sample units, estimating this parameter will be a time consuming and laborious process. Therefore, it is necessary to develop practical and low-cost methodologies, in order to guarantee the execution of works that require a verification of the leaf area and other dimensions with celerity and accuracy.

In this paper, we present a simple, intuitive and low-cost method to calculate leaf dimensions. The method can be implemented in a desktop software or a mobile application, allowing researchers to use the mobile version to perform field measurements and the desktop version if they wish to perform measurements in the laboratory. For the leaf dimensions determination, we need only capture images of the leaves accompanied of a scale pattern printed on paper. The images can be taken by a simple camera. There is no need to purchase extra equipment. Personal devices can be used, at no additional cost.

Our method presents the following characteristics, which are the main contributions of this work:

  • Specific to perform leaf sizing (area, width, length and perimeter);

  • It does not require pre-processing;

  • Automatically detects the scale pattern.

2 Related Work

Several methods for leaf area estimation based on digital images were proposed in last years. Some works use the color information of the images to extract the interest objects and estimate leaf dimensions [6, 16]. Extracting color information may require user intervention at filtering step and even the need of using other software. In our method, we use grayscale images, because the contrast between the objects and the background is enough to separate them, even without the color information.

As in our method, some works used an object of known dimensions as a scale pattern. Jadon [11] chose a coin, but this shows strong reflection when illuminated. Thus, that proposal demonstrates a greater sensitivity to lighting conditions. Easlon et al. [6] and Tech et al. [35] opted for a square. Easlon et al. [6] detect the square automatically using color information. However, as previously mentioned, the color needs to be calibrated by the user. The method of Tech et al. [35] requires the square to be positioned at a specific position in the image. In our method, the square is automatically detected through its shape. Moreover, it can be placed anywhere in the image.

ImageJFootnote 1 software [28] has been used to calculate the leaf area [8, 19]. However, it is a general-purpose image processing software, which requires the user to perform several steps until obtaining the measurement, such as selecting a measurement line known in the image. We propose to create an intuitive computational tool so that the user can open their images in the software and through a few simple steps perform the leaf measurements calculation. In addition, we intend to create a specific tool for leaf morphology analysis, allowing the insertion of new functionalities related to plant physiology. As it is an application widely used by researchers, ImageJ will be used to validate our method results.

There are also some mobile applications for leaf area calculation. The Petiole application [24] uses a very controlled scenario. The calculation starts with a calibration step and the device position should be kept for the area estimation. Moreover, it requires user interaction to identify the leaves position on the image. The application Easy Leaf Area Free [6] needs a color calibration step, which makes the application less intuitive for the user. We propose a simple method that does not require user calibration.

Besides methods based only on computer vision techniques, methods for leaf area measurement with Artificial Neural Networks (ANNs) are appearing recently. Siswantoro and Artadana [30] used a fixed Webcam to capture digital images of the individual leaves. They applied a thresholding step for image segmentation and extract the leaf length, width, area, and perimeter in pixels. Finally, ANN proposed uses the extracted features to estimate the real leaf area. Sa-bouri et al. [26] compared three methods to predict leaf area in cereals: regression models, ANN, and ANN combining fuzzy logic. The neural network developed by the authors is similar to the previously cited work and uses leaf width and length as inputs. Two fluorescent light sources were fixed in specific positions to keep the illumination conditions. The proposal based on neuro-fuzzy inference obtained better performance concerning the work experiments. The advantages of our proposal compared to these networks are automatic detection of the interest objects and no need for training. In addition, while these works need a fixed capture scenario, our method can deal with images captured from different positions and less controlled lighting conditions.

3 Proposed Method

Most existing methods for leaf dimensions estimation depends on user intervention. This user interaction can introduce subjectivity and lead to inaccuracies in results. Other proposals require a specific positioning of scene objects and calibration steps. These constraints make the process difficult. In this work we propose a method to estimate the leaves dimensions in few steps, which do not depend on the user interference. Our proposed method for leaf dimensions calculation is composed of six steps: capture, thresholding, obtaining contours, filtering, detection and dimensions calculation. The Fig. 1 shows an overview of the proposed method. We present images and information extracted in each step.

Fig. 1.
figure 1

Steps of the proposed method. Above each image, we have the notation referring to the information obtained in the respective step.

Full size image

3.1 Capture

In the first step, an image of the leaf (or leaves) is captured so that its dimensions can be estimated. The leaf must be accompanied by a scale pattern whose real size is known. We chose a square pattern printed on paper. This pattern can be placed anywhere on the image and is automatically detected to estimate the leaf dimensions, as described in Sect. 3.5.

The leaf is not a plane surface. It must be planned to avoid underestimating the measures. For this purpose, a glass or acrylic plate may be used to compress the leaf and keep it stretched. We used a commercial clear glass plate in the tests to keep the projection area of the leaf parallel to the surface, avoiding curvatures and folds. Furthermore, the background should be a light color in order to ensure contrast with the interesting objects. Reflection is critical to the thresholding process described in Sect. 3.2 and should be avoided. It is recommended the image be captured at a perpendicular angle to avoid distortion problems. Various capture distances and scale pattern sizes were tested and the results will be discussed in Sect. 4. It is important to mention that our method allows more than one leaf to be captured in the same image. The dimensions of each leaf are individually estimated.

3.2 Thresholding

In the capture step, a color image \(I_c\) is obtained, containing the leaf (or leaves) and the scale pattern. The image is converted to grayscale, resulting in a new image \(I_g\). This conversion and most of the image processing functions were implemented using OpenCV library [2] in C++ programming language.

We applied an Otsu’s thresholding [21] on the \(I_g\) image. This algorithm is based on the histogram image. It finds a threshold t which lies in between two peaks such that variances to both classes are minimal. Based on this threshold t we achieve a binary image \(I_b\) where:

$$\begin{aligned} I_b(x,y) = {\left\{ \begin{array}{ll} 0, &{} \text{ if } I_g(x,y) > t \\ 1, &{} \text{ if } I_g(x,y) \le t \end{array}\right. }. \end{aligned}$$

3.3 Obtaining Contours

In the thresholding step, a binary image \(I_b\) is obtained, where the leaf and the scale pattern are white and the background is black. It is necessary to extract the contours of the interest objects, that is, the leaves and the scale pattern.

In this step, we apply Suzuki et al. [33] algorithm in the \(I_b\) image. This method extracts a set \({\textbf {C}}\):

$$\begin{aligned} {\textbf {C}} = \{{\textbf {C}}_1, {\textbf {C}}_2, ... ,{\textbf {C}}_n\}, \end{aligned}$$

where each \({\textbf {C}}_i\) is a contour, represented by the set of points that compose it:

$$\begin{aligned} {\textbf {C}}_i=\{(x_1, y_1),(x_2, y_2), ..., (x_m, y_m)\}, \end{aligned}$$

where each \((x_j, y_j)\) is the position of a point (pixel) in the image.

3.4 Filtering

In the previous step, a set \({\textbf {C}}\) of n contours is obtained. This set can contain noise, i.e., contours that do not represent interest objects. These contours should be filtered. In general, the bad contours are much smaller (image noise, for example) or much larger (a table contour, for example) than the interest objects. To overcome this problem, a filtering by contour area is performed. The contour areas in pixels are estimated by Green formula [32]. Thus, from C, we obtain the set of areas:

$$\begin{aligned} {\textbf {A}} = \{a_1, a_2, ... ,a_n\}, \end{aligned}$$

where each \(a_i\in {\textbf {A}}\) is the area of the respective contour \({\textbf {C}}_i\in {\textbf {C}}\).

The areas \(a_k\in {\textbf {A}}\) that are not in the range \([a_{min},a_{max}]\) of expected areas, are discarded, as well as their respective contour \({\textbf {C}}_k\). The sets \(C^f\subseteq C\) and \(A^f\subseteq A\) are the filtered sets of contours and areas, respectively. Experimentally, we defined the values \(a_{min} = 10^3\) and \(a_{max}=10^{10}\) which proved to be efficient. However, some noise, such as shadows, may not have been completely removed. It is possible to manually remove these artifacts with a simple tool available in the software. We observed the noise presence in 30% of the analyzed images.

3.5 Detection

In the filtering step, the sets \({\textbf {C}}^f\) and \({\textbf {A}}^f\), are obtained, representing the contours and their respective areas after filtering. In order to automate the process, we need to separate the scale pattern from the leaves. As the scale pattern is a square and it presents a different geometry from the leaves, we can perform a shape-based detection of the square in the contours set.

First, we apply a polygonal approximation on each contour \({\textbf {C}}_i^f\in {\textbf {C}}^f\) using the Douglas-Peucker algorithm [5]. This function reduces each contour \({\textbf {C}}^f_i\) to a simplified contour \({\textbf {C}}^p_i\subseteq {\textbf {C}}^f_i\). Thus, we have the set:

$$\begin{aligned} {\textbf {C}}^p=\{{\textbf {C}}_1^p, {\textbf {C}}_2^p, ..., {\textbf {C}}_n^p\}, \end{aligned}$$

where \({\textbf {C}}_i^p\) is the polygonal approximation of the contour \({\textbf {C}}^f_i\in {\textbf {C}}^f\).

To determine the square contour \({\textbf {C}}^p_s\in {\textbf {C}}^p\), we analyze some conditions. The number of points in \({\textbf {C}}^p_s\) must be equal to 4, i.e., \(|{\textbf {C}}^p_s| = 4\). In addition, the cosine of the angles \(\theta \) formed by the contour sides are calculated. If \(|cos(\theta )| < 0.3\) for all 4 angles, this contour will be \({\textbf {C}}^p_s\). This method was efficient in extracting the square for 98% of the tested images. Thus, we obtain the contour \({\textbf {C}}^p_s\in {\textbf {C}}^p\) and its corresponding \({\textbf {C}}_s\in {\textbf {C}}^f\) in the set of non approximated contours. In terms of area, we have the respective area of the square in pixels \(a_s\in {\textbf {A}}^f\) and the set of leaf areas \({\textbf {A}}^l = {\textbf {A}}^f - \{a_s\}\).

3.6 Dimensions Calculation

At this point, all the interest objects are properly separated and contoured. Therefore, our method is ready to calculate the dimensions of each leaf in the image. To facilitate the calculation of real leaf length and width, the leaves are aligned according to the Principal Component Analysis (PCA) [3]. PCA is applied to each original leaf contour \({\textbf {C}}_i\in {\textbf {C}}^f - \{{\textbf {C}}_s\}\) to find the main leaf direction. PCA returns the centroid \(K_i = (x_k,y_k)\) of each leaf contour and the angle of the main direction \(\theta _i\). We associated this direction with the length of the leaf. Each contour is aligned so that the main direction coincides with the horizontal axis of the image. Thus, each contour point \((x_j,y_j)\in {\textbf {C}}_i\) is rotated by the equation:

$$\begin{aligned} x_j = (x_j - x_k)~cos(\theta _i) - (y_j - y_k)~sin(\theta _i) + x_k,\\ y_j = (x_j - x_k)~sin(\theta _i) + (y_j - y_k)~cos(\theta _i) + x_k. \end{aligned}$$

The real area \(a^r_s\) of the scale pattern in cm\(^2\) is known. We have the scale pattern area \(a_s\) in pixels and each leaf area \(a_i\in A^l\) in pixels. Thus, the real area of each leaf \(a^r_i\) in cm\(^2\) can be obtained by the following equation:

$$\begin{aligned} a^r_i = \frac{a_i a^r_s}{a_s}. \end{aligned}$$

The real perimeter \(p^r_s\) of the scale pattern in cm is known. We calculated the perimeter in pixels of each leaf and the scale pattern. Thus, we achieved the set:

$$\begin{aligned} {\textbf {P}} = \{p_1, p_2, ... ,p_n\} \cup {p_s}, \end{aligned}$$

where, each \(p_i\in {\textbf {P}}\) is the perimeter of the respective leaf contour \({\textbf {C}}_i\in {\textbf {C}}^f\). And \(p_s\) is the scale pattern.

To obtain the real perimeter \(p^r_i\) of the leaf, we use the following equation:

$$\begin{aligned} p^r_i = \frac{p_i p^r_s}{p_s}. \end{aligned}$$
(1)

We know the real side \(d^r_s\) of the scale pattern in cm. The pixel measurement on each side \(d_s\) of the pattern can be obtained by dividing the perimeter \(p_s\) by four. The length \(l_i\) and width \(w_i\), in pixels, of each leaf can be obtained by the rectangular bounding box of the rotated contour.

In this way, the base and height of this rectangle will correspond to leaf length \(l_i\) and width \(w_i\) in pixels, respectively. Thus, the real dimensions of each leaf can be obtained by the following equations:

$$\begin{aligned} w^r_i = \frac{w_i d^r_s}{d_s},&~~~~~ l^r_i = \frac{l_i d^r_s}{d_s}. \end{aligned}$$

As a restriction of the proposed method, the largest between the two dimensions found is considered to be the length of the leaf. Leaves that do not have this characteristic will have their dimensions reversed, that is, the researcher will have to analyze and correct the results manually before performing any analysis.

4 Results and Discussion

In this section, we describe the experiments performed to evaluate our proposed method. We performed controlled tests with known objects and tests with leaves. For capture the images, we used five devices: three smartphones (Samsung Galaxy J2 Core, Motorola Moto C and Motorola One Vision) and two scanners (HP Deskjet 2050 and MultiXpress M5360RX).

Table 1. Pearson’s correlation coefficient (r), RER average (\(\mu \)), RER standard deviation (\(\sigma \)), \(R^2\) and regression equation by dimension for the ellipses pattern.
Full size table

4.1 Metrics

In this subsection, we present the metrics used to evaluate quantitatively our method results. The relative error rate (RER) represents the percentage of the difference between the value estimated \(v_{est}\) and achieved with the standard method \(v_{std}\). It can be calculated as:

$$\begin{aligned} RER(\%) = \frac{|v_{est} - v_{std}|}{v_{std}} \cdot 100. \end{aligned}$$

The RER measure the accuracy of the proposed method in relation to the standard method.

Given a set of values estimated by our method \(V_{est}\) and the respective values achieved with the standard method \(V_{std}\), we analyzed the correlation between these values. We estimated the linear regression equation \(y=ax+b\), the determination coefficient \(R^2\) and the Pearson’s correlation coefficient r, using the Google Sheets software.

4.2 Controlled Tests

In order to evaluate our method accuracy, we performed preliminary experiments with a controlled pattern. For these tests, we used a pattern containing four ellipses and a scale pattern of 4 cm side. We used a glass plate to plan the pattern as explained in the Sect. 3.1 and captured perpendicular images from distances between 20 and 50 cm. In a general analysis, it is possible to see that the method was able to measure the dimensions accurately, achieving a good performance using simple smartphone cameras.

Table 1 presents the results of the ellipse tests. Analyzing the regression equation and the correlation coefficient, it is possible to verify a strong correlation between the values estimated by the proposed method and the expected values. In addition, we achieved \(R^2 > 0.95\) for all dimensions. We achieve RER mean below 3.5% for all the dimensions, which shows the accuracy of our method. Furthermore, we note the regression equations present slope close to 1 and small intercept values.

4.3 Leaf Tests

In this subsection, we introduce the dataset collected for measuring leaf dimensions, and present the results of the tests performed to evaluate the proposed method.

Table 2. \(R^2\) values and regression equations by species and dimensions in relation to the standard method adopted.
Full size table

Data Acquisition. For this work, annual and perennial species were chosen, whose cultures have significant participation in the Brazilian and international Gross Domestic Product (GDP) [10]. All leaves were collected in the morning (before 9:00 am, to guarantee high turgidity), with no apparent/visual symptoms of nutritional deficiency, pathogen attacks, and diseases. The leaves were collected and analyzed on the same day, at the Natural Sciences laboratory of the Instituto Federal de Educação Ciência e Tecnologia de Minas Gerais - Ouro Branco Campus.

The sample is composed of 118 leaves of 6 species: 18 of the bean, 18 of coffee, 11 of cotton, 21 of eucalyptus, 20 of orange, and 30 of soybean. Images were captured between 12 and 50 cm of one or multiple leaves, accompanied by a scale pattern of sides measuring 1, 2, 4, or 6 cm. We used Motorola Moto C and Motorola One Vision smartphones. In addition, the leaves also were scanned accompanied by the 1 cm scale pattern using the printer scanner MultiXpress. Overall, our dataset is composed of 529 leaves images. The images are available at the following link https://bit.ly/42DbyNV.

Method Evaluation. To verify the results accuracy, we compared with two methods, the ImageJ software [28] and a simple, but very widespread manual method. The ImageJ software was used as the standard method. As previously stated in Sect. 2, this application is widely used by researchers in the literature.

The steps used to estimate the leaf dimensions with ImageJ software were: 1) select image file; 2) convert image to grayscale; 3) perform a thresholding; 4) draw a line indicating the side of the scale pattern; 5) set the scale known distance; 6) select the leaf and measure area and perimeter; 7) draw a line indicating the width of the leaf and measure the length; 8) draw a line indicating the length and measure. It is important to note that those steps depends on the user interaction and, consequently, are subject to errors.

Fig. 2.
figure 2

Average RER in relation to standard method. (a) By scale pattern. (b) By capture distance. (c) By species. The error bar represents the standard deviation

Full size image

The manual weight-based method for area estimation [22] starts by tracing the outline of the leaf on a paper, whose grammage g is known. The contour is cut and its weight w is measured by a precision balance. The area a is estimated by the formula \(a = \frac{w}{g}\). The same contour used in the weight-based method is measured with a simple rule to estimate the expected width and length. In addition, a line or string is passed around the contour and then measured with a rule to estimate the expected perimeter.

Table 2 presents the coefficient of determination (\(R^2\)) and the regression equations by species and dimensions in relation to the standard method (ImageJ software). In general, the performance for area and perimeter was satisfactory. For these dimensions, the \(R^2\) values were above 0.96 and the equations presented angular coefficients close to one for all species. Regarding width and length calculation, we note some problems in the cotton results with \(R^2\) values below 0.75. The morphology of the leaves of this species makes it difficult to estimate these dimensions. However, for the other species, our method achieved \(R^2\) values above 0.94 for these dimensions. In general, the results were promising and indicate our method perform similarly to the standard method.

Figure 2c shows the average RER for each species and dimension in relation to the standard method. The errors for perimeter were the smallest, between 1.98 and 3.25%. For area, we achieved errors between 2.32 and 5.40%. Cotton had the highest errors for length and width. As previously mentioned, the leaves morphology makes it difficult these dimensions estimation for cotton. However, for the other species, the average RER for length and width were between 1.65 and 4.31%. Overall, the errors achieved showed the method applicability for measuring leaves dimensions, specially those with simpler morphologies.

Fig. 3.
figure 3

Scatter plot for three methods and all images. (a) Area. (b) Perimeter. (c) Width. (d) Length.

Full size image

Figure 2a presents the average RER for each scale pattern size used. We observe that our method has a worse performance for the 1 cm pattern. The hypothesis for this variation is the manual line selection in the ImageJ procedure. Depending on the direction of this line selection, the real dimension can present a significant difference, especially for images with distortion and the smallest pattern. On the other hand, our method estimates the square measures automatically, without user interaction. For the other scale sizes, the results for the methods was quite similar. In general, the results indicate that the size of the pattern does not significantly influence the method’s performance.

The average RER for each capture distance is presented in Fig. 2b. To facilitate the comparison, we grouped the tests with capture distances shorter than 20 cm. The tests for this group achieved errors slightly higher than the others. As previously explained, the ImageJ method needs a manual line selection in the image. Since images captured more closely can present distortion, the user’s choice when selecting the line can affect the results. For the other distances, the methods performed similarly. This indicates that the capture distance does not have a strong influence, when the capture conditions are respected.

Table 3. Pearson’s correlation coefficient (r), RER average (\(\mu \)) and standard deviation (\(\sigma \)) for Our method \(\times \) Manual methods.
Full size table

Figure 3a, 3c, 3d and 3b present the scatter plots considering all the images used for tests. These plots also include the results of tests performed using the manual methods previously described. Measurements by manual methods are performed per leaf. Thus, the yellow dots on the same line of the graph represents the same leaf. Our method achieved \(R^2\) values above 0.975 for all dimensions. These values were greater than those of manual method. Furthermore, the straight lines of our method presented a slope closer to one, except in the length graph. For this dimension, as previously discussed, our method presents limitation to deal with certain leaves morphology.

Table 3 presents the Pearson’s correlation coefficient r, RER average (\(\mu \)) and RER standard deviation (\(\sigma \)) for our method and manual method in relation to the standard method. Pearson’s coefficients indicate a strong correlation for both methods. In relation to the RER metric, we observe the manual method present higher values. Due to the several manual steps, manual method is more difficult to control and more susceptible to errors.

Discussion. We verify that the proposed method was able to estimate leaf dimensions satisfactorily in comparison to ImageJ, which is widely applied in the literature. In terms of time to estimate the measures, our method is more efficient than the ImageJ software, which requires less user-side intervention. In a comparison made with nine images, the proposed method estimated all dimensions with an average time of 15 s, while ImageJ took 78 s.

However, our method presents some limitations. It is sensitive to the capture angle and distance, illumination, and the noises that may appear in the image. Images taken from an inclined angle suffer from perspective distortion. Depending on the lighting location may cause shadows or reflections in the image. All these factors compromise the calculation of leaf dimensions. These are limitations typical of applications involving image processing, and we intend to propose improvements to minimize such problems.

Leaf morphology is also a limitation of our method, especially on the width and length calculation. These dimensions still need some adjustments for species with more complex leaf morphologies, such as trifoliate or lobed leaves. These morphologies make difficult the PCA alignment, due to the irregular shape. In the ImageJ software, the width and length direction are manually defined. On the other hand, the area and the perimeter have already proved to be quite efficient. Except for the trifoliate bean leaf that overestimated the area in the petiole region, where the leaflets are inserted. This leaf presented problems for both methods, especially for higher capture distances. Finally, the low contrast between the leaf and the background is another limitation of our proposal. When the leaf presents a light hue, the thresholding process does not work correctly. An alternative for these light leaves is to use a light scale pattern and a dark background to increase the contrast.

5 Conclusions

We proposed a simple and low-cost method for leaf dimensions calculation based on image processing. Our method estimates area, perimeter, length, and width for one or more leaves directly from the images. The software automatically detects each leaf in the image and allows estimate statistics such as average and standard deviation. The only information provided by the user is the size of the scale pattern. No additional user interaction is required. The method estimates automatically the location of leaves and scale pattern.

In general, the measurement of leaf morphological parameters (e.g. area, perimeter, length, and width) by this method is an excellent tool to explain plant responses in several areas: evolution, ecophysiology, improvement, management. The results demonstrated the accuracy of the method proposed for this purpose, in addition to its practicality and speed. Moreover, these measurements can be employed in derivations and other parameters, such as the leaf area/ perimeter ratio, the leaf area index, the specific leaf area, and the length/width ratio. To conclude, as future works, we intend to improve the method, especially length and width estimation. Furthermore, we can perform an improvement in the segmentation step, avoiding the detection of noise in the detected contours. Finally, we are developing free mobile and desktop applications from this method that will be made available to any professional and student who wants to perform leaf morphological phenotyping. The source code of these applications can be found in the links https://github.com/KarlaFlorentino/LIMA-desktop (Desktop) and https://github.com/LuizMaurilio/LIMA (Mobile).