DOI: 10.1007/s11099-016-0214-x PHOTOSYNTHETICA 54 (4): 559-566, 2016 559 A novel method for the estimation of soybean chlorophyll content using a smartphone and image analysis J.P.G. RIGON*,+, S. CAPUANI**, D.M. FERNANDES**, and T.M. GUIMARÃES* Sao Paulo State University (UNESP), College of Agricultural Science, Department of Crop Science, Lageado Experimental Farm, José Barbosa de Barros Road 1780, 18603-970 Botucatu, São Paulo, Brazil* Sao Paulo State University (UNESP), College of Agricultural Science, Department of Soil and Environmental Resources, Lageado Experimental Farm, José Barbosa de Barros Road 1780, 18610-307 Botucatu, São Paulo, Brazil** Abstract The development of smartphones, specifically their cameras, and imaging technologies has enabled their use as sensors/measurement tools. Here we aimed to evaluate the applicability of a fast and noninvasive method for the estimation of total chlorophyll (Chl), Chl a, Chl b, and carotenoids (Car) content of soybean plants using a smartphone camera. Single leaf disc images were obtained using a smartphone camera. Subsequently, for the same leaf discs, a Chl meter was used to obtain the relative index of Chl and the photosynthetic pigments were then determined using a classic method. The RGB, HSB and CIELab color models were extracted from the smartphone images and correlated to Chl values obtained using a Chl meter and by a standard laboratory protocol. The smartphone camera was sensitive enough to capture successfully a broad range of Chl and Car contents seen in soybean leaves. Although there was a variation between color models, some of the proposed regressions (e.g., the S and b index from HSB and Lab color models and NRI [RGB model]) were very close to the Chl meter values. Based on our findings, smartphones can be used for rapid and accurate estimation of soybean and Car contents in soybean leaves. Additional key words: camera, color model, mathematical models, nondestructive, photosynthetic pigments, portable equipment. Introduction The amount of solar radiation absorbed by a leaf is a function of the photosynthetic pigment content (Steele et al. 2008). Thus, the contents of chlorophyll (Chl) a and Chl b are linked directly to a photosynthetic potential and primary production (Curran et al. 1990). Carotenoids (Car) are also very important as a mechanism for attenuating the stress caused by excess irradiation (Sikuku et al. 2010). Thus, monitoring changes in the Chl content enables assertions about the plant interaction and the influence of stress factors (Vesali et al. 2015). Nondestructive techniques have been investigated for estimating the Chl content of plants (Rigon et al. 2012a). However, with the development of remote-sensing tech- nology in recent years, image technologies have been used for potential real-time estimation of Chl content and subsequent analysis of photosynthetic properties (Williams et al. 2010). Among the techniques, digital cameras are widely used together with the segmentation of images and color models (Li et al. 2010) and are becoming a new quantitative tool in agriculture (Riccardi et al. 2014). Smart phones and their cameras have provided an opportunity for using their sensors as measurement tools, and computation and analysis can be done without any additional attachments. Digital image processing is increasingly used to estimate variables of interest for agronomic activities (Confalonieri et al. 2013). Smartphone images have already been used to estimate crop water requirements and photosynthetic primary production (Confalonieri et al. 2013). They have also been used to estimate a citrus yield (Gong et al. 2013), and a colorimetric analyzer based on a smart phone can determine the available phosphorus content in soil (Moonrungsee et al. 2015). A recent study created an android application to estimate the Chl content of a corn leaf (Vesali et al. 2015). The image acquisition is reliable for real-time estimation of Chl content. The objectives of the present study were (1) to evaluate the applicability of a fast and nondestructive method to estimate the contents of Chl a and b, total Chl, and Car content and (2) to determine several color models for analysis of digital images acquired with a smart phone camera and validate their applicability for quick estimation. ——— Received 25 October 2015, accepted 26 January 2016, published as online-first 7 March 2016. Corresponding author: jprigon@fca.unesp.br Abbreviations: B – mean values of blue; Car – carotenoids; Chl – chlorophyll; DGCI – dark green color index; G – mean values of green; R – mean values of red; R1 – flovering stage; R3 – pod formation stage; VIgreen– vegetation index. J.P.G. RIGON et al. 560 Materials and methods Field data acquisition: An experiment was conducted at São Paulo State University, UNESP, Botucatu, SP, Brazil. Were collected 80 disc-shaped samples of soybean leaves in the middle third of the plant with 113-mm diameter and different physiological stages (flowering, R1, and pod formation, R3, respectively). The Chl relative rates were individually determined by a portable Chl meter ClorofiLOG-1030 (Falker Agricultural Automation, Brazil). Digital images: Single images were obtained through a smartphone camera (8.0 MP) from the leaf discs. For the acquisition of images, the phone was mounted on a support in the nadir position relative to the leaf discs under identical illumination conditions, as proposed by Riccardi et al. (2014). Munsell images were used (2.5 G 5/5 yellowish green; 5.0 G 5/5 green; 7.5 GY 5/5 green greenish-yellow) for further calibration with known RGB values. Chl extraction: After obtaining the images, the circular samples were placed in test tubes and wrapped in aluminum foil to protect against degradation by light. We used 5 ml of the reagent extractor dimethylsulfoxide, and the tubes were incubated at 70oC for 30 min in accordance with the methods described by Arnon (1949) and Hiscox and Israelstam (1979). A 3-ml portion was transferred to a quartz cuvette, and then the absorbance was measured using a UV-VIS spectrophotometer (UV mini-1240, Shimadzu, Japan) at wavelengths of 480, 646, and 663 nm, simultaneously. The equations described by Wellburn (1994) were used for the quantification of the Chl a, Chl b, and Car contents: Chl 12.47 – 3.62 (1) Chl 25.06 – 6.5 (2) Car . . (3) Image segmentation and calculation of color indices: The images were transferred to a computer and analyzed in Corel Photopaint software (version X4, Corel Corporation, 2008). Using the software histogram function, the center of the circular image was selected, and the size was automatically adjusted to 25 square pixels. Values were obtained for the RGB color model of each image, which is the most commonly used for the representation of digital images. R, G, and B are the mean values of red, green, and blue, while r, g, and b are the RGB values normalized by the following equations: r (4) g (5) b (6) The green vegetation index (VIgreen) was obtained as follows (Gitelson et al. 2002): VI (7) RGB values were converted to the HSB model through algorithms reported by Karcher and Richardson (2003). This color model is represented by cylindrical coordinates in which the angle around the central vertical axis corresponds to the hue (H). The HSB calculation was performed as follows: H = 60 ∗ , , , , , max R, G, B R 60 ∗ , , , , max R, G, B G 60 ∗ , , , , max R, G, B B (8) S = , , , , , , (9) B = max R, G, B (10) The values were then converted to model X, Y, Z and subjected to color model CIE L* a* b* (Robertson 1977) by means of Eq. (11–13): L ∗ – 16 (11) a ∗ 500 / – / (12) b ∗ 200 / – / (13) This model describes all visible colors using L* as the lightness of the color, while a* represents green (a* negative) and red (a* positive), and b* represents the blue (b* negative) and yellow (b* positive). Based on the HSB values, the dark green color index (DGCI) was obtained by Eq. (14) proposed by Karcher and Richardson (2003): DGCI (14) Data were subjected to Pearson′s correlation analysis. The significant data was subjected to linear regression and the Student′s t-test at 95% confidence. Data analysis was performed using Origin 8.6 software and the graphics were obtained using SigmaPlot. ESTIMATION OF CHLOROPHYLL BY SMART PHONE AND IMAGE ANALYSIS 561 Results and discussion Chl pigments by Chl meter: The analytical laboratory extraction of the leaves resulted in a broad range of photo- synthetic pigments, ranging from 75 to 600 µmol m–2 for Chl a, from 8 to 120 µmol m–2 for Chl b, and from 50 to 350 µmol m–2 for Car (Fig. 1). These ranges were necessary for prediction of several conditions of leaf status. Using the classical method, a close relationship was found between the measurement using the portable Chl meter and the pigment contents (Fig. 1). The leaf discs indicated that there was a similar relationship between the degree of leaf greenness, as determined by the portable Chl meter, and the Chl a content, which was extracted by the classical method. The relationship between the methods was best expressed by a quadratic model with a high range of Chl contents and nearly linear relationships. Most scientists agree that nonlinear equations are more appropriate for indicating the relationship between portable meter measurements of Chl and the actual Chl content (Marenco et al. 2009, Mielke et al. 2010, Rigon et al. 2013). The adjusted coefficients of determination of the model were 0.96, 0.92, 0.96, and 0.94 for Chl a and b, total Chl, and Car, respectively. These results were nearly identical to those observed in a recent study for Chl (Novichonok et al. 2016). Another study with soybean plants found similar results with a close relationship between portable Chl meter measurements and Chl measured by the classical method (Markwell et al. 1995). The Chl meter is one of the most used hand-held instruments for a rapid and nondestructive assessment of Chl contents in many crops species, such as quinoa (Riccardi et al. 2014), castor oil, sesame (Rigon et al. 2012a, 2012b), and Surinam cherry (Mielke et al. 2010). However, the reflectance depends on leaf anatomical features, such as surface relief, and/or external architecture, such as trichome density and waxy cuticle (Levizou et al. 2005). Consequently, this relation changes in accordance with the intrinsic characteristics of each species (Marenco et al. 2009). Furthermore, it was observed that in some cases, the determination of Chl b was inaccurate (Rigon et al. 2013). This resulted from the fact that the absorption peak of Chl a was similar to the wavelength emitted by the device, making it more difficult to separate (Neves et al. 2005). Torres-Netto et al. (2005) demonstrated SPAD to be the method with the best performance in terms of precision and the best metrics for both repeatability and reproducibility. Thus, the generated regression models in Fig. 1 may be used to estimate the content of Chl pigments in the leaves of soybean plants with great precision in a fast and efficient way and without any added cost for reagents. Fig 1. Relationship between the readings of the portable chlorophyll meter and the contents of chlorophyll a (A), b (B), total chlorophyll (C), and carotenoids (D) measured analytically in the laboratory for soybean leaves. J.P.G. RIGON et al. 562 RGB color model: A color in the RGB model is described by indicating how much of each of the red (R), green (G), and blue (B) is included, and each color had 256 graduations. The index B was positively correlated with Chl and Car, whereas R and G were negatively correlated (Fig. 2). This was similar to the findings of Yadav et al. (2010), Gupta et al. (2013), and Riccardi et al. (2014). G values were bigger than R and B values in all the images measured, and the R index was almost double of the B index. Thus, the best fit regression models for the single color component index G were about 0.82, 0.79, 0.82, and 0.82 for Chl a, b, total Chl, and Car. Studies have reported that the G index had a better relationship with the Chl content (Vesali et al. 2015). This better correlation can be explained by higher rates. The correlation coefficients for Chl a were similar to those observed by Yadav et al. (2010) for each index in the model RGB content in a study with potato. The possibility of using all the three indices RGB to estimate the Chl content has been demonstrated (Su et al. 2008). Vollmann et al. (2011) analyzed the state of nodulation and soybean Chl content with the use of SPAD and digital image analysis. These investigations of nondestructive methods using RGB image analysis are a relatively new area of research (Gupta et al. 2013). The CIE L*a*b color model: Although it is not common to use the CIE L*a*b color system, a significant linear relationship was observed in all photosynthetic pigments with indices and L*, with ratios for each pigment of about 0.80 and 0.62, respectively. For the correlation with the index b*, the coefficient of determination was about 0.90 for each pigment (Fig. 3). These results were also observed by Wang et al. (2014) in rice, where the b* index had the highest correlation coefficients. In a study of soybean crops, the Chl content in seeds was also observed, and the CIE L*a*b model had excellent correlation with the Chl content, especially the a* index (Sinnecker et al. 2002). In the CIE L*a*b color model, the L* coordinates represent the brightness, while the dimensions a* and b* are close to red–green and yellow–blue, respectively (Fairchild 2005). Interestingly, the best results were with the index b*, Fig. 2. Relationships between chlorophyll and carotenoids contents measured analytically in the laboratory and from an image based on the RGB color model with color indices R (A,D), G (B,E), and B (C,F) for soybean leaves. ESTIMATION OF CHLOROPHYLL BY SMART PHONE AND IMAGE ANALYSIS 563 Fig. 3. Relationships between chlorophyll and Car content measured analytically in the lab and from an image based on the CIE L*a*b color model with indices L (A,D), a (B,E), and b (C,F) in soybean leaves. which represents the yellow–blue color, whereas the Chls are determined by the leaf greening. HSB color model: Karcher and Richardson (2003) observed that the green of vegetation does not appear or represent the green value in the RGB color space exactly. They suggested converting the RGB values to a more intuitive hue, saturation, and brightness (HSB) color spectrum, which is based on the human perception of color. The H index (Hue) that means the pure color (Fairchild 2005), and was the only positively correlated with Chl and Car, while S and B were negatively correlated with photosynthetic pigments (Fig. 4). A higher relationship in the S index was observed with a coefficient of determination of about 0.87. Although the color is the same during the day, the saturation changes. That means the light was better than the pure color from the image captured, and the B index (brightness) was higher than the H index (correlation coefficients of 0.81 and 0.61, respectively). Another study using a smart phone found a relationship between Hue (H index) and SPAD (0.76), but the other indices (S and B) also fitted well (Vesali et al. 2015), as observed by Wang et al. (2014). These variations can be explained by the different species studied and the influence of image accuracy, which is subject to climatic factors and illumination intensity apart from the influences of the sensors, photometric system, and processing system of the camera (Pagola et al. 2009, Wang et al. 2013). DGCI, VIgreen, and NRI indices: Figures were obtained using the described color models. There was a good linear relation (0.84) between DGCI and the Chl and Car contents (Fig. 5). Values near 1 represent a darker green. The advantage of this model is that the amount of red and blue can change the way of how often a green image appears in the RGB model (Karcher and Richardson 2003). DGCI has been among the most widely used indices to estimate the Chl content and SPAD index with high correlations (r > 0.85) in corn (Rorie et al. 2011) and grass (de Lima et al. 2012). The VIgreen index is often used for images above the vegetation covering various plants. Perhaps for this reason, there was no significant correlation to the Chl content in this study. However, in a study of rice, Wang et al. (2014) observed correlations between VIgreen index and both the SPAD and the content of nitrogen, but the coefficients observed were low (0.67) and not effective enough to be used. In this study, however, the NRI index satisfactorily correlated to the SPAD index (0.85), and the J.P.G. RIGON et al. 564 results were higher than those obtained by Wang et al. (2014). Vesali et al. (2015) could not find good results for the relationship between SPAD and leaf N status in maize using this index with a smartphone camera. However, when using the hue index (HSB model), they had the strongest linear relationship with the SPAD value, although the SPAD value was analyzed rather than the Chl and Car content. These differences between species may be related to the nonuniform distribution of Chl in leaves as an effect of the clustered structural organization of Chl molecules in chloroplasts, chloroplasts in cells, and cells in leaves (Fukshansky et al. 1993). Overall, the results showed that the color index can be used to estimate the content of photosynthetic pigments, with the advantage of allowing rapid and repeated meas- ures in the same leaves over time for ontogenetic studies. The results can be used to build applications to work with different color models efficiently using smart phones. Fig. 4. Relationships between chlorophyll and carotenoids content measured analytically in lab and the image based in HSB component model with index H (A,D), S (B,E), and B (C,F) in the leaves of the soybean. ESTIMATION OF CHLOROPHYLL BY SMART PHONE AND IMAGE ANALYSIS 565 Fig. 5. Relationships between chlorophyll and carotenoids contents measured analytically in laboratory and the image based on DGCI index (A,D), VIgreen (B,E), and NRI (C,F) in the leaves of the soybean. Conclusion: The chlorophyll meter showed a proper reliability and may be used to estimate Chl and Car contents accurately, thus saving time and chemical reagents typically used in conventional procedures. The smart phone was accurate and sensitive with good representativeness, which was not diminished even in a broad range of Chl or Car contents in soybean leaves. Although there was variation between color models, some of the proposed methods achieved very close values to the Chl meter values, like the S and B index from the HSB and LAB models and NRI (RGB model). Using these models gave sufficient accuracy and robustness towards use for rapid and accurate estimation. References Arnon D.I.: Copper enzymes in isolated chloroplasts: polyphe- noloxydase in Beta vulgaris. – Plant Physiol. 24:1-15, 1949. Confalonieri R., Foi M., Casa R. et al.: Development of an app for estimating leaf area index using a smartphone. Trueness and precision determination and comparison with other indirect methods. – Comput. Electron. Agr. 96: 67-74, 2013. Curran P.J., Dungan J.L., Gholz H.L.: Exploring the relationship between reflectance red edge and chlorophyll content in slash pine. – Tree Physiol. 7: 33-48, 1990. de Lima C.P., Backes C., Fernandes D.M. et al.: Leaves reflectance index of the bermuda grass to evaluate the nutri- tional status in nitrogen. – Ciênc. Rural. 42: 1568-1574, 2012. Fairchild M.D.: Color Appearance Models, 2nd ed. Pp. 385. Wiley–IS&T, Chichester2005. Fukshansky L., Martinez A., Remisowsky V. et al.: Absorption spectra of leaves corrected for scattering and distributional error: A radiative transfer and absorption statistics treatment. – Photochem. Photobiol. 57: 538-555, 1993. Gitelson A.A., Kaufman Y.J., Stark R.. et al.: Novel algorithms for remote estimation of vegetation fraction. – Remote Sens. Environ. 80: 76-87, 2002. Gong A., Yu J., He Y. et al.: Citrus yield estimation based on images processed by an Android mobile phone. – Biosyst. Eng. 115: 162-170, 2013. J.P.G. RIGON et al. 566 Gupta S.D., Ibaraki Y., Pattanayak A.K.: Development of a digital image analysis method for real-time estimation of chlorophyll content in micropropagated potato plants. – Plant Biotechnol. Rep. 7: 91-97, 2013. Hiscox J.D., Israelstam G.F.: A method for the extraction of chlorophyll from leaf tissue without maceration – Can. J. Bot. 57: 1332-1334, 1979. Karcher D.E., Richardson M.D.: Quantifying turfgrass color using digital image analysis. – Crop Sci. 43: 943-951, 2003. Levizou E., Drilias P., Psaras G.K. et al.: Nondestructive assessment of leaf chemistry and physiology through spectral reflectance measurements may be misleading when changes in trichome density co-occur – New Phytol. 165: 463-472, 2005. Li Y., Chen D., Walker C.N. et al.: Estimating the nitrogen status of crops using a digital camera. – Field Crop. Res. 118: 221- 227, 2010. Marenco R.A., Antezana-Vera S.A., Nascimento H.C.S.: Relationship between specific leaf area, leaf thickness, leaf water content and SPAD-502 readings in six Amazonian tree species. – Photosynthetica 47: 184-190, 2009. Markwell J., Osterman J.C., Mitchell J.L.: Calibration of the Minolta SPAD 502 leaf chlorophyll meter. – Photosynth. Res. 46: 467-472, 1995. Mielke M.S., Schaffer B., Li C.: Use of a SPAD meter to estimate chlorophyll content in Eugenia uniflora L. leaves as affected by contrasting light environments and soil flooding. – Photosynthetica 48: 332-338, 2010. Moonrungsee N., Pencharee S., Jakmunee J.: Colorimetric analyzer based on mobile phone camera for determination of available phosphorus in soil. – Talanta 136: 204-209, 2015. Neves O.S.C., Carvalho J.G., Martins F.A.D. et al.: [Use of SPAD-502 in the evaluation of chlorophyll contents and nutritional status of herbaceous cotton to nitrogen, sulphur, iron and manganese.] – Pesqui. Agropecu. Bras. 40: 517-521, 2005. [In Portuguese] Novichonok E.V., Novichonok A.O., Kurbatova J.A. et al.: Use of the atLEAF+ chlorophyll meter for a nondestructive estimate of chlorophyll content. – Photosynthetica 54: 130-137, 2016. Pagola M., Ortiz R., Irigoyen I. et al.: New method to assess barley nitrogen nutrition status based on image colour analysis, comparison with SPAD–502. – Comput. Electron. Agr. 65: 213-218, 2009. Riccardi M., Mele G., Pulvento C. et al.: Non-destructive evaluation of chlorophyll content in quinoa and amaranth leaves by simple and multiple regression analysis of RGB image components. – Photosynth. Res. 120: 263-272, 2014. Rigon J.P.G., Beltrão N.E.M., Capuani C. et al.: Non-destructive analysis of photosynthetic pigments in leaves of Sesamum indicum (L.). – Rev. Bras. Eng. Agric. Amb. 16: 258-261, 2012a. Rigon J.P.G., Capuani C., Beltrão N.E.M. et al.: Non-destructive determination of photosynthetic pigments in the leaves of castor oil plants. – Acta Sci. Agron. 34: 325-329, 2012b. Rigon J.P.G., Capuani C., Brito Neto J.F. et al.: Indirect measurement of photosynthetic pigments in the leaves of Jatropha curcas. – Semina 34: 669-674, 2013. Robertson A.R.: The CIE 1976 color-difference formulae. – Color Res. Appl. 2: 7-11, 1977. Rorie R.L., Purcell L.C., Mozaffari M. et al.: Association of “greenness” in corn with yield and leaf nitrogen concentration. – Agron. J. 103: 529-535, 2011. Sikuku P.A., Netondo G.W., Onyango J.C. et al.: Chlorophyll fluorescence, protein and chlorophyll content of three nerica rainfed rice varieties under varying irrigation regimes. – ARPN J. Agr. Biol. Sci. 5: 19-25, 2010. Sinnecker P., Salete M., Gomes, O. et al.: Relationship between color (instrumental and visual) and chlorophyll contents in soybean seeds during ripening. – J. Agr. Food. Chem. 50: 3961-3966, 2002. Steele M.R., Gitelson A.A., Rundquist D.C.: A comparison of two techniques for nondestructive measurement of chlorophyll content in grapevine leaves. – Agron. J. 100: 779-782, 2008. Su C.H., Fu C.C., Chang Y.C. et al.: Simultaneous estimation of chlorophyll a and lipid contents in microalgae by three color analysis. – Biotechnol. Bioeng. 99: 1034-1039, 2008. Torres-Netto A., Campostrini E., de Oliveira J.G. et al.: Photosynthetic pigments, nitrogen, chlorophyll a fluorescence and SPAD-502 readings in coffee leaves. – Sci. Hortic.- Amsterdam 104: 199–209, 2005. Vesali F., Omid M., Kaleita A. et al.: Development of an android app to estimate chlorophyll content of corn leaves based on contact imaging. – Comput. Electron. Agr. 116: 211-220, 2015. Vollmann J., Walter H., Sato T. et al.: Digital image analysis and chlorophyll metering for phenotyping the effects of nodulation in soybean. – Comput. Electron. Agr. 75: 190-195, 2011. Wang Y., Wang D., Zhang G. et al.: Estimating nitrogen status of rice using the image segmentation of G-R thresholding method. – Field Crop. Res. 149: 33-39, 2013. Wang Y., Wang D., Shi P.. et al.: Estimating rice chlorophyll content and leaf nitrogen concentration with a digital still color camera under natural light. – Plant Methods 10: 1-11, 2014. Wellburn A.R.: The spectral determination of chlorophylls a and b, as well as total carotenoids, using various solvents with spectrophotometers of different resolution. – J. Plant. Physiol. 144: 307-313, 1994. Williams J.D., Kitchen N.R., Scharf P.C. et al.: Within-field nitrogen response in corn related to aerial photograph color. – Precis. Agric. 11: 291-305, 2010. Yadav S., Ibaraki Y., Gupta D.S.: Estimation of the chlorophyll content of micropropagated potato plants using RGB based image analysis. – Plant Cell Tiss. Org. 100: 183-188, 2010. 11