Efficient multi-class image-based rosemary variety verification and classification model using deep learning

Tsega Asresa^*, Melaku Bayih and Eyeuel Getachew ^*Correspondence: Tsega Asresa, Department of Computer Science, Wolaita Sodo University, Ethiopia, Email:

Author info »

Introduction

Ethiopia is a nation in which agriculture accounts for the economy of the country. In Ethiopia Rosemary is popular by its local name "Yetebes Ketel" which means a leaf used for roasting; the name arises from its widespread use for seasoning meat while roasting [1]. It is also known by an alternative local name as "Azmerino". Rosemary is a perennial shrub; if it is managed properly, it can give economical yield from 4 to 7 years [2]. In Ethiopia, rosemary is found grown in different ecologies; however, the variability study among Rosemary genetic resources was not made. Consequently our knowledge on the variability of agronomic and chemical characters that exists in Ethiopia germplasm is limited. This lack of information is the major hindrance to exploit wealth of rosemary potential in the country. Therefore, to exploit the wealth of rosemary potential for variety development, the available germplasm should be properly evaluated. Therefore this study was conducted with the objective to determine the variability among Rosemary germplasms of Ethiopia for morpho-agronomic and important quality traits. The 45% of the country GDP is agriculture and its production. Rosemary is a well-known aromatic and medicinal plants in Ethiopia and it is an ever green herb that belongs to the shrub family and it is widely used specious in Ethiopia. In Ethiopia there are three types of varity of rosemary such as WG Rosemary I, WG Rosemary II, WG Rosemary III verified by researchers in the Wondo Genet agricultural research center [3].

Nowadays, researchers attempt to develop Rosemary varity identification and verification model to support experts in the area. Multi-class image classification plays an important role in the identification of Rose varity and types (IAVORP) by using a deep learning approach.

Many types of research have been done in Rosemary identification and classification by various researchers [4]. However, those researchers did not answer the questions in what are varity of the Rosemary plant. To automate the system is easily determining the plant's part used for oil production, medicine, cosmetics, perfumery and soon. In the agriculture sector identification of the proposed study will be an automatic aromatic and medicinal plant classification model using the current state-of-the-art of deep learning. Multi-class image classification is playing an important role in the identification of Rosemary Varity Verification Model (RVVM) by using a deep learning approach.

Related works

Agriculture is the back bone of Ethiopian economy around 85% of our peoples are gaining their livelihood from agriculture [5]. The rosemary plant is fragrant that grows as a perennial rounded evergreen shrub. An aromatic, savory note can be added to recipes with the addition of rosemary. Some suggest that rosemary helps strengthen the immune system, ease muscle discomfort and native to the Mediterranean region, rosemary is a fragrant evergreen herb. It features slender, needle-like, gray-green leaves on erect woody stems. And it produces clusters of small, light blue to white flowers typically in the late spring to early summer. However, there are dozens of rosemary varity around the world below, you will find the 12 most popular types of rosemary plants. However, In Ethiopia researchers identify the rosemary varity verification WG Rosmary I, WG Rosmary II, WG Rosmary II, many peoples, botanists, taxonomists’ researchers and farmers have trouble identifying rosemary varity when breeding and chemical (ingredient) extraction in the laboratory. In Addition to this, domain experts losing their time, energy and resources in order to identify those Rosemary types. So, there is a lack of technologies that identifies and classify those Rosemary varity using current state of the art algorithm using multi-class image classification.

Extensive researches have been conducted in the area of plant image classification [6]. However, there is lack of research that classify and identify rosemary varity verification model using the current state of the art methods. Different researchers use different methods to classify the plant images the researcher used the texture based plant image classification model [7]. The proposed Rosemary varity verification model used the rosemary image as the input and the Rosemary varity as the output.

Materials and Methods

Because the research topic is still in its early stages and there aren't many studies done in this field, this study is intended to be experimental research using a mixed research approach (Qualitative and quantitative). The best strategy is an experimental design, which is the process of putting one algorithm into practice and determining its efficiency through empirical analysis [8,9]. The proposed study used convolutional neural network as current state of the art of deep learning. So, the researcher used the current state of the art transfer learning models. In order to automatically extract several distinct properties of plant leaves, this study used CNN, the foundation of deep learning algorithms. The study also used the transfer learning technique, tuning weights and training hyper parameters, and adding batch normalization at the fully connected layer of CNN. In order to create a model that would extract features uniquely and correctly identify medicinal plant components, we used the backside of leaves in this study.

Image acquisition

A high-resolution camera (TECHNO SPARK with 16 MP and SAMSUNG A30S with 25 MP) was used to gather the data. The agricultural research center at Wondo Genet provided these sample data. In order to prevent the manipulation of data, consideration was given to the direction and variation of light intensity when capturing photographs of plant leaves. To precisely extract the vein feature, the backsides of the leaves were photographed. To ensure consistency in the forecast, the focal length varies based on the breadth of the leaves. There are 3284 photos of leaves of medicinal plants in the dataset. There are an equal amount of leaf photos for each plant, yet the number of plants classified in each WG Rosmary I, WG Rosmary II, WG Rosmary III are the classes of the Data set. This rosemary types are the Varity of that are available in WondoGenet agricultural research center the varity are verified by researchers.

As we mentioned the Table 1 the researcher collected total of 3,482 Rosemary leaf images from Wondo Genet agricultural research center (Figure 1).

TABLE 1 Dataset description

Figure 1) Category distribution between rosemary classes

As we illustrated from the above Figure 1 the distribution between the classes of rosemary WG Rosmary I hold higher number of over WG Rosmary II and WG Rosmary III. So we need to balance the dataset using imbalanced learning.

Image preprocessing

The input image was reduced to 150 by 150 in order to shorten the computing time required for training. Normalization in the context of the system architecture refers to image pixel scaling. The integer values of an image pixel range from 1 to 255. Processing a big integer number in CNN can interfere with or slow down learning. As a result, the image's pixels were normalized or scaled, from 0 to 1 (Figure 2).

Figure 2) Intensity transformation between RGR image color channels

As we illustrated in the above Figure 2 the researcher tried to plot the intensity transformation graph between the red, green and blue images. As we seen from the graph the gray level transformation between image color channels near to 250 (Figure 3).

Figure 3) Conversion to gray scale image

As we mention from the above Figure 3 the researcher made different experiments to change the intensity of the original image to the gray scaled one because the model is vary within the two color model.

Feature extraction

Adjusted CNN models like Inception V3, Mobile Net and VGG16 are used to extract helpful leave features and feed them into the classifier. The CNN feature learning layer is where these feature extractions are handled. This is accomplished with the use of the ReLU activation function, which enables the models to learn more quickly and effectively by resolving the vanishing gradient problem, the Maxpooling technique, which minimizes the spatial size of the convolved image features and the filtering kernel, which slides over the image pixels and computes the dot product to produce various image features. The deep learning model automatically extracts the required plant leaf attributes in this feature extraction process. Convolutional neural network structure (Figure 4).

Figure 4) Conversion to gray scale image

As we describe in the following figure the feature learning process of the Rosmary varity classification is done in this step. During the training process of the convolutional neural network, the convolution and the pooling operations are used to extract useful features of the input image.

Proposed model

The present study employed six distinct techniques, namely data collecting, data annotation (labelling), picture preprocessing, feature extraction and training, testing (model evaluation), and categorization of Rosemary components. A high-level system architecture for the investigation is shown in Figure 1. Acquisition of the image and annotation (data labelling) are included in the input image phase. Training and feature extraction are tasks included in the model building process. The sections that follow one another provide a detailed description of each (Figure 5).

Figure 5) Feature extraction

As described in the above Figure 5 the input images are size of 150 × 150 × 3 are push in the convolution layer in order to down sample the size. The convolutional neural network extract features by itself using the max pooling layer. Finally, it goes to the fully connected layer the images are converted in to the 1D vector (Figure 6).

Figure 6) Proposed rosemary varity classifier model design

As we are illustrated from the above Figure 6 the researcher figure out the proposed deep learning based Rosemary varity verification model. The researcher collected the Rosemary leaf images and preprocessed the image, finally the researcher tied to split the images data to training testing set.

Results and Discussion

Result analysis using convolutional neural network

In this section, we used the typical convolutional neural network to provide a few experimental outcomes. Eighty percent of the dataset was utilized for training and twenty percent was used for testing during the experiment. In the first, second and third convolution layers, the researcher utilized 16, 32 and 64 filters, respectively. In this work, 3482 color image datasets were used to build a customized CNN model by the researcher. These datasets have been labeled with the appropriate variance. Deep convolutional neural networks have shown enhanced performance in image categorization in recent years. Because of this, all of the experiments are conducted in hyper parameters using the basic model of the three-layer CNN. In our experiment, we feed a set of 150 × 150 × 3 down-sampled RGB photos into a model. The researcher employed a 2 × 2 max pooling parameter and a 3 × 3 kernel size to build a bespoke CNN model. With the help of the convolutional neural network model, the researcher ran a number of experiments. The data set was split into three groups by the researcher: 70/30, 80/20 and 90/10. For the objectives of testing and training, respectively (Tables 2-4 and Figures 7,8).

TABLE 2 Result analysis using CNN

Figure 7) Training and validation accuracy

TABLE 3 Result using different activation function

Figure 8) Confusion matrix for CNN model

TABLE 4 Result analysis using convolutional neural network

Result analysis using inception V3 network

Google and Google Net developed the effective deep learning architecture known as inception V3, which gets its name from a well-liked internet meme. Although there are numerous layers and neurons in the suggested architecture, computation is only done by one layer. When discussing a deeper network, it's important to remember that the likelihood of the model being overfitting rises with the number of network layers. The inception model adds sparsely linked filters with several layers within a single layer to address this. The model was trained on top of the image net dataset, which consists of around a million photos from the image database, by accepting 299 × 299 × 3 images as input and 1000 classes as output (Tables 5 and 6 and Figures 9, 10).

TABLE 5 Result analysis using inception V3 network

Figure 9) Model accuracy and loss

Figure 10) Confusion matrix for the inception V3

TABLE 6 Result analysis using convolutional neural network

Result analysis using Xception

Francois Chollet proposes the Xception model. Xception is a modification of the inception architecture that uses depth wise separable convolutions in place of the normal inception modules (Table 7).

TABLE 7 Precision recall and F1 score

As we are depected from the above Table 7 58% are precise in WG Rosmary I and 86% is precise in the WG Rosmary II and 91% are WG Rosmary III.

Conclusion

Humans have always used Rosemary plants extensively for medicine, food, decoration, perfumery and oils. They have a significant impact on the economic, social, cultural and environmental aspects of local communities. Botanists, researchers and domain experts are facing difficulty to identify the varity of rosemary using the current state of the art algorithms. In this study, there researcher used an attempt was made to develop an optimal model for rosemary varity verification model. To ensure the study's success, the researchers collected sample leaf images from 3 class of rosemary varity. The leaf samples were obtained from the Wondo Genet Agricultural Research Center (WARC) a total of 3482 images with 3 categories. In this study, we used a deep learning to predict three different types of Rosmary (WG Rosmary I, WG Rosmary II, and WG Rosmary III) using data from the Wondo Genet agricultural research center. This variety of Rosmary was verified for aromatic and threptic plants in the Wondo Genet agricultural research center which included 3484 photos. To that end, the researchers utilized an experimental research design that involved data set preparation for training, and testing to evaluate the classification model. The images of aromatic medicinal plants are resized to 150 × 150 × 3 and fed into the neural network, where the convolutional neural network extracts the features dynamically. To perform training and prediction, we employed a basic CNN. We trained our model using randomly selected example photos that were initially preprocessed into pixels. We obtained 97.07% accuracy and 65% validation accuracy after training. We encountered the problem of low validation accuracy during model training. Increasing the number of CNN layers may improve validation accuracy, increasing the percentage and improving the quality of the data we receive.

References

1. Tigist M, Beemnet G, Muluken M, et al. Rosemary production and utilization. Ethiop Inst Agric Res. 2016;67:67.
2. Beemnet MK, Basazenew D, Desta F. Variability in Ethiopian rosemary (Rosmarinus officinalis L.) collections for agronomic and chemical traits. InFourth Biennial Conference of the Ethiopian Horticultural Science Society. 2013.
3. Banjaw D, Wolde TG, Gebre A, et al. Rosemary (Rosmarinus officinalis L.) variety verification trial at Wondogenet, South Ethiopia. Med Aromat Plants. 2016;5(267):2167-0412.
4. Patil SS, Admuthe LS. Ayurvedic plant leaf classification using image processing techniques and SVM. Intern Res J Engg Technol. 2020;7(09):2395-0056.
5. Ethiopia O. Federal Democratic Republic of Ethiopia. Enhancing economic development and job creation in Addis Ababa: The role of the city administration. 2018.
6. Batchuluun G, Nam SH, Park KR. Deep learning-based plant-image classification using a small training dataset. Mathematics. 2022;10:3091.

   [Crossref] [Google Scholar]

7. Rashad MZ, El-Desouky BS, Khawasik MS. Plants images classification based on textural features using combined classifier. Int J Comput Technol. 2011;3:93-100.

   [Google Scholar]

8. Abbuhl R, Gass S, Mackey A. Experimental research design. Res Methods Linguistics. 2013;1:116-134.
9. McDonough K. Experimental research methods. Routledge Handbook. 2017:562-576.