Luận án Nghiên cứu thiết kế hệ thống phân loại nông sản hiệu suất cao sử dụng công nghệ xử lý ảnh kết hợp trí thông minh nhân tạo

Nội dung text: Luận án Nghiên cứu thiết kế hệ thống phân loại nông sản hiệu suất cao sử dụng công nghệ xử lý ảnh kết hợp trí thông minh nhân tạo

1. International Conference on Control, Mechatronics and Automation, 10 September 2019 Figure 9. Frequency diagram of the standardization of Histogarm from knowledge of understanding of source of the data. As it is known that mango density increased with the volume, then the quality is better and the mango is sweet (Based on regression equation of weight and volume). ANN can be seen as a form of regression equation which can model arbitrary continuous functions where an explicit model relating the functional form of the output to the inputs is known. The first stage in the computer processing of the digital images from camera is to form separate image files of mangoes. This is necessary since locating the mango within the large image would be very computationally expensive. From these resized images, the grey-scale images are formed from the sum of the red and green bands less twice the blue band. Next, the grey- scale images are threshold to form binary images. The Figure 10. Normal P-P balance diagram threshold value is simply found based on experiments for VI. CONCLUSION each type of mango (with reference to several image histograms). The mango images are calibrated for size This study described the method and terminology of by using images of ellipse. several of tolls that are used for image processing and analysis in sorting and classification of mangoes based When using artificial intelligence to determine the on Artificial Intelligence. The digital image processing quality of mangoes including the components of mango is required firstly to preprocess the data of mango images fruit, we can classify them without affecting the bad into a format from which features can be extracted, and value to the quality of mangoes, related to human health. secondly to extract and measure these features. Solving problems in mango classification system combining computer vision and artificial intelligence The fluctuation of mango fruit quality in the market is will help develop smart mango classification system huge. The best harvesting time for fruit quality depends with commercial scale. on many factors including Cat Hoa Loc mango and Cat Chu mango in Vietnam for the best quality when having REFERENCES density from 1.00 -1.02. Fruits are classified by machine [1] Chandra Sekhar Nandi, Bipan Tudu, and Chiranjib Koley, vision techniques and artificial intelligence is more Computer Vision Based Mango Fruit Grading system, International conference on Innovative Engineering uniform in quality than the left harvest by age and Technologies (ICIET’2014) Dec. 28-29, 2014 Bangkok market. Thailand. [2] Tomas U. Ganiron Jr. Size Properties of Mangoes using The mango images used in this study for sorting and Image Analysis, International Association of Engineers (IAENG) South Kowloon, Hong Kong, International Journal blemish detection are obtained using a CCD camera. of Bio-Science and Bio-Technology Vol.6, No.2 (2014), Once shape have been extracted from the mango profile pp.31-42. [3] Emny Harna Yossya, Jhonny Pranataa, Tommy Wijayaa, images and applied to artificial neural network that is Heri Hermawana, Widodo used to combine shape features to form volume estimates Budihartoa, Mango Fruit Sortation System using Neural for the corresponding mango. The testing method used Network and Computer Vision , 2nd International Conference on Computer Science and Computational on ANN and other function approximation methods are Intelligence 2017, ICCSCI 2017, 13-14 October 2017, Bali, explained in this paper. Indonesia. [4] Tajul Rosli B. Razak, Mahmod B. Othman, Mohd Nazari bin Eventually, the features are to be combined to form a Abu Bakar, Khairul Adilah bt Ahmad4, Ab Razak Mansor, Mango Grading By Using Fuzzy Image Analysis, volume estimate of fruit from whose image thay are International Conference on Agricultural, Environment and extracted and measured. Biological Sciences (ICAEBS'2012) May 26-27, 2012 Phuket. In one of its simplest forms, function approximation is [5] Mathieu Ngouajio, William Kirk, and Ronald Goldy, A Simple Model for Rapid and Nondestructive Estimation of determination of a linear regression equation based on a Bell Pepper Fruit Volume, Hort Science 38(4): 509-511, set of data. This linear relationship is a model for 2003. [6] Ms. Seema Banot1, Dr. P.M. Mahajan, A Fruit Detecting and between weight and volume, since one would expect that Grading System Based on Image Processing-Review, the volume of mango would be directly proportional to International Journal Of Innovative Research In Electrical, Electronics, Instrumentation And Control Engineering Vol. its weight, because mango density is usually almost 4, Issue 1, January 2016. constant within a same quality. A model must be formed doi: 10.1109/ICCMA46720.2019.8988603 482
2. International Journal of Machine Learning and Computing, Vol. 10, No. 2, February 2020 Sorting and Classification of Mangoes based on Artificial Intelligence Nguyen Truong Thinh, Nguyen Duc Thong, and Huynh Thanh Cong  the process of producing agricultural products on the one Abstract—For each type of mango, there are different colors, hand reduce human labor, reduce costs, and otherwise meet weights, sizes, shapes and densities. Currently, classification high standards of food safety Processing in difficult markets based on the above features is being carried out mainly by requires high quality is essential. The application of manuals due to farmers' awareness of low accuracy, high costs, automation in agriculture especially in the production and health effects and high costs, costly economically inferior. This study was conducted on three main commercial mango species processing of agricultural products is extremely necessary. of Vietnam as Cat Chu, Cat Hoa Loc and Statue of green skin to World studies of mango classification according to color, size, find out the method of classification of mango with the best volume and almost done in the laboratory but not yet applied quality and accuracy. Research on mango classification based in practice. The quality assessment of mango fruit has not on the color and volume being conducted does not meet the been resolved. So it is necessary to study image processing quality of commercial mangoes and the accuracy is not high. techniques; collect and build a database of photos of some Therefore, a method of mango classification is most effective. In this study, we have proposed and implemented methods, using types of mangoes in Vietnam; studying mango quality algorithms to analyze the content combining statistical methods approaches and techniques, examining mango surfaces that based on image processing techniques to identify commercial are deep, withered, porous, deformed mangoes, ripening on mangoes in Vietnam. The main content of this study is to mango fruit; application of image processing technology, develop an efficient algorithm to design mango classification computer vision combined with artificial intelligence in the system with high quality and accuracy. The goal of the study is problem of mango classification or poor quality. The design to create a system that can classify mangoes in terms of color, volume, size, shape and fruit density. The classification system of high-quality mango classification system based on image using image processing incorporates artificial intelligence processing technology, computer vision combines artificial including the use of CCD cameras, C language programming, intelligence effectively in accordance with the development computer vision and artificial neural networks. The system uses situation of agricultural machines today. the captured mango image, processing the split layer to Currently mangoes are classified by color, volume, size determine the mass, volume and defect on the mango fruit and shape. The quality of the mango fruit is only predicted by surface. Determine the percentage of mango defects to determine the quality of mangoes for export and domestic or the eye of the classification and has not been studied for recycled mangoes. This article is about the development of an application. Case studies of mango classification such as automatic mango classification system to control and evaluate Machine vision-based maturity prediction system for mango quality before packaging and exporting to the market. It harvested mango classification [1] proposed a machine-based is in the research, design and fabrication of mango classification system to classify mangoes by predicting levels maturity to model and the completion of an automatic mango classification replace manual classification system. Prediction of ripeness system using image processing technology combining artificial intelligence. was made from video signals collected by a CCD camera placed above the mango conveyor belt. The recursive feature Index Terms—Fruit classification, mango sorting, image removal technique combined with the vector-based support processing, artificial intelligence, computer vision. (SVM) classifier is used to identify the most relevant features of the original 27 selected features. Finally, optimal aggregation of the number of reduced features is obtained and I. INTRODUCTION used to classify mangoes into four different types according The process of grading mango in Vietnam and the world is to maturity level; Tomas U. Ganiron Jr developed a being carried out mainly by the direct labor of farmers. The size-based mango classification system using image analysis methods used by farmers and distributors to classify techniques [2]. This empirical study aims to develop an agricultural products are through traditional quality testing efficient algorithm to detect and classify mangoes. Using the with time-consuming and less efficient observations or some obtained image, the features of the mango are extracted and types of machines dedicated and result in low productivity, used to determine the mango layer. The characteristics of the high cost, sorting out different types of mangoes is relatively extracted mango are perimeter, area, roundness and defect costly. Research and application of high-tech machinery in rate; The mango classification system uses machine vision and Neural network [3] as a system that can classify ripe or unripe mangoes. The method used to carry out this study was Manuscript received April 9, 2019; revised December 11, 2019. Nguyen Truong Thinh is with the Ho Chi Minh City University of split into several steps: object identification, algorithm Technology and Education, Ho Chi Minh City, Vietnam (e-mail: development, implementation and evaluation. This system is thinhnt@hcmute.edu.vn). implemented in C, Computer Vision and ANN (artificial Nguyen Duc Thong is with Dong Thap University, Vietnam (e-mail: ndthong@dthu.edu.vn). neural networks) so that the system can detect the color of the Huynh Thanh Cong is with Vietnam National University, Ho Chi Minh ripe or unripe mangoes; The research team in Malaysia [4] City, Vietnam (e-mail: htcong@vnuhcm.edu.vn). doi: 10.18178/ijmlc.2020.10.2.945 374
3. International Journal of Machine Learning and Computing, Vol. 10, No. 2, February 2020 1) System with shooting chamber to process color images, represents the same feature. Fourth, both object features and find shape defects and calculate mango volume. window features are extracted from each located area. Fifth, 2) Loadcell system to calculate the weight of each mango. the features are passed to the neural networks and the outputs 3) The system has a wiper mechanism that eliminates of these networks are then combined using the feature unsatisfactory fruits, size, shape. combination strategy to assign an overall class to each region. 4) The system has a classification mechanism used to Finally, the mango is graded, using a set of rules, based on the classify quality of mangoes into trade items. feature type of each located region. An example of a grading Building the principle of operation of mango classification table is shown in Table I. The table shows for each grade, the model using artificial intelligence: Conveyed mango fruit number, type and size of defects that are permissible. brought to the conveyor mounted on the conveyor. In the shooting chamber, there are two cameras for color image Mangoes processing to find defects on the mango fruit surface such as: black spots, bruises, bruises, and shape defects such as waist, Image acquisition using combined damaged broken, the fruit does not meet the color front and back mango requirements, the shape will be eliminated, and the camera will also scan the mango fruit (length, width, height) to Segmentation with convolution filters calculate the volume of the mango. After that, the mango fruit, which meets the requirements of color shape, will be taken to the second conveyor to conduct mass calculations (Fig. 1). Post-processing of the segmented First, the harvested mangoes are cleaned by using a image via AI-based techniques washing solution, then sorted and sorted into commercial mangoes of different types, this is the current stage sorted by Feature extraction as: size, colour, hand. Finally, the mangoes of each classification are defect packaged and transferred to customers (Fig. 2). Synergistic classification by feature combination Expert-system grading Grade Fig. 3. Developed system for mango grading. This table can be easily converted into a rule-based expert Image Processing Computer system. For better results, fuzzy rules can be employed to Chamber Camera emulate expert human graders more closely. The Light segmentation method adopted is based on standard image-processing functions and consists of three stages. Before segmentation, two images of the two surfaces being inspected is acquired using the image from above and Sensor beneath the mango. These images contain some features Conveyor caused by classifications. Fig. 1. Laboratory testbed. The mangoes are rarely perfect spheres, most mangoes are either long (D<L). A simple way to account for variation in mango shape is to use the ratio (R) of length to diameter: Harvesting Cleaning Classification R=L/D. Corrected mango volume will, therefore, have the following equation [5]: Preservation Package Grading Spraying VVVP s s KR 1 (1) where VP is the corrected mango volume, and K is a shape Storage Transportion Users factor that varies with fruit type. After development and Fig. 2. Mango sorting process. rearrangement of Eq. 1, the following equation is obtained: VDL 1.12 6 (2) A. Inspection Process P 3 The inspection routine developed is illustrated in Fig. 3. With D and L in cm and VP in cm . First, two images of front and back surfaces are acquired All of the shape features apart from area are invariant to using two cameras. Second, check areas of the mango are size, since they are measured from profile images normalised found using segmentation modules, each specialised in to unit area. Since none of the shape features shows any detecting a different type of feature. Third, post processing is significant correlation with volume (as opposed to K), and performed to remove false objects and combine areas that since the effects of projection are small, any set of features 376
4. International Journal of Machine Learning and Computing, Vol. 10, No. 2, February 2020 Detection of defects and calculation of defect areas: made. Based on the dependence equation we have found Contour algorithm: Contour is the algorithm used in image from a type of mango Statue of green skin or Cat Chu or Cat processing to separate, extract objects, enabling the following Hoa Loc, for each type of mango we need to calculate the processing to be accurate (Fig. 6c). length and height, we deduce the corresponding volume (Fig. Classification based on area of disability. Calculate 7). Determine the area of the mango image obtained from the approximately the area of a pixel. binary image (borders), determine the length, width and Classification: Find the largest area of disability if the height from this image. Applying formula (1), (2) and disability area is larger or the area of the disability is larger Dependency equation between size and volume (3), we than the area where each disability area has a larger disability deduce the corresponding mango volume. area than allowed, mangoes are removed (Fig. 6d). A. Calculating Mango Volume by Approximate Statistical Results of measuring the actual size of a sample mango Method and the corresponding number of pixels (Fig. 6e). Each type of fruit has its own unique profile, and for each, Graph of the relationship between m and x they will correspond to a certain profile. Mango has the same common profile, quite similar to Elipson. With this method, we use the length and width of each mango to calculate the corresponding volume (Fig. 7). TABLE II: MANGO MASS WHEN DIFFERENT VELOCITIES weight when weight Number order Actual weight v = 6,31 (v/p) v = 4,21 (v/p) 1 307.938 257.5721 263.15 2 240.674 190.308 207.2061 3 246.416 179.152 212.784 4 302.36 256.9158 263.15 5 307.938 254.7831 268.728 6 302.36 240.0178 268.728 Fig. 5. Graph of the relationship between m and x-axis. 7 291.204 240.3459 251.994 8 296.782 245.9239 251.994 9 375.202 311.0551 330.4141 a 10 347.148 309.2505 302.36 ) 7 291.204 240.3459 251.994 8 296.782 245.9239 251.994 b ) c ) d ) Fig. 7. Image analysis determines mango contour to calculate volume. When we determine the length, width, height and actual e volume of the mango, we begin to find a link between them. ) We have 3 input variables (length, width, height) and an output variable (volume), using multivariate regression to find the relationship between them. We just understand that, Fig. 6. Image processing process to calculate mango volume. when we use the actual volume size of the mango to find the dependent equation, then use Kinect to calculate the length, Length (L): 13.69 cm - 426 pixels width, height and with our dependent equation we will find Width (R): 8.51 cm - 281 pixels corresponding. SPSS software supports our multivariate Height (H): 7.28 cm - 258 pixels regression to find dependent equations. We only give the The above word calculates approximately the area of a input variable and the output variable, SPSS will give us the pixel: most accurate dependency equation and related diagrams. 1369 851 SPSS software supports our multivariate regression to find 0.09732 mm2 dependent equations. We only give the input variable and the 4260 2810 output variable, SPSS will give us the most accurate Define the binary image boundary from the program you dependency equation and related diagrams. 378
5. International Journal of Machine Learning and Computing, Vol. 10, No. 2, February 2020 AUTHOR CONTRIBUTIONS [9] B. H. Zhang, W. Q. Huang, J. B. Li et al., “Principles, developments and applications of computer vision for external quality inspection of Nguyen Truong Thinh, Nguyen Duc Thong, Huynh Thanh fruits and vegetables,” Food Research International, vol. 62, 2014, pp. Cong contributed to the analysis and implementation of the 326–343. research, to the analysis of the results and to the writing of the [10] A. Alipasandi, H. Ghaffari, and S. Z. Alibeyglu, “Classification of three varieties of peach fruit using artificial neural network assisted manuscript. All authors discussed the results and contributed with image processing techniques,” International Journal of Agronomy to the final manuscript. Besides, Nguyen Truong Thinh and Plant Production, vol. 4, no. 9, pp. 2179-2186, 2013. conceived the study and were in charge of overall direction [11] M. Khojastehnazhand, M. Omid, and A. Tabatabaeef, “Development of a lemon sorting system based on color and size,” African Journal of and planning. Nguyen Truong Thinh is a corresponding Plant Science, vol. 4, no. 4, pp. 122-127, April 2010. author. [12] M. Rokunuzzaman and H. P. W. Jayasuriya, “Development of a low cost machine vision system for sorting of tomatoes,” Agric Eng Int: CIGR Journal, vol. 15, no. 1, pp. 173-180, 2013. ACKNOWLEDGMENT The authors wish to thank Ho Chi Minh City University of Copyright © 2020 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted Technology and Education, Vietnam. This study was use, distribution, and reproduction in any medium, provided the original supported financially by HCMUTE Open Lab and Ho Chi work is properly cited (CC BY 4.0). Minh City University of Technology and Education, Vietnam. Nguyen Truong Thinh is an associate professor of mechatronics at Ho Chi Minh City University of REFERENCES Technology and Education (HCMUTE). He received his Ph.D in mechanical engineering at Chonnnam [1] C. S. Nandi, B. Tudu, and C. Koley, “Computer vision based mango National University (Korea) in 2010 and obtained a fruit grading system,” in Proc. International Conference on Innovative positive evaluation as an associate professor in 2012. Engineering Technologies, Dec. 28-29, 2014, Bangkok, Thailand. His main research interests are industrial robotics, [2] T. U. Ganiron, “Size properties of mangoes using image analysis,” service robotics, mechatronics, industrial automation. International Journal of Bio-Science and Bio-Technology, vol. 6, no. 2, pp. 31-42, 2014. [3] E. H. Yossya, J. Pranataa, T. Wijayaa, H. Hermawana, and W. Budihartoa, “Mango Fruit Sortation System using Neural Network and Nguyen Duc Thong is a lecturer in physics pedagogy, Computer Vision,” in Proc. 2nd International Conference on chemistry, biology, of Dong Thap University, Computer Science and Computational Intelligence, 2017, Bali, Vietnam. He has got his master degree of science. His Indonesia. main research area is mechanical engineering. He is [4] T. Rosli, B. Razak, M. B. Othman et al., “Mango grading by using now studying a doctorate in mechatronics engineering fuzzy image analysis,” in Proc. International Conference on at Ho Chi Minh City University of Technology and Agricultural, Environment and Biological Sciences, May 26-27, 2012, Education. Phuket. [5] M. Ngouajio, W. Kirk, and R. Goldy, “A simple model for rapid and nondestructive estimation of bell pepper fruit volume,” Hort Science, vol. 38, no. 4, pp. 509-511, 2003. Huynh Thanh Cong is an associate professor of [6] S. Banot and P. M. Mahajan, “A fruit detecting and grading system Mechanical-Power Engineering. He has currently based on image processing-review,” International Journal Of served as the vice-director of Department of Science Innovative Research in Electrical, Electronics, Instrumentation and and Technology, Vietnam National University – Ho Control Engineering, vol. 4, issue 1, January 2016. Chi Minh City, Vietnam. He received his Ph.D. in [7] K. A.Vakilian and J. Massah, “An apple grading system according to mechanical engineering at Sungkyunkwan University. european fruit quality standards using gabor filter and artificial neural His major interests are concerned mechanical networks,” Scientific Study & Research Chemistry & Chemical engineering, power system, internal combustion Engineering, Biotechnology, 2016. engine. [8] J. Gill, A. Girdhar, and T. Singh, “A hybrid intelligent system for fruit grading and sorting,” International Journal on Computer Science and Engineering. 380