Power, Control, and Data Processing Systems

Power, Control, and Data Processing Systems

Empirical Evaluation of Well-known Farsi OCR Engines on the IDPL-PFOD Dataset

Document Type : Original Research

Authors
Fatemeh-sadat Hosseini 1
Elham Shabaninia 2
Hossein Nezamabadi-pour 1
1 Department of Electrical Engineering, Shahid Bahonar University of Kerman, Kerman, Iran
2 Department of Applied mathematics, Graduate University of Advanced Technology, Kerman, Iran
10.30511/pcdp.2025.2052926.1018
Abstract
Optical character recognition (OCR), also referred to as text recognition, extracts text from scanned documents and camera images. OCR has numerous applications in reading forms and cheques, converting archived documents to digital files, and reading books and papers. An accurate OCR system speeds up these processes by eliminating time-consuming manual tasks. Despite advancements, OCR remains challenging for languages like Farsi due to their complex structures and lack of diverse datasets for evaluation. IDPL-PFOD, a new synthetic Farsi dataset, addresses this gap by including various backgrounds, fonts, distortions, and blurs. This paper evaluates Tesseract and EasyOCR on IDPL-PFOD to highlight their limitations. Tesseract and EasyOCR achieve 84.41% and 73.28% overall accuracy, respectively. Tesseract outperforms EasyOCR in most cases but drops to 67.17% on textured backgrounds, while EasyOCR achieves 75.8%. Tesseract reaches its peak accuracy (92.45%) on plain backgrounds and shows robustness to Gaussian blur and slope distortion, with minimal accuracy drops. In contrast, EasyOCR performs poorly on salt-and-pepper noise (4.79%), and moderately on sinewave distortion and Gaussian blur (61.45% and 63.73%). These findings emphasize the need for training OCR engines on more diverse datasets to improve their performance under real-world conditions.
Keywords
IDPL-PFOD Dataset
Optical Character Recognition (OCR)
EasyOCR
Tesseract
OCR engines
Farsi OCR

Subjects

E. A. Santos, "OCR evaluation tools for the 21st century," in Proceedings of the Workshop on Computational Methods for Endangered Languages, 2019, vol. 1. 
[2]    K. Nikoghosyan, "OCR ENGINE COMPARISON-TESSERACT VS. EASYOCR VS. KERAS-OCR," in СОВРЕМЕННАЯ НАУКА И ТЕХНИЧЕСКИЙ ПРОГРЕСС. НОВАЯ ПРОМЫШЛЕННАЯ РЕВОЛЮЦИЯ В ЗЕРКАЛЕ СОВРЕМЕННОЙ НАУКИ, 2022, pp. 61-67. 
[3]    R. R. TN and S. Praveen, "Application of OCR to Design an Intelligent PDF Parser for Technical Documentation."
[4]    M. Ganis, C. L. Wilson, and J. L. Blue, "Neural network-based systems for handprint OCR applications," IEEE Transactions on Image Processing, vol. 7, no. 8, pp. 1097-1112, 1998.
[5]    R. Gossweiler, M. Kamvar, and S. Baluja, "What's up CAPTCHA? A CAPTCHA based on image orientation," in Proceedings of the 18th international conference on World wide web, 2009, pp. 841-850. 
[6]    J. Barwick, "Building an institutional repository at Loughborough University: some experiences," Program, 2007.
[7]    S. Naz, A. I. Umar, S. B. Ahmed, S. H. Shirazi, M. I. Razzak, and I. Siddiqi, "An Ocr system for printed Nasta'liq script: A segmentation based approach," in 17th IEEE International Multi Topic Conference 2014, 2014: IEEE, pp. 255-259. 
[8]    S. S. Tsai, H. Chen, D. Chen, G. Schroth, R. Grzeszczuk, and B. Girod, "Mobile visual search on printed documents using text and low bit-rate features," in 2011 18th IEEE International Conference on Image Processing, 2011: IEEE, pp. 2601-2604. 
[9]    R. Chen, B. C. Desai, and C. Zhou, "CINDI robot: an intelligent Web crawler based on multi-level inspection," in 11th International Database Engineering and Applications Symposium (IDEAS 2007), 2007: IEEE, pp. 93-101. 
[10]    Y. K. Ham, M. S. Kang, H. K. Chung, R.-H. Park, and G. T. Park, "Recognition of raised characters for automatic classification of rubber tires," Optical Engineering, vol. 34, no. 1, pp. 102-109, 1995.
[11]    G. N. DeSouza and A. C. Kak, "Vision for mobile robot navigation: A survey," IEEE transactions on pattern analysis and machine intelligence, vol. 24, no. 2, pp. 237-267, 2002.
[12]    S. Kashef, H. Nezamabadi-pour, and E. Shabaninia, "A review on deep learning approaches for optical character recognition with emphasis on Persian, Arabic and Urdu scripts," Journal of Machine Vision and Image Processing, vol. 8, no. 4, pp. 51-85, 2021.
[13]    P. J. Haghighi, N. Nobile, C. L. He, and C. Y. Suen, "A new large-scale multi-purpose handwritten Farsi database," in International Conference Image Analysis and Recognition, 2009: Springer, pp. 278-286. 
[14]    R. Azmi and E. Kabir, "A New Segmentation Technique for Omnifont Farsi Text," Computational Methods in Engineering, vol. 18, no. 2, pp. 1-10, 1999.
[15]    R. Safabaksh, A. R. Ghanbarian, and G. Ghiasi, "HaFT: A handwritten Farsi text database," in 2013 8th Iranian Conference on Machine Vision and Image Processing (MVIP), 10-12 Sept. 2013 2013, pp. 89-94, doi: 10.1109/IranianMVIP.2013.6779956. 
[16]    S. Torabzadeh and R. Safabaksh, "AUT-PFT: A real world printed Farsi text image dataset," in 2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP), 2015: IEEE, pp. 267-272. 
[17]    F. sadat Hosseini, E. Shabaninia, S. Kashef, and H. Nezamabadi-pour, "IDPL-PFOD: An Image Dataset of Printed Farsi Text for OCR Research," NSURL 2021, p. 1, 2021.
[18]    F. K. Jaiem, S. Kanoun, M. Khemakhem, H. E. Abed, and J. Kardoun, "Database for Arabic printed text recognition research," in International Conference on Image Analysis and Processing, 2013: Springer, pp. 251-259. 
[19]    F. Solimanpour, J. Sadri, and C. Y. Suen, "Standard databases for recognition of handwritten digits, numerical strings, legal amounts, letters and dates in Farsi language," in Tenth International workshop on Frontiers in handwriting recognition, 2006: Suvisoft. 
[20]    Z. Khosrobeygi, H. Veisi, H. Ahmadi, and H. Shabanian, "A rule-based post-processing approach to improve Persian OCR performance," Scientia Iranica, vol. 27, no. 6, pp. 3019-3033, 2020.
[21]    Y. A. Nanehkaran, D. Zhang, S. Salimi, J. Chen, Y. Tian, and N. Al-Nabhan, "Analysis and comparison of machine learning classifiers and deep neural networks techniques for recognition of Farsi handwritten digits," The Journal of Supercomputing, vol. 77, no. 4, pp. 3193-3222, 2021.
[22]    R. Smith, "An overview of the Tesseract OCR engine," in Ninth international conference on document analysis and recognition (ICDAR 2007), 2007, vol. 2: IEEE, pp. 629-633. 
[23]    B. Shi, X. Bai, and C. Yao, "An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition," IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 11, pp. 2298-2304, 2016.
[24]    H. Zhang, E. Whittaker, and I. Kitagishi, "Extending TrOCR for Text Localization-Free OCR of Full-Page Scanned Receipt Images," arXiv preprint arXiv:2212.05525, 2022.
[25]    E. Shabaninia, H. Nezamabadi-pour, and F. Shafizadegan, "Transformers in Action Recognition: A Review on Temporal Modeling," arXiv preprint arXiv:2302.01921, 2022.
[26]    B. Alizadehashraf and S. Roohi, "Persian handwritten character recognition using convolutional neural network," in 2017 10th Iranian Conference on Machine Vision and Image Processing (MVIP), 2017: IEEE, pp. 247-251. 
[27]    Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
[28]    E. Farahbakhsh, E. Kozegar, and M. Soryani, "Improving Persian digit recognition by combining data augmentation and AlexNet," in 2017 10th Iranian Conference on Machine Vision and Image Processing (MVIP), 2017: IEEE, pp. 265-270. 
[29]    S. Khosravi and A. Chalechale, "Chimp Optimization Algorithm to Optimize a Convolutional Neural Network for Recognizing Persian/Arabic Handwritten Words," Mathematical Problems in Engineering, vol. 2022, 2022.
[30]    K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. 
[31]    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708. 
[32]    Y. Yu, X. Si, C. Hu, and J. Zhang, "A review of recurrent neural networks: LSTM cells and network architectures," Neural computation, vol. 31, no. 7, pp. 1235-1270, 2019.
[33]    J. Wang and X. Hu, "Gated recurrent convolution neural network for ocr," Advances in Neural Information Processing Systems, vol. 30, 2017.
[34]    A. Ranjkesh Rashtehroudi, A. Akoushideh, and A. Shahbahrami, "Persian Scene Text Recognition with Convolutional Recurrent Neural Network," Alireza and Shahbahrami, Asadollah, Persian Scene Text Recognition with Convolutional Recurrent Neural Network, 2022.
[35]    Z. Wojna, A. N. Gorban, D.-S. Lee, K. Murphy, Q. Yu, Y. Li, and J. Ibarz, "Attention-based extraction of structured information from street view imagery," in 2017 14th IAPR international conference on document analysis and recognition (ICDAR), 2017, vol. 1: IEEE, pp. 844-850. 
[36]    R. Atienza, "Vision transformer for fast and efficient scene text recognition," in International Conference on Document Analysis and Recognition, 2021: Springer, pp. 319-334. 
[37]    K. Barrere, Y. Soullard, A. Lemaitre, and B. Coüasnon, "Transformers for historical handwritten text recognition," in Doctoral Consortium-ICDAR 2021, 2021. 
[38]    P. Jain, K. Taneja, and H. Taneja, "Which OCR toolset is good and why: A comparative study," Kuwait Journal of Science, vol. 48, no. 2, 2021.
[39]    T. M. Breuel, "The OCRopus open source OCR system," in Document recognition and retrieval XV, 2008, vol. 6815: SPIE, pp. 120-134. 
[40]    D. E. Wood, J. Lu, and B. Langmead, "Improved metagenomic analysis with Kraken 2," Genome biology, vol. 20, pp. 1-13, 2019.
[41]    J. Schulenburg, "GOCR: Open-source character recognition, version 0.39," http://jocr. sourceforge. net/index. html, 2004.
[42]    GNU, "Ocrad - The GNU OCR," 2022. T. Berg-Kirkpatrick, G. Durrett, and D. Klein, "Unsupervised transcription of historical documents," in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2013, pp. 207-217. 
[43]    T. Berg-Kirkpatrick, G. Durrett, and D. Klein, "Unsupervised transcription of historical documents," in Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2013, pp. 207-217.
[44]    "SwiftOCR," 2020. [Online]. Available: https://github.com/NMAC427/SwiftOCR.
[45]    "Attention-OCR." [Online]. Available: https://github.com/emedvedev/attention-ocr.
[46]    D. Rybach, C. Gollan, G. Heigold, B. Hoffmeister, J. Lööf, R. Schlüter, and H. Ney, "The RWTH Aachen University open source speech recognition system," in Tenth Annual Conference of the International Speech Communication Association, 2009. 
[47]    "Simple-OCR-OpenCV." [Online]. Available: https://github.com/goncalopp/simple-ocr-opencv.
[48]    C. Reul et al., "OCR4all—An open-source tool providing a (semi-) automatic OCR workflow for historical printings," Applied Sciences, vol. 9, no. 22, p. 4853, 2019.
[49]    H. Feng, Y. Wang, W. Zhou, J. Deng, and H. Li, "Doctr: Document image transformer for geometric unwarping and illumination correction," arXiv preprint arXiv:2110.12942, 2021.
[50]    "PaddleOCR." [Online]. Available: https://github.com/PaddlePaddle/PaddleOCR.
[51]    M. Troller, "Practical OCR system based on state of art neural networks," Czech Technical University in Prague Dejvice, Czech Republic, 2017. 
[52]    "ClaraOCR." https://directory.fsf.org/wiki/Clara_OCR (accessed.
[53]    "CuneiForm." "EyeOCR." [Online]. Available: https://sourceforge.net/projects/eyeocr/.
[54]    "EyeOCR." [Online]. Available: https://sourceforge.net/projects/eyeocr/.
[55]    "kognition." [Online]. Available: https://sourceforge.net/projects/kognition/.
[56]    "OCRchie." [Online]. Available: https://people.eecs.berkeley.edu/~fateman/kathey/ocrchie.html.
[57]    "orce." [Online]. Available: http://lem.eui.upm.es/ocre.html.
[58]    "Xplab." [Online]. Available: http://www.pattern-lab.de/.
[59]    "hebOCR." [Online]. Available: https://github.com/yaacov/hebocr.
[60]    A. A. Asadi. Shotor Dataset. [Online]. Available: https://www.kaggle.com/amir137825/persianocrdataset/version/2
[61]    Partdpai.ir. Persian Subwords. [Online]. Available: https://github.com/partdpai/persian_subwords
[62]    F. Arefi. Arshasb Persian OCR Dataset. [Online]. Available: https://github.com/persiandataset/Arshasb
[63]    Y. Baek, B. Lee, D. Han, S. Yun, and H. Lee, "Character region awareness for text detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9365-9374. 
[64]    M. Li et al., "Trocr: Transformer-based optical character recognition with pre-trained models," in Proceedings of the AAAI Conference on Artificial Intelligence, 2023, vol. 37, no. 11, pp. 13094-13102. 
[65]    S. Saber, A. Ahmed, and M. Hadhoud, "Robust metrics for evaluating Arabic OCR systems," in International Image Processing, Applications and Systems Conference, 2014: IEEE, pp. 1-6. 
[66]    F. B. Safir, A. Q. Ohi, M. F. Mridha, M. M. Monowar, and M. A. Hamid, "End-to-end optical character recognition for bengali handwritten words," in 2021 National Computing Colleges Conference (NCCC), 2021: IEEE, pp. 1-7
Power, Control, and Data Processing Systems
Volume 2, Issue 1
Winter 2025
Pages 35-47

  • Receive Date 08 February 2025
  • Revise Date 17 March 2025
  • Accept Date 18 March 2025
  • First Publish Date 18 March 2025
  • Publish Date 01 March 2025