Web Tabanlı Otomatik Test Araçları Üzerinde Geniş Bir Araştırma

Year 2023, Volume: 4 Issue: 1, 13 - 28, 10.08.2023

Ruya Samlı , Zeynep Orman

Abstract

Yazılım test sürecinde, daha iyi test sonuçları elde etmek, zamandan ve paradan tasarruf etmek için genellikle otomasyon testi gerekir. Otomasyon testi, özellikle web uygulamaları performans, güvenlik açığı, güvenlik, yük ve strese göre test edildiğinde faydalıdır. Günümüzde, farklı üreticiler tarafından geliştirilmiş, çeşitli özelliklere sahip birçok web tabanlı otomatikleştirilmiş araç bulunmaktadır. Bazıları sadece belirli amaçlar için kullanılabilir ve oldukça iyi sonuçlar verir, bazıları ise daha geniş bir alanda kullanılabilir ancak ortalama sonuçlar verir. Web tabanlı otomatik araçlar, maliyet, lisans, teknik destek, dil desteği, kullanıcı deneyimi, dokümantasyon, tarayıcı desteği, ortam desteği, test türü, donanım gereksinimleri gibi çeşitli kriterlere göre birbirinden farklılık göstermektedir. Belirli bir test süreci için hangi web tabanlı otomatik aracın uygun olduğunu belirlemek genellikle zordur. Bu araçların çoğu lisanslı ve maliyetli olduğundan, deneme yapmak ve uygun otomatik araç üzerinde karar vermek bazen imkansızdır. Literatürdeki bazı çalışmalar, sadece birkaç aracı ve sadece birkaç karşılaştırma kriterini inceleyerek bu sorunu ele almaktadır. Ancak otomatik araçların karşılaştırılması daha detaylı ele alınması gereken önemli bir konudur. Bu yazıda literatürde ilk kez 14 web tabanlı otomatikleştirilmiş araç 20 farklı kritere göre karşılaştırılmakta ve bu kapsamlı incelemenin sonuçları sunulmaktadır.

Keywords

Yazılım testi, Yazılım aracı, web uygulama testi, otomatik yazılım testi

A Comprehensive Overview of Web-Based Automated Testing Tools

Abstract

In the software testing process, automation testing is generally required for obtaining better test results and saving time and money. Automation testing is beneficial, especially when web applications are being tested according to performance, vulnerability, security, load, and stress. Today, there are many web-based automated tools with various properties, which are developed by different producers. Some of them can only be used for specific purposes and produce pretty good results, while others can be used in a broader area but produce average results. Web-based automated tools differ from each other according to various criteria such as cost, license, technical support, language support, user experience, documentation, browser support, environment support, testing type, and hardware requirements. It is often hard to determine which web-based automated tool is appropriate for a specific testing process. Since many of these tools are licensed and costly, it is sometimes impossible to make trials and decide on the proper automated tool. Some studies in the literature address this problem by examining only a few tools and only a few comparison criteria. However, the comparison of automated tools is an important issue that needs to be addressed in more detail. In this paper, for the first time in the literature, 14 web-based automated tools are compared according to 20 different criteria, and the results of this comprehensive review are presented.

Keywords

Software testing, software tool, web application testing, automated testing tool

References

• Abbas, R., Sultan, Z., & Bhatti, S. N. (2017). Comparative Analysis of Automated Load Testing Tools: Apache JMeter, Microsoft Visual Studio (TFS), LoadRunner, Siege. International Conference on Communication Technologies.
• Acunetix, https://www.acunetix.com, Retrieved 13 April 2020.
• Al-Zain, S., Eleyan, D. & Hassouneh, Y. (2013). Comparing GUI Automation Testing Tools for Dynamic Web Applications. Asian Journal of Computer and Information Systems, 1, 38–48.
• Angmo, R., & Sharma, M. (2014). Performance Evaluation of Web Based Automation Testing Tools. International Conference-Confluence The Next Generation Information Technology Summit.
• Bertolino, A. (2007). Software testing research: Achievements, challenges, dreams. Future of Software Engineering Conference.
• Bharti, T., & Dutt, E. V. (2014). Relative Review of Automated Testing Tools: (QTP) Quick Test Professional, Selenium and Test Complete. International Journal of Computer Science Trends and Technology, 2, 110–114.
• Bindal, P., & Gupta, S. (2014). Test Automation Selenium WebDriver using TestNG. Journal of Engineering Computers and Applied Sciences, 3, 18–40.
• Catelani, M., Ciani, L., Scarano, V.L., & Bacioccola, A. (2011). Software automated testing: A solution to maximize the test plan coverage and to increase software reliability and quality in use. Computer Standards and Interfaces, 33, 152–158.
• Chandel, V., Patial, S., & Guleria, S. (2013). Comparative Study of Testing Tools: Apache JMeter and Load Runner. International Journal of Computing and Corporate Research. 3, 1–17.
• Daud, N. I., Bakar, K. A. A., & Hasan, M. S. M. (2014). A Case Study on Web Application Vulnerability Scanning Tools. Science and Information Conference.
• Dubey, N., & Shiwani, S. (2014). Studying and Comparing Automated Testing Tools; Ranorex and TestComplete. International Journal Of Engineering And Computer Science, 3, 5916– 5923.
• Dukes, L., Yuan, X., & Akowuah, F. (2013). A case study on web application security testing with tools and manual testing. IEEE Southeastcon.
• Elbaum, S., Karre, S. & Rothermel, G. (2003). Improving web application testing with user session data. International Conference on Software Engineering.
• Falah, B., Akour, M., & El Marchoum, N. (2015). Testing patterns in action: Designing a test pattern-based süite. International Review on Computers and Software, 10, 489–494. FitNesse, http://docs.fitnesse.org/FrontPage, Retrieved 15 April 2020.
• Gogna, N. (2014). Study of Browser-Based Automated Test Tools WATIR and Selenium. International Journal of Information and Education Technology, 4, 336–339.
• Gogna, N., & Kumari, R. (2011). Comparative Study of Browser Of Browser Based Open Source Testing Tools WATIR And WET. International Journal on Computer Science and Engineering, 3, 1910– 1923.
• Gupta, S., Kumar, S., & Saxena, C. (2015). Review Paper on Comparison of Automation Testing Tools Selenium and QTP. MIT International Journal of Computer Science and Information Technology, 5, 55–57.
• Hedayati, A., Ebrahimzadeh, M., & Sori, A. A. (2015). Investigating Automated Test Patterns in Erratic Tests by Considering Complex Objects. International Journal of Information Technology and Computer Science, 7, 54–59.
• Jain, V., & Rajnish, K. (2018). Comparative Study of Software Automation Testing Tools: OpenScript and Selenium. International Journal of Engineering Research and Application, 8, 29– 33. JMeter, https://jmeter.apache.org, Retrieved 5 April 2020.
• Kakaraparthy, D. (2017). Overview And Analysis Of Automated Testing Tools: Ranorex, Test Complete, Selenium. International Research Journal of Engineering and Technology, 4, 1575–1579.
• Katalon Studio, https://www.katalon.com, Retrieved 1 April 2020.
• Kaur, H., & Gupta, G. (2013). Comparative Study of Automated Testing Tools: Selenium, Quick Test Professional and Testcomplete. International Journal of Engineering Research and Applications, 3, 1739–1743.
• Kaur, M., & Kumari, R. (2011). Comparative Study of Automated Testing Tools: TestComplete and QuickTest Pro. International Journal of Computer Applications, 24, 1–7.
• Kaur, P. (2021). Comparison of Automation Testing Tools for Regression Testing website, International Journal of Innovative Science. Engineering & Technology, 8, 259–265.
• Križanić, J., Grgurić, A., Mošmondor, M., & Lazarevski, P. (2010). Load testing and performance monitoring tools in use with AJAX based web applications. International Convention on Information and Communication Technology, Electronics, and Microelectronics.
• Kundu, S. (2012). Web Testing: Tool, Challenges, and Methods. International Journal of Computer Science, 9, 481–486.
• Kunte, P., & Mane, D. (2017). Automation Testing of Web-based application with Selenium and HP UFT (QTP). International Research Journal of Engineering and Technology, 4, 2579–2583.
• LoadRunner, https://www.microfocus.com/enus/products/loadrunner-professional/overview, Retrieved 3 May 2020.
• Mahmood, H., & Sirshar, M. (2017). A Case Study of Web Based Application by Analyzing Performance of a Testing Tool. International Journal of Education and Management Engineering, 7, 51–58.
• Masso, J., Pino, F. J., Pardo, C., García, F., & Piattini, M. (2020), Risk management in the software life cycle: A systematic literature review. Computer Standards and Interfaces, 71, 103431.
• Meenu, & Kumar, Y. (2015). Comparative Study of Automated Testing Tools: Selenium, SoapUI, HP Unified Functional Testing and Test Complete. Journal of Emerging Technologies and Innovative Research, 2, 42–48.
• Monier, M., & El-Mahdy, M. M. (2015). Evaluation of automated web testing tools. International Journal of Computer Applications Technology and Research. 4, 405–408.
• Patil, D. S., & Pawar, P. B. (2021). Selection Of Test Automation Tools forBFSI Sector Web Based Applications. Natural Volatiles & Essential Oils, 8, 10645–1065.
• Petukhov, A., & Kozlov, D. (2008). Detecting security vulnerabilities in web applications using dynamic analysis with penetration testing. Application Security Conference.
• Prasad, L., Yadav, R., & Vore, N. (2021). A Systematic Literature Review of Automated Software Testing Tool. International Conference on Computing Informatics and Networks, 101– 123.
• Qasaimeh, M., Shamlawi, A., & Khairallah, T. (2018). Black Box Evaluation Of Web Application Scanners: Standards Mapping Approach. Journal of Theoretical and Applied Information Technology, 96,4584–4596.
• QTP, https://www.tutorialspoint.com/qtp, Retrieved 10 May 2020.
• Radhakrishna, S., & Nachamai, M. (2017). Performance Inquisition of Web Services Using Soap UI and Jmeter. IEEE International Conference on Current Trends in Advanced Computing.
• Ranorex Studio, https://www.ranorex.com, Retrieved 12 May 2020.
• SahiPro, https://sahipro.com, Retrieved 2 May 2020.
• Selenium, https://www.selenium.dev, Retrieved 30 May 2020.
• Sharma, M., & Angmo, R. (2014). Web-based Automation Testing and Tools. International Journal of Computer Science and Information Technologies, 5, 908–912.
• Sharmila, S., & Ramadevi, E. (2014). Performance Evaluation and Comparison of Web Application Testing Tools: WAPT Pro and Apache Jmeter. International Journal for Scientific Research and Development, 2, 519–522.
• Singh, I., & Tarika, B. (2014). Comparative Analysis of Open Source Automated Software Testing Tools: Selenium, Sikuli and Watir. International Journal of Information and Computation Technology, 4, 1507–1518.
• Singh, J., & Sharma, M. (2015). Performance Evaluation and Comparison of Sahi Pro and Selenium Webdriver. International Journal of Computer Applications, 129, 23–26.
• Srivastava, N., Kumar, U., & Singh, P. (2021). Software and Performance Testing Tools. Journal of Informatics Electrical and Electronics Engineering, 2, 1–12.
• Telerik Studio, https://www.telerik.com/teststudio, Retrieved 7 September 2021.
• TestComplete, https://smartbear.com/product/testcomplete/over view, Retrieved May 3 2020.
• TestingWhiz, https://www.testing-whiz.com, Retrieved 12 May 2022.
• TestIO, https://test.io, Retrieved 1 May 2022.
• Webload, https://www.radview.com, Retrieved 3 October 2021.