Skip to content

Latest commit

 

History

History
273 lines (211 loc) · 32.3 KB

java-test-automation.md

File metadata and controls

273 lines (211 loc) · 32.3 KB
Raw

Java test automation

A comprehensive curated list of Java test automation frameworks, tools, libraries and software to help software engineers easily bootstrap test automation on Java. Sponsored by http://sdclabs.com

Have questions\issues\problems, join the chat at https://gitter.im/atinfo/awesome-test-automation

Table of content:

Also:

xUnit frameworks

  • JUnit - Common testing framework.
  • TestNG - TestNG - Testing framework.
    • Test Data Supplier - TestNG DataProvider wrapper which helps to supply test data in a more flexible way.
  • Sunshine - Sunshine is a wrapper on Java xUnit test runners (such as TestNG, Junit...) which allows automatically find classes with tests within the jar file, passes them to desired test runner and reports an execution status.

TDD \ ATDD \ BDD

  • tools
    • JBehave - A framework for Behaviour-Driven Development (BDD). BDD is an evolution of test-driven development (TDD) and acceptance-test driven design, and is intended to make these practices more accessible and intuitive to newcomers and experts alike.
    • Cucumber-JVM - A pure Java implementation of Cucumber that supports the most popular programming languages for the JVM.
    • JGiven - A developer-friendly and pragmatic BDD tool for Java. Developers write scenarios in plain Java using a fluent, domain-specific API, JGiven generates reports that are readable by domain experts.
    • easyb - A behavior driven development framework for the Java platform. By using a specification based Domain Specific Language, easyb aims to enable executable, yet readable documentation.
    • Robot Framework - A generic test automation framework for acceptance testing and acceptance test-driven development (ATDD).
    • Spectrum - A BDD-style test runner for Java 8. Inspired by Jasmine, RSpec, and Cucumber.
    • Gauge - Gauge is a light-weight cross-platform test automation tool with the ability to author test cases in the business language.
    • Spock - Specification testing framework for Java and Groovy.
    • Concordion - Flexible, extensible BDD/SBE tool that creates beautiful living documentation using business language
  • extensions
    • cucumber-report-db - Stores results of BDD tests with Cucumber-JVM in a database and provides reporting capabilities.

Model-Based Testing

  • GraphWalker - A framework for Model-Based Testing. The tool reads models in various formats like graphml, dot or json, and then generates tests from the models.

Code analysis and coverage

  • SonarQube - Open source project to manage code quality.
  • Gradle Quality Plugin - Static code analysis for Java and Groovy projects using Checkstyle, PMD, FindBugs and CodeNarc. Plugin implements unified console output for all quality plugins which greatly simplifies developer workflow: only console is required for working with violations and makes it feel the same as java compiler errors.
  • Qulice - Qulice is a static analysis quality control instrument for Java projects. It combines all the best static analysis instruments and pre-configure them. You don't need to use and configure them individually any more.
  • JaCoCo - JaCoCo is a free code coverage library for Java, which has been created by the EclEmma team based on the lessons learned from using and integration existing libraries for many years.

Web UI test automation

  • libraries

    • Selenium - Portable software testing framework for web applications.
    • SikuliX - SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.

  • frameworks and wrappers

    • Selenide - Concise API around Selenium to write stable and readable UI tests.
    • Selenified - An open source framework to simplify Selenium Testing. It provides a wrapper for Selenium calls to add detailed reporting, error handling, simple test setup in a thread-safe manner, and can run either locally or in the cloud (Grid or SauceLabs).
    • Serenity BDD (Thucydides) - An innovative open source library that helps you write more effective automated acceptance tests, and uses these acceptance tests to generates rich documentation and reports about your product and project.
    • htmlelements - A Java framework providing easy-to-use way of interaction with web-page elements in web-page tests.
    • atlassian-selenium - An open-source (BSD) project that aims at facilitating development of functional tests in Selenium/WebDriver libraries.
    • stevia - Open Source QA Automation Testing Framework by Persado
    • darcy - An open source Java 8 framework for modeling user interfaces as page objects with a declarative, automation-library-agnostic DSL
    • Satisfy - An open source Java framework based on Thucydides + Jbehave. Supports to work with WebUI, SOAP, REST, emails, files and generate random data out of the box.
    • JDI Light - is the test Framework for UI test automation that helps to makes your tests fast and sustainable and provide obvious and predictable test run result.
    • Geb Framework - A groovy test automation framework designed for the use with the Webdriver Page Object model and the Spock Framework for (BDD).
    • FluentLenium - FluentLenium helps you writing readable, reusable, reliable and resilient UI functional tests for the browser. FluentLenium provides a Java fluent interface to Selenium, and brings some magic to avoid common issues faced by Selenium users.
    • Selion - builds on top of TestNG and Selenium to provide a set of capabilities that get you up and running with WebDriver in a short time. It can be used for testing web and mobile applications.
    • Frameworkium - automation framework for web, app, and API testing. Integrates: saucelabs, allure, rest-assured, jackson, gson. Published on Jitpack repository.
    • Carina - Carina is a Java-based test automation framework that unites all testing layers: Mobile applications (web, native, hybrid), WEB applications, REST services, Databases.
    • NoraUi - NoraUi, for NOn-Regression Automation for User Interfaces, is a Java framework based on Selenium, Cucumber and Gherkin stack to create GUI testing projects that can be included in the continuous integration chain of single/multi applications web solution builds.
    • Cubano - Cubano is a test automation framework written in Java that provides a structure for developing acceptance and regression tests so your team can hit the ground running and not have to waste time needlessly building and maintaining your own framework.
    • Jalenium - Jalenium is a Java Selenium API which can be easily integrated to any maven Selenium Java project which uses Selenium JAR files.
    • TestZeus - Open source UI Test automation framework built specifically for Salesforce with Autolocators.

  • extensions

    • BrowserMob Proxy - Is a simple utility that makes it easy to capture performance data from browsers, typically written using automation toolkits such as Selenium and Watir.
    • Selenoid - Selenium Hub successor running browsers within containers. Scalable, immutable, self hosted Selenium-Grid on any platform with single binary.
    • Selenium-Grid-Extras - Simplify the management of the Selenium Grid Nodes and stabilize said nodes by cleaning up the test environment after the build has been completed
    • Selenium Grid Extensions - Extend Selenium grid with extra functionality. Execute Sikuli tests in combination with Selenium.
    • Selenium Grid Router is a lightweight server that routes and proxies Selenium Wedriver requests to multiple Selenium hubs.
    • Docker Selenium Grid - A project to provide native video recording support for Selenium Grid and was initially designed to be used with docker-selenium project.
    • Video Recorder Java - This library allows easily record video of your UI tests by just putting couple annotations.
    • Zalenium - Allows anyone to have a disposable and flexible Docker-based Selenium Grid infrastructure featuring video recording, live preview and online/offline dashboards.
    • SikuliFactory - A based PageFactory model for SikuliX.
    • Mailosaur - Java client for email testing/automation via Mailosaur.
    • TrueAutomation.IO - One of the main tasks of TA is to solve problem of working with unstable locators.

Mobile test automation

  • Appium - An open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol.
  • Calabash - A cross-platform test automation framework for Android and iOS native and hybrid applications. Calabash’s easy-to-understand syntax enables even non-technical people to create and execute automated acceptance tests for apps on both of these mobile platforms.
  • Robotium - An Android test automation framework that fully supports native and hybrid applications. Robotium makes it easy to write powerful and robust automatic black-box UI tests for Android applications. With the support of Robotium, test case developers can write function, system and user acceptance test scenarios, spanning multiple Android activities.
  • UIautomator - Provides an efficient way to test UIs. It creates automated functional test cases that can be executed against apps on real Android devices and emulators. It includes a viewer, which is a GUI tool to scan and analyze the UI components of an Android app.
  • Espresso - A pretty new test automation framework that got open-sourced just last year, making it available for developers and testers to hammer out their UIs. Espresso has an API that is small, predictable, easy to learn and built on top of the Android instrumentation framework. You can quickly write concise and reliable Android UI tests with it.

API test automation

  • hikaku - A library that tests if the implementation of a REST-API meets its specification.
  • Karate-DSL - Karate is a BDD javascript framework which enables you to script a sequence of calls to any kind of web-service and assert that the responses are as expected. It makes it really easy to build complex request payloads, traverse data within the responses, and chain data from responses into the next request. Karate's payload validation engine can perform a 'smart compare' of two JSON or XML documents without being affected by white-space or the order in which data-elements actually appear, and you can opt to ignore fields that you choose.
  • Retrofit - A type-safe HTTP client for Android and Java.
  • REST-Assured - A library for testing and validation of REST services in Java.

Windows UI test automation

  • SikuliX - SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.
  • Winium for Desktop - Winium.Desktop is an open source test automation tool for automated testing of Windows application based on WinFroms and WPF platforms.
  • WinAppDriver - Windows Application Driver (WinAppDriver) is a service to support Selenium-like UI Test Automation on Windows Applications. This service supports testing Universal Windows Platform (UWP), Windows Forms (WinForms), Windows Presentation Foundation (WPF), and Classic Windows (Win32) apps on Windows 10 PCs.

Unix \ Linux UI test automation

  • SikuliX - SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.

MacOS UI test automation

  • SikuliX - SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify and control GUI components. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on.

Server side test automation

  • Citrus - Test framework written in Java that is able to create fully automated end-to-end use case tests for enterprise SOA applications. Citrus simulates surrounding interface partners supporting a huge set of different transports and protocols like HTTP, JMS, TCP/IP, FTP, SOAP, XML and JSON.

Virtual environments

  • Vagrant - Easy to configure, reliable tool for building and maintaining portable virtual development environments.
  • Docker - Docker is the world’s leading software container platform. Developers use Docker to eliminate “works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux and Windows Server apps.

Performance & stress & load

  • Apache JMeter - Functional testing and performance measurements.
  • The Grinder - Load testing framework that makes it easy to run a distributed test using many load injector machines.
  • Lightning - Integrate JMeter non-functional tests with CI/CD server.
  • Tsung - Erlang based multi-protocol distributed load testing tool.
  • Gatling - Gatling is an open-source load testing framework based on Scala, Akka and Netty. Is a highly capable load testing tool. It is designed for ease of use, maintainability and high performance. Based on an expressive DSL, the scenarios are self explanatory. They are easy to maintain and can be kept in a version control system. Gatling architecture lets implement virtual users as messages instead of dedicated threads, making them very resource cheap. Thus, running thousands of concurrent virtual users is not an issue.
  • CRUX - (pron. kruhks) is build around Kubernetes (AKS) and JMeter. It makes it easy to incorporate performance tests into your Azure CI/CD pipeline. Each performance build can create and destroy infrastructure necessary for the run, which makes it cost effective. CRUX deploys a distributed JMeter cluster on Kubernetes (AKS) with the size you need, runs tests, destroys infrastructure. If you do not have an AKS, be at ease, you can also run JMeter tests with CRUX directly on build agent with docker.

Security checking

  • BDD-Security - BDD-Security is a security testing framework that uses natural language in a Given, When, Then Gherkin syntax to describe security requirements as features. Those same requirements are also executable as standard unit/integration tests which means they can run as part of the build/test/deploy process.

Build Tools

  • Apache Ant - Apache Ant is a Java library and command-line tool whose mission is to drive processes described in build files as targets and extension points dependent upon each other. The main known usage of Ant is the build of Java applications.
  • Apache Maven - Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information.
  • Gradle - Gradle is an open source build automation system. Gradle can automate the building, testing, publishing, deployment and more of software packages or other types of projects such as generated static websites, generated documentation or indeed anything else.

Continuous Integration

  • Jenkins - Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron.
  • TeamCity - Powerful and user-friendly Continuous Integration and Deployment server that works out of the box.
  • Drone Drone is a self-service Continuous Integration platform for busy development teams.
  • Thundra Foresight Thundra Foresight is a CI observability tool that provides visibility into your test suites.

Reporting

  • ReportPortal - powerful server-client reporting tool. Reduce the effort to work with results. Powered with Machine Learning, Providing historical data (statues) of executions in on click, logs, screenshost and any binary attachement. Trends, flaky test, most failed, longest test via custom widgets and dashboards, which give visibility to the team, leads, managers and falcon-eye view for VPs. Provide ability to categorize fails by custom defect types and utilize power of Machine Learning to detect fails, based on collected patterns. Give benefits of real-time integration: no need to wait execution ending. Any language, any platform. Free, Open Sourced. Java integrations
  • Allure - Open-source framework designed to create test execution reports clear to everyone in the team.
  • Gradle Allure Plugin - 3rd-party Gradle Allure Plugin allows you to integrate Allure into spock, testing and junit tests.
  • ExtentReports - HTML reporting library for .NET and Java which is extremely easy to use and creates beautiful execution reports. It shows test and step summary, test steps and status in a toggle view for quick analysis.
  • ReportNG - ReportNG is a simple HTML reporting plug-in for the TestNG unit-testing framework.
  • Zafira - Zafira is central automation reporting system that is build on the top of Java Spring Framework. It dramatically increases the transparany of test automation results and provides better undestanding of product quality.
  • Difido-reports - This project aims to provide a generic implementation for HTML test reports.
  • cucumber-reporting - This is a Java report publisher primarily created to publish cucumber reports on the Jenkins build server. It publishes pretty html reports with charts showing the results of cucumber runs. It has been split out into a standalone package so it can be used for Jenkins and maven command line as well as any other packaging that might be useful. Generated report has no dependency so can be viewed offline.
  • Cluecumber - This is reporting library for the Cucumber JSON format as it is produced by Cucumber and Karate. It provides clear and concise HTML reports and is highly customizable.

Documentation generation

Contribute to this section

Editors, IDE, consoles

  • IntelliJ IDEA - A free and open-source IDE for Java, Groovy, Scala and Android development.
  • Eclipse - Eclipse is famous for our Java Integrated Development Environment (IDE), C/C++ IDE and PHP IDE.
    • Genie plugin - Functional Test Automation tool for pure AS 3.0 SWF and Flash Games.
  • NetBeans - NetBeans IDE is the official IDE for Java 8. With its editors, code analyzers, and converters, you can quickly and smoothly upgrade your applications to use new Java 8 language constructs, such as lambdas, functional operations, and method references.
  • VSCode Code editing. Redefined.

ORM frameworks and libraries

  • Hibernate - Hibernate ORM enables developers to more easily write applications whose data outlives the application process. As an Object/Relational Mapping (ORM) framework, Hibernate is concerned with data persistence as it applies to relational databases (via JDBC).
  • Morphia - Morphia is a lightweight type-safe library for mapping Java objects to/from MongoDB. Morphia provides a typesafe, and fluent Query API support with (runtime) validation. Morphia uses annotations so there are no XML files to manage or update. Morphia should feel very comfortable for any developer with JPA experience.
  • MyBatis - MyBatis is a first class persistence framework with support for custom SQL, stored procedures and advanced mappings. MyBatis eliminates almost all of the JDBC code and manual setting of parameters and retrieval of results. MyBatis can use simple XML or Annotations for configuration and map primitives, Map interfaces and Java POJOs (Plain Old Java Objects) to database records.

Useful libs

  • StreamEx - Common Java 8 Streams enhancement library.
  • Vavr - Vavr is an object-functional language extension to Java 8, which aims to reduce the lines of code and increase code quality. It provides persistent collections, functional abstractions for error handling, concurrent programming, pattern matching and much more.
  • Lombok - Project Lombok makes Java a spicier language by adding 'handlers' that know how to build and compile simple, boilerplate-free, not-quite-java code.
  • AspectJ - Seamless aspect-oriented extension to the Java. AspectJ enables clean modularization of crosscutting concerns, such as error checking and handling, synchronization, context-sensitive behavior, performance optimizations, monitoring and logging, debugging support, and multi-object protocols.
  • aShot - WebDriver Screenshot utility. Take screenshots, crop, prettify, compare.
  • WireMock is a flexible library for stubbing and mocking web services. Unlike general purpose mocking tools it works by creating an actual HTTP server that your code under test can connect to as it would a real web service.
  • MockServer can be used for mocking any system you integrate with via HTTP or HTTPS (i.e. services, web sites, etc).
  • Mockito is a mocking framework that lets you write beautiful tests with a clean & simple API.
  • PowerMock is a Java framework that allows you to unit test code normally regarded as untestable.
  • XMLUnit - testing and comparing XML output for Java and .NET
  • Awaitility - Awaitility is a DSL that allows you to express expectations of an asynchronous system in a concise and easy to read manner.
  • Beanmother - A library for setting up Java Bean as test data.
  • PIT - Pitest, a tool for mutation testing
  • GreenMail - GreenMail is an open source, intuitive and easy-to-use test suite of email servers for testing purposes. Typical use cases include mail integration testing or a lightweight sandboxed mail server for development.
  • image-comparison - Java Library that compares 2 images with the same sizes and shows the differences visually by drawing rectangles. Some parts of the image can be excluded from the comparison. Can be used for automation qa tests.
  • AssertJ - Powerful fluent assertion framework. Compatible with any xUnit framework.
  • Truth - Fluent assertion framework for Java and Android from Google.
  • JSONassert - JSON assertion library.
  • selenium-to-restassured - Adapter to easily share Selenium WebDriver cookies with Rest Assured back and forth with this Java api.
  • TestRail-JUnit - This Library helps you to integrate your JUnit-based tests with TestRail in an easily manner.
  • TestRail-TestNG - This Library helps you to integrate your TestNG-based tests with TestRail in an easily manner.
  • Allure-Environment-Creator - This tiny Library helps you to serve and store environment data for Allure easily.
  • Owner - The OWNER API is a Java library with the goal of minimizing the code required to handle application configuration via Java properties files.
  • DeepfakeHTTP - DeepfakeHTTP is a web server that uses HTTP dumps as a source for responses. This tool allows you to test clients against REST, GraphQL, and other APIs.
  • Cucable - Cucable is a Maven plugin that simplifies running Cucumber scenarios in parallel by slicing them into the smallest runnable feature files and runner classes.

Test Data

  • Faker - Java implementation of Faker, a library for generating fake data such as names, addresses, and phone numbers.
  • Instancio - Creates fully-populated objects for your unit tests. Includes an extension for JUnit 5.

DataBase test automation

  • DBTestCompare - Application to compare results of two SQL queries. Supports all databases with JDBC drivers provided (tested with Microsoft SQL Server, Teradata, Snowflake, PostgreSQL and MySQL\MariaDB). DBTestCompareGenerator - Tool for generating database tests that can be run with DBTestCompare.

Resources

Websites

Other Awesome Test Automation Lists

Also we are supporting and looking for contribution for project:

Contributing

Your contributions are always welcome!