Skip to content

Latest commit

 

History

History
352 lines (295 loc) · 37.7 KB

python-test-automation.md

File metadata and controls

352 lines (295 loc) · 37.7 KB
Raw

Python test automation

A comprehensive curated list of python test automation frameworks, tools, libraries and software to help software engineers easily bootstrap test automation on python. Sponsored by http://sdclabs.com

Have questions\issues\problems, join the chat at https://gitter.im/atinfo/awesome-test-automation

Table of content:

Also:

xUnit frameworks

  • frameworks
    • unittest - is a library to make unit testing distributed with python out of the box.
    • unittest2 - is a backport of the new features added to the unittest testing framework in Python 2.7 and onwards.
    • pytest - is a mature full-featured Python testing tool that helps you write better programs.
      • plugincompat - Test execution and compatibility checks for pytest plugins
    • test-junkie - Highly configurable modern testing framework.
    • nosetests - is a nicer testing for python.
    • slash - is a testing framework written in Python.
    • lemoncheesecake - is a Python framework intended to functional testing, it provides fixtures, matchers, test organization through test suites hierarchy and metadata (test/suite name, description, tags, properties, links), rich reporting features and various report formats (JSON, XML, HTML, Junit)
  • extensions
    • proboscis - is a Python test framework that extends Python’s built-in unittest module and Nose with features from TestNG.
    • grail - is a library which allows test script creation based on steps.
    • testify - unit test framework, provides Enhanced test fixture setup, Split test suites into buckets for easy parallelization, PEP8 naming conventions & Fancy color test runner with lots of logging / reporting option.
    • trial - Extension of unittest to support writing asynchronous unit tests using Deferreds and new result types ('skip' and 'todo'). Includes a command-line program that does test discovery and integrates with doctest and coverage.
    • subunit - Transparently adds support for running unittest test cases/suites in a separate process : prevents system wide changes by a test destabilising the test runner. It also allows reporting from tests in another process into the unittest framework, giving a single integrated test environment.
    • testresources - Provides a mechanism for managing 'resources' - expensive bits of infrastructure - that are needed by multiple tests. Resources are constructed and free on demand, but with an optional TestSuite?, the test run order is optimised to reduce the number of resource constructions and releases needed. Compatible with unittest.
    • testtools - Useful extensions to unittest derived from custom extensions by projects such as Twisted and Bazaar.
    • Sancho - Sancho 2.1 runs tests, and provides output for tests that fail; Sancho 2.1 does not count tests passed or failed; targets projects that do not maintain failing tests
    • zope.testing - Powerful test runner that includes support for post-mortem debugging of test failures. Also includes profiling and coverage reporting. This is a standalone package that has no dependencies on Zope and works just fine with projects that don't use Zope.
    • pythoscope - Tool that will automatically, or semi-automatically, generate unit tests for legacy systems written in Python.
    • testlib - Gives more power to standard unittest. More assert* methods; support for module level setup/teardown; skip test feature...
    • dutest - An object oriented interface to retrieve unittest test cases out of doctests. Hides initialization from doctests by allowing setUp and tearDown for each interactive example. Allows control over all the options provided by doctest. Specialized classes allow selective test discovery across a package hierarchy.
    • green - Green is a clean, colorful test runner for Python unit tests. Compare it to nose or trial.
    • ddt - Data-Driven tests with unittest
    • pytractor is an extension to the Selenium bindings for Python. Its goal is to make testing of angular.js applications easier with Python.
    • stestr - A parallel Python test runner built around subunit.
    • pytest-django - It allows you to test your Django project/applications with the pytest testing tool.
    • pytest-asyncio - It helps you to test asyncio code with pytest.

TDD \ ATDD \ BDD

  • BDD
    • behave - BDD tool
    • lettuce - BDD tool
      • lettuce-tools - Lettuce Tools are a set of tools and utilities that extend the lettuce out of the box features fill some of the gaps detected in the projects.
    • contexts - Descriptive testing for Python.
    • mamba - The definitive testing tool for Python. Born under the banner of Behavior Driven Development.
    • pyvows - Asynchronous behaviour driven development for Python.
    • pytest-bdd - BDD library for the py.test runner
    • pytest-play - ATDD pytest plugin that let you automate actions, assertions and test metrics executing plain YAML files.
    • robotframework - A generic test automation framework for acceptance testing and acceptance test-driven development (ATDD)
    • radish-bdd - BDD testing tool with supplements
  • Assertions
    • pyshould - Should style asserts based on pyhamcrest.
    • pyhamcrest - Hamcrest matchers for Python.
    • sure - A testing library for python with powerful and flexible assertions.
    • assertpy - Dead simple assertions library for unit testing in Python with a nice fluent API. Supports both Python 2 and 3.
  • Mocking
    • mock - A Python Mocking and Patching Library for Testing.
    • Ludibrio - Elegant test doubles framework in Python (mocks, stubs, spy, and dummies).
    • responses - A utility library for mocking out the requests Python library.
    • doublex - Powerful test doubles framework for Python.
    • freezegun - Travel through time by mocking the datetime module.
    • httpretty - HTTP request mock tool for Python.
    • httmock - A mocking library for requests for Python 2.6+ and 3.2+.
    • pretenders - fake servers for testing.
    • mock-server - Simple mock server for REST and XML-RPC API with admin panel based on tornado.
    • VCR.py - Automatically mock your HTTP interactions to simplify and speed up testing.
    • model-mmomy - Offers a smart way to create fixtures for testing in Django. With a simple and powerful API you can create many objects with a single line of code.
    • django-test-plus - Useful additions to Django's default TestCase.
    • Mockintosh - A mock server generator that's capable to generate RESTful APIs and communicate with the message queues to mimick asynchronous tasks.
    • pytest-mock - Pytest plugin that wraps around the standard unittest.mock package as a fixture.

Test Data manipulation

  • faker - A Python package that generates fake data.
  • fake2db - Fake database generator.
  • ForgeryPy - An easy to use forged data generator for Python. It's a port of forgery.
  • radar - Generate random datetime / time.
  • robotframework-faker - Robot Framework wrapper for faker.
  • hypothesis - Library for property based testing in Python, based on the Haskell library Quickcheck.
  • factory_boy - A fixtures replacement based on thoughtbot’s factory_girl.
  • fauxfactory - Random Data Generator
  • mimesis - Mimesis is fast and extremely easy to use Python package, which helps generate big volumes of fake data for a variety of purposes in a variety of languages

Code analysis

  • ruff – An extremely fast Python linter, written in Rust.
  • pysonar2 - A type inferencer and indexer for Python.
  • coverage - Code coverage measurement
  • pycallgraph - A library that visualises the flow (call graph) of your Python application.
  • Flake8 - The modular source code checker: pep8, pyflakes and co.
  • pylama - Code audit tool for Python and JavaScript.
  • Pylint - A source code analyzer.
  • hacking - OpenStack Hacking Style Checks (flake8 plugins).
  • prospector - A tool to analyse Python code and output information about errors, potential problems, convention violations and complexity.
  • wemake-python-styleguide - The strictest and most opinionated python linter ever!
  • pytest-cov - Pytest plugin which helps you identify parts of code that have been executed by your tests.

Web UI test automation

  • libraries
    • selenium webdriver - Browser automation tool.
    • splinter - A tool for test web applications with a simple for find elements, form actions, and others browser actions.
    • mechanize - Stateful programmatic web browsing in Python.
  • frameworks and wrappers
    • play_selenium - pytest-play plugin that let you drive a browser with Selenium using plain YAML files
    • py.saunter - An opinionated automation framework for use with the Selenium RC and WebDriver libraries.
    • moz-web-qa - A plugin for py.test that provides additional features needed for Mozilla's WebQA projects.
    • testutils sst - A web test framework that uses Python to generate functional browser-based tests.
    • wtframework - Framework for configurable Web Tests in Python.
    • holmium.core - Page objects & Utilities for writing selenium test cases.
    • robotframework-selenium2library - Web testing library for Robot Framework.
    • gocept.selenium - An API for the Selenium remote control that is suited for writing tests and integrates this with your test suite for any WSGI, Plone, Zope 2, ZTK, or Grok application.
    • webium - A Page Object pattern implementation library for Python
    • robotframework-anywherelibrary - A cross platform(desktop browser,android,ios) testing library for Robot Framework that leverages the Selenium 2(WebDriver) libraries internally to control a web browser and appium as mobile test automation framework for use with native and hybrid app.
    • robotframework-pageobjects - Nice implementation of the Page Object pattern with robotframework and selenium, that can work even outside of robotframework. More on this blog post
    • elementium - jQuery-style syntactic sugar for highly reliable automated browser testing in Python
    • slickqa - The slick-webdriver-python project is a wrapper around the python webdriver client bindings.
    • selene - Concise UI tests in Python + Ajax support + PageObjects + Widgets
    • hitch - A high level integration testing framework for service based applications.
    • Needle - Needle is a tool for testing visuals with Selenium and nose. It checks that CSS renders correctly by taking screenshots of portions of a website and comparing them against known good screenshots. It also provides tools for testing calculated CSS values and the position of HTML elements.
    • PyPOM - PyPOM is a Python Page Object Model library for Selenium and Splinter tests.
    • pypom_form - PyPOM based page object model for schema based forms.
    • POM - POM is Page-Object-Model microframework to develop web UI tests easy, quickly and with pleasure.
    • websmith - A Domain Specific Language (DSL) for Web Testing.
    • pages - lightweight page object and component Python library for UI tests.
    • seleniumbase - A simple Python framework for building test automation with Selenium WebDriver. It depends on pytest or noose modules to run. When used with noose a HTML report can be generated in the end of the test run.
    • Toolium - Toolium is a Python wrapper tool of Selenium and Appium libraries to test web and mobile applications in a single project. It provides a way of choosing and configuring the driver through a configuration file, implements a Page Object pattern and includes a simple visual testing solution.
  • extensions
    • webdriver_manager - The main idea is to simplify managemet of binary drivers for different browsers.
    • pytest_splinter - pytest spinter and selenium integration for anyone interested in browser interaction in tests.
    • lettuce_webdriver - Selenium webdriver for lettuce.
    • Browsermob Proxy - A python wrapper for Browsermob Proxy.
    • FireRobot - A Firefox extension that allows you to create Robot Framework/Selenium tests, in a more efficient way.
    • pytractor - Python selenium extensions for testing angular.js apps
    • Selenium-Requests - Extends Selenium WebDriver classes to include the request function from the Requests library, while doing all the needed cookie and request headers handling.
    • selenium-respectful - Minimalist Selenium WebDriver wrapper to work within rate limits of any amount of websites simultaneously. Parallel processing friendly.
    • Mailosaur - Python client for email testing/automation via Mailosaur.
    • gauge - Gauge is a light weight cross-platform test automation tool. It provides the ability to author test cases in the business language, actually written in golang with python language driver.
    • Golem - Golem is a complete test automation tool and framework for end-to-end testing. It creates powerful, robust and maintainable test suites, it's easy to learn even without a lot of programming knowledge. It is based on Selenium Webdriver and it can be extended using Python
    • Pylenium.io - Pylenium brings the best of Selenium, Cypress and Python into one package.
    • Playwright - Playwright is a Python library to automate Chromium, Firefox and WebKit browsers with a single API. Playwright delivers automation that is ever-green, capable, reliable and fast.

Mobile test automation

  • appium - An open source test automation framework for use with native, hybrid and mobile web apps. It drives iOS and Android apps using the WebDriver protocol.
  • Winium.StoreApps - An open source test automation tool for Windows Store apps, tested on emulators. It is Selenium Remote WebDriver implementation.
  • robotframework-androidlibrary - A Robot Framework test library for all your Android automation needs.
  • robotframework-appiumlibrary - An appium testing library for RobotFramework.
  • robotframework-ioslibrary - A Robot Framework test library for all your iOS automation needs.
  • uiautomator - Python wrapper of Android uiautomator test tool, it works on Android 4.1+ simply with Android device attached via adb.
  • ATX - Smart phone automation tool. Support iOS, Android, WebApp and game.
  • Airtest - Airtest is a cross-platform automated testing framework focusing mainly on games, but can also be used for native apps.

Windows UI test automation

  • Winium.Desktop - An open source test automation tool for automated testing of Windows application based on WinForms and WPF platforms. It is Selenium Remote WebDriver implementation.
  • PyAutoGUI - A cross-platform module for GUI automation for human beings. Control the keyboard and mouse from a Python script.
  • robotframework-autoitlibrary - A Windows GUI testing library for Robot Framework
  • autopy - A simple, cross-platform GUI automation toolkit for Python.
  • UISoup - This library supports UI-related testing using Python on Windows and Mac OS. (Only Python x86 is supported)
  • pywinauto - Very pythonic object-oriented Windows GUI automation library. Now it supports 64-bit Py2 and Py3.
  • SikuliX - OpenCV based GUI test framework that uses image recognision to locate item to interact with, script from python 2.7.
  • AutoItDriverServer - Selenium server to control/drive AutoIt via (Remote)WebDriver API.

Unix \ Linux UI test automation

  • ldtp - Cross Platform Linux Desktop (GUI Application) Testing Project.
  • fMBT - Python libraries for multiplatform GUI testing.
  • SikuliX - OpenCV based GUI test framework that uses image recognision to locate item to interact with, script from python 2.7.

MacOS UI test automation

  • PyAutoGUI - A cross-platform module for GUI automation for human beings. Control the keyboard and mouse from a Python script.
  • ATOMac - Python library to fully enable GUI testing of Mac applications via the Apple Accessibility API
  • SikuliX - OpenCV based GUI test framework that uses image recognision to locate item to interact with, script from python 2.7.

REST API Testing

  • Rester - Framework for testing (RESTful) HTTP APIs
  • pyresttest - A REST testing and API microbenchmarking tool
  • siesta - Python REST Client
  • play_requests - pytest-play plugin driving the famous python requests library for making HTTP calls using plain YAML files
  • gabbi - a tool for running HTTP tests where requests and responses are expressed as declarations in YAML files.
  • Schemathesis - Schemathesis is a tool for property-based testing of applications based on Open API & Swagger specs. It reads the application schema and generates test cases which will ensure that your application is compliant with its schema. Includes pytest & unittest integrations.
  • httprunner - A simple & elegant, yet powerful HTTP(S) testing framework. Support record and generate testcases with HAR support.
  • tavern - A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

Virtual environments

  • virtualenv - A tool to create isolated Python environments.
  • virtualenvwrapper - A set of extensions to Ian Bicking’s virtualenv tool.
  • tox - A generic virtualenv management and test command line tool to test code on different python versions.
  • PyVirtualDisplay - A python wrapper for Xvfb, Xephyr and Xvnc for GUI testing and\or automatic GUI screenshot.
  • pew - A set of tools to manage multiple virtual environments.
  • Vex - Run a command in the named virtualenv.
  • PyRun - A one-file, no-installation-needed version of Python.
  • pyenv - Simple Python version management.
  • Tapper - all-embracing test infrastructure.
  • Poetry - Easy dependency management in Python projects.

Performance & stress & load

  • Multi-Mechanize - An open source framework for performance and load testing.
  • locust - Scalable user load testing tool written in Python.
  • Yandex.Tank - Load and performance benchmark tool.
  • pytest-benchmark - A py.test fixture for benchmarking code.
  • Airspeed Velocity - A simple Python benchmarking tool with web-based reporting
  • The Grinder - Load testing framework that makes it easy to run a distributed test using many load injector machines. This includes common cases such as HTTP web servers, SOAP and REST web services, and application servers (CORBA, RMI, JMS, EJBs), as well as custom protocols. Flexible Scripting Test scripts are written in the powerful Jython and Clojure languages.
  • Tsung - is an open-source multi-protocol distributed load testing tool. It can be used to stress HTTP, WebDAV, SOAP, PostgreSQL, MySQL, LDAP and Jabber/XMPP servers.
  • taurus - Automation-friendly framework for Continuous Testing
  • molotov - Simple Python 3.5+ tool to write load tests.
  • Awesome pytest speedup - A checklist of best practices to speed up your pytest suite.

Security checking

  • sparta - A GUI application which simplifies network infrastructure penetration testing by aiding the penetration tester in the scanning and enumeration phase.
  • OWASP ZAP - An API for the usage of OWASP ZAP-features from a Python-script.
  • pythem - pythem is a multi-purpose pentest framework written in Python. It has been developed to be used by security researchers and security professionals. The tool intended to be used only for acts within the law. I am not liable for any undue and unlawful act practiced by this tool, for more information, read the license. Only runs on GNU/Linux OS.
  • SQLMap - sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.
  • fsociety - A Penetration Testing Framework, you will have every script that a hacker needs.

Continuous Integration

  • buildbot - A system to automate the compile/test cycle required by most software projects to validate code changes.
  • jenkins - An extensible open source continuous integration server.
  • travis-ci - Free Hosted Continuous Integration Platform.

Reporting

  • ReportPortal.io - pytest, RobotFramework, etc. powerfull results management and analytics for test automation reports. Powered with Machine Learning. Real-time integration and reports, visualization of trends and statistics, custom dashboards and widgets, gives you real visibility into the state of test automation. Integral part of CI/CD with TA and Continous Testing. Server-client application, can be used for any type of automated tests. Free and OpenSourced, GitHub link.
  • allure pytest - Allure adapter for PyTest framework.
  • HTMLTestRunner - An extension to the Python standard library's unittest module. It generates easy to use HTML test reports.
  • unittest-xml-reporting - A unittest test runner that can save test results to XML files that can be consumed by a wide range of tools, such as build systems, IDEs and continuous integration servers.
  • PrettyTable - Python library to generate nice table reports right in console.
  • pytest-html - Plugin for generating HTML reports for PyTest results.

Documentation generation

  • Sphinx - Python Documentation generator.
  • MkDocs - Markdown friendly documentation generator.
  • Pycco - The original quick-and-dirty, hundred-line-long, literate-programming-style documentation generator.

Editors, IDE, consoles

  • pycharm - Smart code editor provides first-class support for Python
  • pydev - Full featured python ide based on eclipse
  • sublime - A sophisticated text editor for code, markup and prose.
  • ipython - A command shell for interactive computing in multiple programming languages, originally developed for the Python programming language
  • VSCode - Very Neat Editor for python with loads of plugins

Useful libs

Libraries that may help you to build better test automation.

  • requests - An Apache2 Licensed HTTP library, written in Python, for human beings.
  • WebTest - WebTest helps you test your WSGI-based web applications.
  • lxml - Most feature-rich and easy-to-use library for processing XML and HTML in the Python language.
  • suds - A lightweight SOAP python client for consuming Web Services.
  • fabric - A Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks.
  • BeautifulSoup - Package for parsing HTML documents (including having malformed markup, i.e. non-closed tags, so named after Tag soup). It creates a parse tree for parsed pages that can be used to extract data from HTML, which is useful for web scraping.
  • Soupy - It is a wrapper around BeautifulSoup that makes it easier to build complex queries when wrangling web data.
  • PyQuery - Pyquery allows you to make jquery queries on xml documents. The API is as much as possible the similar to jquery. pyquery uses lxml for fast xml and html manipulation.
  • waiting - python utility library for waiting for predicates.
  • Comcast - simulating shitty network connections so you can build better systems.
  • dateutil - Extensions to the standard Python datetime module.
  • python-tesseract - Wrapper class for tesseract OCR (Linux & Mac & Windows)
  • pywinrm - A Python client for Windows Remote Management (WinRM). This allows you to invoke commands on target Windows machines from any machine that can run Python. WinRM allows you to call native objects in Windows. These include, but are not limited to, running batch scripts, powershell scripts and fetching WMI variables. For more information on WinRM, please visit Microsoft's WinRM site.
  • fig - Fast, isolated development environments using Docker.
  • gitapi - Pure-Python API for git.
  • Pyro4 - Pyro enables you to build applications in which objects can talk to each other over the network, with minimal programming effort.
  • keyboard - Hook and simulate global keyboard events on Windows and Linux.
  • Errbot - Errbot is a chatbot, a daemon that connects to your favorite chat service and brings your tools into the conversation.
  • tappy - tappy is a set of tools for working with the Test Anything Protocol (TAP) in Python. TAP is a line based test protocol for recording test data in a standard way.
  • pyscreenshot - The pyscreenshot module can be used to copy the contents of the screen to a PIL or Pillow image memory using various back-ends. Replacement for the ImageGrab Module, which works on Windows only, so Windows users don't need this library.
  • TBVaccine - TBVaccine is a utility that pretty-prints Python tracebacks. It automatically highlights lines you care about and deemphasizes lines you don't, and colorizes the various elements in a traceback to make it easier to parse.
  • PyPattyrn - PyPattyrn is a python package aiming to make it easier and faster to implement design patterns into your own projects.
  • Spyne - Spyne is a Python RPC toolkit that makes it easy to expose online services that have a well-defined API using multiple protocols and transports.
  • Pexpect - Pexpect makes Python a better tool for controlling other applications.
  • devtools-proxy - Proxy for Chrome DevTools. Fully compatible with Selenium and ChromeDriver
  • extratools - 145+ extra higher-level functional tools that go beyond standard library’s itertools, functools, etc. and popular third-party libraries like toolz, fancy, and more-itertools.
  • retrying - Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.
  • mitmproxy - mitmproxy is a free and open source interactive HTTPS proxy.
  • Python MSS - An ultra fast cross-platform multiple screenshots module in pure Python using ctypes.
  • Loguru - Loguru is a library which aims to bring enjoyable logging in Python.
  • prettyprinter - Syntax-highlighting, declarative and composable pretty printer for Python 3.5+
  • jsonschema - JSON Schema implementation for Python. Powerful tool for validating the structure of JSON data.

Resources

Where to discover new libraries, information, tools, etc.

Websites

Other Awesome Test Automation Lists

Also we are supporting and looking for contribution for project:

Contributing

Your contributions are always welcome!