DM-2 v1.2 Major Changes and new Features

This version brings some major changes in the package structure and extracted UI properties, which were simply unavoidable for the new features (i.e. element visibility & keyboard identification).
The good news is the device interface and exploration model should be stable now and in particular the model is finally easily extendable again, the bad news is that previously stored models are not compatible with the new version and existing code bases will require some adaption like package path imports, class name fixes and adaptions to use the new UI properties.

requirements

• the new android gradle version will probably require you to install android build tools 28.03 or higher
• Even though DM-2 still supports Android 6.0 (API 23), we strongly recommend to use Android 7 or higher (API 24+) to completely benefit from the new features. This restriction comes from the underying UiAutomator as some of the API functions required for the new features (like input field hint texts) are only available since API 24.

If you experience build errors you may need to delete your gradle cache as gradle seams to have some trouble to fetch the right dependencies after version changes sometimes.

new features

• proper UI visibility detection by analyzation of overlying (even popup) windows and rendering order
  • the property visibleAreas can be used during exploration to determine which areas of an UI element are visible to the user
  • visibleBounds is the overall visible bounding box but may be partially overlapped by other elements, therefore visibleAreas should be used to determine action target coordinates
  • boundaries are the boundaries as defined in the UI hierarchy, these may be bigger than the visibleArea if the element is partially hidden or out of screen
• the isKeyboard property can be checked to identify (soft) keyboard elements
• an UI element now stores both its parent and child hashes, which allows for easy traversal in the hierarchy within a state
• we found the property hintText, which seams to contain the initial display text of input fields
• the id computation and widget/state generation functions can now easily be overwritten via inheritance, which allows for customization of the similarity metrics
• we now have Action-Queues which allow to send a sequence of actions wich is completely executed on device before a result state is retrieved. This can be e.g. useful to fill all available edit-fields before clicking anything, as state similarity still has some issues to recognize states as similar when for example a clear icon appears after filling an edit field.
• we have a new function navigateTo which allows to scroll to an element which is currently out of screen and interact with it
• you can configure the quality of screen-shots via configuration UiAutomatorServer.imgQuality and if images are allowed to be fetched asynchronously via UiAutomatorServer.delayedImgFetch which speeds up the exploration noticeable
• model parsing now offers the option to parse models parallel or sequential (mostly for debugging)
• critical performance and bug fixes in model parsing were applied

To allow for better modularity in exploration and analysis modules, the exploration model and device interface are now separate git submodules which can be easily included separately via jitpack (repository maven { url 'https://jitpack.io' }) with

compile "com.github.hotzkow:platformInterfaceLib:b48c1c48e3"
compile "com.github.hotzkow:explorationModel:8c6168d3f2"

In theory this allows to implement the device interface on arbitrary platforms like web or other mobile platforms and to reuse ones base analysis/exploration approach based on the generated exploration model.
We hope this simplifies the integration and reduces the effort to integrate new ideas and features.
We would be happy to integrate useful merge requests to improve our framework.

The gradle build of DM-2 will automatically use the jitpack repository if the git submodules are not initialized and use the local code otherwise.