python-ups-utils - Python modules and CLI utilities for working with UPS
Optional, pre-install for virtualenv-based installation:
$ virtualenv -p python3 venv # or $ pyvenv python3 venv $ source venv/bin/activate
This provides a self-contained installation and run-time area with some Python features (such as setuptools
) that your OS may not have.
The actual installation is then done:
$ python setup.py install [--user]
You may include --user
to install into your home area instead into the system area.
This package is also available from PyPI as ups-utils
and can be installed with
$ pip install ups-utils
Alternatively, to set up for development so that any edits to the source are immediately available:
$ python setup.py develop
The UPS ecosystem and this package makes use of some specific terms.
- flavor
- a platform description
- qualifiers
- a list of tags indicating build variant
- version
- a string representing a release of a software package, usually in the form
vXX_YY_ZZ
- product
- a build of a package of a given version, flavor and qualifiers
- suite
- a cohesive set of products
- repository
- a directory on disk holding installed products
- manifest
- a text file listing the products in a suite
- mirror
- a server holding manifests and products packaged as tar files
Added to this lexicon are the concepts:
- view
- a single, unified directory tree which has merges the distinct file system trees of a number of products. It is equivalent to a Spack view.
There are a number of command line interfaces to the ups
modules.
urman
- UPS repository manager
The urman
program is divided into a number of commands. It has
some general command line help as well as per-command help:
$ urman Usage: urman [OPTIONS] COMMAND [ARGS]... ... $ urman init --help Usage: urman init [OPTIONS] ...
Additional examples are given below with details omitted. See the per-command help for these details.
You can make queries on an existing UPS repository. For example to see what products are available (similar to ups list -aK+
) or which products are considered “top level” (have no other products which depend on them.
$ urman -z /products avail $ urman -z /products top
You can also produce a tree of dependencies in “dot” format and render that into a graphic if you have the GraphViz package installed:
$ urman -z /products depend <package> <version> -f <flavor> -q <quals> -F dot -o depend.dot [--full] $ dot -Tpng -odepend.png depend.dot $ display depend.png
To initialize a directory as a UPS repository run:
$ urman -z /path/to/products init
You can “install” a suite of products based on a suite name, its version and probably a list of qualifiers:
$ urman -z /path/to/products install lbne v02_05_01 -q e5:prof [--dryrun]
The “installation” process consists merely of downloading the associated compressed tar files and unpacking them. Even with a fast Internet connection this can take an hour, depending on the size of the suite.
Purging a UPS repository is done based no a given product. Only this product and any product on which it depends are subject to purging. If any candidate for purging is still needed by any remaining product it is not purged.
$ urman -z /products top ... larsoft v02_05_01 -f Linux64bit+2.6-2.12 -z /data4/lbne/bv/pp-v02_05_01/products-urman -q e5:noifdh:prof larsoft v02_05_01 -f Linux64bit+2.6-2.12 -z /data4/lbne/bv/pp-v02_05_01/products-urman -q e5:prof ... $ urman -z /products purge larsoft v02_05_01 [--dryrun]
Watch this space.
TBD. A configuration file mechanism will be added to allow setting command line options defaults and other parameters.
The bulk of the functionality of this package is in the form of Python modules in the ups.
namespace. There is fairly good unit test coverage of these modules in the ./tests/ sub-directory. These tests can be exercised by first “installing” for development as above and adding the “nose” package and running its nosetests
command
$ pip install nose $ nosetests [-s]
Note, ./tests/test_ups_commands.py downloads the source for UPS and builds it into ./products/.
Many UPS operations are slow on any reasonably sized repository. The packages combats that by maintaining a cache of per-repository information. By default this cache resides in ~/.ups-util/cache/
.
If you are lucky there is even more documentation under ./doc/.