Testing, building, and releasing¶

This section explains our testing, building, and release process.

Linting and testing¶

We lint with flake8. and run tests with nosetests.

We generally follow PEP 8, with a few exceptions which are listed in setup.cfg.

Tests are primarily set up via doctest, though a few are set up using unittest and live in lib5c.<some_module>.tests.test_<some_feature>.py.

Extra dependencies needed for linting and testing are listed in extras_require['test'] in setup.py. These include:

nose>=1.3.7
nose-exclude>=0.5.0
flake8>=3.4.1

To lint, run

$ flake8 lib5c

To run tests, run

$ nosetests

Our CI pipeline (hosted by Codeship) installs the package (using the frozen package versions from requirements.txt) and runs these two commands.

Committing¶

The git repository for lib5c is https://bitbucket.org/creminslab/lib5c/.

We roughly try to follow GitHub Flow when committing, trying to never allow the master branch to contain breaking code.

Building¶

We build Python wheels and source distributions, as well as a Docker image.

Versioning¶

Before building, you can optionally tag the release with a version number. We try to follow semantic versioning whenever possible. Our version tags do not include a leading “v” (e.g., we would use 0.5.3 as a tag rather than v0.5.3). To tag, run

$ git tag 0.5.1
$ git push --tags

If you don’t tag the release, versioneer will automatically fill in a PEP 440 compliant version for you.

Python wheel and source distribution¶

To build the wheel for lib5c, run

$ python setup.py bdist_wheel

To build the source distribution for lib5c, run

$ python setup.py sdist

Docker image¶

First, ensure you’ve built the Python wheel and that it exists as dist/lib5c-<version>-py2-none-any.whl.

To build the lib5c Docker image, we pass the version name into the Dockerfile as a build-arg. To build, run

$ docker build --build-arg VERSION=$(python lib5c/_version.py) -t creminslab/lib5c:$(git describe --tags --dirty --always) .
$ docker build --build-arg VERSION=$(python lib5c/_version.py) -t creminslab/lib5c:latest .

Note that we tag the Docker image with the direct output of git describe, since Docker image tags can’t contain plus signs.

Releasing¶

Upon paper acceptance, we will release to PyPI and to Docker Hub.

PyPI¶

To build both the wheel and source distribution and upload them to PyPI, run

$ python setup.py sdist bdist_wheel upload

Docker Hub¶

To push the Docker image to Docker Hub, run

$ docker push creminslab/lib5c:$(git describe --tags --dirty --always)
$ docker push creminslab/lib5c:latest

Documentation¶

Docs are built using Sphinx.

Extra dependencies needed for building docs are listed in extras_require['docs'] in setup.py. These include:

Sphinx>=1.7.2
sphinx-rtd-theme>=0.3.0
nbsphinx>=0.3.5
ipykernel>=4.10.0

The exact doc build environment is specified in docs/requirements.txt

We also require Pandoc.

Documentation pages are stored in docs/.

The documentation is built by running

$ sphinx-apidoc -o docs/ -f -T -e lib5c/ "**/tests" "**/*legacy*.py"
$ sphinx-build -b dirhtml docs/ docs_build/

The documentation is then pushed to this Bitbucket repository under the lib5c/ folder.

Cheat sheet¶

In the future, more of these steps will be integrated into the CI/CD pipeline.

The commands explained above are collected all in one place in the cheat sheet below:

# test (check extras_require['test'] in setup.py for requirements)
flake8 lib5c
nosetests

# git commit
git commit -m 'commit message'
git push

# git tag (optional)
git tag 0.5.1
git push --tags

# build wheel
python setup.py bdist_wheel

# build and upload to pypi
python setup.py sdist bdist_wheel upload

# local Docker build
docker build --build-arg VERSION=$(python lib5c/_version.py) -t creminslab/lib5c:$(git describe --tags --dirty --always) .
docker build --build-arg VERSION=$(python lib5c/_version.py) -t creminslab/lib5c:latest .

# push Docker image
docker push creminslab/lib5c:$(git describe --tags --dirty --always)
docker push creminslab/lib5c:latest

# build docs
sphinx-apidoc -o docs/ -f -T -e lib5c/ "**/tests" "**/*legacy*.py"
sphinx-build -b dirhtml docs/ docs_build/