Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test custom with scheduler fixture #61

Merged
merged 2 commits into from
Sep 21, 2018
Merged

Conversation

kain88-de
Copy link
Member

@kain88-de kain88-de commented Sep 19, 2018

This allows to add new schedulers in the future and have pytest
automatically create the tests for us. The should also catch more
bugs.

Fixes #57

Changes made in this Pull Request:

  • make scheduler fixture more common to use.
  • add testing helper module

PR Checklist

  • Tests?
  • Docs?
  • CHANGELOG updated?
  • Issue raised/referenced?

@orbeckst
Copy link
Member

Not sure why the tests fail with

___________ ERROR at setup of test_AnalysisFromFunction[distributed] ___________
file /home/travis/build/MDAnalysis/pmda/pmda/test/test_custom.py, line 26
  def test_AnalysisFromFunction(scheduler):
file /home/travis/build/MDAnalysis/pmda/pmda/testing.py, line 26
  @pytest.fixture(scope='session', params=('distributed', 'multiprocessing'))
  def scheduler(request, client):
E       fixture 'client' not found
>       available fixtures: cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespace, monkeypatch, no_cover, pytestconfig, record_property, record_xml_attribute, record_xml_property, recwarn, scheduler, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

Can we not stack parametrized fixtures?

@richardjgowers
Copy link
Member

richardjgowers commented Sep 20, 2018 via email

@kain88-de
Copy link
Member Author

kain88-de commented Sep 20, 2018 via email

@kain88-de
Copy link
Member Author

OK the fixtures work now. The tests fail because of actual bugs.

@@ -164,7 +164,7 @@ def _conclude(self, ):
@staticmethod
def _reduce(res, result_single_frame):
""" 'add' action for an accumulator"""
if res == []:
if isinstance(res, list) and len(res) == 0:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VOD555 Can you have a look. Your old version triggered a deprecation warning when res is a numpy array.

@codecov
Copy link

codecov bot commented Sep 20, 2018

Codecov Report

Merging #61 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #61   +/-   ##
=======================================
  Coverage   99.27%   99.27%           
=======================================
  Files           7        7           
  Lines         274      274           
  Branches       26       26           
=======================================
  Hits          272      272           
  Misses          1        1           
  Partials        1        1
Impacted Files Coverage Δ
pmda/rdf.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d56fd7e...58b1281. Read the comment docs.

This allows to add new schedulers in the future and have pytest
automatically create the tests for us. The should also catch more
bugs.
The comparison is deprecated is `res` is a numpy array. This is more robust.
@kain88-de kain88-de force-pushed the test-with-scheduler-fixture branch 2 times, most recently from 9e83b15 to 58b1281 Compare September 20, 2018 13:13
@kain88-de
Copy link
Member Author

Ready for review

Copy link
Member

@orbeckst orbeckst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Can we now use this for other tests as well, not only for the custom analysis class?

@VOD555
Copy link
Collaborator

VOD555 commented Sep 20, 2018

@kain88-de I run the tests for rdf.py, rms.py and contacts.py, and they all returned the deprecation warning:

 /home/shujie/anaconda3/envs/py36/lib/python3.6/site-packages/dask/base.py:835: UserWarning: 
The get= keyword has been deprecated. Please use the scheduler= keyword instead with the 
name of the desired scheduler like 'threads' or 'processes'
    warnings.warn("The get= keyword has been deprecated. "

I think the cause is line775 in parallel.py

        scheduler_kwargs = {'get': scheduler.get}

where we use a get= keyword, whcih is now deprecated in the new version of dask.

@kain88-de
Copy link
Member Author

kain88-de commented Sep 21, 2018 via email

@kain88-de kain88-de merged commit 3d78fc7 into master Sep 21, 2018
@kain88-de kain88-de deleted the test-with-scheduler-fixture branch September 21, 2018 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Convenient test for different schedulers
4 participants