Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Boost dash::sort with even more parallelism #611

Open
wants to merge 99 commits into
base: development
Choose a base branch
from
Open

[WIP] Boost dash::sort with even more parallelism #611

wants to merge 99 commits into from

Commits on Nov 21, 2018

  1. added a specific configuration for Intel KNL

    Roger Kowalewski committed Nov 21, 2018
    Configuration menu
    Copy the full SHA
    0c608f6 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2018

  1. Add Intel Parallel STL for ICC > 2018

    Roger Kowalewski committed Nov 22, 2018
    Configuration menu
    Copy the full SHA
    f3679f5 View commit details
    Browse the repository at this point in the history

  2. include MPI linker flags in compiler wrapper

    Roger Kowalewski committed Nov 22, 2018
    Configuration menu
    Copy the full SHA
    2d36a52 View commit details
    Browse the repository at this point in the history

  3. include parallel stl in dash::sort

    Roger Kowalewski committed Nov 22, 2018
    Configuration menu
    Copy the full SHA
    21ba37a View commit details
    Browse the repository at this point in the history

  4. Merge branch 'fix-constexpr-icc19' into feat-sort-parallel-stl

    Roger Kowalewski committed Nov 22, 2018
    Configuration menu
    Copy the full SHA
    c08a21e View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2018

  1. refactor and centralize parallel sort

    Roger Kowalewski committed Nov 27, 2018
    Configuration menu
    Copy the full SHA
    9cb3fd4 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2018

  1. different value type for test

    Roger Kowalewski committed Nov 28, 2018
    Configuration menu
    Copy the full SHA
    d9732a9 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/development' into feat-sort-high… 

    …-parallelism
    Roger Kowalewski committed Nov 28, 2018
    Configuration menu
    Copy the full SHA
    52cb8c3 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2018

  1. rename pstl config to algo_pstl

    Roger Kowalewski committed Nov 30, 2018
    Configuration menu
    Copy the full SHA
    46a4d5b View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'dash/development' into feat-sort-high-p… 

    …arallelism
    Roger Kowalewski committed Nov 30, 2018
    Configuration menu
    Copy the full SHA
    a0e8b71 View commit details
    Browse the repository at this point in the history

Commits on Dec 2, 2018

  1. Find Parallel STL properly on SuperMUC environment

    Roger Kowalewski committed Dec 2, 2018
    Configuration menu
    Copy the full SHA
    cf5bef2 View commit details
    Browse the repository at this point in the history

  2. remove unknown flag

    Roger Kowalewski committed Dec 2, 2018
    Configuration menu
    Copy the full SHA
    ba99e30 View commit details
    Browse the repository at this point in the history

  3. use parallelism according to locality information

    Roger Kowalewski committed Dec 2, 2018
    Configuration menu
    Copy the full SHA
    47cfcbc View commit details
    Browse the repository at this point in the history

  4. fix detection of pstl in OpenHPC stack

    Roger Kowalewski committed Dec 2, 2018
    Configuration menu
    Copy the full SHA
    4f76d60 View commit details
    Browse the repository at this point in the history

Commits on Dec 6, 2018

  1. minor refactoring for the sorting algorithm

    Roger Kowalewski committed Dec 6, 2018
    Configuration menu
    Copy the full SHA
    57dfd3f View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2018

  1. added exclusive scan operation in DASH and DART

    Roger Kowalewski committed Dec 9, 2018
    Configuration menu
    Copy the full SHA
    1fe1c4d View commit details
    Browse the repository at this point in the history

  2. remove erroneuous logging

    Roger Kowalewski committed Dec 9, 2018
    Configuration menu
    Copy the full SHA
    a903131 View commit details
    Browse the repository at this point in the history

  3. refactor to use more efficient MPI communication

    Roger Kowalewski committed Dec 9, 2018
    Configuration menu
    Copy the full SHA
    1d89255 View commit details
    Browse the repository at this point in the history

  4. fix trace numbering

    Roger Kowalewski committed Dec 9, 2018
    Configuration menu
    Copy the full SHA
    4e2e594 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2018

  1. do not send empty messages

    Roger Kowalewski committed Dec 10, 2018
    Configuration menu
    Copy the full SHA
    a63d717 View commit details
    Browse the repository at this point in the history

  2. Use std::async for merging

    @pascalj
    pascalj committed Dec 10, 2018
    Configuration menu
    Copy the full SHA
    763fc01 View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2018

  1. Remove unused file dash/internal/Config.in.h

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    daf928d View commit details
    Browse the repository at this point in the history

  2. Parse git commit hash and add it as compiler macro

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    a124c91 View commit details
    Browse the repository at this point in the history

  3. Add compiler version to benchmark output

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    6fd6246 View commit details
    Browse the repository at this point in the history

  4. Move specification of DASH version into head file

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    35e3955 View commit details
    Browse the repository at this point in the history

  5. Re-add version to CMake file and generate Version.h from it

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    a7d9fe4 View commit details
    Browse the repository at this point in the history

  6. Do not define DASH_GIT_COMMIT in Version.h unless the git commit is a… 

    …vailable
    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    83c3c16 View commit details
    Browse the repository at this point in the history

  7. Print DART environment variables in BenchmarkParams

    @devreal
    devreal authored and Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    a63ef36 View commit details
    Browse the repository at this point in the history

  8. fix tracing

    Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    1a631cd View commit details
    Browse the repository at this point in the history

  9. make strided iterator constexpr

    Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    76085ad View commit details
    Browse the repository at this point in the history

  10. rename detail namespace to impl namespace

    Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    c9849bb View commit details
    Browse the repository at this point in the history

  11. fix missing traces

    Roger Kowalewski committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    24965f8 View commit details
    Browse the repository at this point in the history

  12. Move the copy_async future into lambda 

    It went out of scope too fast.
    @pascalj
    pascalj committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    758138c View commit details
    Browse the repository at this point in the history

  13. Merge branch 'feat-sort-high-parallelism' of github.com:dash-project/… 

    …dash into feat-sort-high-parallelism
    @pascalj
    pascalj committed Dec 11, 2018
    Configuration menu
    Copy the full SHA
    c8e7806 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2018

  1. change put to get variant for correct merging -> Tests not 

    working
    Roger Kowalewski committed Dec 12, 2018
    Configuration menu
    Copy the full SHA
    6a303ea View commit details
    Browse the repository at this point in the history

Commits on Dec 13, 2018

  1. fix a bug in communication / merge overlap

    Roger Kowalewski committed Dec 13, 2018
    Configuration menu
    Copy the full SHA
    26ac2b5 View commit details
    Browse the repository at this point in the history

Commits on Dec 14, 2018

  1. Use std::merge instread of inplace_merge for the last step

    @pascalj
    pascalj committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    239fc7c View commit details
    Browse the repository at this point in the history

  2. Use the sort_comp lambda also for merging

    @pascalj
    pascalj committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    ce06c8f View commit details
    Browse the repository at this point in the history

  3. temporary commit to simplify the sort algorithm and to reduce lots of 

    communication
    Roger Kowalewski committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    3a01410 View commit details
    Browse the repository at this point in the history

  4. remove two all-to-all communication steps

    Roger Kowalewski committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    ef63a7e View commit details
    Browse the repository at this point in the history

  5. Try to compile with threadsupport on CI

    @pascalj
    pascalj committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    54c87df View commit details
    Browse the repository at this point in the history

  6. fix deadlock if we have a local empty range at the beginning

    Roger Kowalewski committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    c49f892 View commit details
    Browse the repository at this point in the history

  7. fix missing information in trace

    Roger Kowalewski committed Dec 14, 2018
    Configuration menu
    Copy the full SHA
    1d421d7 View commit details
    Browse the repository at this point in the history

Commits on Dec 16, 2018

  1. make GlobLocalMemoryPool thread-safe

    Roger Kowalewski committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    9d082a6 View commit details
    Browse the repository at this point in the history

  2. temporarily disable strange ThreadsafeTest.ConcurrentAlgorithm

    Roger Kowalewski committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    f1b18ce View commit details
    Browse the repository at this point in the history

  3. disable another thread safety test

    Roger Kowalewski committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    a025c19 View commit details
    Browse the repository at this point in the history

  4. Add thread pool

    @pascalj
    pascalj committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    376ba84 View commit details
    Browse the repository at this point in the history

  5. Move all-to-all and merge to its own file

    @pascalj
    pascalj committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    2bb4744 View commit details
    Browse the repository at this point in the history

  6. Use std::future<result_t> for the thread pool

    @pascalj
    pascalj committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    1fc1b8e View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'dash/development' into feat-sort-high-p… 

    …arallelism
    Roger Kowalewski committed Dec 16, 2018
    Configuration menu
    Copy the full SHA
    ae9304f View commit details
    Browse the repository at this point in the history

Commits on Dec 17, 2018

  1. add a config file to manage node level parallelism

    Roger Kowalewski committed Dec 17, 2018
    1 Configuration menu
    Copy the full SHA
    f3cac18 View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2018

  1. Use operator[] for std::map access in threads 

    map.at() will compare find(idx) against end() which can cause false
negatives during bounds checking in a multithreading context.
    @pascalj
    pascalj committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    8b8d842 View commit details
    Browse the repository at this point in the history

  2. Use dash_get_handle instead of async_copy 

    We have an ideal case for this here, no overlapping or tiling.
    @pascalj
    pascalj committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    8be6273 View commit details
    Browse the repository at this point in the history

  3. Test whether the range dependencies are waitable

    @pascalj
    pascalj committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    8749a5c View commit details
    Browse the repository at this point in the history

  4. fix tbb config since it is neither swappable nor movable

    Roger Kowalewski committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    0d568a8 View commit details
    Browse the repository at this point in the history

  5. fix identation again

    Roger Kowalewski committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    53633f8 View commit details
    Browse the repository at this point in the history

  6. fix narrowing conversion

    Roger Kowalewski committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    62cd0ee View commit details
    Browse the repository at this point in the history

  7. Use a threadpool of the configured size 

    This way it won't be configured ad std::thread::hardware_concurrency().
    @pascalj
    pascalj committed Dec 18, 2018
    Configuration menu
    Copy the full SHA
    4e9c3e0 View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2018

  1. Remove constexpr in NodeParallelismConfig 

    It breaks the build for gcc < 7.2, see
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66297
    @pascalj
    pascalj committed Dec 19, 2018
    Configuration menu
    Copy the full SHA
    7fb4bc5 View commit details
    Browse the repository at this point in the history

  2. add the possibility to pass an explicit output iterator

    Roger Kowalewski committed Dec 19, 2018
    Configuration menu
    Copy the full SHA
    ab39386 View commit details
    Browse the repository at this point in the history

Commits on Dec 20, 2018

  1. Prepare support for non-inplace sort and enhance documentation

    Roger Kowalewski committed Dec 20, 2018
    Configuration menu
    Copy the full SHA
    23aa200 View commit details
    Browse the repository at this point in the history

  2. minor refactoring

    Roger Kowalewski committed Dec 20, 2018
    Configuration menu
    Copy the full SHA
    13da0d9 View commit details
    Browse the repository at this point in the history

Commits on Dec 21, 2018

  1. minor refactoring to support sort and merge strategies via templates

    Roger Kowalewski committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    c926b82 View commit details
    Browse the repository at this point in the history

  2. minor changes

    Roger Kowalewski committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    02f6d1d View commit details
    Browse the repository at this point in the history

  3. minor changes

    Roger Kowalewski committed Dec 21, 2018
    Configuration menu
    Copy the full SHA
    26499b3 View commit details
    Browse the repository at this point in the history

Commits on Dec 25, 2018

  1. refactoring for preparation of non-in-place sort

    Roger Kowalewski committed Dec 25, 2018
    Configuration menu
    Copy the full SHA
    07ca18f View commit details
    Browse the repository at this point in the history

Commits on Dec 26, 2018

  1. use a unique pointer for the local buffer instead of a vector

    Roger Kowalewski committed Dec 26, 2018
    Configuration menu
    Copy the full SHA
    5374432 View commit details
    Browse the repository at this point in the history

  2. remove duplicate code

    Roger Kowalewski committed Dec 26, 2018
    Configuration menu
    Copy the full SHA
    448b802 View commit details
    Browse the repository at this point in the history

  3. add cpp17 monotonic buffer resource

    Roger Kowalewski committed Dec 26, 2018
    Configuration menu
    Copy the full SHA
    d6df5ae View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2018

  1. minmax in separate function

    Roger Kowalewski committed Dec 27, 2018
    Configuration menu
    Copy the full SHA
    f5f09a7 View commit details
    Browse the repository at this point in the history

  2. add a very simple integer range class

    Roger Kowalewski committed Dec 27, 2018
    Configuration menu
    Copy the full SHA
    ba855c0 View commit details
    Browse the repository at this point in the history

  3. simplify and remove some ugly stuff

    Roger Kowalewski committed Dec 27, 2018
    Configuration menu
    Copy the full SHA
    2cd3d95 View commit details
    Browse the repository at this point in the history

Commits on Dec 28, 2018

  1. rely on copy ellision instead of explicit moves

    Roger Kowalewski committed Dec 28, 2018
    Configuration menu
    Copy the full SHA
    0e3067e View commit details
    Browse the repository at this point in the history

  2. fix an error after refactoring

    Roger Kowalewski committed Dec 28, 2018
    Configuration menu
    Copy the full SHA
    531e8a6 View commit details
    Browse the repository at this point in the history

  3. minor changes

    Roger Kowalewski committed Dec 28, 2018
    Configuration menu
    Copy the full SHA
    b2d51a0 View commit details
    Browse the repository at this point in the history

  4. added non in-place sort: tests cannot be passed

    Roger Kowalewski committed Dec 28, 2018
    Configuration menu
    Copy the full SHA
    32ccc26 View commit details
    Browse the repository at this point in the history

  5. minor refactoring

    Roger Kowalewski committed Dec 28, 2018
    Configuration menu
    Copy the full SHA
    deb6545 View commit details
    Browse the repository at this point in the history

Commits on Dec 31, 2018

  1. Add non-inplace merge 

    When the input- and out buffer are not the same, we can use a series of
non inplace buffers.
    @pascalj
    pascalj committed Dec 31, 2018
    Configuration menu
    Copy the full SHA
    9c4eebe View commit details
    Browse the repository at this point in the history

  2. Handle non-inplace edge case with n=1

    @pascalj
    pascalj committed Dec 31, 2018
    Configuration menu
    Copy the full SHA
    a220257 View commit details
    Browse the repository at this point in the history

Commits on Jan 1, 2019

  1. remove branches in loops

    Roger Kowalewski committed Jan 1, 2019
    Configuration menu
    Copy the full SHA
    e051f2b View commit details
    Browse the repository at this point in the history

  2. simplify the local data container

    Roger Kowalewski committed Jan 1, 2019
    Configuration menu
    Copy the full SHA
    4c45890 View commit details
    Browse the repository at this point in the history

  3. remove non-existent include

    Roger Kowalewski committed Jan 1, 2019
    Configuration menu
    Copy the full SHA
    ab8fbf0 View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2019

  1. add unit test which uses sort instead of merge as a final step

    Roger Kowalewski committed Jan 2, 2019
    Configuration menu
    Copy the full SHA
    53afd7c View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2019

  1. Switch to alltoallv for data exchange

    @pascalj
    pascalj committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    dd0e08a View commit details
    Browse the repository at this point in the history

  2. Add comment for send counts

    @pascalj
    pascalj committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    66f391e View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2019

  1. Check if value_type is supported by dart

    @pascalj
    pascalj committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    3a22430 View commit details
    Browse the repository at this point in the history

Commits on Jan 24, 2019

  1. minor changes

    Roger Kowalewski committed Jan 24, 2019
    Configuration menu
    Copy the full SHA
    9f44af8 View commit details
    Browse the repository at this point in the history

  2. log number of iterations even in release build temporarily

    Roger Kowalewski committed Jan 24, 2019
    Configuration menu
    Copy the full SHA
    bc85377 View commit details
    Browse the repository at this point in the history

Commits on Jan 25, 2019

  1. minor changes

    Roger Kowalewski committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    02d8fbe View commit details
    Browse the repository at this point in the history

  2. enable thread support for algorithms independent of DART

    Roger Kowalewski committed Jan 25, 2019
    Configuration menu
    Copy the full SHA
    2537a35 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2019

  1. some minor refactorings and const correctness

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    79a9cba View commit details
    Browse the repository at this point in the history

  2. remove unnecessary barrier

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    889e38f View commit details
    Browse the repository at this point in the history

  3. remove empty file

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    8a3436d View commit details
    Browse the repository at this point in the history

  4. Revert "make GlobLocalMemoryPool thread-safe" 

    This reverts commit 9d082a6.
    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    f2b9c08 View commit details
    Browse the repository at this point in the history

  5. reenable test

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    264bdd3 View commit details
    Browse the repository at this point in the history

  6. dash::sort: remove merge implementation as not need for now

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    cb08349 View commit details
    Browse the repository at this point in the history

  7. remove thread support in ci

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    48f9a53 View commit details
    Browse the repository at this point in the history

  8. no magic numbers

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    cf39716 View commit details
    Browse the repository at this point in the history

  9. replace ugly macros with lovely constexprs

    Roger Kowalewski committed Feb 5, 2019
    Configuration menu
    Copy the full SHA
    151a6b9 View commit details
    Browse the repository at this point in the history