Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sequence Parallel MLP #3259

Merged
merged 19 commits into from
Oct 29, 2024
Merged

Sequence Parallel MLP #3259

merged 19 commits into from
Oct 29, 2024

Commits on Oct 16, 2024

  1. progress - working for mlp f/b

    @cowanmeg
    cowanmeg committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    1ba4aec View commit details
    Browse the repository at this point in the history

  2. repro error

    @cowanmeg
    cowanmeg committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    10ce34a View commit details
    Browse the repository at this point in the history

  3. reduce redundant code

    @cowanmeg
    cowanmeg committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    f9bdbab View commit details
    Browse the repository at this point in the history

  4. remove debug prints

    @cowanmeg
    cowanmeg committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    a4adf33 View commit details
    Browse the repository at this point in the history

  5. lint

    @cowanmeg
    cowanmeg committed Oct 16, 2024
    Configuration menu
    Copy the full SHA
    1048b93 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. remove recomputation of gemms in backward

    @cowanmeg
    cowanmeg committed Oct 17, 2024
    Configuration menu
    Copy the full SHA
    122ed48 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2024

  1. update input output dtypes

    @cowanmeg
    cowanmeg committed Oct 18, 2024
    Configuration menu
    Copy the full SHA
    c484707 View commit details
    Browse the repository at this point in the history

  2. remove extra cast

    @cowanmeg
    cowanmeg authored Oct 18, 2024
    Configuration menu
    Copy the full SHA
    2ac4e8e View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. updates

    @cowanmeg
    cowanmeg committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    567d962 View commit details
    Browse the repository at this point in the history

  2. Sequence parallel mlp

    @cowanmeg
    cowanmeg committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    36a03bf View commit details
    Browse the repository at this point in the history

  3. update comments

    @cowanmeg
    cowanmeg authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    44afa49 View commit details
    Browse the repository at this point in the history

  4. Add comment

    @cowanmeg
    cowanmeg authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    988fc8b View commit details
    Browse the repository at this point in the history

  5. merge and cleanup

    @cowanmeg
    cowanmeg committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    5fdc960 View commit details
    Browse the repository at this point in the history

  6. remove comments

    @cowanmeg
    cowanmeg committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    d8fbe97 View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2024

  1. refactor and fix

    @cowanmeg
    cowanmeg committed Oct 23, 2024
    Configuration menu
    Copy the full SHA
    0960986 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2024

  1. fix dropout mask sharding

    @cowanmeg
    cowanmeg committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    ab08a23 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' of https://github.com/nvidia/Fuser into normaliza… 

    …tion-did
    @cowanmeg
    cowanmeg committed Oct 24, 2024
    Configuration menu
    Copy the full SHA
    f236393 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2024

  1. fix single device multidevice

    @cowanmeg
    cowanmeg committed Oct 25, 2024
    Configuration menu
    Copy the full SHA
    952d262 View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2024

  1. Remove setDeviceMesh

    @cowanmeg
    cowanmeg authored Oct 28, 2024
    Configuration menu
    Copy the full SHA
    989591e View commit details
    Browse the repository at this point in the history