Skip to content

Commit

Permalink
Move to ml-energy
Browse files Browse the repository at this point in the history
  • Loading branch information
jaywonchung committed Sep 24, 2023
1 parent 52c1867 commit 4ffca5e
Show file tree
Hide file tree
Showing 11 changed files with 31 additions and 29 deletions.
8 changes: 5 additions & 3 deletions .github/workflows/push_docker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,9 @@ jobs:
id: meta
uses: docker/metadata-action@v4
with:
images: symbioticlab/zeus
images: |
symbioticlab/zeus
mlenergy/zeus
tags: |
type=ref,event=branch
type=ref,event=tag
Expand All @@ -55,6 +57,6 @@ jobs:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=registry,ref=symbioticlab/zeus:buildcache
cache-to: type=registry,ref=symbioticlab/zeus:buildcache,mode=max
cache-from: type=registry,ref=mlenergy/zeus:buildcache
cache-to: type=registry,ref=mlenergy/zeus:buildcache,mode=max
build-args: TARGETARCH=amd64
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# Build instructions
# If you're building this image locally, make sure you specify `TARGETARCH`.
# Currently, this image supports `amd64` and `arm64`. For instance:
# docker build -t symbioticlab/zeus:master --build-arg TARGETARCH=amd64 .
# docker build -t mlenergy/zeus:master --build-arg TARGETARCH=amd64 .

FROM nvidia/cuda:11.8.0-base-ubuntu22.04

Expand Down
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@
</div>

[![NSDI23 paper](https://custom-icon-badges.herokuapp.com/badge/NSDI'23-paper-b31b1b.svg)](https://www.usenix.org/conference/nsdi23/presentation/you)
[![Docker Hub](https://badgen.net/docker/pulls/symbioticlab/zeus?icon=docker&label=Docker%20pulls)](https://hub.docker.com/r/symbioticlab/zeus)
[![Docker Hub](https://badgen.net/docker/pulls/symbioticlab/zeus?icon=docker&label=Docker%20pulls)](https://hub.docker.com/r/mlenergy/zeus)
[![Slack workspace](https://badgen.net/badge/icon/Join%20workspace/611f69?icon=slack&label=Slack)](https://join.slack.com/t/zeus-ml/shared_invite/zt-1najba5mb-WExy7zoNTyaZZfTlUWoLLg)
[![Homepage build](https://github.com/SymbioticLab/Zeus/actions/workflows/deploy_homepage.yaml/badge.svg)](https://github.com/SymbioticLab/Zeus/actions/workflows/deploy_homepage.yaml)
[![Apache-2.0 License](https://custom-icon-badges.herokuapp.com/github/license/SymbioticLab/Zeus?logo=law)](/LICENSE)
[![Homepage build](https://github.com/ml-energy/zeus/actions/workflows/deploy_homepage.yaml/badge.svg)](https://github.com/ml-energy/zeus/actions/workflows/deploy_homepage.yaml)
[![Apache-2.0 License](https://custom-icon-badges.herokuapp.com/github/license/ml-energy/zeus?logo=law)](/LICENSE)

---
**Project News**
Expand Down Expand Up @@ -134,7 +134,7 @@ docker run -it \
--gpus all `# Mount all GPUs` \
--cap-add SYS_ADMIN `# Needed to change the power limit of the GPU` \
--ipc host `# PyTorch DataLoader workers need enough shm` \
symbioticlab/zeus:latest \
mlenergy/zeus:latest \
bash
```

Expand All @@ -154,7 +154,7 @@ Refer to [Extending Zeus](https://ml.energy/zeus/extend/) for details.

## Carbon-Aware Zeus

The use of GPUs for training DNNs results in high carbon emissions and energy consumption. Building on top of Zeus, we introduce *Chase* -- a carbon-aware solution. *Chase* dynamically controls the energy consumption of GPUs; adapts to shifts in carbon intensity during DNN training, reducing carbon footprint with minimal compromises on training performance. To proactively adapt to shifting carbon intensity, a lightweight machine learning algorithm is used to forecast the carbon intensity of the upcoming time frame. For more details on Chase, please refer to our [paper](https://symbioticlab.org/publications/files/chase:ccai23/chase-ccai23.pdf) and the [chase branch](https://github.com/SymbioticLab/Zeus/tree/chase).
The use of GPUs for training DNNs results in high carbon emissions and energy consumption. Building on top of Zeus, we introduce *Chase* -- a carbon-aware solution. *Chase* dynamically controls the energy consumption of GPUs; adapts to shifts in carbon intensity during DNN training, reducing carbon footprint with minimal compromises on training performance. To proactively adapt to shifting carbon intensity, a lightweight machine learning algorithm is used to forecast the carbon intensity of the upcoming time frame. For more details on Chase, please refer to our [paper](https://symbioticlab.org/publications/files/chase:ccai23/chase-ccai23.pdf) and the [chase branch](https://github.com/ml-energy/zeus/tree/chase).


## Citation
Expand Down
2 changes: 1 addition & 1 deletion docs/extend.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ There are two ways to run Zeus: trace-driven and end-to-end.
### Trace-driven Zeus

The Zeus simulator ([`Simulator`][zeus.simulate.Simulator]) accepts one [`BatchSizeOptimizer`][zeus.policy.BatchSizeOptimizer] and [`PowerLimitOptimizer`][zeus.policy.PowerLimitOptimizer] in its constructor.
A full-example can be found in [`examples/trace_driven`](https://github.com/SymbioticLab/Zeus/tree/master/examples/trace_driven/).
A full-example can be found in [`examples/trace_driven`](https://github.com/ml-energy/zeus/tree/master/examples/trace_driven/).

### End-to-end Zeus

Expand Down
8 changes: 4 additions & 4 deletions docs/getting_started/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ We encourage users to do everything inside a Docker container spawned with our p

## Zeus Docker image

We provide a pre-built Docker image in [Docker Hub](https://hub.docker.com/r/symbioticlab/zeus){.external}.
We provide a pre-built Docker image in [Docker Hub](https://hub.docker.com/r/mlenergy/zeus){.external}.
On top of the `nvidia/cuda:11.8.0-devel-ubuntu22.04` image, the following are added:

1. Miniconda3 23.3.1, PyTorch 2.0.1, torchvision 0.15.2
Expand All @@ -26,7 +26,7 @@ On top of the `nvidia/cuda:11.8.0-devel-ubuntu22.04` image, the following are ad
!!! Tip
If you want to build our Docker image locally, you should specify `TARGETARCH` to be one of `amd64` or `arm64` based on your environment's architecture:
```sh
docker build -t symbioticlab/zeus:master --build-arg TARGETARCH=amd64 .
docker build -t mlenergy/zeus:master --build-arg TARGETARCH=amd64 .
```


Expand All @@ -44,7 +44,7 @@ docker run -it \
--gpus all \ # (1)!
--cap-add SYS_ADMIN \ # (2)!
--ipc host \ # (3)!
symbioticlab/zeus:latest \
mlenergy/zeus:latest \
bash
```

Expand All @@ -64,7 +64,7 @@ docker run -it \
--ipc host \ # (3)!
-v $(pwd):/workspace/zeus \ # (4)!
-v /data/imagenet:/data/imagenet:ro \
symbioticlab/zeus:latest \
mlenergy/zeus:latest \
bash
```

Expand Down
6 changes: 3 additions & 3 deletions docs/getting_started/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ We provide ready-to-go [Docker images](environment.md).
After going through the prerequisites, [`GlobalPowerLimitOptimizer`][zeus.optimizer.power_limit.GlobalPowerLimitOptimizer] into your training script.

Refer to
[our integration example with ImageNet](https://github.com/SymbioticLab/Zeus/tree/master/examples/imagenet/)
[our integration example with ImageNet](https://github.com/ml-energy/zeus/tree/master/examples/imagenet/)
for complete running examples for single-GPU and multi-GPU data parallel training.

```python
Expand Down Expand Up @@ -110,7 +110,7 @@ for epoch in range(100):

!!! Info
We plan to integrate [`ZeusMaster`][zeus.run.ZeusMaster] with an MLOps platform like [KubeFlow](https://www.kubeflow.org/).
Let us know about your preferences, use cases, and expectations by [posting an issue](https://github.com/SymbioticLab/Zeus/issues/new?assignees=&labels=&template=feature_request.md&title=Regarding%20Integration%20with%20MLOps%20Platroms)!
Let us know about your preferences, use cases, and expectations by [posting an issue](https://github.com/ml-energy/zeus/issues/new?assignees=&labels=&template=feature_request.md&title=Regarding%20Integration%20with%20MLOps%20Platroms)!

The cost-optimal batch size is located *across* multiple job runs using a Multi-Armed Bandit algorithm.
First, go through the steps for non-recurring jobs.
Expand All @@ -119,4 +119,4 @@ Then, you can use [`ZeusMaster`][zeus.run.ZeusMaster] to drive recurring jobs an

This example will come in handy:

- [Running trace-driven simulation on single recurring jobs and the Alibaba GPU cluster trace](https://github.com/SymbioticLab/Zeus/tree/master/examples/trace_driven){.external}
- [Running trace-driven simulation on single recurring jobs and the Alibaba GPU cluster trace](https://github.com/ml-energy/zeus/tree/master/examples/trace_driven){.external}
8 changes: 4 additions & 4 deletions docs/getting_started/installing_and_building.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This document explains how to install the [`zeus`][zeus] Python package.
!!! Tip
We encourage users to utilize our Docker image. Please refer to [Environment setup](./environment.md). Quick command:
```bash
docker run -it --gpus all --cap-add SYS_ADMIN --shm-size 64G symbioticlab/zeus:latest bash
docker run -it --gpus all --cap-add SYS_ADMIN --ipc host mlenergy/zeus:latest bash
```


Expand Down Expand Up @@ -33,15 +33,15 @@ pip install zeus-ml
If you would like to follow the `HEAD`:

```bash
git clone https://github.com/SymbioticLab/Zeus.git zeus
git clone https://github.com/ml-energy/zeus.git zeus
cd zeus
pip install .
```

For those would like to make changes to the source code and run them, we suggest an editable installation:

```bash
git clone https://github.com/SymbioticLab/Zeus.git zeus
git clone https://github.com/ml-energy/zeus.git zeus
cd zeus
pip install -e .
```
Expand All @@ -62,7 +62,7 @@ All dependencies are pre-installed if you're using our Docker image.

### Configuration

You can change the `#define`s in lines 28 to 31 in [`zemo/zemo.hpp`](https://github.com/SymbioticLab/Zeus/tree/master/zeus_monitor/zemo/zemo.hpp) to configure what information from the GPU is polled.
You can change the `#define`s in lines 28 to 31 in [`zemo/zemo.hpp`](https://github.com/ml-energy/zeus/tree/master/zeus_monitor/zemo/zemo.hpp) to configure what information from the GPU is polled.
By default, only the momentary power draw of the GPU will be collected.

### Building the power monitor
Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ hide:
</div>

!!! Success "Join the Zeus Slack workspace!"
Zeus exceeded 100k+ pulls on [Docker Hub](https://hub.docker.com/r/symbioticlab/zeus)!
Zeus exceeded 100k+ pulls on [Docker Hub](https://hub.docker.com/r/mlenergy/zeus)!
The Zeus team is always happy to chat with Zeus users and help out.
Reach out to us in the [Zeus Slack workspace](https://join.slack.com/t/zeus-ml/shared_invite/zt-1najba5mb-WExy7zoNTyaZZfTlUWoLLg).

Expand Down Expand Up @@ -100,7 +100,7 @@ Zeus is part of [The ML.ENERGY Initiative](https://ml.energy).
## Getting Started

Refer to [Getting Started](getting_started/index.md) for instructions on environment setup, installation, and integration.
We also provide integration [examples](https://github.com/SymbioticLab/Zeus/tree/master/examples) in our GitHub repository.
We also provide integration [examples](https://github.com/ml-energy/zeus/tree/master/examples) in our GitHub repository.

## Extending Zeus

Expand Down
8 changes: 4 additions & 4 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ site_description: Deep learning energy measurement and optimization
edit_uri: ""

# Repository
repo_name: SymbioticLab/Zeus
repo_url: https://github.com/SymbioticLab/Zeus
repo_name: ml-energy/zeus
repo_url: https://github.com/ml-energy/zeus

# Copyright
copyright: Copyright &copy; 2023 Jae-Won Chung
Expand Down Expand Up @@ -122,7 +122,7 @@ extra:
social:
- name: Zeus GitHub repository
icon: fontawesome/brands/github
link: https://github.com/SymbioticLab/Zeus
link: https://github.com/ml-energy/zeus
- name: Zeus Docker Hub registry
icon: fontawesome/brands/docker
link: https://hub.docker.com/r/symbioticlab/zeus
link: https://hub.docker.com/r/mlenergy/zeus
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ dependencies = [
dynamic = ["version"]

[project.urls]
Repository = "https://github.com/SymbioticLab/Zeus"
Repository = "https://github.com/ml-energy/zeus"
Homepage = "https://ml.energy/zeus"
Documentation = "https://ml.energy/zeus"

Expand Down
2 changes: 1 addition & 1 deletion zeus/run/dataloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ class ZeusDataLoader(DataLoader):
with five specific steps, which are noted below in the comments.
Please refer to
[our integration example with ImageNet](https://github.com/SymbioticLab/Zeus/tree/master/examples/imagenet/train.py)
[our integration example with ImageNet](https://github.com/ml-energy/zeus/tree/master/examples/imagenet/train.py)
for a complete example.
```python
Expand Down

0 comments on commit 4ffca5e

Please sign in to comment.