Skip to content

Commit

Permalink
updated for simpler install, improved CLI (#20)
Browse files Browse the repository at this point in the history 
* updated installation, recursive clone

* obligatory history essay

* added capability to load pyttipanna params

* fixed bug with pyttipanna incorporation
  • Loading branch information
@dmarx
dmarx authored Feb 13, 2022
1 parent b284d2a commit b50e79c
Show file tree
Hide file tree
Showing 2 changed files with 174 additions and 110 deletions.
228 changes: 127 additions & 101 deletions pyttitools-PYTTI.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,46 +6,33 @@
"id": "WksDx4_To_su"
},
"source": [
"#pytti: python text to image\n",
"# PYTTI-TOOLS!\n",
"\n",
"---\n",
"This is a closed beta. Leak it if you must, information wants to be free.\n",
"## A brief history of this notebook\n",
"\n",
"[pytti is made possible by supporters like you.](https://www.patreon.com/sportsracer48) [Thank you.](https://www.youtube.com/watch?v=TexDW6nEhgU)\n",
"The tools and techniques below were pioneered in 2021 by a diverse and distributed collection of amazingly talented ML practitioners, researchers, and artists. The short version of this history is that Katherine Crowson ([@RiversHaveWings](https://twitter.com/RiversHaveWings)) published a notebook inspired by work done by [@advadnoun](https://twitter.com/advadnoun). Katherine's notebook spawned a litany of variants, each with their own twist on the technique or adding a feature to someone else's work. Henry Rachootin ([@sportsracer48](https://twitter.com/sportsracer48)) collected several of the most interesting notebooks and stuck the important bits together with bublegum and scotch tape. Thus was born PYTTI, and there was much rejoicing in sportsracer48's patreon, where it was shared in closed beta for several months so sportsracer48 wouldn't get buried under tech support requests (or so he hoped).\n",
"\n",
"PYTTI rapidly gained a reputation as one of the most powerful tools available for generating CLIP-guided images. In late November, @sportsracer48 released the last version in his closed beta: the \"pytti 5 beta\" notebook. David Marx ([@DigThatData](https://twitter.com/DigThatData)) offered to help tidy up the mess a few weeks later, and sportsracer48 encouraged him to run wild with it. Henry didn't realize he'd been speaking with someone who had recently quit their job and had a lot of time on their hands, and David's contributions snowballed into [PYTTI-Tools](https://github.com/pytti-tools)!\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "Z0hRb20yxxsc"
},
"outputs": [],
"source": [
"# @title Licensed under the MIT License\n",
"# Copyleft (c) 2021 Henry Rachootin\n",
"## How is PYTTI-Tools different from PYTTI 5 Beta?\n",
"\n",
"# Permission is hereby granted, free of charge, to any person obtaining a copy\n",
"# of this software and associated documentation files (the \"Software\"), to deal\n",
"# in the Software without restriction, including without limitation the rights\n",
"# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n",
"# copies of the Software, and to permit persons to whom the Software is\n",
"# furnished to do so, subject to the following conditions:\n",
"Right now, not very. The main user-visible changes are:\n",
"\n",
"# The above copyright notice and this permission notice shall be included in\n",
"# all copies or substantial portions of the Software.\n",
"* Local use is now a first-class citizen\n",
"* PyTTI is installable and can be run as a CLI tool\n",
"* Using PyTTI on the command line gives you magic powers\n",
"* PyTTI supports tensorboard, meaning it also integrates with tools like MLFlow and WandB\n",
"* Bug fixes and slightly saner code\n",
"\n",
"# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
"# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n",
"# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n",
"# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n",
"# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n",
"# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n",
"# THE SOFTWARE."
"## Call to action!\n",
"\n",
"My hope is that rather than continuing to fork off messy notebooks with minor changes between them, pytti-tools will become a central hub for organizing and sharing related techniques for this kind of generative art in a way that will enable methods to be shared and combined more fluidly than the 2021 paradigm of doing everything in colab permitted. \n",
"\n",
"If you're interested in contributing (even if you aren't a coder and just have an idea for something to add to the documentation), please visit our issue tracker: https://github.com/pytti-tools/pytti-core/issues\n",
"\n",
"Please help me untangle this thing before it swallows me whole. Thanks!\n",
"\n",
"`--The Management`\n"
]
},
{
Expand Down Expand Up @@ -271,46 +258,6 @@
"Run the cells in this section once for each runtime, or after a factory reset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "a9C9tARLzyzq"
},
"outputs": [],
"source": [
"#@title 1.1 Mount google drive (optional)\n",
"#@markdown Mounting your drive is optional but recommended. You can even restore from google randomly\n",
"#@markdown kicking you out if you mount your drive.\n",
"from google.colab import drive\n",
"drive.mount('/content/drive', force_remount = True)\n",
"!mkdir -p /content/drive/MyDrive/pytti_test\n",
"%cd /content/drive/MyDrive/pytti_test"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "RosI5DYxtjh0"
},
"outputs": [],
"source": [
"#@title 1.2 NVIDIA-SMI (optional)\n",
"#@markdown View information about your runtime GPU.\n",
"#@markdown Google will connect you to an industrial strength GPU, which is needed to run\n",
"#@markdown this notebook. You can also disable error checking on your GPU to get some\n",
"#@markdown more VRAM, at a marginal cost to stability. You will have to restart the runtime after\n",
"#@markdown disabling it.\n",
"enable_error_checking = False#@param {type:\"boolean\"}\n",
"if enable_error_checking:\n",
" !nvidia-smi\n",
"else:\n",
" !nvidia-smi\n",
" !nvidia-smi -i 0 -e 0"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand All @@ -328,26 +275,10 @@
"\n",
"\n",
"def clone_reqs():\n",
" !git clone https://github.com/pytti-tools/pytti-core\n",
" !git clone https://github.com/openai/CLIP\n",
" !git clone https://github.com/CompVis/taming-transformers\n",
" !git clone https://github.com/pytti-tools/AdaBins\n",
" !git clone https://github.com/pytti-tools/GMA\n",
" !git clone --branch dev --recurse-submodules -j8 https://github.com/pytti-tools/pytti-core\n",
"\n",
"def flush_reqs():\n",
" !rm -r pytti-core\n",
" !rm -r CLIP \n",
" !rm -r taming-transformers\n",
" !rm -r AdaBins\n",
" !rm -r GMA\n",
"\n",
"def dl_adabins():\n",
" !mkdir -p ./pretrained\n",
" if not path_exists('./pretrained/AdaBins_nyu.pt'):\n",
" !gdown https://drive.google.com/uc?id=1lvyZZbC9NLcS8a__YPcUP7rDiIpbRpoF\n",
" if not path_exists('AdaBins_nyu.pt'):\n",
" !gdown https://drive.google.com/uc?id=1zgGJrkFkJbRouqMaWArXE4WF_rhj-pxW\n",
" !mv AdaBins_nyu.pt ./pretrained/AdaBins_nyu.pt\n",
"\n",
"def install_everything():\n",
" if path_exists('./pytti-core'):\n",
Expand All @@ -364,21 +295,23 @@
" \"https://github.com/pytti-tools/pytti-notebook/issues/new\",\n",
" exc_info=True)\n",
"\n",
" clone_reqs()\n",
" !git clone --branch dev --recurse-submodules -j8 https://github.com/pytti-tools/pytti-core\n",
"\n",
" !pip install -r pytti-core/requirements.txt\n",
" !pip install ./AdaBins\n",
" !pip install ./CLIP\n",
" !pip install ./GMA\n",
" !pip install kornia pytorch-lightning\n",
" !pip install jupyter gdown loguru einops PyGLM ftfy regex tqdm hydra-core exrex\n",
" !pip install seaborn adjustText bunch matplotlib-label-lines\n",
" \n",
" !pip install ./pytti-core/vendor/AdaBins\n",
" !pip install ./pytti-core/vendor/CLIP\n",
" !pip install ./pytti-core/vendor/GMA\n",
" !pip install ./pytti-core/vendor/taming-transformers\n",
" !pip install ./pytti-core\n",
"\n",
" dl_adabins()\n",
" \n",
" from pytti.Notebook import change_tqdm_color\n",
" change_tqdm_color()\n",
" !mkdir -p images_out\n",
" !mkdir -p videos\n",
"\n",
" from pytti.Notebook import change_tqdm_color\n",
" change_tqdm_color()\n",
"\n",
"try:\n",
" from adjustText import adjust_text\n",
Expand All @@ -395,13 +328,54 @@
" change_tqdm_color()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "a9C9tARLzyzq"
},
"outputs": [],
"source": [
"#@title 1.1 Mount google drive (optional)\n",
"#@markdown Mounting your drive is optional but recommended. You can even restore from google randomly\n",
"#@markdown kicking you out if you mount your drive.\n",
"from google.colab import drive\n",
"drive.mount('/content/drive', force_remount = True)\n",
"!mkdir -p /content/drive/MyDrive/pytti_test\n",
"%cd /content/drive/MyDrive/pytti_test"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "RosI5DYxtjh0"
},
"outputs": [],
"source": [
"#@title 1.2 NVIDIA-SMI (optional)\n",
"#@markdown View information about your runtime GPU.\n",
"#@markdown Google will connect you to an industrial strength GPU, which is needed to run\n",
"#@markdown this notebook. You can also disable error checking on your GPU to get some\n",
"#@markdown more VRAM, at a marginal cost to stability. You will have to restart the runtime after\n",
"#@markdown disabling it.\n",
"enable_error_checking = False#@param {type:\"boolean\"}\n",
"if enable_error_checking:\n",
" !nvidia-smi\n",
"else:\n",
" !nvidia-smi\n",
" !nvidia-smi -i 0 -e 0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PcayyBJjE-qy"
},
"source": [
"# Step 2: Run it!\n",
"# Step 2: Configure Experiment\n",
"\n",
"Edit the parameters, or load saved parameters, then run the model."
]
},
Expand Down Expand Up @@ -962,6 +936,22 @@
" raise RuntimeError(f\"WARNING: Drive is not mounted.\\nERROR: video videos/{filename} does not exist.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Batch Settings\n",
"\n",
"Be Advised: google may penalize you for sustained colab GPU utilization, even if you are a PRO+ subscriber. Tread lightly with batch runs, you don't wanna end up in GPU jail."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"FYI: the batch setting feature below may not work at present. We recommend using the CLI for batch jobs, see usage instructions at https://github.com/pytti-tools/pytti-core . The code below will probably be removed in the near future."
]
},
{
"cell_type": "markdown",
"metadata": {
Expand All @@ -982,6 +972,9 @@
"outputs": [],
"source": [
"#@title batch settings\n",
"\n",
"# ngl... this probably doesn't work right now.\n",
"\n",
"from os.path import exists as path_exists\n",
"if path_exists('/content/drive/MyDrive/pytti_test'):\n",
" %cd /content/drive/MyDrive/pytti_test\n",
Expand Down Expand Up @@ -1101,6 +1094,38 @@
"if glob.glob(f'images_out/{namespace}/*.png'):\n",
" print(f\"WARNING: images_out/{namespace} contains images. Batch indicies may not match filenames unless restoring.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "Z0hRb20yxxsc"
},
"outputs": [],
"source": [
"# @title Licensed under the MIT License\n",
"# Copyleft (c) 2021 Henry Rachootin\n",
"# Copyright (c) 2022 David Marx\n",
"\n",
"# Permission is hereby granted, free of charge, to any person obtaining a copy\n",
"# of this software and associated documentation files (the \"Software\"), to deal\n",
"# in the Software without restriction, including without limitation the rights\n",
"# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n",
"# copies of the Software, and to permit persons to whom the Software is\n",
"# furnished to do so, subject to the following conditions:\n",
"\n",
"# The above copyright notice and this permission notice shall be included in\n",
"# all copies or substantial portions of the Software.\n",
"\n",
"# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n",
"# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n",
"# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n",
"# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n",
"# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n",
"# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN\n",
"# THE SOFTWARE."
]
}
],
"metadata": {
Expand All @@ -1117,7 +1142,8 @@
"name": "python3"
},
"language_info": {
"name": "python"
"name": "python",
"version": "3.9.9"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit b50e79c

Please sign in to comment.