unsupported operand type(s) for %: 'int' and 'NoneType'

[fancy45daddy]

❓ Questions and Help

I follow the https://github.com/pytorch/xla/blob/master/contrib/kaggle/pytorch-xla-2-0-on-kaggle.ipynb

but the code in image = pipeline(prompt, callback=lambda *args: xm.mark_step(), generator=generator).images[0]
get

TypeError                                 Traceback (most recent call last)
Cell In[8], line 4
      1 generator = torch.Generator().manual_seed(0)
      2 # xm.mark_step compiles and executes the graph after each iteration.
      3 # The first few steps will be much slower than the rest.
----> 4 image = pipeline(prompt, callback=lambda *args: xm.mark_step(), generator=generator).images[0]
      5 image

File /usr/local/lib/python3.8/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    112 @functools.wraps(func)
    113 def decorate_context(*args, **kwargs):
    114     with ctx_factory():
--> 115         return func(*args, **kwargs)

File /usr/local/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py:1035, in StableDiffusionPipeline.__call__(self, prompt, height, width, num_inference_steps, timesteps, sigmas, guidance_scale, negative_prompt, num_images_per_prompt, eta, generator, latents, prompt_embeds, negative_prompt_embeds, ip_adapter_image, ip_adapter_image_embeds, output_type, return_dict, cross_attention_kwargs, guidance_rescale, clip_skip, callback_on_step_end, callback_on_step_end_tensor_inputs, **kwargs)
   1033 if i == len(timesteps) - 1 or ((i + 1) > num_warmup_steps and (i + 1) % self.scheduler.order == 0):
   1034     progress_bar.update()
-> 1035     if callback is not None and i % callback_steps == 0:
   1036         step_idx = i // getattr(self.scheduler, "order", 1)
   1037         callback(step_idx, t, latents)

TypeError: unsupported operand type(s) for %: 'int' and 'NoneType'

how to fix the problem?

[JackCaoG]

that code is most likely deprecated right now, do you just need to run some code to confirm it works or you want to run stable diffusion specifically? Otherwise I would recommend examples under https://github.com/pytorch/xla/tree/master/examples

[fancy45daddy]

I want to run stable diffusion specifically. I can not find any stable diffusion code in https://github.com/pytorch/xla/tree/master/examples. I want to get a minimum working example for stable diffusion in huggingface diffusers.

[JackCaoG]

@zpcore can you share our sd reference code that can be run?

[fancy45daddy]

If you can please give me a minimum working example for stable diffustion and use all tpu cores in kaggle v3-8. Not just one tpu core.

[zpcore]

@zpcore can you share our sd reference code that can be run?

I need to check if the code is compatible with tpu v3 or not in kaggle. Will update.

Just for reference for now, here is the script we use for SD2: https://github.com/pytorch-tpu/diffusers/blob/main/examples/text_to_image/train_text_to_image_xla.py, and we use this command to run:

export XLA_DISABLE_FUNCTIONALIZATION=0
export PROFILE_DIR=/tmp/home/profile/
export CACHE_DIR=/tmp/home/xla_cache
export DATASET_NAME=lambdalabs/naruto-blip-captions
export OUTPUT_DIR=/tmp/home/output/
export PROFILE_DURATION=80000
export PER_HOST_BATCH_SIZE=8
export TRAIN_STEPS=50
export RESOLUTION=512
python diffusers/examples/text_to_image/train_text_to_image_xla.py \
--pretrained_model_name_or_path=stabilityai/stable-diffusion-2-base \
--dataset_name=$DATASET_NAME --resolution=$RESOLUTION --center_crop --random_flip \
--train_batch_size=$PER_HOST_BATCH_SIZE --max_train_steps=$TRAIN_STEPS \
--learning_rate=1e-06 --mixed_precision=bf16 --profile_duration=$PROFILE_DURATION \
--output_dir=$OUTPUT_DIR --dataloader_num_workers=8 \
--loader_prefetch_size=4 --device_prefetch_size=4 --loader_prefetch_factor=4

[fancy45daddy]

@zpcore i do not need to train the model. i need to use tpu to produce picture. right now i only can run 1 tpu core. i need to use all cores in kaggle v3-8. could you help to check issue #8234？

[zpcore]

The kaggle code is kind of outdated. A fast work around is to manual add callback_steps=5 before line 1036 in diffusers package file like /home//.local/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py.

Just wrap the code in def process(rank):... and call torch_xla.distributed.xla_multiprocessing.spawn(process, start_method='fork') and it should automatically use 8 cores in v3-8.