StableDiffusionPipeline.from_pretrained でload errorが出る問題の対処

blog.vtryo.me

このブログを書いたときにコーディングした内容だと動作しなくなっていた。大したことはないのだが、普段コードを書かない人がこれに遭遇するとどう修正したらいいかわからない思うので修正メモを残す。

問題箇所

上記のブログでは、このようなコードを紹介した。

!pip install diffusers==0.7.2 transformers scipy ftfy

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline

model_id = "CompVis/stable-diffusion-v1-4"
device = "cuda"

pipe = StableDiffusionPipeline.from_pretrained(model_id, use_auth_token='<YOUR_ACCESS_TOKEN>')
pipe = pipe.to(device)

# Write what you want to draw
prompt = "Back view of beautiful girl, walking far away looking at map, elaborate modern building background, hopeful, concept art, comic, wallpaper, by Makoto Shinaki"

num = 0
for num in range(2):
  with autocast("cuda"):
      image = pipe(prompt, guidance_scale=7.5).images[0]

  image.save(str(num) + "_girl.png")
  image

で、これを実行すると以下のようなエラーに出くわすかもしれない（出くわした）。

diffusers/pipeline_utils.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
    596                         class_obj()
    597 
--> 598                     raise ValueError(
    599                         f"The component {class_obj} of {pipeline_class} cannot be loaded as it does not seem to have"
    600                         f" any of the loading methods defined in {ALL_IMPORTABLE_CLASSES}."

ValueError: The component <class 'transformers.models.clip.image_processing_clip.CLIPImageProcessor'> of <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> cannot be loaded as it does not seem to have any of the loading methods defined in {'ModelMixin': ['save_pretrained', 'from_pretrained'], 'SchedulerMixin': ['save_config', 'from_config'], 'DiffusionPipeline': ['save_pretrained', 'from_pretrained'], 'OnnxRuntimeModel': ['save_pretrained', 'from_pretrained'], 'PreTrainedTokenizer': ['save_pretrained', 'from_pretrained'], 'PreTrainedTokenizerFast': ['save_pretrained', 'from_pretrained'], 'PreTrainedModel': ['save_pretrained', 'from_pretrained'], 'FeatureExtractionMixin': ['save_pretrained', 'from_pretrained']}.

結論

今回変更したのは2箇所。

diffusersのバージョンを0.9.0に固定する。これは、上記の問題が修正されたバージョン

!pip install diffusers==0.9.0 transformers scipy ftfy

モデルをダウンロードする際、ログインしなくてよくなったのでToken指定を削除

pipe = StableDiffusionPipeline.from_pretrained(model_id)

全体像

import torch
from torch import autocast
from diffusers import StableDiffusionPipeline

model_id = "CompVis/stable-diffusion-v1-4"
device = "cuda"

pipe = StableDiffusionPipeline.from_pretrained(model_id)
pipe = pipe.to(device)

# Write what you want to draw
prompt = "Back view of beautiful girl, walking far away looking at map, elaborate modern building background, hopeful, concept art, comic, wallpaper, by Makoto Shinaki"

num = 0
for num in range(2):
  with autocast("cuda"):
      image = pipe(prompt, guidance_scale=7.5).images[0]

  image.save(str(num) + "_girl.png")
  image