Why Does Stablediffusionpipeline Return Black Images When Generating Multiple Images at Once?

I am using the StableDiffusionPipeline from the Hugging Face Diffusers library in Python 3.10.2, on an M2 Mac (I tagged it because this might be the issue). When I try to generate 1 image from 1 prompt, the output looks fine, but when I try to generate multiple images using the same prompt, the images are all either black squares or a random image (see example below). What could be the issue?

My code is as follows (where I change n_imgs from 1 to more than 1 to break it):

from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained("runwayml/stable-diffusion-v1-5")
pipe = pipe.to("mps")  # for M1/M2 chips
pipe.enable_attention_slicing()

prompt = "a photo of an astronaut driving a car on mars"

# First-time "warmup" pass (because of weird M1 behaviour)
_ = pipe(prompt, num_inference_steps=1)

# generate images
n_imgs = 1
imgs = pipe([prompt] * n_imgs).images

I also tried setting num_images_per_prompt instead of creating a list of repeated prompts in the pipeline call, but this gave the same bad results.

Example output (for multiple images):

[edit/update]: When I generate the images in a loop surrounding the pipe call instead of passing an iterable to the pipe call, it does work:

# generate images
n_imgs = 3
for i in range(n_imgs):
    img = pipe(prompt).images[0]
    # do something with img

But it is still a mystery to me as to why.

3 Answers

Apparently it is indeed an Apple Silicon (M1/M2) issue, of which Hugging Face is not yet sure why this is happening, see this GitHub issue for more details.

I think it might be a PyTorch issue given that a pure MPS version of the code (in Swift) worked fine last time I tested:

import MetalPerformanceShadersGraph

let graph = MPSGraph()
let x = graph.constant(1, shape: [32, 4096, 4096], dataType: .float32)
let y = graph.constant(1, shape: [32, 4096, 1], dataType: .float32)
let z = graph.matrixMultiplication(primary: x, secondary: y, name: nil)
let device = MTLCreateSystemDefaultDevice()!
let buf = device.makeBuffer(length: 16384)!
let td = MPSGraphTensorData(buf, shape: [64, 64], dataType: .int32)
let cmdBuf = MPSCommandBuffer(from: device.makeCommandQueue()!)
graph.encode(to: cmdBuf, feeds: [:], targetOperations: nil, resultsDictionary: [z:td], executionDescriptor: nil)
cmdBuf.commit()

See this thread for details:

You will encounter this error now if you ran that or anything related to MPS using M1/M2 apple devices.

NotImplementedError: The operator 'aten::index.Tensor' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on . As a temporary fix, you can set the environment variable PYTORCH_ENABLE_MPS_FALLBACK=1 to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

I'm actually looking for a workaround here too in case you guys know.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

David Miller

David Miller

Executive Financial & Market Analyst

David Miller brings 15 years of experience in global economics, personal finance strategy, and market dynamics. He specializes in turning complex economic trends into actionable insights for everyday readers.

Share this article
Twitter Facebook Pinterest