Getting a Error When Running Gptneoxforcausallm from Transformers Library: Nameerror: Name 'Init_Empty_Weights' Is Not Defined
I'm Trying to Run Openassistant's Pythia-12B Model but I'm Getting the Following Error: Nameerror: Name 'Init_Empty_Weights' Is Not Defined I Have Accelerate...
I'm trying to run OpenAssistant's pythia-12b model but I'm getting the following error:
NameError: name 'init_empty_weights' is not defined
I have Accelerate installed, and I'm running Transformers version 4.25.1
from transformers import AutoModelForCausalLM, GPTNeoXForCausalLM, AutoTokenizer
Model = "OpenAssistant/oasst-sft-1-pythia-12b"
tokenizer = AutoTokenizer.from_pretrained(Model, cache_dir='models_hf')
model = GPTNeoXForCausalLM.from_pretrained(Model, device_map="auto", load_in_8bit=True, cache_dir='models_hf',low_cpu_mem_usage=True)
message = "Hello, How are you?"
inp = "<|prompter|>"+message+"<|endoftext|><|assistant|>"
data = tokenizer([inp], return_tensors="pt")
data = {k: v.to(model.device) for k, v in data.items() if k in ("input_ids", "attention_mask")}
outputs = model.generate(**data)
print(tokenizer.decode(outputs[0]))
1 Answer
In my case, installing accelerate package and re-starting the runtime was enough. The same issue was also discussed here.
Alexander Ross has covered the video game industry for a decade, writing deep dives on game design, esports tournaments, VR developments, and gaming culture.
