Skip to content

[Bug] torch.load raise UnpicklingError #53

@Mr-brillianter

Description

@Mr-brillianter

Checklist before your report.

  • I have verified that the issue exists against the master branch of AdaSeq.
  • I have read the relevant section in the contribution guide on reporting bugs.
  • I have checked the issues list for similar or identical bug reports.
  • I have checked the pull requests list for existing proposed fixes.
  • I have checked the commit log to find out if the bug was already fixed in the master branch.

What happened?

Where should I modify the config or how should I fix this error raised within the lib?

  • I've tried set TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 but failed. I guess it does not take effect as ENV is not set within the lib.
  • I also tried torch.serialization.add_safe_globals but there are too many variables may need to add.

Python traceback

show/hide

UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Can only build Tensor, Parameter, OrderedDict or types allowlisted via `add_safe_globals`, but got <class 'numpy.dtypes.UInt32DType'>

Operating system

Ubuntu 22.04 LTS

Python version

3.11.11

Output of pip freeze

show/hide

absl-py==2.2.0
accelerate==1.10.0
adaseq==0.6.6
addict==2.4.0
aiofiles==24.1.0
aiohappyeyeballs==2.6.1
aiohttp==3.12.15
aiosignal==1.4.0
albucore==0.0.24
albumentations==2.0.8
alias-free-torch==0.0.6
aliyun-python-sdk-core==2.16.0
aliyun-python-sdk-kms==2.16.5
annotated-types==0.7.0
antlr4-python3-runtime==4.9.3
anyio==4.9.0
apex==0.1
appdirs==1.4.4
argon2-cffi==25.1.0
argon2-cffi-bindings==21.2.0
arrow==1.3.0
asttokens==3.0.0
astunparse==1.6.3
async-lru==2.0.5
attrdict==2.0.1
attrs==25.3.0
audioread==3.0.1
auto_gptq==0.7.1
autoawq==0.2.9
av==15.0.0
babel==2.17.0
basicsr==1.4.2
beartype==0.21.0
beautifulsoup4==4.13.4
binpacking==1.5.2
biopython==1.85
bitarray==3.2.0
bitsandbytes==0.47.0
bitstring==4.3.1
black==25.1.0
bleach==6.2.0
blis==0.7.11
blobfile==3.0.0
bmt-clipit==1.0
boto3==1.40.11
botocore==1.40.11
Brotli==1.1.0
cachetools==5.5.2
catalogue==2.0.10
certifi==2025.1.31
cffi==1.17.1
cfgv==3.4.0
charset-normalizer==3.4.1
chumpy==0.70
cityscapesScripts==2.2.4
click==8.1.8
clip==1.0
cloudpathlib==0.16.0
cloudpickle==3.1.1
colorama==0.4.6
coloredlogs==15.0.1
comm==0.2.2
confection==0.1.5
ConfigArgParse==1.7.1
contourpy==1.3.2
control-ldm==0.1.0
cpm-kernels==1.0.11
crcmod==1.7
cryptography==45.0.6
cycler==0.12.1
cymem==2.0.11
Cython==0.29.36
dacite==1.9.2
dataclasses==0.6
datasets==3.2.0
ddpm-guided-diffusion==0.0.0
debugpy==1.8.14
decorator==4.4.2
decord==0.6.0
deepspeed==0.17.4
defusedxml==0.7.1
descartes==1.1.0
detectron2 @ git+https://github.com/facebookresearch/detectron2.git@65184fc057d4fab080a98564f6b60fae0b94edc4
dgl==2.1.0+cu121
diffusers==0.34.0
dill==0.3.7
diskcache==5.6.3
Distance==0.1.3
distlib==0.4.0
distro==1.9.0
dotenv==0.9.9
easydict==1.13
edit-distance==1.0.6
editdistance==0.8.1
einops==0.8.1
embeddings==0.0.8
emoji==2.14.1
espnet-tts-frontend==0.0.3
et_xmlfile==2.0.0
evalscope==0.17.1
evaluate==0.4.5
executing==2.2.0
expecttest==0.3.0
face-alignment==1.4.1
fairscale==0.4.13
fairseq @ https://github.com/liyaodev/fairseq/releases/download/v0.12.3.1/fairseq-0.12.3.1-cp311-cp311-linux_x86_64.whl
fastai==2.8.3
fastapi==0.116.1
fastcore==1.8.7
fastdownload==0.0.7
fastjsonschema==2.21.1
fastprogress==1.0.3
fasttext==0.9.3
fasttransform==0.0.2
ffmpeg==1.4
ffmpeg-python==0.2.0
ffmpy==0.6.1
filelock==3.17.0
fire==0.7.1
flake8==7.3.0
flash-attn==2.7.1.post4
flatbuffers==25.2.10
fonttools==4.58.1
fqdn==1.5.1
frozenlist==1.7.0
fsspec==2023.10.0
ftfy==6.3.1
funasr==1.2.7
func-timeout==4.3.5
funtextprocessing==0.1.1
future==1.0.0
fuzzywuzzy==0.18.0
fvcore==0.1.5.post20221221
g2p==2.2.2
g2p-en==2.1.0
gast==0.6.0
gekko==1.3.0
google-auth==2.40.3
google-genai==1.30.0
google-pasta==0.2.0
gradio==5.42.0
gradio_client==1.11.1
greenlet==3.2.4
groovy==0.1.2
grpcio==1.71.0
h11==0.16.0
h5py==3.13.0
hdbscan==0.8.40
hf-xet==1.1.7
hjson==3.1.0
httpcore==1.0.9
httpx==0.28.1
huggingface-hub==0.34.4
human-eval==1.0.3
humanfriendly==10.0
hydra-core==1.3.2
hyper-connections==0.2.1
HyperPyYAML==1.2.2
identify==2.6.13
idna==3.10
imageio==2.37.0
imageio-ffmpeg @ https://modelscope.oss-cn-beijing.aliyuncs.com/packages/imageio_ffmpeg-0.4.9-py3-none-any.whl#sha256=a42128ee8dbfaa35f5e9ada072b5d64928f044f126b472a3c917de2d46a2dd98
imgaug==0.4.0
immutabledict==4.2.1
importlib_metadata==8.7.0
inflect==7.0.0
iniconfig==2.1.0
interegular==0.3.3
iopath==0.1.9
ipdb==0.13.13
ipykernel==6.29.5
ipython==9.0.2
ipython_pygments_lexers==1.1.1
ipywidgets==8.1.7
isoduration==20.11.0
isort==6.0.1
jaconv==0.4.0
jamo==0.4.1
jedi==0.19.2
jieba==0.42.1
Jinja2==3.1.6
jiter==0.10.0
jmespath==0.10.0
joblib==1.4.2
json-tricks==3.17.3
json5==0.12.0
json_repair==0.49.0
jsonlines==4.0.0
jsonplus==0.8.0
jsonpointer==3.0.0
jsonschema==4.24.0
jsonschema-specifications==2025.4.1
jupyter==1.1.1
jupyter-console==6.6.3
jupyter-events==0.12.0
jupyter-lsp==2.2.5
jupyter_client==8.6.3
jupyter_core==5.8.1
jupyter_server==2.16.0
jupyter_server_terminals==0.5.3
jupyterlab==4.4.6
jupyterlab_pygments==0.3.0
jupyterlab_server==2.27.3
jupyterlab_widgets==3.0.15
kaldiio==2.18.1
kantts==1.0.1
keras==3.9.0
kiwisolver==1.4.8
kornia==0.8.1
kornia_rs==0.1.9
langcodes==3.5.0
langdetect==1.0.9
language_data==1.3.0
lap @ git+https://github.com/gatagat/lap.git@d35979164b6d27f2d886860cc1fe641023f522d7
lark==1.2.2
latex2sympy2_extended==1.10.2
lazy_loader==0.4
Levenshtein==0.27.1
libclang==18.1.1
librosa==0.11.0
lightning-utilities==0.15.2
llvmlite==0.44.0
lmdb==1.6.2
lmdeploy==0.6.2
local-attention==1.11.2
lpips==0.1.4
lxml==5.3.1
lyft-dataset-sdk==0.0.8
marisa-trie==1.3.0
Markdown==3.7
markdown-it-py==3.0.0
MarkupSafe==3.0.2
matplotlib==3.10.3
matplotlib-inline==0.1.7
mccabe==0.7.0
mdurl==0.1.2
megatron-util==1.3.2
MinDAEC @ https://modelscope.oss-cn-beijing.aliyuncs.com/releases/dependencies/MinDAEC-0.0.2-py3-none-any.whl#sha256=ba64c619611bc3faf1ccd6004bec1178ee27f1beab24835fd2e2f3130b37b25d
mir_eval==0.8.2
mistune==3.1.3
ml-dtypes==0.3.2
ml_collections==1.1.0
mmcls==0.25.0
mmcv-full @ file:///tmp/mmcv
mmdet==2.28.2
mmdet3d==1.0.0a1
mmengine-lite==0.10.7
mmsegmentation==0.30.0
mock==5.2.0
modelscope @ file:///tmp/modelscope
more-itertools==10.7.0
moviepy==1.0.3
mpi4py==4.1.0
mpmath==1.3.0
ms-agent==1.1.2
ms-funcodec==0.2.0
ms-opencompass==0.1.6
ms-swift @ file:///tmp/ms-swift
ms-vlmeval==0.0.18
msgpack==1.1.1
multidict==6.6.4
multiprocess==0.70.15
munkres==1.1.4
murmurhash==1.0.13
mypy_extensions==1.1.0
namex==0.0.8
narwhals==2.1.2
nbclient==0.10.2
nbconvert==7.16.6
nbformat==5.10.4
nerfacc==0.2.2
nest-asyncio==1.6.0
networkx==3.4.2
ninja==1.13.0
nltk==3.9.1
nodeenv==1.9.1
notebook==7.4.5
notebook_shim==0.2.4
numba==0.61.2
numpy==1.26.4
nuscenes-devkit==1.1.9
nvdiffrast @ file:///nvdiffrast
nvidia-cublas-cu12==12.4.5.8
nvidia-cuda-cupti-cu12==12.4.127
nvidia-cuda-nvrtc-cu12==12.4.127
nvidia-cuda-runtime-cu12==12.4.127
nvidia-cudnn-cu12==9.1.0.70
nvidia-cufft-cu12==11.2.1.3
nvidia-curand-cu12==10.3.5.147
nvidia-cusolver-cu12==11.6.1.9
nvidia-cusparse-cu12==12.3.1.170
nvidia-cusparselt-cu12==0.6.2
nvidia-ml-py==12.575.51
nvidia-nccl-cu12==2.21.5
nvidia-nvjitlink-cu12==12.4.127
nvidia-nvtx-cu12==12.4.127
omegaconf==2.3.0
onnx==1.18.0
onnxruntime==1.22.1
onnxsim==0.4.36
open_clip_torch==3.1.0
openai==1.99.9
openai-whisper==20250625
OpenCC==1.1.9
opencv-python==4.11.0.86
opencv-python-headless==4.12.0.88
openpyxl==3.1.5
opt_einsum==3.4.0
optimum==1.27.0
optree==0.14.1
orjson==3.11.2
oss2==2.19.1
outlines==0.0.41
overrides==7.7.0
packaging==24.2
pai-easycv==0.11.7
paint-ldm==0.1.0
pandas==2.3.1
pandocfilters==1.5.1
panopticapi==0.1
panphon==0.22.2
parso==0.8.4
pathlib_abc==0.1.1
pathspec==0.12.1
pathy==0.11.0
peft==0.11.1
pexpect==4.9.0
phaseaug==1.0.1
pickleshare==0.7.5
pillow==11.1.0
platformdirs==4.3.7
plotly==6.3.0
pluggy==1.6.0
plum-dispatch==2.5.7
plyfile==1.1.2
pointnet2 @ git+https://github.com/gxd1994/Pointnet2.PyTorch.git@8e25e3a22386d27286f5cd415153f0002317591b#subdirectory=pointnet2
pooch==1.8.2
portalocker==3.1.1
pre_commit==4.3.0
preshed==3.0.10
prettytable==3.16.0
proglog==0.1.12
prometheus_client==0.22.1
prompt_toolkit==3.0.50
propcache==0.3.2
protobuf==3.20.3
psutil==7.0.0
ptflops==0.7.4
ptyprocess==0.7.0
pure_eval==0.2.3
py-cpuinfo==9.0.0
py-sound-connect==0.1.0
pyarrow==21.0.0
pyarrow-hotfix==0.7
pyasn1==0.6.1
pyasn1_modules==0.4.2
pybind11==3.0.0
pyclipper==1.3.0.post6
pycocoevalcap==1.2
pycocotools==2.0.10
pycodestyle==2.14.0
pycparser==2.22
pycryptodome==3.23.0
pycryptodomex==3.23.0
pydantic==2.11.7
pydantic_core==2.33.2
pyDeprecate==0.3.2
pydot==4.0.1
pydub==0.25.1
pyflakes==3.4.0
Pygments==2.19.1
PyMCubes==0.1.4
pynini==2.1.5
pynndescent==0.5.13
pynvml==12.0.0
pyparsing==3.2.3
pypinyin==0.44.0
pyquaternion==0.9.9
pysptk==1.0.1
pytest==8.4.1
pythainlp==5.1.2
python-crfsuite==0.9.11
python-dateutil==2.9.0.post0
python-dotenv==1.1.1
python-json-logger==3.3.0
python-Levenshtein==0.27.1
python-multipart==0.0.20
pytorch-lightning==1.7.7
pytorch-metric-learning==2.8.1
pytorch-wavelets==1.3.0
pytorch-wpe==0.0.1
pytorch3d @ git+https://github.com/facebookresearch/pytorch3d.git@75ebeeaea0908c5527e7b1e305fbc7681382db47
pytz==2025.2
pyvi==0.1.1
PyWavelets==1.9.0
PyYAML==6.0.2
pyzmq==26.4.0
qwen-omni-utils==0.0.8
qwen-vl-utils==0.0.11
rank-bm25==0.2.2
RapidFuzz==3.13.0
referencing==0.36.2
regex==2024.11.6
requests==2.32.3
rfc3339-validator==0.1.4
rfc3986-validator==0.1.1
rich==13.9.4
rotary-embedding-torch==0.8.9
rouge==1.0.1
rouge-chinese==1.0.3
rouge-score==0.1.2
rpds-py==0.25.1
rsa==4.9.1
ruamel.yaml==0.18.14
ruamel.yaml.clib==0.2.12
ruff==0.12.9
s3transfer==0.13.1
sacrebleu==2.5.1
sacremoses==0.1.1
safehttpx==0.1.6
safetensors==0.6.2
scikit-image==0.25.2
scikit-learn==1.7.1
scipy==1.12.0
seaborn==0.13.2
semantic-version==2.10.0
Send2Trash==1.8.3
sentence-transformers==5.1.0
sentencepiece==0.2.1
seqeval==1.2.2
Shapely==1.8.4
shellingham==1.5.4
shortuuid==1.0.13
shotdetect-scenedetect-lgss==0.0.4
simplejson==3.20.1
simsimd==6.5.0
six==1.17.0
sklearn-crfsuite==0.5.0
smart-open==6.4.0
smplx==0.1.28
sniffio==1.3.1
sortedcontainers==2.4.0
soundfile==0.13.1
soupsieve==2.7
sox==1.5.0
soxr==0.5.0.post1
spacy==3.7.0
spacy-legacy==3.0.12
spacy-loggers==1.0.5
speechbrain==1.0.3
srsly==2.5.1
sse-starlette==3.0.2
stack-data==0.6.3
stanza==1.10.1
starlette==0.47.2
stringzilla==3.12.6
sty==1.0.6
subword-nmt==0.3.8
sympy==1.13.1
tabulate==0.9.0
taming-transformers-rom1504==0.0.6
tb-nightly==2.20.0a20250605
tenacity==9.1.2
tensorboard==2.16.2
tensorboard-data-server==0.7.2
tensorboardX==2.6.4
tensorflow==2.16.1
tensorflow-estimator==2.15.0
tensorflow-io-gcs-filesystem==0.37.1
termcolor==2.5.0
terminado==0.18.1
terminaltables==3.1.10
text-unidecode==1.3
text2sql-lgesql==1.3.0
tf-slim==1.1.0
tf_keras==2.16.0
thinc==8.2.5
thop==0.1.1.post2209072238
threadpoolctl==3.6.0
tifffile==2025.6.1
tiktoken==0.11.0
timeout-decorator==0.5.0
timm==1.0.19
tinycss2==1.4.0
tokenizers==0.21.4
tomlkit==0.13.3
torch==2.6.0
torch-complex==0.4.4
torch-scatter==2.1.2
torchaudio==2.3.1
torchdata==0.11.0
torchmetrics==0.11.4
torchsde==0.2.6
torchsummary==1.5.1
torchvision==0.21.0
tornado==6.5.1
tqdm==4.67.1
traitlets==5.14.3
trampoline==0.1.2
transformers==4.55.2
transformers-stream-generator==0.0.5
trimesh==2.35.39
triton==3.2.0
trl==0.20.0
ttsfrd==0.1.0
typeguard==2.13.3
typer==0.9.4
types-python-dateutil==2.9.0.20250516
typing==3.7.4.3
typing-inspection==0.4.1
typing_extensions==4.12.2
tzdata==2025.2
ujson==5.10.0
umap-learn==0.5.9.post2
unicodecsv==0.14.1
unicodedata2==16.0.0
Unidecode==1.4.0
uri-template==1.3.0
urllib3==2.3.0
utils==1.0.2
uvicorn==0.35.0
validators==0.35.0
versioneer==0.29
videofeatures-clipit==1.0
virtualenv==20.34.0
wasabi==1.1.3
wcwidth==0.2.13
weasel==0.3.4
webcolors==24.11.1
webencodings==0.5.1
websocket-client==1.8.0
websockets==15.0.1
Werkzeug==3.1.3
wget==3.2
widgetsnbextension==4.0.14
word2number==1.1
wrapt==1.17.2
xlsxwriter==3.2.5
xtcocotools @ git+https://github.com/jin-s13/xtcocoapi.git@70a6aaa759d39a5d7ff919d3c7973daf3d244ecf
xxhash==3.5.0
yacs==0.1.8
yapf==0.30.0
yarl==1.20.1
zhconv==1.4.3
zipp==3.23.0
zstandard==0.23.0

How to reproduce

show/hide


Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions