-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
问题确认 Search before asking
- 我已经查询历史issue(包括open与closed),没有发现相似的bug。I have searched the open and closed issues and found no similar bug report.
Bug描述 Describe the Bug
5090上安装paddlepaddle3.2.0之后运行示例的视盘分割数据集使用的pp_liteseg模型出现一下错误
D:\Software\Anaconda\envs\paddle3\lib\site-packages\paddle\nn\layer\norm.py:818: UserWarning: When training, we now always track global mean and variance.
warnings.warn(
D:\Software\Anaconda\envs\paddle3\lib\site-packages\paddle\utils\decorator_utils.py:420: Warning:
Non compatible API. Please refer to https://www.paddlepaddle.org.cn/documentation/docs/en/develop/guides/model_convert/convert_from_pytorch/api_difference/torch/torch.max.html first.
warnings.warn(
W1201 06:26:02.397688 33280 multiply_fwd_func.cc:81] got different data type, run type promotion automatically, this may cause data type been changed.
W1201 06:26:02.690557 33280 backward.cc:462] While running Node (Conv2dGradNodeFinal) raises an EnforceNotMet exception
Traceback (most recent call last):
File "D:\Project\2025_4_month\PaddleSeg-release-2.10\tools\train.py", line 264, in
main(args)
File "D:\Project\2025_4_month\PaddleSeg-release-2.10\tools\train.py", line 234, in main
train(model,
File "D:\Project\2025_4_month\PaddleSeg-release-2.10\paddleseg\core\train.py", line 242, in train
scaled.backward() # do backward
File "D:\Software\Anaconda\envs\paddle3\lib\site-packages\paddle\base\wrapped_decorator.py", line 50, in wrapper
return decorated(*args, **kwargs)
File "D:\Software\Anaconda\envs\paddle3\lib\site-packages\paddle\base\framework.py", line 722, in impl
return func(*args, **kwargs)
File "D:\Software\Anaconda\envs\paddle3\lib\site-packages\paddle\base\dygraph\tensor_patch_methods.py", line 368, in backward
core.eager.run_backward([self], grad_tensor, retain_graph)
OSError: (External) CUDNN error(4000), CUDNN_STATUS_INTERNAL_ERROR.
[Hint: Please search for the error code(4000) on website (https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnStatus_t) to get Nvidia's official solution and advice about CUDNN Error.] (at ../paddle/phi/kernels/gpudnn/conv_cudnn_v7.h:480)
复现环境 Environment
paddlepaddle==3.2.0
paddleseg==2.10
Bug描述确认 Bug description confirmation
- 我确认已经提供了Bug复现步骤、代码改动说明、以及环境信息,确认问题是可以复现的。I confirm that the bug replication steps, code change instructions, and environment information have been provided, and the problem can be reproduced.
是否愿意提交PR? Are you willing to submit a PR?
- 我愿意提交PR!I'd like to help by submitting a PR!