name | about | labels |
---|---|---|
Bug Report | Use this template for reporting a bug | kind/bug |
[CI][MS]联合CI-910B 模型导出失败,padv3算子切分策略报错
Ascend
/GPU
/CPU
) / 硬件环境:Please delete the backend not involved / 请删除不涉及的后端:
/device ascend/GPU/CPU/kirin/等其他芯片
Software Environment / 软件环境 (Mandatory / 必填):
-- MindSpore version (e.g., 1.7.0.Bxxx) :
-- Python version (e.g., Python 3.7.5) :
-- OS platform and distribution (e.g., Linux Ubuntu 16.04):
-- GCC/Compiler version (if compiled from source):
Excute Mode / 执行模式 (Mandatory / 必填)(PyNative
/Graph
):
Please delete the mode not involved / 请删除不涉及的模式:
/mode pynative
/mode graph
[ERROR] PARALLEL(459695,ffffa32580b0,python):2024-03-14-01:14:09.398.959 [mindspore/ccsrc/frontend/parallel/ops_info/pad_info.cc:101] CheckStrategy] PadV3Info6363: the padding dimension of input can not be split, the strategy of input is [const vector]{1, 8, 1, 1}, and the paddings flag is [const vector]{0, 1, 0, 0}
[ERROR] PARALLEL(459695,ffffa32580b0,python):2024-03-14-01:14:09.399.048 [mindspore/ccsrc/frontend/parallel/ops_info/operator_info.cc:1126] InitForCostModelWithAutoRepeatCalc] PadV3Info6363: CheckStrategy failed.
[ERROR] PARALLEL(459695,ffffa32580b0,python):2024-03-14-01:14:09.399.066 [mindspore/ccsrc/frontend/parallel/ops_info/operator_info.cc:1068] Init] PadV3Info6363 : Init failed.
Traceback (most recent call last):
File "/data2/wxr/xiaoyi_test/script/pangu_am_deploy/workspace/CI/noah/910B/pangu_am_deploy-release-v0.9.10/pangu_sigma/evaluate.py", line 250, in load_model
predict_layout = model_predict.infer_predict_layout(inputs_np, experts_np, attention_mask, position_ids, init_true, batch_valid_length, lora_ids_np, skip_backend_compile=skip_backend_compile)
File "/home/miniconda3/envs/ci39/lib/python3.9/site-packages/mindspore/train/model.py", line 1879, in infer_predict_layout
predict_net.compile(*predict_data)
File "/home/miniconda3/envs/ci39/lib/python3.9/site-packages/mindspore/nn/cell.py", line 963, in compile
_cell_graph_executor.compile(self, *self._compile_args, phase=self.phase,
File "/home/miniconda3/envs/ci39/lib/python3.9/site-packages/mindspore/common/api.py", line 1584, in compile
result = self._graph_executor.compile(obj, args, kwargs, phase, self._use_vm_mode())
RuntimeError: Failure:operator PadV3 init failed
Please assign maintainer to check this issue.
请为此issue分配处理人。
@188******92
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
感谢您的反馈,您可以评论//mindspore-assistant更快获取帮助,更多标签可以查看标签列表:
根因分析:
In MindSpore, padding order starts from the last dimension and goes backward (same as PyTorch), but GE padding order
starts from the first dimension and goes forward. So the purpose of this pass is to adapt MindSpore PadV3 op to Ascend
GE PadV3 op. Namely, reverse the padding order.
Main steps:
已补充st测试用例,见关联pr。
3.16验证通过,导出、静态转动态、推理均成功,日志见http://10.90.67.50:8080/jenkins/job/Combined_Pipeline_910B_PanguSigma_Inference/276/console
使用的软件包,
MindSpore 2.3.0.B010-20240315231353
Ascend HDK 24.1.RC1.B031-20240307094942-32
Milan-ASL V100R001C17B214
登录 后才可以发表评论