1.6K Star 6K Fork 2.3K

GVPMindSpore / mindspore

 / 详情

Euler系统在Mindspore1.5.0.B110版本使用cann版本CANN 5.0.3.B076测试resnet50 1p模型训练,每个step比8p的每个step耗时多3ms。

WIP
RFC
Opened this issue  
2021-10-18 20:45

【Atlas800型号9000】【模型训练】euler系统在Mindspore1.5.0.B110版本使用cann版本CANN 5.0.3.B076测试fresnet50 1p模型训练,每个step比8p的每个step耗时多3ms。

Environment

Uncomment only one /device <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:
/device ascend

Related testcase

resnet50模型单卡性能测试:batch_size=256 进行modelzoo renset50模型单卡性能

Steps to reproduce the issue

  1. get code from https://github.com/mindspore-ai/mindspore/tree/master
  2. 环境10.174.216.206 root/HUAwei123
  3. 路径:/home/CI_daily/B110/models-master/official/cv/resnet/scripts
  4. 执行命令:bash run_standalone_train.sh /home/data/ImageNet2012/train /home/CI_daily/B110/models-master/official/cv/resnet/config/resnet50_imagenet2012_Boost_config.yaml

Describe the current behavior

8卡性能:
输入图片说明

Describe the expected behavior

单卡性能:

输入图片说明

Related log / screenshot

Special notes for this issue

euler系统在Mindspore1.5.0.B110版本使用cann版本5.0.3.B076测试resnet50模型单p训练,每个step的时间下降3ms。

定位开发:林清客 00356578

Comments (6)

caimengping createdBug-Report
caimengping set related repository to MindSpore/mindspore
Expand operation logs

Please assign maintainer to check this issue.
请为这个issue分配处理人, @fangwenyi @chengxiaoli

Please add labels (comp or sig),also you can visit "https://gitee.com/mindspore/community/blob/master/sigs/dx/docs/labels.md" to find more.
为了让问题更快得到响应,请您为该issue打上 组件(comp)或兴趣组(sig) 标签,打上标签的问题可以直接推送给责任人进行处理。更多的标签可以查看
https://gitee.com/mindspore/community/blob/master/sigs/dx/docs/labels.md
以组件问题为例,如果你发现问题是data组件造成的,你可以这样评论:
//comp/data
当然你也可以向data SIG组求助,可以这样写:
//comp/data
//sig/data
如果是一个简单的问题,你可以留给刚进入社区的小伙伴来回答,这时候你可以这样写:
//good-first-issue
恭喜你,你已经学会了使用命令来打标签,接下来就在下面的评论里打上标签吧!

hello, @caimengping @caimengping , we suggest you add some labels like:
你好, @caimengping @caimengping , 建议您为这个issue打上标签:
//comp/train

i-robot added
 
kind/bug
label
caimengping changed description

你好,问题已经收到,会尽快帮助分析定位问题。请耐心等待下~

chengxiaoli changed issue state from TODO to ACCEPTED
chengxiaoli set priority to Main
chengxiaoli set assignee to linqingke
chengxiaoli assigned collaborator chengxiaoli
caimengping changed title
fangwenyi added
 
mindspore-assistant
label
linqingke assigned collaborator linqingke
linqingke changed assignee from linqingke to guozhijian

产品单卡性能数据存在瓶颈,Q4试验异构方案。

fangwenyi set deadline to 2021-12-31
fangwenyi removed
 
kind/bug
label
fangwenyi changed issue type from Bug-Report to RFC
fangwenyi changed related project from MindSpore Bug Tracking System to not set
fangwenyi set milestone to B-SIG-Data

@caimengping 计划Q4解决,及时关注

fangwenyi changed issue state from ACCEPTED to WIP
fangwenyi changed milestone from B-SIG-Data to not set
fangwenyi changed priority from Main to Not specified
fangwenyi changed assignee from guozhijian to linqingke
fangwenyi unassigned collaborator linqingke
fangwenyi assigned collaborator guozhijian

Sign in to comment

Status
Assignees
Projects
Milestones
Pull Requests
Successfully merging a pull request will close this issue.
Branches
Planed to start   -   Planed to end
-
Top level
Priority
Duration (hours)
Confirm
参与者(6)
6575151 linqingke 1584444037
Python
1
https://git.oschina.net/mindspore/mindspore.git
git@git.oschina.net:mindspore/mindspore.git
mindspore
mindspore
mindspore

Search

182229 41614e54 1850385 182230 7885ed45 1850385