1.6K Star 6K Fork 2.3K

GVPMindSpore / mindspore

 / 详情

[CI] ST probabilistic core dump in gate<test_topk_op.py, test_math_ops.py>

DONE
Bug-Report member
Opened this issue  
2021-10-21 19:32
name about labels
Bug Report Use this template for reporting a bug kind/bug

Environment

  • Hardware Environment(Ascend/GPU/CPU):

Uncomment only one /device <> line, hit enter to put that in a new line, and remove leading whitespaces from that line:

/device cpu

  • Software Environment:
    -- MindSpore version (source or binary):
    -- Python version (e.g., Python 3.7.5):
    -- OS platform and distribution (e.g., Linux Ubuntu 16.04):
    -- GCC/Compiler version (if compiled from source):

Related testcase

test_math_ops.py::test_logaddexp
test_topk_op.py::test_topk

Steps to reproduce the issue

  1. code compile
  2. install mindspore*.whl
  3. run testcase

Describe the current behavior

Testcase probabilistic core dump

Describe the expected behavior

These testcases run success

Related log / screenshot

URL: https://build.mindspore.cn/blue/organizations/jenkins/MindSpore_Gitee_Gate/detail/MindSpore_Gitee_Gate/106167/pipeline
https://build.mindspore.cn/blue/organizations/jenkins/MindSpore_Gitee_Gate/detail/MindSpore_Gitee_Gate/106150/pipeline/613
输入图片说明
输入图片说明

Special notes for this issue

cpu用例概率性失败,导致门禁堵塞

Comments (8)

wmzheng2020 set priority to Serious
wmzheng2020 createdBug-Report
wmzheng2020 set assignee to 范吉斌
wmzheng2020 set related repository to MindSpore/mindspore

Please add labels (comp or sig),also you can visit "https://gitee.com/mindspore/community/blob/master/sigs/dx/docs/labels.md" to find more.
为了让问题更快得到响应,请您为该issue打上 组件(comp)或兴趣组(sig) 标签,打上标签的问题可以直接推送给责任人进行处理。更多的标签可以查看
https://gitee.com/mindspore/community/blob/master/sigs/dx/docs/labels.md
以组件问题为例,如果你发现问题是data组件造成的,你可以这样评论:
//comp/data
当然你也可以向data SIG组求助,可以这样写:
//comp/data
//sig/data
如果是一个简单的问题,你可以留给刚进入社区的小伙伴来回答,这时候你可以这样写:
//good-first-issue
恭喜你,你已经学会了使用命令来打标签,接下来就在下面的评论里打上标签吧!

i-robot added
 
kind/bug
label
wmzheng2020 assigned collaborator 杨林枫
wmzheng2020 set start time to 2021-10-21
wmzheng2020 set deadline to 2021-10-22
wmzheng2020 set branch to master
wmzheng2020 changed title
wmzheng2020 changed description
wmzheng2020 added
 
stat/occasionally
label

每次都挂在不同用例,非cpu算子内部问题,大概率框架流程问题,转给黎明奇继续处理。

范吉斌 assigned collaborator 范吉斌
范吉斌 changed assignee from 范吉斌 to limingqi107
limingqi107 changed issue state from TODO to VALIDATION
limingqi107 assigned collaborator limingqi107
limingqi107 changed assignee from limingqi107 to wmzheng2020

Appearance & Root Cause

pyNative单算子插入cast场景,会概率出现执行完后tensor先析构device_address,memoryManageActor再执行FreeMemory消息里访问了device_address导致出现core

Fix Solution

确保memoryManageActor再先执行完FreeMemory,再退出执行流程触发tensor析构,增加loopcountActor控制
!25472:fix the coredump probability of pyNative free memory

wmzheng2020 changed milestone from B-ComponentTest to B-SolutionTest
wmzheng2020 changed issue state from VALIDATION to DONE
wmzheng2020 unassigned collaborator 杨林枫
wmzheng2020 unassigned collaborator 范吉斌

Sign in to comment

Status
Assignees
Projects
Milestones
Pull Requests
Successfully merging a pull request will close this issue.
Branches
Planed to start   -   Planed to end
-
Top level
Priority
Duration (hours)
Confirm
参与者(4)
Python
1
https://git.oschina.net/mindspore/mindspore.git
git@git.oschina.net:mindspore/mindspore.git
mindspore
mindspore
mindspore

Search