代码拉取完成,页面将自动刷新
506
[Dataset] Multiprocessing support for Pyfunc
已合并
What type of PR is this?
Uncomment only one
/kind <>
line, hit enter to put that in a new line, and remove leading whitespaces from that line:/kind bug
/kind task
/kind feature
What does this PR do / why do we need it:
Add multiprocessing support for Pyfunc
Which issue(s) this PR fixes:
Part 1 of https://gitee.com/mindspore/dashboard/issues?id=I1D9CQ
Special notes for your reviewers:
This add multiprocess support for Pyfunc.
The following pieces are involved.
- Add new option python_multiprocess (default to false) to mapOp.
- On iterator construction, create process pool in python. (pool will be GCed along with iterator)
- Wrap each user pyfunc to make it executed in process pool.