Feature: Light weight AI Sub-System
We would like Harmony OS to support the Light weight AI Sub-System: A robust framework to support AI application in OHOS. The framework which can cater to different application / use case requirements. The Light weight AI fits in the OHOS as an optional module which can be used on High capacity devices or light weight devices.
Work in progress
Need PMC approval: Light weight AI run-time interfaces kit
Need PMC approval: Light weight AI run-time framework
Need PMC approval: Third party dlpack used in Light weight AI run-time framework
PR sample app: Sample Applications using Light weight AI run-time
Motivation
Background
Efficient RAM Reduction :
Our Solution is Customizable, based on the memory requirement can choose between:
Efficient ROM reduction :
Depending on the need to the application, users can choose the optimization level which meets their requirements.
Proposed interface
Run-time
/**
* @yutong zhang Constructor, initializes the runtime and allocate necessary memory.
*
* @chancelai graph_byte Pointer to the graph data of the model.
* @chancelai param_byte Pointer to the param data of the model.
* @chancelai kernel_funcs The list of kernel functions used for this model.
*
/
Runtime(const char graph_byte, const char* params_byte, std::vector* kernel_funcs);
Set-Input
/**
* @yutong zhang Function to set the input tensor to the model.
*
* @chancelai name The name of the input tensor for which input need to set.
* @chancelai tensor The input tensor value to be set for execution of model.
*
/
void SetInput(const char name, DLTensor* tensor);
Run
/**
* @yutong zhang Function to execute the model.
*
*/
void Run();
Get-Result
/**
* @yutong zhang Return tensor for given output index after execution.
*
* @chancelai index The output index.
* @chancelai tensor The output pointer where the data is copied.
*
/
void GetOutput(int32_t index, DLTensor tensor);
Sample Application
Sample application has been implemented to use the above service. The Sample application is maintained in sample camera apps.
The Sample applications generated in the bin can be invoked to see the results
Sample Results
Future Enhancements
Currently the Light-weight AI framework is already work in progress as per the above mentioned PR’s and is waiting for approval to create repo.
Apart from the current optimized framework capabilities, the following enhancements are being planned to be supported.
Feature Enhancements
-- Memory planner (30-Oct)
-- CMSIS-NN (30-Oct)
-- Camera integrated demo (30-Oct)
-- TVM Model Conversion (30-Nov)
-- Enhanced Tiny kernels (30-Nov)
-- 4-bit quantized kernels (30-Dec)
-- AMP (30-Dec)
Model Zoo
-- Person detection
-- Image recognition
-- Speech detection
-- MNIST
-- Magic wand
Hardware support
-- Currently the Hi3516 and Hi3518 boards are being supported
-- Plan to support additional hardware in future (Q1-2021)
-- NXP IMXULL
-- Audrino
-- Sparkfun
Roadmap summary
Close the issue temporarily. it will be open the issue again or create a new issue after the feature having some progress
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。
登录 后才可以发表评论