432 Star 2.3K Fork 523

OpenHarmony / docs

[Feature Request] Add Light-weight AI framework to OHOS

需求
已完成
VinodSKumar  Opened this issue

[Feature Request]

Feature: Light weight AI Sub-System
We would like Harmony OS to support the Light weight AI Sub-System: A robust framework to support AI application in OHOS. The framework which can cater to different application / use case requirements. The Light weight AI fits in the OHOS as an optional module which can be used on High capacity devices or light weight devices.

Light Weight AI subsystem positioning

Work in progress
Need PMC approval: Light weight AI run-time interfaces kit
Need PMC approval: Light weight AI run-time framework
Need PMC approval: Third party dlpack used in Light weight AI run-time framework
PR sample app: Sample Applications using Light weight AI run-time

Motivation

  • Low Energy Consumption
    -- Micro-controllers consume very little energy and are efficient
    -- Micro-controllers are usually not wired and typically rely on batteries or energy harvesting.
  • Flexibility
    -- We can add AI ability to various devices without network connectivity, which are normally constrained by bandwidth, power and / or latency.
    -- Micro-controllers are very common and the possibilities are endless when we bring AI to micro-controllers.
  • Cost
    -- Hardware limitation and design specific to program limitation
    -- Little external third-party configuration
  • Privacy
    -- Users do not have to worry about privacy as no data is required to leave the device and the computation will happen on the device itself.

Background
Efficient RAM Reduction :
Our Solution is Customizable, based on the memory requirement can choose between:

  • Shared params solutions
  • On demand memory allocation

Efficient ROM reduction :
Depending on the need to the application, users can choose the optimization level which meets their requirements.

Proposed interface
Run-time
/**
* @yutong zhang Constructor, initializes the runtime and allocate necessary memory.
*
* @chancelai graph_byte Pointer to the graph data of the model.
* @chancelai param_byte Pointer to the param data of the model.
* @chancelai kernel_funcs The list of kernel functions used for this model.
*
/
Runtime(const char
graph_byte, const char* params_byte, std::vector* kernel_funcs);

Set-Input
/**
* @yutong zhang Function to set the input tensor to the model.
*
* @chancelai name The name of the input tensor for which input need to set.
* @chancelai tensor The input tensor value to be set for execution of model.
*
/
void SetInput(const char
name, DLTensor* tensor);

Run
/**
* @yutong zhang Function to execute the model.
*
*/
void Run();

Get-Result
/**
* @yutong zhang Return tensor for given output index after execution.
*
* @chancelai index The output index.
* @chancelai tensor The output pointer where the data is copied.
*
/
void GetOutput(int32_t index, DLTensor
tensor);

Sample Application
Sample application has been implemented to use the above service. The Sample application is maintained in sample camera apps.
The Sample applications generated in the bin can be invoked to see the results

Sample Results
Sample Application list and results

Future Enhancements
Currently the Light-weight AI framework is already work in progress as per the above mentioned PR’s and is waiting for approval to create repo.
Apart from the current optimized framework capabilities, the following enhancements are being planned to be supported.

  • Feature Enhancements
    -- Memory planner (30-Oct)
    -- CMSIS-NN (30-Oct)
    -- Camera integrated demo (30-Oct)
    -- TVM Model Conversion (30-Nov)
    -- Enhanced Tiny kernels (30-Nov)
    -- 4-bit quantized kernels (30-Dec)
    -- AMP (30-Dec)

  • Model Zoo
    -- Person detection
    -- Image recognition
    -- Speech detection
    -- MNIST
    -- Magic wand

  • Hardware support
    -- Currently the Hi3516 and Hi3518 boards are being supported
    -- Plan to support additional hardware in future (Q1-2021)
    -- NXP IMXULL
    -- Audrino
    -- Sparkfun

Roadmap summary

输入图片说明

重现步骤

报错信息

+1 2
Attachments
total 2 participants

Comments (1)

dongjinguang 2020-11-12 20:22 member

Close the issue temporarily. it will be open the issue again or create a new issue after the feature having some progress

Sign in to comment

Assignees
Labels
feature
Projects
Milestones
Branches
Planed to start
Not set
Planed to end
Not set
Top level
Priority
1
https://git.oschina.net/openharmony/docs.git
git@git.oschina.net:openharmony/docs.git
openharmony
docs
docs

Search

152606 8668e384 1899542 133635 2cd7d36e 1899542