当前仓库属于暂停状态,部分功能使用受限,详情请查阅 仓库状态说明
4 Star 4 Fork 11

OpenHarmony / distributeddatamgr_datamgr
暂停

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

分布式数据服务组件

简介

分布式数据服务(Distributed Data Service,DDS) 提供不同设备间数据库数据分布式的能力。通过结合帐号、应用和数据库三元组,分布式数据服务对数据进行隔离。在通过可信认证的设备间,分布式数据服务支持数据相互同步,为用户提供在多种终端设备上一致的数据访问体验。

分布式数据服务支撑OpenHarmony系统上分布式管理数据,包含五部分:

  • 服务接口

    分布式数据服务提供专门的数据库创建、数据访问、数据订阅等接口给内部其他部件调用,接口支持KV数据模型,支持常用的数据类型,同时确保接口的兼容性、易用性和可发布性。

  • 服务组件

    服务组件负责服务内元数据管理、权限管理、备份和恢复管理以及多用户管理等、同时负责初始化底层分布式DB的存储组件、同步组件和通信适配层。

  • 存储组件

    存储组件负责数据的访问、数据的缩减、事务、快照,以及数据合并和冲突解决等特性。

  • 同步组件

    同步组件连结了存储组件与通信组件,其目标是保持在线设备间的数据库数据一致性,包括将本地产生的未同步数据同步给其他设备,接收来自其他设备发送过来的数据,并合并到本地设备中。

  • 通信适配层

    通信适配层负责调用底层公共通信层的接口完成通信管道的创建、连接,接收设备上下线消息,维护已连接和断开设备列表的元数据,同时将设备上下线信息发送给上层同步组件,同步组件维护连接的设备列表,同步数据时根据该列表,调用通信适配层的接口将数据封装并发送给连接的设备。

通过调用分布式数据服务接口实现分布式数据库创建、访问、订阅功能,服务接口通过操作服务组件提供的能力,将数据存储至存储组件,存储组件调用同步组件实现将数据同步,同步组件使用通信适配层将数据同步至远端设备,远端设备通过同步组件接收数据,并更新至本端存储组件。

图 1 数据分布式运作示意图

目录

/foundation/distributeddatamgr/datamgr_service
├── interfaces                    # 接口层代码
│   └── innerkits                 # Native接口
│   └── jskits                    # JavaScript接口
├── services                      # 服务层代码
│   └── distributeddataservice    # 分布式数据服务实现
└── test                          # 测试用例资源

约束

  • 如需使用分布式数据服务完整功能,需要申请ohos.permission.DISTRIBUTED_DATASYNC权限。

  • 分布式数据服务的数据模型仅支持KV数据模型,不支持外键、触发器等关系型数据库中的技术点。

  • 分布式数据服务支持的KV数据模型规格:

    • 设备协同数据库,Key最大支持896Byte,Value最大支持4MB。
    • 单版本数据库,Key最大支持1KB,Value最大支持4MB。
    • 每个程序最多支持同时打开16个DB。
  • 由于支持的存储类型不完全相同等原因,分布式数据服务无法完全代替业务沙箱内数据库数据的存储功能,开发人员需要确定要做分布式同步的数据,把这些数据保存到分布式数据服务中。

  • 分布式数据服务当前不支持自定义冲突解决策略。

  • 分布式数据服务当前流控机制针对KvStore的接口1秒最大访问1000次,1分钟最大访问10000次。KvManager的接口1秒最大访问50次,1分钟最大访问500次。

说明

分布式数据服务几个基本概念:

  • KV数据模型

    “KV数据模型”是“Key-Value数据模型”的简称,“Key-Value”即“键-值”。它是一种NoSQL类型数据库,其数据以键值对的形式进行组织、索引和存储。

    KV数据模型适合不涉及过多数据关系和业务关系的业务数据存储,比SQL数据库存储拥有更好的读写性能,同时因在分布式场景中降低了数据库版本兼容和数据同步过程中冲突解决的复杂度而被广泛使用。分布式数据库也是基于KV数据模型,对外提供KV类型的访问接口。

  • 分布式数据库事务性

    分布式数据库事务支持本地事务(和传统数据库的事务概念一致)和同步事务,同步事务是指在设备之间同步数据时,是以本地事务为单位进行同步,一次本地事务的修改要么都同步成功,要么都同步失败。

  • 分布式数据库一致性

    在分布式场景中一般会涉及多个设备,组网内设备之间看到的数据是否一致称为分布式数据库的一致性。分布式数据库一致性可以分为强一致性弱一致性最终一致性

    • 强一致性:是指某一设备成功增、删、改数据后,组网内设备对该数据的读取操作都将得到更新后的值。
    • 弱一致性:是指某一设备成功增、删、改数据后,组网内设备可能能读取到本次更新数据,也可能读取不到,不能保证在多长时间后每个设备的数据一定是一致的。
    • 最终一致性:是指某一设备成功增、删、改数据后,组网内设备可能读取不到本次更新数据,但在某个时间窗口之后组网内设备的数据能够达到一致状态。

    强一致性对分布式数据的管理要求非常高,在服务器的分布式场景可能会遇到。因为移动终端设备的不常在线、以及无中心的特性,分布式数据服务不支持强一致,只支持最终一致性。

  • 分布式数据库同步

    底层通信组件完成设备发现和认证,会通知分布式数据服务设备上线。收到设备上线的消息后分布式数据服务可以在两个设备之间建立加密的数据传输通道,利用该通道在两个设备之间进行数据同步。

    分布式数据服务提供了两种同步模式:手动同步自动同步模式手动同步模式支持指定同步的设备列表和同步模式(PULL、PUSH和PULL_PUSH三种同步模式)。自动同步模式由分布式数据库来完成数据同步(同步时机包括设备上线、修改数据等),业务不感知同步操作。

  • 单版本分布式数据库

    单版本是指数据在本地保存是以单个KV条目为单位的方式保存,对每个Key最多只保存一个条目项,当数据在本地被用户修改时,不管它是否已经被同步出去,均直接在这个条目上进行修改。同步也以此为基础,按照它在本地被写入或更改的顺序将当前最新一次修改逐条同步至远端设备。

  • 设备协同分布式数据库

    设备协同分布式数据库建立在单版本分布式数据库之上,对于存入的KV数据中的Key前面拼接了本设备的DeviceID标识符,这样能保证每个设备产生的数据严格隔离,底层按照设备的维度管理这些数据,设备协同分布式数据库支持以设备的维度查询分布式数据,但是不支持修改远端设备同步过来的数据。

  • 分布式数据库冲突解决策略

    分布式数据库多设备提交冲突场景,在给提交冲突做合并的过程中,如果多个设备同时修改了同一数据,则称这种场景为数据冲突。数据冲突采用默认冲突解决策略,基于提交时间戳,取时间戳较大的提交数据,当前不支持定制冲突解决策略。

  • 数据库Schema化管理与谓词查询

    单版本数据库支持在创建和打开数据库时指定Schema,数据库根据Schema定义感知KV记录的Value格式,以实现对Value值结构的检查,并基于Value中的字段实现索引建立和支持谓词查询。

  • 分布式数据库备份能力

    提供分布式数据库备份能力,业务通过设置backup属性为true,可以触发分布式数据服务每日备份。当分布式数据库发生损坏,分布式数据服务会删除损坏数据库,并且从备份数据库中恢复上次备份的数据。如果不存在备份数据库,则创建一个新的数据库。同时支持加密数据库的备份能力。

相关仓

分布式数据管理子系统

distributeddatamgr_datamgr

third_party_sqlite

Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: (a) You must give any other recipients of the Work or Derivative Works a copy of this License; and (b) You must cause any modified files to carry prominent notices stating that You changed the files; and (c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and (d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS

简介

Distributed data manager that provides the capability to store data in the databases of different devices | 分布式数据管理服务模块,提供多设备数据分布式存储能力 展开 收起
C++ 等 2 种语言
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
1
https://gitee.com/openharmony/distributeddatamgr_datamgr.git
git@gitee.com:openharmony/distributeddatamgr_datamgr.git
openharmony
distributeddatamgr_datamgr
distributeddatamgr_datamgr
master

搜索帮助