加入 Gitee

与超过 1200万开发者一起发现、参与优秀开源项目，私有仓库也完全免费：）

克隆/下载

README_EN.md 4.01 KB

# LarkMidTable

[![License](https://img.shields.io/badge/license-Apache%202-4EB1BA.svg)](https://www.apache.org/licenses/LICENSE-2.0.html)

English | [中文](README.md)

Lark Chinese name Skylark,  sky stands for big data, and bird stands for ordinary and freedom.

LarkMidTable is a one-stop open source data center that realizes metadata management, data warehouse development, data quality management, data visualization, and realizes a product that efficiently empowers the data front desk and provides data services.

# **Product vision**

1. Meet many small businesses and provide one-stop solutions.

2. Make world-class products comparable to Google、Microsoft、Apple manufacturers.

3. Create value, generate value, and make the world a better place.

# Technology selection

| Frame name                                                   | Frame use                                 | The main function                                            |
| ------------------------------------------------------------ | ----------------------------------------- | ------------------------------------------------------------ |
| [Atlas](http://atlas.apache.org/)                            | Metadata management                       | Core capabilities of metadata governance including data classification, centralized policy engine, data blood relationship, security and lifecycle management |
| [Dolphin](https://github.com/apache/incubator-dolphinscheduler) | Task scheduling                           | Visual DAG workflow task scheduling system                   |
| [Flink](https://github.com/apache/flink)                     | Offline and real-time computing framework | Based on Flink, one-stop solution to batch processing problems |
| [Hive](https://github.com/apache/hive)                       | data storage                              | MR-based data warehouse tools                                |
| [Kylin](https://github.com/apache/kylin)                     | Analyze the database                      | Open source, distributed analytical data warehouse           |
| [Kafka](https://github.com/apache/kafka)                     | Message middleware                        | LinkedIn is implemented in Scala language and supports parallel loading of hadoop data |
| [K8S](https://github.com/kubernetes/kubernetes)              | Container deployment                      | Deploying containerized applications is simple and efficient |
| [Zookeeper](https://github.com/apache/zookeeper)             | Distributed coordination service          | Unified naming service, configuration management, cluster management, queue management |

# Product architecture diagram

| Applicable industry | E-commerce field    | The financial sector  | Communication field | Industrial field        | ...  |
| ------------------- | ------------------- | --------------------- | ------------------- | ----------------------- | ---- |
| Data Center         | database            | Data quality          | Metadata management | Real-time report        | ...  |
| Data platform       | Offline calculation | Real-time calculation | Business algorithm  | Artificial intelligence | ...  |
| Data collection     | Buried log          | Alarm log             | IoT data            | Clickstream data        | ...  |

# **Quick Start**

Please click [Quick Start](https://github.com/wxgzgl/LarkMidTable/blob/master/userGuid.md)

Resource Library [R&D Resource Library]( https://github.com/wxgzgl/LarkMidTable/blob/master/docs/list.md)

Development Specification [Vipshop Development Specification](https://vipshop.github.io/vjtools/#/standard/)

# **Development Plan**

To be completed:

Task management, log management, resource monitoring

completed:

Login function [Complete] Project management [Complete] User management [Complete] JSON formatting [Complete]

Actuator management [Complete] Linux installation and deployment [Complete] Data source management [Complete]

[Development Log](https://github.com/wxgzgl/Lark/tree/master/docs/notes/202009.md)

# **Technology Exchange**

一键复制编辑原始数据按行查看历史

提交于 2021-10-14 22:55 . update

LarkMidTable

English | 中文

Lark Chinese name Skylark, sky stands for big data, and bird stands for ordinary and freedom.

LarkMidTable is a one-stop open source data center that realizes metadata management, data warehouse development, data quality management, data visualization, and realizes a product that efficiently empowers the data front desk and provides data services.

Product vision

Meet many small businesses and provide one-stop solutions.
Make world-class products comparable to Google、Microsoft、Apple manufacturers.
Create value, generate value, and make the world a better place.

Technology selection

Frame name	Frame use	The main function
Atlas	Metadata management	Core capabilities of metadata governance including data classification, centralized policy engine, data blood relationship, security and lifecycle management
Dolphin	Task scheduling	Visual DAG workflow task scheduling system
Flink	Offline and real-time computing framework	Based on Flink, one-stop solution to batch processing problems
Hive	data storage	MR-based data warehouse tools
Kylin	Analyze the database	Open source, distributed analytical data warehouse
Kafka	Message middleware	LinkedIn is implemented in Scala language and supports parallel loading of hadoop data
K8S	Container deployment	Deploying containerized applications is simple and efficient
Zookeeper	Distributed coordination service	Unified naming service, configuration management, cluster management, queue management

Product architecture diagram

Applicable industry	E-commerce field	The financial sector	Communication field	Industrial field	...
Data Center	database	Data quality	Metadata management	Real-time report	...
Data platform	Offline calculation	Real-time calculation	Business algorithm	Artificial intelligence	...
Data collection	Buried log	Alarm log	IoT data	Clickstream data	...

Quick Start

Please click Quick Start

Resource Library R&D Resource Library

Development Specification Vipshop Development Specification

Development Plan

To be completed:

Task management, log management, resource monitoring

completed:

Login function [Complete] Project management [Complete] User management [Complete] JSON formatting [Complete]

Actuator management [Complete] Linux installation and deployment [Complete] Data source management [Complete]

Development Log

Technology Exchange

1

https://gitee.com/jiudu800/LarkMidTable.git

git@gitee.com:jiudu800/LarkMidTable.git

jiudu800

LarkMidTable

LarkMidTable

master