1 Star 0 Fork 0

BeyondStorage / beyond-tp

Create your Gitee Account
Explore and code with more than 6 million developers,Free private repositories !:)
Sign up
Clone or Download
Notice: Creating folder will generate an empty file .keep, because not support in Git


go version license unit test status beyond-tp dev

beyond-tp is a data migration service.

This project is now under high development, please do not apply to production environment

Contributor Guide

You are always welcome to act as a contributor to beyond-tp.

Since beyond-tp is an application under BeyondStorage, you should follow the contributor guide of the community.

Please feel free to submit issues, create a pull request, or just star this repo :)

For more details, please refer to: https://beyondstorage.io/docs/general/contributor-guide

Development Requirement


Beyond-tp uses protobuf for RPC call and data serialization. So protobuf tools are needed when generate code from .proto files, related components are listed below:


protoc is the Protocol Compiler. To install protoc, you can take Protocol Compiler Installation as a reference.

protoc-gen-go & protoc-gen-go-grpc

protoc-gen-go and protoc-gen-go-grpc are two plugins used to generate Go code from .proto files.

Install them using Go 1.16 or higher by running:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

This will install protoc-gen-go and protoc-gen-go-grpc binary files in $GOBIN. Set the $GOBIN environment variable to change the installation location. It must be in your $PATH for the protocol buffer compiler to find it.

You can also specify a version number by replacing latest after @.

Since we develop beyond-tp with:

➜  ~ protoc --version
libprotoc 3.15.7
➜  ~ protoc-gen-go --version
protoc-gen-go v1.25.0
➜  ~ protoc-gen-go-grpc --version
protoc-gen-go-grpc 1.1.0

So we recommend you to use these tools equal or higher than this.


Beyond-tp uses flutter to conduct frontend web pages. v2.0.1 or higher version is recommended for development.

To install flutter, you can take flutter installation as a reference.

Notice: For some network reason, you can also get help from https://flutter.cn/community/china.


Build Binary

make build

Run make build to build the binary file used to start a beyond-tp server or staff.

Build Frontend Pages

make build-frontend

Run make build-frontend to build frontend pages.

Notice that beyond-tp uses flutter to build web pages, so please make sure flutter is well installed, and added in $PATH directories.

For more details, please run make help as a reference.

Generate Code

make generate

Run make generate to generate the latest code by GraphQL schema and grpc .proto files.

Notice that beyond-tp uses protobuf tools to generate grpc code, so please make sure protobuf is well installed, and added in $PATH directories for the protocol buffer compiler to find it.


Start a Server

Server used as a web server as well as a task manager. It allows you to manage beyond-tp visually, and support a rpc server to communicate with Staffs. You can easily start a beyond-tp server by:

beyondtp server --db /path/to/db --host localhost --port 7436 --rpc-port 7000

This command will start a beyond-tp server, including a web server, with local db indicated by db flag. You can visit the home page at localhost:7436/ui in your web browser.

Run beyondtp server --help for more examples and flags' usage.

Start a Staff

Staff used as a task executor. It can run a migration task, distributed by Server. You can easily start a beyond-tp staff by:

beyondtp staff --host localhost --manager localhost:7000

This command will start a beyond-tp staff connected to the manager localhost:7000. The manager's host and port are specified in server command by flag host and rpc-port.

Run beyondtp staff --help for more examples and flags' usage.

What's more

We support to use environment variable to replace flags. All the flags follow these rules:

  • All flags correspond to environment variable with prefix BEYONDTP_
  • Kebab-case (log-level) to Snake_case (LOG_LEVEL) with upper case
  • Flags for specific commands should add command prefix, e.g. BEYONDTP_SERVER_PORT and BEYONDTP_STAFF_PORT for --port flag under server command and staff command
  • Global flags do not need to add command prefix, e.g. BEYONDTP_LOG_LEVEL

If the same flag set by both environment variable and command flag, command flag will be higher priority than environment variable, for example:

BEYONDTP_LOG_LEVEL=debug bin/beyondtp server --db badger --log-level=warn

The final log level would be warn instead of debug.

For more command usage, please run beyondtp --help as a reference.

Useful links

Repository Comments ( 0 )

Sign in to post a comment


Neutral data migration service expand collapse
Dart and 6 more languages


No release





Load More
can not load any more