22 Star 59 Fork 15

MarkHsiu / MiniMQ

加入 Gitee
与超过 1200万 开发者一起发现、参与优秀开源项目,私有仓库也完全免费 :)
免费加入
克隆/下载
贡献代码
同步代码
取消
提示: 由于 Git 不支持空文件夾,创建文件夹后会生成空的 .keep 文件
Loading...
README
Apache-2.0

#MiniMQ 目前项目是起步阶段,是自己学习netty的延伸出来。因为写这个项目,又发现了一个有趣好玩的通信框架talent-aio,这个在真的很快,用起来很爽,源码很值得看,也许将来完善后,可以替代netty,最重要是中国人写的,必须支持,up up up。近期看了阿里新出来的 《阿里巴巴Java开发手册》,感觉蛮好的 ,可以作为项目的编码标准。 设计原则参考dubbo作者梁飞的一篇分享《架构设计原则》。

  • 第一阶段主要学习一些别人的轮子MQ,看看简单的MQ设计思路,多方借鉴好的结构
     >>> https://github.com/tang-jie/AvatarMQ.git    
         	作者有 [博客园](http://www.cnblogs.com/jietang/) 的专栏专门讲解 ,是自己学习其他MQ产品练习出来的项目。代码不单单是来看的,也是来写的。
     >>> https://github.com/uncodecn/uncode-mq.git    
         	作者oschina [blog](https://my.oschina.net/uncode),设计还算完整,但是性能部分很多没有考虑,比如序列化部分还是使用jdk的io。  
     >>> https://git.oschina.net/xuxueli0323/xxl-mq   
        	作者 [blog](http://www.cnblogs.com/xuxueli/p/4918535.html)   
     >>> https://git.oschina.net/blackbeans/kiteq
        	go语言版本 
     >>> https://git.oschina.net/itteam/sf-mq
        	被坑,难怪看着设计这么复杂,原来是阿里MQ的3.x版本 
     >>> https://git.oschina.net/rushmore/zbus    
        	比较活跃,也是比较完整可靠的非主流MQ实现 
     >>> https://github.com/adyliu/jafka  
        	Kafka的克隆版本,由原来的scala换成java实现 
     >>> https://github.com/fusesource/mqtt-client  
        	mqtt,作为以为高效客户端连接的参考 
     >>> https://github.com/killme2008/Metamorphosis/
     		rocketmq的早期版本,Meta.任何好的框架都是由简入繁,学习早期的设计,反而更有利于理解它的进化
  • 第二阶段阅读和参看主流java类的MQ框架,比如RocketMQ,ActiveMQ.以阿里的RocketMQ为主,也会参看 Kafka(scala),RabbitMQ(Erlang),ZeroMQ(C/C++)
   >>> https://github.com/apache/incubator-rocketmq 
   >>> https://github.com/apache/activemq
  • 第三阶会重写这个项目部分依赖的框架,作为下一步的学习

#模块划分

  • Broker 简单来说就是消息队列服务器实体。
    • factory 消费者管理等
    • server 提供客户端连接
    • task 消息推送
    • store 消息存储管理
    • topic:路由关键字 Routing Key,
  • core 工具类,和常量配置
  • Message 消息实体,客户端和服务端传递信息的载体
  • client 连接服务端
    • Producer:消息的生产者,主要用来发送消息给消费者。
    • Consumer:消息的消费者, 主要用来接收生产者的消息。 主要用来控制生产者和消费者之间的发送与接收消息的对应关系。
  • Channel:消息通道 在客户端的每个连接里,可建立多个channel,每个channel代表一个会话任务。
  • Serialize 序列化接口和实现

#zookeeper数据结构

  • broker启动后注册到zookeeper,通知cosumer、producer更新broker信息
  • cosumer启动后,获取所有的broker信息,尝试连接包含topic的broker
  • producer启动, 获取所有的broker信息,
  • broker等待producer发送消息,注册接受的topic到对应的brokerID下,并通知cosumer更新broker信息
Borker
/minimq/broker/${brokerID}(ip:port)/${topic}

#开发工具

  • eclipse 4.5+,IDEA也可以
  • maven3.x
  • git
  • JDK1.8+

#编码原则

  • 尽量用单例,类不要有可以存储数据的属性(数据传输对象除外)
  • 建议使用ConcurrentHashMap 来数据内存数据,并估算容器大小
  • 面向接口编程,序列化、通信等模块通过注入实现接口调用
  • 日志输出不能使用System.out ,使用log4j并严格控制日志数据级别,最好使用xml配置异步处理,性能影响太大

#任务说明

  • 默认为protostuff 序列化方式,支持java序列化
  • netty作为默认通信方式 ,扩展talent-aio通信方式
  • 参考dubbo,用spring作为bean管理和功能扩展
  • 存储方案 msyql 或是 redis 或是 磁盘
  • 集群管理 zookeeper
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document. "Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License. "Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity. "You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License. "Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files. "Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types. "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below). "Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof. "Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution." "Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work. 2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form. 3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed. 4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions: You must give any other recipients of the Work or Derivative Works a copy of this License; and You must cause any modified files to carry prominent notices stating that You changed the files; and You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License. You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions. 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file. 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License. 8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages. 9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability. END OF TERMS AND CONDITIONS APPENDIX: How to apply the Apache License to your work To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "{}" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives. Copyright 2017 MarkHsiu Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

简介

微型消息服务中间件框架,正在进化中。。。 展开 收起
Java
Apache-2.0
取消

发行版

暂无发行版

贡献者

全部

近期动态

加载更多
不能加载更多了
Java
1
https://gitee.com/markhsiu/MiniMQ.git
git@gitee.com:markhsiu/MiniMQ.git
markhsiu
MiniMQ
MiniMQ
master

搜索帮助