3 Star 32 Fork 4

哒哒哒耳文 / squirrel

Create your Gitee Account
Explore and code with more than 12 million developers,Free private repositories !:)
Sign up
This repository doesn't specify license. Please pay attention to the specific project description and its upstream code dependency when using it.
Clone or Download
contribute
Sync branch
Cancel
Notice: Creating folder will generate an empty file .keep, because not support in Git
Loading...
README

squirrel

介绍

squirrel(居哩猫)是一款基于Java语言开发,借助webmagic框架实现的聚合科技资讯爬虫项目,享受24H科技新闻速览。

技术交流学习使用,请勿用于任何商业用途!!!

软件架构

  1. webmagic爬虫核心框架
  2. quartz任务调度框架

目前完成的数据源

  1. 36Kr
  2. iyiou
  3. NewSeed
  4. Jiemian
  5. PingWest
  6. ChinaVenture
  7. Pedaily
  8. tmtpost
  9. technode
  10. qubi8
  11. itjuzu
  12. hexun-tech

以上数据源作为参考,内容是一个不断修正的过程。

效果演示

enter image description here

enter image description here

技术交流

微信号:oddubbo

e-mail:zscao@daervin.com

改进记录

2019-12-08 优化重复标题计算逻辑

2019-12-10 增加阻塞队列,数据顺序落地,从而达到多源展示的效果

2019-12-12 优化部分数据源,新增两个数据源

Empty file

About

squirrel(聚哩猫)是一款基于Java语言开发,借助webmagic框架实现的聚合科技资讯爬虫项目,享受24H科技新闻速览。 expand collapse
Cancel

Releases

No release

Contributors

All

Activities

Load More
can not load any more
Java
1
https://gitee.com/zscao/squirrel.git
git@gitee.com:zscao/squirrel.git
zscao
squirrel
squirrel
master

Search