Inviso is a lightweight tool that provides the ability to search for Hadoop jobs, visualize the performance, and view cluster utilization.
REST API for Job History: REST endpoint to load an entire job history file as a json object.
ElasticSearch: Search over jobs and correlate Hadoop jobs for Pig and Hive scripts.
Python Scripts: Scripts to index job configurations into ElasticSearch for querying. These scripts can accommodate a pub/sub model for use with SQS or some other queuing service to better distribute the load or allow other systems to know about job events.
Web UI: Provides an interface to serach and visualize jobs and cluster data.
Inviso is easy to setup given a Hadoop cluster. To get a quick preview, it is easiest to configure Inviso on the NameNode/ResourceManager host.
> wget http://<mirror>/.../apache-tomcat-7.0.55.tar.gz
> tar -xzf apache-tomcat-7.0.55.tar.gz
> rm -r apache-tomcat-7.0.55/webapps/*
> wget http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.tar.gz
> tar -xzf elasticsearch-1.3.2.tar.gz
> git clone https://github.com/Netflix/inviso.git
> cd inviso
> ./gradlew assemble
> cd ..
> cp inviso/trace-mr2/build/libs/inviso#mr2#v0.war apache-tomcat-7.0.55/webapps/
> ln -s `pwd`/inviso/web-ui/public apache-tomcat-7.0.55/webapps/ROOT
> ./elasticsearch-1.3.2/bin/elasticsearch -d
> curl -XPUT http://localhost:9200/inviso -d @inviso/elasticsearch/mappings/config-settings.json
{"acknowledged":true}
> curl -XPUT http://localhost:9200/inviso-cluster -d @inviso/elasticsearch/mappings/cluster-settings.json
{"acknowledged":true}
> ./apache-tomcat-7.0.55/bin/startup.sh
> virtualenv venv
> source venv/bin/activate
> pip install -r inviso/jes/requirements.txt
> cd inviso/jes/
> cp settings_default.py settings.py
> python jes.py
> python index_cluster_stats.py
#Run in a cron or loop
> while true; do sleep 60s; python jes.py; done&
> while true; do sleep 60s; python index_cluster_stats.py; done&
An alternate way of starting the inviso project would be via docker. If you already have docker installed, you can run the following command:
docker run -d -p 8080:8080 savaki/inviso
This will launch inviso in your container running on port 8080.
Enjoy!
此处可能存在不合适展示的内容,页面不予展示。您可通过相关编辑功能自查并修改。
如您确认内容无涉及 不当用语 / 纯广告导流 / 暴力 / 低俗色情 / 侵权 / 盗版 / 虚假 / 无价值内容或违法国家有关法律法规的内容,可点击提交进行申诉,我们将尽快为您处理。