Sign in Sign up
Explore Enterprise Education Gitee Premium Gitee AI
Scan WeChat QR to Pay
Cancel
Complete
Watch
Unwatch Watching Releases Only Ignoring
1 Star 1 Fork 2

杜森 / ambari-deploy

Code Issues 0 Pull Requests 0 Wiki Insights Pipelines
Service
Create your Gitee Account
Explore and code with more than 12 million developers，Free private repositories ！：）
Sign up
Clone or Download
README.md 13.91 KB
AI 代码解读
Copy Edit Raw Blame History
dusen authored 2023-09-02 14:35 . [IMP] desensitization

Ambari && HDP 离线自动化安装

先在一台机器安装 Ambari，然后通过 Ambari Web 界面安装 HDP。

  • Ambari 版本: 2.7.6.0-22
  • HDP 版本: 3.3.2.0-002
  • MySQL 版本: 8.0.33
  • Java 版本: 1.8
  • 系统版本: Centos 7* / Rocky 8*

前提

  1. 从公司云盘下载软件包 ambari-parcels.20230831.tar.gz 到脚本执行机器中。
wget -O /opt/ambari-parcels.20230831.tar.gz http://127.0.0.1:80/owncloud/index.php/s/TSgLjsyB54ciyzh/download
  1. 把压缩包解压到 /var/www/html 目录下
mkdir -p /var/www/html
tar -zxvf /opt/ambari-parcels.20230831.tar.gz -C /var/www/html/

一、Ambari 安装

0. 配置环境变量

  • 需要手动补充该文件中的配置项
  • ./00_env

1. 配置集群间ssh免密

2. 配置所有节点的 hosts

3. 初始化系统环境

4. 安装/配置 httpd

5. 安装/配置 java

6. 安装/配置 chrony

7. 安装/配置 mysql

8. 安装/配置 ambari agent

09. 安/配置装 ambari server

10. 配置 hdp

二、HDP 安装

  • hdp-01
  • hdp-02
  • hdp-03
  • hdp-04
  • hdp-05
  • hdp-06
  • hdp-07
  • hdp-08
  • hdp-09
  • hdp-10

三、其它

  1. hue 缺少 MySQL-python
yum install -y mysql-devel

source /usr/hdp/current/hue/build/env/bin/activate
pip uninstall mysqlclient
pip uninstall mysql-python
pip install mysql-python
  1. ambari-parcels.6.3.1466458.tar.gz
/var/www/html/ambari-parcels
|-- ambari
|   `-- 2.7.6.0-22
|       |-- ambari
|       |   |-- ambari-agent-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-infra-manager-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-infra-solr-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-infra-solr-client-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-logsearch-logfeeder-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-logsearch-portal-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-metrics-collector-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-metrics-common-2.7.6.0-22.noarch.rpm
|       |   |-- ambari-metrics-grafana-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-metrics-hadoop-sink-2.7.6.0-22.x86_64.rpm
|       |   |-- ambari-metrics-monitor-2.7.6.0-22.x86_64.rpm
|       |   `-- ambari-server-2.7.6.0-22.x86_64.rpm
|       |-- ambari.repo
|       |-- repodata
|       |   |-- 4d38958a349ed790a806167fe455b1089afb9b7cbbde38d3c1f741a3f3df7ac1-filelists.sqlite.bz2
|       |   |-- 4ecfcec536aca6847e6f566e349f45305e3472062ad9cf2073d443db7104af42-filelists.xml.gz
|       |   |-- 80fa976a237c77bace278a8bdf25c6e06593f0d15116f7da87bd52f006f1d810-other.sqlite.bz2
|       |   |-- aa7c6efcb599f94e67d59a3d314af171db7468f9c2d1530ca0fcbdf62438a561-primary.sqlite.bz2
|       |   |-- bfbeb4a9134eb86fdaf4271e7c9e9924d8ff7beee102501afb9ec74318c42662-primary.xml.gz
|       |   |-- f6bb518f714608643604ece4a63019f317592be9bfd838f414ca73d1c3439f99-other.xml.gz
|       |   `-- repomd.xml
|       `-- RPM-GPG-KEY
|           `-- RPM-GPG-KEY-Jenkins
|-- hdp
|   `-- 3.3.2.0-002
|       |-- atlas-metadata
|       |   |-- atlas-metadata_3_3_2_0_002-2.3.0.3.3.2.0-002.x86_64.rpm
|       |   |-- atlas-metadata_3_3_2_0_002-hbase-plugin-2.3.0.3.3.2.0-002.x86_64.rpm
|       |   |-- atlas-metadata_3_3_2_0_002-hive-plugin-2.3.0.3.3.2.0-002.x86_64.rpm
|       |   |-- atlas-metadata_3_3_2_0_002-kafka-plugin-2.3.0.3.3.2.0-002.x86_64.rpm
|       |   |-- atlas-metadata_3_3_2_0_002-sqoop-plugin-2.3.0.3.3.2.0-002.x86_64.rpm
|       |   `-- atlas-metadata_3_3_2_0_002-storm-plugin-2.3.0.3.3.2.0-002.x86_64.rpm
|       |-- bigtop-jsvc
|       |   `-- bigtop-jsvc-1.0.15-002.x86_64.rpm
|       |-- dolphinscheduler
|       |   `-- dolphinscheduler_3_3_2_0_002-3.1.7.3.3.2.0-002.x86_64.rpm
|       |-- doris
|       |   |-- doris_3_3_2_0_002-be-1.2.6.3.3.2.0-002.x86_64.rpm
|       |   |-- doris_3_3_2_0_002-client-1.2.6.3.3.2.0-002.x86_64.rpm
|       |   |-- doris_3_3_2_0_002-fe-1.2.6.3.3.2.0-002.x86_64.rpm
|       |   `-- doris_3_3_2_0_002-hdfs_broker-1.2.6.3.3.2.0-002.x86_64.rpm
|       |-- flink
|       |   `-- flink_3_3_2_0_002-1.16.2.3.3.2.0-002.x86_64.rpm
|       |-- hadoop
|       |   |-- hadoop_3_3_2_0_002-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-client-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-datanode-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-fuse-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-journalnode-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-namenode-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-secondarynamenode-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-hdfs-zkfc-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-httpfs-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-httpfs-server-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-libhdfs-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-mapreduce-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-mapreduce-historyserver-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-nodemanager-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-proxyserver-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-registrydns-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-resourcemanager-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   |-- hadoop_3_3_2_0_002-yarn-timelinereader-3.3.6.3.3.2.0-002.x86_64.rpm
|       |   `-- hadoop_3_3_2_0_002-yarn-timelineserver-3.3.6.3.3.2.0-002.x86_64.rpm
|       |-- hbase
|       |   |-- hbase_3_3_2_0_002-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   |-- hbase_3_3_2_0_002-doc-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   |-- hbase_3_3_2_0_002-master-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   |-- hbase_3_3_2_0_002-regionserver-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   |-- hbase_3_3_2_0_002-rest-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   |-- hbase_3_3_2_0_002-thrift2-2.5.5.3.3.2.0-002.x86_64.rpm
|       |   `-- hbase_3_3_2_0_002-thrift-2.5.5.3.3.2.0-002.x86_64.rpm
|       |-- HDP-3.3.2.0-001-PATCH.xml
|       |-- HDP-3.3.2.0-001.xml
|       |-- hdp.repo
|       |-- hdp-select
|       |   `-- hdp-select-3.3.2.0-002.x86_64.rpm
|       |-- hive
|       |   |-- hive_3_3_2_0_002-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-hcatalog-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-hcatalog-server-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-jdbc-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-metastore-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-server2-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-server-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   |-- hive_3_3_2_0_002-webhcat-3.1.3.3.3.2.0-002.x86_64.rpm
|       |   `-- hive_3_3_2_0_002-webhcat-server-3.1.3.3.3.2.0-002.x86_64.rpm
|       |-- hue
|       |   `-- hue_3_3_2_0_002-4.11.0.3.3.2.0-002.x86_64.rpm
|       |-- impala
|       |   |-- impala_3_3_2_0_002-4.1.2.3.3.2.0-002.x86_64.rpm
|       |   `-- impala_3_3_2_0_002-shell-4.1.2.3.3.2.0-002.x86_64.rpm
|       |-- kafka
|       |   `-- kafka_3_3_2_0_002-3.5.1.3.3.2.0-002.x86_64.rpm
|       |-- knox
|       |   `-- knox_3_3_2_0_002-2.0.0.3.3.2.0-002.x86_64.rpm
|       |-- kudu
|       |   `-- kudu_3_3_2_0_002-1.16.0.3.3.2.0-002.x86_64.rpm
|       |-- kyuubi
|       |   `-- kyuubi_3_3_2_0_002-1.7.1.3.3.2.0-002.x86_64.rpm
|       |-- livy2
|       |   `-- livy2_3_3_2_0_002-0.8.0.3.3.2.0-002.x86_64.rpm
|       |-- ozone
|       |   `-- ozone_3_3_2_0_002-1.3.0.3.3.2.0-002.x86_64.rpm
|       |-- phoenix
|       |   `-- phoenix_3_3_2_0_002-5.1.3.3.3.2.0-002.x86_64.rpm
|       |-- ranger
|       |   |-- ranger_3_3_2_0_002-admin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-atlas-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-hbase-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-hdfs-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-hive-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-kafka-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-kms-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-knox-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-ozone-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-solr-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-spark3-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-storm-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-tagsync-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   |-- ranger_3_3_2_0_002-usersync-2.4.0.3.3.2.0-002.x86_64.rpm
|       |   `-- ranger_3_3_2_0_002-yarn-plugin-2.4.0.3.3.2.0-002.x86_64.rpm
|       |-- repodata
|       |   |-- 280033bd2242c3aa32363393e9dfd496891fbb93f5bf3f45eb6ad14901af29f0-other.xml.gz
|       |   |-- 3558b8d59c8d6fb67b92b00a40bbd5d42b190d09096a938501ebf2af0313f2c3-primary.xml.gz
|       |   |-- 470e1bdcb9ce5e9012903395f2676c5e840e490e2de087718c0b0c0798307e1a-other.sqlite.bz2
|       |   |-- 9e4bb5de077f681cf6bab57cdb29b1a824d2095ef166859f45143fa220a11fd6-filelists.xml.gz
|       |   |-- a5dc22675309b04a2759313d03eff6f633a39112b87ecf49d0e99b48f7ba9188-filelists.sqlite.bz2
|       |   |-- bf1bdfd71cad4d81e589e8d100da1268ffb8f6b9c489a4cffe8f405719e74ceb-primary.sqlite.bz2
|       |   `-- repomd.xml
|       |-- seatunnel
|       |   `-- seatunnel_3_3_2_0_002-2.3.2.3.3.2.0-002.x86_64.rpm
|       |-- spark2
|       |   |-- spark2_3_3_2_0_002-2.4.8.3.3.2.0-002.x86_64.rpm
|       |   |-- spark2_3_3_2_0_002-master-2.4.8.3.3.2.0-002.x86_64.rpm
|       |   |-- spark2_3_3_2_0_002-python-2.4.8.3.3.2.0-002.x86_64.rpm
|       |   |-- spark2_3_3_2_0_002-worker-2.4.8.3.3.2.0-002.x86_64.rpm
|       |   `-- spark2_3_3_2_0_002-yarn-shuffle-2.4.8.3.3.2.0-002.x86_64.rpm
|       |-- spark3
|       |   |-- spark3_3_3_2_0_002-3.3.2.3.3.2.0-002.x86_64.rpm
|       |   |-- spark3_3_3_2_0_002-master-3.3.2.3.3.2.0-002.x86_64.rpm
|       |   |-- spark3_3_3_2_0_002-python-3.3.2.3.3.2.0-002.x86_64.rpm
|       |   |-- spark3_3_3_2_0_002-worker-3.3.2.3.3.2.0-002.x86_64.rpm
|       |   `-- spark3_3_3_2_0_002-yarn-shuffle-3.3.2.3.3.2.0-002.x86_64.rpm
|       |-- spark-atlas-connector
|       |   `-- spark-atlas-connector_3_3_2_0_002-0.1.0.3.3.2.0-002.x86_64.rpm
|       |-- sqoop
|       |   |-- sqoop_3_3_2_0_002-1.4.7.3.3.2.0-002.x86_64.rpm
|       |   `-- sqoop_3_3_2_0_002-metastore-1.4.7.3.3.2.0-002.x86_64.rpm
|       |-- tez
|       |   `-- tez_3_3_2_0_002-0.10.2.3.3.2.0-002.x86_64.rpm
|       `-- zookeeper
|           |-- zookeeper_3_3_2_0_002-3.8.1.3.3.2.0-002.x86_64.rpm
|           `-- zookeeper_3_3_2_0_002-server-3.8.1.3.3.2.0-002.x86_64.rpm
|-- hdp-utils
|   `-- 1.1.0.22
|       |-- hdp-utils.repo
|       |-- openblas
|       |   |-- openblas-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-devel-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-openmp-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-openmp64-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-openmp64_-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-Rblas-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-serial64-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-serial64_-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-static-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-threads-0.2.19-4.el7.x86_64.rpm
|       |   |-- openblas-threads64-0.2.19-4.el7.x86_64.rpm
|       |   `-- openblas-threads64_-0.2.19-4.el7.x86_64.rpm
|       |-- repodata
|       |   |-- 03caa5dc00c5e38c9d3a33abe86589986f1efb825d982b73dd377889fc70ae2b-filelists.sqlite.bz2
|       |   |-- 392f74ff6f3c9e0e8ef0b933efd5f9d5b08977dd3396772336ee7d71051d460a-filelists.xml.gz
|       |   |-- 395136b97d8b1d5ec9cc5c77ff7de68568d3865675840b3305f6aa42796ac663-other.sqlite.bz2
|       |   |-- 5e1eca162a63e4ea0ecf2bf7cdc394ad97f8ec9966810f18b764e98c3f8bc6e0-primary.xml.gz
|       |   |-- b129db4961099c4fee5efb06e88f9918f2087e524b14de83925efc376027fe36-primary.sqlite.bz2
|       |   |-- b4f6be336c4deaf2eddd958c6dab1d975d6b0274f2b336e58f9075190cc09a5f-other.xml.gz
|       |   `-- repomd.xml
|       |-- RPM-GPG-KEY
|       |   `-- RPM-GPG-KEY-Jenkins
|       `-- snappy
|           |-- snappy-1.1.0-3.el7.i686.rpm
|           |-- snappy-1.1.0-3.el7.x86_64.rpm
|           |-- snappy-devel-1.1.0-3.el7.i686.rpm
|           `-- snappy-devel-1.1.0-3.el7.x86_64.rpm
`-- others
    |-- jdk-8u202-linux-x64.tar.gz
    |-- mysql-8.0-el7-bundle-rpms.tar.gz
    `-- mysql-8.0-el8-bundle-rpms.tar.gz

41 directories, 152 files

四、Refs

  1. HDP3.3.1.0-004版本: https://mp.weixin.qq.com/s/FrktFj2qgjsldxHlRgszDg
  2. Ambari-2.7.6和HDP-3.3.1安装: http://mp.weixin.qq.com/s?__biz=MzkwODM4NTc4Mw==&mid=2247483799&idx=2&sn=0152829912f367e40aff1d9867ae8f19&chksm=c0cb8d0ff7bc0419bad2649b7d682b43acf0e86e19bb064e92fbda46412ff309a1e8ae7294c2&scene=21#wechat_redirect
  3. HDP3.3.1 安装指南（1）: http://mp.weixin.qq.com/s?__biz=MzkwODM4NTc4Mw==&mid=2247483867&idx=1&sn=0270ea5e84002fa42fa5f541380808f3&chksm=c0cb8d43f7bc045503870fbe3114661924f3954c219cc3f018ce7c12af6f21c856efd7f55097&scene=21#wechat_redirect
  4. Centos7/8 RPMS: https://centos.pkgs.org/
1
https://gitee.com/ddusen/ambari-deploy.git
git@gitee.com:ddusen/ambari-deploy.git
ddusen
ambari-deploy
ambari-deploy
master
Going to Help Center

Search

Git 命令在线学习 如何在 Gitee 导入 GitHub 仓库
Git 仓库基础操作
企业版和社区版功能对比
SSH 公钥设置
如何处理代码冲突
仓库体积过大，如何减小？
如何找回被删除的仓库数据
Gitee 产品配额说明
GitHub仓库快速导入Gitee及同步更新
什么是 Release（发行版）
将 PHP 项目自动发布到 packagist.org
Comment
Repository Report
Back to the top