https://www.809508860.xyz/tags/tez/ Recent content in Tez on Colben Notes Hugo zh-cn Tue, 03 Feb 2026 11:00:00 +0800 https://www.809508860.xyz/post/hdp3.3/ Fri, 28 Jul 2023 14:00:00 +0800 https://www.809508860.xyz/post/hdp3.3/ <h2 id="环境">环境</h2> <table> <thead> <tr> <th>主机名</th> <th>地址</th> <th>数据目录</th> <th>组件</th> </tr> </thead> <tbody> <tr> <td>hdp-nn</td> <td>192.168.8.1/24</td> <td>/data/hdp-nn</td> <td>Namenode Spark</td> </tr> <tr> <td>hdp-snn</td> <td>192.168.8.2/24</td> <td>/data/hdp-snn</td> <td>SecondaryNamenode</td> </tr> <tr> <td>hdp-rm</td> <td>192.168.8.3/24</td> <td>-</td> <td>ResourceManager</td> </tr> <tr> <td>hdp-worker0</td> <td>192.168.8.10/24</td> <td>/data/hdp-dn</td> <td>Datanode NodeManager Spark</td> </tr> <tr> <td>hdp-worker1</td> <td>192.168.8.11/24</td> <td>/data/hdp-dn</td> <td>Datanode NodeManager Spark</td> </tr> <tr> <td>hive-hs</td> <td>192.168.8.20/24</td> <td>-</td> <td>HiveServer2 Tez</td> </tr> <tr> <td>hive-ms</td> <td>192.168.8.21/24</td> <td>-</td> <td>HiveMetastore Tez</td> </tr> </tbody> </table> <h2 id="部署-hadoop-集群">部署 hadoop 集群</h2> <h3 id="服务器初始配置">服务器初始配置</h3> <ul> <li>在<strong>全部主机</strong>上执行如下操作</li> <li>禁用防火墙</li> <li>禁用 selinux</li> <li>配置时间同步</li> <li>配置主机名解析，修改 /etc/hosts，增加如下内容 <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span><span class="lnt">2 </span><span class="lnt">3 </span><span class="lnt">4 </span><span class="lnt">5 </span><span class="lnt">6 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="c1"># hadoop</span> </span></span><span class="line"><span class="cl">192.168.8.1 hdp-nn </span></span><span class="line"><span class="cl">192.168.8.2 hdp-snn </span></span><span class="line"><span class="cl">192.168.8.3 hdp-dn </span></span><span class="line"><span class="cl">192.168.8.10 hdp-worker0 </span></span><span class="line"><span class="cl">192.168.8.11 hdp-worker1 </span></span></code></pre></td></tr></table> </div> </div></li> </ul> <h3 id="ssh-免密登录">ssh 免密登录</h3> <ul> <li> <p>在 <strong>hdp-nn</strong> 配置 ssh 免密登录 hdp-nn、hdp-snn 和 hdp-workerX</p> https://www.809508860.xyz/post/hdp2/ Tue, 23 May 2023 10:00:00 +0800 https://www.809508860.xyz/post/hdp2/ <h2 id="环境">环境</h2> <table> <thead> <tr> <th>主机名</th> <th>地址</th> <th>数据目录</th> <th>组件</th> </tr> </thead> <tbody> <tr> <td>编译服务器</td> <td>-</td> <td>-</td> <td>各种编译工具</td> </tr> <tr> <td>hdp-nn</td> <td>192.168.8.1/24</td> <td>/data/hdp-nn</td> <td>Namenode Spark</td> </tr> <tr> <td>hdp-snn</td> <td>192.168.8.2/24</td> <td>/data/hdp-snn</td> <td>SecondaryNamenode</td> </tr> <tr> <td>hdp-rm</td> <td>192.168.8.3/24</td> <td>-</td> <td>ResourceManager</td> </tr> <tr> <td>hdp-slave0</td> <td>192.168.8.10/24</td> <td>/data/hdp-dn</td> <td>Datanode NodeManager Spark</td> </tr> <tr> <td>hdp-slave1</td> <td>192.168.8.11/24</td> <td>/data/hdp-dn</td> <td>Datanode NodeManager Spark</td> </tr> <tr> <td>hive-hs</td> <td>192.168.8.20/24</td> <td>-</td> <td>HiveServer2 Tez</td> </tr> <tr> <td>hive-ms</td> <td>192.168.8.21/24</td> <td>-</td> <td>HiveMetastore Tez</td> </tr> <tr> <td>hbase-m</td> <td>192.168.8.30/24</td> <td>-</td> <td>HbaseMaster</td> </tr> <tr> <td>hbase-bm</td> <td>192.168.8.31/24</td> <td>-</td> <td>HbaseBackupMaster</td> </tr> <tr> <td>hbase-rs0</td> <td>192.168.8.32/24</td> <td>-</td> <td>HbaseRigionServer</td> </tr> <tr> <td>hbase-rs1</td> <td>192.168.8.33/24</td> <td>-</td> <td>HbaseRigionServer</td> </tr> </tbody> </table> <h2 id="部署-hadoop-集群">部署 hadoop 集群</h2> <h3 id="服务器初始配置">服务器初始配置</h3> <ul> <li>在<strong>全部主机</strong>上执行如下操作</li> <li>禁用防火墙</li> <li>禁用 selinux</li> <li>配置时间同步</li> <li>配置主机名解析，修改 /etc/hosts，增加如下内容 <div class="highlight"><div class="chroma"> <table class="lntable"><tr><td class="lntd"> <pre tabindex="0" class="chroma"><code><span class="lnt">1 </span><span class="lnt">2 </span><span class="lnt">3 </span><span class="lnt">4 </span><span class="lnt">5 </span><span class="lnt">6 </span></code></pre></td> <td class="lntd"> <pre tabindex="0" class="chroma"><code class="language-fallback" data-lang="fallback"><span class="line"><span class="cl"># hadoop </span></span><span class="line"><span class="cl">192.168.8.1 hdp-nn </span></span><span class="line"><span class="cl">192.168.8.2 hdp-snn </span></span><span class="line"><span class="cl">192.168.8.3 hdp-dn </span></span><span class="line"><span class="cl">192.168.8.10 hdp-slave0 </span></span><span class="line"><span class="cl">192.168.8.11 hdp-slave1 </span></span></code></pre></td></tr></table> </div> </div></li> </ul> <h3 id="ssh-免密登录">ssh 免密登录</h3> <ul> <li> <p>在 <strong>hdp-nn</strong> 配置 ssh 免密登录 hdp-nn、hdp-snn 和 hdp-slaveX</p>