<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>无名老卒BLOG</title>
  
  <subtitle>自己不能胜任的事情，切莫轻易答应别人，一旦答应了别人，就必须实践自己的诺言。</subtitle>
  <link href="/atom.xml" rel="self"/>
  
  <link href="https://www.wumingx.com/"/>
  <updated>2022-12-13T12:16:26.960Z</updated>
  <id>https://www.wumingx.com/</id>
  
  <author>
    <name>wumingx</name>
    
  </author>
  
  <generator uri="http://hexo.io/">Hexo</generator>
  
  <entry>
    <title>GlusterFS简介以及部署</title>
    <link href="https://www.wumingx.com/linux/GlusterFS.html"/>
    <id>https://www.wumingx.com/linux/GlusterFS.html</id>
    <published>2022-03-12T16:46:47.000Z</published>
    <updated>2022-12-13T12:16:26.960Z</updated>
    
    <content type="html"><![CDATA[<h1 id="GlusterFS简介"><a href="#GlusterFS简介" class="headerlink" title="GlusterFS简介"></a>GlusterFS简介</h1><p>PB级容量、高可用、读写性能、基于文件系统级别共享、分布式、无metadata(元数据)的存储方式。</p><p>GlusterFS(GNU ClusterFile System)是一种全对称的开源分布式文件系统，所谓全对称是指GlusterFS采用弹性哈希算法，没有中心节点，所有节点全部平等。GlusterFS配置方便，稳定性好，可轻松达到PB级容量，数千个节点。</p><p>2011年被红帽收购，之后推出了基于GlusterFS的Red Hat Storage Server，增加了针对KVM的许多特性，可用作为KVM存储image存储集群，也可以为LB或HA提供存储。</p><a id="more"></a><p>GlusterFS重要特性：全对称架构、支持多种卷类型(类似RAID0/1/5/10/01)、支持卷级别的压缩、支持FUSE、支持NFS、支持SMB、支持Hadoop、支持OpenStack、与oVirt深度整合(对应RHEL红帽企业级虚拟化)。</p><h1 id="GlusterFS重要概念"><a href="#GlusterFS重要概念" class="headerlink" title="GlusterFS重要概念"></a>GlusterFS重要概念</h1><ul><li>birck：GlusterFS的存储单元，以节点服务器目录形式展现；类似于Linux上块设备（block device）的概念。</li><li>volume：多个brick的逻辑集合；类似于LVM中的Volume。</li><li>metadata：元数据，用于描述文件、目录等的信息；</li><li>self-heal：用于后台运行检测副本卷中文件和目录的不一致性并解决这些不一致；</li><li>FUSE：Filesystem Userspace是一个可加载的内核模块，其支持非特权用户创建自己的文件系统而不需要修改内核代码通过在用户空间运行文件系统的代码通过FUSE代码与内核进行桥接；</li><li>Gluster Server：数据存储服务器，即组成GlusterFS存储集群的节点；</li><li>Gluster Client：使用GlusterFS存储服务的服务器，如KVM、OpenStack、LB RealServer、HA node。</li></ul><h1 id="GlusterFS部署"><a href="#GlusterFS部署" class="headerlink" title="GlusterFS部署"></a>GlusterFS部署</h1><h2 id="准备工作"><a href="#准备工作" class="headerlink" title="准备工作"></a>准备工作</h2><div class="note primary">            <p>由于GlusterFS并没有服务器与元数据等概念，因此所有服务器的设置都相同。如果操作时都用ip地址，不使用主机名，那么就不需要做hosts绑定，建议还是使用主机名方式。</p><p>部署时GlusterFS至少需要两台服务器搭建（建议三台），服务器配置最好相同，每个服务器两块磁盘，一块是用于安装系统，一块是用于GlusterFS。</p>          </div><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@GlusterFS-master ~]<span class="comment"># vim /etc/hosts</span></span><br><span class="line">.....</span><br><span class="line">172.31.0.6  node6</span><br><span class="line">172.31.0.7  node7</span><br><span class="line">172.31.0.8  node8</span><br></pre></td></tr></table></figure><p>本文直接使用IP方式进行测试。有三台服务器，分别为 <code>172.31.0.6/7/8</code>。</p><h2 id="软件包安装"><a href="#软件包安装" class="headerlink" title="软件包安装"></a>软件包安装</h2><p>根据 <a href="https://docs.gluster.org/en/latest/Quick-Start-Guide/Quickstart/" target="_blank" rel="noopener">官方Quickstart</a>，可以使用</p><p>服务端</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">yum -y install centos-release-gluster</span><br><span class="line">yum install -y glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma glusterfs-geo-replication glusterfs-devel</span><br></pre></td></tr></table></figure><p>客户端只需要<code>glusterfs、glusterfs-fuse</code></p><h2 id="创建集群"><a href="#创建集群" class="headerlink" title="创建集群"></a>创建集群</h2><p>分别启动glusterd服务(并添加开机自启动):</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">systemctl start glusterd</span><br><span class="line">systemctl <span class="built_in">enable</span> glusterd</span><br></pre></td></tr></table></figure><p>创建集群(任意节点上执行一下操作，向集群中添加节点):</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">gluster peer probe 172.31.0.7</span><br><span class="line">gluster peer probe 172.31.0.8</span><br></pre></td></tr></table></figure><p>从集群中去除节点(该节点中不能存在卷中正在使用的brick)：<code>gluster peer detach 172.31.0.7</code>。不需要添加自己，只需要添加其他节点即可</p><p>查看集群状态：<code>gluster peer status</code></p><h1 id="volume模式"><a href="#volume模式" class="headerlink" title="volume模式"></a>volume模式</h1><p>GlusterFS中的volume的模式有很多中，包括以下几种：</p><ul><li><strong>分布卷（默认模式）</strong>：即DHT, 也叫 分布卷: 将文件以hash算法随机分布到 一台服务器节点中存储。</li><li><strong>复制模式</strong>：即AFR, 创建volume 时带 replica x 数量: 将文件复制到 replica x 个节点中。</li><li><strong>条带模式</strong>：即Striped, 创建volume 时带 stripe x 数量： 将文件切割成数据块，分别存储到 stripe x 个节点中 ( 类似raid 0 )。</li><li><strong>分布式条带模式</strong>：最少需要4台服务器才能创建。 创建volume 时 stripe 2 server = 4 个节点： 是DHT 与 Striped 的组合型。</li><li><strong>分布式复制模式</strong>：最少需要4台服务器才能创建。 创建volume 时 replica 2 server = 4 个节点：是DHT 与 AFR 的组合型。</li><li><strong>条带复制卷模式</strong>：最少需要4台服务器才能创建。 创建volume 时 stripe 2 replica 2 server = 4 个节点： 是 Striped 与 AFR 的组合型。</li><li><strong>三种模式混合</strong>： 至少需要8台 服务器才能创建。 stripe 2 replica 2 , 每4个节点 组成一个 组。</li></ul><h2 id="分布卷"><a href="#分布卷" class="headerlink" title="分布卷"></a>分布卷</h2><p>分布式卷也成为哈希卷，多个文件以文件为单位在多个brick上，使用哈希算法随机存储。</p><ul><li>应用场景：大量小文件</li><li>优点：读/写性能好</li><li>缺点：如果存储或服务器故障，该brick上的数据将丢失</li></ul><p><img src="/assets/885374-20181029092703454-1731342742.png" alt="img"></p><p>创建分布式卷：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># volumn_name：卷名</span></span><br><span class="line"><span class="comment"># node1：节点名</span></span><br><span class="line"><span class="comment"># /data/br1：可以理解为节点上的目录，这个目录最好是一个单独的分区(分区类型最好为逻辑卷的方式，这样易于操作系统级别的存储空间扩展)</span></span><br><span class="line">gluster volume create volume_name node1:/data/br1 node2:/data/br1</span><br><span class="line"></span><br><span class="line"><span class="comment"># 启动这个卷:</span></span><br><span class="line">gluster volume start volume_name</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看这个卷的信息：</span></span><br><span class="line">gluster volume info volume_name</span><br></pre></td></tr></table></figure><h2 id="复制卷"><a href="#复制卷" class="headerlink" title="复制卷"></a>复制卷</h2><p>多个文件在多个brick上复制多份，brick的数目要与需要复制的份数相等，建议brick分布在不同的服务器上。</p><p>应用场景：对可靠性高和读写性能要求高的场景</p><p>优点：读写性能好</p><p>缺点：写性能差</p><p> <img src="/assets/885374-20181029092757774-1316952986.png" alt></p><p>创建复制卷：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 加上replica参数就完成了复制卷，后接文件保存的份数</span></span><br><span class="line">gluster volume create volume_name replica 2 node1:/data/br1 node2:/data/br1</span><br><span class="line"></span><br><span class="line"><span class="comment"># 启动这个卷:</span></span><br><span class="line">gluster volume start volume_name</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看这个卷的信息：</span></span><br><span class="line">gluster volume info volume_name</span><br></pre></td></tr></table></figure><h2 id="条带卷"><a href="#条带卷" class="headerlink" title="条带卷"></a>条带卷</h2><p>将大文件分成条带，存放在多个brick上，默认条带大小128k，应用场景为大文件，缺点：可靠性低，brick故障会导致数据全部丢失</p><p><img src="/assets/885374-20181029092816362-887072224.png" alt></p><p>创建条带卷：stripe：条带个数，<code>gluster volume create volume_name stripe 2 node1:/data/br1 node2:/data/br1</code></p><h2 id="分布式条带卷"><a href="#分布式条带卷" class="headerlink" title="分布式条带卷"></a>分布式条带卷</h2><p>多个文件在多个节点哈希存储，每个文件再多分条带在多个brick上存储</p><p> <img src="/assets/885374-20181029092828816-50382001.png" alt></p><ul><li>应用场景：读/写性能高的大量大文件场景</li><li>优点：高并发支持</li><li>缺点：没有冗余，可靠性差</li><li>brick数是stripe的倍数</li></ul><p>创建分布式条带卷：<code>gluster volume create volume_name stripe 2 node1:/data/br1 node2:/data/br1 node3:/data/br1 node4:/data/br1</code></p><p>启动这个卷: <code>gluster volume start volume_name</code></p><p>查看这个卷的信息：<code>gluster volume info volume_name</code></p><p>创建时没有具体的选项，来指定卷的类型，只根据stripe和brick数量分配</p><h2 id="分布式复制卷"><a href="#分布式复制卷" class="headerlink" title="分布式复制卷"></a>分布式复制卷</h2><p>多个文件在多个节点上哈希存储，在多个brick复制多份存储。</p><p><img src="/assets/885374-20181029092845265-1485325814.png" alt></p><ul><li>应用场景：大量文件读和可靠性要求高的场景</li><li>优点：高可靠，读性能高</li><li>缺点：牺牲存储空间，写性能差</li><li>brick数量是replica的倍数</li></ul><p>创建：<code>gluster volume create volume_name replica 2 node1:/data/br1 node2:/data/br1 node3:/data/br1 node4:/data/br1</code></p><h2 id="配置客户端使用卷"><a href="#配置客户端使用卷" class="headerlink" title="配置客户端使用卷"></a>配置客户端使用卷</h2><p>将卷挂载到本地的mysql目录：<code>mount –t glusterfs node1:/volume_name /mysql</code></p><p>设置开机自动挂载</p><p><code>vim /etc/fstab</code>加入：<code>node1:/volume_name /mysql glusterfs defaults,_netdev 0 0</code></p><p>使用<code>mount -a</code>检测并挂载测试</p><p>其他挂在方式(NFS、Samba)参考：</p><p><strong><a href="http://www.mamicode.com/info-detail-1925105.html" target="_blank" rel="noopener">http://www.mamicode.com/info-detail-1925105.html</a></strong></p><h1 id="实操"><a href="#实操" class="headerlink" title="实操"></a>实操</h1><p>以172.31.0.6 172.31.0.7 172.31.0.8这三台机器为例，演示安装和配置过程：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 安装</span></span><br><span class="line">yum -y install centos-release-gluster</span><br><span class="line">yum install -y glusterfs glusterfs-server glusterfs-fuse glusterfs-rdma glusterfs-geo-replication glusterfs-devel</span><br><span class="line"><span class="comment"># 启动服务</span></span><br><span class="line">systemctl start glusterd</span><br><span class="line">systemctl <span class="built_in">enable</span> glusterd</span><br><span class="line">mkdir /cache1/glusterd/ /gfsshare/ -p</span><br><span class="line"><span class="comment"># 在0.6的机器运行，添加peer</span></span><br><span class="line">gluster peer probe 172.31.0.7</span><br><span class="line">gluster peer probe 172.31.0.8</span><br><span class="line"><span class="comment"># 添加volume，使用三副本的方式</span></span><br><span class="line">gluster volume create hidsdata replica 3 172.31.0.6:/cache1/glusterd/ 172.31.0.7:/cache1/glusterd/ 172.31.0.8:/cache1/glusterd/</span><br><span class="line">gluster volume start hidsdata</span><br><span class="line">mount -t glusterfs 172.31.0.6:hidsdata /gfsshare/ <span class="comment">#手工</span></span><br><span class="line"><span class="built_in">echo</span> <span class="string">"172.31.0.6:hidsdata /gfsshare glusterfs defaults,_netdev 0 0"</span> &gt;&gt;/etc/fstab</span><br></pre></td></tr></table></figure><p>查看peer状态：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root@wlpx09w8npgsdp ~]<span class="comment"># gluster peer status</span></span><br><span class="line">Number of Peers: 2</span><br><span class="line"></span><br><span class="line">Hostname: 172.31.0.7</span><br><span class="line">Uuid: e9208e52-1d8e-4251-9f39-9df46537a31a</span><br><span class="line">State: Peer <span class="keyword">in</span> Cluster (Connected)</span><br><span class="line"></span><br><span class="line">Hostname: 172.31.0.8</span><br><span class="line">Uuid: b82d5eb1-391b-48d8-8d1e-d004274c6bd0</span><br><span class="line">State: Peer <span class="keyword">in</span> Cluster (Connected)</span><br></pre></td></tr></table></figure><p>查看volume的状态</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line">[root@wlpx09w8npgsdp ~]<span class="comment"># gluster volume list</span></span><br><span class="line">hidsdata</span><br><span class="line">[root@wlpx09w8npgsdp ~]<span class="comment"># </span></span><br><span class="line">[root@wlpx09w8npgsdp ~]<span class="comment"># gluster volume info hidsdata </span></span><br><span class="line"> </span><br><span class="line">Volume Name: hidsdata</span><br><span class="line">Type: Replicate</span><br><span class="line">Volume ID: f4d5905c-0e3a-448d-a1d6-ea73c7a5259f</span><br><span class="line">Status: Started</span><br><span class="line">Snapshot Count: 0</span><br><span class="line">Number of Bricks: 1 x 3 = 3</span><br><span class="line">Transport-type: tcp</span><br><span class="line">Bricks:</span><br><span class="line">Brick1: 172.31.0.6:/cache1/glusterd</span><br><span class="line">Brick2: 172.31.0.7:/cache1/glusterd</span><br><span class="line">Brick3: 172.31.0.8:/cache1/glusterd</span><br><span class="line">Options Reconfigured:</span><br><span class="line">cluster.granular-entry-heal: on</span><br><span class="line">storage.fips-mode-rchecksum: on</span><br><span class="line">transport.address-family: inet</span><br><span class="line">nfs.disable: off</span><br><span class="line">performance.client-io-threads: off</span><br><span class="line"></span><br><span class="line">[root@wlpx09w8npgsdp ~]<span class="comment"># gluster volume status hidsdata </span></span><br><span class="line">Status of volume: hidsdata</span><br><span class="line">Gluster process                             TCP Port  RDMA Port  Online  Pid</span><br><span class="line">------------------------------------------------------------------------------</span><br><span class="line">Brick 172.31.0.6:/cache1/glusterd           49152     0          Y       20608</span><br><span class="line">Brick 172.31.0.7:/cache1/glusterd           49152     0          Y       54538</span><br><span class="line">Brick 172.31.0.8:/cache1/glusterd           49152     0          Y       117976</span><br><span class="line">Self-heal Daemon on localhost               N/A       N/A        Y       123716</span><br><span class="line">Self-heal Daemon on 172.31.0.8              N/A       N/A        Y       118014</span><br><span class="line">Self-heal Daemon on 172.31.0.7              N/A       N/A        Y       54556</span><br><span class="line"> </span><br><span class="line">Task Status of Volume hidsdata</span><br><span class="line">------------------------------------------------------------------------------</span><br><span class="line">There are no active volume tasks</span><br></pre></td></tr></table></figure><h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><blockquote><p><a href="https://www.cnblogs.com/fansik/p/9868831.html" target="_blank" rel="noopener">GlusterFS部署</a></p><p><a href="https://www.cnblogs.com/linuxk/p/9449566.html" target="_blank" rel="noopener">GlusterFS学习之路（三）客户端挂载和管理GlusterFS卷</a></p><p><a href="https://www.cnblogs.com/luxf0/p/15798677.html" target="_blank" rel="noopener">【beegfs相关】beegfs集群部署</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;GlusterFS简介&quot;&gt;&lt;a href=&quot;#GlusterFS简介&quot; class=&quot;headerlink&quot; title=&quot;GlusterFS简介&quot;&gt;&lt;/a&gt;GlusterFS简介&lt;/h1&gt;&lt;p&gt;PB级容量、高可用、读写性能、基于文件系统级别共享、分布式、无metadata(元数据)的存储方式。&lt;/p&gt;
&lt;p&gt;GlusterFS(GNU ClusterFile System)是一种全对称的开源分布式文件系统，所谓全对称是指GlusterFS采用弹性哈希算法，没有中心节点，所有节点全部平等。GlusterFS配置方便，稳定性好，可轻松达到PB级容量，数千个节点。&lt;/p&gt;
&lt;p&gt;2011年被红帽收购，之后推出了基于GlusterFS的Red Hat Storage Server，增加了针对KVM的许多特性，可用作为KVM存储image存储集群，也可以为LB或HA提供存储。&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="GlusterFS" scheme="https://www.wumingx.com/tags/GlusterFS/"/>
    
  </entry>
  
  <entry>
    <title>shell常用功能的写法</title>
    <link href="https://www.wumingx.com/script/tutorial.html"/>
    <id>https://www.wumingx.com/script/tutorial.html</id>
    <published>2021-11-12T05:48:42.000Z</published>
    <updated>2022-12-13T12:16:26.975Z</updated>
    
    <content type="html"><![CDATA[<h1 id="测试IO与网络速度"><a href="#测试IO与网络速度" class="headerlink" title="测试IO与网络速度"></a>测试IO与网络速度</h1><p>使用 <code>wget -qO- bench.sh | bash</code> 进行测试</p><h1 id="判断系统版本"><a href="#判断系统版本" class="headerlink" title="判断系统版本"></a>判断系统版本</h1><p>使用系统文件来做判断。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="title">get_opsy</span></span>() &#123;</span><br><span class="line">    [ -f /etc/redhat-release ] &amp;&amp; awk <span class="string">'&#123;print $0&#125;'</span> /etc/redhat-release &amp;&amp; <span class="built_in">return</span></span><br><span class="line">    [ -f /etc/os-release ] &amp;&amp; awk -F<span class="string">'[= "]'</span> <span class="string">'/PRETTY_NAME/&#123;print $3,$4,$5&#125;'</span> /etc/os-release &amp;&amp; <span class="built_in">return</span></span><br><span class="line">    [ -f /etc/lsb-release ] &amp;&amp; awk -F<span class="string">'[="]+'</span> <span class="string">'/DESCRIPTION/&#123;print $2&#125;'</span> /etc/lsb-release &amp;&amp; <span class="built_in">return</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><a id="more"></a>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;测试IO与网络速度&quot;&gt;&lt;a href=&quot;#测试IO与网络速度&quot; class=&quot;headerlink&quot; title=&quot;测试IO与网络速度&quot;&gt;&lt;/a&gt;测试IO与网络速度&lt;/h1&gt;&lt;p&gt;使用 &lt;code&gt;wget -qO- bench.sh | bash&lt;/code&gt; 进行测试&lt;/p&gt;
&lt;h1 id=&quot;判断系统版本&quot;&gt;&lt;a href=&quot;#判断系统版本&quot; class=&quot;headerlink&quot; title=&quot;判断系统版本&quot;&gt;&lt;/a&gt;判断系统版本&lt;/h1&gt;&lt;p&gt;使用系统文件来做判断。&lt;/p&gt;
&lt;figure class=&quot;highlight bash&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;function&quot;&gt;&lt;span class=&quot;title&quot;&gt;get_opsy&lt;/span&gt;&lt;/span&gt;() &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    [ -f /etc/redhat-release ] &amp;amp;&amp;amp; awk &lt;span class=&quot;string&quot;&gt;&#39;&amp;#123;print $0&amp;#125;&#39;&lt;/span&gt; /etc/redhat-release &amp;amp;&amp;amp; &lt;span class=&quot;built_in&quot;&gt;return&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    [ -f /etc/os-release ] &amp;amp;&amp;amp; awk -F&lt;span class=&quot;string&quot;&gt;&#39;[= &quot;]&#39;&lt;/span&gt; &lt;span class=&quot;string&quot;&gt;&#39;/PRETTY_NAME/&amp;#123;print $3,$4,$5&amp;#125;&#39;&lt;/span&gt; /etc/os-release &amp;amp;&amp;amp; &lt;span class=&quot;built_in&quot;&gt;return&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    [ -f /etc/lsb-release ] &amp;amp;&amp;amp; awk -F&lt;span class=&quot;string&quot;&gt;&#39;[=&quot;]+&#39;&lt;/span&gt; &lt;span class=&quot;string&quot;&gt;&#39;/DESCRIPTION/&amp;#123;print $2&amp;#125;&#39;&lt;/span&gt; /etc/lsb-release &amp;amp;&amp;amp; &lt;span class=&quot;built_in&quot;&gt;return&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="shell教程" scheme="https://www.wumingx.com/categories/script/"/>
    
    
      <category term="shell" scheme="https://www.wumingx.com/tags/shell/"/>
    
  </entry>
  
  <entry>
    <title>ansible系列之六:常见疑问知识点</title>
    <link href="https://www.wumingx.com/linux/ansible-sex.html"/>
    <id>https://www.wumingx.com/linux/ansible-sex.html</id>
    <published>2021-10-06T13:10:50.000Z</published>
    <updated>2022-12-13T12:16:26.969Z</updated>
    
    <content type="html"><![CDATA[<h1 id="问题一：如何进行调试"><a href="#问题一：如何进行调试" class="headerlink" title="问题一：如何进行调试"></a>问题一：如何进行调试</h1><p>当我们写完一个很大的一个playbook时，需要指定位置停止运行，要怎么办呢？方法如下：</p><ul><li>在playbook配置 <code>any_errors_fatal: &quot;true&quot;</code>，这时当出现异常时，会直接停止。</li></ul><p><a href="https://blog.csdn.net/qq_36417677/article/details/105342211" target="_blank" rel="noopener">https://blog.csdn.net/qq_36417677/article/details/105342211</a></p><p><a href="https://blog.51cto.com/zouqingyun/1882367" target="_blank" rel="noopener">https://blog.51cto.com/zouqingyun/1882367</a></p><a id="more"></a><div class="note default | primary | success | info | warning | danger">                      </div>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;问题一：如何进行调试&quot;&gt;&lt;a href=&quot;#问题一：如何进行调试&quot; class=&quot;headerlink&quot; title=&quot;问题一：如何进行调试&quot;&gt;&lt;/a&gt;问题一：如何进行调试&lt;/h1&gt;&lt;p&gt;当我们写完一个很大的一个playbook时，需要指定位置停止运行，要怎么办呢？方法如下：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;在playbook配置 &lt;code&gt;any_errors_fatal: &amp;quot;true&amp;quot;&lt;/code&gt;，这时当出现异常时，会直接停止。&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;a href=&quot;https://blog.csdn.net/qq_36417677/article/details/105342211&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://blog.csdn.net/qq_36417677/article/details/105342211&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href=&quot;https://blog.51cto.com/zouqingyun/1882367&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://blog.51cto.com/zouqingyun/1882367&lt;/a&gt;&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="ansible" scheme="https://www.wumingx.com/tags/ansible/"/>
    
  </entry>
  
  <entry>
    <title>基于centos7.6制作自定义的ISO镜像</title>
    <link href="https://www.wumingx.com/linux/centos-iso.html"/>
    <id>https://www.wumingx.com/linux/centos-iso.html</id>
    <published>2021-09-07T02:14:20.000Z</published>
    <updated>2022-09-14T07:03:26.904Z</updated>
    
    <content type="html"><![CDATA[<p>以CentOS 7.6为例</p><p>全自动安装CentOS，在生产环境下可以有效减少安装的时间，以及可以集成一些软件到镜像上，做一个定制化的处理，所以有必要进行定制化。这样就可以实现2个功能：</p><ul><li>制作完成的ISO在安装过程中可以选择自定义的软件包</li><li>制作完成的ISO全自动安装，不需要在安装过程中进行任何选择操作</li></ul><a id="more"></a><h1 id="基础准备与介绍"><a href="#基础准备与介绍" class="headerlink" title="基础准备与介绍"></a>基础准备与介绍</h1><p>先下载原始的镜像， <a href="https://vault.centos.org/" target="_blank" rel="noopener">http://vault.centos.org/</a> 由于我是需要做centos 7.6的镜像，直接打开 <a href="https://vault.centos.org/7.6.1810/isos/x86_64/" target="_blank" rel="noopener">https://vault.centos.org/7.6.1810/isos/x86_64/</a> 下载DVD的镜像，注意，尽量选择这个镜像，minimal版本的包太少了，有些还会缺少。</p><p>下载好ISO，在虚机机上面安装，安装过程省略，自行百度。此次实验仅使用最小化来安装。</p><p>我此次是使用的是KVM进行虚拟化的，具体可以参考：<a href="/linux/centos-kvm.html">实战CentOS7安装且使用KVM虚拟机</a></p><h1 id="挂载光盘及复制"><a href="#挂载光盘及复制" class="headerlink" title="挂载光盘及复制"></a>挂载光盘及复制</h1><p>将原始ISO镜像复制到虚拟机里面，然后进行挂载。</p><h2 id="安装工具"><a href="#安装工具" class="headerlink" title="安装工具"></a>安装工具</h2><p>安装制作发行版的工具</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum -y install anaconda repodata createrepo mkisofs rsync</span><br></pre></td></tr></table></figure><h2 id="挂载光盘，同步文件"><a href="#挂载光盘，同步文件" class="headerlink" title="挂载光盘，同步文件"></a>挂载光盘，同步文件</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">mkdir /mnt/cdrom</span><br><span class="line">mount -o loop CentOS-7-x86_64-DVD-1810.iso /mnt/cdrom/   <span class="comment">##挂载iso到/mnt/cdrom文件下</span></span><br><span class="line">/usr/bin/rsync -a --exclude=Packages/ --exclude=repodata/ /mnt/cdrom/ /ISO/ <span class="comment">##同步/mnt/cdrom/下的文件到ISO/路径下，除了Packages和repodata文件夹</span></span><br><span class="line">mkdir -p /ISO/&#123;Packages,repodata&#125;  <span class="comment">##在ISO/文件夹下新建Packages和repodata文件夹</span></span><br></pre></td></tr></table></figure><p>挂载光盘之后，各分区的作用：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">.</span><br><span class="line">├── CentOS_BuildTag  <span class="comment"># 系统版本构建标签 </span></span><br><span class="line">├── EFI      <span class="comment"># UEFI 启动模式下必须文件，Legacy模式下是非必须文件</span></span><br><span class="line">├── EULA     <span class="comment"># 最终用户许可协议</span></span><br><span class="line">├── GPL      <span class="comment"># 通用公用许可证／执照(General Public License)</span></span><br><span class="line">├── images   <span class="comment"># 启动映像文件</span></span><br><span class="line">├── isolinux <span class="comment"># 存放光盘启动时的安装界面信息</span></span><br><span class="line">├── LiveOS   <span class="comment"># 存储了映像文件</span></span><br><span class="line">├── Packages <span class="comment"># 系统自带rpm包软件</span></span><br><span class="line">├── repodata <span class="comment"># 系统rpm包metadate源数据</span></span><br><span class="line">├── RPM-GPG-KEY-CentOS-7 <span class="comment"># rpm的GPG校验公钥</span></span><br><span class="line">├── RPM-GPG-KEY-CentOS-Testing-7 <span class="comment"># 同上</span></span><br><span class="line">└── TRANS.TBL <span class="comment"># 提供比ISO9660标准约定的基本文件名更加灵活的文件名, 用简约符号代表目录、文件、链接;</span></span><br><span class="line">discinfo    <span class="comment">#文件是安装价质的识别信息</span></span><br><span class="line">.treeinfo   <span class="comment">#文件是系统版本，创建时间及文件目录树结构信息</span></span><br><span class="line">ks.cfg     <span class="comment">#文件是无人值守自动化安装配置文件</span></span><br></pre></td></tr></table></figure><h2 id="复制rpm包"><a href="#复制rpm包" class="headerlink" title="复制rpm包"></a>复制rpm包</h2><p>使用脚本拷贝相关软件包到/ISO/Packages目录下，centos6安装之后是可以从/root/install.log读取安装了哪些安装包，但在Centos7的/root下面并没有install.log文件。于是我们可以通过下面脚本用<code>rpm -qa</code>的方式得到所需安装的软件包。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line">rpm -qa  &gt;package.txt</span><br><span class="line">DVD=<span class="string">'/mnt/cdrom/Packages'</span></span><br><span class="line">NEW_DVD=<span class="string">'/ISO/Packages'</span></span><br><span class="line"><span class="keyword">while</span> <span class="built_in">read</span> LINE</span><br><span class="line"><span class="keyword">do</span></span><br><span class="line">cp <span class="variable">$&#123;DVD&#125;</span>/<span class="variable">$&#123;LINE&#125;</span>*.rpm /<span class="variable">$&#123;NEW_DVD&#125;</span> || <span class="built_in">echo</span> <span class="string">"<span class="variable">$LINE</span> don't cp......."</span></span><br><span class="line"><span class="keyword">done</span> &lt; package.txt</span><br><span class="line">rm -f package.txt</span><br></pre></td></tr></table></figure><p>或者使用 <code>awk &#39;{print $0}&#39;  package.txt |xargs -i cp /mnt/cdrom/Packages/{}.rpm /ISO/Packages/</code> 来复制即可。</p><h1 id="下载自定义软件包以及依赖包"><a href="#下载自定义软件包以及依赖包" class="headerlink" title="下载自定义软件包以及依赖包"></a>下载自定义软件包以及依赖包</h1><p>配置yum下载指定软件包列表（rpms_list.txt）以及所有依赖包</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/sh   </span></span><br><span class="line"><span class="keyword">for</span> line <span class="keyword">in</span> `cat /root/rpms_list.txt`;<span class="keyword">do</span></span><br><span class="line">    <span class="built_in">echo</span> <span class="string">"download file&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;&gt;and it's denpendies"</span><span class="variable">$line</span></span><br><span class="line">    yum install -y --downloadonly --downloaddir=/root/<span class="built_in">test</span>/ <span class="variable">$line</span></span><br><span class="line"><span class="comment">#或者先安装yum -y install yum-utils再yumdownloader $line</span></span><br><span class="line"><span class="keyword">done</span></span><br><span class="line"><span class="comment">#yum查找.so或者某个依赖在哪个rpm包中</span></span><br><span class="line">yum provides &#123;.so名或者依赖名字&#125;</span><br><span class="line"><span class="comment">#拷贝包到指定目录</span></span><br><span class="line">cp /root/<span class="built_in">test</span>/* /ISO/Packages/</span><br></pre></td></tr></table></figure><p>这一步其实是可以写成脚本，不过需要做一下特殊的判断。</p><h2 id="修改isolinux-isolinux-cfg文件"><a href="#修改isolinux-isolinux-cfg文件" class="headerlink" title="修改isolinux/isolinux.cfg文件"></a>修改isolinux/isolinux.cfg文件</h2><p>这里指定ks文件引导：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">label linux</span><br><span class="line">  menu label ^Install CentOS 7.6 1810</span><br><span class="line">  menu default</span><br><span class="line">  kernel vmlinuz</span><br><span class="line">  <span class="comment">#append initrd=initrd.img inst.stage2=hd:LABEL=CentOS\x207\x20x86_64 quiet</span></span><br><span class="line">  append ks=cdrom:/ks.cfg initrd=initrd.img inst.stage2=hd:LABEL=CentOS76</span><br></pre></td></tr></table></figure><ul><li>ks为ks.cfg文件位置；</li><li>inst.stage2为安装介质位置，hd:LABEL为介质标签，例如CentOS7。这个和后续生成ISO镜像文件的命令genisoimage的参数-V有关。最好是把所有hd:LABEL后面的值都替换为一个后面会用到的字符串，如CentOS76</li><li>modprobe.blacklist=nouveau; 禁用nouveau驱动安装，用于NVIDIA驱动的安装准备工作；</li><li>net.ifnames=0 biosdevname=0； 用于禁用centos7的”一致性网络设备命名法”.</li></ul><h2 id="生成ks-cfg文件"><a href="#生成ks-cfg文件" class="headerlink" title="生成ks.cfg文件"></a>生成ks.cfg文件</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cp /root/anaconda-ks.cfg /ISO/isolinux/ks.cfg</span><br></pre></td></tr></table></figure><p>在%packages与%end中间加入需要自定义安装的包组。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">%packages</span><br><span class="line">@^web-server-environment</span><br><span class="line">@base</span><br><span class="line">@core</span><br><span class="line">@java-platform</span><br><span class="line">@python-web</span><br><span class="line">@web-server</span><br><span class="line">kexec-tools</span><br><span class="line">@&#123;自定义包组的groupid&#125;</span><br><span class="line">%end</span><br></pre></td></tr></table></figure><h2 id="制作修改comps文件"><a href="#制作修改comps文件" class="headerlink" title="制作修改comps文件"></a>制作修改comps文件</h2><h3 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h3><p>先来解释下comps文件。</p><p>进入<code>/mnt/cdrom/repodata</code> 目录，将<code>-x86_64-comps.xml</code>文件拷贝到/ISO/repodata路径下，并重命名成comps.xml。由于centos6.5下的comps.xml文件名很长，这里“*”为省略符，实际操作时输入完整文件名。</p><p>comps文件以<code>group</code>来区分包，以<code>environment</code>来区分环境，例如centos的mini版本为<code>minimal</code>，其核心包为<code>core</code>。类似于其格式，可定制自己的rpm包，建立自己的<code>group id</code>和<code>name</code>，包含自己的<code>language</code>及描述，最重要的是<code>packagelist</code>，类型<code>default</code>为默认的，<code>mandatory</code>为强制的。如下是默认的comps格式：</p><ul><li>先定义好一个group的分组，里面包括了package的信息，</li></ul><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?xml version="1.0" encoding="UTF-8"?&gt;</span></span><br><span class="line"><span class="meta">&lt;!DOCTYPE comps PUBLIC "-//CentOS//DTD Comps info//EN" "comps.dtd"&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">comps</span>&gt;</span></span><br><span class="line">  <span class="tag">&lt;<span class="name">group</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">id</span>&gt;</span>core<span class="tag">&lt;/<span class="name">id</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">name</span>&gt;</span>Core<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">name</span> <span class="attr">xml:lang</span>=<span class="string">"zh_CN"</span>&gt;</span>核心<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">description</span>&gt;</span>Smallest possible installation.<span class="tag">&lt;/<span class="name">description</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">description</span> <span class="attr">xml:lang</span>=<span class="string">"zh_CN"</span>&gt;</span>最小可能安装。<span class="tag">&lt;/<span class="name">description</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">default</span>&gt;</span>false<span class="tag">&lt;/<span class="name">default</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">uservisible</span>&gt;</span>false<span class="tag">&lt;/<span class="name">uservisible</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">packagelist</span>&gt;</span></span><br><span class="line">      <span class="tag">&lt;<span class="name">packagereq</span> <span class="attr">type</span>=<span class="string">"mandatory"</span>&gt;</span>audit<span class="tag">&lt;/<span class="name">packagereq</span>&gt;</span></span><br><span class="line">      <span class="tag">&lt;<span class="name">packagereq</span> <span class="attr">type</span>=<span class="string">"default"</span>&gt;</span>aic94xx-firmware<span class="tag">&lt;/<span class="name">packagereq</span>&gt;</span></span><br><span class="line">      <span class="tag">&lt;<span class="name">packagereq</span> <span class="attr">type</span>=<span class="string">"optional"</span>&gt;</span>tboot<span class="tag">&lt;/<span class="name">packagereq</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">packagelist</span>&gt;</span></span><br><span class="line">  <span class="tag">&lt;/<span class="name">group</span>&gt;</span></span><br><span class="line">  <span class="tag">&lt;<span class="name">environment</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">id</span>&gt;</span>minimal<span class="tag">&lt;/<span class="name">id</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">name</span>&gt;</span>Minimal Install<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">name</span> <span class="attr">xml:lang</span>=<span class="string">"zh_CN"</span>&gt;</span>最小安装<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">description</span>&gt;</span>Basic functionality.<span class="tag">&lt;/<span class="name">description</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">description</span> <span class="attr">xml:lang</span>=<span class="string">"zh_CN"</span>&gt;</span>基本功能。<span class="tag">&lt;/<span class="name">description</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">display_order</span>&gt;</span>5<span class="tag">&lt;/<span class="name">display_order</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">grouplist</span>&gt;</span></span><br><span class="line">      <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>core<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">      <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>core<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">grouplist</span>&gt;</span></span><br><span class="line">  <span class="tag">&lt;/<span class="name">environment</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">comps</span>&gt;</span></span><br></pre></td></tr></table></figure><p>然后在ks.cfg文件里面定义好package就可以实现自动安装，使用 <code>@</code> 表明安装一个group，</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">%packages</span><br><span class="line">@^minimal</span><br><span class="line">@core</span><br><span class="line">kexec-tools</span><br></pre></td></tr></table></figure><h3 id="精思化操作"><a href="#精思化操作" class="headerlink" title="精思化操作"></a>精思化操作</h3><p>这个方法比较麻烦，但是对于ks.cfg的维护比较规范，即定义好group分组，再然后选择指定的分组进行安装，方法如下：</p><p>1) 进入/ISO目录，将“*-x86_64-comps.xml”文件拷贝到/ISO/repodata路径下，并重命名成comps.xml。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cp /media/repodata/*-x86_64-comps.xml  /ISO/repodata/comps.xml</span><br></pre></td></tr></table></figure><p>2) 编辑comps文件</p><p>(1) 添加定制rpm包</p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">group</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">id</span>&gt;</span>&#123;自定义包组的groupid&#125;<span class="tag">&lt;/<span class="name">id</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">name</span>&gt;</span>My Dependencies<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">default</span>&gt;</span>true<span class="tag">&lt;/<span class="name">default</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">uservisible</span>&gt;</span>true<span class="tag">&lt;/<span class="name">uservisible</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">packagelist</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">packagereq</span> <span class="attr">type</span>=<span class="string">"default"</span>&gt;</span>python-webob<span class="tag">&lt;/<span class="name">packagereq</span>&gt;</span></span><br><span class="line">        ......</span><br><span class="line">    <span class="tag">&lt;/<span class="name">packagelist</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">group</span>&gt;</span></span><br></pre></td></tr></table></figure><p>根据rpms_list.txt拼接packagereq</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/sh</span></span><br><span class="line"><span class="keyword">for</span> line <span class="keyword">in</span> `cat rpms_list.txt`</span><br><span class="line">    <span class="keyword">do</span></span><br><span class="line">        <span class="built_in">echo</span> <span class="string">" &lt;packagereq type=\"default\"&gt;"</span><span class="variable">$line</span><span class="string">"&lt;/packagereq&gt;"</span> &gt;&gt; /root/package.txt</span><br><span class="line">    <span class="keyword">done</span></span><br></pre></td></tr></table></figure><p>(2) 在指定的环境中添加定制的groupid</p><figure class="highlight xml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">environment</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">id</span>&gt;</span>web-server-environment<span class="tag">&lt;/<span class="name">id</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">name</span>&gt;</span>Basic Web Server<span class="tag">&lt;/<span class="name">name</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">description</span>&gt;</span>Server for serving static and dynamic internet content. <span class="tag">&lt;/<span class="name">description</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">display_order</span>&gt;</span>30<span class="tag">&lt;/<span class="name">display_order</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">grouplist</span>&gt;</span></span><br><span class="line">          <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>base<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">          <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>core<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">          <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>web-server<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">          <span class="tag">&lt;<span class="name">groupid</span>&gt;</span>&#123;自定义包组的groupid&#125;<span class="tag">&lt;/<span class="name">groupid</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">grouplist</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">environment</span>&gt;</span></span><br></pre></td></tr></table></figure><p>切换到ISO/路径下，由comps.xml生成repodata包。注意当有新包加入，或者更新comps.xml文件，均需要重新生成repodata文件夹</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">createrepo -g repodata/comps.xml ./</span><br></pre></td></tr></table></figure><h3 id="粗放式方法"><a href="#粗放式方法" class="headerlink" title="粗放式方法"></a>粗放式方法</h3><p>从上面的方法可以看出，整理一个comps.xml相对比较麻烦，还有一个方法，是不修改这个文件，使用以下方法即可：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cp /media/repodata/*-x86_64-comps.xml  /ISO/repodata/comps.xml</span><br><span class="line">createrepo -g repodata/comps.xml ./</span><br></pre></td></tr></table></figure><p>但这样做的结果是需要在ks.cfg文件里面维护全部的包名，如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">%packages --ignoremissing</span><br><span class="line">abrt-2.1.11-52.el7.centos.x86_64</span><br><span class="line">abrt-addon-ccpp-2.1.11-52.el7.centos.x86_64</span><br><span class="line">abrt-addon-kerneloops-2.1.11-52.el7.centos.x86_64</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h2 id="制作ISO镜像"><a href="#制作ISO镜像" class="headerlink" title="制作ISO镜像"></a>制作ISO镜像</h2><p>注意参数中的-V，和上面的isolinux.cfg文件有关。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> /ISO</span><br><span class="line">genisoimage -joliet-long -V CentOS7 -o CentOS-7-6.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -R -J -v -cache-inodes -T -eltorito-alt-boot -e images/efiboot.img -no-emul-boot /ISO</span><br></pre></td></tr></table></figure><h2 id="制作镜像MD5值"><a href="#制作镜像MD5值" class="headerlink" title="制作镜像MD5值"></a>制作镜像MD5值</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">implantisomd5 /ISO/CentOS-7-6.iso</span><br></pre></td></tr></table></figure><h2 id="Hybird模式"><a href="#Hybird模式" class="headerlink" title="Hybird模式"></a>Hybird模式</h2><p>采用“hybird模式”（混合模式），操作系统可以直接刻录成物理光盘，也可以直接做成可引导的U盘。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">isohybrid -v /ISO/CentOS-7-6.iso</span><br></pre></td></tr></table></figure><h1 id="安装系统"><a href="#安装系统" class="headerlink" title="安装系统"></a>安装系统</h1><p>只要不出错，一般十来分钟就可以安装完成，但如果出错了，需要使用以下方法来做排查：</p><p><strong>由于是<code>text</code>模式，则进入安装界面，会有一个类似图形安装界面的排版，里面可以选择。所有选择均为<code>x</code>的时候，表示正常无误;有<code>!</code>的话，则为错误。</strong>错误时需要查看日志，其中<code>Alt+F1</code>快捷键可以进入<code>main</code>界面，<code>Alt+F2</code>快捷键可以进入<code>Shell</code>界面等等。在<code>Shell</code>界面中，可以从<code>/tmp/packaging.log</code>中找到rpm包日志，可以从<code>/tmp/anaconda.log</code>中找到安装过程中的报错日志;可以从<code>/run/install/repo</code>路径下找到外部文件夹；</p><p>当选项均为<code>x</code>时，即表示正常无误，可以不进行操作，其自动进入下一步安装过程，直至安装成功，然后自动重启，显示登录界面。</p><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p><a href="https://zhuanlan.zhihu.com/p/140972579" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/140972579</a></p><p><a href="https://o-my-chenjian.com/2017/11/20/DIY-A-CentOS7-System/" target="_blank" rel="noopener">定制个自己的CentOS7系统</a></p><p><a href="https://blog.csdn.net/chyq112366/article/details/90107753" target="_blank" rel="noopener">https://blog.csdn.net/chyq112366/article/details/90107753</a></p><p><a href="https://yangfannie.com/1930.html" target="_blank" rel="noopener">https://yangfannie.com/1930.html</a></p><p><a href="https://cloud.tencent.com/developer/article/1729640" target="_blank" rel="noopener">https://cloud.tencent.com/developer/article/1729640</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;以CentOS 7.6为例&lt;/p&gt;
&lt;p&gt;全自动安装CentOS，在生产环境下可以有效减少安装的时间，以及可以集成一些软件到镜像上，做一个定制化的处理，所以有必要进行定制化。这样就可以实现2个功能：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;制作完成的ISO在安装过程中可以选择自定义的软件包&lt;/li&gt;
&lt;li&gt;制作完成的ISO全自动安装，不需要在安装过程中进行任何选择操作&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="centos" scheme="https://www.wumingx.com/tags/centos/"/>
    
  </entry>
  
  <entry>
    <title>aws上传私有化centos镜像到AMI的方法</title>
    <link href="https://www.wumingx.com/tools/aws-ami.html"/>
    <id>https://www.wumingx.com/tools/aws-ami.html</id>
    <published>2021-08-08T14:32:19.000Z</published>
    <updated>2022-12-13T12:13:23.970Z</updated>
    
    <content type="html"><![CDATA[<h1 id="为什么需要"><a href="#为什么需要" class="headerlink" title="为什么需要"></a>为什么需要</h1><p>aws的云主机要选择自己系统的时候，就需要通过自有的AMI镜像来操作的，但是AWS是不支持直接上传ISO包的，所以只能通过aws-cli来上传到S3存储桶里面，然后再导入到AMI镜像中。</p><div class="note success">            <p>导入到AMI的前提是已经在虚拟机上面安装好了系统，然后配置为DHCP，关机之后就可以导入。在windows平台可以使用vmware、HyperV、virtualBox等虚拟机软件，在导出格式时，尽量选择使用ova的格式。但由于某些原因，我们这次使用centos的KVM来做虚拟化。</p>          </div><a id="more"></a><h1 id="安装镜像"><a href="#安装镜像" class="headerlink" title="安装镜像"></a>安装镜像</h1><p>我们使用kvm来安装需要的镜像，由于aws不支持qcow2格式，所以直接使用了raw格式：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">qemu-img create -f raw -o size=5G /data/raw/raw-disk01.raw</span><br><span class="line">virt-install --name wsos2 --virt-type kvm --ram 32768 --vcpus=16 --cdrom=/cache2/WSOS.iso --disk path=/cache2/raw-disk01.raw --network=bridge:virbr0 --graphics vnc,listen=0.0.0.0,port=5968,password=xxx --noautoconsole</span><br></pre></td></tr></table></figure><p>运行之后，可以使用VNC来观察安装的进展。</p><p>kvm具体使用方法可以参考 <a href="/linux/centos-kvm.html">实战CentOS7安装且使用KVM虚拟机</a> </p><p>安装完成之后，把网卡配置为DHCP，这一步是必须的，然后直接关机即可。这样就准备好了虚拟化的文件了。</p><h1 id="AMI制作"><a href="#AMI制作" class="headerlink" title="AMI制作"></a>AMI制作</h1><p>主要有几步：</p><ul><li>安装与配置awscli</li><li>创建服务角色以方便S3的上传</li><li>导入镜像</li></ul><h2 id="安装与配置awscli"><a href="#安装与配置awscli" class="headerlink" title="安装与配置awscli"></a>安装与配置awscli</h2><p>使用 pip 安装 AWS CLI：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pip install awscli --upgrade --user</span><br></pre></td></tr></table></figure><p>更多安装方法参考：<a href="https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-install.html" target="_blank" rel="noopener">https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-install.html</a></p><p>配置AWS CLI:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ aws configure</span><br><span class="line">AWS Access Key ID [None]: xxx</span><br><span class="line">AWS Secret Access Key [None]: xxxxxx</span><br><span class="line">Default region name [None]: us-west-2</span><br><span class="line">Default output format [None]: json</span><br></pre></td></tr></table></figure><p>第一次运行的时候需要配置CLI，AWS CLI 会提示您输入四条信息，并将它们存储在名为 default 的配置文件（一个设置集合）中。每当您运行的 AWS CLI 命令未明确指定要使用的配置文件时，就会使用该配置文件。<br>参考：<a href="https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-configure.html" target="_blank" rel="noopener">https://docs.aws.amazon.com/zh_cn/cli/latest/userguide/cli-chap-configure.html</a></p><h2 id="创建服务角色"><a href="#创建服务角色" class="headerlink" title="创建服务角色"></a>创建服务角色</h2><p>1、创建名为 trust-policy.json 的文件：</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">   <span class="attr">"Version"</span>: <span class="string">"2019-04-01"</span>,</span><br><span class="line">   <span class="attr">"Statement"</span>: [</span><br><span class="line">      &#123;</span><br><span class="line">         <span class="attr">"Effect"</span>: <span class="string">"Allow"</span>,</span><br><span class="line">         <span class="attr">"Principal"</span>: &#123; <span class="attr">"Service"</span>: <span class="string">"vmie.amazonaws.com"</span> &#125;,</span><br><span class="line">         <span class="attr">"Action"</span>: <span class="string">"sts:AssumeRole"</span>,</span><br><span class="line">         <span class="attr">"Condition"</span>: &#123;</span><br><span class="line">            <span class="attr">"StringEquals"</span>:&#123;</span><br><span class="line">               <span class="attr">"sts:Externalid"</span>: <span class="string">"vmimport"</span></span><br><span class="line">            &#125;</span><br><span class="line">         &#125;</span><br><span class="line">      &#125;</span><br><span class="line">   ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>2、使用 create-role 命令创建名为 vmimport 的角色，并向 VM Import/Export 提供对该角色的访问权。请确保指定 trust-policy.json 文件的完整路径，并且为路径添加 file:// 前缀。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">aws iam create-role --role-name vmimport --assume-role-policy-document file://trust-policy.json</span><br></pre></td></tr></table></figure><p>3、创建名为 role-policy.json 的文件并编写下面的策略，其中，disk-image-file-bucket 为存储磁盘映像的存储桶：</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">   <span class="attr">"Version"</span>:<span class="string">"2012-10-17"</span>,</span><br><span class="line">   <span class="attr">"Statement"</span>:[</span><br><span class="line">      &#123;</span><br><span class="line">         <span class="attr">"Effect"</span>:<span class="string">"Allow"</span>,</span><br><span class="line">         <span class="attr">"Action"</span>:[</span><br><span class="line">            <span class="string">"s3:GetBucketLocation"</span>,</span><br><span class="line">            <span class="string">"s3:GetObject"</span>,</span><br><span class="line">            <span class="string">"s3:ListBucket"</span> </span><br><span class="line">         ],</span><br><span class="line">         <span class="attr">"Resource"</span>:[</span><br><span class="line">            <span class="string">"arn:aws-cn:s3:::hongsin"</span>,</span><br><span class="line">            <span class="string">"arn:aws-cn:s3:::hongsin/*"</span></span><br><span class="line">         ]</span><br><span class="line">      &#125;,</span><br><span class="line">      &#123;</span><br><span class="line">         <span class="attr">"Effect"</span>:<span class="string">"Allow"</span>,</span><br><span class="line">         <span class="attr">"Action"</span>:[</span><br><span class="line">            <span class="string">"ec2:ModifySnapshotAttribute"</span>,</span><br><span class="line">            <span class="string">"ec2:CopySnapshot"</span>,</span><br><span class="line">            <span class="string">"ec2:RegisterImage"</span>,</span><br><span class="line">            <span class="string">"ec2:Describe*"</span></span><br><span class="line">         ],</span><br><span class="line">         <span class="attr">"Resource"</span>:<span class="string">"*"</span></span><br><span class="line">      &#125;</span><br><span class="line">   ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>其中</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">"arn:aws-cn:s3:::disk-image-file-bucket"</span>,</span><br><span class="line"><span class="string">"arn:aws-cn:s3:::disk-image-file-bucket/*"</span></span><br></pre></td></tr></table></figure><p>arn:aws-cn:s3为AWS中国区域，disk-image-file-bucket 为存储磁盘映像的存储桶：为存储桶的名称<br>4、使用put-role-policy命令将策略挂载到之前创建的角色，请指定 role-policy.json 文件位置的完整路径</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">aws iam put-role-policy --role-name vmimport --policy-name vmimport --policy-document file://role-policy.json</span><br></pre></td></tr></table></figure><h2 id="导入镜像"><a href="#导入镜像" class="headerlink" title="导入镜像"></a>导入镜像</h2><p>使用以下命令导入镜像，</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">aws ec2 import-image --platform Linux --license-type BYOL --no-encrypted --description _imagedescription_ --architecture x86_64 --disk-containers Format=Raw,UserBucket=<span class="string">"&#123;S3Bucket=centosbucket,S3Key=wsos.raw&#125;"</span></span><br></pre></td></tr></table></figure><p>运行之后会返回一个ID，这时就可以使用 <code>aws ec2 describe-import-image-tasks --import-task-ids import-ami-0c0a99df27707ef13</code> 来查看进度，几个状态为：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">包括的状态值如下：</span><br><span class="line">active — 正在运行导入任务。</span><br><span class="line">deleting — 正在取消导入任务。</span><br><span class="line">deleted — 导入任务已取消。</span><br><span class="line">updating — 导入状态正在更新。</span><br><span class="line">validating — 正在验证导入的映像。</span><br><span class="line">validated — 已验证导入的映像。</span><br><span class="line">converting — 正在将导入的映像转换成 AMI。</span><br><span class="line">completed — 导入任务已完成，并且 AMI 已准备就绪，随时可以使用。</span><br></pre></td></tr></table></figure><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p><a href="https://www.unixso.com/Windows/aws-AMI.html" target="_blank" rel="noopener">AWS镜像AMI制作</a></p><p><a href="https://docs.aws.amazon.com/zh_cn/vm-import/latest/userguide/vmie_prereqs.html" target="_blank" rel="noopener">官方文档：VM Import说明</a></p><p><a href="https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/7/html/deploying_red_hat_enterprise_linux_7_on_public_cloud_platforms/deploying-a-virtual-machine-on-aws_cloud-content" target="_blank" rel="noopener">将 RED HAT ENTERPRISE LINUX 镜像部署为 EC2 实例</a></p><p><a href="https://access.redhat.com/documentation/zh-cn/red_hat_enterprise_linux/8/html/composing_a_customized_rhel_system_image/creating-cloud-images-with-composer_composing-a-customized-rhel-system-image" target="_blank" rel="noopener">第 6 章 使用镜像构建器准备并上传云镜像</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;为什么需要&quot;&gt;&lt;a href=&quot;#为什么需要&quot; class=&quot;headerlink&quot; title=&quot;为什么需要&quot;&gt;&lt;/a&gt;为什么需要&lt;/h1&gt;&lt;p&gt;aws的云主机要选择自己系统的时候，就需要通过自有的AMI镜像来操作的，但是AWS是不支持直接上传ISO包的，所以只能通过aws-cli来上传到S3存储桶里面，然后再导入到AMI镜像中。&lt;/p&gt;
&lt;div class=&quot;note success&quot;&gt;
            &lt;p&gt;导入到AMI的前提是已经在虚拟机上面安装好了系统，然后配置为DHCP，关机之后就可以导入。在windows平台可以使用vmware、HyperV、virtualBox等虚拟机软件，在导出格式时，尽量选择使用ova的格式。但由于某些原因，我们这次使用centos的KVM来做虚拟化。&lt;/p&gt;
          &lt;/div&gt;
    
    </summary>
    
      <category term="工具教程" scheme="https://www.wumingx.com/categories/tools/"/>
    
    
      <category term="ami" scheme="https://www.wumingx.com/tags/ami/"/>
    
      <category term="aws" scheme="https://www.wumingx.com/tags/aws/"/>
    
  </entry>
  
  <entry>
    <title>zookeeper集群迁移实战</title>
    <link href="https://www.wumingx.com/linux/zookeeper-move.html"/>
    <id>https://www.wumingx.com/linux/zookeeper-move.html</id>
    <published>2021-06-18T09:44:15.000Z</published>
    <updated>2021-06-18T09:48:38.017Z</updated>
    
    <content type="html"><![CDATA[<p>线上环境需要做迁移，所以就需要动手了。</p><a id="more"></a><h2 id="环境说明"><a href="#环境说明" class="headerlink" title="环境说明"></a>环境说明</h2><p>需要将以下zk迁移至新的机器上</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">server.1=172.31.0.4:2888:3888</span><br><span class="line">server.2=172.31.0.4:2889:3889</span><br><span class="line">server.3=172.31.0.4:2890:3890</span><br></pre></td></tr></table></figure><p>新机器上面的zk先不要启动</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">server.6=172.31.0.5:2888:3888</span><br><span class="line">server.7=172.31.0.5:2889:3889</span><br><span class="line">server.8=172.31.0.5:2890:3890</span><br></pre></td></tr></table></figure><p>手工操作</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">start) /opt/zookeeper/zookeeperServer0/bin/zkServer.sh start /opt/zookeeper/zk1/conf/zoo.cfg</span><br><span class="line">stop) /opt/zookeeper/zookeeperServer0/bin/zkServer.sh stop /opt/zookeeper/zk1/conf/zoo.cfg</span><br><span class="line">status) /opt/zookeeper/zookeeperServer0/bin/zkServer.sh status /opt/zookeeper/zk1/conf/zoo.cfg</span><br><span class="line">restart) /opt/zookeeper/zookeeperServer0/bin/zkServer.sh restart /opt/zookeeper/zk1/conf/zoo.cfg</span><br></pre></td></tr></table></figure><h1 id="总体原则"><a href="#总体原则" class="headerlink" title="总体原则"></a>总体原则</h1><h1 id="详细步骤"><a href="#详细步骤" class="headerlink" title="详细步骤"></a>详细步骤</h1><p>zk集群中有半数以上节点存活即可提供服务，使用这个特性可以把3个节点的集群扩容到5个节点的集群，然后在依次缩容，减少的3个节点的集群。通过这个过程达到迁移的目的。</p><p>准备集群环境：</p><div class="table-container"><table><thead><tr><th style="text-align:center">Myid</th><th style="text-align:center">IP</th><th>端口</th></tr></thead><tbody><tr><td style="text-align:center">1</td><td style="text-align:center">172.31.0.4</td><td>2181</td></tr><tr><td style="text-align:center">2</td><td style="text-align:center">172.31.0.4</td><td>2182</td></tr><tr><td style="text-align:center">3</td><td style="text-align:center">172.31.0.4</td><td>2183</td></tr><tr><td style="text-align:center">6</td><td style="text-align:center">172.31.0.5</td><td>2181</td></tr><tr><td style="text-align:center">7</td><td style="text-align:center">172.31.0.5</td><td>2182</td></tr><tr><td style="text-align:center">8</td><td style="text-align:center">172.31.0.5</td><td>2183</td></tr></tbody></table></div><div class="note primary">            <p>旧集群是172.31.0.4，新集群是172.31.0.5。<code>新集群的myid必须要比旧集群的大</code>，不然服务启动不了。</p>          </div><h2 id="扩容"><a href="#扩容" class="headerlink" title="扩容"></a>扩容</h2><ol><li><p>修改myid为 6 7的配置，启动这2个进程:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">server.1=172.31.0.4:2888:3888</span><br><span class="line">server.2=172.31.0.4:2889:3889</span><br><span class="line">server.3=172.31.0.4:2890:3890</span><br><span class="line">server.6=172.31.0.5:2888:3888</span><br><span class="line">server.7=172.31.0.5:2889:3889</span><br></pre></td></tr></table></figure><p>使用 <code>/opt/zookeeper/zookeeperServer0/bin/zkServer.sh status /opt/zookeeper/zk1/conf/zoo.cfg</code> 这个命令来查看状态，如果是follower即可。</p><p>启动节点4和节点5的zk服务，并在leader节点查看数据是否同步。在leader上面查询followers数：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@gl7wid90gwcewn ~]<span class="comment"># echo mntr|nc 172.31.0.5 2182 |grep followers</span></span><br><span class="line">zk_followers 4</span><br><span class="line">zk_synced_followers 4</span><br></pre></td></tr></table></figure><p>这时可以看到zk_followers为4，zk_synced_followers表示已经同步的follower。</p></li><li><p>依次修改配置<code>myid 1 2 3</code>这三个进程的配置，并重启服务。配置的话，将 <code>server.6 server.7</code> 这两个配置添加到原有的旧zk配置里面。然后按以下规则进行重启：</p><ul><li><code>重启时应保证先重启myid最小的机器，由小向大进行重启。</code></li><li><code>Leader无论其myid大小，都放到最后重启。</code>ZooKeeper的机制中，myid大的会向小的发起连接，而小的不会向大的发起连接。因此如果最后重启myid最小的机器，则其可能无法加入集群</li><li>重启follower节点无影响，当重启leader节点时会触发leader选举，zxid最新的默认优先当选新的leader，当zxid相同，myid最大的优先当选新的leader</li><li>重启完成之后，需要确认这5个zk进程的状态是否是正常的。</li></ul></li></ol><h2 id="缩容"><a href="#缩容" class="headerlink" title="缩容"></a>缩容</h2><p>将5节点缩容为3节点，将<code>myid 1 2</code>停止zk服务下线，假设这两个节点为follower，修改<code>myid 3 6 7</code>配置，按照重启原则重启这个3个节点的zk服务：</p><ol><li><p>直接将myid 1 2的zk服务停止</p></li><li><p>按照重启原则对<code>myid 3 6 7</code>的服务进行重启：</p><ul><li><p>由于现在leader是在<code>myid 7</code>上面，这时就放最后面重启</p></li><li><p>先修改<code>myid 3</code>的zoo.cfg的配置，将server修改为如下，然后重启zk</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">server.3=172.31.0.4:2890:3890</span><br><span class="line">server.6=172.31.0.5:2888:3888</span><br><span class="line">server.7=172.31.0.5:2889:3889</span><br></pre></td></tr></table></figure><ul><li>再修改<code>myid 6</code>的zoo.cfg的配置，重启ZK</li><li>最后面修改<code>myid 7</code>的zoo.cfg的配置，重启ZK</li><li>再次查看状态，可以看到leader已经是在<code>myid 6</code>上面了。</li></ul></li></ol><h2 id="最后一台迁移"><a href="#最后一台迁移" class="headerlink" title="最后一台迁移"></a>最后一台迁移</h2><p>按上面的方式，先扩容，再缩容即可。一定要按照重启原则进行重启。注意，最后迁移时，只用4个进程，不会受影响。</p><blockquote><p><a href="https://nurhier.github.io/2017/07/19/migrate-zookeeper/index.html" target="_blank" rel="noopener">https://nurhier.github.io/2017/07/19/migrate-zookeeper/index.html</a></p><p><a href="https://cloud.tencent.com/developer/article/1406910" target="_blank" rel="noopener">https://cloud.tencent.com/developer/article/1406910</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;线上环境需要做迁移，所以就需要动手了。&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="zookeeper" scheme="https://www.wumingx.com/tags/zookeeper/"/>
    
  </entry>
  
  <entry>
    <title>rocketMQ集群化部署以及常见运维命令</title>
    <link href="https://www.wumingx.com/linux/rocketMQ.html"/>
    <id>https://www.wumingx.com/linux/rocketMQ.html</id>
    <published>2021-06-18T09:14:22.000Z</published>
    <updated>2021-06-18T09:48:38.008Z</updated>
    
    <content type="html"><![CDATA[<h1 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h1><p>打开 <a href="http://rocketmq.apache.org/docs/quick-start/" target="_blank" rel="noopener">官方Quick Start</a> 下载带bin版本的rocketmq，这样就不需要再次编译安装了，可以直接使用。如果下载的是带source的下载包的话，则必须使用 <code>mvn -Prelease-all -DskipTests clean install -U</code> 来打包。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://mirrors.bfsu.edu.cn/apache/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zip</span><br></pre></td></tr></table></figure><p>然后解压到指定目录即可。如果只要单机启动的话，就比较简单，运行以下命令即可：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">nohup sh bin/mqnamesrv &amp;</span><br><span class="line">nohup sh bin/mqbroker -c ./conf/broker.conf &amp;</span><br></pre></td></tr></table></figure><a id="more"></a><h1 id="集群部署配置"><a href="#集群部署配置" class="headerlink" title="集群部署配置"></a>集群部署配置</h1><h2 id="模式介绍"><a href="#模式介绍" class="headerlink" title="模式介绍"></a>模式介绍</h2><p>一般部署方法有以下几种：</p><ul><li>单Master模式：这种方式风险较大，一旦Broker重启或者宕机时，会导致整个服务不可用。不建议线上环境使用,可以用于本地测试。</li><li>多Master模式：一个集群无Slave，全是Master，例如2个Master或者3个Master，这种模式的优缺点如下：<ul><li>优点：配置简单，单个Master宕机或重启维护对应用无影响，在磁盘配置为RAID10时，即使机器宕机不可恢复情况下，由于RAID10磁盘非常可靠，消息也不会丢（异步刷盘丢失少量消息，同步刷盘一条不丢），性能最高；</li><li>缺点：单台机器宕机期间，这台机器上未被消费的消息在机器恢复之前不可订阅，消息实时性会受到影响。</li></ul></li><li>多Master多Slave模式-异步复制：每个Master配置一个Slave，有多对Master-Slave，HA采用异步复制方式，主备有短暂消息延迟（毫秒级），这种模式的优缺点如下：<ul><li>优点：即使磁盘损坏，消息丢失的非常少，且消息实时性不会受影响，同时Master宕机后，消费者仍然可以从Slave消费，而且此过程对应用透明，不需要人工干预，性能同多Master模式几乎一样；</li><li>缺点：Master宕机，磁盘损坏情况下会丢失少量消息。</li></ul></li><li>多Master多Slave模式-同步双写：每个Master配置一个Slave，有多对Master-Slave，HA采用同步双写方式，即只有主备都写成功，才向应用返回成功，这种模式的优缺点如下：<ul><li>优点：数据与服务都无单点故障，Master宕机情况下，消息无延迟，服务可用性与数据可用性都非常高；</li><li>缺点：性能比异步复制模式略低（大约低10%左右），发送单个消息的RT会略高，且目前版本在主节点宕机后，备机不能自动切换为主机。</li></ul></li><li>Dledger集群：<code>RocketMQ-on-DLedger Group</code> 是指一组<strong>「相同名称的 Broker」</strong>，至少需要 3 个节点，通过 <strong>「Raft」</strong> 自动选举出一个 Leader，其余节点作为 Follower，并在 Leader 和 Follower 之间复制数据以保证高可用。</li></ul><h2 id="同步双写"><a href="#同步双写" class="headerlink" title="同步双写"></a>同步双写</h2><p>在conf目录下有以下目录，分别代表了不同的模式，<code>2m-2s-async</code> 表示多Master多Slave模式-异步复制，<code>2m-2s-sync</code> 表示多Master多Slave模式-同步复制，<code>2m-noslave</code> 表示多Master，<code>broker.conf</code> 表示单机配置，还有一种是dledger是自动选举模式，必须三台机器</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># ll conf/</span></span><br><span class="line">total 36</span><br><span class="line">drwxr-xr-x 2 root root   118 Oct 23  2020 2m-2s-async</span><br><span class="line">drwxr-xr-x 2 root root   118 Jun  8 17:45 2m-2s-sync</span><br><span class="line">drwxr-xr-x 2 root root    91 Dec  4  2020 2m-noslave</span><br><span class="line">-rw-r--r-- 1 root root   949 Oct 23  2020 broker.conf</span><br><span class="line">drwxr-xr-x 2 root root    72 Dec  4  2020 dledger</span><br></pre></td></tr></table></figure><p>我们以同步双写的方法来配置。</p><h3 id="配置解析"><a href="#配置解析" class="headerlink" title="配置解析"></a>配置解析</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#mq-master01节点配置/data/rocketmq/conf/2m-2s-sync/broker-a.properties</span></span><br><span class="line">[root@mq-master01 ~]<span class="comment"># vim /data/rocketmq/conf/2m-2s-sync/broker-a.properties</span></span><br><span class="line"><span class="comment">#所属集群名字</span></span><br><span class="line">brokerClusterName=rocketmq-cluster</span><br><span class="line"><span class="comment">#broker名字，注意此处不同的配置文件填写的不一样  例如：在a.properties 文件中写 broker-a  在b.properties 文件中写 broker-b</span></span><br><span class="line">brokerName=broker<span class="_">-a</span></span><br><span class="line"><span class="comment">#0 表示 Master，&gt;0 表示 Slave</span></span><br><span class="line">brokerId=0</span><br><span class="line"><span class="comment">#nameServer地址，这里nameserver是单台，如果nameserver是多台集群的话，就用分号分割（即namesrvAddr=ip1:port1;ip2:port2;ip3:port3）</span></span><br><span class="line">namesrvAddr=192.168.10.207:9876;</span><br><span class="line"><span class="comment">#在发送消息时，自动创建服务器不存在的topic，默认创建的队列数。由于是4个broker节点，所以设置为4</span></span><br><span class="line">defaultTopicQueueNums=4</span><br><span class="line"><span class="comment">#是否允许 Broker 自动创建Topic，建议线下开启，线上关闭</span></span><br><span class="line">autoCreateTopicEnable=<span class="literal">true</span></span><br><span class="line"><span class="comment">#是否允许 Broker 自动创建订阅组，建议线下开启，线上关闭</span></span><br><span class="line">autoCreateSubscriptionGroup=<span class="literal">true</span></span><br><span class="line"><span class="comment">#Broker 对外服务的监听端口</span></span><br><span class="line">listenPort=10911</span><br><span class="line"><span class="comment">#删除文件时间点，默认凌晨 4点</span></span><br><span class="line">deleteWhen=04</span><br><span class="line"><span class="comment">#文件保留时间，默认 48 小时</span></span><br><span class="line">fileReservedTime=120</span><br><span class="line"><span class="comment">#commitLog每个文件的大小默认1G</span></span><br><span class="line">mapedFileSizeCommitLog=1073741824</span><br><span class="line"><span class="comment">#ConsumeQueue每个文件默认存30W条，根据业务情况调整</span></span><br><span class="line">mapedFileSizeConsumeQueue=300000</span><br><span class="line"><span class="comment">#destroyMapedFileIntervalForcibly=120000</span></span><br><span class="line"><span class="comment">#redeleteHangedFileInterval=120000</span></span><br><span class="line"><span class="comment">#检测物理文件磁盘空间</span></span><br><span class="line">diskMaxUsedSpaceRatio=88</span><br><span class="line"><span class="comment">#存储路径</span></span><br><span class="line">storePathRootDir=/data/rocketmq/store</span><br><span class="line"><span class="comment">#commitLog 存储路径</span></span><br><span class="line">storePathCommitLog=/data/rocketmq/store/commitlog</span><br><span class="line"><span class="comment">#消费队列存储路径存储路径</span></span><br><span class="line">storePathConsumeQueue=/data/rocketmq/store/consumequeue</span><br><span class="line"><span class="comment">#消息索引存储路径</span></span><br><span class="line">storePathIndex=/data/rocketmq/store/index</span><br><span class="line"><span class="comment">#checkpoint 文件存储路径</span></span><br><span class="line">storeCheckpoint=/data/rocketmq/store/checkpoint</span><br><span class="line"><span class="comment">#abort 文件存储路径</span></span><br><span class="line">abortFile=/data/rocketmq/store/abort</span><br><span class="line"><span class="comment">#限制的消息大小</span></span><br><span class="line">maxMessageSize=65536</span><br><span class="line"><span class="comment">#flushCommitLogLeastPages=4</span></span><br><span class="line"><span class="comment">#flushConsumeQueueLeastPages=2</span></span><br><span class="line"><span class="comment">#flushCommitLogThoroughInterval=10000</span></span><br><span class="line"><span class="comment">#flushConsumeQueueThoroughInterval=60000</span></span><br><span class="line"><span class="comment">#Broker 的角色</span></span><br><span class="line"><span class="comment">#- ASYNC_MASTER 异步复制Master</span></span><br><span class="line"><span class="comment">#- SYNC_MASTER 同步双写Master</span></span><br><span class="line"><span class="comment">#- SLAVE</span></span><br><span class="line">brokerRole=SYNC_MASTER                       <span class="comment">#要配置为MASTER或SLAVE的角色</span></span><br><span class="line"><span class="comment">#刷盘方式</span></span><br><span class="line"><span class="comment">#- ASYNC_FLUSH 异步刷盘</span></span><br><span class="line"><span class="comment">#- SYNC_FLUSH 同步刷盘</span></span><br><span class="line">flushDiskType=SYNC_FLUSH</span><br><span class="line"><span class="comment">#checkTransactionMessageEnable=false</span></span><br><span class="line"><span class="comment">#发消息线程池数量</span></span><br><span class="line"><span class="comment">#sendMessageThreadPoolNums=128</span></span><br><span class="line"><span class="comment">#拉消息线程池数量</span></span><br><span class="line"><span class="comment">#pullMessageThreadPoolNums=128</span></span><br><span class="line"> </span><br><span class="line"><span class="comment">#mq-master02节点配置的是/data/rocketmq/conf/2m-2s-sync/broker-b.properties      #就下面三行配置不一样，其他配置行都一样！</span></span><br><span class="line">[root@mq-master02 software]<span class="comment"># vim /data/rocketmq/conf/2m-2s-sync/broker-b.properties     </span></span><br><span class="line">......</span><br><span class="line">brokerName=broker-b           </span><br><span class="line">brokerId=0</span><br><span class="line">brokerRole=MASTER  </span><br><span class="line"> </span><br><span class="line"><span class="comment">#mq-slave01节点配置的是/data/rocketmq/conf/2m-2s-sync/broker-a-s.properties</span></span><br><span class="line">[root@mq-slave01 software]<span class="comment"># vim /data/rocketmq/conf/2m-2s-sync/broker-a-s.properties</span></span><br><span class="line">......</span><br><span class="line">brokerName=broker<span class="_">-a</span>          <span class="comment">#注意这一行的名称要和master保持一致</span></span><br><span class="line">brokerId=1                   <span class="comment">#这个ID要跟master的不一致！</span></span><br><span class="line">brokerRole=SLAVE             <span class="comment">#要配置为从</span></span><br><span class="line"> </span><br><span class="line"><span class="comment">#mq-slave02节点配置的是/data/rocketmq/conf/2m-2s-sync/broker-b-s.properties</span></span><br><span class="line">[root@mq-slave02 software]<span class="comment"># vim /data/rocketmq/conf/2m-2s-sync/broker-b-s.properties</span></span><br><span class="line">......</span><br><span class="line">brokerName=broker-b         <span class="comment">#注意这一行的名称要和master的保持一致</span></span><br><span class="line">brokerId=1                  <span class="comment">#这个ID要跟master的不一致</span></span><br><span class="line">brokerRole=SLAVE            <span class="comment">#要配置为从</span></span><br></pre></td></tr></table></figure><h3 id="详细配置"><a href="#详细配置" class="headerlink" title="详细配置"></a>详细配置</h3><p>以下是我的环境，2台机器互为主备，这边使用配置文件进行区分，实际上2台机器上面的配置文件名是可以一样的。</p><div class="table-container"><table><thead><tr><th>配制文件</th><th>IP</th><th>角色</th></tr></thead><tbody><tr><td>broker-a.properties</td><td>172.31.0.21</td><td>SYNC_MASTER</td></tr><tr><td>broker-a-s.properties</td><td>172.31.0.22</td><td>SLAVE</td></tr><tr><td>broker-b.properties</td><td>172.31.0.22</td><td>SYNC_MASTER</td></tr><tr><td>broker-b-s.properties</td><td>172.31.0.21</td><td>SLAVE</td></tr></tbody></table></div><p>以下是  172.31.0.21 的配置文件，主备关系是通过<code>brokerName+brokerId+brokerRole</code>来确认的，实际上配置文件都是差不多的。同时注意目录路径是有区别的：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line">[root@8fptpfxk957bjm 2m-2s-sync]<span class="comment"># cat broker-a.properties </span></span><br><span class="line">brokerClusterName=DefaultCluster</span><br><span class="line">brokerName=broker<span class="_">-a</span></span><br><span class="line">brokerId=0</span><br><span class="line">deleteWhen=04</span><br><span class="line">fileReservedTime=48</span><br><span class="line">brokerRole=SYNC_MASTER</span><br><span class="line">flushDiskType=ASYNC_FLUSH</span><br><span class="line"></span><br><span class="line">autoCreateTopicEnable=<span class="literal">true</span></span><br><span class="line">autoCreateSubscriptionGroup=<span class="literal">true</span></span><br><span class="line">listenPort=10911</span><br><span class="line">haListenPort=10912</span><br><span class="line">mapedFileSizeCommitLog=1073741824</span><br><span class="line">mapedFileSizeConsumeQueue=300000</span><br><span class="line">diskMaxUsedSpaceRatio=88</span><br><span class="line">storePathRootDir=/usr/<span class="built_in">local</span>/rocketmq/data/broker<span class="_">-a</span></span><br><span class="line">storePathCommitLog=/usr/<span class="built_in">local</span>/rocketmq/data/broker<span class="_">-a</span>/commitlog</span><br><span class="line">maxMessageSize=65536</span><br><span class="line">brokerIP1=172.31.0.21</span><br><span class="line">namesrvAddr=172.31.0.21:9876;172.31.0.22:9876</span><br><span class="line">[root@8fptpfxk957bjm 2m-2s-sync]<span class="comment"># </span></span><br><span class="line">[root@8fptpfxk957bjm 2m-2s-sync]<span class="comment"># cat broker-b-s.properties </span></span><br><span class="line">brokerClusterName=DefaultCluster</span><br><span class="line">brokerName=broker-b</span><br><span class="line">brokerId=1</span><br><span class="line">deleteWhen=04</span><br><span class="line">fileReservedTime=48</span><br><span class="line">brokerRole=SLAVE</span><br><span class="line">flushDiskType=ASYNC_FLUSH</span><br><span class="line"></span><br><span class="line">autoCreateTopicEnable=<span class="literal">true</span></span><br><span class="line">autoCreateSubscriptionGroup=<span class="literal">true</span></span><br><span class="line">listenPort=10923</span><br><span class="line">haListenPort=10924</span><br><span class="line">mapedFileSizeCommitLog=1073741824</span><br><span class="line">mapedFileSizeConsumeQueue=300000</span><br><span class="line">diskMaxUsedSpaceRatio=88</span><br><span class="line">storePathRootDir=/usr/<span class="built_in">local</span>/rocketmq/data/broker-b<span class="_">-s</span></span><br><span class="line">storePathCommitLog=/usr/<span class="built_in">local</span>/rocketmq/data/broker-b<span class="_">-s</span>/commitlog</span><br><span class="line">maxMessageSize=65536</span><br><span class="line">brokerIP1=172.31.0.21</span><br><span class="line">namesrvAddr=172.31.0.21:9876;172.31.0.22:9876</span><br></pre></td></tr></table></figure><p>以下是  172.31.0.22 的配置文件，</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br></pre></td><td class="code"><pre><span class="line">[root@27zrgari9hd937 2m-2s-sync]<span class="comment"># cat broker-b.properties </span></span><br><span class="line">brokerClusterName=DefaultCluster</span><br><span class="line">brokerName=broker-b</span><br><span class="line">brokerId=0</span><br><span class="line">deleteWhen=04</span><br><span class="line">fileReservedTime=48</span><br><span class="line">brokerRole=SYNC_MASTER</span><br><span class="line">flushDiskType=ASYNC_FLUSH</span><br><span class="line"></span><br><span class="line">autoCreateTopicEnable=<span class="literal">true</span></span><br><span class="line">autoCreateSubscriptionGroup=<span class="literal">true</span></span><br><span class="line">listenPort=10911</span><br><span class="line">haListenPort=10912</span><br><span class="line">mapedFileSizeCommitLog=1073741824</span><br><span class="line">mapedFileSizeConsumeQueue=300000</span><br><span class="line">diskMaxUsedSpaceRatio=88</span><br><span class="line">storePathRootDir=/usr/<span class="built_in">local</span>/rocketmq/data/broker-b</span><br><span class="line">storePathCommitLog=/usr/<span class="built_in">local</span>/rocketmq/data/broker-b/commitlog</span><br><span class="line">maxMessageSize=65536</span><br><span class="line">brokerIP1=172.31.0.22</span><br><span class="line">namesrvAddr=172.31.0.21:9876;172.31.0.22:9876</span><br><span class="line">[root@27zrgari9hd937 2m-2s-sync]<span class="comment">#  </span></span><br><span class="line">[root@27zrgari9hd937 2m-2s-sync]<span class="comment"># </span></span><br><span class="line">[root@27zrgari9hd937 2m-2s-sync]<span class="comment"># cat broker-a-s.properties </span></span><br><span class="line">brokerClusterName=DefaultCluster</span><br><span class="line">brokerName=broker<span class="_">-a</span></span><br><span class="line">brokerId=1</span><br><span class="line">deleteWhen=04</span><br><span class="line">fileReservedTime=48</span><br><span class="line">brokerRole=SLAVE</span><br><span class="line">flushDiskType=ASYNC_FLUSH</span><br><span class="line"></span><br><span class="line">autoCreateTopicEnable=<span class="literal">true</span></span><br><span class="line">autoCreateSubscriptionGroup=<span class="literal">true</span></span><br><span class="line">listenPort=10923</span><br><span class="line">haListenPort=10924</span><br><span class="line">mapedFileSizeCommitLog=1073741824</span><br><span class="line">mapedFileSizeConsumeQueue=300000</span><br><span class="line">diskMaxUsedSpaceRatio=88</span><br><span class="line">storePathRootDir=/usr/<span class="built_in">local</span>/rocketmq/data/broker<span class="_">-a</span><span class="_">-s</span></span><br><span class="line">storePathCommitLog=/usr/<span class="built_in">local</span>/rocketmq/data/broker<span class="_">-a</span><span class="_">-s</span>/commitlog</span><br><span class="line">maxMessageSize=65536</span><br><span class="line">brokerIP1=172.31.0.22</span><br><span class="line">namesrvAddr=172.31.0.21:9876;172.31.0.22:9876</span><br></pre></td></tr></table></figure><p>同时要注意配置storePathRootDir以及storePathCommitLog的路径。</p><h3 id="java启动配置"><a href="#java启动配置" class="headerlink" title="java启动配置"></a>java启动配置</h3><p>官方默认的启动脚本所使用的内存比较大，由于是测试环境，需要修改一下：</p><ul><li><p>bin/mqnamesrv：对应的脚本是 <code>sh ${ROCKETMQ_HOME}/bin/runserver.sh</code> ，按以下方式修改：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 修改内存大小</span></span><br><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -server -Xms1g -Xmx1g -Xmn512m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"</span> </span><br><span class="line"></span><br><span class="line"><span class="comment"># 默认的日志是配置是在 $user.home/logs/rocketmqlogs/下，需要修改：</span></span><br><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -Dons.client.logRoot=/usr/local/rocketmq"</span></span><br><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -Duser.home=/usr/local/rocketmq"</span></span><br></pre></td></tr></table></figure></li><li><p>bin/mqbroker：对应的脚本是<code>sh ${ROCKETMQ_HOME}/bin/runbroker.sh</code>:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -server -Xms2g -Xmx2g -Xmn2g"</span></span><br><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -Dons.client.logRoot=/usr/local/rocketmq"</span></span><br><span class="line">JAVA_OPT=<span class="string">"<span class="variable">$&#123;JAVA_OPT&#125;</span> -Duser.home=/usr/local/rocketmq"</span></span><br></pre></td></tr></table></figure></li></ul><h3 id="启动脚本"><a href="#启动脚本" class="headerlink" title="启动脚本"></a>启动脚本</h3><p>nohup方式管理的方法如下：</p><ul><li>nameserver</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">启动：<span class="built_in">cd</span> /usr/<span class="built_in">local</span>/rocketmq/bin &amp;&amp; nohup ./mqnamesrv &amp;</span><br><span class="line">停止：<span class="built_in">cd</span> /usr/<span class="built_in">local</span>/rocketmq/bin &amp;&amp; sh mqshutdown namesrv</span><br></pre></td></tr></table></figure><ul><li>broker</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">启动：cd /usr/local/rocketmq/ &amp;&amp; nohup sh bin/mqbroker -c conf/2m-2s-sync/broker-a.properties &amp;</span><br><span class="line">停止：cd /usr/local/rocketmq/ &amp;&amp; sh bin/mqshutdown broker</span><br></pre></td></tr></table></figure><p>比较麻烦，还是通过systemctl来接管。</p><p>根据实际的情况修改以下配置：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line">[root@8fptpfxk957bjm 2m-2s-sync]<span class="comment"># systemctl cat rocketmq-namesrv.service </span></span><br><span class="line"><span class="comment"># /usr/lib/systemd/system/rocketmq-namesrv.service</span></span><br><span class="line">[Unit]</span><br><span class="line">Description=rocketmq-namesrv</span><br><span class="line">After=network.target</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=simple </span><br><span class="line">ExecStart=/usr/<span class="built_in">local</span>/rocketmq/bin/mqnamesrv</span><br><span class="line">ExecStop=/usr/<span class="built_in">local</span>/rocketmq/bin/mqshutdown namesrv</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br><span class="line"></span><br><span class="line">[root@8fptpfxk957bjm 2m-2s-sync]<span class="comment"># systemctl cat rocketmq-broker-b-s</span></span><br><span class="line"><span class="comment"># /usr/lib/systemd/system/rocketmq-broker-b-s.service</span></span><br><span class="line">[Unit]</span><br><span class="line">Description=rocketmq-broker-b<span class="_">-s</span></span><br><span class="line">After=network.target</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=simple </span><br><span class="line"></span><br><span class="line">ExecStart=/usr/<span class="built_in">local</span>/rocketmq/bin/mqbroker -c /usr/<span class="built_in">local</span>/rocketmq/conf/2m-2s-sync/broker-b-s.properties</span><br><span class="line">ExecStop=/usr/<span class="built_in">local</span>/rocketmq/bin/mqshutdown broker</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br></pre></td></tr></table></figure><p>然后start相对应的服务即可。</p><h3 id="常用命令"><a href="#常用命令" class="headerlink" title="常用命令"></a>常用命令</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 生产消息</span></span><br><span class="line"><span class="built_in">export</span> NAMESRV_ADDR=localhost:9876</span><br><span class="line">sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer</span><br><span class="line"> SendResult [sendStatus=SEND_OK, msgId= ...</span><br><span class="line"></span><br><span class="line"><span class="comment"># 消费消息</span></span><br><span class="line">sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer</span><br><span class="line"> ConsumeMessageThread_%d Receive New Messages: [MessageExt...</span><br><span class="line"> </span><br><span class="line"><span class="comment"># broker的写权限关闭</span></span><br><span class="line">bin/mqadmin updateBrokerConfig -b 192.168.x.x:10911 -n 192.168.x.x:9876 -k brokerPermission -v 4</span><br><span class="line"></span><br><span class="line"><span class="comment"># 恢复该节点的写权限</span></span><br><span class="line">bin/mqadmin updateBrokerConfig -b 192.168.x.x:10911 -n 192.168.x.x:9876 -k brokerPermission -v 6</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看集群信息，集群、BrokerName、BrokerId、TPS等信息</span></span><br><span class="line">./bin/mqadmin clusterList -n localhost:9876</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取全部topic</span></span><br><span class="line">./bin/mqadmin topicList -n localhost:9876 -c DevCluster &gt; topiclist</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取topic 路由信息</span></span><br><span class="line">./bin/mqadmin topicRoute -t demo-cluster -n localhost:9876</span><br><span class="line"></span><br><span class="line"><span class="comment"># 获取topic offset</span></span><br><span class="line">./bin/mqadmin topicStatus -t demo-cluster -n localhost:9876</span><br><span class="line"></span><br><span class="line"><span class="comment"># 打印Topic订阅关系、TPS、积累量、24h读写总量等信息</span></span><br><span class="line">./bin/mqadmin statsAll  -n localhost:9876</span><br><span class="line"></span><br><span class="line"><span class="comment"># 修改broker 参数</span></span><br><span class="line">./bin/mqadmin updateBrokerConfig -n localhost:9876 -b 10.0.xxx.2:10911 -k waitTimeMillsInSendQueue -v 500 -c TestCluster</span><br><span class="line"></span><br><span class="line"><span class="comment"># 发送消息</span></span><br><span class="line">./bin/mqadmin sendMessage -n localhost:9876 -t <span class="built_in">test</span> -p <span class="string">"this is test"</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 消费</span></span><br><span class="line">./bin/mqadmin consumeMessage -n localhost:9876 -t <span class="built_in">test</span></span><br></pre></td></tr></table></figure><p>有关 mqadmin 的命令说明，可以访问： <a href="https://github.com/apache/rocketmq/blob/master/docs/cn/operation.md" target="_blank" rel="noopener">mqadmin管理工具</a></p><h1 id="⁨rocketmq-console部署"><a href="#⁨rocketmq-console部署" class="headerlink" title="⁨rocketmq-console部署"></a>⁨rocketmq-console部署</h1><p>rocketmq-console是可以管理rocketmq集群的，先下载rocketmq-externals</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git <span class="built_in">clone</span> https://github.com/apache/rocketmq-externals.git</span><br></pre></td></tr></table></figure><p>下载会比较慢，自己想办法解决：</p><p>修改‎⁨⁨rocketmq-externals⁩/⁨rocketmq-console⁩/⁨src⁩/⁨main⁩/⁨resources⁩/application.properties</p><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">server.address</span>=<span class="number">0.0</span>.<span class="number">0.0</span></span><br><span class="line"><span class="attr">server.port</span>=<span class="number">8088</span></span><br><span class="line"><span class="attr">rocketmq.config.namesrvAddr</span>=<span class="number">127.0</span>.<span class="number">0.1</span>:<span class="number">9876</span></span><br></pre></td></tr></table></figure><p>修改‎⁨⁨rocketmq-externals⁩/⁨rocketmq-console/pom.xml，与服务端版本一致</p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">&lt;rocketmq.version&gt;4.7.1&lt;/rocketmq.version&gt;</span><br></pre></td></tr></table></figure><p>打包, 打包期间报错修改pom.xml中的jar 版本</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> rocketmq-externals/rocketmq-console</span><br><span class="line">mvn clean package -Dmaven.test.skip=<span class="literal">true</span></span><br></pre></td></tr></table></figure><p>启动 <code>java -jar target/rocketmq-console-ng-2.0.0.jar</code>，访问 <a href="http://localhost:8088" target="_blank" rel="noopener">http://localhost:8088</a> 即可。</p><p>如果想自定义参数，可以直接使用配置文件来做自定义：</p><ul><li>users.properties，配置控制台用户名和密码。</li></ul><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># This file supports hot change, any change will be auto-reloaded without Console restarting.</span></span><br><span class="line"><span class="comment"># Format: a user per line, username=password[,N] #N is optional, 0 (Normal User); 1 (Admin)</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Define Admin</span></span><br><span class="line"><span class="attr">admin</span>=admin,<span class="number">1</span></span><br></pre></td></tr></table></figure><ul><li><p>application.properties：配置访问端口</p><figure class="highlight ini"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">server.contextPath</span>=</span><br><span class="line"><span class="attr">server.port</span>=<span class="number">8088</span></span><br><span class="line"></span><br><span class="line"><span class="comment">### SSL setting</span></span><br><span class="line"><span class="comment">#server.ssl.key-store=classpath:rmqcngkeystore.jks</span></span><br><span class="line"><span class="comment">#server.ssl.key-store-password=rocketmq</span></span><br><span class="line"><span class="comment">#server.ssl.keyStoreType=PKCS12</span></span><br><span class="line"><span class="comment">#server.ssl.keyAlias=rmqcngkey</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#spring.application.index=true</span></span><br><span class="line"><span class="attr">spring.application.name</span>=rocketmq-console</span><br><span class="line"><span class="attr">spring.http.encoding.charset</span>=UTF-<span class="number">8</span></span><br><span class="line"><span class="attr">spring.http.encoding.enabled</span>=<span class="literal">true</span></span><br><span class="line"><span class="attr">spring.http.encoding.force</span>=<span class="literal">true</span></span><br><span class="line"><span class="attr">logging.config</span>=classpath:logback.xml</span><br><span class="line"><span class="comment">#if this value is empty,use env value rocketmq.config.namesrvAddr  NAMESRV_ADDR | now, you can set it in ops page.default localhost:9876</span></span><br><span class="line"><span class="attr">rocketmq.config.namesrvAddr</span>=<span class="number">127.0</span>.<span class="number">0.1</span>:<span class="number">9876</span></span><br><span class="line"><span class="comment">#if you use rocketmq version &lt; 3.5.8, rocketmq.config.isVIPChannel should be false.default true</span></span><br><span class="line"><span class="attr">rocketmq.config.isVIPChannel</span>=</span><br><span class="line"><span class="comment">#rocketmq-console's data path:dashboard/monitor</span></span><br><span class="line"><span class="attr">rocketmq.config.dataPath</span>=/usr/local/rocketmq/data/broker-a</span><br><span class="line"><span class="comment">#set it false if you don't want use dashboard.default true</span></span><br><span class="line"><span class="attr">rocketmq.config.enableDashBoardCollect</span>=<span class="literal">true</span></span><br><span class="line"><span class="comment">#set the message track trace topic if you don't want use the default one</span></span><br><span class="line"><span class="attr">rocketmq.config.msgTrackTopicName</span>=</span><br><span class="line"><span class="attr">rocketmq.config.ticketKey</span>=ticket</span><br><span class="line"></span><br><span class="line"><span class="comment">#Must create userInfo file: $&#123;rocketmq.config.dataPath&#125;/users.properties if the login is required</span></span><br><span class="line"><span class="attr">rocketmq.config.loginRequired</span>=<span class="literal">true</span></span><br></pre></td></tr></table></figure></li></ul><p>然后使用 <code>java -jar rocketmq-console-ng-2.0.0.jar -Dspring.config.location=./</code> 来启动。</p><p>使用方法，请参考 <a href="https://blog.csdn.net/ccgshigao/article/details/108765838" target="_blank" rel="noopener">RocketMQ系列：rocketmq运维控制台使用详解（全网独家）</a></p><h1 id="mqadmin详解"><a href="#mqadmin详解" class="headerlink" title="mqadmin详解"></a>mqadmin详解</h1><p>为了方便命令运行，将 <code>export PATH=$PATH:/usr/local/rocketmq/bin/</code>加入到/etc/profile。</p><h2 id="集群状态查看"><a href="#集群状态查看" class="headerlink" title="集群状态查看"></a>集群状态查看</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 集群状态</span></span><br><span class="line">[root@8fptpfxk957bjm bin]<span class="comment"># sh mqadmin clusterList 2&gt;/dev/null </span></span><br><span class="line"><span class="comment">#Cluster Name     #Broker Name            #BID  #Addr                  #Version                #InTPS(LOAD)       #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE</span></span><br><span class="line">DefaultCluster    broker<span class="_">-a</span>                0     172.31.0.21:10911      V4_8_0                   0.00(0,0ms)         0.00(0,0ms)          0 450874.19 0.0095</span><br><span class="line">DefaultCluster    broker<span class="_">-a</span>                1     172.31.0.22:10923      V4_8_0                   0.00(0,0ms)         0.00(0,0ms)          0 450874.19 0.0110</span><br><span class="line">DefaultCluster    broker-b                0     172.31.0.22:10911      V4_8_0                   0.00(0,0ms)         0.00(0,0ms)          0 450874.19 0.0110</span><br><span class="line">DefaultCluster    broker-b                1     172.31.0.21:10923      V4_8_0                   0.00(0,0ms)         0.00(0,0ms)          0 450874.19 0.0095</span><br></pre></td></tr></table></figure><h2 id="topic相关命令"><a href="#topic相关命令" class="headerlink" title="topic相关命令"></a>topic相关命令</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># topicList</span></span><br><span class="line">[root@8fptpfxk957bjm ~]<span class="comment"># mqadmin topicList -n localhost:9876 2&gt;/dev/null |egrep -v "OFFSET_MOVED_EVENT|%RETRY%|BenchmarkTest|broker-|DefaultCluster|SELF_TEST_TOPIC" </span></span><br><span class="line">RMQ_SYS_TRANS_HALF_TOPIC</span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">TBW102</span><br><span class="line">SCHEDULE_TOPIC_XXXX</span><br><span class="line">TopicTest</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看 Topic 消息队列offset</span></span><br><span class="line">[root@8fptpfxk957bjm bin]<span class="comment"># sh mqadmin topicStatus -t TopicTest 2&gt;/dev/null </span></span><br><span class="line"><span class="comment">#Broker Name                      #QID  #Min Offset           #Max Offset             #Last Updated</span></span><br><span class="line">broker<span class="_">-a</span>                          0     0                     125                     2021-06-08 17:19:50,639</span><br><span class="line">broker<span class="_">-a</span>                          1     0                     125                     2021-06-08 17:19:50,640</span><br><span class="line">broker<span class="_">-a</span>                          2     0                     125                     2021-06-08 17:19:50,641</span><br><span class="line">broker<span class="_">-a</span>                          3     0                     125                     2021-06-08 17:19:50,641</span><br><span class="line">broker-b                          0     0                     125                     2021-06-08 17:19:49,100</span><br><span class="line">broker-b                          1     0                     125                     2021-06-08 17:19:49,100</span><br><span class="line">broker-b                          2     0                     125                     2021-06-08 17:19:49,101</span><br><span class="line">broker-b                          3     0                     125                     2021-06-08 17:19:49,102</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看 Topic 路由信息，数据保存在哪个broker上，读写队列是多少等信息</span></span><br><span class="line">[root@8fptpfxk957bjm bin]<span class="comment"># sh mqadmin topicRoute -t TopicTest 2&gt;/dev/null </span></span><br><span class="line">&#123;</span><br><span class="line"><span class="string">"brokerDatas"</span>:[</span><br><span class="line">&#123;</span><br><span class="line"><span class="string">"brokerAddrs"</span>:&#123;0:<span class="string">"172.31.0.22:10911"</span>,1:<span class="string">"172.31.0.21:10923"</span></span><br><span class="line">&#125;,</span><br><span class="line"><span class="string">"brokerName"</span>:<span class="string">"broker-b"</span>,</span><br><span class="line"><span class="string">"cluster"</span>:<span class="string">"DefaultCluster"</span></span><br><span class="line">&#125;,</span><br><span class="line">&#123;</span><br><span class="line"><span class="string">"brokerAddrs"</span>:&#123;0:<span class="string">"172.31.0.21:10911"</span>,1:<span class="string">"172.31.0.22:10923"</span></span><br><span class="line">&#125;,</span><br><span class="line"><span class="string">"brokerName"</span>:<span class="string">"broker-a"</span>,</span><br><span class="line"><span class="string">"cluster"</span>:<span class="string">"DefaultCluster"</span></span><br><span class="line">&#125;</span><br><span class="line">],</span><br><span class="line"><span class="string">"filterServerTable"</span>:&#123;&#125;,</span><br><span class="line"><span class="string">"queueDatas"</span>:[</span><br><span class="line">&#123;</span><br><span class="line"><span class="string">"brokerName"</span>:<span class="string">"broker-b"</span>,</span><br><span class="line"><span class="string">"perm"</span>:6,</span><br><span class="line"><span class="string">"readQueueNums"</span>:4,</span><br><span class="line"><span class="string">"topicSynFlag"</span>:0,</span><br><span class="line"><span class="string">"writeQueueNums"</span>:4</span><br><span class="line">&#125;,</span><br><span class="line">&#123;</span><br><span class="line"><span class="string">"brokerName"</span>:<span class="string">"broker-a"</span>,</span><br><span class="line"><span class="string">"perm"</span>:6,</span><br><span class="line"><span class="string">"readQueueNums"</span>:4,</span><br><span class="line"><span class="string">"topicSynFlag"</span>:0,</span><br><span class="line"><span class="string">"writeQueueNums"</span>:4</span><br><span class="line">&#125;</span><br><span class="line">]</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">[root@8fptpfxk957bjm ~]<span class="comment"># mqadmin topicRoute -t TopicTest -n localhost:9876 2&gt;/dev/null |grep readQueueNums</span></span><br><span class="line"><span class="string">"readQueueNums"</span>:4,</span><br><span class="line"><span class="string">"readQueueNums"</span>:4,</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查询TOPIC被哪些Consumer Group订阅了</span></span><br><span class="line">[root@8fptpfxk957bjm bin]<span class="comment"># sh mqadmin statsAll 2&gt;/dev/null </span></span><br><span class="line"><span class="comment">#Topic                            #Consumer Group                  #Accumulation      #InTPS     #OutTPS   #InMsg24Hour  #OutMsg24Hour</span></span><br><span class="line">SCHEDULE_TOPIC_XXXX                                                           0        0.00                          0    NO_CONSUMER</span><br><span class="line">RMQ_SYS_TRANS_HALF_TOPIC                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line"><span class="built_in">test</span>                                                                          0        0.00                          0    NO_CONSUMER</span><br><span class="line">DefaultCluster_REPLY_TOPIC                                                    0        0.00                          0    NO_CONSUMER</span><br><span class="line">broker-b                                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line">BenchmarkTest                                                                 0        0.00                          0    NO_CONSUMER</span><br><span class="line">OFFSET_MOVED_EVENT                                                            0        0.00                          0    NO_CONSUMER</span><br><span class="line">TopicTest                                                                     0        0.00                       1000    NO_CONSUMER</span><br><span class="line">broker<span class="_">-a</span>                                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line">TBW102                                                                        0        0.00                          0    NO_CONSUMER</span><br><span class="line">SELF_TEST_TOPIC                                                               0        0.00                          0    NO_CONSUMER</span><br><span class="line">DefaultCluster                                                                0        0.00                          0    NO_CONSUMER</span><br></pre></td></tr></table></figure><h2 id="消息相关命令"><a href="#消息相关命令" class="headerlink" title="消息相关命令"></a>消息相关命令</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 发送消息</span></span><br><span class="line">[root@8fptpfxk957bjm src]<span class="comment"># mqadmin sendMessage -n localhost:9876 -t test -p "this is test"</span></span><br><span class="line"><span class="comment">#Broker Name                      #QID  #Send Result            #MsgId</span></span><br><span class="line">broker<span class="_">-a</span>                          14    SEND_OK                 7F0000015DFD6FF3C5B52B7569C90000</span><br><span class="line"></span><br><span class="line"><span class="comment"># 消费消息</span></span><br><span class="line">[root@8fptpfxk957bjm src]<span class="comment"># mqadmin consumeMessage -n localhost:9876 -t test</span></span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=15] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=15] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=13] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=13] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=11] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=11] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=9] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=9] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=7] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=7] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=5] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=5] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=3] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=3] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=1] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=1] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">Consume ok</span><br><span class="line">MSGID: 7F0000015DFD6FF3C5B52B7569C90000 MessageExt [brokerName=broker<span class="_">-a</span>, queueId=14, storeSize=182, queueOffset=0, sysFlag=0, bornTimestamp=1623205915082, bornHost=/172.31.0.21:52474, storeTimestamp=1623205915092, storeHost=/172.31.0.21:10911, msgId=AC1F001500002A9F0000000000018C48, commitLogOffset=101448, bodyCRC=4604200, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message&#123;topic=<span class="string">'test'</span>, flag=0, properties=&#123;MIN_OFFSET=0, MAX_OFFSET=1, UNIQ_KEY=7F0000015DFD6FF3C5B52B7569C90000, CLUSTER=DefaultCluster, WAIT=<span class="literal">true</span>&#125;, body=[116, 104, 105, 115, 32, 105, 115, 32, 116, 101, 115, 116], transactionId=<span class="string">'null'</span>&#125;] BODY: this is <span class="built_in">test</span></span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=14] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=1</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=14] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=12] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=12] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=10] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=10] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=8] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=8] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=6] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=6] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=4] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=4] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=2] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=2] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker<span class="_">-a</span>, queueId=0] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line">MessageQueue [topic=<span class="built_in">test</span>, brokerName=broker-b, queueId=0] <span class="built_in">print</span> msg finished. status=NO_NEW_MSG, offset=0</span><br><span class="line"></span><br><span class="line"><span class="comment"># 将读写队列配置为4</span></span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># mqadmin updateTopic -c DefaultCluster -n localhost:9876 -t test -r 4 -w 4 2&gt;/dev/null</span></span><br><span class="line">create topic to 172.31.0.21:10911 success.</span><br><span class="line">create topic to 172.31.0.22:10911 success.</span><br><span class="line">TopicConfig [topicName=<span class="built_in">test</span>, readQueueNums=4, writeQueueNums=4, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=<span class="literal">false</span>]</span><br><span class="line"></span><br><span class="line"><span class="comment"># 打印Topic订阅关系、TPS、积累量、24h读写总量等信息</span></span><br><span class="line">[root@8fptpfxk957bjm src]<span class="comment"># mqadmin statsAll  -n localhost:9876 2&gt;/dev/null</span></span><br><span class="line"><span class="comment">#Topic                            #Consumer Group                  #Accumulation      #InTPS     #OutTPS   #InMsg24Hour  #OutMsg24Hour</span></span><br><span class="line">RMQ_SYS_TRANS_HALF_TOPIC                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line"><span class="built_in">test</span>                                                                          0        0.00                          0    NO_CONSUMER</span><br><span class="line">BenchmarkTest                                                                 0        0.00                          0    NO_CONSUMER</span><br><span class="line">OFFSET_MOVED_EVENT                                                            0        0.00                          0    NO_CONSUMER</span><br><span class="line">TBW102                                                                        0        0.00                          0    NO_CONSUMER</span><br><span class="line">SELF_TEST_TOPIC                                                               0        0.00                          0    NO_CONSUMER</span><br><span class="line">DefaultCluster                                                                0        0.00                          0    NO_CONSUMER</span><br><span class="line">SCHEDULE_TOPIC_XXXX                                                           0        0.00                          0    NO_CONSUMER</span><br><span class="line">DefaultCluster_REPLY_TOPIC                                                    0        0.00                          0    NO_CONSUMER</span><br><span class="line">broker-b                                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line">TopicTest                         please_rename_unique_group_name_          605        0.00        0.00           1000           2586</span><br><span class="line">broker<span class="_">-a</span>                                                                      0        0.00                          0    NO_CONSUMER</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看消费的情况，不带-g是否总体的情况</span></span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># mqadmin consumerProgress -n localhost:9876 2&gt;/dev/null</span></span><br><span class="line"><span class="comment">#Group                            #Count  #Version                 #Type  #Model          #TPS     #Diff Total</span></span><br><span class="line">please_rename_unique_group_name_  0       OFFLINE                                         0        0</span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># </span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看具体consumer的消费情况</span></span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># mqadmin consumerProgress -n localhost:9876 -g please_rename_unique_group_name_4 2&gt;/dev/null </span></span><br><span class="line"><span class="comment">#Topic                            #Broker Name                      #QID  #Broker Offset        #Consumer Offset      #Client IP           #Diff                 #LastTime</span></span><br><span class="line">%RETRY%please_rename_unique_grou  broker<span class="_">-a</span>                          0     0                     0                     N/A                  0                     N/A</span><br><span class="line">%RETRY%please_rename_unique_grou  broker-b                          0     0                     0                     N/A                  0                     N/A</span><br><span class="line">TopicTest                         broker<span class="_">-a</span>                          0     125                   125                   N/A                  0                     2021-06-08 17:19:50</span><br><span class="line">TopicTest                         broker<span class="_">-a</span>                          1     125                   125                   N/A                  0                     2021-06-08 17:19:50</span><br><span class="line">TopicTest                         broker<span class="_">-a</span>                          2     125                   125                   N/A                  0                     2021-06-08 17:19:50</span><br><span class="line">TopicTest                         broker<span class="_">-a</span>                          3     125                   125                   N/A                  0                     2021-06-08 17:19:50</span><br><span class="line">TopicTest                         broker-b                          0     125                   125                   N/A                  0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         broker-b                          1     125                   125                   N/A                  0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         broker-b                          2     125                   125                   N/A                  0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         broker-b                          3     125                   125                   N/A                  0                     2021-06-08 17:19:49</span><br><span class="line"></span><br><span class="line">Consume TPS: 0.00</span><br><span class="line">Diff Total: 0</span><br><span class="line"></span><br><span class="line"><span class="comment"># 以broker为角度，查看consume的消费情况</span></span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># mqadmin brokerConsumeStats -n localhost:9876 -b 172.31.0.22:10911 2&gt;/dev/null</span></span><br><span class="line"><span class="comment">#Topic                            #Group                            #Broker Name                      #QID  #Broker Offset        #Consumer Offset      #Diff                 #LastTime</span></span><br><span class="line">TopicTest                         please_rename_unique_group_name_4  broker-b                          0     125                   125                   0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         please_rename_unique_group_name_4  broker-b                          1     125                   125                   0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         please_rename_unique_group_name_4  broker-b                          2     125                   125                   0                     2021-06-08 17:19:49</span><br><span class="line">TopicTest                         please_rename_unique_group_name_4  broker-b                          3     125                   125                   0                     2021-06-08 17:19:49</span><br><span class="line"></span><br><span class="line">Diff Total: 0</span><br><span class="line"></span><br><span class="line"><span class="comment"># 创建订阅组</span></span><br><span class="line">[root@8fptpfxk957bjm rocketmq]<span class="comment"># mqadmin updateSubGroup -n localhost:9876 -c DefaultCluster -g my_consumer_01</span></span><br><span class="line">RocketMQLog:WARN No appenders could be found <span class="keyword">for</span> logger (io.netty.util.internal.PlatformDependent0).</span><br><span class="line">RocketMQLog:WARN Please initialize the logger system properly.</span><br><span class="line">create subscription group to 172.31.0.21:10911 success.</span><br><span class="line">create subscription group to 172.31.0.22:10911 success.</span><br><span class="line">SubscriptionGroupConfig [groupName=my_consumer_01, consumeEnable=<span class="literal">true</span>, consumeFromMinEnable=<span class="literal">false</span>, consumeBroadcastEnable=<span class="literal">false</span>, retryQueueNums=1, retryMaxTimes=16, brokerId=0, whichBrokerWhenConsumeSlowly=1, notifyConsumerIdsChangedEnable=<span class="literal">true</span>]</span><br></pre></td></tr></table></figure><p>可以使用命令创建订阅组，但是没有找到方法怎么样去跟topic关联起来。所以在rocketmq里面，不需要创建消费组。</p><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p><a href="https://www.cnblogs.com/jing99/p/13166602.html" target="_blank" rel="noopener">RocketMQ学习之安装部署及基础讲解</a></p><p><a href="https://github.com/apache/rocketmq/tree/master/docs/cn" target="_blank" rel="noopener">Apache RocketMQ开发者指南</a></p><p><a href="https://juejin.cn/post/6844904199805730824" target="_blank" rel="noopener">RocketMQ-Dledger集群搭建</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;安装&quot;&gt;&lt;a href=&quot;#安装&quot; class=&quot;headerlink&quot; title=&quot;安装&quot;&gt;&lt;/a&gt;安装&lt;/h1&gt;&lt;p&gt;打开 &lt;a href=&quot;http://rocketmq.apache.org/docs/quick-start/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;官方Quick Start&lt;/a&gt; 下载带bin版本的rocketmq，这样就不需要再次编译安装了，可以直接使用。如果下载的是带source的下载包的话，则必须使用 &lt;code&gt;mvn -Prelease-all -DskipTests clean install -U&lt;/code&gt; 来打包。&lt;/p&gt;
&lt;figure class=&quot;highlight bash&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;wget https://mirrors.bfsu.edu.cn/apache/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zip&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;然后解压到指定目录即可。如果只要单机启动的话，就比较简单，运行以下命令即可：&lt;/p&gt;
&lt;figure class=&quot;highlight bash&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;nohup sh bin/mqnamesrv &amp;amp;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;nohup sh bin/mqbroker -c ./conf/broker.conf &amp;amp;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="rocketMQ" scheme="https://www.wumingx.com/tags/rocketMQ/"/>
    
  </entry>
  
  <entry>
    <title>kafka集群的迁移与扩容(转)</title>
    <link href="https://www.wumingx.com/linux/kafka-reassign-partitions.html"/>
    <id>https://www.wumingx.com/linux/kafka-reassign-partitions.html</id>
    <published>2021-06-08T06:31:56.000Z</published>
    <updated>2021-09-07T01:24:58.075Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>此篇文章转载自 <a href="[https://birdben.github.io/2016/12/27/Kafka/Kafka%E5%AD%A6%E4%B9%A0%EF%BC%88%E5%85%AD%EF%BC%89Kafka%E9%9B%86%E7%BE%A4%E7%9A%84%E8%BF%81%E7%A7%BB%E4%B8%8E%E6%89%A9%E5%AE%B9/](https://birdben.github.io/2016/12/27/Kafka/Kafka学习（六）Kafka集群的迁移与扩容/">Kafka学习（六）Kafka集群的迁移与扩容</a>) ，目前很符合我的预期了，如有需要，可以直接到原站点查看。</p><a id="more"></a><div class="note success">            <p>Kafka的集群扩容实际上就是把Topic的Partition移动到新加的集群节点上。我们只需要copy一份Kafka的安装目录到新的节点机器上，修改一下相关配置文件（broker.id，logs.dir等等），但是新添加的Kafka节点机器是不会自动分配数据的，所以需要我们手动操作。</p>          </div><p>Kafka的扩容可以细分为以下三种：</p><ul><li>增加节点</li><li>增加Topic副本</li><li>增加分区</li></ul><p>具体的步骤有两种方式:</p><ul><li>通过<code>–topics-to-move-json-file和–broker-list</code>批量生成新的Topic分区信息，然后根据该信息执行转移操作。</li><li>手动写要移动的topic信息，更灵活，但是在大量Topic和Partition的情况下非常繁琐并且容易出错。</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">kafka-reassign-partitions.sh命令的三种模式</span><br><span class="line"></span><br><span class="line">generate模式：给需要重新分配的Topic，自动生成reassign plan（执行计划），但并不执行</span><br><span class="line">execute模式：根据指定的reassign plan（json文件）重新分配partition或者replication</span><br><span class="line">verify模式：检查重新分配是否完成</span><br><span class="line"></span><br><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file json_file --broker-list <span class="string">"brokerIds"</span> --generate</span><br><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file json_file --execute</span><br><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file json_file --verify</span><br></pre></td></tr></table></figure><h1 id="Kafka动态增加节点（Node）"><a href="#Kafka动态增加节点（Node）" class="headerlink" title="Kafka动态增加节点（Node）"></a>Kafka动态增加节点（Node）</h1><p>新添加的Kafka节点机器是不会自动分配数据的，所以无法分担集群的负载，除非我们新建一个Topic，此时新的Topic会使用新添加的Kafka节点机器。如果我们想新添加的Kafka节点机器能够分担集群存储，需要手动将部分分区移动到新添加的Kafka节点机器上。</p><p>我们原来的Kafka节点分别是0，1，2，现在要加入3，4两个新节点</p><p>topicMove.json</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">    <span class="string">"topics"</span>:[</span><br><span class="line">        &#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>&#125;</span><br><span class="line">    ],</span><br><span class="line">    <span class="string">"version"</span>:1</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>生成迁移的计划</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --topics-to-move-json-file topicMove.json --broker-list <span class="string">"0,1,2,3,4"</span> --generate</span><br><span class="line">Current partition replica assignment</span><br><span class="line"></span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:3,<span class="string">"replicas"</span>:[1,2,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:1,<span class="string">"replicas"</span>:[2,1,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[1,2,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:2,<span class="string">"replicas"</span>:[0,2,1]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:5,<span class="string">"replicas"</span>:[0,1,2]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:4,<span class="string">"replicas"</span>:[2,0,1]&#125;]&#125;</span><br><span class="line">Proposed partition reassignment configuration</span><br><span class="line"></span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:3,<span class="string">"replicas"</span>:[2,0,1]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:1,<span class="string">"replicas"</span>:[0,3,4]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[4,2,3]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:2,<span class="string">"replicas"</span>:[1,4,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:5,<span class="string">"replicas"</span>:[4,3,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:4,<span class="string">"replicas"</span>:[3,1,2]&#125;]&#125;</span><br></pre></td></tr></table></figure><p>上面是原来的所有partition在各个节点的分布情况，下面是加入4，5两个新节点之后所有partition在各个节点的分布情况</p><p>将生成的执行计划保存为add_node.json文件</p><p>重新分配partition（其实这里是添加新的节点）</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_node.json --execute</span><br><span class="line">Current partition replica assignment</span><br><span class="line"></span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:3,<span class="string">"replicas"</span>:[1,2,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:1,<span class="string">"replicas"</span>:[2,1,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[1,2,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:2,<span class="string">"replicas"</span>:[0,2,1]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:5,<span class="string">"replicas"</span>:[0,1,2]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:4,<span class="string">"replicas"</span>:[2,0,1]&#125;]&#125;</span><br><span class="line"></span><br><span class="line">Save this to use as the --reassignment-json-file option during rollback</span><br><span class="line">Successfully started reassignment of partitions &#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[4,2,3]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:4,<span class="string">"replicas"</span>:[3,1,2]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:5,<span class="string">"replicas"</span>:[4,3,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:2,<span class="string">"replicas"</span>:[1,4,0]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:3,<span class="string">"replicas"</span>:[2,0,1]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"logstash_test"</span>,<span class="string">"partition"</span>:1,<span class="string">"replicas"</span>:[0,3,4]&#125;]&#125;</span><br></pre></td></tr></table></figure><p>查看执行的状态</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_node.json --verify</span><br><span class="line">ERROR: Assigned replicas (X,X) don<span class="string">'t match the list of replicas for reassignment (X,X) for partition [logstash_test,1]</span></span><br></pre></td></tr></table></figure><p>假设出现类似这样的错误，他并不是真的出错，而是指目前仍在复制数据中。再过一段时间再运行verify命令，他就会消失(加入完成拷贝)</p><p>当然也可以不使用generate先生成执行计划，而是自己直接手动编辑生成add_node.json文件内容，然后直接execute执行，但是这样容易出错，Kafka节点比较少的时候推荐使用。</p><p>注意：其实通过generate的结果我们也可以看出，其实不管是扩容，减容，迁移，其实都是重新分配Topic或者Partition的过程。json文件的内容都是指定Topic下的Partition要移动到哪个Node上。</p><h1 id="Kafka动态增加Topic副本（Replication）"><a href="#Kafka动态增加Topic副本（Replication）" class="headerlink" title="Kafka动态增加Topic副本（Replication）"></a>Kafka动态增加Topic副本（Replication）</h1><p>查看当前node_log的Topic信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line">Topic:node_logPartitionCount:1ReplicationFactor:1Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 2Replicas: 2Isr: 2</span><br></pre></td></tr></table></figure><p>add_replication.json</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  <span class="attr">"version"</span>: <span class="number">1</span>,</span><br><span class="line">  <span class="attr">"partitions"</span>: [</span><br><span class="line">    &#123; <span class="attr">"topic"</span>: <span class="string">"node_log"</span>,<span class="attr">"partition"</span>: <span class="number">0</span>,<span class="attr">"replicas"</span>: [ <span class="number">0</span>,<span class="number">1</span>,<span class="number">2</span> ] &#125;</span><br><span class="line">  ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>重新分配partition（其实这里是添加副本replication）</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_replication.json --execute</span><br><span class="line">Current partition replica assignment</span><br><span class="line"></span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[2]&#125;]&#125;</span><br><span class="line"></span><br><span class="line">Save this to use as the --reassignment-json-file option during rollback</span><br><span class="line">Successfully started reassignment of partitions &#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[0,1,2]&#125;]&#125;</span><br></pre></td></tr></table></figure><p>查看执行的状态</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_replication.json --verify</span><br><span class="line">Status of partition reassignment:</span><br><span class="line">Reassignment of partition [node_log,0] completed successfully</span><br></pre></td></tr></table></figure><p>如果遇到下面的错误很有可能是json文件格式有错误，仔细检查修正重新运行即可</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Partitions reassignment failed due to Partition reassignment data file add_replication.json is empty</span><br><span class="line">kafka.common.AdminCommandFailedException: Partition reassignment data file add_replication.json is empty</span><br><span class="line">at kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:120)</span><br><span class="line">at kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:52)</span><br><span class="line">at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)</span><br></pre></td></tr></table></figure><p>查看添加replication后的node_log的Topic信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line">Topic:node_logPartitionCount:1ReplicationFactor:2Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 0Replicas: 0,1,2Isr: 1,0,2</span><br></pre></td></tr></table></figure><h1 id="Kafka动态增加分区（Parition）"><a href="#Kafka动态增加分区（Parition）" class="headerlink" title="Kafka动态增加分区（Parition）"></a>Kafka动态增加分区（Parition）</h1><p>查看当前node_log的Topic信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line">Topic:node_logPartitionCount:1ReplicationFactor:2Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 0Replicas: 0,1,2Isr: 1,0,2</span><br></pre></td></tr></table></figure><p>add_partition.json</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  <span class="attr">"version"</span>: <span class="number">1</span>,</span><br><span class="line">  <span class="attr">"partitions"</span>: [</span><br><span class="line">    &#123; <span class="attr">"topic"</span>: <span class="string">"node_log"</span>,<span class="attr">"partition"</span>: <span class="number">0</span>,<span class="attr">"replicas"</span>: [ <span class="number">0</span>,<span class="number">1</span>,<span class="number">2</span> ] &#125;,</span><br><span class="line">    &#123; <span class="attr">"topic"</span>: <span class="string">"node_log"</span>,<span class="attr">"partition"</span>: <span class="number">1</span>,<span class="attr">"replicas"</span>: [ <span class="number">0</span>,<span class="number">1</span>,<span class="number">2</span> ] &#125;,</span><br><span class="line">    &#123; <span class="attr">"topic"</span>: <span class="string">"node_log"</span>,<span class="attr">"partition"</span>: <span class="number">2</span>,<span class="attr">"replicas"</span>: [ <span class="number">0</span>,<span class="number">1</span>,<span class="number">2</span> ] &#125;</span><br><span class="line">  ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>重新分配partition</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_partition.json --execute</span><br><span class="line">Current partition replica assignment</span><br><span class="line"></span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[0,1,2]&#125;]&#125;</span><br><span class="line"></span><br><span class="line">Save this to use as the --reassignment-json-file option during rollback</span><br><span class="line">[2016-12-30 12:22:17,195] ERROR Skipping reassignment of partition [node_log,1] since it doesn<span class="string">'t exist (kafka.admin.ReassignPartitionsCommand)</span></span><br><span class="line"><span class="string">[2016-12-30 12:22:17,207] ERROR Skipping reassignment of partition [node_log,2] since it doesn'</span>t exist (kafka.admin.ReassignPartitionsCommand)</span><br><span class="line">Successfully started reassignment of partitions &#123;<span class="string">"version"</span>:1,<span class="string">"partitions"</span>:[&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:0,<span class="string">"replicas"</span>:[0,1,2]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:1,<span class="string">"replicas"</span>:[0,1,2]&#125;,&#123;<span class="string">"topic"</span>:<span class="string">"node_log"</span>,<span class="string">"partition"</span>:2,<span class="string">"replicas"</span>:[0,1,2]&#125;]&#125;</span><br></pre></td></tr></table></figure><p>查看执行的状态</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_partition.json --verify</span><br><span class="line">Status of partition reassignment:</span><br><span class="line">ERROR: Assigned replicas (1,2,0) don<span class="string">'t match the list of replicas for reassignment (0,1,2) for partition [node_log,1]</span></span><br><span class="line"><span class="string">ERROR: Assigned replicas (2,0,1) don'</span>t match the list of replicas <span class="keyword">for</span> reassignment (0,1,2) <span class="keyword">for</span> partition [node_log,2]</span><br><span class="line">Reassignment of partition [node_log,0] completed successfully</span><br><span class="line">Reassignment of partition [node_log,1] failed</span><br><span class="line">Reassignment of partition [node_log,2] failed</span><br></pre></td></tr></table></figure><p>报错是因为我们只有一个partition0，没有partition1，partition2，所以跳过了重新分配partition的过程。</p><p>我们需要先增加partition的数量，我们把partition的数量变成6个</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --zookeeper localhost:2181 --alter --topic node_log --partitions 6</span><br><span class="line">WARNING: If partitions are increased <span class="keyword">for</span> a topic that has a key, the partition logic or ordering of the messages will be affected</span><br><span class="line">Adding partitions succeeded!</span><br></pre></td></tr></table></figure><p>查看添加partition后的node_log的Topic信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line">Topic:node_logPartitionCount:6ReplicationFactor:3Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 0Replicas: 0,1,2Isr: 1,0,2</span><br><span class="line">Topic: node_logPartition: 1Leader: 1Replicas: 1,2,0Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 2Leader: 2Replicas: 2,0,1Isr: 2,0,1</span><br><span class="line">Topic: node_logPartition: 3Leader: 0Replicas: 0,2,1Isr: 0,2,1</span><br><span class="line">Topic: node_logPartition: 4Leader: 1Replicas: 1,0,2Isr: 1,0,2</span><br><span class="line">Topic: node_logPartition: 5Leader: 2Replicas: 2,1,0Isr: 2,1,0</span><br></pre></td></tr></table></figure><p>重新执行reassign，然后查看执行状态</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-reassign-partitions.sh --zookeeper localhost:2181 --reassignment-json-file ~/Downloads/add_partition.json --verify</span><br><span class="line">Status of partition reassignment:</span><br><span class="line">Reassignment of partition [node_log,0] completed successfully</span><br><span class="line">Reassignment of partition [node_log,1] completed successfully</span><br><span class="line">Reassignment of partition [node_log,2] completed successfully</span><br></pre></td></tr></table></figure><p>重新分配执行完成之后，再次查看node_log的Topic信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line"></span><br><span class="line">Topic:node_logPartitionCount:6ReplicationFactor:3Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 0Replicas: 0,1,2Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 1Leader: 0Replicas: 0,1,2Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 2Leader: 0Replicas: 0,1,2Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 3Leader: 0Replicas: 0,2,1Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 4Leader: 1Replicas: 1,0,2Isr: 1,2,0</span><br><span class="line">Topic: node_logPartition: 5Leader: 2Replicas: 2,1,0Isr: 1,2,0</span><br></pre></td></tr></table></figure><p>执行reassign之后，我们发现0-3的partition的leader都是0，这是因为我们之前突然扩展partition到6个，而我们在reassign的之后只指定了0-2的partition的分配，这样是不合理的，所以我们使用kafka自带的重分区工具进行处理，即上文的 <code>--generate</code>：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash </span></span><br><span class="line">brokerid=<span class="string">"0,1,2"</span></span><br><span class="line">zkserver=<span class="string">"localhost:2181"</span></span><br><span class="line">kafkapath=<span class="string">"/opt/kafka/kafkaServer/bin"</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">for</span> i <span class="keyword">in</span> `cat /tmp/topics.txt`;<span class="keyword">do</span></span><br><span class="line"><span class="built_in">echo</span> <span class="string">"==================&gt; <span class="variable">$i</span>"</span></span><br><span class="line"><span class="built_in">echo</span> &#123;\<span class="string">"topics\":[ &#123;\"topic\": \"<span class="variable">$i</span>\"&#125;],\"version\":1&#125;  &gt;/root/topicMove.json</span></span><br><span class="line"><span class="string"><span class="variable">$kafkapath</span>/kafka-reassign-partitions.sh --zookeeper <span class="variable">$zkserver</span> --topics-to-move-json-file /root/topicMove.json --broker-list "</span><span class="variable">$brokerid</span><span class="string">" --generate |grep -A1 Proposed |grep version &gt;/root/move.json</span></span><br><span class="line"><span class="string">echo <span class="variable">$kafkapath</span>/kafka-reassign-partitions.sh --zookeeper <span class="variable">$zkserver</span> --reassignment-json-file /root/move.json --execute</span></span><br><span class="line"><span class="string"><span class="variable">$kafkapath</span>/kafka-reassign-partitions.sh --zookeeper <span class="variable">$zkserver</span> --reassignment-json-file /root/move.json --execute</span></span><br><span class="line"><span class="string">sleep 2</span></span><br><span class="line"><span class="string">while (( 1 ));do </span></span><br><span class="line"><span class="string"><span class="variable">$kafkapath</span>/kafka-reassign-partitions.sh --zookeeper <span class="variable">$zkserver</span> --reassignment-json-file /root/move.json --verify &gt;/tmp/verify.txt</span></span><br><span class="line"><span class="string">num_part=`cat /tmp/verify.txt |grep 'Reassignment of partition' |wc -l` </span></span><br><span class="line"><span class="string">num_success=`cat /tmp/verify.txt |grep 'completed successfully' |wc -l`</span></span><br><span class="line"><span class="string">if [ <span class="variable">$num_part</span> -ne <span class="variable">$num_success</span> ];then</span></span><br><span class="line"><span class="string">echo "</span>[INFO] Reassignment of partition num: <span class="variable">$num_part</span>, completed successfully num: <span class="variable">$num_success</span> <span class="string">"</span></span><br><span class="line"><span class="string">sleep 2</span></span><br><span class="line"><span class="string">else</span></span><br><span class="line"><span class="string">break</span></span><br><span class="line"><span class="string">fi</span></span><br><span class="line"><span class="string">done</span></span><br><span class="line"><span class="string">done</span></span><br></pre></td></tr></table></figure><p>这里使用了一个简单的脚本进行数据的迁移，只需要只topic名写到/tmp/topics.txt即可进行了。此脚本虽然没有做其他的异常判断，但也是可以进行topic的批量迁移的。</p><h1 id="assigned-replicas和preferred-replica"><a href="#assigned-replicas和preferred-replica" class="headerlink" title="assigned replicas和preferred replica"></a>assigned replicas和preferred replica</h1><p>但是这样我们发现还是有点小问题，partition2的leader仍然是0，这是为什么呢？。这里先普及一下assigned replicas和preferred replica。</p><p>每个partitiion的所有replicas叫做”assigned replicas”，”assigned replicas”中的第一个replicas叫”preferred replica”，刚创建的topic一般”preferred replica”是leader。leader replica负责所有的读写。但随着时间推移，broker可能会停机，会导致leader迁移，导致机群的负载不均衡。</p><p>我们这里preferred replica已经是2了，但是leader却不是2，这样我们需要重新选举一下leader，需要使用kafka-preferred-replica-election.sh来调整。</p><p>两种操作方式：</p><ul><li>对所有Topics进行操作</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-preferred-replica-election.sh --zookeeper localhost:2181</span><br></pre></td></tr></table></figure><ul><li>对某个Topic进行操作（json文件中指定要操作的Topic）</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-preferred-replica-election.sh --zookeeper localhost:2181 --path-to-json-file json_file</span><br></pre></td></tr></table></figure><p>注意：这里我们只需要调整node_log的Topic的partition2的leader。</p><p>leaderTopic.json</p><figure class="highlight json"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line"> <span class="attr">"partitions"</span>:</span><br><span class="line">  [</span><br><span class="line">    &#123;<span class="attr">"topic"</span>:<span class="string">"node_log"</span>,<span class="attr">"partition"</span>:<span class="number">2</span>&#125;</span><br><span class="line">  ]</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>重新选举leader</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-preferred-replica-election.sh --zookeeper localhost:2181 --path-to-json-file leaderTopic.json</span><br><span class="line">Successfully started preferred replica election <span class="keyword">for</span> partitions Set([node_log,2])</span><br></pre></td></tr></table></figure><p>再次查看Topic情况，发现leader也是我们所期望的均匀分配了</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic node_log</span><br><span class="line">Topic:node_logPartitionCount:6ReplicationFactor:2Configs:</span><br><span class="line">Topic: node_logPartition: 0Leader: 0Replicas: 0,1Isr: 0,1</span><br><span class="line">Topic: node_logPartition: 1Leader: 1Replicas: 1,2Isr: 2,1</span><br><span class="line">Topic: node_logPartition: 2Leader: 2Replicas: 2,0Isr: 2,0</span><br><span class="line">Topic: node_logPartition: 3Leader: 0Replicas: 0,1Isr: 0,1</span><br><span class="line">Topic: node_logPartition: 4Leader: 1Replicas: 1,2Isr: 2,1</span><br><span class="line">Topic: node_logPartition: 5Leader: 2Replicas: 2,0Isr: 0,2</span><br></pre></td></tr></table></figure><h1 id="修改topic配置"><a href="#修改topic配置" class="headerlink" title="修改topic配置"></a>修改topic配置</h1><p>kafka有一个全局配置定义了全部topic的属性，我们也可以对某些特定的topic进行一些特殊的配置。</p><p>单独修改一个topic的配置：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 查看配置</span></span><br><span class="line">[root@IPV6-11114 ~]<span class="comment"># /opt/kafka/bin/kafka-configs.sh --describe --zookeeper 10.0.1.71:5001 --entity-type topics --entity-name KAFKA_OPERATION_AUDIT_LOG</span></span><br><span class="line">Configs <span class="keyword">for</span> topic <span class="string">'KAFKA_OPERATION_AUDIT_LOG'</span> are unclean.leader.election.enable=<span class="literal">true</span></span><br><span class="line">[root@IPV6-11114 ~]<span class="comment"># </span></span><br><span class="line"><span class="comment"># 删除配置</span></span><br><span class="line">[root@IPV6-11114 ~]<span class="comment"># /opt/kafka/bin/kafka-configs.sh --alter --zookeeper 10.0.1.71:5001 --entity-type topics --entity-name KAFKA_OPERATION_AUDIT_LOG --delete-config unclean.leader.election.enable</span></span><br><span class="line">Completed Updating config <span class="keyword">for</span> entity: topic <span class="string">'KAFKA_OPERATION_AUDIT_LOG'</span>.</span><br></pre></td></tr></table></figure><p>此外，使用命令操作kafka也有可能会不太友好，可以使用Kafka Manager工具来实现界面化管理，有兴趣的朋友可以试试。</p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;此篇文章转载自 &lt;a href=&quot;[https://birdben.github.io/2016/12/27/Kafka/Kafka%E5%AD%A6%E4%B9%A0%EF%BC%88%E5%85%AD%EF%BC%89Kafka%E9%9B%86%E7%BE%A4%E7%9A%84%E8%BF%81%E7%A7%BB%E4%B8%8E%E6%89%A9%E5%AE%B9/](https://birdben.github.io/2016/12/27/Kafka/Kafka学习（六）Kafka集群的迁移与扩容/&quot;&gt;Kafka学习（六）Kafka集群的迁移与扩容&lt;/a&gt;) ，目前很符合我的预期了，如有需要，可以直接到原站点查看。&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="kafka" scheme="https://www.wumingx.com/tags/kafka/"/>
    
  </entry>
  
  <entry>
    <title>shell三剑客之一：grep详解</title>
    <link href="https://www.wumingx.com/script/shell-grep.html"/>
    <id>https://www.wumingx.com/script/shell-grep.html</id>
    <published>2021-05-09T13:51:31.000Z</published>
    <updated>2021-06-16T06:20:19.855Z</updated>
    
    <content type="html"><![CDATA[<h2 id="一、简介"><a href="#一、简介" class="headerlink" title="一、简介"></a>一、简介</h2><p>Linux系统中grep命令是一种强大的文本搜索工具，它能使用正则表达式搜索文本，并把匹 配的行打印出来。grep全称是<code>Global Regular Expression Print</code>，表示全局正则表达式版本，它的使用权限是所有用户。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">fgrep == grep -F</span><br><span class="line">egrep == grep -E</span><br></pre></td></tr></table></figure><a id="more"></a><h2 id="二、格式"><a href="#二、格式" class="headerlink" title="二、格式"></a>二、格式</h2><p><code>grep [OPTIONS] PATTERN [FILE...]</code></p><h2 id="三、主要参数"><a href="#三、主要参数" class="headerlink" title="三、主要参数"></a>三、主要参数</h2><p>grep 如果不加参数的话，则是会使用==普通的正则的表达式==进行匹配。</p><p>[options]主要参数：</p><ul><li><p>帮助信息</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">-V  --version，显示版本</span><br><span class="line">--<span class="built_in">help</span>  显示帮忙信息</span><br></pre></td></tr></table></figure></li><li><p>模式选择</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">-G  --basic-regexp</span><br><span class="line">    普通的正则的表达式进行匹配，这是默认选项，可省略</span><br><span class="line">    </span><br><span class="line">-E  --extended-regexp</span><br><span class="line">    等同于egrep，可以使用扩展的正则表达式来匹配</span><br><span class="line"></span><br><span class="line">-P  --perl-regexp</span><br><span class="line">    使用Perl正则表达式匹配，一般不使用</span><br><span class="line"></span><br><span class="line">-F  --fixed-strings</span><br><span class="line">    视为字符串进行匹配，如果字符串出现了.*，也认为为普通的字符串，即不使用正则表达式进行匹配</span><br></pre></td></tr></table></figure><ul><li><p>匹配控制</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">-e pattern </span><br><span class="line">    --regexp=pattern</span><br><span class="line">    多条件匹配</span><br><span class="line">-f file</span><br><span class="line">    --file=file</span><br><span class="line">    指定文件进行多条件匹配，一行为一个条件</span><br><span class="line">-i/-y</span><br><span class="line">    --ignore-case</span><br><span class="line">    不区分大小写</span><br><span class="line">-v</span><br><span class="line">    --invert-match</span><br><span class="line">    不显示包含匹配字符的所有行</span><br><span class="line">-w</span><br><span class="line">    --word-regexp</span><br><span class="line">    只匹配整个单词，而不是字符串的一部分</span><br><span class="line">-x</span><br><span class="line">    --line-regexp</span><br><span class="line">    只显示整行都匹配</span><br></pre></td></tr></table></figure></li><li><p>一般输出控制</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">-c  --count</span><br><span class="line">    只输出匹配行的计数</span><br><span class="line">--color</span><br><span class="line">    将查找到的子符以有颜色的输出</span><br><span class="line">-q  --quiet/--silent</span><br><span class="line">    匹配字符不显示任何信息，如果有匹配到，返回状态码0；没有匹配则返回状态码为非0</span><br><span class="line">-s  --no-messages</span><br><span class="line">    不显示不存在或无匹配文本的错误信息。在第七版unix的系统下等同于-q，所以此参数建议不在要shell中使用</span><br><span class="line">-o  --only-matching</span><br><span class="line">    只输出文件中匹配到的部分</span><br><span class="line">-l  --files-with-matches</span><br><span class="line">    查询多文件时只输出包含匹配字符的文件名</span><br><span class="line"></span><br><span class="line">-L  --files-without-match</span><br><span class="line">    查询多文件时只输出不包含匹配字符的文件名</span><br></pre></td></tr></table></figure></li><li><p>输出行前缀控制</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">-b --byte-offset</span><br><span class="line">    打印样式匹配所位于的字符或字节偏移，选项 -b -o 一般总是配合使用。并没有什么用处</span><br><span class="line">-H  --with-filename</span><br><span class="line">    查询多文件时会显示文件名，这是默认的选项</span><br><span class="line">-h  --no-filename</span><br><span class="line">    查询多文件时不显示文件名</span><br><span class="line">-n  --line-number</span><br><span class="line">    显示匹配行及行号</span><br></pre></td></tr></table></figure></li><li><p>上下文控制 </p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">-A num</span><br><span class="line">    --after-context=num</span><br><span class="line">    除了显示符合范本样式的那一行之外，并显示该行之后的内容</span><br><span class="line">-B num</span><br><span class="line">    --before-context=num</span><br><span class="line">    除了显示符合范本样式的那一行之外，并显示该行之前的内容</span><br><span class="line">-C num</span><br><span class="line">    --context=num</span><br><span class="line">    匹配的上下文分别显示[number]行</span><br><span class="line">--group-separator=string</span><br><span class="line">    如果查找的内容是多个结果，且又使用-A -B -C这三个参数之后，使用string来代替分隔，默认为--为分隔符</span><br><span class="line">--no-group-separator</span><br><span class="line">    不输出分隔符</span><br></pre></td></tr></table></figure><ul><li>文件和目录控制</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">-a  --text，此用法等同于--binary-files=text</span><br><span class="line">    将 binary 文件以 text 文件的方式搜寻数据，此方法要配合zgrep来查询压缩包的字符</span><br><span class="line">-I  等同于--binary-files=without-match</span><br><span class="line">    </span><br><span class="line">-r   --recursive，等同于-directories=recurse</span><br><span class="line">    递归查询目录下的所有子目录以及文件，不包括符号链接</span><br><span class="line">-R  --dereference-recursive</span><br><span class="line">    同-r，但其查询时，会查询符号链接</span><br><span class="line">-d  --directories=action</span><br><span class="line">    默认的action为<span class="built_in">read</span>，还有一个action为recurse，这个就等同于-r</span><br><span class="line">--binary-files=<span class="built_in">type</span></span><br><span class="line">    <span class="built_in">type</span>为以下类型：</span><br><span class="line">    1、默认为binary</span><br><span class="line">    2、without-match，等同于-I</span><br><span class="line">    3、text，等同于-a</span><br><span class="line">--exclude=glob</span><br><span class="line">    排除一些目录或者文件，不进行匹配</span><br><span class="line">--exclude-from=file</span><br><span class="line">    以文件形式排除</span><br><span class="line">--exclude-dir=glob</span><br><span class="line">    跳过目录</span><br><span class="line">--include=glob</span><br><span class="line">    简单理解为在哪些文件下查询</span><br></pre></td></tr></table></figure><h2 id="三、实例"><a href="#三、实例" class="headerlink" title="三、实例"></a>三、实例</h2><ul><li>-o 只显示匹配到的数据</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ grep eth.* ifcfg-eth0</span><br><span class="line">DEVICE=eth0</span><br><span class="line"></span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -o eth.* ifcfg-eth0</span><br><span class="line">eth0</span><br></pre></td></tr></table></figure><ul><li>-a 配合zgrep来使用</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ tar zcvf /tmp/ifcfg.tgz ifcfg-*</span><br><span class="line">ifcfg-eth0</span><br><span class="line">ifcfg-eth0:317</span><br><span class="line">ifcfg-eth1</span><br><span class="line">ifcfg-eth2</span><br><span class="line">ifcfg-eth3</span><br><span class="line">ifcfg-lo</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep IPADDR /tmp/ifcfg.tgz</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -a IPADDR /tmp/ifcfg.tgz</span><br><span class="line"><span class="comment">#以上方法都匹配不到数据</span></span><br><span class="line">[op@PSR-17208 network-scripts]$ zgrep -a IPADDR /tmp/ifcfg.tgz</span><br><span class="line">IPADDR=218.75.158.52</span><br><span class="line">IPADDR=198.19.5.150</span><br><span class="line">IPADDR=111.23.5.150</span><br><span class="line">IPADDR=127.0.0.1</span><br></pre></td></tr></table></figure><ul><li><p>-F 使用普通字符串来查看</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ <span class="built_in">echo</span> <span class="string">"1.1.1.1"</span> |fgrep <span class="string">'1.*'</span></span><br><span class="line">[op@PSR-17208 network-scripts]$ <span class="built_in">echo</span> <span class="string">"1.1.1.1"</span> |grep <span class="string">'1.*'</span></span><br><span class="line">1.1.1.1</span><br></pre></td></tr></table></figure></li><li><p>-A 显示匹配后 -B显示匹配前的数据 -C 显示前后行</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ grep -n GATEWAY -A2 -B1 ifcfg-eth0</span><br><span class="line">3-NETMASK=255.255.255.128</span><br><span class="line">4:GATEWAY=218.75.158.2</span><br><span class="line">5-BOOTPROTO=static</span><br><span class="line">6-ONBOOT=yes</span><br><span class="line"></span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -n GATEWAY -C 1 ifcfg-eth0</span><br><span class="line">3-NETMASK=255.255.255.128</span><br><span class="line">4:GATEWAY=218.75.158.2</span><br><span class="line">5-BOOTPROTO=static</span><br><span class="line"></span><br><span class="line">[op@11009815 network-scripts]$ dmidecode |grep -C 1 --group-separator=<span class="string">'-------------'</span> Product</span><br><span class="line">Manufacturer: Dell Inc.</span><br><span class="line">Product Name: PowerEdge R710</span><br><span class="line">Version: Not Specified</span><br><span class="line">-------------</span><br><span class="line">Manufacturer: Dell Inc.</span><br><span class="line">Product Name: 0XDX06</span><br><span class="line">Version: A09</span><br></pre></td></tr></table></figure><ul><li><p>-e 指定多个匹配条件；可以参数-E；-f 将匹配条件放在文件下 </p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ cat /tmp/a.txt </span><br><span class="line">GATEWAY</span><br><span class="line">NETMASK</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -f /tmp/a.txt ifcfg-eth0</span><br><span class="line">NETMASK=255.255.255.128</span><br><span class="line">GATEWAY=218.75.158.2</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -e -f /tmp/a.txt ifcfg-eth0</span><br><span class="line"></span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -e GATEWAY -e NETMASK ifcfg-eth0</span><br><span class="line">NETMASK=255.255.255.128</span><br><span class="line">GATEWAY=218.75.158.2</span><br></pre></td></tr></table></figure></li><li><p>-w 只显示全字符合的列 -x 只显示全列符合的列</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[op@PSR-17208 network-scripts]$ grep -w GATEWA ifcfg-eth0</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -w GATEWAY ifcfg-eth0</span><br><span class="line">GATEWAY=218.75.158.2</span><br><span class="line"></span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -x GATEWAY ifcfg-eth0</span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -x GATEWAY=218.75.158.2 ifcfg-eth0</span><br><span class="line">GATEWAY=218.75.158.2</span><br></pre></td></tr></table></figure><ul><li>-r/-R 递归查询</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#在包涵的目录下查询</span></span><br><span class="line">[op@11009815 network-scripts]$ grep -n DEVICE . -R --include <span class="string">"ifcfg-eth*"</span> --include <span class="string">"ifcfg-bond*"</span></span><br><span class="line">./ifcfg-eth3:9:DEVICE=eth3</span><br><span class="line">./ifcfg-bond0:1:DEVICE=bond0</span><br><span class="line">./ifcfg-eth2:9:DEVICE=eth2</span><br><span class="line">./ifcfg-bond0:317:1:DEVICE=bond0:317</span><br><span class="line">./ifcfg-eth1:1:DEVICE=eth1</span><br><span class="line">./ifcfg-eth0:1:DEVICE=eth0</span><br><span class="line"></span><br><span class="line"><span class="comment"># exclude 排除目录</span></span><br><span class="line">[op@PSR-17208 network-scripts]$ grep -n BOOTPROTO . -R --exclude <span class="string">"ifup-aliases"</span> --exclude <span class="string">"network-functions"</span> --exclude <span class="string">"ifup-plusb"</span> --exclude <span class="string">"ifdown-eth"</span> --exclude <span class="string">"ifup-eth"</span> --exclude <span class="string">"ifup"</span></span><br><span class="line">./ifcfg-eth2:5:BOOTPROTO=static</span><br><span class="line">./ifcfg-eth0:5:BOOTPROTO=static</span><br><span class="line">./ifcfg-eth3:5:BOOTPROTO=static</span><br><span class="line">./ifcfg-eth0:317:2:BOOTPROTO=static</span><br><span class="line">./ifcfg-eth1:5:BOOTPROTO=none</span><br></pre></td></tr></table></figure><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p>官方文档：<a href="http://www.gnu.org/software/grep/manual/grep.html" target="_blank" rel="noopener">http://www.gnu.org/software/grep/manual/grep.html</a></p><p><a href="https://wenzhiquan.github.io/2016/09/06/2016-09-06-grep/" target="_blank" rel="noopener">https://wenzhiquan.github.io/2016/09/06/2016-09-06-grep/</a></p><p><a href="http://www.51niux.com/?id=90" target="_blank" rel="noopener">http://www.51niux.com/?id=90</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h2 id=&quot;一、简介&quot;&gt;&lt;a href=&quot;#一、简介&quot; class=&quot;headerlink&quot; title=&quot;一、简介&quot;&gt;&lt;/a&gt;一、简介&lt;/h2&gt;&lt;p&gt;Linux系统中grep命令是一种强大的文本搜索工具，它能使用正则表达式搜索文本，并把匹 配的行打印出来。grep全称是&lt;code&gt;Global Regular Expression Print&lt;/code&gt;，表示全局正则表达式版本，它的使用权限是所有用户。&lt;/p&gt;
&lt;figure class=&quot;highlight bash&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;fgrep == grep -F&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;egrep == grep -E&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="shell教程" scheme="https://www.wumingx.com/categories/script/"/>
    
    
      <category term="shell" scheme="https://www.wumingx.com/tags/shell/"/>
    
  </entry>
  
  <entry>
    <title>实例讲解strace的用法</title>
    <link href="https://www.wumingx.com/performance/strace.html"/>
    <id>https://www.wumingx.com/performance/strace.html</id>
    <published>2021-05-09T12:50:10.000Z</published>
    <updated>2021-05-09T14:10:11.768Z</updated>
    
    <content type="html"><![CDATA[<h1 id="使用说明"><a href="#使用说明" class="headerlink" title="使用说明"></a>使用说明</h1><p>strace是一个非常简单的工具，它可以跟踪系统调用的执行。最简单的方式，它可以从头到尾跟踪binary的执行，然后以一行文本输出系统调用的名字，参数和返回值。</p><p>strace的最简单的用法就是执行一个指定的命令，如：$ strace cat /dev/null 在命令结束之后它也就退出了。在命令执行的过程中，strace会记录和解析命令进程的所有系统调用以及这个进程所接收到的所有的信号值。</p><div class="note success">            <p>对于网络慢的问题，可以使用tcpdump抓包分析慢的问题；而如果就是本机一个程序慢呢？就需要使用strace来分析了</p>          </div><a id="more"></a><p>以下是详细参数：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br></pre></td><td class="code"><pre><span class="line">-c 统计每一系统调用的所执行的时间,次数和出错的次数等.</span><br><span class="line">-d 输出strace关于标准错误的调试信息.</span><br><span class="line">-f 跟踪由fork调用所产生的子进程.</span><br><span class="line">-ff 如果提供-o filename,则所有进程的跟踪结果输出到相应的filename.pid中,pid是各进程的进程号.</span><br><span class="line">-F 尝试跟踪vfork调用.在-f时,vfork不被跟踪.</span><br><span class="line">-h 输出简要的帮助信息.</span><br><span class="line">-i 输出系统调用的入口指针.</span><br><span class="line">-q 禁止输出关于脱离的消息.</span><br><span class="line">-r 打印出相对时间关于,,每一个系统调用.</span><br><span class="line">-t 在输出中的每一行前加上时间信息.</span><br><span class="line">-tt 在输出中的每一行前加上时间信息,微秒级.</span><br><span class="line">-ttt 微秒级输出,以秒了表示时间.</span><br><span class="line">-T 显示每一调用所耗的时间.</span><br><span class="line">-v 输出所有的系统调用.一些调用关于环境变量,状态,输入输出等调用由于使用频繁,默认不输出.</span><br><span class="line">-V 输出strace的版本信息.</span><br><span class="line">-x 以十六进制形式输出非标准字符串</span><br><span class="line">-xx 所有字符串以十六进制形式输出.</span><br><span class="line">-a column设置返回值的输出位置.默认 为40.</span><br><span class="line">-e expr指定一个表达式,用来控制如何跟踪.格式如下:</span><br><span class="line">[qualifier=][!]value1[,value2]...</span><br><span class="line">qualifier只能是 trace,abbrev,verbose,raw,signal,<span class="built_in">read</span>,write其中之一.value是用来限定的符号或数字.默认的 qualifier是 trace.感叹号是否定符号.例如:</span><br><span class="line">-eopen等价于 -e trace=open,表示只跟踪open调用.而-etrace!=open表示跟踪除了open以外的其他调用.有两个特殊的符号 all 和 none.</span><br><span class="line">注意有些shell使用!来执行历史记录里的命令,所以要使用\\.</span><br><span class="line">-e trace=<span class="built_in">set</span>只跟踪指定的系统 调用.例如:-e trace=open,close,rean,write表示只跟踪这四个系统调用.默认的为<span class="built_in">set</span>=all.</span><br><span class="line">-e trace=file只跟踪有关文件操作的系统调用.</span><br><span class="line">-e trace=process只跟踪有关进程控制的系统调用.</span><br><span class="line">-e trace=network跟踪与网络有关的所有系统调用.</span><br><span class="line">-e strace=signal跟踪所有与系统信号有关的 系统调用</span><br><span class="line">-e trace=ipc跟踪所有与进程通讯有关的系统调用</span><br><span class="line">-e abbrev=<span class="built_in">set</span>设定 strace输出的系统调用的结果集.-v 等与 abbrev=none.默认为abbrev=all.</span><br><span class="line">-e raw=<span class="built_in">set</span>将指 定的系统调用的参数以十六进制显示.</span><br><span class="line">-e signal=<span class="built_in">set</span>指定跟踪的系统信号.默认为all.如 signal=!SIGIO(或者signal=!io),表示不跟踪SIGIO信号.</span><br><span class="line">-e <span class="built_in">read</span>=<span class="built_in">set</span>输出从指定文件中读出 的数据.例如:</span><br><span class="line">-e <span class="built_in">read</span>=3,5</span><br><span class="line">-e write=<span class="built_in">set</span>输出写入到指定文件中的数据.</span><br><span class="line">-o filename将strace的输出写入文件filename</span><br><span class="line">-p pid跟踪指定的进程pid.</span><br><span class="line">-s strsize指定输出的字符串的最大长度.默认为32.文件名一直全部输出.</span><br><span class="line">-u username以username 的UID和GID执行被跟踪的命令</span><br></pre></td></tr></table></figure><p>比较常用的参数有：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">-p 跟踪指定的进程</span><br><span class="line">-o filename 默认strace将结果输出到stdout。通过-o可以将输出写入到filename文件中</span><br><span class="line">-ff 常与-o选项一起使用，不同进程(子进程)产生的系统调用输出到filename.PID文件</span><br><span class="line">-r 打印每一个系统调用的相对时间</span><br><span class="line">-t 在输出中的每一行前加上时间信息。</span><br><span class="line">-tt 时间确定到微秒级。还可以使用-ttt打印相对时间</span><br><span class="line">-s 指定每一行输出字符串的长度,默认是32。</span><br><span class="line">-c 统计每种系统调用所执行的时间，调用次数，出错次数。</span><br><span class="line">-e expr 输出过滤器，通过表达式，可以过滤出掉你不想要输出</span><br></pre></td></tr></table></figure><p>统计每项调用的时间占比</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">[root@localhost ~]<span class="comment"># strace -cp 2920</span></span><br><span class="line">Process 2920 attached</span><br><span class="line">^CProcess 2920 detached</span><br><span class="line">% time     seconds  usecs/call     calls    errors syscall</span><br><span class="line">------ ----------- ----------- --------- --------- ----------------</span><br><span class="line"> 69.44    0.002775           6       442           epoll_wait</span><br><span class="line">  5.86    0.000234           4        63        63 connect</span><br><span class="line">  5.16    0.000206           4        46           write</span><br><span class="line">  3.53    0.000141           2        73           close</span><br><span class="line">  3.40    0.000136           2        63           socket</span><br><span class="line">  2.53    0.000101           1        70        23 sendto</span><br><span class="line">  2.05    0.000082           1       139         1 setsockopt</span><br><span class="line">  1.83    0.000073           1        73           fcntl</span><br><span class="line">  1.70    0.000068           1        61           epoll_ctl</span><br><span class="line">  1.45    0.000058           1        86        38 recvfrom</span><br><span class="line">  1.43    0.000057           1       103        10 getsockopt</span><br><span class="line">  0.60    0.000024           1        40           semop</span><br><span class="line">  0.55    0.000022           1        20        10 accept</span><br><span class="line">  0.40    0.000016           1        20           ioctl</span><br><span class="line">  0.08    0.000003           0        10           getsockname</span><br><span class="line">------ ----------- ----------- --------- --------- ----------------</span><br><span class="line">100.00    0.003996                  1309       145 total</span><br></pre></td></tr></table></figure><p>跟踪28979进程的所有系统调用（-e trace=all），并统计系统调用的花费时间，以及开始时间（并以可视化的时分秒格式显示），最后将记录结果存在output.txt文件里面</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">strace -o output.txt -T -tt -e trace=all -p 28979</span><br></pre></td></tr></table></figure><p>使用-e跟踪指定的系统调用</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@localhost ~]<span class="comment"># strace -e open cat a.sh </span></span><br><span class="line">open(<span class="string">"/etc/ld.so.cache"</span>, O_RDONLY|O_CLOEXEC) = 3</span><br><span class="line">open(<span class="string">"/usr/lib64/libc.so.6"</span>, O_RDONLY|O_CLOEXEC) = 3</span><br><span class="line">open(<span class="string">"/usr/lib/locale/locale-archive"</span>, O_RDONLY|O_CLOEXEC) = 3</span><br><span class="line">open(<span class="string">"a.sh"</span>, O_RDONLY)                  = 3</span><br></pre></td></tr></table></figure><h1 id="实例一：Ping延时5秒"><a href="#实例一：Ping延时5秒" class="headerlink" title="实例一：Ping延时5秒"></a>实例一：Ping延时5秒</h1><h2 id="故障现象"><a href="#故障现象" class="headerlink" title="故障现象"></a>故障现象</h2><p>线上有一台机器反馈，跟xxx.com交互很慢，在使用ping命令去测试时，发现经常会出现卡住的情况，ping一个包就花费的时间用了5s，但实际上时延很小，如下所示：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[root@12002838 ~]<span class="comment"># time ping -c 1 xxx.com</span></span><br><span class="line">PING xxx.com.cdngtm.com (1.2.3.4) 56(84) bytes of data.</span><br><span class="line">64 bytes from 1.2.3.4: icmp_seq=1 ttl=45 time=35.8 ms</span><br><span class="line"></span><br><span class="line">--- xxx.com.cdngtm.com ping statistics ---</span><br><span class="line">1 packets transmitted, 1 received, 0% packet loss, time 0ms</span><br><span class="line">rtt min/avg/max/mdev = 35.852/35.852/35.852/0.000 ms</span><br><span class="line"></span><br><span class="line">real0m0.224s</span><br><span class="line">user0m0.000s</span><br><span class="line">sys0m0.002s</span><br><span class="line">[root@12002838 ~]<span class="comment">#</span></span><br><span class="line">[root@12002838 ~]<span class="comment"># time ping -c 1 xxx.com</span></span><br><span class="line">PING xxx.com.cdngtm.com (1.2.3.4) 56(84) bytes of data.</span><br><span class="line">64 bytes from 1.2.3.4: icmp_seq=1 ttl=45 time=35.7 ms</span><br><span class="line"></span><br><span class="line">--- xxx.com.cdngtm.com ping statistics ---</span><br><span class="line">1 packets transmitted, 1 received, 0% packet loss, time 0ms</span><br><span class="line">rtt min/avg/max/mdev = 35.783/35.783/35.783/0.000 ms</span><br><span class="line"></span><br><span class="line">real0m5.603s</span><br><span class="line">user0m0.000s</span><br><span class="line">sys0m0.002s</span><br></pre></td></tr></table></figure><p>再接着测试，会发现使用<code>ping -c 1 xxx.com -n</code>或者直接<code>ping 1.2.3.4</code>时，就很正常了。</p><h2 id="分析过程"><a href="#分析过程" class="headerlink" title="分析过程"></a>分析过程</h2><p>从上面的测试中可以看出，从网络连通性来说，是没有问题的，因为时延很低。而ping -n的这个参数的意思是指对解析出来的IP不进行反向解析，所以我们有理由怀疑是反向解析出来了问题。</p><p>但是使用<code>tcpdump -i any -nns0 host 1.2.3.4</code>抓包发现，未没有发现异常情况。</p><p>所以怀疑是系统上面有什么异常导致的，使用strace命令来跟踪下，-tt显示时间，-T显示调用时间，输出如下：<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br></pre></td><td class="code"><pre><span class="line">[op@12002824 ~]$ strace -tt -T ping -c 1 xxx.com</span><br><span class="line">13:42:35.810276 execve(<span class="string">"/usr/bin/ping"</span>, [<span class="string">"ping"</span>, <span class="string">"-c"</span>, <span class="string">"1"</span>, <span class="string">"xxx.com"</span>], [/* 29 vars */]) = 0 &lt;0.000541&gt;</span><br><span class="line">13:42:35.811240 brk(0)                  = 0x7ff9d7b47000 &lt;0.000187&gt;</span><br><span class="line">13:42:35.811671 fcntl(0, F_GETFD)       = 0 &lt;0.000140&gt;</span><br><span class="line">13:42:35.812035 fcntl(1, F_GETFD)       = 0 &lt;0.000107&gt;</span><br><span class="line">13:42:35.812336 fcntl(2, F_GETFD)       = 0 &lt;0.000047&gt;</span><br><span class="line">13:42:35.812542 access(<span class="string">"/etc/suid-debug"</span>, F_OK) = -1 ENOENT (No such file or directory) &lt;0.000056&gt;</span><br><span class="line">13:42:35.812853 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6427000 &lt;0.000097&gt;</span><br><span class="line">13:42:35.813064 access(<span class="string">"/etc/ld.so.preload"</span>, R_OK) = -1 ENOENT (No such file or directory) &lt;0.000052&gt;</span><br><span class="line">13:42:35.813223 open(<span class="string">"/etc/ld.so.cache"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000025&gt;</span><br><span class="line">13:42:35.813307 fstat(3, &#123;st_mode=S_IFREG|0644, st_size=59413, ...&#125;) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.813356 mmap(NULL, 59413, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff9d6418000 &lt;0.000015&gt;</span><br><span class="line">13:42:35.813400 close(3)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.813442 open(<span class="string">"/usr/lib64/libcap.so.2"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000012&gt;</span><br><span class="line">13:42:35.813481 <span class="built_in">read</span>(3, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0 \26\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000009&gt;</span><br><span class="line">13:42:35.813519 fstat(3, &#123;st_mode=S_IFREG|0755, st_size=20024, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.813561 mmap(NULL, 2114112, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff9d6002000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.813599 mprotect(0x7ff9d6006000, 2093056, PROT_NONE) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.813638 mmap(0x7ff9d6205000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7ff9d6205000 &lt;0.000012&gt;</span><br><span class="line">13:42:35.813684 close(3)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.813722 open(<span class="string">"/usr/lib64/libidn.so.11"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000012&gt;</span><br><span class="line">13:42:35.813763 <span class="built_in">read</span>(3, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0\0000\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000009&gt;</span><br><span class="line">13:42:35.813801 fstat(3, &#123;st_mode=S_IFREG|0755, st_size=208928, ...&#125;) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.813839 mmap(NULL, 2302416, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff9d5dcf000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.813875 mprotect(0x7ff9d5e01000, 2093056, PROT_NONE) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.813910 mmap(0x7ff9d6000000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x31000) = 0x7ff9d6000000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.813954 close(3)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.813991 open(<span class="string">"/usr/lib64/libc.so.6"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000010&gt;</span><br><span class="line">13:42:35.814030 <span class="built_in">read</span>(3, <span class="string">"\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0P%\2\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814067 fstat(3, &#123;st_mode=S_IFREG|0755, st_size=2173512, ...&#125;) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.814104 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6417000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814143 mmap(NULL, 3981792, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff9d5a02000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814179 mprotect(0x7ff9d5bc5000, 2093056, PROT_NONE) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.814217 mmap(0x7ff9d5dc4000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c2000) = 0x7ff9d5dc4000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.814257 mmap(0x7ff9d5dca000, 16864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff9d5dca000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.814298 close(3)                = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.814336 open(<span class="string">"/usr/lib64/libattr.so.1"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000011&gt;</span><br><span class="line">13:42:35.814375 <span class="built_in">read</span>(3, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0\320\23\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814413 fstat(3, &#123;st_mode=S_IFREG|0755, st_size=19888, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814450 mmap(NULL, 2113904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7ff9d57fd000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.814488 mprotect(0x7ff9d5801000, 2093056, PROT_NONE) = 0 &lt;0.000010&gt;</span><br><span class="line">13:42:35.814524 mmap(0x7ff9d5a00000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x7ff9d5a00000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.814569 close(3)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814613 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6416000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814651 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6414000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814691 arch_prctl(ARCH_SET_FS, 0x7ff9d6414740) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.814775 mprotect(0x7ff9d5dc4000, 16384, PROT_READ) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.814817 mprotect(0x7ff9d5a00000, 4096, PROT_READ) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814862 mprotect(0x7ff9d6000000, 4096, PROT_READ) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814900 mprotect(0x7ff9d6205000, 4096, PROT_READ) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814957 mprotect(0x7ff9d6634000, 4096, PROT_READ) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.814994 mprotect(0x7ff9d6428000, 4096, PROT_READ) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.815029 munmap(0x7ff9d6418000, 59413) = 0 &lt;0.000013&gt;</span><br><span class="line">13:42:35.815117 brk(0)                  = 0x7ff9d7b47000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815152 brk(0x7ff9d7b68000)     = 0x7ff9d7b68000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.815186 brk(0)                  = 0x7ff9d7b68000 &lt;0.000007&gt;</span><br><span class="line">13:42:35.815224 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, NULL) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.815264 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;CAP_NET_ADMIN|CAP_NET_RAW, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815303 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, NULL) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.815340 capset(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;0, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815378 prctl(PR_SET_KEEPCAPS, 1) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815414 getuid()                = 1000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815448 setuid(1000)            = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.815481 prctl(PR_SET_KEEPCAPS, 0) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815515 getuid()                = 1000 &lt;0.000007&gt;</span><br><span class="line">13:42:35.815554 geteuid()               = 1000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815601 open(<span class="string">"/usr/lib/locale/locale-archive"</span>, O_RDONLY|O_CLOEXEC) = 3 &lt;0.000012&gt;</span><br><span class="line">13:42:35.815642 fstat(3, &#123;st_mode=S_IFREG|0644, st_size=106070960, ...&#125;) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.815680 mmap(NULL, 106070960, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7ff9cf2d4000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.815720 close(3)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815774 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, NULL) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815810 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;0, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815849 capset(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;CAP_NET_RAW, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.815889 socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3 &lt;0.000013&gt;</span><br><span class="line">13:42:35.815935 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, NULL) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.815970 capget(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;CAP_NET_RAW, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816007 capset(&#123;_LINUX_CAPABILITY_VERSION_3, 0&#125;, &#123;0, CAP_NET_ADMIN|CAP_NET_RAW, 0&#125;) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.816075 <span class="built_in">stat</span>(<span class="string">"/etc/resolv.conf"</span>, &#123;st_mode=S_IFREG|0644, st_size=72, ...&#125;) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.816125 open(<span class="string">"/etc/host.conf"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.816168 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=9, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816205 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6426000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.816242 <span class="built_in">read</span>(4, <span class="string">"multi on\n"</span>, 4096) = 9 &lt;0.000011&gt;</span><br><span class="line">13:42:35.816287 <span class="built_in">read</span>(4, <span class="string">""</span>, 4096)       = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816323 close(4)                = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.816357 munmap(0x7ff9d6426000, 4096) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.816396 open(<span class="string">"/etc/resolv.conf"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000010&gt;</span><br><span class="line">13:42:35.816436 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=72, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816473 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6426000 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816510 <span class="built_in">read</span>(4, <span class="string">"nameserver 119.29.29.29\nnameserv"</span>..., 4096) = 72 &lt;0.000010&gt;</span><br><span class="line">13:42:35.816551 <span class="built_in">read</span>(4, <span class="string">""</span>, 4096)       = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816585 close(4)                = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.816620 munmap(0x7ff9d6426000, 4096) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.816657 uname(&#123;sys=<span class="string">"Linux"</span>, node=<span class="string">"12002824"</span>, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.816707 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 &lt;0.000012&gt;</span><br><span class="line">13:42:35.816746 connect(4, &#123;sa_family=AF_LOCAL, sun_path=<span class="string">"/var/run/nscd/socket"</span>&#125;, 110) = -1 ENOENT (No such file or directory) &lt;0.000018&gt;</span><br><span class="line">13:42:35.816853 close(4)                = 0 &lt;0.000010&gt;</span><br><span class="line">13:42:35.816893 socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 4 &lt;0.000009&gt;</span><br><span class="line">13:42:35.816929 connect(4, &#123;sa_family=AF_LOCAL, sun_path=<span class="string">"/var/run/nscd/socket"</span>&#125;, 110) = -1 ENOENT (No such file or directory) &lt;0.000011&gt;</span><br><span class="line">13:42:35.816970 close(4)                = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817009 open(<span class="string">"/etc/nsswitch.conf"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000010&gt;</span><br><span class="line">13:42:35.817047 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=1717, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817085 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6426000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817122 <span class="built_in">read</span>(4, <span class="string">"#\n# /etc/nsswitch.conf\n#\n# An ex"</span>..., 4096) = 1717 &lt;0.000010&gt;</span><br><span class="line">13:42:35.817170 <span class="built_in">read</span>(4, <span class="string">""</span>, 4096)       = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817205 close(4)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817239 munmap(0x7ff9d6426000, 4096) = 0 &lt;0.000010&gt;</span><br><span class="line">13:42:35.817284 open(<span class="string">"/etc/ld.so.cache"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000010&gt;</span><br><span class="line">13:42:35.817322 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=59413, ...&#125;) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.817358 mmap(NULL, 59413, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7ff9d6418000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817394 close(4)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817433 open(<span class="string">"/usr/lib64/libnss_files.so.2"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.817473 <span class="built_in">read</span>(4, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0\320!\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817510 fstat(4, &#123;st_mode=S_IFREG|0755, st_size=62184, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817549 mmap(NULL, 2173048, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7ff9cf0c1000 &lt;0.000012&gt;</span><br><span class="line">13:42:35.817587 mprotect(0x7ff9cf0cd000, 2093056, PROT_NONE) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.817626 mmap(0x7ff9cf2cc000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0xb000) = 0x7ff9cf2cc000 &lt;0.000012&gt;</span><br><span class="line">13:42:35.817669 mmap(0x7ff9cf2ce000, 22648, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff9cf2ce000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817711 close(4)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817755 mprotect(0x7ff9cf2cc000, 4096, PROT_READ) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.817799 munmap(0x7ff9d6418000, 59413) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.817844 open(<span class="string">"/etc/hosts"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.817885 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=271, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.817922 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6426000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.817959 <span class="built_in">read</span>(4, <span class="string">"127.0.0.1   localhost localhost."</span>..., 4096) = 271 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818006 <span class="built_in">read</span>(4, <span class="string">""</span>, 4096)       = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818042 close(4)                = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.818076 munmap(0x7ff9d6426000, 4096) = 0 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818115 open(<span class="string">"/etc/ld.so.cache"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818153 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=59413, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818190 mmap(NULL, 59413, PROT_READ, MAP_PRIVATE, 4, 0) = 0x7ff9d6418000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.818226 close(4)                = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.818264 open(<span class="string">"/usr/lib64/libnss_dns.so.2"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.818303 <span class="built_in">read</span>(4, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0\260\20\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818340 fstat(4, &#123;st_mode=S_IFREG|0755, st_size=31824, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818378 mmap(NULL, 2121984, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7ff9ceeba000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818416 mprotect(0x7ff9ceebf000, 2097152, PROT_NONE) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.818454 mmap(0x7ff9cf0bf000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x5000) = 0x7ff9cf0bf000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818499 close(4)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818538 open(<span class="string">"/usr/lib64/libresolv.so.2"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.818577 <span class="built_in">read</span>(4, <span class="string">"\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0&gt;\0\1\0\0\0@9\0\0\0\0\0\0"</span>..., 832) = 832 &lt;0.000009&gt;</span><br><span class="line">13:42:35.818614 fstat(4, &#123;st_mode=S_IFREG|0755, st_size=106848, ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818652 mmap(NULL, 2198016, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 4, 0) = 0x7ff9ceca1000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818690 mprotect(0x7ff9cecb7000, 2093056, PROT_NONE) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.818727 mmap(0x7ff9ceeb6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 4, 0x15000) = 0x7ff9ceeb6000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.818773 mmap(0x7ff9ceeb8000, 6656, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7ff9ceeb8000 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818812 close(4)                = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.818862 mprotect(0x7ff9ceeb6000, 4096, PROT_READ) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.818953 mprotect(0x7ff9cf0bf000, 4096, PROT_READ) = 0 &lt;0.000010&gt;</span><br><span class="line">13:42:35.818991 munmap(0x7ff9d6418000, 59413) = 0 &lt;0.000013&gt;</span><br><span class="line">13:42:35.819050 socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 4 &lt;0.000011&gt;</span><br><span class="line">13:42:35.819088 connect(4, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"119.29.29.29"</span>)&#125;, 16) = 0 &lt;0.000014&gt;</span><br><span class="line">13:42:35.819142 poll([&#123;fd=4, events=POLLOUT&#125;], 1, 0) = 1 ([&#123;fd=4, revents=POLLOUT&#125;]) &lt;0.000009&gt;</span><br><span class="line">13:42:35.819184 sendto(4, <span class="string">"\357\235\1\0\0\1\0\0\0\0\0\0\6omsapi\7snstack\2cn\0\0"</span>..., 35, MSG_NOSIGNAL, NULL, 0) = 35 &lt;0.000024&gt;</span><br><span class="line">13:42:35.819238 poll([&#123;fd=4, events=POLLIN&#125;], 1, 5000) = 1 ([&#123;fd=4, revents=POLLIN&#125;]) &lt;0.019979&gt;</span><br><span class="line">13:42:35.839285 ioctl(4, FIONREAD, [108]) = 0 &lt;0.000055&gt;</span><br><span class="line">13:42:35.839388 recvfrom(4, <span class="string">"\357\235\201\200\0\1\0\3\0\0\0\0\6omsapi\7snstack\2cn\0\0"</span>..., 1024, 0, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"119.29.29.29"</span>)&#125;, [16]) = 108 &lt;0.000014&gt;</span><br><span class="line">13:42:35.839450 close(4)                = 0 &lt;0.000016&gt;</span><br><span class="line">13:42:35.839515 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 4 &lt;0.000012&gt;</span><br><span class="line">13:42:35.839551 connect(4, &#123;sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr(<span class="string">"163.177.221.143"</span>)&#125;, 16) = 0 &lt;0.000012&gt;</span><br><span class="line">13:42:35.839629 getsockname(4, &#123;sa_family=AF_INET, sin_port=htons(41990), sin_addr=inet_addr(<span class="string">"218.107.205.242"</span>)&#125;, [16]) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839667 close(4)                = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.839707 setsockopt(3, SOL_RAW, ICMP_FILTER, ~(ICMP_ECHOREPLY|ICMP_DEST_UNREACH|ICMP_SOURCE_QUENCH|ICMP_REDIRECT|ICMP_TIME_EXCEEDED|ICMP_PARAMETERPROB), 4) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839745 setsockopt(3, SOL_IP, IP_RECVERR, [1], 4) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.839786 setsockopt(3, SOL_SOCKET, SO_SNDBUF, [324], 4) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839825 setsockopt(3, SOL_SOCKET, SO_RCVBUF, [65536], 4) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839861 getsockopt(3, SOL_SOCKET, SO_RCVBUF, [131072], [4]) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839906 fstat(1, &#123;st_mode=S_IFCHR|0620, st_rdev=makedev(136, 4), ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.839944 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6426000 &lt;0.000009&gt;</span><br><span class="line">13:42:35.839985 write(1, <span class="string">"PING xxx.com.cdngtm.com ("</span>..., 71PING xxx.com.cdngtm.com (163.177.221.143) 56(84) bytes of data.</span><br><span class="line">) = 71 &lt;0.000016&gt;</span><br><span class="line">13:42:35.840030 setsockopt(3, SOL_SOCKET, SO_TIMESTAMP, [1], 4) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.840066 setsockopt(3, SOL_SOCKET, SO_SNDTIMEO, <span class="string">"\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span>, 16) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840102 setsockopt(3, SOL_SOCKET, SO_RCVTIMEO, <span class="string">"\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"</span>, 16) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840137 getpid()                = 20102 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840170 rt_sigaction(SIGINT, &#123;0x7ff9d642fbd0, [], SA_RESTORER|SA_INTERRUPT, 0x7ff9d5a382f0&#125;, NULL, 8) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840214 rt_sigaction(SIGALRM, &#123;0x7ff9d642fbd0, [], SA_RESTORER|SA_INTERRUPT, 0x7ff9d5a382f0&#125;, NULL, 8) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840252 rt_sigaction(SIGQUIT, &#123;0x7ff9d642fbc0, [], SA_RESTORER|SA_INTERRUPT, 0x7ff9d5a382f0&#125;, NULL, 8) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.840288 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840326 ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, &#123;B38400 opost isig icanon <span class="built_in">echo</span> ...&#125;) = 0 &lt;0.000008&gt;</span><br><span class="line">13:42:35.840364 ioctl(1, TIOCGWINSZ, &#123;ws_row=42, ws_col=210, ws_xpixel=0, ws_ypixel=0&#125;) = 0 &lt;0.000007&gt;</span><br><span class="line">13:42:35.840401 sendmsg(3, &#123;msg_name(16)=&#123;sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr(<span class="string">"163.177.221.143"</span>)&#125;, msg_iov(1)=[&#123;<span class="string">"\10\0\5\273N\206\0\1K\272\277]\0\0\0\0\315\322\f\0\0\0\0\0\20\21\22\23\24\25\26\27"</span>..., 64&#125;], msg_controllen=0, msg_flags=0&#125;, 0) = 64 &lt;0.000023&gt;</span><br><span class="line">13:42:35.840460 setitimer(ITIMER_REAL, &#123;it_interval=&#123;0, 0&#125;, it_value=&#123;10, 0&#125;&#125;, NULL) = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.840500 recvmsg(3, &#123;msg_name(16)=&#123;sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr(<span class="string">"163.177.221.143"</span>)&#125;, msg_iov(1)=[&#123;<span class="string">"E\0\0T\315\374\0\0005\1\216\r\243\261\335\217\332k\315\362\0\0\r\273N\206\0\1K\272\277]"</span>..., 192&#125;], msg_controllen=32, &#123;cmsg_len=32, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...&#125;, msg_flags=0&#125;, 0) = 84 &lt;0.020365&gt;</span><br><span class="line">13:42:35.860923 <span class="built_in">stat</span>(<span class="string">"/etc/resolv.conf"</span>, &#123;st_mode=S_IFREG|0644, st_size=72, ...&#125;) = 0 &lt;0.000016&gt;</span><br><span class="line">13:42:35.861009 open(<span class="string">"/etc/hosts"</span>, O_RDONLY|O_CLOEXEC) = 4 &lt;0.000015&gt;</span><br><span class="line">13:42:35.861060 fstat(4, &#123;st_mode=S_IFREG|0644, st_size=271, ...&#125;) = 0 &lt;0.000011&gt;</span><br><span class="line">13:42:35.861103 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7ff9d6425000 &lt;0.000011&gt;</span><br><span class="line">13:42:35.861143 <span class="built_in">read</span>(4, <span class="string">"127.0.0.1   localhost localhost."</span>..., 4096) = 271 &lt;0.000015&gt;</span><br><span class="line">13:42:35.861198 <span class="built_in">read</span>(4, <span class="string">""</span>, 4096)       = 0 &lt;0.000009&gt;</span><br><span class="line">13:42:35.861235 close(4)                = 0 &lt;0.000046&gt;</span><br><span class="line">13:42:35.861323 munmap(0x7ff9d6425000, 4096) = 0 &lt;0.000022&gt;</span><br><span class="line">13:42:35.861408 socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 4 &lt;0.000015&gt;</span><br><span class="line">13:42:35.861466 connect(4, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"119.29.29.29"</span>)&#125;, 16) = 0 &lt;0.000019&gt;</span><br><span class="line">13:42:35.861528 poll([&#123;fd=4, events=POLLOUT&#125;], 1, 0) = 1 ([&#123;fd=4, revents=POLLOUT&#125;]) &lt;0.000010&gt;</span><br><span class="line">13:42:35.861583 sendto(4, <span class="string">"\34\312\1\0\0\1\0\0\0\0\0\0\003143\003221\003177\003163\7in-"</span>..., 46, MSG_NOSIGNAL, NULL, 0) = 46 &lt;0.000021&gt;</span><br><span class="line">13:42:35.861636 poll([&#123;fd=4, events=POLLIN&#125;], 1, 5000) = 0 (Timeout) &lt;5.005176&gt;</span><br><span class="line"></span><br><span class="line">13:42:40.866995 socket(PF_INET, SOCK_DGRAM|SOCK_CLOEXEC|SOCK_NONBLOCK, IPPROTO_IP) = 5 &lt;0.000133&gt;</span><br><span class="line">13:42:40.867341 connect(5, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"223.5.5.5"</span>)&#125;, 16) = 0 &lt;0.000134&gt;</span><br><span class="line">13:42:40.867662 poll([&#123;fd=5, events=POLLOUT&#125;], 1, 0) = 1 ([&#123;fd=5, revents=POLLOUT&#125;]) &lt;0.000198&gt;</span><br><span class="line">13:42:40.868077 sendto(5, <span class="string">"\34\312\1\0\0\1\0\0\0\0\0\0\003143\003221\003177\003163\7in-"</span>..., 46, MSG_NOSIGNAL, NULL, 0) = 46 &lt;0.000100&gt;</span><br><span class="line">13:42:40.868271 poll([&#123;fd=5, events=POLLIN&#125;], 1, 3000) = 1 ([&#123;fd=5, revents=POLLIN&#125;]) &lt;0.018325&gt;</span><br><span class="line">13:42:40.886880 ioctl(5, FIONREAD, [121]) = 0 &lt;0.000097&gt;</span><br><span class="line">13:42:40.887278 recvfrom(5, <span class="string">"\34\312\201\203\0\1\0\0\0\1\0\0\003143\003221\003177\003163\7in-"</span>..., 1024, 0, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"223.5.5.5"</span>)&#125;, [16]) = 121 &lt;0.000200&gt;</span><br><span class="line">13:42:40.887805 close(4)                = 0 &lt;0.000149&gt;</span><br><span class="line">13:42:40.888173 close(5)                = 0 &lt;0.000047&gt;</span><br><span class="line">13:42:40.888424 write(1, <span class="string">"64 bytes from 163.177.221.143: i"</span>..., 6264 bytes from 163.177.221.143: icmp_seq=1 ttl=53 time=20.4 ms</span><br><span class="line">) = 62 &lt;0.000121&gt;</span><br><span class="line">13:42:40.888708 write(1, <span class="string">"\n"</span>, 1</span><br><span class="line">)       = 1 &lt;0.000045&gt;</span><br><span class="line">13:42:40.888867 write(1, <span class="string">"--- xxx.com.cdngtm.com pi"</span>..., 50--- xxx.com.cdngtm.com ping statistics ---</span><br><span class="line">) = 50 &lt;0.000036&gt;</span><br><span class="line">13:42:40.889016 write(1, <span class="string">"1 packets transmitted, 1 receive"</span>..., 601 packets transmitted, 1 received, 0% packet loss, time 0ms</span><br><span class="line">) = 60 &lt;0.000011&gt;</span><br><span class="line">13:42:40.889073 write(1, <span class="string">"rtt min/avg/max/mdev = 20.426/20"</span>..., 53rtt min/avg/max/mdev = 20.426/20.426/20.426/0.000 ms</span><br><span class="line">) = 53 &lt;0.000008&gt;</span><br><span class="line">13:42:40.889139 exit_group(0)           = ?</span><br><span class="line">13:42:40.889364 +++ exited with 0 +++</span><br></pre></td></tr></table></figure></p><p>根据strace的输出，可以看到，<code>13:42:35.861636 poll([{fd=4, events=POLLIN}], 1, 5000) = 0 (Timeout) &lt;5.005176&gt;</code>这里卡死了5s，结合上下文对比一下，</p><p>使用dig来查看一下：<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">[root@12002838 ~]<span class="comment"># dig -x 1.2.3.4 @119.29.29.29</span></span><br><span class="line"></span><br><span class="line">; &lt;&lt;&gt;&gt; DiG 9.9.4-RedHat-9.9.4-18.el7 &lt;&lt;&gt;&gt; -x 1.2.3.4 @119.29.29.29</span><br><span class="line">;; global options: +cmd</span><br><span class="line">;; Got answer:</span><br><span class="line">;; -&gt;&gt;HEADER&lt;&lt;- opcode: QUERY, status: NXDOMAIN, id: 64491</span><br><span class="line">;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1</span><br><span class="line"></span><br><span class="line">;; OPT PSEUDOSECTION:</span><br><span class="line">; EDNS: version: 0, flags:; udp: 4096</span><br><span class="line">;; QUESTION SECTION:</span><br><span class="line">;111.242.61.183.in-addr.arpa.INPTR</span><br><span class="line"></span><br><span class="line">;; AUTHORITY SECTION:</span><br><span class="line">61.183.in-addr.arpa.600INSOAsoa. dns.guangzhou.gd.cn. 2016012139 10800 3600 604800 86400</span><br><span class="line"></span><br><span class="line">;; Query time: 42 msec</span><br><span class="line">;; SERVER: 119.29.29.29<span class="comment">#53(119.29.29.29)</span></span><br><span class="line">;; WHEN: Mon Nov 04 17:01:13 CST 2019</span><br><span class="line">;; MSG SIZE  rcvd: 114</span><br><span class="line"></span><br><span class="line">[root@12002838 ~]<span class="comment"># dig -x 1.2.3.4 @119.29.29.29</span></span><br><span class="line"></span><br><span class="line">; &lt;&lt;&gt;&gt; DiG 9.9.4-RedHat-9.9.4-18.el7 &lt;&lt;&gt;&gt; -x 1.2.3.4 @119.29.29.29</span><br><span class="line">;; global options: +cmd</span><br><span class="line">;; connection timed out; no servers could be reached</span><br></pre></td></tr></table></figure></p><p>结合抓包来分析的话，可以看到反向解析异常了。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[op@12002838 ~]$ tcpdump -r ptr-119.29.29.29.pcap</span><br><span class="line">reading from file ptr-119.29.29.29.pcap, link-type LINUX_SLL (Linux cooked)</span><br><span class="line">17:01:13.110935 IP 1.146.21.211.58085 &gt; pdns.dnspod.cn.domain: 64491+ [1au] PTR? 111.242.61.183.in-addr.arpa. (56)</span><br><span class="line">17:01:13.153168 IP pdns.dnspod.cn.domain &gt; 1.146.21.211.58085: 64491 NXDomain 0/1/1 (114)</span><br><span class="line">17:01:15.718078 IP 1.146.21.211.10569 &gt; pdns.dnspod.cn.domain: 10577+ [1au] PTR? 111.242.61.183.in-addr.arpa. (56)</span><br><span class="line">17:01:20.718195 IP 1.146.21.211.10569 &gt; pdns.dnspod.cn.domain: 10577+ [1au] PTR? 111.242.61.183.in-addr.arpa. (56)</span><br><span class="line">17:01:25.718336 IP 1.146.21.211.10569 &gt; pdns.dnspod.cn.domain: 10577+ [1au] PTR? 111.242.61.183.in-addr.arpa. (56)</span><br></pre></td></tr></table></figure><h1 id="实例二：zk客户端连接慢"><a href="#实例二：zk客户端连接慢" class="headerlink" title="实例二：zk客户端连接慢"></a>实例二：zk客户端连接慢</h1><p>最近在部署zookeeper的时候，在本机使用<code>/cache1/zookeeper/bin/zkCli.sh -server 10.8.50.189</code>连接ZK时非常慢，需要等待10几秒才能由CONNECTING变成CONNECTED，如下图：</p><p><img src="/assets/image-20210509211717840.png" alt></p><p>刚开始怀疑是网络问题，使用tcpdump抓包之后，发现zookeeper也是隔了15S左右才发现的建链请求。</p><p>使用<code>strace -f -TT -t -o 189.txt /cache1/zookeeper/bin/zkCli.sh -server 10.8.50.189</code>来跟踪，这时会将zkCli.sh所加载的子进程一起打印到189.txt的结果里面。我们取最后面一列，时间大于1秒的结果来分析</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line">[root@misa ~]<span class="comment"># cat 189.txt |egrep -v 'si_signo|unfinished|exited with|exit_group' |awk 'BEGIN&#123;FS="[&lt;&gt;]+"&#125;&#123;if($(NF-1)&gt;1)print $0&#125;'</span></span><br><span class="line">13695 17:55:28.198956 &lt;... futex resumed&gt; ) = ? ERESTARTSYS (To be restarted <span class="keyword">if</span> SA_RESTART is <span class="built_in">set</span>) &lt;1.241380&gt;</span><br><span class="line">13705 17:55:29.181802 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000035&gt;</span><br><span class="line">13705 17:55:30.182504 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000074&gt;</span><br><span class="line">13705 17:55:31.182737 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000081&gt;</span><br><span class="line">13705 17:55:32.286019 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000100&gt;</span><br><span class="line">13726 17:55:32.995436 &lt;... poll resumed&gt; ) = 0 (Timeout) &lt;5.004650&gt;</span><br><span class="line">13705 17:55:33.286386 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000060&gt;</span><br><span class="line">13711 17:55:33.294683 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000069&gt;</span><br><span class="line">13710 17:55:33.294901 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000050&gt;</span><br><span class="line">13709 17:55:33.295178 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000063&gt;</span><br><span class="line">13712 17:55:33.295427 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000062&gt;</span><br><span class="line">13705 17:55:34.286639 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000061&gt;</span><br><span class="line">13705 17:55:35.286925 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000109&gt;</span><br><span class="line">13705 17:55:36.287219 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000071&gt;</span><br><span class="line">13705 17:55:37.287529 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000072&gt;</span><br><span class="line">13726 17:55:38.000944 &lt;... poll resumed&gt; ) = 0 (Timeout) &lt;5.005081&gt;</span><br><span class="line">13705 17:55:38.287743 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000067&gt;</span><br><span class="line">13711 17:55:38.294922 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000071&gt;</span><br><span class="line">13710 17:55:38.295131 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000059&gt;</span><br><span class="line">13709 17:55:38.295420 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000060&gt;</span><br><span class="line">13712 17:55:38.295642 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000073&gt;</span><br><span class="line">13705 17:55:39.288386 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000053&gt;</span><br><span class="line">13705 17:55:40.288629 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000065&gt;</span><br><span class="line">13705 17:55:41.288916 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000013&gt;</span><br><span class="line">13705 17:55:42.289220 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000056&gt;</span><br><span class="line">13726 17:55:43.008426 &lt;... poll resumed&gt; ) = 0 (Timeout) &lt;5.005070&gt;</span><br><span class="line">13705 17:55:43.289487 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000055&gt;</span><br><span class="line">13711 17:55:43.295181 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000061&gt;</span><br><span class="line">13710 17:55:43.295363 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000048&gt;</span><br><span class="line">13709 17:55:43.295633 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000005&gt;</span><br><span class="line">13712 17:55:43.295859 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;5.000047&gt;</span><br><span class="line">13705 17:55:44.289740 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000077&gt;</span><br><span class="line">13705 17:55:45.290048 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000095&gt;</span><br><span class="line">13705 17:55:46.290380 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000088&gt;</span><br><span class="line">13705 17:55:47.290651 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000055&gt;</span><br><span class="line">13726 17:55:48.012491 &lt;... poll resumed&gt; ) = 0 (Timeout) &lt;5.003801&gt;</span><br><span class="line">13711 17:55:48.045227 &lt;... futex resumed&gt; ) = 0 &lt;4.749875&gt;</span><br><span class="line">13710 17:55:48.045447 &lt;... futex resumed&gt; ) = 0 &lt;4.749921&gt;</span><br><span class="line">13709 17:55:48.045658 &lt;... futex resumed&gt; ) = 0 &lt;4.749881&gt;</span><br><span class="line">13712 17:55:48.045848 &lt;... futex resumed&gt; ) = 0 &lt;4.749845&gt;</span><br><span class="line">13727 17:55:48.062549 &lt;... futex resumed&gt; ) = 0 &lt;20.085666&gt;</span><br><span class="line">13705 17:55:48.290963 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000139&gt;</span><br><span class="line">13705 17:55:49.291661 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000052&gt;</span><br><span class="line">13705 17:55:50.291893 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000062&gt;</span><br><span class="line">13705 17:55:51.292141 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000072&gt;</span><br><span class="line">13696 17:55:51.310297 &lt;... <span class="built_in">read</span> resumed&gt; <span class="string">"\r"</span>, 1) = 1 &lt;23.003900&gt;</span><br><span class="line">13709 17:55:51.311488 &lt;... futex resumed&gt; ) = 0 &lt;3.247963&gt;</span><br><span class="line">13711 17:55:51.311853 &lt;... futex resumed&gt; ) = 0 &lt;3.247834&gt;</span><br><span class="line">13712 17:55:51.312082 &lt;... futex resumed&gt; ) = 0 &lt;3.247748&gt;</span><br><span class="line">13710 17:55:51.312409 &lt;... futex resumed&gt; ) = 0 &lt;3.247753&gt;</span><br><span class="line">13729 17:55:51.320436 &lt;... futex resumed&gt; ) = 0 &lt;23.014312&gt;</span><br><span class="line">13705 17:55:52.344542 &lt;... futex resumed&gt; ) = -1 ETIMEDOUT (Connection timed out) &lt;1.000074&gt;</span><br><span class="line">13695 17:55:52.806638 &lt;... futex resumed&gt; ) = ? ERESTARTSYS (To be restarted <span class="keyword">if</span> SA_RESTART is <span class="built_in">set</span>) &lt;24.550556&gt;</span><br><span class="line">13687 17:55:52.806675 &lt;... wait4 resumed&gt; 0x7fff0dcc7610, 0, NULL) = ? ERESTARTSYS (To be restarted <span class="keyword">if</span> SA_RESTART is <span class="built_in">set</span>) &lt;25.861684&gt;</span><br><span class="line">13708 17:55:52.806868 &lt;... futex resumed&gt; ) = 0 &lt;25.566062&gt;</span><br><span class="line">13729 17:55:52.821612 &lt;... futex resumed&gt; ) = 0 &lt;1.476969&gt;</span><br><span class="line">13918 17:55:52.828485 _exit(0)          = ?</span><br><span class="line">13714 17:55:52.830662 _exit(0)          = ?</span><br></pre></td></tr></table></figure><p>然后就对上述日志进行排查，发现了以下的异常：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">13726 17:55:27.990116 uname(&#123;sys=<span class="string">"Linux"</span>, node=<span class="string">"misa"</span>, ...&#125;) = 0 &lt;0.000029&gt;</span><br><span class="line">13726 17:55:27.990215 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 44 &lt;0.000050&gt;</span><br><span class="line">13726 17:55:27.990327 connect(44, &#123;sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr(<span class="string">"114.114.114.114"</span>)&#125;, 16) = 0 &lt;0.000050&gt;</span><br><span class="line">13726 17:55:27.990439 poll([&#123;fd=44, events=POLLOUT&#125;], 1, 0) = 1 ([&#123;fd=44, revents=POLLOUT&#125;]) &lt;0.000033&gt;</span><br><span class="line">13726 17:55:27.990529 sendto(44, <span class="string">"\247\214\1\0\0\1\0\0\0\0\0\0\003189\00250\0018\00210\7in-addr"</span>..., 42, MSG_NOSIGNAL, NULL, 0) = 42 &lt;0.000075&gt;</span><br><span class="line">13726 17:55:27.990754 poll([&#123;fd=44, events=POLLIN&#125;], 1, 5000 &lt;unfinished ...&gt;</span><br><span class="line">13726 17:55:32.995436 &lt;... poll resumed&gt; ) = 0 (Timeout) &lt;5.004650&gt;</span><br></pre></td></tr></table></figure><p>线路13726有发起对114 DNS的请求，但是我们不清楚是什么域名，那就再次动用tcdump来分析问题：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@misa ~]<span class="comment"># tcpdump -i any -nns0 port 53 </span></span><br><span class="line">tcpdump: verbose output suppressed, use -v or -vv <span class="keyword">for</span> full protocol decode</span><br><span class="line">listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes</span><br><span class="line">21:36:42.254545 IP 10.8.50.190.42709 &gt; 114.114.114.114.53: 51804+ PTR? 189.50.8.10.in-addr.arpa. (42)</span><br><span class="line">21:36:47.258444 IP 10.8.50.190.42709 &gt; 114.114.114.114.53: 51804+ PTR? 189.50.8.10.in-addr.arpa. (42)</span><br><span class="line">21:36:52.264034 IP 10.8.50.190.59759 &gt; 114.114.114.114.53: 28250+ PTR? 189.50.8.10.in-addr.arpa. (42)</span><br><span class="line">21:36:57.269104 IP 10.8.50.190.59759 &gt; 114.114.114.114.53: 28250+ PTR? 189.50.8.10.in-addr.arpa. (42)</span><br></pre></td></tr></table></figure><p>那就问题很明显了，还是遇到了ptr反向解析的问题了。我们用host命令来测试下是不是很慢：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@misa ~]<span class="comment"># time host 10.8.50.191</span></span><br><span class="line">;; connection timed out; no servers could be reached</span><br><span class="line"></span><br><span class="line">real0m10.007s</span><br><span class="line">user0m0.004s</span><br><span class="line">sys0m0.003s</span><br></pre></td></tr></table></figure><p>也可以用 <code>dig -x 10.8.50.189</code> 来测试下，证明都是很慢。所以极大的可能是反向解析的问题，为了验证确认是这个问题导致的，我把nameserver都注释掉，发现一切正常了，秒级连上zk。</p><p>但问题是，为什么zk会去反向解析这个IP呢？换句话说，怎么样解决这个问题呢？</p><p>第一个想法是通过zk的参数去关闭反向解析，但翻查了官方文档 <a href="https://zookeeper.apache.org/doc/r3.6.3/zookeeperAdmin.html" target="_blank" rel="noopener">https://zookeeper.apache.org/doc/r3.6.3/zookeeperAdmin.html</a> 之后，发现没有找到有这个选项。此方法不通。</p><p>第二个方法是想通过强制解析下ptr就可以了，但是/etc/hosts是不支持的，难道为了这个问题，还得去搭建一下DNS服务器吗？不值得。</p><p>思考了半天，我在/etc/hosts加上主机名的解析，即<code>10.8.50.189    misa</code>，这样是否可以呢？</p><p>神奇的发现，加上之后问题解决了，并且抓包也没有反向解析了，这是为什么呢？求大神指导。</p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;使用说明&quot;&gt;&lt;a href=&quot;#使用说明&quot; class=&quot;headerlink&quot; title=&quot;使用说明&quot;&gt;&lt;/a&gt;使用说明&lt;/h1&gt;&lt;p&gt;strace是一个非常简单的工具，它可以跟踪系统调用的执行。最简单的方式，它可以从头到尾跟踪binary的执行，然后以一行文本输出系统调用的名字，参数和返回值。&lt;/p&gt;
&lt;p&gt;strace的最简单的用法就是执行一个指定的命令，如：$ strace cat /dev/null 在命令结束之后它也就退出了。在命令执行的过程中，strace会记录和解析命令进程的所有系统调用以及这个进程所接收到的所有的信号值。&lt;/p&gt;
&lt;div class=&quot;note success&quot;&gt;
            &lt;p&gt;对于网络慢的问题，可以使用tcpdump抓包分析慢的问题；而如果就是本机一个程序慢呢？就需要使用strace来分析了&lt;/p&gt;
          &lt;/div&gt;
    
    </summary>
    
      <category term="性能优化" scheme="https://www.wumingx.com/categories/performance/"/>
    
    
      <category term="strace" scheme="https://www.wumingx.com/tags/strace/"/>
    
  </entry>
  
  <entry>
    <title>&#39;zookeeper</title>
    <link href="https://www.wumingx.com/person/zookeeper.html"/>
    <id>https://www.wumingx.com/person/zookeeper.html</id>
    <published>2021-01-05T08:27:58.000Z</published>
    <updated>2021-07-08T02:22:53.184Z</updated>
    
    <content type="html"><![CDATA[<a id="more"></a><div class="note default | primary | success | info | warning | danger">                      </div><span class="label default">默认</span>  <span class="label primary">主要</span>  <span class="label success">成功</span>  <span class="label info">信息</span>  <span class="label warning">警告</span>  <span class="label danger">危险</span> <p>机器重启会</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 已经建立了连接</span></span><br><span class="line">[root@localhost ~]<span class="comment"># ss -tn |grep 192.168.1.99:22</span></span><br><span class="line">ESTAB      0      0      192.168.1.222:47070              192.168.1.99:22                 </span><br><span class="line">[root@localhost ~]<span class="comment"># ss -tn |grep 192.168.1.99:22//192.168.1.99重启后连接断开</span></span><br><span class="line">[root@localhost ~]<span class="comment"># </span></span><br><span class="line">[root@localhost ~]<span class="comment">#</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 抓包可以看到重启时，可以收到FIN包</span></span><br><span class="line">[root@localhost ~]<span class="comment"># tcpdump -r 1.pcap -nn</span></span><br><span class="line">reading from file 1.pcap, link-type EN10MB (Ethernet)</span><br><span class="line">10:15:24.473770 IP 192.168.1.99.22 &gt; 192.168.1.222.47070: Flags [F.], seq 3091611374, ack 1145214609, win 298, length 0</span><br><span class="line">10:15:24.473979 IP 192.168.1.222.47070 &gt; 192.168.1.99.22: Flags [P.], seq 1:61, ack 1, win 88, length 60</span><br><span class="line">10:15:24.474162 IP 192.168.1.222.47070 &gt; 192.168.1.99.22: Flags [F.], seq 61, ack 1, win 88, length 0</span><br><span class="line">10:15:24.478742 IP 192.168.1.99.22 &gt; 192.168.1.222.47070: Flags [R], seq 3091611375, win 0, length 0</span><br><span class="line">10:15:24.483751 IP 192.168.1.99.22 &gt; 192.168.1.222.47070: Flags [R], seq 3091611375, win 0, length 0</span><br></pre></td></tr></table></figure>]]></content>
    
    <summary type="html">
    
      
      
        &lt;a id=&quot;more&quot;&gt;&lt;/a&gt;
&lt;div class=&quot;note default | primary | success | info | warning | danger&quot;&gt;
            
          &lt;/div&gt;
&lt;span class=&quot;label 
      
    
    </summary>
    
      <category term="person" scheme="https://www.wumingx.com/categories/person/"/>
    
    
      <category term="tag" scheme="https://www.wumingx.com/tags/tag/"/>
    
  </entry>
  
  <entry>
    <title>zookeeper &amp; kafka部署和使用</title>
    <link href="https://www.wumingx.com/linux/zk-kafka.html"/>
    <id>https://www.wumingx.com/linux/zk-kafka.html</id>
    <published>2021-01-05T08:27:35.000Z</published>
    <updated>2021-09-04T13:31:48.820Z</updated>
    
    <content type="html"><![CDATA[<h1 id="zookeeper"><a href="#zookeeper" class="headerlink" title="zookeeper"></a>zookeeper</h1><h2 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h2><p>ZooKeeper是一个开源的分布式应用程序协调服务，是Google的Chubby一个开源的实现。ZooKeeper为分布式应用提供一致性服务，提供的功能包括：分布式同步（Distributed Synchronization）、命名服务（Naming Service）、集群维护（Group Maintenance）、分布式锁（Distributed Lock）等，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。</p><p>ZooKeeper本身可以以单机模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。</p><a id="more"></a><p>ZooKeeper主要有领导者（Leader）、跟随者（Follower）和观察者（Observer）三种角色。</p><div class="table-container"><table><thead><tr><th>角色</th><th>说明</th></tr></thead><tbody><tr><td>领导者（Leader）</td><td>为客户端提供读和写的服务，负责投票的发起和决议，更新系统状态。</td></tr><tr><td>跟随者（Follower）</td><td>为客户端提供读服务，如果是写服务则转发给Leader。在选举过程中参与投票。</td></tr><tr><td>观察者（Observer）</td><td>为客户端提供读服务器，如果是写服务则转发给Leader。不参与选举过程中的投票，也不参与“过半写成功”策略。在不影响写性能的情况下提升集群的读性能。此角色于zookeeper3.3系列新增的角色。</td></tr></tbody></table></div><h2 id="部署"><a href="#部署" class="headerlink" title="部署"></a>部署</h2><p>根据官方说明，如果使用java 1.8的版本的话，需要使用u211以上的版本，但实测u161也是可以使用的。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">yum install java-1.8.0-openjdk -y</span><br><span class="line"></span><br><span class="line"><span class="comment"># https://zookeeper.apache.org/releases.html</span></span><br><span class="line">wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz</span><br><span class="line"></span><br><span class="line">tar zxf apache-zookeeper-3.6.2-bin.tar.gz</span><br><span class="line">cp -r apache-zookeeper-3.6.2-bin /usr/<span class="built_in">local</span>/zookeeper-3.6.2</span><br><span class="line"></span><br><span class="line"><span class="built_in">cd</span> /usr/<span class="built_in">local</span>/zookeeper-3.6.2/conf</span><br><span class="line">cp zoo_sample.cfg zoo.cfg</span><br></pre></td></tr></table></figure><p>zoo.cfg的配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">tickTime=2000</span><br><span class="line">initLimit=10</span><br><span class="line">syncLimit=5</span><br><span class="line">dataDir=/usr/local/zookeeper-3.6.2/data</span><br><span class="line">clientPort=2181</span><br><span class="line"></span><br><span class="line">maxClientCnxns=512</span><br><span class="line">autopurge.snapRetainCount=30</span><br><span class="line">autopurge.purgeInterval=2</span><br><span class="line"></span><br><span class="line">server.13=192.168.0.13:2888:3888</span><br><span class="line">server.14=192.168.0.14:2888:3888</span><br><span class="line">server.15=192.168.0.15:2888:3888</span><br></pre></td></tr></table></figure><ul><li><p><strong>tickTime</strong>：心跳基本时间单位，毫秒级，ZK基本上所有的时间都是这个时间的整数倍。</p></li><li><p><strong>initLimit</strong>：tickTime的个数，表示在leader选举结束后，followers与leader同步需要的时间，如果followers比较多或者说leader的数据灰常多时，同步时间相应可能会增加，那么这个值也需要相应增加。当然，这个值也是follower和observer在开始同步leader的数据时的最大等待时间(setSoTimeout)</p></li><li><strong>syncLimit</strong>：tickTime的个数，这时间容易和上面的时间混淆，它也表示follower和observer与leader交互时的最大等待时间，只不过是在与leader同步完毕之后，进入正常请求转发或ping等消息交互时的超时时间</li><li><strong>dataDir</strong>：内存数据库快照存放地址，如果没有指定事务日志存放地址(dataLogDir)，默认也是存放在这个路径下，建议两个地址分开存放到不同的设备上。</li><li><strong>clientPort</strong>：配置ZK监听客户端连接的端口</li><li><strong>dataLogDir</strong>：将事务日志存储在该路径下，比较重要，这个日志存储的设备效率会影响ZK的写吞吐量。</li><li><strong>server.serverid=host:tickpot:electionport</strong><ul><li>server：固定写法</li><li>serverid：每个服务器的指定ID（必须处于1-255之间，必须每一台机器不能重复）</li><li>host：主机名，直接用IP更容易识别</li><li>tickpot：心跳通信端口</li><li>electionport：选举端口</li><li>这里的serverid要写入到<code>/usr/local/zookeeper-3.6.2/data/myid</code>下，注意每台机器都不一样。</li></ul></li></ul><p>由于/usr/local/zookeeper-3.6.2/bin目录下面是zookeeper的运行命令所在位置，可以将他加入到系统变量中去。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@iloqg8n3yb9mje data]<span class="comment"># cat /etc/profile.d/zookeeper.sh </span></span><br><span class="line"><span class="built_in">export</span> ZOOKEEPER_HOME=/usr/<span class="built_in">local</span>/zookeeper-3.6.2/</span><br><span class="line"><span class="built_in">export</span> PATH=<span class="variable">$PATH</span>:<span class="variable">$ZOOKEEPER_HOME</span>/bin</span><br></pre></td></tr></table></figure><h2 id="服务启动"><a href="#服务启动" class="headerlink" title="服务启动"></a>服务启动</h2><p>启动服务，使用<code>zkServer.sh start</code>，但我们可以使用以下systemd来接管。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">$ systemctl cat zookeeper</span><br><span class="line"><span class="comment"># /etc/systemd/system/zookeeper.service</span></span><br><span class="line">[Unit]</span><br><span class="line">Description=Apache Zookeeper server</span><br><span class="line">Documentation=http://zookeeper.apache.org</span><br><span class="line">Requires=network.target remote-fs.target</span><br><span class="line">After=network.target remote-fs.target</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=forking</span><br><span class="line">User=root</span><br><span class="line">Group=root</span><br><span class="line">ExecStart=/usr/<span class="built_in">local</span>/zookeeper-3.6.2/bin/zkServer.sh start /usr/<span class="built_in">local</span>/zookeeper-3.6.2/conf/zoo.cfg</span><br><span class="line">ExecStop=/usr/<span class="built_in">local</span>/zookeeper-3.6.2/bin/zkServer.sh stop /usr/<span class="built_in">local</span>/zookeeper-3.6.2/conf/zoo.cfg</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br></pre></td></tr></table></figure><h2 id="集群状态查看"><a href="#集群状态查看" class="headerlink" title="集群状态查看"></a>集群状态查看</h2><p>使用<code>zkServer.sh status</code>查看集群状态；需要等三台全部启动之后才会显示MODE的状态，如果只启动一台的话，就会显示<code>Error contacting service. It is probably not running.</code>；</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@iabnt3f4lvt0va data]<span class="comment"># zkServer.sh start</span></span><br><span class="line">ZooKeeper JMX enabled by default</span><br><span class="line">Using config: /usr/<span class="built_in">local</span>/zookeeper-3.6.2/bin/../conf/zoo.cfg</span><br><span class="line">Starting zookeeper ... STARTED</span><br><span class="line">[root@iabnt3f4lvt0va data]<span class="comment"># zkServer.sh status</span></span><br><span class="line">ZooKeeper JMX enabled by default</span><br><span class="line">Using config: /usr/<span class="built_in">local</span>/zookeeper-3.6.2/bin/../conf/zoo.cfg</span><br><span class="line">Client port found: 2181. Client address: localhost. Client SSL: <span class="literal">false</span>.</span><br><span class="line">Mode: leader  <span class="comment"># 备显示为follower</span></span><br></pre></td></tr></table></figure><p>如果三台都启动了，但没有显示mode，可以查看<code>/usr/local/zookeeper-3.6.2/logs</code>下面的日志进行排查。</p><p>zookeeper的操作，运行<code>/usr/local/zookeeper-3.6.2/bin/zkCli.sh -server 192.168.0.14</code>：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line">[zk: localhost:2181(CONNECTED) 0] ls /</span><br><span class="line">[zookeeper]</span><br><span class="line">[zk: localhost:2181(CONNECTED) 1] create /cpu intel</span><br><span class="line">Created /cpu</span><br><span class="line">[zk: localhost:2181(CONNECTED) 2] ls /</span><br><span class="line">[cpu, zookeeper]</span><br><span class="line">[zk: localhost:2181(CONNECTED) 3] get /cpu</span><br><span class="line">intel</span><br><span class="line">[zk: localhost:2181(CONNECTED) 10] get -s /cpu</span><br><span class="line">intel</span><br><span class="line">cZxid = 0x200000007</span><br><span class="line">ctime = Mon Jan 04 10:47:26 CST 2021</span><br><span class="line">mZxid = 0x200000007</span><br><span class="line">mtime = Mon Jan 04 10:47:26 CST 2021</span><br><span class="line">pZxid = 0x200000007</span><br><span class="line">cversion = 0</span><br><span class="line">dataVersion = 0</span><br><span class="line">aclVersion = 0</span><br><span class="line">ephemeralOwner = 0x0</span><br><span class="line">dataLength = 5</span><br><span class="line">numChildren = 0</span><br><span class="line">[zk: localhost:2181(CONNECTED) 4] <span class="built_in">set</span> /cpu AMD</span><br><span class="line">[zk: localhost:2181(CONNECTED) 5] get /cpu</span><br><span class="line">AMD</span><br><span class="line">[zk: localhost:2181(CONNECTED) 6] delete</span><br><span class="line">delete      deleteall   </span><br><span class="line">[zk: localhost:2181(CONNECTED) 6] delete /cpu</span><br><span class="line">[zk: localhost:2181(CONNECTED) 7] ls /</span><br><span class="line">[zookeeper]</span><br></pre></td></tr></table></figure><p>zxid：一个事务编号，zookeeper集群内部的所有事务，都有一个全局的唯一的顺序的编号，由两部分组成： 就是一个 64位的长整型 long：</p><ul><li><strong>高32位: 用来标识leader关系是否改变，如  0x2</strong></li><li><strong>低32位： 用来做当前这个leader领导期间的全局的递增的事务编号，如  00000007</strong></li><li><code>get -s</code>可以使用查看zxid的详细数据。</li></ul><div class="table-container"><table><thead><tr><th><strong>状态属性</strong></th><th><strong>说明</strong></th></tr></thead><tbody><tr><td>cZxid</td><td>数据节点创建时的事务ID</td></tr><tr><td>ctime</td><td>数据节点创建时的时间</td></tr><tr><td>mZxid</td><td>数据节点最后一次更新时的事务ID</td></tr><tr><td>mtime</td><td>数据节点最后一次更新时的时间</td></tr><tr><td>pZxid</td><td>数据节点的子节点列表最后一次被修改（是子节点列表变更，而不是子节点内容变更）时的事务ID</td></tr><tr><td>cversion</td><td>子节点的版本号</td></tr><tr><td>dataVersion</td><td>数据节点的版本号</td></tr><tr><td>aclVersion</td><td>数据节点的ACL版本号</td></tr><tr><td>ephemeralOwner</td><td>如果节点是临时节点，则表示创建该节点的会话的SessionID；如果节点是持久节点，则该属性值为0</td></tr><tr><td>dataLength</td><td>数据内容的长度</td></tr><tr><td>numChildren</td><td>数据节点当前的子节点个数</td></tr></tbody></table></div><blockquote><p><a href="https://zookeeper.apache.org/doc/current/zookeeperStarted.html" target="_blank" rel="noopener">https://zookeeper.apache.org/doc/current/zookeeperStarted.html</a></p><p><a href="https://www.cnblogs.com/qingyunzong/p/8619184.html" target="_blank" rel="noopener">Zookeeper学习之路 （二）集群搭建</a></p><p><a href="https://ken.io/note/zookeeper-cluster-deploy-guide" target="_blank" rel="noopener">ZooKeeper集群部署指南</a></p></blockquote><h1 id="kafka"><a href="#kafka" class="headerlink" title="kafka"></a>kafka</h1><p>Kafka是一个开源的分布式消息引擎/消息中间件，同时Kafka也是一个流处理平台。Kakfa支持以发布/订阅的方式在应用间传递消息，同时并基于消息功能添加了Kafka Connect、Kafka Streams以支持连接其他系统的数据等)</p><p>Kafka最核心的最成熟的还是他的消息引擎，所以Kafka大部分应用场景还是用来作为消息队列削峰平谷。另外，Kafka也是目前性能最好的消息中间件。</p><p>在Kafka集群(Cluster)中，一个Kafka节点就是一个Broker，消息由Topic来承载，可以存储在1个或多个Partition中。发布消息的应用为Producer、消费消息的应用为Consumer，多个Consumer可以促成Consumer Group共同消费一个Topic中的消息。</p><div class="table-container"><table><thead><tr><th>概念/对象</th><th>简单说明</th></tr></thead><tbody><tr><td>Broker</td><td>Kafka节点</td></tr><tr><td>Topic</td><td>主题，用来承载消息，可以理解为数据库里面的表</td></tr><tr><td>Partition</td><td>分区，用于主题分片存储，可以理解为多机器存储，比如说设置分区为3，kafka会根据一定的算法动态存储到不同的broker上</td></tr><tr><td>Producer</td><td>生产者，向主题发布消息的应用</td></tr><tr><td>Consumer</td><td>消费者，从主题订阅消息的应用</td></tr><tr><td>Consumer Group</td><td>消费者组，由多个消费者组成</td></tr></tbody></table></div><h2 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h2><h3 id="下载"><a href="#下载" class="headerlink" title="下载"></a>下载</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> /usr/<span class="built_in">local</span>/src</span><br><span class="line"><span class="comment"># http://kafka.apache.org/downloads.html</span></span><br><span class="line">wget https://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.7.0/kafka_2.12-2.7.0.tgz</span><br><span class="line">tar zxf kafka_2.12-2.7.0.tgz</span><br><span class="line">cp -r kafka_2.12-2.7.0 /usr/<span class="built_in">local</span>/kakfa</span><br><span class="line"><span class="built_in">cd</span> /usr/<span class="built_in">local</span>/kafka/config</span><br></pre></td></tr></table></figure><h3 id="修改配置"><a href="#修改配置" class="headerlink" title="修改配置"></a>修改配置</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">$ cat server.properties |grep -v <span class="string">'^#'</span> |grep -v <span class="string">'^$'</span></span><br><span class="line"><span class="comment"># Broker唯一标识，和zookeeper的myid性质一样</span></span><br><span class="line">broker.id=13</span><br><span class="line"><span class="comment"># 监听信息，PLAINTEXT表示明文传输，这个最好是直接配置IP进行通信</span></span><br><span class="line">listeners=PLAINTEXT://192.168.0.13:9092</span><br><span class="line"><span class="comment"># borker进行网络处理的线程数</span></span><br><span class="line">num.network.threads=3</span><br><span class="line"><span class="comment"># borker进行I/O处理的线程数</span></span><br><span class="line">num.io.threads=8</span><br><span class="line"><span class="comment"># 发送缓冲区buffer大小，数据不是一下子就发送的，先回存储到缓冲区了到达一定的大小后在发送，能提高性能</span></span><br><span class="line">socket.send.buffer.bytes=102400</span><br><span class="line"><span class="comment"># kafka接收缓冲区大小，当数据到达一定大小后在序列化到磁盘</span></span><br><span class="line">socket.receive.buffer.bytes=102400</span><br><span class="line"><span class="comment"># 这个参数是向kafka请求消息或者向kafka发送消息的请请求的最大数，这个值不能超过java的堆栈大小</span></span><br><span class="line">socket.request.max.bytes=104857600</span><br><span class="line"><span class="comment"># kafka数据存放地址，可以填写多个。用”,”间隔</span></span><br><span class="line">log.dirs=/usr/<span class="built_in">local</span>/kafka/logs</span><br><span class="line"><span class="comment"># 默认分区数</span></span><br><span class="line">num.partitions=1</span><br><span class="line"><span class="comment"># 每个数据目录用来日志恢复的线程数目</span></span><br><span class="line">num.recovery.threads.per.data.dir=1</span><br><span class="line">offsets.topic.replication.factor=1</span><br><span class="line">transaction.state.log.replication.factor=1</span><br><span class="line">transaction.state.log.min.isr=1</span><br><span class="line"><span class="comment"># 默认消息的最大持久化时间，168小时，7天</span></span><br><span class="line">log.retention.hours=168</span><br><span class="line"><span class="comment"># kafka的消息是以追加的形式落地到文件，当超过这个值的时候，kafka会新起一个文件，默认为1G</span></span><br><span class="line">log.segment.bytes=1073741824</span><br><span class="line"><span class="comment"># 每隔300000毫秒去检查上面配置的log失效时间</span></span><br><span class="line">log.retention.check.interval.ms=300000</span><br><span class="line"><span class="comment"># ZooKeeper服务器地址，多台用”,”间隔</span></span><br><span class="line">zookeeper.connect=192.168.0.13:2181,192.168.0.14:2181,192.168.0.15:2181</span><br><span class="line"><span class="comment"># 设置zookeeper的连接超时时间</span></span><br><span class="line">zookeeper.connection.timeout.ms=18000</span><br><span class="line">group.initial.rebalance.delay.ms=0</span><br></pre></td></tr></table></figure><h3 id="启动服务"><a href="#启动服务" class="headerlink" title="启动服务"></a>启动服务</h3><p>命令运行之前，最好是把软件包配置到其他机器上再来运行。</p><p>运行命令<code>kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties</code>就可以运行命令了。第一次运行时，可能会报错，可以将daemon先去掉，这样就可以输出报错日志了。</p><p>另外可以使用systemctl来管理服务：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 修改环境变量</span></span><br><span class="line">$ cat /etc/profile.d/kafka.sh </span><br><span class="line"><span class="built_in">export</span> KAFKA_HOME=/usr/<span class="built_in">local</span>/kafka</span><br><span class="line"><span class="built_in">export</span> PATH=<span class="variable">$PATH</span>:<span class="variable">$KAFKA_HOME</span>/bin</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">$ systemctl cat kafka</span><br><span class="line"><span class="comment"># /etc/systemd/system/kafka.service</span></span><br><span class="line">[Unit]</span><br><span class="line">Description=Apache Kafka server (broker)</span><br><span class="line">Documentation=http://kafka.apache.org/documentation.html</span><br><span class="line">Requires=network.target remote-fs.target</span><br><span class="line">After=network.target remote-fs.target kafka-zookeeper.service</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=simple</span><br><span class="line">User=root</span><br><span class="line">Group=root</span><br><span class="line"><span class="comment">#Environment=JAVA_HOME=/usr/java/jdk1.8.0_102</span></span><br><span class="line">ExecStart=/usr/<span class="built_in">local</span>/kafka/bin/kafka-server-start.sh /usr/<span class="built_in">local</span>/kafka/config/server.properties</span><br><span class="line">ExecStop=/usr/<span class="built_in">local</span>/kafka/bin/kafka-server-stop.sh</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br></pre></td></tr></table></figure><h3 id="配置性能优化"><a href="#配置性能优化" class="headerlink" title="配置性能优化"></a>配置性能优化</h3><h4 id="消息大小相关参数"><a href="#消息大小相关参数" class="headerlink" title="消息大小相关参数"></a>消息大小相关参数</h4><p>以下参数优化来源于 <a href="https://blog.csdn.net/qq_32907195/article/details/118969743" target="_blank" rel="noopener">彻底搞懂 Kafka 消息大小相关参数设置的规则</a></p><ul><li>max.request.size：producer端发送消息的最大的大小</li><li>batch.size：producer：非常重要的参数，收集到一批消息再发送到 broker，比每条消息都请求一次 broker，性能会有显著的提高，但 batch.size 设置得非常大又会给机器内存带来极大的压力，因此需要在项目中合理地增减 batch.size 值，才能提高 producer 的吞吐量。</li><li><code>message.max.bytes</code>：Broker 端对 Producer 发送过来的消息也有一定的大小限制,默认值为1M</li><li>max.message.bytes：它只针对某个主题生效，可动态配置，可覆盖全局的 message.max.bytes，好处就是可以针对不同主题去设置 Broker 接收消息的大小，而且不用重启 Broker。</li><li>fetch.max.byte：这个参数决定消费者单次从 Broker 获取消息的最大字节数</li><li><p><code>replica.fetch.max.bytes</code>：限制副本拉取分区中消息的大小，尽可能要设置为比message.max.bytes更大，也可以设置为一样大的值</p></li><li><p>socket.request.max.bytes：socket请求的最大数值，为了防止serverOOM，<code>message.max.bytes一定要小于socket.request.max.bytes</code></p></li></ul><blockquote><p>从broker的优化角度，message.max.bytes、replica.fetch.max.bytes、socket.request.max.bytes这三个值必须调大。</p></blockquote><h4 id="内部topic优化"><a href="#内部topic优化" class="headerlink" title="内部topic优化"></a>内部topic优化</h4><p>我们知道老版本的消费位移信息是存储的zookeeper 中的， 但是zookeeper 并不适合频繁的写入查询操作，所以在新版本的中消费位移信息存放在了<code>__consumer_offsets</code>内置topic中，所以这个topic很重要，但默认配置下，一个分区只有一个副本，当该副本所在的broker宕机，consumer_offsets只有一份副本，该分区宕机。使用该分区存储消费分组offset位置的消费者均会收到影响，offset无法提交，从而导致生产者可以发送消息但消费者不可用。所以需要设置该字段的值大于1。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">offsets.topic.replication.factor=3</span><br><span class="line">transaction.state.log.replication.factor=3</span><br><span class="line">transaction.state.log.min.isr=2</span><br><span class="line"></span><br><span class="line"><span class="comment"># kafla在启动时进行日志恢复和在关闭时进行数据刷新的线程数</span></span><br><span class="line">num.recovery.threads.per.data.dir=5</span><br></pre></td></tr></table></figure><h4 id="其他参数"><a href="#其他参数" class="headerlink" title="其他参数"></a>其他参数</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 默认值为False。默认情况下leader不能从非ISR的副本列表里选择；在异常情况下，比如ISR内的副本都不可用了，此时如果该字段设置为False，服务会直接挂掉；</span></span><br><span class="line"><span class="comment"># 如果该字段设为True，即允许从非ISR列表中选择leader，那么服务尽管有可能丢失数据，却依然可以继续使用。所以这个参数必须参考业务特性来决定是否打开。</span></span><br><span class="line">unclean.leader.election.enable=<span class="literal">true</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 默认值是1，当生产者将acks设置为“ all”（或“ -1”）时，min.insync.replicas指定必须确认写入才能使成功写入的最小备份副本数量。</span></span><br><span class="line">min.insync.replicas=2</span><br><span class="line"></span><br><span class="line"><span class="comment"># 默认值为1，默认副本数</span></span><br><span class="line">default.replication.factor=3</span><br><span class="line"></span><br><span class="line"><span class="comment"># 是否自动平衡broker之间的分配策略</span></span><br><span class="line">auto.leader.rebalance.enable=<span class="literal">true</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># leader的不平衡比例，若是超过这个数值，会对分区进行重新的平衡</span></span><br><span class="line">leader.imbalance.per.broker.percentage=10</span><br><span class="line"></span><br><span class="line"><span class="comment"># leader中进行复制的线程数，增大这个数值会增加relipca的IO</span></span><br><span class="line">num.replica.fetchers=3</span><br><span class="line"></span><br><span class="line"><span class="comment"># 是否删除topic</span></span><br><span class="line">delete.topic.enable=<span class="literal">true</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 默认主题的分区数</span></span><br><span class="line">num.partitions=9</span><br></pre></td></tr></table></figure><blockquote><p><a href="https://www.jdon.com/53953" target="_blank" rel="noopener">https://www.jdon.com/53953</a></p></blockquote><h2 id="相关操作"><a href="#相关操作" class="headerlink" title="相关操作"></a>相关操作</h2><h3 id="创建Topic"><a href="#创建Topic" class="headerlink" title="创建Topic"></a>创建Topic</h3><p>随便找一台机器创建测试Tpoic：test，这里我们指定了3个副本、1个分区:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@4n1eq6wnfvdwvj logs]<span class="comment"># kafka-topics.sh --create --bootstrap-server 192.168.0.14:9092 --replication-factor 3 --partitions 1 --topic test</span></span><br><span class="line">Created topic <span class="built_in">test</span>.</span><br><span class="line"><span class="comment"># 三台机器上都有这个tpoic</span></span><br><span class="line">[root@4n1eq6wnfvdwvj logs]<span class="comment"># kafka-topics.sh --list --bootstrap-server 192.168.0.14:9092 </span></span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">[root@4n1eq6wnfvdwvj logs]<span class="comment"># kafka-topics.sh --list --bootstrap-server 192.168.0.15:9092 </span></span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">[root@4n1eq6wnfvdwvj logs]<span class="comment"># kafka-topics.sh --list --bootstrap-server 192.168.0.13:9092 </span></span><br><span class="line"><span class="built_in">test</span></span><br></pre></td></tr></table></figure><h3 id="生产数据"><a href="#生产数据" class="headerlink" title="生产数据"></a>生产数据</h3><p>运行命令之后，输入内容就是一条消息：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@4n1eq6wnfvdwvj logs]<span class="comment"># kafka-console-producer.sh --broker-list 192.168.0.13:9092 --topic test</span></span><br><span class="line">&gt;<span class="built_in">test</span> by wumingx  </span><br><span class="line">&gt;date：20210114</span><br><span class="line">&gt;</span><br></pre></td></tr></table></figure><h3 id="消费数据"><a href="#消费数据" class="headerlink" title="消费数据"></a>消费数据</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 从头开始消费</span></span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># kafka-console-consumer.sh --bootstrap-server 192.168.0.14:9092 --topic test --from-beginning</span></span><br><span class="line"><span class="built_in">test</span> by wumingx</span><br><span class="line">date：20210114</span><br><span class="line">^CProcessed a total of 2 messages</span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># kafka-console-consumer.sh --bootstrap-server 192.168.0.15:9092 --topic test --from-beginning</span></span><br><span class="line"><span class="built_in">test</span> by wumingx</span><br><span class="line">date：20210114</span><br><span class="line">^CProcessed a total of 2 messages</span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># </span></span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># </span></span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># kafka-console-consumer.sh --bootstrap-server 192.168.0.13:9092 --topic test --from-beginning</span></span><br><span class="line"><span class="built_in">test</span> by wumingx</span><br><span class="line">date：20210114</span><br><span class="line">^CProcessed a total of 2 messages</span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># </span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 可以对consumer进行分组，那么只允许组内的一个消费者进行消费，其他消费者则不行</span></span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># kafka-console-consumer.sh --bootstrap-server 192.168.0.13:9092 --topic test --from-beginning --group testgroup</span></span><br><span class="line"><span class="built_in">test</span> by wumingx</span><br><span class="line">date：20210114</span><br><span class="line">^CProcessed a total of 2 messages</span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># </span></span><br><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># kafka-console-consumer.sh --bootstrap-server 192.168.0.14:9092 --topic test --from-beginning --group testgroup</span></span><br><span class="line">^CProcessed a total of 0 messages</span><br></pre></td></tr></table></figure><h3 id="查看leader"><a href="#查看leader" class="headerlink" title="查看leader"></a>查看leader</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建一个名为cpu的topic，分为4个分区，3副本</span></span><br><span class="line">[root@iloqg8n3yb9mje logs]<span class="comment"># kafka-topics.sh --create --bootstrap-server 192.168.0.13:9092 --replication-factor 3 --partitions 4 --topic cpu</span></span><br><span class="line"><span class="comment"># 查看topic列表，只有名字</span></span><br><span class="line">[root@iloqg8n3yb9mje logs]<span class="comment"># kafka-topics.sh --list --bootstrap-server 192.168.0.14:9092</span></span><br><span class="line">__consumer_offsets</span><br><span class="line">cpu</span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">[root@iloqg8n3yb9mje logs]<span class="comment"># kafka-topics.sh --list --zookeeper 192.168.0.14:2181</span></span><br><span class="line">__consumer_offsets</span><br><span class="line">cpu</span><br><span class="line"><span class="built_in">test</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#查看cpu这个topic的属性，可以看到分区0的leader是在14上，分区1是在15上。</span></span><br><span class="line">[root@iloqg8n3yb9mje logs]<span class="comment"># kafka-topics.sh --describe --zookeeper 192.168.0.14:2181 --topic cpu</span></span><br><span class="line">Topic: cpuPartitionCount: 4ReplicationFactor: 3Configs: </span><br><span class="line">Topic: cpuPartition: 0Leader: 14Replicas: 14,15,13Isr: 14,15,13</span><br><span class="line">Topic: cpuPartition: 1Leader: 15Replicas: 15,13,14Isr: 15,13,14</span><br><span class="line">Topic: cpuPartition: 2Leader: 13Replicas: 13,14,15Isr: 13,14,15</span><br><span class="line">Topic: cpuPartition: 3Leader: 14Replicas: 14,13,15Isr: 14,13,15</span><br></pre></td></tr></table></figure><p>Leader会跟踪与其保持同步的Replica列表，该列表称为ISR（即in-sync Replica）。</p><h3 id="kafka-in-zookeeper"><a href="#kafka-in-zookeeper" class="headerlink" title="kafka in zookeeper"></a>kafka in zookeeper</h3><p><img src="/assets/1228818-20180508101652574-1613892176.png" alt></p><h4 id="查看id和topics"><a href="#查看id和topics" class="headerlink" title="查看id和topics"></a>查看id和topics</h4><p>/brokers/topics/[topic] :存储某个topic的partitions所有分配信息</p><p>运行<code>zkCli.sh</code></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[zk: localhost:2181(CONNECTED) 0] ls /brokers/ids </span><br><span class="line">[13, 14, 15]<span class="comment"># 表示三台机器</span></span><br><span class="line">[zk: localhost:2181(CONNECTED) 1] ls /brokers/topics </span><br><span class="line">[__consumer_offsets, cpu, <span class="built_in">test</span>]<span class="comment"># 有cpu test这2个topics</span></span><br><span class="line">[zk: localhost:2181(CONNECTED) 2] get /brokers/topics/cpu </span><br><span class="line">&#123;<span class="string">"version"</span>:2,<span class="string">"partitions"</span>:&#123;<span class="string">"2"</span>:[13,14,15],<span class="string">"1"</span>:[15,13,14],<span class="string">"0"</span>:[14,15,13],<span class="string">"3"</span>:[14,13,15]&#125;,<span class="string">"adding_replicas"</span>:&#123;&#125;,<span class="string">"removing_replicas"</span>:&#123;&#125;&#125;</span><br><span class="line">[zk: localhost:2181(CONNECTED) 7] get /brokers/ids/13 </span><br><span class="line">&#123;<span class="string">"listener_security_protocol_map"</span>:&#123;<span class="string">"PLAINTEXT"</span>:<span class="string">"PLAINTEXT"</span>&#125;,<span class="string">"endpoints"</span>:[<span class="string">"PLAINTEXT://192.168.0.13:9092"</span>],<span class="string">"jmx_port"</span>:-1,<span class="string">"features"</span>:&#123;&#125;,<span class="string">"host"</span>:<span class="string">"192.168.0.13"</span>,<span class="string">"timestamp"</span>:<span class="string">"1609747952920"</span>,<span class="string">"port"</span>:9092,<span class="string">"version"</span>:5&#125;</span><br></pre></td></tr></table></figure><h4 id="partition状态信息"><a href="#partition状态信息" class="headerlink" title="partition状态信息"></a>partition状态信息</h4><p>/brokers/topics/[topic]/partitions/[0…N]  其中[0..N]表示partition索引号</p><p>/brokers/topics/[topic]/partitions/[partitionId]/state</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[zk: localhost:2181(CONNECTED) 5] ls /brokers/topics/cpu/partitions </span><br><span class="line">[0, 1, 2, 3]</span><br><span class="line">[zk: localhost:2181(CONNECTED) 6] get /brokers/topics/cpu/partitions/0/state </span><br><span class="line">&#123;<span class="string">"controller_epoch"</span>:6,<span class="string">"leader"</span>:14,<span class="string">"version"</span>:1,<span class="string">"leader_epoch"</span>:0,<span class="string">"isr"</span>:[14,15,13]&#125;</span><br></pre></td></tr></table></figure><p>最后一条表示partitions 0这个分区的leader在14这台机器上面；controller_epoch表示kafka集群中的中央控制器选举次数；leader_epoch表示 该partition leader选举次数。</p><h4 id="controller信息"><a href="#controller信息" class="headerlink" title="controller信息"></a>controller信息</h4><ul><li>controller_epoch：此值为一个数字,kafka集群中第一个broker第一次启动时为1，以后只要集群中center controller中央控制器所在broker变更或挂掉，就会重新选举新的center controller，每次center controller变更controller_epoch值就会 + 1; </li><li>controller：存储center controller中央控制器所在kafka broker的信息</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[zk: localhost:2181(CONNECTED) 8] get /controller_epoch </span><br><span class="line">6</span><br><span class="line">[zk: localhost:2181(CONNECTED) 9] get /controller</span><br><span class="line">&#123;<span class="string">"version"</span>:1,<span class="string">"brokerid"</span>:13,<span class="string">"timestamp"</span>:<span class="string">"1609747992848"</span>&#125;</span><br></pre></td></tr></table></figure><blockquote><p><a href="https://ken.io/note/kafka-cluster-deploy-guide" target="_blank" rel="noopener">Kafka集群部署指南</a></p><p><a href="https://www.cnblogs.com/qingyunzong/p/9007107.html" target="_blank" rel="noopener">Kafka学习之路 （五）Kafka在zookeeper中的存储</a></p><p><a href="https://www.cnblogs.com/binyue/p/10308754.html" target="_blank" rel="noopener">为什么Kafka速度那么快</a></p><p><a href="https://blog.csdn.net/lizhitao/article/details/39499283" target="_blank" rel="noopener">apache kafka技术分享系列(目录索引)</a></p><p><a href="https://knner.wang/2019/12/05/install-and-config-zookeeper-and-kafka-cluster.html" target="_blank" rel="noopener">安装配置Zookeeper和Kafka集群</a></p><p><a href="https://yhyr.github.io/2019/01/01/Kafka%E5%AE%9E%E8%B7%B5%E4%B9%8B%E5%B8%B8%E7%94%A8%E5%91%BD%E4%BB%A4/" target="_blank" rel="noopener">Kafka实践之常用命令</a></p><p><a href="https://segmentfault.com/a/1190000021824942" target="_blank" rel="noopener">Kafka文件存储机制</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;zookeeper&quot;&gt;&lt;a href=&quot;#zookeeper&quot; class=&quot;headerlink&quot; title=&quot;zookeeper&quot;&gt;&lt;/a&gt;zookeeper&lt;/h1&gt;&lt;h2 id=&quot;简介&quot;&gt;&lt;a href=&quot;#简介&quot; class=&quot;headerlink&quot; title=&quot;简介&quot;&gt;&lt;/a&gt;简介&lt;/h2&gt;&lt;p&gt;ZooKeeper是一个开源的分布式应用程序协调服务，是Google的Chubby一个开源的实现。ZooKeeper为分布式应用提供一致性服务，提供的功能包括：分布式同步（Distributed Synchronization）、命名服务（Naming Service）、集群维护（Group Maintenance）、分布式锁（Distributed Lock）等，简化分布式应用协调及其管理的难度，提供高性能的分布式服务。&lt;/p&gt;
&lt;p&gt;ZooKeeper本身可以以单机模式安装运行，不过它的长处在于通过分布式ZooKeeper集群（一个Leader，多个Follower），基于一定的策略来保证ZooKeeper集群的稳定性和可用性，从而实现分布式应用的可靠性。&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="kafka" scheme="https://www.wumingx.com/tags/kafka/"/>
    
      <category term="zookeeper" scheme="https://www.wumingx.com/tags/zookeeper/"/>
    
  </entry>
  
  <entry>
    <title>keepalived几个配置问题讲解</title>
    <link href="https://www.wumingx.com/linux/keepalived.html"/>
    <id>https://www.wumingx.com/linux/keepalived.html</id>
    <published>2020-12-24T09:52:34.000Z</published>
    <updated>2020-12-27T15:17:12.482Z</updated>
    
    <content type="html"><![CDATA[<h1 id="配置"><a href="#配置" class="headerlink" title="配置"></a>配置</h1><p>使用keepalived做主备，其中一台设置为master，一台设置为backup。当master出现异常后，backup自动切换为master。当backup成为master后，master恢复正常后会再次抢占成为master，<strong>导致不必要的主备切换。</strong><code>因此可以将两台keepalived初始状态均配置为backup，设置不同的优先级，优先级高的设置nopreempt解决异常恢复后再次抢占的问题。</code></p><p>有如下配置表示意思也比较简单，VIP为192.168.0.18，2台机器的初始state都是BACKUP，machineA的优先级是15，machineB的优先级是13，配置了<code>/root/1.sh</code>这个来检测服务是否正常。</p><p>machineA机器配置：</p><a id="more"></a><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># cat /etc/keepalived/keepalived.conf </span></span><br><span class="line">! Configuration File <span class="keyword">for</span> keepalived</span><br><span class="line">global_defs &#123;</span><br><span class="line">   router_id  iloqg8n3yb9mje</span><br><span class="line">   script_user root</span><br><span class="line">   enable_script_security</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">vrrp_script check_mysql &#123;</span><br><span class="line">    script <span class="string">"/root/1.sh"</span></span><br><span class="line">    interval 10</span><br><span class="line">    timeout 5</span><br><span class="line">    weight 5</span><br><span class="line">    fall 3</span><br><span class="line">&#125;</span><br><span class="line">vrrp_instance VI_1 &#123;</span><br><span class="line">    state BACKUP</span><br><span class="line">    nopreempt</span><br><span class="line">    interface eth0</span><br><span class="line">    virtual_router_id 18</span><br><span class="line">    priority 15</span><br><span class="line">    advert_int 1<span class="comment">#检查间隔，默认1秒 VRRP心跳包的发送周期，单位为s 组播信息发送间隔，两个节点设置必须一样</span></span><br><span class="line">    authentication &#123;</span><br><span class="line">        auth_type PASS</span><br><span class="line">        auth_pass 1002</span><br><span class="line">    &#125;</span><br><span class="line">    track_script &#123;</span><br><span class="line">        check_mysql</span><br><span class="line">    &#125;</span><br><span class="line">    virtual_ipaddress &#123;</span><br><span class="line">       192.168.0.18 dev eth0 label eth0:0</span><br><span class="line">    &#125;</span><br><span class="line">    notify_master <span class="string">"/root/2.sh master"</span></span><br><span class="line">    notify_backup <span class="string">"/root/2.sh backup"</span></span><br><span class="line">    notify_fault <span class="string">"/root/2.sh fault"</span></span><br><span class="line">    notify <span class="string">"/root/2.sh notify..."</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>machineB配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">global_defs &#123;</span><br><span class="line">   router_id 4n1eq6wnfvdwvj</span><br><span class="line">   script_user root</span><br><span class="line">   enable_script_security</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">vrrp_script check_mysql &#123;</span><br><span class="line">    script &quot;/root/1.sh&quot;</span><br><span class="line">    interval 10</span><br><span class="line">    timeout 5</span><br><span class="line">    weight 5</span><br><span class="line">    fall 3</span><br><span class="line">&#125;</span><br><span class="line">vrrp_instance VI_1 &#123;</span><br><span class="line">    state BACKUP</span><br><span class="line">    nopreempt</span><br><span class="line">    interface eth0</span><br><span class="line">    virtual_router_id 18</span><br><span class="line">    priority 13</span><br><span class="line">    advert_int 1</span><br><span class="line">    authentication &#123;</span><br><span class="line">        auth_type PASS</span><br><span class="line">        auth_pass 1002</span><br><span class="line">    &#125;</span><br><span class="line">    track_script &#123;</span><br><span class="line">        check_mysql</span><br><span class="line">    &#125;</span><br><span class="line">    virtual_ipaddress &#123;</span><br><span class="line">       192.168.0.18 dev eth0 label eth0:0</span><br><span class="line">    &#125;</span><br><span class="line">    notify_master &quot;/root/2.sh master&quot;</span><br><span class="line">    notify_backup &quot;/root/2.sh backup&quot;</span><br><span class="line">    notify_fault &quot;/root/2.sh fault&quot;</span><br><span class="line">    notify &quot;/root/3.sh&quot;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>/root/1.sh配置如下：这个脚本用来检测服务是否正常，这个为了测试，设置当 <code>/etc/keepalived/down</code>这个文件存在返回值为0，反之为1</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line">/bin/<span class="built_in">test</span> -f /etc/keepalived/down &amp;&amp; <span class="built_in">exit</span> 0 || <span class="built_in">exit</span> 1</span><br></pre></td></tr></table></figure><h1 id="vrrp-script"><a href="#vrrp-script" class="headerlink" title="vrrp_script"></a>vrrp_script</h1><h2 id="配置-1"><a href="#配置-1" class="headerlink" title="配置"></a>配置</h2><p>vrrp_script是指通过脚本来检测服务是否正常，通过 <code>man keepalived.conf</code> 查看其参数的意思。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">vrrp_script &lt;SCRIPT_NAME&gt; &#123;</span><br><span class="line">           script &lt;STRING&gt;|&lt;QUOTED-STRING&gt; <span class="comment"># path of the script to execute，需要运行的脚本，返回值为0表示正常，</span></span><br><span class="line">           interval &lt;INTEGER&gt;  <span class="comment"># seconds between script invocations, default 1 second ，脚本运行时间，即隔多少秒去检测</span></span><br><span class="line">           timeout &lt;INTEGER&gt;   <span class="comment"># seconds after which script is considered to have failed，脚本运行的超时时间。</span></span><br><span class="line">           weight &lt;INTEGER:-254..254&gt;  <span class="comment"># adjust priority by this weight, default 0</span></span><br><span class="line">           rise &lt;INTEGER&gt;              <span class="comment"># required number of successes for OK transition，配置几次检测成功才认为服务正常</span></span><br><span class="line">           fall &lt;INTEGER&gt;              <span class="comment"># required number of successes for KO transition，配置几次检测失败才认为服务异常</span></span><br><span class="line">           user USERNAME [GROUPNAME]   <span class="comment"># user/group names to run script under</span></span><br><span class="line">                                       <span class="comment">#   group default to group of user</span></span><br><span class="line">           init_fail                   <span class="comment"># assume script initially is in failed state，配置初始时失败状态</span></span><br><span class="line">        &#125;</span><br></pre></td></tr></table></figure><p>以上文的配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">vrrp_script check_mysql &#123;</span><br><span class="line">    script &quot;/root/1.sh&quot;</span><br><span class="line">    interval 10</span><br><span class="line">    timeout 5</span><br><span class="line">    weight 5</span><br><span class="line">    fall 3</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>我们把/etc/keepalived/down目录删除之后，machineA，<code>17:45:06</code>有第一次检测异常，后面再过了20秒之后，直接提示了failed，同时优先级从20减为了15。说明需要达到fall的次数之后才会切优先级。以下是从message日志里面看到的：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">Dec 24 17:45:06 iloqg8n3yb9mje Keepalived_vrrp[109141]: Script `check_mysql` now returning 1</span><br><span class="line">Dec 24 17:45:26 iloqg8n3yb9mje Keepalived_vrrp[109141]: VRRP_Script(check_mysql) failed (exited with status 1)</span><br><span class="line">Dec 24 17:45:26 iloqg8n3yb9mje Keepalived_vrrp[109141]: (VI_1) Changing effective priority from 20 to 15</span><br></pre></td></tr></table></figure><p>machineB，17:44:45检测到正常之后，就直接调整优先级了，说明rise的默认值为1。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">Dec 24 17:44:15 4n1eq6wnfvdwvj Keepalived_vrrp[51077]: /root/1.sh exited with status 1</span><br><span class="line">Dec 24 17:44:25 4n1eq6wnfvdwvj Keepalived_vrrp[51077]: /root/1.sh exited with status 1</span><br><span class="line">Dec 24 17:44:35 4n1eq6wnfvdwvj Keepalived_vrrp[51077]: /root/1.sh exited with status 1</span><br><span class="line">Dec 24 17:44:45 4n1eq6wnfvdwvj Keepalived_vrrp[51077]: VRRP_Script(check_mysql) succeeded</span><br><span class="line">Dec 24 17:44:46 4n1eq6wnfvdwvj Keepalived_vrrp[51077]: VRRP_Instance(VI_1) Changing effective priority from 13 to 18</span><br></pre></td></tr></table></figure><p>日志显示优先级有做了切换，但是其他事情都没有做，VIP未没有正常切换。这是为什么呢？</p><h2 id="原因分析"><a href="#原因分析" class="headerlink" title="原因分析"></a>原因分析</h2><p>参考 <a href="https://www.cnblogs.com/arjenlee/p/9258188.html" target="_blank" rel="noopener">keepalived之vrrp_script详解</a> 的说法：</p><p><strong>vrrp_script 里的script返回值为0时认为检测成功，其它值都会当成检测失败；</strong></p><ol><li><strong>weight 为正时</strong>，<strong>脚本检测成功时此weight会加到priority上</strong>，检测失败时不加；<ol><li>主失败:<ol><li>主 priority &lt; 从 priority + weight 时会切换。</li></ol></li><li>主成功：<ol><li>主 priority + weight &gt; 从 priority + weight 时，主依然为主</li></ol></li></ol></li><li><strong>weight 为负时</strong>，脚本检测成功时此weight不影响priority，<strong>检测失败时priority – abs(weight)</strong><ol><li>主失败:<ol><li>主 priority – abs(weight) &lt; 从priority 时会切换主从</li></ol></li><li>主成功:<ol><li>主 priority &gt; 从priority 主依然为主</li></ol></li></ol></li></ol><p>实测并不是这个结论，比较怀疑是版号不一致导致出现的结论不一样，但不管怎么说，VIP并未发生切换，所以跟想像中的不一样。</p><p>突发奇想，如果在vrrp_script不配置weight值，会怎么样呢？以下都是在machineA上面显示的日志：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 当脚本check_mysql检测失败的时候，VI_1这个实例就进入了FAULT状态</span></span><br><span class="line">Dec 27 10:42:08 iloqg8n3yb9mje Keepalived_vrrp[120464]: Script `check_mysql` now returning 1</span><br><span class="line">Dec 27 10:42:28 iloqg8n3yb9mje Keepalived_vrrp[120464]: VRRP_Script(check_mysql) failed (exited with status 1)</span><br><span class="line">Dec 27 10:42:28 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Entering FAULT STATE</span><br><span class="line"></span><br><span class="line"><span class="comment"># 当脚本check_mysql恢复正常时，由于配置了nopreempt，VI_1这个实例就进入了BACKUP状态，注意machineA的优先级更高</span></span><br><span class="line">Dec 27 10:47:28 iloqg8n3yb9mje Keepalived_vrrp[120464]: Script `check_mysql` now returning 0</span><br><span class="line">Dec 27 10:47:28 iloqg8n3yb9mje Keepalived_vrrp[120464]: VRRP_Script(check_mysql) succeeded</span><br><span class="line">Dec 27 10:47:28 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Entering BACKUP STATE</span><br><span class="line"></span><br><span class="line"><span class="comment"># machineB失败时，machineA就主动进入了MASTER状态</span></span><br><span class="line">Dec 27 10:48:40 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Backup received priority 0 advertisement</span><br><span class="line">Dec 27 10:48:41 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Receive advertisement timeout</span><br><span class="line">Dec 27 10:48:41 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Entering MASTER STATE</span><br><span class="line">Dec 27 10:48:41 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) setting VIPs.</span><br><span class="line">Dec 27 10:48:41 iloqg8n3yb9mje Keepalived_vrrp[120464]: Sending gratuitous ARP on eth0 <span class="keyword">for</span> 192.168.0.18</span><br><span class="line">Dec 27 10:48:41 iloqg8n3yb9mje Keepalived_vrrp[120464]: (VI_1) Sending/queueing gratuitous ARPs on eth0 <span class="keyword">for</span> 192.168.0.18</span><br></pre></td></tr></table></figure><div class="note success">            <p>由此，可以说明 <code>vrrp_script可以不配置weight值</code>，并且也不需要配置这个值，以避免意外情况发生。</p>          </div><p>另外，如果有遇到如下报错：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">Dec 24 17:41:50 iloqg8n3yb9mje Keepalived_vrrp[108697]: WARNING - default user &apos;keepalived_script&apos; for script execution does not exist - please create.</span><br><span class="line">Dec 24 17:41:50 iloqg8n3yb9mje Keepalived_vrrp[108697]: SECURITY VIOLATION - scripts are being executed but script_security not enabled.</span><br></pre></td></tr></table></figure><p>应该不会影响，但是可以在global配置项里面加上之后就不会有这个提示了。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">script_user root</span><br><span class="line">enable_script_security</span><br></pre></td></tr></table></figure><p>那么直接在vrrp_script下面写成 <code>script &quot;test -f /etc/keepalived/down &amp;&amp; exit 0 || exit 1&quot;</code>是否可以呢？经测试是有问题的。</p><h1 id="notify"><a href="#notify" class="headerlink" title="notify"></a>notify</h1><p><strong>notify的用法：</strong></p><ul><li>notify_master:当当前节点成为master时，通知脚本执行任务(一般用于启动某服务，比如nginx,haproxy等</li><li>notify_backup:当当前节点成为backup时，通知脚本执行任务(一般用于关闭某服务，比如nginx,haproxy等)</li><li>notify_fault：当当前节点出现故障，执行的任务; </li><li>notify<code>表示只要状态切换都会调用的脚本，并且该脚本是在以上三个脚本执行之后再调用的</code></li></ul><p>根据文档所写，notify会自动传以下参数：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$1 = &quot;GROUP&quot;|&quot;INSTANCE&quot;</span><br><span class="line">$2 = name of the group or instance</span><br><span class="line">$3 = target state of transition (&quot;MASTER&quot;|&quot;BACKUP&quot;|&quot;FAULT&quot;)</span><br><span class="line">$4 = priority value</span><br></pre></td></tr></table></figure><p>所以要使用notify时，不需要接参数，跟其他的三个是有所区别的。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">notify_master &quot;/root/2.sh master&quot;</span><br><span class="line">notify_backup &quot;/root/2.sh backup&quot;</span><br><span class="line">notify_fault &quot;/root/2.sh fault&quot;</span><br><span class="line">notify &quot;/root/3.sh&quot;</span><br></pre></td></tr></table></figure><p>脚本内容很简单，只是打印日志出来而出，如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[root@4n1eq6wnfvdwvj ~]<span class="comment"># cat 2.sh </span></span><br><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line"></span><br><span class="line"><span class="built_in">echo</span> <span class="string">"`date +"</span>%F %T<span class="string">"` <span class="variable">$1</span>"</span> &gt;&gt;/tmp/fdm.txt</span><br><span class="line"></span><br><span class="line">[root@4n1eq6wnfvdwvj ~]<span class="comment"># cat 3.sh </span></span><br><span class="line"><span class="meta">#!/bin/bash</span></span><br><span class="line"></span><br><span class="line">TYPE=<span class="variable">$1</span></span><br><span class="line">NAME=<span class="variable">$2</span></span><br><span class="line">STATE=<span class="variable">$3</span></span><br><span class="line"><span class="keyword">case</span> <span class="variable">$STATE</span> <span class="keyword">in</span></span><br><span class="line">        <span class="string">"MASTER"</span>) <span class="built_in">echo</span> <span class="string">"`date +"</span>%F %T<span class="string">"` notify <span class="variable">$1</span> <span class="variable">$2</span> MASTER..."</span> &gt;&gt;/tmp/fdm.txt</span><br><span class="line">                  ;;</span><br><span class="line">        <span class="string">"BACKUP"</span>) <span class="built_in">echo</span> <span class="string">"`date +"</span>%F %T<span class="string">"` notify <span class="variable">$1</span> <span class="variable">$2</span> BACKUP..."</span> &gt;&gt;/tmp/fdm.txt</span><br><span class="line">                  ;;</span><br><span class="line">        <span class="string">"FAULT"</span>)  <span class="built_in">echo</span> <span class="string">"`date +"</span>%F %T<span class="string">"` notify <span class="variable">$1</span> <span class="variable">$2</span> FAULT..."</span> &gt;&gt;/tmp/fdm.txt</span><br><span class="line">                  <span class="built_in">exit</span> 0</span><br><span class="line">                  ;;</span><br><span class="line">        *)        <span class="built_in">echo</span> <span class="string">"`date +"</span>%F %T<span class="string">"` NO TYPE:<span class="variable">$1</span> <span class="variable">$2</span>"</span> &gt;&gt;/tmp/fdm.txt</span><br><span class="line">                  <span class="built_in">exit</span> 1</span><br><span class="line">                  ;;</span><br><span class="line"><span class="keyword">esac</span></span><br></pre></td></tr></table></figure><p>输出的日志如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">2020-12-27 22:31:12 backup</span><br><span class="line">2020-12-27 22:31:12 notify INSTANCE VI_1 BACKUP...</span><br><span class="line">2020-12-27 22:31:12 fault</span><br><span class="line">2020-12-27 22:31:12 notify INSTANCE VI_1 FAULT...</span><br></pre></td></tr></table></figure><p>可以看到，notify的通知在notify_backup的后面。</p><h1 id="脑裂问题"><a href="#脑裂问题" class="headerlink" title="脑裂问题"></a>脑裂问题</h1><p>上文所述的都是业务服务异常了，导致的切换。那主备2台机器不通的情况下，keepalived会做什么操作呢？</p><p><strong>VRRP控制报文只有一种：</strong>VRRP通告(advertisement)，使用通过<code>advert_int 1</code>这个参数来发送通告包的时延，默认是1秒发一次通告包。使用IP多播数据包进行封装，组地址为224.0.0.18，发布范围只限于同一局域网内。这保证了VRID在不同网络中可以重复使用。为了减少网络带宽消耗只有主控路由器才可以周期性的发送VRRP通告报文。<strong>备份路由器在连续三个通告间隔内收不到VRRP</strong>或收到优先级为0的通告后启动新的一轮VRRP选举。</p><p>一般情况下，只有主服务器会发VRRP的通告。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[root@4n1eq6wnfvdwvj ~]<span class="comment"># tcpdump -i any -nns0 vrrp </span></span><br><span class="line">tcpdump: verbose output suppressed, use -v or -vv <span class="keyword">for</span> full protocol decode</span><br><span class="line">listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes</span><br><span class="line">23:00:22.574251 IP 192.168.0.15 &gt; 224.0.0.18: VRRPv2, Advertisement, vrid 18, prio 15, authtype simple, intvl 1s, length 20</span><br><span class="line">23:00:23.574399 IP 192.168.0.15 &gt; 224.0.0.18: VRRPv2, Advertisement, vrid 18, prio 15, authtype simple, intvl 1s, length 20</span><br><span class="line">23:00:24.574420 IP 192.168.0.15 &gt; 224.0.0.18: VRRPv2, Advertisement, vrid 18, prio 15, authtype simple, intvl 1s, length 20</span><br><span class="line">23:00:25.574504 IP 192.168.0.15 &gt; 224.0.0.18: VRRPv2, Advertisement, vrid 18, prio 15, authtype simple, intvl 1s, length 20</span><br><span class="line">23:00:26.574580 IP 192.168.0.15 &gt; 224.0.0.18: VRRPv2, Advertisement, vrid 18, prio 15, authtype simple, intvl 1s, length 20</span><br></pre></td></tr></table></figure><p>如果在主服务器上设置iptables规则，<code>date +&quot;%F %T&quot;;iptables -I OUTPUT -p vrrp -j DROP</code>将vrrp协议发出的包禁掉，命令运行的时间为 <code>2020-12-27 22:53:50</code>，那么观察下备服务器的进入MASTER的时间：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">Dec 27 22:53:54 iloqg8n3yb9mje Keepalived_vrrp[123054]: (VI_1) Receive advertisement timeout</span><br><span class="line">Dec 27 22:53:54 iloqg8n3yb9mje Keepalived_vrrp[123054]: (VI_1) Entering MASTER STATE</span><br><span class="line">Dec 27 22:53:54 iloqg8n3yb9mje Keepalived_vrrp[123054]: (VI_1) setting VIPs.</span><br></pre></td></tr></table></figure><p>从上可以看出，vrrp的通告包超时了，节点进入了MASTER状态，那VIP生效的时间会延迟一秒：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[root@iloqg8n3yb9mje ~]<span class="comment"># for i in `seq 1 100`;do ip -4 -o addr |grep 192.168.0.18 -q &amp;&amp; echo "`date +"%F %T"` have 192.168.0.18" || echo `date +"%F %T"` no~~~;sleep 1;done</span></span><br><span class="line">2020-12-27 22:53:50 no~~~</span><br><span class="line">2020-12-27 22:53:51 no~~~</span><br><span class="line">2020-12-27 22:53:52 no~~~</span><br><span class="line">2020-12-27 22:53:53 no~~~</span><br><span class="line">2020-12-27 22:53:54 no~~~</span><br><span class="line">2020-12-27 22:53:55 have 192.168.0.18</span><br><span class="line">2020-12-27 22:53:56 have 192.168.0.18</span><br></pre></td></tr></table></figure><p>所以一般脑裂问题的排查思路有：</p><ul><li>virtual_router_id必须一样</li><li>防火墙将vrrp广播包给过滤掉了</li><li>机器负载异常，导致机器无法正常发送、或者收到vrrp包之后没有足够的时间进行CPU的处理，这样建议可以尝试增加advert_int时间</li><li>网卡异常等</li></ul><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p><a href="http://arganzheng.life/keepalived-in-action.html" target="_blank" rel="noopener">keepalived实战</a></p><p><a href="https://www.cnblogs.com/kevingrace/p/6248941.html" target="_blank" rel="noopener">Keepalived基础知识-运维小结</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;配置&quot;&gt;&lt;a href=&quot;#配置&quot; class=&quot;headerlink&quot; title=&quot;配置&quot;&gt;&lt;/a&gt;配置&lt;/h1&gt;&lt;p&gt;使用keepalived做主备，其中一台设置为master，一台设置为backup。当master出现异常后，backup自动切换为master。当backup成为master后，master恢复正常后会再次抢占成为master，&lt;strong&gt;导致不必要的主备切换。&lt;/strong&gt;&lt;code&gt;因此可以将两台keepalived初始状态均配置为backup，设置不同的优先级，优先级高的设置nopreempt解决异常恢复后再次抢占的问题。&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;有如下配置表示意思也比较简单，VIP为192.168.0.18，2台机器的初始state都是BACKUP，machineA的优先级是15，machineB的优先级是13，配置了&lt;code&gt;/root/1.sh&lt;/code&gt;这个来检测服务是否正常。&lt;/p&gt;
&lt;p&gt;machineA机器配置：&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="keepalived" scheme="https://www.wumingx.com/tags/keepalived/"/>
    
  </entry>
  
  <entry>
    <title>openwrt出现dnsmasq部分域名解析不了的问题</title>
    <link href="https://www.wumingx.com/others/openwrt-dnsmasq.html"/>
    <id>https://www.wumingx.com/others/openwrt-dnsmasq.html</id>
    <published>2020-10-25T14:26:26.000Z</published>
    <updated>2020-10-25T15:24:17.935Z</updated>
    
    <content type="html"><![CDATA[<h1 id="前言"><a href="#前言" class="headerlink" title="前言"></a>前言</h1><p>最近几天在访问一些常用网站的时候，如<code>ping.chinaz.com</code>，出现了域名无法解析的情况，由于是常用网站，才不得不进行问题排查。网上搜索了几个方法，都是不行，如以下方法：</p><ul><li>在 <code>网络 --&gt; DHCP/DNS --&gt; 常规配置</code> 中，重绑定保护取消掉  <a href="[https://lisupy.github.io/2020/02/08/openwrt%E8%B7%AF%E7%94%B1%E5%99%A8dnsmasq%E9%83%A8%E5%88%86%E5%9F%9F%E5%90%8D%E6%97%A0%E6%B3%95%E8%A7%A3%E6%9E%90/](https://lisupy.github.io/2020/02/08/openwrt路由器dnsmasq部分域名无法解析/">openwrt路由器dnsmasq部分域名无法解析</a>)</li><li>在/etc/dnsmasq.conf添加配置no-resolv和server等：<a href="https://oracleblog.org/its-my-life/can-not-resolv-some-domain-name-in-openwrt/" target="_blank" rel="noopener">https://oracleblog.org/its-my-life/can-not-resolv-some-domain-name-in-openwrt/</a></li><li>dnsmasq进程无法启动：<a href="http://www.openwrt.pro/post-44.html" target="_blank" rel="noopener">http://www.openwrt.pro/post-44.html</a></li></ul><a id="more"></a><h1 id="原理"><a href="#原理" class="headerlink" title="原理"></a>原理</h1><p>对于dnsmasq就不多介绍了，主要功能是可以实现DNS解析以及DHCP分配IP的功能，openwrt环境提供了uci方式配置dnsmasq，配置文件是<code>/etc/config/dhcp.conf</code>，dnsmasq本身也有配置文件，比如<code>/etc/dnsmasq.conf</code>，它们之间的关系如下：</p><ul><li>uci配置文件：<code>/etc/config/dhcp.conf</code>，该文件能够通过luci界面进行修改，后台把它转换成dnsmasq能够识别的配置项。</li><li>dnsmasq本身的配置文件：<code>/etc/dnsmasq.conf</code>，通常情况下，dnsmasq运行时会读取该文件，但默认情况下，该文件全是注释，看似不起任何作用。</li><li>运行时配置文件：<code>/var/dnsmasq.conf.cfgxxxx</code></li></ul><p>使用ps来查看dnsmasq加载的配置文件，如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">root@fangdm:~<span class="comment">#ps |grep dnsmasq</span></span><br><span class="line"> 2243 root      1108 S    grep dnsmasq</span><br><span class="line">16520 dnsmasq   2124 S    /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf.cfg01411c -k -x /var/run/dnsmasq/dnsmasq.cfg01</span><br><span class="line">root@fangdm:~<span class="comment">#</span></span><br></pre></td></tr></table></figure><p>可以发现<code>/var/dnsmasq.conf.cfgxxxx</code>是真实有效的配置文件。打开配置如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line">root@fangdm:~<span class="comment"># cat /var/etc/dnsmasq.conf.cfg01411c</span></span><br><span class="line"><span class="comment"># auto-generated config file from /etc/config/dhcp</span></span><br><span class="line">conf-file=/etc/dnsmasq.conf</span><br><span class="line">dhcp-authoritative</span><br><span class="line">domain-needed</span><br><span class="line">filterwin2k</span><br><span class="line">no-negcache</span><br><span class="line"><span class="built_in">log</span>-queries=extra</span><br><span class="line">localise-queries</span><br><span class="line"><span class="built_in">read</span>-ethers</span><br><span class="line"><span class="built_in">enable</span>-ubus</span><br><span class="line">expand-hosts</span><br><span class="line"><span class="built_in">bind</span>-dynamic</span><br><span class="line"><span class="built_in">local</span>-service</span><br><span class="line">cache-size=9999</span><br><span class="line">domain=lan</span><br><span class="line">server=/lan/</span><br><span class="line">interface=br-lan</span><br><span class="line">dhcp-leasefile=/tmp/dhcp.leases</span><br><span class="line">resolv-file=/tmp/resolv.conf.auto</span><br><span class="line">dhcp-broadcast=tag:needs-broadcast</span><br><span class="line">addn-hosts=/tmp/hosts</span><br><span class="line">conf-dir=/tmp/dnsmasq.d</span><br><span class="line">user=dnsmasq</span><br><span class="line">group=dnsmasq</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">dhcp-ignore-names=tag:dhcp_bogus_hostname</span><br><span class="line">conf-file=/usr/share/dnsmasq/dhcpbogushostname.conf</span><br><span class="line"></span><br><span class="line">srv-host=_vlmcs._tcp,fangdm,1688,0,100</span><br><span class="line"></span><br><span class="line">bogus-priv</span><br><span class="line">conf-file=/usr/share/dnsmasq/rfc6761.conf</span><br><span class="line">dhcp-range=<span class="built_in">set</span>:lan,192.168.1.100,192.168.1.249,255.255.255.0,24h</span><br><span class="line">no-dhcp-interface=pppoe-wan</span><br></pre></td></tr></table></figure><p>从上可以看出：</p><div class="note success">            <p>1、dnsmasq运行时读取的配置文件是<code>/var/dnsmasq.conf.cfgxxxx</code>，该配置文件由<code>/etc/config/dhcp.conf</code>转换而来；从文件中的配置项<code>conf-file=/etc/dnsmasq.conf</code>来看，dnsmasq运行时还会读取<code>/etc/dnsmasq.conf</code>。</p><p>2、<code>server=/chinaz.com/114.114.114.114</code>表示chinaz.com这个域名以及 子域名都是通过 114来解析 ，而配置 <code>server=114.114.114.114</code> 则表示全部域名都走114解析。</p>          </div><p>dnsmasq在运行的时候，会将/etc/resolv.conf配置为127.0.0.1</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">root@fangdm:~<span class="comment"># cat /etc/resolv.conf</span></span><br><span class="line">search lan</span><br><span class="line">nameserver 127.0.0.1</span><br></pre></td></tr></table></figure><p>与此同时，<code>/var/dnsmasq.conf.cfgxxxx</code>有指定了<code>resolv-file=/tmp/resolv.conf.auto</code>，这个文件的内容如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">root@fangdm:~<span class="comment"># cat /tmp/resolv.conf.auto</span></span><br><span class="line"><span class="comment"># Interface wan</span></span><br><span class="line">nameserver 218.85.152.99</span><br><span class="line">nameserver 218.85.157.99</span><br><span class="line"><span class="comment"># Interface wan_6</span></span><br><span class="line">nameserver 240e:14:e000::1</span><br><span class="line">nameserver 240e:14:6000::1</span><br></pre></td></tr></table></figure><p>dnsmasq会使用这些DNS做为上游的DNS来解析的。</p><h1 id="排查"><a href="#排查" class="headerlink" title="排查"></a>排查</h1><p>经过了多个方法排查，确认了一点，我所遇到的问题，之前没有过遇到过，或者没有人分享过。不过经过排查之后，可以确认的是：dnsmasq进程是可以正常启 动的，上游DNS设置也是正确的。那为什么还会失败呢？再次观察<code>/var/dnsmasq.conf.cfgxxxx</code>，发现有一个配置项为：<code>conf-dir=/tmp/dnsmasq.d</code>，表示此目录下面还有配置文件，dnsmasq-adbyby.conf 以及 dnsmasq-go.conf，指向到/var/etc/dnsmasq-adbyby.d或者是/var/etc/dnsmasq-go.d上，打开一个文件看下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">root@fangdm:/tmp/etc/dnsmasq-go.d<span class="comment"># grep chinaz 01-pollution.conf</span></span><br><span class="line">server=/chinaz.com/127.0.0.1<span class="comment">#7453</span></span><br><span class="line">root@fangdm:/tmp/etc/dnsmasq-go.d<span class="comment">#</span></span><br><span class="line">root@fangdm:/tmp/etc/dnsmasq-go.d<span class="comment"># netstat -tunpl |grep 7453</span></span><br><span class="line">tcp        0      0 127.0.0.1:7453          0.0.0.0:*               LISTEN      24377/pdnsd</span><br><span class="line">udp        0      0 127.0.0.1:7453          0.0.0.0:*                           24377/pdnsd</span><br></pre></td></tr></table></figure><p>竟然看到了chinaz.com这个域名是走 7453端口，而这个端口的进程是pdnsd。查到这边，就明白了，很多之前有听说pdnsd可以用来加速dnsmasq效果的 ，其实现在dnsmasq也支持缓存了 ，没有必要使用了，而且此次问题的来源正是 pdnsd，直接 卸载就立马正常了。</p><h1 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h1><p>现在回头去看原因，是非常简单的，但分析的时候还是很费脑，跑去看dnsmasq配置文件含义，dnsmasq的启动脚本以及openwrt的界面配置等一些操作，总结来说，还是对openwrt不熟悉，不然直接去查dnsmasq的日志即可。</p><p>在openwrt的Luci是可以 开启dns记录日志 的  ，但是并没有说明 <code>log-queries=extra</code>  这个配置项是记录到哪里了，后续如果有问题可以在/etc/dnsmasq.conf配置文件中加入以下参数，即可现实日志的记录：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">log-dhcp</span><br><span class="line">log-queries</span><br><span class="line">log-facility=/tmp/dnsmasq.log</span><br></pre></td></tr></table></figure><h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><blockquote><p><a href="https://e-mailky.github.io/2018-07-14-dnsmasq" target="_blank" rel="noopener">dnsmasq详解及配置</a></p><p><a href="https://my.oschina.net/u/2404183/blog/4292974" target="_blank" rel="noopener">openwrt上安装smartdns加速访问</a></p><p><a href="http://blog.geeky.win/2020/20200727-01.html" target="_blank" rel="noopener">在openwrt环境下配置dnsmasq</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;前言&quot;&gt;&lt;a href=&quot;#前言&quot; class=&quot;headerlink&quot; title=&quot;前言&quot;&gt;&lt;/a&gt;前言&lt;/h1&gt;&lt;p&gt;最近几天在访问一些常用网站的时候，如&lt;code&gt;ping.chinaz.com&lt;/code&gt;，出现了域名无法解析的情况，由于是常用网站，才不得不进行问题排查。网上搜索了几个方法，都是不行，如以下方法：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;在 &lt;code&gt;网络 --&amp;gt; DHCP/DNS --&amp;gt; 常规配置&lt;/code&gt; 中，重绑定保护取消掉  &lt;a href=&quot;[https://lisupy.github.io/2020/02/08/openwrt%E8%B7%AF%E7%94%B1%E5%99%A8dnsmasq%E9%83%A8%E5%88%86%E5%9F%9F%E5%90%8D%E6%97%A0%E6%B3%95%E8%A7%A3%E6%9E%90/](https://lisupy.github.io/2020/02/08/openwrt路由器dnsmasq部分域名无法解析/&quot;&gt;openwrt路由器dnsmasq部分域名无法解析&lt;/a&gt;)&lt;/li&gt;
&lt;li&gt;在/etc/dnsmasq.conf添加配置no-resolv和server等：&lt;a href=&quot;https://oracleblog.org/its-my-life/can-not-resolv-some-domain-name-in-openwrt/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://oracleblog.org/its-my-life/can-not-resolv-some-domain-name-in-openwrt/&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;dnsmasq进程无法启动：&lt;a href=&quot;http://www.openwrt.pro/post-44.html&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;http://www.openwrt.pro/post-44.html&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="其他" scheme="https://www.wumingx.com/categories/others/"/>
    
    
      <category term="openwrt" scheme="https://www.wumingx.com/tags/openwrt/"/>
    
      <category term="dnsmasq" scheme="https://www.wumingx.com/tags/dnsmasq/"/>
    
  </entry>
  
  <entry>
    <title>git使用方法小结</title>
    <link href="https://www.wumingx.com/linux/git.html"/>
    <id>https://www.wumingx.com/linux/git.html</id>
    <published>2020-09-20T07:17:47.000Z</published>
    <updated>2020-09-20T07:46:38.836Z</updated>
    
    <content type="html"><![CDATA[<h1 id="本地文件添加到github"><a href="#本地文件添加到github" class="headerlink" title="本地文件添加到github"></a>本地文件添加到github</h1><ul><li>第一步生成ssh key</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh-keygen -t rsa -C &quot;your_email@youremail.com&quot;</span><br></pre></td></tr></table></figure><p>后面的 your_email@youremail.com 改为你在github上注册的邮箱，之后会要求确认路径和输入密码，我们这使用默认的一路回车就行。</p><p>成功的话会在当前用户~/下生成.ssh文件夹，如我的目录是<code>C:\Users\xxx\ .ssh</code>，打开id_rsa.pub，复制里面的key(可以把这个公钥文件删除，只保留id_rsa私钥文件)。</p><p>回到github上，进入 Account Settings（账户配置），左边选择<code>SSH Keys，Add SSH Key,title</code>随便填，粘贴在你电脑上生成的key。</p><a id="more"></a><p>验证是否成功的方法：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ssh -T git@github.com</span><br></pre></td></tr></table></figure><p>在git bash下输入上面的，如果有看到<code>You&#39;ve successfully authenticated, but GitHub does not provide shell access</code> 。这就表示已成功连上github。</p><ul><li><p>第二步建立git的仓库，在github上面新建一个repository即可。如果是为了保存资料，是可以新建repository时选择private。这样就不会公开了。创建好了之后，会有以下提示：</p><p>新仓库，需要做的事情：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">echo</span> <span class="string">"# hexo"</span> &gt;&gt; README.md</span><br><span class="line">git init</span><br><span class="line">git add README.md</span><br><span class="line">git commit -m <span class="string">"first commit"</span></span><br><span class="line">git branch -M master</span><br><span class="line">git remote add origin git@github.com:fang141x/hexo.git</span><br><span class="line">git push -u origin master</span><br></pre></td></tr></table></figure><p>已有仓库，</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">git remote add origin git@github.com:fang141x/hexo.git</span><br><span class="line">git branch -M master</span><br><span class="line">git push -u origin master</span><br></pre></td></tr></table></figure></li><li><p>第三步建立git的本地仓库。之前有建立好本地仓库就可以跳过这一步了。</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">cd</span> /E/shell</span><br><span class="line"><span class="built_in">echo</span> <span class="string">"# shell"</span> &gt;&gt; README.md</span><br><span class="line"><span class="comment">#初始化本地仓库，这时会在目录下创建.git</span></span><br><span class="line">git init</span><br><span class="line"><span class="comment">#将项目的所有文件添加到仓库中</span></span><br><span class="line">git add .</span><br><span class="line"><span class="comment">#将add的文件commit到仓库</span></span><br><span class="line">git commit -m <span class="string">"first commit"</span></span><br></pre></td></tr></table></figure><ul><li>第四步，上传代码</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#配置username和email，github每次commit都会需要这2个参数</span></span><br><span class="line">git config --global user.name <span class="string">"xxx"</span></span><br><span class="line">git config --global user.email <span class="string">"xxx@qq.com"</span></span><br><span class="line"></span><br><span class="line"><span class="comment">#将本地的仓库关联到github上</span></span><br><span class="line">git remote add origin git@github.com:xxx/xxx.git</span><br><span class="line"><span class="comment">#将代码到github远程仓库</span></span><br><span class="line">git push -u origin master</span><br></pre></td></tr></table></figure><p>这样就完成了git上传代码的过程。</p><p>有时一些文件不需要同步，我们可以在仓库的目录下创建 <code>.gitignore</code> 文件，把不需要同步的文件名写上去就了。以下表示 不同步assets文件名，但是<code>_posts/assets</code>这个目录需要同步。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ cat .gitignore </span><br><span class="line">assets</span><br><span class="line">!_posts/assets</span><br></pre></td></tr></table></figure><h1 id="同步数据"><a href="#同步数据" class="headerlink" title="同步数据"></a>同步数据</h1><ul><li>从github同步代码到本地</li></ul><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd /f/</span><br><span class="line">git clone git@github.com:fang141x/shell.git</span><br></pre></td></tr></table></figure><p>首先进入f盘，直接使用git clone就可以了。如上，就会在F:/shell/上面创建一个目录，注意，此目录不能存在。</p><ul><li>同步远程文件到本地<br>输入 git remote -v，查看远程仓库：<code>git pull origin master</code>从远端同步数据到本地</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">$ git remote -v</span><br><span class="line">origin  git@github.com:fang141x/shell.git (fetch)</span><br><span class="line">origin  git@github.com:fang141x/shell.git (push)</span><br><span class="line"></span><br><span class="line">fangdm@DESKTOP-ILP480A MINGW64 ~/shell (master)</span><br><span class="line">$ git pull origin master</span><br><span class="line">remote: Counting objects: 8, <span class="keyword">done</span>.</span><br><span class="line">remote: Compressing objects: 100% (6/6), <span class="keyword">done</span>.</span><br><span class="line">remote: Total 8 (delta 4), reused 6 (delta 2), pack-reused 0</span><br><span class="line">Unpacking objects: 100% (8/8), <span class="keyword">done</span>.</span><br><span class="line">From github.com:fang141x/shell</span><br><span class="line"> * branch            master     -&gt; FETCH_HEAD</span><br><span class="line">   f1bdc2c..ed75d6b  master     -&gt; origin/master</span><br><span class="line">Updating f1bdc2c..ed75d6b</span><br><span class="line">Fast-forward</span><br><span class="line"> debug.log                                          | 2102 ++++++++++++++++++++</span><br><span class="line"> ...00\237\345\221\275\344\273\244\351\233\206.qbl<span class="string">" |  164 ++</span></span><br><span class="line"><span class="string"> 2 files changed, 2266 insertions(+)</span></span><br><span class="line"><span class="string"> create mode 100644 debug.log</span></span><br><span class="line"><span class="string"> create mode 100644 "</span>\351\273\230\350\256\244\345\277\253\351\200\237\345\221\275\344\273\244\351\233\206.qbl<span class="string">"</span></span><br></pre></td></tr></table></figure><h1 id="本地仓库"><a href="#本地仓库" class="headerlink" title="本地仓库"></a>本地仓库</h1><p>本地仓库可以分为工作区和版本库。由 git 维护的三棵”树”组成。</p><ul><li>第一个是你的 工作目录，它持有实际文件；</li><li>第二个是 暂存区（Index/stage），它像个缓存区域，临时保存你的改动；</li><li>最后是 HEAD，它指向你最后一次提交的结果。</li></ul><p><img src="/assets/trees.png" alt="image"></p><p>你可以提出更改（把它们添加到暂存区），使用如下命令：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">git add &lt;filename&gt;</span><br><span class="line">git add *</span><br></pre></td></tr></table></figure><p>这是 git 基本工作流程的第一步；使用如下命令以实际提交改动：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git commit -m &quot;代码提交信息&quot;</span><br></pre></td></tr></table></figure><p>现在，你的改动已经提交到了 HEAD，但是还没到你的远端仓库。</p><h1 id="分支管理"><a href="#分支管理" class="headerlink" title="分支管理"></a>分支管理</h1><h2 id="本地仓库操作"><a href="#本地仓库操作" class="headerlink" title="本地仓库操作"></a>本地仓库操作</h2><ul><li>建立分支</li></ul><p>创建dev分支，然后切换到dev分支，<code>git checkout -b dev</code>相当于2条命令，<code>git branch dev、git checkout dev</code></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git checkout -b dev</span></span><br><span class="line">Switched to a new branch <span class="string">'dev'</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git branch</span></span><br><span class="line">* dev</span><br><span class="line">  master</span><br></pre></td></tr></table></figure><p>git branch命令会列出所有分支，当前分支前面会标一个*号。</p><ul><li>修改文件且commit</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#echo 'Creating a new branch is quick.' &gt;&gt;test.txt</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git add test.txt</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git commit -m "branch test"</span></span><br><span class="line">[dev 35132aa] branch <span class="built_in">test</span></span><br><span class="line"> 1 files changed, 1 insertions(+), 0 deletions(-)</span><br></pre></td></tr></table></figure><ul><li>合并分支：<code>git merge命令用于合并指定分支到当前分支</code>。</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git checkout master</span></span><br><span class="line">Switched to branch <span class="string">'master'</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git branch</span></span><br><span class="line">  dev</span><br><span class="line">* master</span><br><span class="line"></span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt</span></span><br><span class="line">Git is a distributed version control system.</span><br><span class="line"></span><br><span class="line">Git is free software.</span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git merge dev</span></span><br><span class="line">Updating 69345bb..35132aa</span><br><span class="line">Fast-forward</span><br><span class="line"> test.txt |    1 +</span><br><span class="line"> 1 files changed, 1 insertions(+), 0 deletions(-)</span><br></pre></td></tr></table></figure><p>注意到上面的Fast-forward信息，Git告诉我们，这次合并是“快进模式”，也就是直接把master指向dev的当前提交，所以合并速度非常快。我们可以使用—no-ff来关闭fast forward，这里Git就会在merge时生成一个新的commit，这样，从分支历史上就可以看出分支信息。</p><ul><li>删除分支：合并完成后，就可以放心地删除dev分支了：</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git branch -d dev</span></span><br><span class="line">Deleted branch dev (was 35132aa).</span><br><span class="line">[root@home gitstudy]<span class="comment">#git branch</span></span><br><span class="line">* master</span><br></pre></td></tr></table></figure><h2 id="远程仓库操作"><a href="#远程仓库操作" class="headerlink" title="远程仓库操作"></a>远程仓库操作</h2><p>我们也可以直接将分支同步到github上。</p><ul><li>将新分支发布在github上</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">$ git push origin dev</span><br><span class="line">Counting objects: 3, <span class="keyword">done</span>.</span><br><span class="line">Delta compression using up to 4 threads.</span><br><span class="line">Compressing objects: 100% (2/2), <span class="keyword">done</span>.</span><br><span class="line">Writing objects: 100% (3/3), 280 bytes | 280.00 KiB/s, <span class="keyword">done</span>.</span><br><span class="line">Total 3 (delta 1), reused 0 (delta 0)</span><br><span class="line">remote: Resolving deltas: 100% (1/1), completed with 1 <span class="built_in">local</span> object.</span><br><span class="line">To github.com:fang141x/python.git</span><br><span class="line"> * [new branch]      dev -&gt; dev</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment">#在github远程端删除一个分(分支名前的冒号代表删除)</span></span><br><span class="line">$ git push origin :dev</span><br><span class="line">To github.com:fang141x/python.git</span><br><span class="line"> - [deleted]         dev</span><br></pre></td></tr></table></figure><ul><li>将github上的分支同步下来</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">$ git checkout -b dev origin/dev</span><br><span class="line">Switched to a new branch <span class="string">'dev'</span></span><br><span class="line">Branch dev <span class="built_in">set</span> up to track remote branch dev from origin.</span><br><span class="line">$ git branch</span><br><span class="line">* dev</span><br><span class="line">  master</span><br><span class="line">$ git add test.txt</span><br><span class="line">$ git commit -m <span class="string">"add name"</span></span><br><span class="line">[dev d9229d5] add name</span><br><span class="line"> 1 file changed, 2 insertions(+), 1 deletion(-)</span><br><span class="line"></span><br><span class="line">$ git push origin dev</span><br><span class="line">Counting objects: 3, <span class="keyword">done</span>.</span><br><span class="line">Delta compression using up to 4 threads.</span><br><span class="line">Compressing objects: 100% (2/2), <span class="keyword">done</span>.</span><br><span class="line">Writing objects: 100% (3/3), 282 bytes | 141.00 KiB/s, <span class="keyword">done</span>.</span><br><span class="line">Total 3 (delta 1), reused 0 (delta 0)</span><br><span class="line">remote: Resolving deltas: 100% (1/1), completed with 1 <span class="built_in">local</span> object.</span><br><span class="line">To github.com:fang141x/python.git</span><br><span class="line">   364e514..d9229d5  dev -&gt; dev</span><br></pre></td></tr></table></figure><ul><li>Git鼓励大量使用分支：</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">查看分支：git branch</span><br><span class="line">创建分支：git branch &lt;name&gt;</span><br><span class="line">切换分支：git checkout &lt;name&gt;</span><br><span class="line">创建+切换分支：git checkout -b &lt;name&gt;</span><br><span class="line">合并某分支到当前分支：git merge &lt;name&gt;</span><br><span class="line">删除分支：git branch -d &lt;name&gt;</span><br></pre></td></tr></table></figure><h2 id="解决冲突"><a href="#解决冲突" class="headerlink" title="解决冲突"></a>解决冲突</h2><p>如果在合并分支时，在master里面又修改了同一个文件，同时也commit到同步库，这时进行合并就会出现异常了。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#这时合并就会提示有冲突了</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git merge feature1</span></span><br><span class="line">Auto-merging test.txt</span><br><span class="line">CONFLICT (content): Merge conflict <span class="keyword">in</span> test.txt</span><br><span class="line">Automatic merge failed; fix conflicts and <span class="keyword">then</span> commit the result.</span><br><span class="line"></span><br><span class="line"><span class="comment">#这时文件内容会发现变化</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt</span></span><br><span class="line">Git is a distributed version control system.</span><br><span class="line"></span><br><span class="line">Git is free software.</span><br><span class="line"><span class="built_in">test</span></span><br><span class="line">Creating a new branch is quick.</span><br><span class="line">&lt;&lt;&lt;&lt;&lt;&lt;&lt; HEAD</span><br><span class="line">Creating a new branch is quick &amp; simple.</span><br><span class="line">=======</span><br><span class="line">Creating a new branch is quick AND simple.</span><br><span class="line">&gt;&gt;&gt;&gt;&gt;&gt;&gt; feature1</span><br><span class="line"></span><br><span class="line"><span class="comment">#使用git status就可以看到是哪个文件有冲突了</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line"><span class="comment"># Unmerged paths:</span></span><br><span class="line"><span class="comment">#   (use "git add/rm &lt;file&gt;..." as appropriate to mark resolution)</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment">#both modified:      test.txt</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line">no changes added to commit (use <span class="string">"git add"</span> and/or <span class="string">"git commit -a"</span>)</span><br></pre></td></tr></table></figure><p>我们需要手工修改这个文件之后，再次add/commit就可以了。也就是有冲突了，就需要手工解决。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git add test.txt</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git commit -m "conflict fixed"</span></span><br><span class="line">[master 562533a] conflict fixed</span><br><span class="line">[root@home gitstudy]<span class="comment">#git log --graph --pretty=oneline --abbrev-commit</span></span><br><span class="line">*   562533a conflict fixed</span><br><span class="line">|\</span><br><span class="line">| * fe70421 And simple</span><br><span class="line">* | 55c7613 &amp; simple</span><br><span class="line">|/</span><br><span class="line">* 35132aa branch <span class="built_in">test</span></span><br><span class="line">* 69345bb remove test.txt</span><br><span class="line">* 56a0e8f a.txt</span><br><span class="line">* 3994ba3 add dd</span><br><span class="line">* 164dd36 add test.txt</span><br><span class="line">* ccd4067 add test.txt</span><br></pre></td></tr></table></figure><p>可以用<code>gitlog --graph</code>命令可以看到分支合并图。<br>修复bug时，我们会通过创建新的bug分支进行修复，然后合并，最后删除；当手头工作没有完成时，先把工作现场git stash一下，然后去修复bug，修复后，再git stash pop，回到工作现场。如果要丢弃一个没有被合并过的分支，可以通过<code>git branch -D &lt;name&gt;</code>强行删除。</p><h1 id="git其他常用命令"><a href="#git其他常用命令" class="headerlink" title="git其他常用命令"></a>git其他常用命令</h1><ul><li>git status</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line">nothing to commit (working directory clean)</span><br></pre></td></tr></table></figure><p>Git告诉我们当前没有需要提交的修改，而且，工作目录是干净（working directory clean）的。</p><ul><li><p>git diff 查看修改了哪些内容</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git diff test.txt </span></span><br><span class="line">diff --git a/test.txt b/test.txt</span><br><span class="line">index a4c0a67..bb0f9ef 100644</span><br><span class="line">--- a/test.txt</span><br><span class="line">+++ b/test.txt</span><br><span class="line">@@ -1,3 +1,4 @@</span><br><span class="line"> Git is a distributed version control system.</span><br><span class="line"> </span><br><span class="line"> Git is free software.</span><br><span class="line">+<span class="built_in">test</span></span><br></pre></td></tr></table></figure></li><li><p>git log查看commit的提交日志</p></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git log --pretty=oneline</span></span><br><span class="line">3994ba33b9236a1d80e3146babf64202e9ed832b add dd</span><br><span class="line">164dd363a2f66bfe8f289798a3d93f913416cce7 add test.txt</span><br><span class="line">ccd40670223d1d2aedfb073b72b08dedc7f12044 add test.txt</span><br></pre></td></tr></table></figure><ul><li>git reset 可以根据commitid来指定回退版本</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#此为回退到前一个版本</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git reset --hard HEAD^</span></span><br><span class="line">HEAD is now at 164dd36 add test.tx</span><br></pre></td></tr></table></figure><p>可以指定commit id来需要回退或者到哪个版本下。<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt </span></span><br><span class="line">Git is a distributed version control system.</span><br><span class="line">Git is free software.</span><br><span class="line">[root@home gitstudy]<span class="comment">#</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git reset --hard 3994ba33b9</span></span><br><span class="line">HEAD is now at 3994ba3 add dd</span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt </span></span><br><span class="line">Git is a distributed version control system.</span><br><span class="line">Git is free software.</span><br><span class="line"><span class="built_in">test</span></span><br></pre></td></tr></table></figure></p><ul><li>git reflog 用来记录reset的日志</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#git reflog</span></span><br><span class="line">ccd4067 HEAD@&#123;0&#125;: ccd40670223d1d2: updating HEAD</span><br><span class="line">3994ba3 HEAD@&#123;1&#125;: 3994ba33b9: updating HEAD</span><br><span class="line">164dd36 HEAD@&#123;2&#125;: HEAD^: updating HEAD</span><br><span class="line">3994ba3 HEAD@&#123;3&#125;: commit: add dd</span><br><span class="line">164dd36 HEAD@&#123;4&#125;: commit: add test.txt</span><br></pre></td></tr></table></figure><ul><li>git checkout 其实是用版本库里的版本替换工作区的版本，无论工作区是修改还是删除，都可以“一键还原”。</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line">[root@home gitstudy]<span class="comment">#echo 'checkout' &gt;test.txt </span></span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt </span></span><br><span class="line">checkout</span><br><span class="line"><span class="comment">#修改文件后，status查看会提示要git add，或者使用git checkout来放弃修改</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line"><span class="comment"># Changed but not updated:</span></span><br><span class="line"><span class="comment">#   (use "git add &lt;file&gt;..." to update what will be committed)</span></span><br><span class="line"><span class="comment">#   (use "git checkout -- &lt;file&gt;..." to discard changes in working directory)</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment">#modified:   test.txt</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line">no changes added to commit (use <span class="string">"git add"</span> and/or <span class="string">"git commit -a"</span>)</span><br><span class="line">[root@home gitstudy]<span class="comment">#git add test.txt </span></span><br><span class="line"></span><br><span class="line"><span class="comment">#这边git add了一下，使用status查看可以看到需要给commit或者使用git reset HEAD file可以把暂存区的修改撤销掉（unstage）</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line"><span class="comment"># Changes to be committed:</span></span><br><span class="line"><span class="comment">#   (use "git reset HEAD &lt;file&gt;..." to unstage)</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment">#modified:   test.txt</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git reset HEAD test.txt </span></span><br><span class="line">Unstaged changes after reset:</span><br><span class="line">Mtest.txt</span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line"><span class="comment"># Changed but not updated:</span></span><br><span class="line"><span class="comment">#   (use "git add &lt;file&gt;..." to update what will be committed)</span></span><br><span class="line"><span class="comment">#   (use "git checkout -- &lt;file&gt;..." to discard changes in working directory)</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment">#modified:   test.txt</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line">no changes added to commit (use <span class="string">"git add"</span> and/or <span class="string">"git commit -a"</span>)</span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt </span></span><br><span class="line">checkout</span><br><span class="line">[root@home gitstudy]<span class="comment">#git checkout -- test.txt </span></span><br><span class="line">[root@home gitstudy]<span class="comment">#cat test.txt </span></span><br><span class="line">Git is a distributed version control system.</span><br><span class="line"></span><br><span class="line">Git is free software.</span><br><span class="line"><span class="built_in">test</span></span><br></pre></td></tr></table></figure><ul><li>git rm 从版本库里面删除文件</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#先加一个新文件</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#echo '333' &gt;a.txt</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git add a.txt </span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git commit -m 'a.txt'</span></span><br><span class="line">[master 56a0e8f] a.txt</span><br><span class="line"> 1 files changed, 1 insertions(+), 0 deletions(-)</span><br><span class="line"> create mode 100644 a.txt</span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line">nothing to commit (working directory clean)</span><br><span class="line"><span class="comment">#再删除</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#rm -f a.txt </span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git rm a.txt</span></span><br><span class="line">rm <span class="string">'a.txt'</span></span><br><span class="line">[root@home gitstudy]<span class="comment">#git commit -m "remove a.txt"</span></span><br><span class="line">[master 69345bb] remove a.txt</span><br><span class="line"> 1 files changed, 0 insertions(+), 1 deletions(-)</span><br><span class="line"> delete mode 100644 a.txt</span><br><span class="line">[root@home gitstudy]<span class="comment">#git status</span></span><br><span class="line"><span class="comment"># On branch master</span></span><br><span class="line">nothing to commit (working directory clean)</span><br></pre></td></tr></table></figure><p>标签</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">命令git tag &lt;name&gt;用于新建一个标签，默认为HEAD，也可以指定一个commit id；</span><br><span class="line">git tag -a &lt;tagname&gt; -m <span class="string">"blablabla..."</span>可以指定标签信息；</span><br><span class="line">git tag -s &lt;tagname&gt; -m <span class="string">"blablabla..."</span>可以用PGP签名标签；</span><br><span class="line"></span><br><span class="line">命令git tag可以查看所有标签。</span><br><span class="line">命令git push origin &lt;tagname&gt;可以推送一个本地标签；</span><br><span class="line">命令git push origin --tags可以推送全部未推送过的本地标签；</span><br><span class="line">命令git tag -d &lt;tagname&gt;可以删除一个本地标签；</span><br><span class="line">命令git push origin :refs/tags/&lt;tagname&gt;可以删除一个远程标签。</span><br></pre></td></tr></table></figure><h1 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h1><ul><li><p>管理修改</p><p>每次修改，如果不add到暂存区，那就不会加入到commit中</p></li><li><p>撤销修改</p><ul><li><p>场景1：当你改乱了工作区某个文件的内容，想直接丢弃工作区的修改时，用命令<code>git checkout -- file</code>。</p></li><li><p>场景2：当你不但改乱了工作区某个文件的内容，还添加到了暂存区时，想丢弃修改，分两步，第一步用命令<code>git reset HEAD file</code>，就回到了场景1，第二步按场景1操作。</p></li></ul></li><li><p>删除文件，在本地删除了一个文件之后</p><ul><li>场景1：确实要从版本库中删除该文件，那就用命令g<code>it rm</code>删掉，并且<code>git commit</code></li><li>场景2：删错了，因为版本库里还有呢，所以可以使用<code>git checkout -- file</code>，很轻松地把误删的文件恢复到最新版本</li></ul></li></ul><p>最后引用 <a href="https://zhuanlan.zhihu.com/p/94008510" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/94008510</a> 的流程图来结尾：</p><p><img src="/assets/git.jpg" alt="git"></p>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;本地文件添加到github&quot;&gt;&lt;a href=&quot;#本地文件添加到github&quot; class=&quot;headerlink&quot; title=&quot;本地文件添加到github&quot;&gt;&lt;/a&gt;本地文件添加到github&lt;/h1&gt;&lt;ul&gt;
&lt;li&gt;第一步生成ssh key&lt;/li&gt;
&lt;/ul&gt;
&lt;figure class=&quot;highlight plain&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;ssh-keygen -t rsa -C &amp;quot;your_email@youremail.com&amp;quot;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
&lt;p&gt;后面的 your_email@youremail.com 改为你在github上注册的邮箱，之后会要求确认路径和输入密码，我们这使用默认的一路回车就行。&lt;/p&gt;
&lt;p&gt;成功的话会在当前用户~/下生成.ssh文件夹，如我的目录是&lt;code&gt;C:\Users\xxx\ .ssh&lt;/code&gt;，打开id_rsa.pub，复制里面的key(可以把这个公钥文件删除，只保留id_rsa私钥文件)。&lt;/p&gt;
&lt;p&gt;回到github上，进入 Account Settings（账户配置），左边选择&lt;code&gt;SSH Keys，Add SSH Key,title&lt;/code&gt;随便填，粘贴在你电脑上生成的key。&lt;/p&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="git" scheme="https://www.wumingx.com/tags/git/"/>
    
  </entry>
  
  <entry>
    <title>openssl手工生成证书以及acme.sh自动部署证书</title>
    <link href="https://www.wumingx.com/tcpip/openssl-acmesh.html"/>
    <id>https://www.wumingx.com/tcpip/openssl-acmesh.html</id>
    <published>2020-09-12T14:34:09.000Z</published>
    <updated>2022-11-20T16:40:55.337Z</updated>
    
    <content type="html"><![CDATA[<h1 id="证书简介"><a href="#证书简介" class="headerlink" title="证书简介"></a>证书简介</h1><p>有关https是如何保证数据的安全证的，可以参考文章：<a href="/tcpip/https-theory.html">详解https加密通信原理</a> ，以下简介证书相关的知识点。</p><p>我们一般在服务器上面可以使用openssl命令来生成证书的.而一般是使用X509的证书链。x509证书一般会用到三类文件，<code>key，csr，crt</code>。</p><ul><li><p>key是私用密钥，openssl格式，通常是rsa算法</p></li><li><p>csr是证书请求文件，用于申请证书。在申请的时候，必须使用自己的私钥来签署申请，还可以设定一个密钥。</p></li><li>crt是证书文件（windows下面的csr，其实是crt），是签署人用自己的key给你签署的凭证。</li></ul><a id="more"></a><p>除了以上格式,openssl还有如下后缀名的文件：</p><ul><li>.crl格式：证书吊销列表，Certificate Revocation List的缩写</li><li>.pem格式：用于导出，导入证书时候的证书的格式，有证书开头，结尾的格式</li><li>.p12 “或者 “.pfx” : 用于实现存储许多加密对象在一个单独的文件中。通常用它来打包<strong>一个私钥及有关的 X.509 证书</strong>，或者打包信任链的全部项目。</li></ul><p>在部署HTTPS证书时，不同的服务器我们需要用到不同格式的证书文件，常见的证书文件格式有以下几种：</p><ul><li>PEM<ul><li>适用于Apache、Nginx、Candy Server等Web服务器</li><li>常见的文件后缀为.pem、.crt、.cer</li><li>可以存放证书或私钥，或者两者都包含</li><li>.key后缀一般只用于证书私钥文件</li></ul></li><li>PFX<ul><li>适用于IIS等Web服务器</li><li>常见的文件后缀为.pfx、.p12</li><li>同时包含证书和私钥，且一般有密码保护</li></ul></li><li>JKS<ul><li>适用于Tomcat、Weblogic、JBoss、Jetty等Web服务器</li><li>常见的文件后缀为.jks</li></ul></li></ul><h1 id="手工生成证书"><a href="#手工生成证书" class="headerlink" title="手工生成证书"></a>手工生成证书</h1><h2 id="单域名版"><a href="#单域名版" class="headerlink" title="单域名版"></a>单域名版</h2><p>有时我们需要在内网环境下测试https,可以使用以下方法进行操作:</p><p>生成私钥key:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">// 生成rsa私钥，des3算法，openssl格式，2048位强度。</span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl genrsa -des3 -out in.key 2048</span></span><br><span class="line">Generating RSA private key, 2048 bit long modulus</span><br><span class="line">.............................................................+++</span><br><span class="line">................................................+++</span><br><span class="line">e is 65537 (0x10001)</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> in.key:</span><br><span class="line">139631662581648:error:28069065:lib(40):UI_set_result:result too small:ui_lib.c:831:You must <span class="built_in">type</span> <span class="keyword">in</span> 4 to 1023 characters //提示必须输入4位以上的密码</span><br><span class="line">Enter pass phrase <span class="keyword">for</span> in.key:</span><br><span class="line">Verifying - Enter pass phrase <span class="keyword">for</span> in.key:</span><br><span class="line"></span><br><span class="line">// 可以通过以下方法去除密码</span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl rsa -in in.key -out out.key</span></span><br><span class="line">Enter pass phrase <span class="keyword">for</span> in.key:</span><br><span class="line">writing RSA key</span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># ll</span></span><br><span class="line">total 8</span><br><span class="line">-rw-r--r-- 1 root root 1751 Sep 12 23:03 in.key</span><br><span class="line">-rw-r--r-- 1 root root 1679 Sep 12 23:03 out.key</span><br></pre></td></tr></table></figure><p>使用<code>openssl genrsa -out in.key 2048</code> 就默认是没有密码的.</p><p>生成证书请求文件csr:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl req -new -key out.key -out out.csr</span></span><br><span class="line">You are about to be asked to enter information that will be incorporated</span><br><span class="line">into your certificate request.</span><br><span class="line">What you are about to enter is what is called a Distinguished Name or a DN.</span><br><span class="line">There are quite a few fields but you can leave some blank</span><br><span class="line">For some fields there will be a default value,</span><br><span class="line">If you enter <span class="string">'.'</span>, the field will be left blank.</span><br><span class="line">-----</span><br><span class="line">Country Name (2 letter code) [XX]:CN// 输入国家代码</span><br><span class="line">State or Province Name (full name) []:FJ// 输入省份</span><br><span class="line">Locality Name (eg, city) [Default City]:XM// 输入城市</span><br><span class="line">Organization Name (eg, company) [Default Company Ltd]:XM// 输入组织机构(或公司名</span><br><span class="line">Organizational Unit Name (eg, section) []:XM// 输入机构部门</span><br><span class="line">Common Name (eg, your name or your server<span class="string">'s hostname) []:wumingx.com  //这里填入证书的域名,如果要生成泛域名证书,要以*.domain.com开头</span></span><br><span class="line"><span class="string">Email Address []:8@wumingx.com     // 你的邮箱地址</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">Please enter the following '</span>extra<span class="string">' attributes</span></span><br><span class="line"><span class="string">to be sent with your certificate request</span></span><br><span class="line"><span class="string">A challenge password []:// 你的证书密码，如果不想设置密码，可以直接回车</span></span><br><span class="line"><span class="string">An optional company name []:</span></span><br></pre></td></tr></table></figure><p>生成证书:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl x509 -req -days 365 -in out.csr -signkey out.key -out out.crt</span></span><br><span class="line">Signature ok</span><br><span class="line">subject=/C=CN/ST=FJ/L=XM/O=XM/OU=XM/CN=wumingx.com/emailAddress=8@wumingx.com</span><br><span class="line">Getting Private key</span><br></pre></td></tr></table></figure><p>以上三步可以简写为：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">openssl genrsa -out xxx.key 2048</span><br><span class="line">openssl req -new -x509 -days 3650 -key xxx.key -out xxx.crt -subj <span class="string">"/C=CN/ST=FJ/L=XM/O=WS/OU=WS/CN=domain1/CN=domain2"</span></span><br></pre></td></tr></table></figure><p>可以使用命令 <code>openssl x509 -in out.crt -noout -text</code> 来查看证书的信息，也可以使用以下命令查看证书的有效期以及匹配的域名:</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl x509 -in out.crt -noout -startdate -enddate -subject //可以用-dates代替-startdate -enddate</span></span><br><span class="line">notBefore=Sep 12 15:43:58 2020 GMT</span><br><span class="line">notAfter=Sep 12 15:43:58 2021 GMT</span><br><span class="line">subject= /C=CN/ST=FJ/L=XM/O=XM/OU=XM/CN=wumingx.com/emailAddress=8@wumingx.com</span><br><span class="line"></span><br><span class="line">// 验证域名是否匹配证书</span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl x509 -in out.crt -noout -checkhost wumingx.com</span></span><br><span class="line">Hostname wumingx.com does match certificate</span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># </span></span><br><span class="line">[root@VM_0_6_centos ssl]<span class="comment"># openssl x509 -in out.crt -noout -checkhost www.wumingx.com</span></span><br><span class="line">Hostname www.wumingx.com does NOT match certificate</span><br></pre></td></tr></table></figure><h2 id="多域名版"><a href="#多域名版" class="headerlink" title="多域名版"></a>多域名版</h2><p>多域名以及IP就需要使用V3版本的证书了。</p><h3 id="方法一"><a href="#方法一" class="headerlink" title="方法一"></a>方法一</h3><p>先用 openssl req 生成证书请求，会顺带生成私钥：<code>openssl req -nodes -new -keyout server.key -out server.csr -subj &quot;/C=CN/ST=FJ/L=XM/O=ws/CN=ws&quot;</code></p><ol><li><code>-nodes</code>， 密钥文件不加密。不加此参数会要求 “Enter PEM pass phrase：”</li><li><code>-keyout</code>，密钥生成路径</li><li><code>-out</code>，证书请求文件生成路径</li><li><code>-subj</code>，证书的国家地区等信息</li></ol><p>生成自签名证书，带上 SAN 扩展信息：<code>openssl x509 -req -sha256 -days 3650 -in server.csr -signkey server.key -out server.crt -extfile &lt;(printf &quot;subjectAltName=DNS:localhost,DNS:example.com,IP:192.168.1.1,IP:127.0.0.1&quot;)</code></p><ol><li><code>-sha256</code>，使用的加密方法</li><li><code>-days</code>，证书有效期</li><li><code>-in</code>，证书请求文件路径</li><li><code>-signkey</code>，用于提供自签名时的私钥文件</li><li><code>-out</code>，证书生成路径</li><li><code>-extfile</code>，指定签名时包含要添加到证书中的扩展项的文件，自签名时候可以这样使用</li></ol><blockquote><p><a href="https://www.iszy.cc/posts/u62i8r/" target="_blank" rel="noopener">https://www.iszy.cc/posts/u62i8r/</a></p></blockquote><p>备注：可以使用以下命令检查线上域名https证书的情况：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">openssl s_client -servername www.baidu.com -connect 1.2.3.4:443 | openssl x509 -noout -dates # -connect可以接IP，也可以接域名</span><br><span class="line">nmap -sV --script ssl-cert,ssl-enum-ciphers -p 443 www.baidu.com #可以使用此命令查看域名证书支持的版本</span><br></pre></td></tr></table></figure><h2 id="双向证书"><a href="#双向证书" class="headerlink" title="双向证书"></a>双向证书</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">openssl genrsa -out ca.key 2048</span><br><span class="line">openssl req -new -out ca.csr -key ca.key -subj <span class="string">"/C=CN/ST=FJ/L=XM/O=root/CN=root"</span></span><br><span class="line">openssl x509 -req -<span class="keyword">in</span> ca.csr -out ca.crt -signkey ca.key -CAcreateserial -days 3650</span><br><span class="line"></span><br><span class="line">openssl genrsa -out server.key 2048</span><br><span class="line">openssl req -new -out server.csr -key server.key -subj <span class="string">"/C=CN/ST=FJ/L=XM/O=root/CN=server"</span></span><br><span class="line">openssl x509 -req -<span class="keyword">in</span> server.csr -out server.crt -signkey server.key -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">openssl genrsa -out client.key 2048</span><br><span class="line">openssl req -new -out client.csr -key client.key -subj <span class="string">"/C=CN/ST=FJ/L=XM/O=root/CN=client"</span></span><br><span class="line">openssl x509 -req -<span class="keyword">in</span> client.csr -out client.crt -signkey client.key -CA ca.crt -CAkey ca.key -CAcreateserial -days 3650</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">[root@localhost fdm]<span class="comment"># openssl verify -CAfile ca.crt server.crt </span></span><br><span class="line">server.crt: OK</span><br><span class="line">[root@localhost fdm]<span class="comment"># openssl verify -CAfile ca.crt client.crt </span></span><br><span class="line">client.crt: OK</span><br><span class="line"></span><br><span class="line"><span class="comment"># 开启一个端口</span></span><br><span class="line">openssl s_server -accept 8090 -key server.key -cert server.crt -CAfile ca.crt -Verify 1</span><br><span class="line">openssl s_client -connect localhost:8090 -key client.key -cert client.crt -CAfile ca.crt -showcerts</span><br></pre></td></tr></table></figure><blockquote><p><a href="https://blog.csdn.net/easylife206/article/details/107776854" target="_blank" rel="noopener">https://blog.csdn.net/easylife206/article/details/107776854</a></p></blockquote><h1 id="acme-sh"><a href="#acme-sh" class="headerlink" title="acme.sh"></a>acme.sh</h1><p>由于https的证书是需要费用的，可以使用letsencrypt 生成免费的证书，所以我们可以借助acme.sh来申请泛域名的证书，且可以实现自动更新证书。</p><h2 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h2><p>安装很简单, 一个命令:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl  https://get.acme.sh | sh</span><br></pre></td></tr></table></figure><p>安装过程进行了以下几步:</p><ol><li>把 acme.sh 安装到你的 <strong>home</strong> 目录下:</li></ol><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">~/.acme.sh/</span><br></pre></td></tr></table></figure><p>并创建 一个 bash 的 alias, 方便你的使用: <code>alias acme.sh=~/.acme.sh/acme.sh</code>，将这一行写入到.bash_profile即可</p><p>2). 自动为你创建 cronjob, 每天 0:00 点自动检测所有的证书, 如果快过期了, 需要更新, 则会自动更新证书.</p><p>更高级的安装选项请参考: <a href="https://github.com/Neilpang/acme.sh/wiki/How-to-install" target="_blank" rel="noopener">https://github.com/Neilpang/acme.sh/wiki/How-to-install</a></p><p><strong>安装过程不会污染已有的系统任何功能和文件</strong>, 所有的修改都限制在安装目录中: <code>~/.acme.sh/</code></p><h2 id="生成证书"><a href="#生成证书" class="headerlink" title="生成证书"></a>生成证书</h2><p>生成证书有2种方式进行验证，http 和 dns 验证。申请泛域名证书只能通过DNS认证。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos ~]<span class="comment"># acme.sh  --issue -d *.wumingx.com -d wumingx.com --webroot /home/wwwroot/wumingx.com/</span></span><br><span class="line">[Mon Sep 14 22:43:14 CST 2020] Using CA: https://acme-v02.api.letsencrypt.org/directory</span><br><span class="line">[Mon Sep 14 22:43:14 CST 2020] Creating domain key</span><br><span class="line">[Mon Sep 14 22:43:14 CST 2020] The domain key is here: /root/.acme.sh/*.wumingx.com/*.wumingx.com.key</span><br><span class="line">[Mon Sep 14 22:43:14 CST 2020] Multi domain=<span class="string">'DNS:*.wumingx.com,DNS:wumingx.com'</span></span><br><span class="line">[Mon Sep 14 22:43:14 CST 2020] Getting domain auth token <span class="keyword">for</span> each domain</span><br><span class="line">[Mon Sep 14 22:43:25 CST 2020] Getting webroot <span class="keyword">for</span> domain=<span class="string">'*.wumingx.com'</span></span><br><span class="line">[Mon Sep 14 22:43:25 CST 2020] Error, can not get domain token entry *.wumingx.com <span class="keyword">for</span> http-01</span><br><span class="line">[Mon Sep 14 22:43:25 CST 2020] The supported validation types are: dns-01 , but you specified: http-01</span><br><span class="line">[Mon Sep 14 22:43:25 CST 2020] Please add <span class="string">'--debug'</span> or <span class="string">'--log'</span> to check more details.</span><br><span class="line">[Mon Sep 14 22:43:25 CST 2020] See: https://github.com/acmesh-official/acme.sh/wiki/How-to-debug-acme.sh</span><br></pre></td></tr></table></figure><p>有报错：<strong>The supported validation types are: dns-01 , but you specified: http-01</strong></p><h3 id="http"><a href="#http" class="headerlink" title="http"></a>http</h3><p>http 方式需要在你的网站根目录下放置一个文件, 来验证你的域名所有权,完成验证. 然后就可以生成证书了.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">acme.sh  --issue  -d mydomain.com -d www.mydomain.com  --webroot  /home/wwwroot/mydomain.com/</span><br></pre></td></tr></table></figure><p>只需要指定域名, 并指定域名所在的网站根目录. <strong>acme.sh</strong> 会全自动的生成验证文件, 并放到网站的根目录, 然后自动完成验证. 最后会聪明的删除验证文件. 整个过程没有任何副作用。如果是nginx，可以指定 <code>acme.sh --issue  -d mydomain.com   --nginx nginx.conf</code> 就可以了，从nginx配置里面自动完验证。</p><h3 id="DNS"><a href="#DNS" class="headerlink" title="DNS"></a>DNS</h3><p>支持手工和自动的方法，推荐使用自动的方法。以dnspod为例，先申请API。在安全设置，申请API Token。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">申请API之后继续执行申请，acme脚本会使用添加的API自动在DNS服务商处对域名添加TXT记录</span><br><span class="line"></span><br><span class="line">export DP_Id=&quot;申请的API ID&quot;</span><br><span class="line">export DP_Key=&quot;申请的API Key&quot;</span><br><span class="line"> ~/.acme.sh/acme.sh  --issue  --dns dns_dp  -d *.wumingx.com</span><br></pre></td></tr></table></figure><p>以下是操作过程：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos ~]<span class="comment"># export DP_Id="YOU ID"</span></span><br><span class="line">[root@VM_0_6_centos ~]<span class="comment"># export DP_Key="YOU KEY"</span></span><br><span class="line">[root@VM_0_6_centos ~]<span class="comment"># acme.sh   --issue   --dns dns_dp -d *.wumingx.com -d wumingx.com</span></span><br><span class="line">[Mon Sep 14 22:51:19 CST 2020] Using CA: https://acme-v02.api.letsencrypt.org/directory</span><br><span class="line">[Mon Sep 14 22:51:19 CST 2020] Multi domain=<span class="string">'DNS:*.wumingx.com,DNS:wumingx.com'</span></span><br><span class="line">[Mon Sep 14 22:51:19 CST 2020] Getting domain auth token <span class="keyword">for</span> each domain</span><br><span class="line">[Mon Sep 14 22:51:28 CST 2020] Getting webroot <span class="keyword">for</span> domain=<span class="string">'*.wumingx.com'</span></span><br><span class="line">[Mon Sep 14 22:51:28 CST 2020] Getting webroot <span class="keyword">for</span> domain=<span class="string">'wumingx.com'</span></span><br><span class="line">[Mon Sep 14 22:51:28 CST 2020] Adding txt value: mPC7BuynLkelvBKO2gDIAVkzwexIpXghxKFAGxml_kg <span class="keyword">for</span> domain: _acme-challenge.wumingx.com</span><br><span class="line">[Mon Sep 14 22:51:29 CST 2020] Adding record</span><br><span class="line">[Mon Sep 14 22:51:29 CST 2020] The txt record is added: Success.</span><br><span class="line">[Mon Sep 14 22:51:29 CST 2020] Adding txt value: Vp-lFa0CzR3eIqCbZjgulwN-KG3vCUdYSjmNp4C_aU4 <span class="keyword">for</span> domain: _acme-challenge.wumingx.com</span><br><span class="line">[Mon Sep 14 22:51:29 CST 2020] Adding record</span><br><span class="line">[Mon Sep 14 22:51:30 CST 2020] The txt record is added: Success.</span><br><span class="line">[Mon Sep 14 22:51:30 CST 2020] Let<span class="string">'s check each DNS record now. Sleep 20 seconds first.</span></span><br><span class="line"><span class="string">[Mon Sep 14 22:51:51 CST 2020] Checking wumingx.com for _acme-challenge.wumingx.com</span></span><br><span class="line"><span class="string">[Mon Sep 14 22:51:55 CST 2020] Domain wumingx.com '</span>_acme-challenge.wumingx.com<span class="string">' success.</span></span><br><span class="line"><span class="string">[Mon Sep 14 22:51:55 CST 2020] Checking wumingx.com for _acme-challenge.wumingx.com</span></span><br><span class="line"><span class="string">[Mon Sep 14 22:51:58 CST 2020] Domain wumingx.com '</span>_acme-challenge.wumingx.com<span class="string">' success.</span></span><br><span class="line"><span class="string">[Mon Sep 14 22:51:58 CST 2020] All success, let'</span>s <span class="built_in">return</span></span><br><span class="line">[Mon Sep 14 22:51:58 CST 2020] Verifying: *.wumingx.com</span><br><span class="line">[Mon Sep 14 22:52:08 CST 2020] Success</span><br><span class="line">[Mon Sep 14 22:52:08 CST 2020] Verifying: wumingx.com</span><br><span class="line">[Mon Sep 14 22:52:14 CST 2020] Success</span><br><span class="line">[Mon Sep 14 22:52:14 CST 2020] Removing DNS records.</span><br><span class="line">[Mon Sep 14 22:52:14 CST 2020] Removing txt: mPC7BuynLkelvBKO2gDIAVkzwexIpXghxKFAGxml_kg <span class="keyword">for</span> domain: _acme-challenge.wumingx.com</span><br><span class="line">[Mon Sep 14 22:52:15 CST 2020] Removed: Success</span><br><span class="line">[Mon Sep 14 22:52:15 CST 2020] Removing txt: Vp-lFa0CzR3eIqCbZjgulwN-KG3vCUdYSjmNp4C_aU4 <span class="keyword">for</span> domain: _acme-challenge.wumingx.com</span><br><span class="line">[Mon Sep 14 22:52:16 CST 2020] Removed: Success</span><br><span class="line">[Mon Sep 14 22:52:16 CST 2020] Verify finished, start to sign.</span><br><span class="line">[Mon Sep 14 22:52:16 CST 2020] Lets finalize the order.</span><br><span class="line">[Mon Sep 14 22:52:16 CST 2020] Le_OrderFinalize=<span class="string">'https://acme-v02.api.letsencrypt.org/acme/finalize/96628938/5183873632'</span></span><br><span class="line">[Mon Sep 14 22:52:21 CST 2020] Downloading cert.</span><br><span class="line">[Mon Sep 14 22:52:21 CST 2020] Le_LinkCert=<span class="string">'https://acme-v02.api.letsencrypt.org/acme/cert/0406e69631e78d603c65f61d1527169be11c'</span></span><br><span class="line">[Mon Sep 14 22:52:22 CST 2020] Cert success.</span><br><span class="line">-----BEGIN CERTIFICATE-----</span><br><span class="line">....</span><br><span class="line">-----END CERTIFICATE-----</span><br><span class="line">[Mon Sep 14 22:52:22 CST 2020] Your cert is <span class="keyword">in</span>  /root/.acme.sh/*.wumingx.com/*.wumingx.com.cer</span><br><span class="line">[Mon Sep 14 22:52:22 CST 2020] Your cert key is <span class="keyword">in</span>  /root/.acme.sh/*.wumingx.com/*.wumingx.com.key</span><br><span class="line">[Mon Sep 14 22:52:22 CST 2020] The intermediate CA cert is <span class="keyword">in</span>  /root/.acme.sh/*.wumingx.com/ca.cer</span><br><span class="line">[Mon Sep 14 22:52:22 CST 2020] And the full chain certs is there:  /root/.acme.sh/*.wumingx.com/fullchain.cer</span><br></pre></td></tr></table></figure><p>完成之后，dnspod的信息会保存在<code>/root/.acme.sh/account.conf</code>，下次就不需要输入了。</p><p>有关续约，有如下方法：</p><ul><li><p>手工方法：使用<code>acme.sh --renew --force -d *.wumingx.com</code> 就可以强制更新证书</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">[root@VM_0_6_centos .acme.sh]<span class="comment"># acme.sh --renew -d *.wumingx.com --force</span></span><br><span class="line">[Wed Sep 16 22:15:34 CST 2020] Renew: <span class="string">'*.wumingx.com'</span></span><br><span class="line">[Wed Sep 16 22:15:36 CST 2020] Using CA: https://acme-v02.api.letsencrypt.org/directory</span><br><span class="line">[Wed Sep 16 22:15:36 CST 2020] Signing from existing CSR.</span><br><span class="line">[Wed Sep 16 22:15:36 CST 2020] Getting domain auth token <span class="keyword">for</span> each domain</span><br><span class="line">[Wed Sep 16 22:15:41 CST 2020] Getting webroot <span class="keyword">for</span> domain=<span class="string">'*.wumingx.com'</span></span><br><span class="line">[Wed Sep 16 22:15:41 CST 2020] Getting webroot <span class="keyword">for</span> domain=<span class="string">'wumingx.com'</span></span><br><span class="line">[Wed Sep 16 22:15:42 CST 2020] *.wumingx.com is already verified, skip dns-01.</span><br><span class="line">[Wed Sep 16 22:15:42 CST 2020] wumingx.com is already verified, skip dns-01.</span><br><span class="line">[Wed Sep 16 22:15:42 CST 2020] Verify finished, start to sign.</span><br><span class="line">[Wed Sep 16 22:15:42 CST 2020] Lets finalize the order.</span><br><span class="line">[Wed Sep 16 22:15:42 CST 2020] Le_OrderFinalize=<span class="string">'https://acme-v02.api.letsencrypt.org/acme/finalize/96628938/5216400077'</span></span><br><span class="line">[Wed Sep 16 22:15:44 CST 2020] Downloading cert.</span><br><span class="line">[Wed Sep 16 22:15:44 CST 2020] Le_LinkCert=<span class="string">'https://acme-v02.api.letsencrypt.org/acme/cert/042c57fcab6c76af999a89ac1220a379cd33'</span></span><br><span class="line">[Wed Sep 16 22:15:46 CST 2020] Cert success.</span><br><span class="line">-----BEGIN CERTIFICATE-----</span><br><span class="line">...</span><br><span class="line">-----END CERTIFICATE-----</span><br><span class="line">[Wed Sep 16 22:15:46 CST 2020] Your cert is <span class="keyword">in</span>  /root/.acme.sh/*.wumingx.com/*.wumingx.com.cer </span><br><span class="line">[Wed Sep 16 22:15:46 CST 2020] The intermediate CA cert is <span class="keyword">in</span>  /root/.acme.sh/*.wumingx.com/ca.cer </span><br><span class="line">[Wed Sep 16 22:15:46 CST 2020] And the full chain certs is there:  /root/.acme.sh/*.wumingx.com/fullchain.cer</span><br></pre></td></tr></table></figure></li><li><p>定时方法：申请后acme.sh脚本会自动添加一条crontab定时任务，以便脚本自动续期，如果没有自动续期，可以执行以下命令 <code>32 0 * * * &quot;/root/.acme.sh&quot;/acme.sh --cron --home &quot;/root/.acme.sh&quot; &gt; /dev/null</code>。</p></li></ul><p>之后再配合一些简单的脚本就可以实现https通用证书的部署了。</p><h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><blockquote><p><a href="https://ningyu1.github.io/site/post/51-ssl-cert/" target="_blank" rel="noopener">Openssl生成自签名证书，简单步骤</a></p><p><a href="https://blog.csdn.net/cowbin2012/article/details/100134114" target="_blank" rel="noopener">openssl生成CA证书</a></p><p><a href="https://www.cnblogs.com/tugenhua0707/p/10927722.html" target="_blank" rel="noopener">使用openssl 生成免费证书</a></p><p><a href="https://github.com/acmesh-official/acme.sh/wiki/%E8%AF%B4%E6%98%8E" target="_blank" rel="noopener">acmd.sh官方说明</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;证书简介&quot;&gt;&lt;a href=&quot;#证书简介&quot; class=&quot;headerlink&quot; title=&quot;证书简介&quot;&gt;&lt;/a&gt;证书简介&lt;/h1&gt;&lt;p&gt;有关https是如何保证数据的安全证的，可以参考文章：&lt;a href=&quot;/tcpip/https-theory.html&quot;&gt;详解https加密通信原理&lt;/a&gt; ，以下简介证书相关的知识点。&lt;/p&gt;
&lt;p&gt;我们一般在服务器上面可以使用openssl命令来生成证书的.而一般是使用X509的证书链。x509证书一般会用到三类文件，&lt;code&gt;key，csr，crt&lt;/code&gt;。&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;p&gt;key是私用密钥，openssl格式，通常是rsa算法&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;csr是证书请求文件，用于申请证书。在申请的时候，必须使用自己的私钥来签署申请，还可以设定一个密钥。&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;crt是证书文件（windows下面的csr，其实是crt），是签署人用自己的key给你签署的凭证。&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="抓包分析" scheme="https://www.wumingx.com/categories/tcpip/"/>
    
    
      <category term="openssl" scheme="https://www.wumingx.com/tags/openssl/"/>
    
      <category term="acme.sh" scheme="https://www.wumingx.com/tags/acme-sh/"/>
    
  </entry>
  
  <entry>
    <title>实战CentOS7安装且使用KVM虚拟机</title>
    <link href="https://www.wumingx.com/linux/centos-kvm.html"/>
    <id>https://www.wumingx.com/linux/centos-kvm.html</id>
    <published>2020-08-22T08:13:12.000Z</published>
    <updated>2023-10-16T08:25:44.421Z</updated>
    
    <content type="html"><![CDATA[<h1 id="镜像格式"><a href="#镜像格式" class="headerlink" title="镜像格式"></a>镜像格式</h1><h2 id="介绍"><a href="#介绍" class="headerlink" title="介绍"></a>介绍</h2><ul><li>RAW格式是直接给云服务器进行读写的文件。RAW不支持动态增长空间，是镜像中I/O性能最好的一种格式。</li><li><p>QCOW2格式镜像是QEMU模拟器支持的一种磁盘镜像，是用一个文件的形式来表示一块固定大小的块设备磁盘。与普通的RAW格式镜像相比，QCOW2格式有如下几个特性：</p><ul><li>支持更小的磁盘占用。</li><li>支持写时拷贝（CoW，Copy-On-Write），镜像文件只反映底层磁盘变化。</li><li>支持快照，可以包含多个历史快照。</li><li>支持压缩和加密，可以选择ZLIB压缩和AES加密。</li></ul></li><li><p>VMDK是VMware创建的虚拟硬盘格式。一个VMDK文件代表VMFS（云服务器文件系统）在云服务器上的一个物理硬盘驱动。</p></li><li>VHD是微软提供的一种虚拟硬盘文件格式。VHD文件格式可以被压缩成单个文件存放到宿主机的文件系统上，主要包括云服务器启动所需的文件系统。微软在Windows Server 2012中的Hyper-V引入的一个新版本的VHD格式，称为VHDX。与VHD格式相比，VHDX具有更大的存储容量。它在电源故障期间提供数据损坏保护，并且优化了磁盘结构对齐方式，以防止新的大扇区物理磁盘性能降级。</li><li>vdi（Virtual Disk Image）是Oracle的VirtualBox虚拟机中的存储格式</li></ul><a id="more"></a><h2 id="格式转化"><a href="#格式转化" class="headerlink" title="格式转化"></a>格式转化</h2><p>在日常使用镜像的过程中，您可能需要将某格式的镜像转换为其他格式，qemu-img工具支持vhd、vmdk、qcow2、raw、vhdx、qcow、vdi或qed格式的镜像之间相互转换。在centos上，直接使用<code>yum install qemu-img</code>即可。</p><h3 id="基本语法"><a href="#基本语法" class="headerlink" title="基本语法"></a>基本语法</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">qemu-img command [command options]</span><br></pre></td></tr></table></figure><h3 id="主要参数"><a href="#主要参数" class="headerlink" title="主要参数"></a>主要参数</h3><ul><li>info ： 查看镜像的信息；</li><li>create： 创建镜像；</li><li>check： 检查镜像；</li><li>convert： 转化镜像的格式；</li><li>snapshot ：管理镜像的快照；</li><li>rebase： 在已有的镜像的基础上创建新的镜像；</li><li>resize： 增加或减小镜像大小</li></ul><h3 id="创建镜像-create"><a href="#创建镜像-create" class="headerlink" title="创建镜像, create"></a>创建镜像, create</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img create -f raw -o size=5G /data/raw/raw-disk01.raw  </span><br><span class="line">Formatting '/data/raw/raw-disk01.raw', fmt=raw size=5368709120</span><br></pre></td></tr></table></figure><ul><li>-f fmt: 执行格式</li><li>-o options: list of format specific options in a name1=value1,name2=value2… format. size=5G, 5G大小</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu cache2]<span class="comment"># qemu-img create -f raw -o size=1G t1.raw</span></span><br><span class="line">Formatting <span class="string">'t1.raw'</span>, fmt=raw size=1073741824 </span><br><span class="line">[root@ubuntu cache2]<span class="comment"># qemu-img create -f qcow2 -o size=1G t2.raw</span></span><br><span class="line">Formatting <span class="string">'t2.raw'</span>, fmt=qcow2 size=1073741824 encryption=off cluster_size=65536 lazy_refcounts=off </span><br><span class="line">[root@ubuntu cache2]<span class="comment"># ll -h t1.raw t2.raw </span></span><br><span class="line">-rw-r--r-- 1 root root 1.0G Aug 22 23:30 t1.raw</span><br><span class="line">-rw-r--r-- 1 root root 193K Aug 22 23:31 t2.raw</span><br></pre></td></tr></table></figure><p>从显示的文件大小上面是可以看出区别的。</p><h3 id="查看镜像的信息-info"><a href="#查看镜像的信息-info" class="headerlink" title="查看镜像的信息, info"></a>查看镜像的信息, info</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img info /data/raw/raw-disk01.raw  </span><br><span class="line">image: /data/raw/raw-disk01.raw</span><br><span class="line">file format: raw</span><br><span class="line">virtual size: 5.0G (5368709120 bytes)</span><br><span class="line">disk size: 0</span><br></pre></td></tr></table></figure><h3 id="镜像格式转换-convert"><a href="#镜像格式转换-convert" class="headerlink" title="镜像格式转换, convert"></a>镜像格式转换, convert</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img convert -f raw -O qcow2 /data/raw/raw-disk01.raw /data/raw/qcow2-disk01.qcow2</span><br><span class="line">[root@ubuntu ~]# qemu-img  info /data/raw/qcow2-disk01.qcow2</span><br><span class="line">image: /data/raw/qcow2-disk01.qcow2</span><br><span class="line">file format: qcow2</span><br><span class="line">virtual size: 5.0G (5368709120 bytes)</span><br><span class="line">disk size: 196K</span><br><span class="line">cluster_size: 65536</span><br><span class="line">Format specific information:</span><br><span class="line">    compat: 1.1</span><br><span class="line">    lazy refcounts: false</span><br></pre></td></tr></table></figure><ul><li>-f: 指定原镜像的格式，会自动检查可以省略</li><li>-O: 指定目标镜像格式</li></ul><h3 id="镜像快照-snapshot"><a href="#镜像快照-snapshot" class="headerlink" title="镜像快照, snapshot"></a>镜像快照, snapshot</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img snapshot /data/raw/qcow2-disk01.qcow2 -c /data/raw/qcow2-disk01.qcow2.snapshot </span><br><span class="line">[root@ubuntu ~]# qemu-img snapshot /data/raw/qcow2-disk01.qcow2 -l</span><br><span class="line">Snapshot list:</span><br><span class="line">ID        TAG                 VM SIZE                DATE       VM CLOCK</span><br><span class="line">1         /data/raw/qcow2-disk01.qcow2.snapshot      0 2019-06-24 20:42:51   00:00:00.000</span><br></pre></td></tr></table></figure><ul><li>-c: 创建快照</li><li>-l: 列出快照</li><li>-d: 删除快照<br>*ps: raw不支持快照，只有qcow2支持快照</li></ul><h3 id="调整镜像大小-resize"><a href="#调整镜像大小-resize" class="headerlink" title="调整镜像大小, resize"></a>调整镜像大小, resize</h3><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img resize /data/raw/disk02.raw +2G</span><br><span class="line">Image resized.</span><br><span class="line">[root@ubuntu ~]# qemu-img info /data/raw/disk02.raw              </span><br><span class="line">image: /data/raw/disk02.raw</span><br><span class="line">file format: raw</span><br><span class="line">virtual size: 7.0G (7516192768 bytes)</span><br><span class="line">disk size: 0</span><br></pre></td></tr></table></figure><p>ps: raw格式镜像大小都可以调整，qcow2格式镜像只能增大，不能缩小</p><h3 id="检查镜像，check"><a href="#检查镜像，check" class="headerlink" title="检查镜像，check"></a>检查镜像，check</h3><p>对磁盘镜像文件进行一致性检查，查找镜像文件中的错误，目前仅支持对“qcow2”、“qed”、“vdi”格式文件的检查。qcow2是QEMU 0.8.3版本引入的镜像文件格式，也是目前使用最广泛的格式。qed（QEMU enhanced disk）是从QEMU 0.14版开始加入的增强磁盘文件格式，为了避免qcow2格式的一些缺点，也为了提高性能，不过目前还不够成熟</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@ubuntu ~]# qemu-img check -f qcow2 /data/raw/qcow2-disk01.qcow2   </span><br><span class="line">No errors were found on the image.</span><br><span class="line">Image end offset: 393216</span><br></pre></td></tr></table></figure><ul><li>-f fmt: 指定文件的格式，如果不指定格式qemu-img会自动检测</li></ul><h1 id="安装KVM环境"><a href="#安装KVM环境" class="headerlink" title="安装KVM环境"></a>安装KVM环境</h1><h2 id="检测是否支持KVM"><a href="#检测是否支持KVM" class="headerlink" title="检测是否支持KVM"></a>检测是否支持KVM</h2><p>KVM 是基于 x86 虚拟化扩展(Intel VT 或者 AMD-V) 技术的虚拟机软件，所以查看 CPU 是否支持 VT 技术，就可以判断是否支持KVM。有返回结果，如果结果中有vmx（Intel）或svm(AMD)字样，就说明CPU的支持的。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">cat /proc/cpuinfo | egrep <span class="string">'vmx|svm'</span></span><br><span class="line"></span><br><span class="line">flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm arat epb pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm xsaveopt cqm_llc cqm_occup_llc</span><br></pre></td></tr></table></figure><p>关闭SELinux，将 /etc/sysconfig/selinux 中的 <code>SELinux=enforcing</code> 修改为 <code>SELinux=disabled</code>。</p><h2 id="安装方法"><a href="#安装方法" class="headerlink" title="安装方法"></a>安装方法</h2><p>通过yum就可以安装kvm基础包和管理工具了。kvm相关安装包及其作用:</p><ul><li><code>qemu-kvm</code> 主要的KVM程序包</li><li><code>python-virtinst</code> 创建虚拟机所需要的命令行工具和程序库</li><li><code>virt-manager</code> GUI虚拟机管理工具</li><li><code>virt-top</code> 虚拟机统计命令</li><li><code>virt-viewer</code> GUI连接程序，连接到已配置好的虚拟机</li><li><code>libvirt</code> C语言工具包，提供libvirt服务</li><li><code>libvirt-client</code> 为虚拟客户机提供的C语言工具包</li><li><code>virt-install</code> 基于libvirt服务的虚拟机创建命令</li><li><code>bridge-utils</code> 创建和管理桥接设备的工具</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 安装 kvm </span></span><br><span class="line"><span class="comment"># ------------------------</span></span><br><span class="line"><span class="comment"># yum -y install qemu-kvm python-virtinst libvirt libvirt-python virt-manager libguestfs-tools bridge-utils virt-install</span></span><br><span class="line"></span><br><span class="line">yum -y install qemu-kvm libvirt virt-install bridge-utils </span><br><span class="line"></span><br><span class="line"><span class="comment"># 重启宿主机，以便加载 kvm 模块</span></span><br><span class="line"><span class="comment"># ------------------------</span></span><br><span class="line">reboot</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看KVM模块是否被正确加载</span></span><br><span class="line"><span class="comment"># ------------------------</span></span><br><span class="line">lsmod | grep kvm</span><br><span class="line"></span><br><span class="line">kvm_intel             162153  0</span><br><span class="line">kvm                   525259  1 kvm_intel</span><br></pre></td></tr></table></figure><p>开启kvm服务，并且设置其开机自动启动</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">systemctl start libvirtd</span><br><span class="line">systemctl <span class="built_in">enable</span> libvirtd</span><br></pre></td></tr></table></figure><p>查看状态操作结果，如<code>Active: active (running)</code>，说明运行情况良好</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">systemctl status libvirtd</span><br><span class="line">systemctl is-enabled libvirtd</span><br><span class="line"></span><br><span class="line">● libvirtd.service - Virtualization daemon</span><br><span class="line">   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)</span><br><span class="line">   Active: active (running) since 二 2001-01-02 11:29:53 CST; 1h 41min ago</span><br><span class="line">     Docs: man:libvirtd(8)</span><br><span class="line">           http://libvirt.org</span><br></pre></td></tr></table></figure><h2 id="安装虚拟机"><a href="#安装虚拟机" class="headerlink" title="安装虚拟机"></a>安装虚拟机</h2><p>有2种方式，经过个人测试，建议使用VNC来创建，兼容性更好。</p><h3 id="通过VNC来远程"><a href="#通过VNC来远程" class="headerlink" title="通过VNC来远程"></a>通过VNC来远程</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 先创建一个磁盘</span></span><br><span class="line">[root@localhost cache2]<span class="comment"># qemu-img create -f qcow2 test.qcow2 10G</span></span><br><span class="line">Formatting <span class="string">'test.qcow2'</span>, fmt=qcow2 size=10737418240 encryption=off cluster_size=65536 lazy_refcounts=off</span><br><span class="line"><span class="comment"># 查看磁盘信息</span></span><br><span class="line">[root@localhost cache2]<span class="comment"># qemu-img info test.qcow2</span></span><br><span class="line">image: test.qcow2</span><br><span class="line">file format: qcow2</span><br><span class="line">virtual size: 10G (10737418240 bytes)</span><br><span class="line">disk size: 196K</span><br><span class="line">cluster_size: 65536</span><br><span class="line">Format specific information:</span><br><span class="line">    compat: 1.1</span><br><span class="line">    lazy refcounts: <span class="literal">false</span></span><br><span class="line"><span class="comment"># 创建虚拟机，最好是使用qemu-img create先创建好qcow2文件</span></span><br><span class="line">[root@localhost cache2]$ virt-install --name centOS78 --virt-type kvm --ram 8192 --vcpus=4 --cdrom=/tmp/CentOS-7-x86_64-Minimal-2003.iso --disk path=/cache2/centos78.qcow2,size=40,format=qcow2 --network=bridge:br0 --graphics vnc,listen=0.0.0.0,port=5962,password=xxxxx --noautoconsole</span><br><span class="line"></span><br><span class="line">Starting install...</span><br><span class="line">Allocating <span class="string">'centos78.qcow2'</span>                                                                                                  |  40 GB  00:00:00</span><br><span class="line">Domain installation still <span class="keyword">in</span> progress. You can reconnect to</span><br><span class="line">the console to complete the installation process.</span><br></pre></td></tr></table></figure><p>通过 <a href="https://www.realvnc.com/en/connect/download/viewer/" target="_blank" rel="noopener">https://www.realvnc.com/en/connect/download/viewer/</a> 下载这个工具就可以通过VNC来远程了。运行完成之后，centos7的系统可以使用 <code>grubby --update-kernel=ALL --args=&quot;console=ttyS0&quot;</code>来更新下，这时就可以实现console了。</p><h3 id="通过重定向来远程"><a href="#通过重定向来远程" class="headerlink" title="通过重定向来远程"></a>通过重定向来远程</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">virt-install --virt-type=kvm --name=centos88 --vcpus=2 --memory=4096 --location=/tmp/CentOS-7-x86_64-Minimal-2003.iso --disk path=/cache2/centos88.qcow2,size=40,format=qcow2 --network bridge=br0 --graphics none --extra-args=<span class="string">'console=ttyS0'</span> --force</span><br></pre></td></tr></table></figure><p>创建成功之后，可以通过<code>virsh console centos88</code>来远程上机器操作了。</p><h1 id="添加硬件"><a href="#添加硬件" class="headerlink" title="添加硬件"></a>添加硬件</h1><h2 id="磁盘相关"><a href="#磁盘相关" class="headerlink" title="磁盘相关"></a>磁盘相关</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="comment"># 加磁盘</span></span><br><span class="line">[root@12038849 ~]<span class="comment"># virsh attach-disk slone /cache2/slone/data.qcow2 hdc --subdriver qcow2 --persistent </span></span><br><span class="line">Disk attached successfully</span><br><span class="line"><span class="comment"># 查看磁盘</span></span><br><span class="line">[root@12038849 slone]<span class="comment"># virsh domblklist slone </span></span><br><span class="line">Target     Source</span><br><span class="line">------------------------------------------------</span><br><span class="line">hda        /cache2/slone/system.qcow2</span><br><span class="line">hdb        -</span><br><span class="line">sdb        /cache2/slone/data.qcow2</span><br><span class="line"></span><br><span class="line"><span class="comment"># 卸载磁盘</span></span><br><span class="line">[root@12038849 slone]<span class="comment"># virsh detach-disk slone /cache2/slone/data.qcow2 </span></span><br><span class="line">Disk detached successfully</span><br></pre></td></tr></table></figure><blockquote><p><a href="https://opengers.github.io/virtualization/kvm-online-add-device/" target="_blank" rel="noopener">https://opengers.github.io/virtualization/kvm-online-add-device/</a></p></blockquote><h2 id="快照"><a href="#快照" class="headerlink" title="快照"></a>快照</h2><p>常用使用方法：</p><ul><li><p>创建默认快照（一般为一串数字）：<code>virsh snapshot-create 虚拟机名称</code></p></li><li><p>创建自定义名称快照：<code>virsh snapshot-create-as 虚拟机名称 快照名称</code></p></li><li>查看虚拟机相关的快照：<code>virsh snapshot-list 虚拟机名</code></li><li>删除快照：<code>virsh snapshot-delete 虚拟机名 快照名称</code></li><li>恢复快照：<code>virsh snapshot-revert 虚拟机名称 快照名称</code></li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[root@localhost Plcm-SoftMcuMain]<span class="comment"># virsh snapshot-list --domain centos88</span></span><br><span class="line"> Name                 Creation Time             State</span><br><span class="line">------------------------------------------------------------</span><br><span class="line"></span><br><span class="line">[root@localhost Plcm-SoftMcuMain]<span class="comment">#</span></span><br><span class="line">[root@localhost Plcm-SoftMcuMain]<span class="comment"># virsh snapshot-create-as --domain centos88 centos78init</span></span><br><span class="line">Domain snapshot centos78init created</span><br><span class="line">[root@localhost Plcm-SoftMcuMain]<span class="comment"># virsh snapshot-list --domain centos88</span></span><br><span class="line"> Name                 Creation Time             State</span><br><span class="line">------------------------------------------------------------</span><br><span class="line"> centos78init         2020-07-28 11:31:58 +0800 running</span><br></pre></td></tr></table></figure><h2 id="克隆虚拟机"><a href="#克隆虚拟机" class="headerlink" title="克隆虚拟机"></a>克隆虚拟机</h2><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 暂停原始虚拟机</span></span><br><span class="line">virsh shutdown centos72</span><br><span class="line">virt-clone -o centos72 -n centos.112 -f /home/vms/centos.112.qcow2 -m 00:00:00:00:00:01</span><br><span class="line">virt-clone -o centos88 -n centos.112 --file /home/vms/centos.112.qcow2 --nonsparse</span><br><span class="line"></span><br><span class="line"><span class="comment"># 实例</span></span><br><span class="line">[root@12038849 template]<span class="comment"># virt-clone -o slone -n slthree -f /cache2/slthree/system.qcow2 -f /cache2/slthree/data.qcow2</span></span><br><span class="line">Allocating <span class="string">'system.qcow2'</span>                                                                                                                                                                  | 200 GB  00:00:12     </span><br><span class="line">Allocating <span class="string">'data.qcow2'</span>                                                                                                                                                                    | 1.0 TB  00:00:00     </span><br><span class="line">Clone <span class="string">'slthree'</span> created successfully.</span><br></pre></td></tr></table></figure><p><code>virt-clone</code> 参数介绍</p><ul><li><code>--version</code> 查看版本。</li><li><code>-h，--help</code> 查看帮助信息。</li><li><code>--connect=URI</code> 连接到虚拟机管理程序 libvirt 的URI。</li><li><code>-o 原始虚拟机名称</code> 原始虚拟机名称，必须为关闭或者暂停状态。</li><li><code>-n 新虚拟机名称</code> —name 新虚拟机名称。</li><li><code>--auto-clone</code> 从原来的虚拟机配置自动生成克隆名称和存储路径。</li><li><code>-u NEW_UUID, --uuid=NEW_UUID</code> 克隆虚拟机的新的UUID，默认值是一个随机生成的UUID。</li><li><code>-m NEW_MAC, --mac=NEW_MAC</code> 设置一个新的mac地址，默认为随机生成 MAC。</li><li><code>-f NEW_DISKFILE, --file=NEW_DISKFILE</code> 为新客户机使用新的磁盘镜像文件地址。</li><li><code>--force-copy=TARGET</code> 强制复制设备。</li><li><code>--nonsparse</code> 不使用稀疏文件复制磁盘映像。</li></ul><h2 id="虚拟机迁移"><a href="#虚拟机迁移" class="headerlink" title="虚拟机迁移"></a>虚拟机迁移</h2><p>需求：hids01需要迁移一台虚拟机到hids02、hids03的机器上。</p><p>先将以下文件scp到其他的机器上：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[root@hids01 hids]<span class="comment"># virsh domblklist hids </span></span><br><span class="line">Target     Source</span><br><span class="line">------------------------------------------------</span><br><span class="line">hda        /cache1/hids/system.qcow2</span><br><span class="line">hdb        /cache1/hids/data.qcow2</span><br><span class="line">hdc        -</span><br><span class="line"></span><br><span class="line">[root@hids01 hids]<span class="comment"># virsh dumpxml hids &gt;hids.xml</span></span><br></pre></td></tr></table></figure><p>在hids02、hids03的机器上面修改hids.xml文件，修改name、uuid、mac这三项即可</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">cat hids.xml |egrep &quot;&lt;name&gt;|mac address|uuid&quot;</span><br><span class="line"></span><br><span class="line">  &lt;name&gt;hids&lt;/name&gt;</span><br><span class="line">  &lt;uuid&gt;5a1a650a-7ef4-47ae-91fa-973f108b3142&lt;/uuid&gt;</span><br><span class="line">      &lt;mac address=&apos;52:54:00:84:46:41&apos;/&gt;</span><br></pre></td></tr></table></figure><ul><li>mac地址的生成方法：<code>for i in {1..6}; do printf &quot;%0.2X:&quot; $[ $RANDOM % 0x100 ]; done | sed &#39;s/:$/\n/&#39;</code></li><li>uuid的生成方法：<code>uuidgen</code></li></ul><p>接下来按以下操作：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">[root@hids02 hids]<span class="comment"># virsh define --file hids.xml</span></span><br><span class="line">Domain hids defined from hids.xml</span><br><span class="line"></span><br><span class="line">[root@hids02 hids]<span class="comment"># virsh edit hids </span></span><br><span class="line">Domain hids XML configuration edited.</span><br><span class="line"></span><br><span class="line">[root@hids02 hids]<span class="comment"># virsh list --all</span></span><br><span class="line"> Id    Name                           State</span><br><span class="line">----------------------------------------------------</span><br><span class="line"> -     hids                           shut off</span><br><span class="line"></span><br><span class="line">[root@hids02 hids]<span class="comment"># virsh start hids </span></span><br><span class="line">Domain hids started</span><br></pre></td></tr></table></figure><h2 id="动态更改cpu数量和内存大小"><a href="#动态更改cpu数量和内存大小" class="headerlink" title="动态更改cpu数量和内存大小"></a>动态更改cpu数量和内存大小</h2><p>动态调整，如果超过给虚拟机分配的最大内存，需要重启虚拟机。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">virsh list --all</span><br><span class="line"><span class="meta">#</span><span class="bash">  Id    名称                         状态</span></span><br><span class="line"><span class="meta">#</span><span class="bash"> ----------------------------------------------------</span></span><br><span class="line"><span class="meta">#</span><span class="bash">  2     working112                     running</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 更改CPU</span></span><br><span class="line">virsh setvcpus working112 --maximum 4 --config</span><br><span class="line"><span class="meta">#</span><span class="bash"> 更改内存</span></span><br><span class="line">virsh setmaxmem working112 1048576 --config</span><br><span class="line"><span class="meta">#</span><span class="bash"> 查看信息</span></span><br><span class="line">virsh dominfo working112</span><br></pre></td></tr></table></figure><h2 id="网络模式"><a href="#网络模式" class="headerlink" title="网络模式"></a>网络模式</h2><p>KVM 虚拟机默认是基于 NAT 的网络配置，只有同一宿主机的虚拟键之间可以互相访问，跨宿主机是不能访问；如果要实现跨宿主机可以访问，需要将网络模式配置成桥接模式。</p><h3 id="NAT模式"><a href="#NAT模式" class="headerlink" title="NAT模式"></a>NAT模式</h3><p>NAT(Network Address Translation网络地址翻译)，NAT方式是kvm安装后的默认方式。它支持主机与虚拟机的互访，同时也支持虚拟机访问互联网，但不支持外界访问虚拟机。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">virsh net-edit default <span class="comment"># 如果要创建或者修改NAT网络，要先编辑default.xml：</span></span><br><span class="line">&lt;network&gt;</span><br><span class="line">  &lt;name&gt;default&lt;/name&gt;</span><br><span class="line">  &lt;uuid&gt;ab22bbdf-9889-4fea-978c-67c177e586d6&lt;/uuid&gt;</span><br><span class="line">  &lt;forward mode=<span class="string">'nat'</span>/&gt;</span><br><span class="line">  &lt;bridge name=<span class="string">'virbr0'</span> stp=<span class="string">'on'</span> delay=<span class="string">'0'</span>/&gt;</span><br><span class="line">  &lt;mac address=<span class="string">'52:54:00:a4:0c:91'</span>/&gt;</span><br><span class="line">  &lt;ip address=<span class="string">'192.168.122.1'</span> netmask=<span class="string">'255.255.255.0'</span>&gt;</span><br><span class="line">    &lt;dhcp&gt;</span><br><span class="line">      &lt;range start=<span class="string">'192.168.122.2'</span> end=<span class="string">'192.168.122.254'</span>/&gt;</span><br><span class="line">    &lt;/dhcp&gt;</span><br><span class="line">  &lt;/ip&gt;</span><br><span class="line">&lt;/network&gt;</span><br><span class="line"></span><br><span class="line">virsh net-list --all</span><br><span class="line"></span><br><span class="line"> Name                 State      Autostart     Persistent</span><br><span class="line">----------------------------------------------------------</span><br><span class="line"> default              active     no            no</span><br></pre></td></tr></table></figure><p>其中virbr0是由宿主机虚拟机支持模块安装时产生的虚拟网络接口，也是一个switch和bridge，负责把内容分发到各虚拟机。几个虚拟机管理模块产生的接口关系如下图:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">┌───────────────────────┐                      </span><br><span class="line">│         HOST          │                      </span><br><span class="line">│ ┌──────┐              │   ┌─────────────────┐</span><br><span class="line">│ │ br0  │─┬──────┐     │   │Virtual Machine 1│</span><br><span class="line">│ └──────┘ │      │     │   │   ┌──────┐      │</span><br><span class="line">│     │    │  ┌───────┐ │ ─ │   │ br0  │      │</span><br><span class="line">│     │    │  │ vnet0 │─│┘  │   └──────┘      │</span><br><span class="line">│ ┌──────┐ │  └───────┘ │   └─────────────────┘</span><br><span class="line">│ │virbr0│ │  ┌───────┐ │   ┌─────────────────┐</span><br><span class="line">│ │ -nic │ └──│ vnet1 │─│┐  │Virtual Machine 2│</span><br><span class="line">│ └──────┘    └───────┘ │   │                 │</span><br><span class="line">│ ┌──────┐              │└ ─│   ┌──────┐      │</span><br><span class="line">│ │ eno0 │              │   │   │ br0  │      │</span><br><span class="line">│ └──────┘              │   │   └──────┘      │</span><br><span class="line">│ ┌──────┐              │   └─────────────────┘</span><br><span class="line">│ │ eno1 │              │</span><br><span class="line">│ └──────┘              │</span><br><span class="line">└───────────────────────┘</span><br></pre></td></tr></table></figure><p>从图上可以看出，虚拟接口和物理接口之间没有连接关系，所以虚拟机只能在通过虚拟的网络访问外部世界，无法从网络上定位和访问虚拟主机。</p><p>virbr0是一个桥接器，接收所有到网络192.168.122.*的内容。从下面命令可以验证：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">brctl show</span><br><span class="line"><span class="comment"># 输出结果</span></span><br><span class="line">bridge namebridge idSTP enabledinterfaces</span><br><span class="line">br08000.001631f44361noeth7</span><br><span class="line">vnet0</span><br><span class="line">virbr08000.525400a40c91yesvirbr0-nic</span><br><span class="line"></span><br><span class="line">ip route</span><br><span class="line"><span class="comment"># default via 192.168.188.1 dev br0</span></span><br><span class="line"><span class="comment"># 169.254.0.0/16 dev br0  scope link  metric 1012</span></span><br><span class="line"><span class="comment"># 192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1</span></span><br><span class="line"><span class="comment"># 192.168.188.0/24 dev br0  proto kernel  scope link  src 192.168.188.132</span></span><br></pre></td></tr></table></figure><p>同时，虚拟机支持模块会修改iptables规则，通过命令可以查看：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">iptables -t nat -L -nv</span><br><span class="line"><span class="comment"># 输出结果</span></span><br><span class="line">-A POSTROUTING -s 192.168.122.0/24 -d 224.0.0.0/24 -j RETURN</span><br><span class="line">-A POSTROUTING -s 192.168.122.0/24 -d 255.255.255.255/32 -j RETURN</span><br><span class="line">-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p tcp -j MASQUERADE --to-ports 1024-65535</span><br><span class="line">-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -p udp -j MASQUERADE --to-ports 1024-65535</span><br><span class="line">-A POSTROUTING -s 192.168.122.0/24 ! -d 192.168.122.0/24 -j MASQUERADE</span><br><span class="line"></span><br><span class="line">iptables -t filter -S |egrep <span class="string">"192.168|virbr0"</span></span><br><span class="line"><span class="comment"># 输出结果</span></span><br><span class="line">-A INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT</span><br><span class="line">-A INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT</span><br><span class="line">-A INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT</span><br><span class="line">-A INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT</span><br><span class="line">-A FORWARD -d 192.168.122.0/24 -o virbr0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT</span><br><span class="line">-A FORWARD -s 192.168.122.0/24 -i virbr0 -j ACCEPT</span><br><span class="line">-A FORWARD -i virbr0 -o virbr0 -j ACCEPT</span><br><span class="line">-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable</span><br><span class="line">-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable</span><br><span class="line">-A OUTPUT -o virbr0 -p udp -m udp --dport 68 -j ACCEPT</span><br></pre></td></tr></table></figure><p>如果没有default的话，或者需要扩展自己的虚拟网络，可以使用命令重新安装NAT。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">virsh net-define /usr/share/libvirt/networks/default.xml</span><br></pre></td></tr></table></figure><p>此命令定义一个虚拟网络，default.xml的内容：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">&lt;network&gt;</span><br><span class="line">  &lt;name&gt;default&lt;/name&gt;</span><br><span class="line">  &lt;bridge name=&quot;virbr0&quot; /&gt;</span><br><span class="line">  &lt;forward/&gt;</span><br><span class="line">  &lt;ip address=&quot;192.168.122.1&quot; netmask=&quot;255.255.255.0&quot;&gt;</span><br><span class="line">    &lt;dhcp&gt;</span><br><span class="line">      &lt;range start=&quot;192.168.122.2&quot; end=&quot;192.168.122.254&quot; /&gt;</span><br><span class="line">    &lt;/dhcp&gt;</span><br><span class="line">  &lt;/ip&gt;</span><br><span class="line">&lt;/network&gt;</span><br></pre></td></tr></table></figure><p>也可以修改xml，创建自己的虚拟网络。</p><p>重新加载和激活配置：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">virsh  net-define /etc/libvirt/qemu/networks/default.xml</span><br></pre></td></tr></table></figure><p>标记为自动启动：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">virsh net-autostart default</span><br><span class="line"># Network default marked as autostarted</span><br><span class="line"></span><br><span class="line">virsh net-start default</span><br></pre></td></tr></table></figure><p>启动网络：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">virsh net-start default</span><br><span class="line"># Network default started</span><br></pre></td></tr></table></figure><p>网络启动后可以用命令brctl show 查看和验证。</p><p>总结一下：如果想自定义一个网络，假设是需要创建名为<code>management</code>的NAT网络，<code>vi  /usr/share/libvirt/networks/management.xml</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">&lt;network&gt;</span><br><span class="line">  &lt;name&gt;management&lt;/name&gt;</span><br><span class="line">  &lt;bridge name=&quot;virbr1&quot;/&gt;</span><br><span class="line">  &lt;forward/&gt;</span><br><span class="line">  &lt;ip address=&quot;192.168.123.1&quot; netmask=&quot;255.255.255.0&quot;&gt;</span><br><span class="line">    &lt;dhcp&gt;</span><br><span class="line">      &lt;range start=&quot;192.168.123.2&quot; end=&quot;192.168.123.254&quot;/&gt;</span><br><span class="line">    &lt;/dhcp&gt;</span><br><span class="line">  &lt;/ip&gt;</span><br><span class="line">&lt;/network&gt;</span><br></pre></td></tr></table></figure><p>启用新建的NAT网络</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">virsh net-define /usr/share/libvirt/networks/management.xml</span><br><span class="line">virsh net-start management</span><br><span class="line">virsh net-autostart management</span><br></pre></td></tr></table></figure><h3 id="Bridge模式配置"><a href="#Bridge模式配置" class="headerlink" title="Bridge模式配置"></a>Bridge模式配置</h3><p>Bridge方式即虚拟网桥的网络连接方式，是客户机和子网里面的机器能够互相通信。可以使虚拟机成为网络中具有独立IP的主机。<strong>桥接网络</strong>（也叫 <strong>物理设备共享</strong>）被用作把一个物理设备复制到一台虚拟机。网桥多用作高级设置，特别是主机多个网络接口的情况。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">┌─────────────────────────┐      ┌─────────────────┐</span><br><span class="line">│          HOST           │      │Virtual Machine 1│</span><br><span class="line">│ ┌──────┐      ┌───────┐ │      │    ┌──────┐     │</span><br><span class="line">│ │ br0  │──┬───│ vnet0 │─│─ ─ ─ │    │ br0  │     │</span><br><span class="line">│ └──────┘  │   └───────┘ │      │    └──────┘     │</span><br><span class="line">│     │     │             │      └─────────────────┘</span><br><span class="line">│     │     │   ┌───────┐ │      ┌─────────────────┐</span><br><span class="line">│ ┌──────┐  └───│ vnet1 │─│─     │Virtual Machine 2│</span><br><span class="line">│ │ eno0 │      └───────┘ │ │    │    ┌──────┐     │</span><br><span class="line">│ └──────┘                │  ─ ─ │    │ br0  │     │</span><br><span class="line">│ ┌──────┐                │      │    └──────┘     │</span><br><span class="line">│ │ eno1 │                │      └─────────────────┘</span><br><span class="line">│ └──────┘                │</span><br><span class="line">└─────────────────────────┘</span><br></pre></td></tr></table></figure><p>通过<a href="https://jaywcjlove.github.io/linux-command/c/ip.html" target="_blank" rel="noopener">ip</a> 命令查看宿主机配置文件的名字</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">ip addr</span><br><span class="line"></span><br><span class="line">6: eno1: &lt;BROADCAST,MULTICAST,UP,LOWER_UP&gt; mtu 1500 qdisc mq state UP qlen 1000</span><br><span class="line">    link/ether 38:63:bb:44:cf:6c brd ff:ff:ff:ff:ff:ff</span><br><span class="line">    inet 192.168.188.132/24 brd 192.168.188.255 scope global dynamic eno1</span><br><span class="line">       valid_lft 2822sec preferred_lft 2822sec</span><br><span class="line">    inet6 fe80::3a63:bbff:fe44:cf6c/64 scope link</span><br><span class="line">       valid_lft forever preferred_lft forever</span><br></pre></td></tr></table></figure><p>可以看到上面<code>eno1</code>是有获取到ip地址的，相对应的文件在<code>/etc/sysconfig/network-scripts/</code>目录中，<code>ifcfg-eno1</code> 宿主机的物理网卡配置文件</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> cat ifcfg-eno1</span></span><br><span class="line">DEVICE=eno1</span><br><span class="line">TYPE="Ethernet"</span><br><span class="line">BOOTPROTO="static"</span><br><span class="line">ONBOOT="yes"</span><br><span class="line"></span><br><span class="line">BRIDGE="br0" # 指定桥接网卡的名称</span><br></pre></td></tr></table></figure><p><code>ifcfg-br0</code> 桥接网卡配置在同一个目录中。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> cat ifcfg-br0</span></span><br><span class="line">DEVICE="br0"</span><br><span class="line">TYPE="bridge"</span><br><span class="line">BOOTPROTO="static"</span><br><span class="line">ONBOOT="yes"</span><br><span class="line">TYPE=bridge  # 将制定为桥接类型</span><br><span class="line">IPADDR=192.168.188.133  # 设置IP地址</span><br><span class="line">PREFIX=24               # 设置子网掩码</span><br><span class="line">GATEWAY=192.168.188.1   # 设置网关</span><br></pre></td></tr></table></figure><p>配置好之后，通过<a href="https://jaywcjlove.github.io/linux-command/c/systemctl.html" target="_blank" rel="noopener">systemctl</a> 命令重启网卡。</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">ifup eno1 # 激活网卡</span><br><span class="line">ifup br0 # 激活桥接网卡</span><br><span class="line"><span class="meta">#</span><span class="bash"> 两种重启网络的方法</span></span><br><span class="line">systemctl restart network.service</span><br><span class="line">service network restart</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 校验桥接接口</span></span><br><span class="line">brctl show</span><br><span class="line"></span><br><span class="line">bridge name bridge id   STP enabled interfaces</span><br><span class="line">br0   8000.3863bb44cf6c no    eno1</span><br><span class="line">              vnet0</span><br><span class="line">virbr0    8000.525400193f0f yes   virbr0-nic</span><br></pre></td></tr></table></figure><h2 id="端口转发"><a href="#端口转发" class="headerlink" title="端口转发"></a>端口转发</h2><p>现在我们还以上述VM为例，目前该KVM的公网IP为<code>211.11.61.7</code>，VM的IP为<code>192.168.188.115</code>，现在我要求通过访问KVM的2222端口访问VM的22端口。</p><p>编辑<code>vi /etc/rc.d/rc.local</code> 添加下面命令，达到开机重启配置网络转发规则。</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"># 启动网络转发规则</span><br><span class="line">iptables -t nat -A : -s 192.168.188.0/24 -j SNAT --to-source 211.11.61.7</span><br><span class="line"></span><br><span class="line">iptables -t nat -A POSTROUTING -s 192.168.188.0/24 -j SNAT --to-source 211.11.61.7</span><br><span class="line">iptables -t nat -A PREROUTING -d  211.11.61.7 -p tcp --dport 2222  -j DNAT --to-dest 192.168.188.115:22</span><br><span class="line">iptables -t nat -A PREROUTING -d  211.11.61.7 -p tcp --dport 2221  -j DNAT --to-dest 192.168.188.115:21</span><br><span class="line"></span><br><span class="line"># 实际效果可以通过外网连接虚拟机</span><br><span class="line">ssh -p 2222 root@211.11.61.7</span><br></pre></td></tr></table></figure><h1 id="常用命令"><a href="#常用命令" class="headerlink" title="常用命令"></a>常用命令</h1><p>virsh常用</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">virsh list --all           <span class="comment"># 查看所有运行和没有运行的虚拟机</span></span><br><span class="line">virsh list                 <span class="comment"># 查看在运行的虚拟机</span></span><br><span class="line">virsh dumpxml vm-name      <span class="comment"># 查看kvm虚拟机配置文件</span></span><br><span class="line">virsh start vm-name        <span class="comment"># 启动kvm虚拟机</span></span><br><span class="line">virsh shutdown vm-name     <span class="comment"># 正常关机</span></span><br><span class="line"></span><br><span class="line">virsh destroy vm-name      <span class="comment"># 非正常关机，强制关闭虚拟机（相当于物理机直接拔掉电源）</span></span><br><span class="line">virsh undefine vm-name     <span class="comment"># 删除vm的配置文件</span></span><br><span class="line"></span><br><span class="line">ls /etc/libvirt/qemu</span><br><span class="line"><span class="comment"># 查看删除结果，Centos-6.6的配置文件被删除，但磁盘文件不会被删除</span></span><br><span class="line"></span><br><span class="line">virsh define file-name.xml <span class="comment"># 根据配置文件定义虚拟机</span></span><br><span class="line">virsh <span class="built_in">suspend</span> vm-name      <span class="comment"># 挂起，终止</span></span><br><span class="line">virsh resumed vm-name      <span class="comment"># 恢复被挂起的虚拟机</span></span><br><span class="line">virsh autostart vm-name    <span class="comment"># 开机自启动vm</span></span><br><span class="line">virsh console &lt;虚拟机名称&gt;   <span class="comment"># 连接虚拟机</span></span><br></pre></td></tr></table></figure><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p><a href="https://github.com/jaywcjlove/handbook/blob/master/docs/CentOS/CentOS7%E5%AE%89%E8%A3%85KVM%E8%99%9A%E6%8B%9F%E6%9C%BA%E8%AF%A6%E8%A7%A3.md" target="_blank" rel="noopener">CentOS7安装KVM虚拟机详解</a></p><p><a href="https://loveyu.org/5538.html" target="_blank" rel="noopener">https://loveyu.org/5538.html</a></p><p><a href="https://www.leolan.top/index.php/posts/134.html" target="_blank" rel="noopener">https://www.leolan.top/index.php/posts/134.html</a></p><p><a href="https://www.cnblogs.com/kevingrace/p/8377645.html" target="_blank" rel="noopener">Centos7.4安装kvm虚拟机（使用virt-manager管理）</a></p><p><a href="https://blog.csdn.net/linzhaolover/article/details/52209477" target="_blank" rel="noopener">https://blog.csdn.net/linzhaolover/article/details/52209477</a></p><p><a href="https://www.cnblogs.com/wshenjin/p/11079469.html" target="_blank" rel="noopener">KVM之磁盘管理工具qemu-img小结</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;镜像格式&quot;&gt;&lt;a href=&quot;#镜像格式&quot; class=&quot;headerlink&quot; title=&quot;镜像格式&quot;&gt;&lt;/a&gt;镜像格式&lt;/h1&gt;&lt;h2 id=&quot;介绍&quot;&gt;&lt;a href=&quot;#介绍&quot; class=&quot;headerlink&quot; title=&quot;介绍&quot;&gt;&lt;/a&gt;介绍&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;RAW格式是直接给云服务器进行读写的文件。RAW不支持动态增长空间，是镜像中I/O性能最好的一种格式。&lt;/li&gt;
&lt;li&gt;&lt;p&gt;QCOW2格式镜像是QEMU模拟器支持的一种磁盘镜像，是用一个文件的形式来表示一块固定大小的块设备磁盘。与普通的RAW格式镜像相比，QCOW2格式有如下几个特性：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;支持更小的磁盘占用。&lt;/li&gt;
&lt;li&gt;支持写时拷贝（CoW，Copy-On-Write），镜像文件只反映底层磁盘变化。&lt;/li&gt;
&lt;li&gt;支持快照，可以包含多个历史快照。&lt;/li&gt;
&lt;li&gt;支持压缩和加密，可以选择ZLIB压缩和AES加密。&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;VMDK是VMware创建的虚拟硬盘格式。一个VMDK文件代表VMFS（云服务器文件系统）在云服务器上的一个物理硬盘驱动。&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;VHD是微软提供的一种虚拟硬盘文件格式。VHD文件格式可以被压缩成单个文件存放到宿主机的文件系统上，主要包括云服务器启动所需的文件系统。微软在Windows Server 2012中的Hyper-V引入的一个新版本的VHD格式，称为VHDX。与VHD格式相比，VHDX具有更大的存储容量。它在电源故障期间提供数据损坏保护，并且优化了磁盘结构对齐方式，以防止新的大扇区物理磁盘性能降级。&lt;/li&gt;
&lt;li&gt;vdi（Virtual Disk Image）是Oracle的VirtualBox虚拟机中的存储格式&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="Linux基础" scheme="https://www.wumingx.com/categories/linux/"/>
    
    
      <category term="kvm" scheme="https://www.wumingx.com/tags/kvm/"/>
    
  </entry>
  
  <entry>
    <title>k8s入门教程十五:使用ceph实现storageclass动态存储</title>
    <link href="https://www.wumingx.com/k8s/kubernetes-ceph-storageclass.html"/>
    <id>https://www.wumingx.com/k8s/kubernetes-ceph-storageclass.html</id>
    <published>2020-06-10T11:58:26.000Z</published>
    <updated>2020-07-13T15:33:44.201Z</updated>
    
    <content type="html"><![CDATA[<h1 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h1><p>从上文我们可知，虽然K8S提供了PVC的方式进行存储的便利性，但是PV的创建还是要手工的，使用起来不是很方便，在k8s 1.4以后，kubernetes提供了一种更加方便的动态创建PV的方式，即StorageClass。使用StorageClass时无需预先创建固定大小的PV来等待使用者创建PVC使用，而是直接创建PVC即可使用。</p><p>本文主要使用Rook搭建一个ceph集群，然后使用StorageClass来直接创建PVC。</p><h1 id="Rook与ceph"><a href="#Rook与ceph" class="headerlink" title="Rook与ceph"></a>Rook与ceph</h1><p><a href="https://rook.io/" target="_blank" rel="noopener">Rook</a> 是基于<code>Kubernetes</code>之上，提供一键部署存储系统的编排系统。Rook 将存储软件转变成自我管理、自我扩展和自我修复的存储服务，通过自动化部署、启动、配置、供应、扩展、升级、迁移、灾难恢复、监控和资源管理来实现。Rook 底层使用云原生容器管理、调度和编排平台提供的能力来提供这些功能。</p><a id="more"></a><p>Rook 使用底层云本机容器管理、调度和编排平台提供的工具来实现它自身的功能。Rook 目前支持Ceph、NFS、Minio Object Store和CockroachDB。</p><p>ook使用Kubernetes原语使Ceph存储系统能够在Kubernetes上运行。下图说明了Ceph Rook如何与Kubernetes集成：</p><p><img src="/assets/5c33112b3d5d3.png" alt></p><p>随着Rook在Kubernetes集群中运行，Kubernetes应用程序可以挂载由Rook管理的<strong>块设备和文件系统</strong>，或者可以使用S3 / Swift API提供<strong>对象存储</strong>。</p><h2 id="Rook组件"><a href="#Rook组件" class="headerlink" title="Rook组件"></a>Rook组件</h2><p>Rook的主要组件有两个，功能如下：</p><ol><li>Rook Operator：Rook oprerator是一个简单的容器，具有引导和监视存储集群所需的全部功能。oprerator将启动并监控ceph monitor pods和OSDs的守护进程，它提供基本的RADOS存储。oprerator通过初始化运行服务所需的pod和其他组件来管理池，对象存储（S3 / Swift）和文件系统的CRD。<ul><li>Rook与Kubernetes交互的组件</li><li>整个Rook集群只有一个</li></ul></li><li>Rook Agent<ul><li>与Rook Operator交互，执行命令</li><li>每个Kubernetes的Node上都会启动一个</li><li>不同的存储系统，启动的Agent是不同的</li></ul></li></ol><h2 id="Rook-amp-Ceph框架"><a href="#Rook-amp-Ceph框架" class="headerlink" title="Rook &amp; Ceph框架"></a>Rook &amp; Ceph框架</h2><p>使用Rook部署Ceph集群的架构图如下：</p><p><img src="/assets/5c331183ea477.png" alt="img"></p><p>从上面可以看出，通过Rook部署完Ceph集群后，就可以提供<code>Volume Claim</code>给Kubernetes集群里的App使用了。</p><h2 id="ceph简介"><a href="#ceph简介" class="headerlink" title="ceph简介"></a>ceph简介</h2><p>Ceph 是一个分布式存储系统，具备大规模、高性能、无单点失败的特点。Ceph 是一种高度可扩展的分布式存储解决方案，用于具有多年生产部署的<strong>块存储，对象存储和共享文件系统。</strong></p><p>Ceph 包括多个组件：</p><ul><li><strong>Ceph Monitors(MON)</strong>：负责生成集群票选机制。所有的集群节点都会向 Mon 进行汇报，并在每次状态变更时进行共享信息。</li><li><strong>Ceph Object Store Devices(OSD)</strong>：负责在本地文件系统保存对象，并通过网络提供访问。通常 OSD 守护进程会绑定在集群的一个物理盘上，Ceph 客户端直接和 OSD 打交道。</li><li><strong>Ceph Manager(MGR)</strong>：提供额外的监控和界面给外部的监管系统使用。</li><li><strong>Reliable Autonomic Distributed Object Stores</strong>：Ceph 存储集群的核心。这一层用于为存储数据提供一致性保障，执行数据复制、故障检测以及恢复等任务。</li></ul><p>为了在 Ceph 上进行读写，客户端首先要联系 MON，获取最新的集群地图，其中包含了集群拓扑以及数据存储位置的信息。Ceph 客户端使用集群地图来获知需要交互的 OSD，从而和特定 OSD 建立联系。</p><p>而对应的三种存储类型对应的区别如下：</p><ul><li>共享文件存储：类似我们平时看到的文件，有点类似NFS</li><li>块存储：硬盘等裸设备的存储方式，ceph创建时，就是创建一个虚拟的硬盘</li><li>对象存储：用 S3 兼容接口开放存储服务。</li></ul><h2 id="Rook部署"><a href="#Rook部署" class="headerlink" title="Rook部署"></a>Rook部署</h2><p>本次的测试环境如下：</p><ul><li><p>三台k8s v1.17.2节点</p><p>| 主机名 |    IP地址    |<br>| :——: | :—————: |<br>| master | 192.168.1.60 |<br>| node1  | 192.168.1.61 |<br>| node2  | 192.168.1.62 |</p></li><li><p>rook 1.2版本</p></li></ul><p>本次的主要资源列表：</p><ul><li>官方文档：<a href="https://rook.io/docs/rook/v1.2/ceph-quickstart.html" target="_blank" rel="noopener">https://rook.io/docs/rook/v1.2/ceph-quickstart.html</a></li><li>YAML文件列表：<a href="https://github.com/rook/rook/tree/release-1.2/cluster/examples/kubernetes/ceph" target="_blank" rel="noopener">https://github.com/rook/rook/tree/release-1.2/cluster/examples/kubernetes/ceph</a></li></ul><h2 id="Rook-Operator部署"><a href="#Rook-Operator部署" class="headerlink" title="Rook Operator部署"></a>Rook Operator部署</h2><p>克隆rook github仓库到本地，然后运行common.yaml 与 operator.yaml 两个资源清单文件：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">git <span class="built_in">clone</span> --single-branch --branch release-1.2 https://github.com/rook/rook.git</span><br><span class="line"><span class="built_in">cd</span> cluster/examples/kubernetes/ceph</span><br><span class="line">kubectl create -f common.yaml</span><br><span class="line">kubectl create -f operator.yaml</span><br></pre></td></tr></table></figure><p>由于docker镜像比较大，建议先下载以下镜像：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">rook/ceph:v1.2.4</span><br><span class="line">ceph/ceph:v14.2.7</span><br><span class="line">quay.io/cephcsi/cephcsi:v1.2.2</span><br><span class="line">quay.io/k8scsi/csi-snapshotter:v1.2.2</span><br><span class="line">quay.io/k8scsi/csi-provisioner:v1.4.0</span><br><span class="line">quay.io/k8scsi/csi-node-driver-registrar:v1.2.0</span><br><span class="line">quay.io/k8scsi/csi-attacher:v1.2.0</span><br></pre></td></tr></table></figure><p>操作完成之后，必须保证rook-ceph-operator有在正常运行(1.2版本的namespace为<code>rook-ceph</code>)：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@master Rook]<span class="comment"># kubectl get pods -n rook-ceph -o wide -l app=rook-ceph-operator</span></span><br><span class="line">NAME                                  READY   STATUS    RESTARTS   AGE     IP          NODE    NOMINATED NODE   READINESS GATES</span><br><span class="line">rook-ceph-operator-69d86f8597-7txd9   1/1     Running   37         7h49m   10.44.0.8   node1   &lt;none&gt;           &lt;none&gt;</span><br></pre></td></tr></table></figure><p>部署rook-ceph-operator过程中，会触发以DaemonSet的方式在集群部署Agent和Discoverpods。operator会在集群内的每个主机创建两个pod:rook-discover,rook-ceph-agent。</p><h2 id="ceph集群创建"><a href="#ceph集群创建" class="headerlink" title="ceph集群创建"></a>ceph集群创建</h2><p>现在 Rook Operator 处于 Running 状态，接下来我们就可以创建 Ceph 集群了。为了使集群在重启后不受影响，请确保设置的 <code>dataDirHostPath</code> 属性值为有效得主机路径，同时要保证有5G以上的空闲空间。</p><p>官方提供了3个yaml用来创建集群：</p><ul><li><code>cluster.yaml</code>: 该文件包含生产存储集群的通用设置。至少需要三个节点。</li><li><code>cluster-test.yaml</code>: 未配置冗余的测试群集的设置。只需要一个节点。</li><li><code>cluster-minimal.yaml</code>: 仅使用一个ceph-mon和一个ceph-mgr创建一个集群，因此Ceph仪表板可用于其余集群配置。</li></ul><p>如果你有自定义了<code>dataDirHostPath</code> ，就需要修改。</p><p>当检查到<strong>Rook operator, agent, and discover</strong> pods已经是running状态后，就可以部署rook cluster了。</p><p>同时，Ceph 有一个 Dashboard 工具，我们可以在上面查看集群的状态，包括总体运行状态，mgr、osd 和其他 Ceph 进程的状态，查看池和 PG 状态，以及显示守护进程的日志等等。只需要设置<code>dashboard.enable=true</code>即可，这样 Rook Operator 就会启用 ceph-mgr dashboard 模块</p><p>就直接 <code>kubectl apply -f cluster.yaml</code>即可。这时就需要漫长的等待了，因为需要下载镜像，必须保证以下pod都是在正常运行：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">[root@master Rook]<span class="comment"># kubectl get pods -n rook-ceph -o wide</span></span><br><span class="line">NAME                                               READY   STATUS             RESTARTS   AGE    IP             NODE     NOMINATED NODE   READINESS GATES</span><br><span class="line">csi-cephfsplugin-2rmx5                             3/3     Running            36         104d   192.168.1.60   master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-cephfsplugin-ctf49                             3/3     Running            9          104d   192.168.1.62   node2    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-cephfsplugin-gjt57                             3/3     Running            33         104d   192.168.1.61   node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-cephfsplugin-provisioner-5c85564f4c-frj6m      4/4     Running            50         8h     10.44.0.2      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-cephfsplugin-provisioner-5c85564f4c-lx8jm      4/4     Running            55         9d     10.32.0.2      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-rbdplugin-bhmdv                                3/3     Running            9          104d   192.168.1.62   node2    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-rbdplugin-chh2t                                3/3     Running            34         104d   192.168.1.61   node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-rbdplugin-provisioner-7bb78d6c66-8s62m         5/5     Running            79         8h     10.44.0.10     node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-rbdplugin-provisioner-7bb78d6c66-s9gq4         5/5     Running            70         9d     10.32.0.9      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">csi-rbdplugin-rcrcc                                3/3     Running            36         104d   192.168.1.60   master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-crashcollector-master-75f895557f-8zhlh   1/1     Running            1          14d    10.32.0.8      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-crashcollector-node1-59866fd66-fv5gg     1/1     Running            0          9h     10.44.0.7      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-crashcollector-node2-54c9469bcf-kh4zj    1/1     Running            0          8h     10.36.0.2      node2    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-mon<span class="_">-a</span>-848bc78f6d-4xmv6                   1/1     Running            0          8h     10.36.0.3      node2    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-mon-b-5dbdbc4df-2jpr7                    1/1     Running            1          14d    10.32.0.3      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-mon-c-6f6558f959-x7mnd                   1/1     Running            1          9h     10.44.0.5      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-operator-69d86f8597-7txd9                1/1     Running            38         8h     10.44.0.8      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-osd-0-6fd84c65c4-d8q8b                   1/1     Running            0          8h     10.36.0.1      node2    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-osd-1-67bf5c7686-q4cgk                   1/1     Running            178        14d    10.32.0.5      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-osd-2-6b65ddbdf9-p5m8f                   1/1     Running            0          9h     10.44.0.6      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-ceph-osd-prepare-master-lxgjt                 0/1     Completed          0          16h    10.32.0.13     master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-discover-6x98l                                1/1     Running            10         104d   10.44.0.1      node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-discover-fg9zh                                1/1     Running            11         104d   10.32.0.6      master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">rook-discover-mxlwt                                1/1     Running            3          104d   10.36.0.6      node2    &lt;none&gt;           &lt;none&gt;</span><br></pre></td></tr></table></figure><p>如果pod运行失败，则需要使用<code>kubectl describe pod -n rook-ceph xxx</code>查看具体失败的原因。</p><h3 id="Ceph-Dashboard"><a href="#Ceph-Dashboard" class="headerlink" title="Ceph Dashboard"></a>Ceph Dashboard</h3><p>前文说到，设置<code>dashboard.enable=true</code>即可，这样 Rook Operator 就会启用 ceph-mgr dashboard 模块，并将创建一个 Kubernetes Service 来暴露该服务，将启用端口 7000 进行 https 访问，如果 Ceph 集群部署成功了，我们可以使用下面的命令来查看 Dashboard 的 Service：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]# kubectl get svc -n rook-ceph</span><br><span class="line">NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE</span><br><span class="line">csi-cephfsplugin-metrics   ClusterIP   10.96.0.151      &lt;none&gt;        8080/TCP,8081/TCP   105d</span><br><span class="line">csi-rbdplugin-metrics      ClusterIP   10.97.104.240    &lt;none&gt;        8080/TCP,8081/TCP   105d</span><br><span class="line">rook-ceph-mgr              ClusterIP   10.105.202.253   &lt;none&gt;        9283/TCP            105d</span><br><span class="line">rook-ceph-mgr-dashboard    NodePort    10.104.133.172   &lt;none&gt;        7000:30024/TCP      105d</span><br><span class="line">rook-ceph-mon-a            ClusterIP   10.99.134.244    &lt;none&gt;        6789/TCP,3300/TCP   105d</span><br><span class="line">rook-ceph-mon-b            ClusterIP   10.99.12.131     &lt;none&gt;        6789/TCP,3300/TCP   105d</span><br><span class="line">rook-ceph-mon-c            ClusterIP   10.97.137.167    &lt;none&gt;        6789/TCP,3300/TCP   105d</span><br></pre></td></tr></table></figure><p>这里的 <code>rook-ceph-mgr</code> 服务用于报告 Prometheus metrics 指标数据的，而后面的的 <code>rook-ceph-mgr-dashboard</code> 服务就是我们的 Dashboard 服务，如果在集群内部我们可以通过 DNS 名称 <code>http://rook-ceph-mgr-dashboard.rook-ceph:7000</code> 或者 CluterIP <code>http://10.104.133.172:7000</code> 来进行访问，如果想要外部访问的话，则使用<code>kubectl edit svc -n rook-ceph rook-ceph-mgr-dashboard</code>将TYPE修改为NodePort即可。</p><p>这时就可以使用 <code>http://&lt;NodeIp&gt;:30024</code> 就可以访问到 Dashboard 了。但是在访问的时候需要我们登录才能够访问，Rook 创建了一个默认的用户 admin，并在运行 Rook 的命名空间中生成了一个名为 <code>rook-ceph-dashboard-admin-password</code> 的 Secret，要获取密码，可以运行以下命令：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl -n rook-ceph get secret rook-ceph-dashboard-password -o jsonpath="&#123;['data']['password']&#125;" | base64 --decode &amp;&amp; echo</span></span><br><span class="line">xjDadabefO</span><br><span class="line">[root@master ~]<span class="comment">#</span></span><br></pre></td></tr></table></figure><h3 id="Rook工具箱安装"><a href="#Rook工具箱安装" class="headerlink" title="Rook工具箱安装"></a>Rook工具箱安装</h3><p>要验证集群是否处于正常状态，我们可以使用 <a href="https://rook.io/docs/rook/v1.2/ceph-toolbox.html" target="_blank" rel="noopener">Rook 工具箱</a> 来运行 <code>ceph status</code> 命令查看。</p><p>Rook 工具箱是一个用于调试和测试 Rook 的常用工具容器，该工具基于 CentOS 镜像，所以可以使用 <code>yum</code> 来轻松安装更多的工具包。 我们这里用 Deployment 控制器来部署 Rook 工具箱，<code>kubectl create -f toolbox.yaml</code>；</p><p>一旦 toolbox 的 Pod 运行成功后，我们就可以使用下面的命令进入到工具箱内部进行操作：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='&#123;.items[0].metadata.name&#125;') bash</span><br></pre></td></tr></table></figure><p>工具箱中的所有可用工具命令均已准备就绪，可满足您的故障排除需求。例如：</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">ceph status</span><br><span class="line">ceph osd status</span><br><span class="line">ceph df</span><br><span class="line">rados df</span><br></pre></td></tr></table></figure><p>比如现在我们要查看集群的状态，需要满足下面的条件才认为是健康的：</p><ul><li>所有 mons 应该达到法定数量</li><li>mgr 应该是激活状态</li><li>至少有一个 OSD 处于激活状态</li><li>如果不是 HEALTH_OK 状态，则应该查看告警或者错误信息</li></ul><p>以下为实操部分：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br></pre></td><td class="code"><pre><span class="line">[root@master Rook]<span class="comment"># kubectl -n rook-ceph exec -it $(kubectl -n rook-ceph get pod -l "app=rook-ceph-tools" -o jsonpath='&#123;.items[0].metadata.name&#125;') bash</span></span><br><span class="line">bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_TIME: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">[root@node2 /]<span class="comment">#</span></span><br><span class="line">[root@node2 /]<span class="comment">#</span></span><br><span class="line">[root@node2 /]<span class="comment"># ceph status</span></span><br><span class="line">  cluster:</span><br><span class="line">    id:     892bda12-220d-4ec3-a0be-2e123c7d4a66</span><br><span class="line">    health: HEALTH_OK</span><br><span class="line"></span><br><span class="line">  services:</span><br><span class="line">    mon: 3 daemons, quorum a,b,c (age 2h)</span><br><span class="line">    mgr: a(active, since 2h)</span><br><span class="line">    osd: 3 osds: 3 up (since 2h), 3 <span class="keyword">in</span> (since 2h)</span><br><span class="line"></span><br><span class="line">  data:</span><br><span class="line">    pools:   0 pools, 0 pgs</span><br><span class="line">    objects: 0 objects, 0 B</span><br><span class="line">    usage:   56 GiB used, 231 GiB / 286 GiB avail</span><br><span class="line">    pgs:</span><br><span class="line"><span class="comment"># 所有 OSD 的状态</span></span><br><span class="line">[root@node2 /]<span class="comment"># ceph osd status</span></span><br><span class="line">+----+--------+-------+-------+--------+---------+--------+---------+-----------+</span><br><span class="line">| id |  host  |  used | avail | wr ops | wr data | rd ops | rd data |   state   |</span><br><span class="line">+----+--------+-------+-------+--------+---------+--------+---------+-----------+</span><br><span class="line">| 0  | node2  | 12.3G | 83.0G |    0   |     0   |    0   |     0   | exists,up |</span><br><span class="line">| 1  | master | 25.3G | 70.1G |    0   |     0   |    0   |     0   | exists,up |</span><br><span class="line">| 2  | node1  | 18.0G | 77.3G |    0   |     0   |    0   |     0   | exists,up |</span><br><span class="line">+----+--------+-------+-------+--------+---------+--------+---------+-----------+</span><br><span class="line">[root@node2 /]<span class="comment"># ceph osd df</span></span><br><span class="line">ID CLASS WEIGHT  REWEIGHT SIZE    RAW USE DATA    OMAP   META AVAIL   %USE  VAR  PGS STATUS</span><br><span class="line"> 1   hdd 0.09319  1.00000  95 GiB  25 GiB  70 GiB 26 KiB  0 B  70 GiB 26.51 1.36   0     up</span><br><span class="line"> 2   hdd 0.09319  1.00000  95 GiB  18 GiB  77 GiB 26 KiB  0 B  77 GiB 18.94 0.97   0     up</span><br><span class="line"> 0   hdd 0.09319  1.00000  95 GiB  12 GiB  83 GiB 26 KiB  0 B  83 GiB 12.98 0.67   0     up</span><br><span class="line">                    TOTAL 286 GiB  56 GiB 231 GiB 80 KiB  0 B 231 GiB 19.47</span><br><span class="line">MIN/MAX VAR: 0.67/1.36  STDDEV: 5.54</span><br><span class="line"></span><br><span class="line"><span class="comment"># 显示 Pool 和总体用量</span></span><br><span class="line">[root@node2 /]<span class="comment"># rados df</span></span><br><span class="line">POOL_NAME USED OBJECTS CLONES COPIES MISSING_ON_PRIMARY UNFOUND DEGRADED RD_OPS RD WR_OPS WR USED COMPR UNDER COMPR</span><br><span class="line"></span><br><span class="line">total_objects    0</span><br><span class="line">total_used       56 GiB</span><br><span class="line">total_avail      231 GiB</span><br><span class="line">total_space      286 GiB</span><br></pre></td></tr></table></figure><h3 id="ceph的挂载方法"><a href="#ceph的挂载方法" class="headerlink" title="ceph的挂载方法"></a>ceph的挂载方法</h3><p>创建完ceph集群之后，那我们要如何使用呢？ </p><h4 id="块设备"><a href="#块设备" class="headerlink" title="块设备"></a>块设备</h4><p>如果需要挂载ceph的分区，首先需要创建一个POD：<code>kubectl create -f direct-mount.yaml</code>，完成之后，查看且进入POD：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl -n rook-ceph get pod -l app=rook-direct-mount</span></span><br><span class="line">NAME                                 READY   STATUS    RESTARTS   AGE</span><br><span class="line">rook-direct-mount-6c9d444887-fgj5g   1/1     Running   0          161m</span><br><span class="line">[root@master ~]<span class="comment">#</span></span><br><span class="line">[root@master ~]<span class="comment"># kubectl -n rook-ceph exec -it rook-direct-mount-6c9d444887-fgj5g -- bash</span></span><br><span class="line">bash: warning: setlocale: LC_CTYPE: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_COLLATE: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">bash: warning: setlocale: LC_TIME: cannot change locale (en_US.UTF-8): No such file or directory</span><br><span class="line">[root@master /]<span class="comment">#</span></span><br></pre></td></tr></table></figure><p>进入之后，创建一个10MB的RBD：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">[root@master /]<span class="comment"># rbd create replicapool/test --size 10</span></span><br><span class="line">[root@master /]<span class="comment"># rbd info replicapool/test</span></span><br><span class="line">rbd image <span class="string">'test'</span>:</span><br><span class="line">size 10 MiB <span class="keyword">in</span> 3 objects</span><br><span class="line">order 22 (4 MiB objects)</span><br><span class="line">snapshot_count: 0</span><br><span class="line">id: 163a80e6351c31</span><br><span class="line">block_name_prefix: rbd_data.163a80e6351c31</span><br><span class="line">format: 2</span><br><span class="line">features: layering</span><br><span class="line">op_features:</span><br><span class="line">flags:</span><br><span class="line">create_timestamp: Sun Jun 14 15:56:33 2020</span><br><span class="line">access_timestamp: Sun Jun 14 15:56:33 2020</span><br><span class="line">modify_timestamp: Sun Jun 14 15:56:33 2020</span><br><span class="line">[root@master /]<span class="comment"># rbd map replicapool/test</span></span><br><span class="line">/dev/rbd1</span><br><span class="line">[root@master /]<span class="comment"># lsblk | grep rbd</span></span><br><span class="line">rbd0   253:0    0   20G  0 disk</span><br><span class="line">rbd1   253:16   0   10M  0 disk</span><br></pre></td></tr></table></figure><p>使用<code>lsblk</code> 就可以看到创建出来的块设备了。这个就可以理解为普通的磁盘分区，我们可以直接格式化且挂载他：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line">[root@master /]<span class="comment"># mkfs.ext4 -m0 /dev/rbd1</span></span><br><span class="line">mke2fs 1.42.9 (28-Dec-2013)</span><br><span class="line">Discarding device blocks: <span class="keyword">done</span></span><br><span class="line">Filesystem label=</span><br><span class="line">OS <span class="built_in">type</span>: Linux</span><br><span class="line">Block size=1024 (<span class="built_in">log</span>=0)</span><br><span class="line">Fragment size=1024 (<span class="built_in">log</span>=0)</span><br><span class="line">Stride=4096 blocks, Stripe width=4096 blocks</span><br><span class="line">2560 inodes, 10240 blocks</span><br><span class="line">0 blocks (0.00%) reserved <span class="keyword">for</span> the super user</span><br><span class="line">First data block=1</span><br><span class="line">Maximum filesystem blocks=10485760</span><br><span class="line">2 block groups</span><br><span class="line">8192 blocks per group, 8192 fragments per group</span><br><span class="line">1280 inodes per group</span><br><span class="line">Superblock backups stored on blocks:</span><br><span class="line">8193</span><br><span class="line"></span><br><span class="line">Allocating group tables: <span class="keyword">done</span></span><br><span class="line">Writing inode tables: <span class="keyword">done</span></span><br><span class="line">Creating journal (1024 blocks): <span class="keyword">done</span></span><br><span class="line">Writing superblocks and filesystem accounting information: <span class="keyword">done</span></span><br><span class="line"></span><br><span class="line">[root@master /]<span class="comment"># mkdir /tmp/rook-volume</span></span><br><span class="line">[root@master /]<span class="comment"># mount /dev/rbd1 /tmp/rook-volume</span></span><br><span class="line">[root@master /]<span class="comment"># df -h |grep rook</span></span><br><span class="line">/dev/rbd1       8.7M  172K  8.4M   2% /tmp/rook-volume</span><br></pre></td></tr></table></figure><p>这样就像使用本地磁盘一样了。</p><p>其他的使用方法见：<a href="https://rook.io/docs/rook/v1.2/direct-tools.html" target="_blank" rel="noopener">https://rook.io/docs/rook/v1.2/direct-tools.html</a></p><h1 id="实例：wordpress"><a href="#实例：wordpress" class="headerlink" title="实例：wordpress"></a>实例：wordpress</h1><p>我们建一个wordpress+mysql+StorageClass的例子，参考了官网的例子：<a href="https://rook.io/docs/rook/v1.2/ceph-block.html" target="_blank" rel="noopener">https://rook.io/docs/rook/v1.2/ceph-block.html</a></p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">ceph.rook.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">CephBlockPool</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">replicapool</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">rook-ceph</span></span><br><span class="line"><span class="attr">spec:</span></span><br><span class="line"><span class="attr">  failureDomain:</span> <span class="string">host</span></span><br><span class="line"><span class="attr">  replicated:</span></span><br><span class="line"><span class="attr">    size:</span> <span class="number">3</span></span><br><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">storage.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">StorageClass</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">   name:</span> <span class="string">rook-ceph-block</span></span><br><span class="line"><span class="comment"># Change "rook-ceph" provisioner prefix to match the operator namespace if needed</span></span><br><span class="line"><span class="attr">provisioner:</span> <span class="string">rook-ceph.rbd.csi.ceph.com</span></span><br><span class="line"><span class="attr">parameters:</span></span><br><span class="line">    <span class="comment"># clusterID is the namespace where the rook cluster is running</span></span><br><span class="line"><span class="attr">    clusterID:</span> <span class="string">rook-ceph</span></span><br><span class="line">    <span class="comment"># Ceph pool into which the RBD image shall be created</span></span><br><span class="line"><span class="attr">    pool:</span> <span class="string">replicapool</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># RBD image format. Defaults to "2".</span></span><br><span class="line"><span class="attr">    imageFormat:</span> <span class="string">"2"</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># RBD image features. Available for imageFormat: "2". CSI RBD currently supports only `layering` feature.</span></span><br><span class="line"><span class="attr">    imageFeatures:</span> <span class="string">layering</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># The secrets contain Ceph admin credentials.</span></span><br><span class="line">    <span class="string">csi.storage.k8s.io/provisioner-secret-name:</span> <span class="string">rook-csi-rbd-provisioner</span></span><br><span class="line">    <span class="string">csi.storage.k8s.io/provisioner-secret-namespace:</span> <span class="string">rook-ceph</span></span><br><span class="line">    <span class="string">csi.storage.k8s.io/node-stage-secret-name:</span> <span class="string">rook-csi-rbd-node</span></span><br><span class="line">    <span class="string">csi.storage.k8s.io/node-stage-secret-namespace:</span> <span class="string">rook-ceph</span></span><br><span class="line"></span><br><span class="line">    <span class="comment"># Specify the filesystem type of the volume. If not specified, csi-provisioner</span></span><br><span class="line">    <span class="comment"># will set default as `ext4`.</span></span><br><span class="line">    <span class="string">csi.storage.k8s.io/fstype:</span> <span class="string">xfs</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># Delete the rbd volume when a PVC is deleted</span></span><br><span class="line"><span class="attr">reclaimPolicy:</span> <span class="string">Delete</span></span><br></pre></td></tr></table></figure><p>将上述yaml保存为storageclass.yaml，然后 <code>kubectl create -f storageclass.yaml</code>，再查看创建好的资源：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@master Rook]<span class="comment"># kubectl -n rook-ceph get cephblockpools.ceph.rook.io</span></span><br><span class="line">NAME          AGE</span><br><span class="line">replicapool   2m28s</span><br><span class="line">[root@master Rook]<span class="comment"># kubectl -n rook-ceph get storageclasses.storage.k8s.io</span></span><br><span class="line">NAME              PROVISIONER                  RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE</span><br><span class="line">rook-ceph-block   rook-ceph.rbd.csi.ceph.com   Delete          Immediate           <span class="literal">false</span>                  2m8s</span><br></pre></td></tr></table></figure><p>下载相关的资料：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">wget https://raw.githubusercontent.com/rook/rook/release-1.2/cluster/examples/kubernetes/mysql.yaml</span><br><span class="line">wget https://raw.githubusercontent.com/rook/rook/release-1.2/cluster/examples/kubernetes/wordpress.yaml</span><br><span class="line">kubectl apply -f mysql.yaml</span><br><span class="line"><span class="comment"># 先将svc的type修改为NodePort</span></span><br><span class="line">kubectl apply -f wordpress.yaml</span><br></pre></td></tr></table></figure><p>创建成功之后，查看PVC已经成功绑定了2个了。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl get pvc</span></span><br><span class="line">NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS      AGE</span><br><span class="line">mysql-pv-claim   Bound    pvc-9b861f8f-f32c-4b0e-90cc-ac15931d1677   20Gi       RWO            rook-ceph-block   24h</span><br><span class="line">wp-pv-claim      Bound    pvc-57724038-8e8c-40c3-b636-f7ece2bf194f   20Gi       RWO            rook-ceph-block   24h</span><br><span class="line">[root@master ~]<span class="comment"># lsblk |grep rbd</span></span><br><span class="line">rbd0   253:0    0   20G  0 disk /var/lib/kubelet/pods/4399e9ab-55ec-4ec2-9979-6a968e486f0e/volumes/kubernetes.io~csi/pvc-57724038-8e8c-40c3-b636-f7ece2bf194f/mount</span><br></pre></td></tr></table></figure><p>使用 <code>lsblk |grep rbd</code> 可以看到有一块rbd是挂载在master上面的，也可以看到具体的文件内容，那为什么可看到呢？这是因为wp的pod是运行在master上面。</p><p>最后清理掉测试的文件：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">kubectl delete -f wordpress.yaml</span><br><span class="line">kubectl delete -f mysql.yaml</span><br><span class="line">kubectl delete -n rook-ceph cephblockpools.ceph.rook.io replicapool</span><br><span class="line">kubectl delete storageclass rook-ceph-block</span><br></pre></td></tr></table></figure><h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><blockquote><p><a href="https://rook.io/docs/rook/v1.2/ceph-quickstart.html" target="_blank" rel="noopener">https://rook.io/docs/rook/v1.2/ceph-quickstart.html</a></p><p><a href="https://www.qikqiak.com/post/deploy-ceph-cluster-with-rook/" target="_blank" rel="noopener">使用 Rook 快速搭建 Ceph 集群</a></p><p><a href="https://k.i4t.com/kubernetes_ceph_storageclass.html" target="_blank" rel="noopener">K8s企业实践使用storageclass实现动态存储</a></p><p><a href="https://blog.fleeto.us/post/the-ultimate-rook-and-ceph-survival-guide/" target="_blank" rel="noopener">Rook &amp; Ceph 简介</a></p><p><a href="https://www.cnblogs.com/breezey/p/8849501.html" target="_blank" rel="noopener">kubernetes挂载ceph rbd和cephfs的方法</a></p><p><a href="https://www.gmlyo.com/tags/Kubernetes部署rook-ceph存储系统/" target="_blank" rel="noopener">Kubernetes部署rook+ceph存储系统</a></p><p><a href="http://www.yangguanjun.com/2018/12/22/rook-ceph-practice-part1/" target="_blank" rel="noopener">Kubernetes上使用Rook部署Ceph系统并提供PV服务</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;简介&quot;&gt;&lt;a href=&quot;#简介&quot; class=&quot;headerlink&quot; title=&quot;简介&quot;&gt;&lt;/a&gt;简介&lt;/h1&gt;&lt;p&gt;从上文我们可知，虽然K8S提供了PVC的方式进行存储的便利性，但是PV的创建还是要手工的，使用起来不是很方便，在k8s 1.4以后，kubernetes提供了一种更加方便的动态创建PV的方式，即StorageClass。使用StorageClass时无需预先创建固定大小的PV来等待使用者创建PVC使用，而是直接创建PVC即可使用。&lt;/p&gt;
&lt;p&gt;本文主要使用Rook搭建一个ceph集群，然后使用StorageClass来直接创建PVC。&lt;/p&gt;
&lt;h1 id=&quot;Rook与ceph&quot;&gt;&lt;a href=&quot;#Rook与ceph&quot; class=&quot;headerlink&quot; title=&quot;Rook与ceph&quot;&gt;&lt;/a&gt;Rook与ceph&lt;/h1&gt;&lt;p&gt;&lt;a href=&quot;https://rook.io/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Rook&lt;/a&gt; 是基于&lt;code&gt;Kubernetes&lt;/code&gt;之上，提供一键部署存储系统的编排系统。Rook 将存储软件转变成自我管理、自我扩展和自我修复的存储服务，通过自动化部署、启动、配置、供应、扩展、升级、迁移、灾难恢复、监控和资源管理来实现。Rook 底层使用云原生容器管理、调度和编排平台提供的能力来提供这些功能。&lt;/p&gt;
    
    </summary>
    
      <category term="Docker&amp;k8s" scheme="https://www.wumingx.com/categories/k8s/"/>
    
    
      <category term="kubernetes" scheme="https://www.wumingx.com/tags/kubernetes/"/>
    
  </entry>
  
  <entry>
    <title>k8s入门教程十四:安全与RBAC</title>
    <link href="https://www.wumingx.com/k8s/kubernetes-rbac.html"/>
    <id>https://www.wumingx.com/k8s/kubernetes-rbac.html</id>
    <published>2020-06-06T15:15:42.000Z</published>
    <updated>2020-06-14T16:24:24.800Z</updated>
    
    <content type="html"><![CDATA[<h1 id="简介"><a href="#简介" class="headerlink" title="简介"></a>简介</h1><p>Kubernetes 作为一个分布式集群的管理工具，保证集群的安全性是其一个重要的任务。<strong>API Server 是集群内部各个组件通信的中介，也是外部控制的入口。</strong>所以 Kubernetes 的安全机制基本就是围绕保护 API Server 来设计的。<strong>Kubernetes 使用了认证（Authentication）、鉴权（Authorization）、准入控制（Admission Control）三步来保证API Server的安全。</strong></p><a id="more"></a><h2 id="认证Authentication"><a href="#认证Authentication" class="headerlink" title="认证Authentication"></a>认证Authentication</h2><p>认证的方法有：</p><ul><li>HTTP Token 认证：通过一个 Token 来识别合法用户：HTTP Token 的认证是用一个很长的特殊编码方式的并且难以被模仿的字符串 - Token 来表达客户的一<br>种方式。Token 是一个很长的很复杂的字符串，每一个 Token 对应一个用户名存储在 API Server 能访问的文件中。当客户端发起 API 调用请求时，需要在 HTTP Header 里放入 Token。</li><li>HTTP Base 认证：通过 用户名+密码 的方式认证。用户名+：+密码 用 BASE64 算法进行编码后的字符串放在 HTTP Request 中的 Heather；Authorization 域里发送给服务端，服务端收到后进行编码，获取用户名及密码</li><li>最严格的 HTTPS 证书认证：基于 CA 根证书签名的客户端身份认证方式</li></ul><p>认证是包括两种类型的:</p><ul><li><code>Kubenetes组件对API Server的访问：kubectl、Controller Manager、Scheduler、kubelet、kubeproxy</code><ul><li>Controller Manager、Scheduler 与 API Server 在同一台机器，所以直接使用API Server的非安全端口访问， <code>--insecure-bind-address=127.0.0.1</code></li><li>kubectl、kubelet、kube-proxy 访问 API Server 就都需要证书进行 HTTPS 双向认证</li><li>实现方式：kubeconfig 文件包含集群参数（CA证书、API Server地址），客户端参数（上面生成的证书和私钥），集群context 信息（集群名称、用户名）。Kubenetes 组件通过启动时指定不同的 kubeconfig 文件可以切换到不同的集群</li></ul></li><li><code>Kubernetes 管理的 Pod 对容器的访问：Pod（dashborad 也是以 Pod 形式运行）。</code><ul><li>Pod中的容器访问API Server。因为Pod的创建、销毁是动态的，所以要为它手动生成证书就不可行了。Kubenetes使用了Service Account解决Pod 访问API Server的认证问题</li><li>Kubernetes 设计了一种资源对象叫做 Secret，分为两类，一种是用于 ServiceAccount 的 service-accounttoken，另一种是用于保存用户自定义保密信息的 Opaque。<strong>ServiceAccount 中用到包含三个部分：Token、ca.crt、namespace</strong><ul><li>token是使用 API Server 私钥签名的 JWT。用于访问API Server时，Server端认证</li><li>ca.crt，根证书。用于Client端验证API Server发送的证书</li><li>namespace, 标识这个service-account-token的作用域名空间</li></ul></li></ul></li></ul><p>认证总体上如下图：</p><p><img src="/assets/1591532963530.png" alt></p><h2 id="鉴权Authorization"><a href="#鉴权Authorization" class="headerlink" title="鉴权Authorization"></a>鉴权Authorization</h2><p>鉴权是确定请求方有哪些资源的权限。API Server 目前支持以下几种授权策略 （通过 API Server 的启动参数 “—authorization-mode” 设置）</p><ul><li>AlwaysDeny：表示拒绝所有的请求，一般用于测试</li><li>AlwaysAllow：允许接收所有请求，如果集群不需要授权流程，则可以采用该策略</li><li>ABAC（Attribute-Based Access Control）：基于属性的访问控制，表示使用用户配置的授权规则对用户请求进行匹配和控制</li><li>Webbook：通过调用外部 REST 服务对用户进行授权</li><li>RBAC（Role-Based Access Control）：<strong>基于角色的访问控制，现行默认规则。</strong>相对其它访问控制方式，拥有以下优势：<ul><li>对集群中的资源和非资源均拥有完整的覆盖</li><li>整个 RBAC 完全由几个 API 对象完成，同其它 API 对象一样，可以用 kubectl 或 API 进行操作</li><li>可以在运行时进行调整，无需重启 API Server</li></ul></li></ul><h2 id="准入控制Admission-Control"><a href="#准入控制Admission-Control" class="headerlink" title="准入控制Admission Control"></a>准入控制Admission Control</h2><p>准入控制是API Server的插件集合，通过添加不同的插件，实现额外的准入控制规则。甚至于API Server的一些主要的功能都需要通过 Admission Controllers 实现，比如 ServiceAccount。</p><p>列举几个插件的功能：</p><ul><li>NamespaceLifecycle： 防止在不存在的 namespace 上创建对象，防止删除系统预置 namespace，删除namespace 时，连带删除它的所有资源对象。</li><li>LimitRanger：确保请求的资源不会超过资源所在 Namespace 的 LimitRange 的限制。</li><li>ServiceAccount： 实现了自动化添加 ServiceAccount。</li><li>ResourceQuota：确保请求的资源不会超过资源的 ResourceQuota 限制。</li></ul><h1 id="ServiceAccount"><a href="#ServiceAccount" class="headerlink" title="ServiceAccount"></a>ServiceAccount</h1><p>Service account是为了方便Pod里面的进程调用Kubernetes API或其他外部服务而设计的。它与User account不同</p><ul><li>User account是为人设计的，而service account则是为Pod中的进程调用Kubernetes API而设计；</li><li>User account是跨namespace的，而service account则是仅局限它所在的namespace；</li><li>每个namespace都会自动创建一个default service account</li><li>Token controller检测service account的创建，并为它们创建<a href="https://www.kubernetes.org.cn/secret" target="_blank" rel="noopener">secret</a></li><li><p>开启ServiceAccount Admission Controller后</p><ul><li>每个Pod在创建后都会自动设置spec.serviceAccount为default（除非指定了其他ServiceAccout）</li><li>验证Pod引用的service account已经存在，否则拒绝创建</li><li>如果Pod没有指定ImagePullSecrets，则把service account的ImagePullSecrets加到Pod中</li><li>每个container启动后都会挂载该service account的token和ca.crt到/var/run/secrets/kubernetes.io/serviceaccount/</li></ul><p>当创建 pod 的时候，如果没有指定一个 service account，系统会自动在与该pod 相同的 namespace 下为其指派一个default service account。而pod和apiserver之间进行通信的账号，称为serviceAccountName。</p></li></ul><p>当创建 pod 的时候，如果没有指定一个 service account，系统会自动在与该pod 相同的 namespace 下为其指派一个default service account。而pod和apiserver之间进行通信的账号，称为serviceAccountName。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl get pods nginx-geekbang-76579f4dfd-54d85 -o yaml |grep serviceAccountName</span></span><br><span class="line">  serviceAccountName: default</span><br><span class="line">[root@master ~]<span class="comment"># kubectl describe pods nginx-geekbang-76579f4dfd-54d85  |egrep "Mounts|from"</span></span><br><span class="line">    Mounts:</span><br><span class="line">      /etc/nginx/conf.d/ from www-conf (rw)</span><br><span class="line">      /usr/share/nginx/html from www-data (rw)</span><br><span class="line">      /var/run/secrets/kubernetes.io/serviceaccount from default-token-wqbdz (ro)</span><br></pre></td></tr></table></figure><p>从上面可以看到每个Pod无论定义与否都会有个存储卷，这个存储卷为<code>default-token-***</code>的token令牌，这就是pod和serviceaccount认证信息。通过secret进行定义，由于认证信息属于敏感信息，所以需要保存在secret资源当中，并以存储卷的方式挂载到Pod当中。查看sa以及secret：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl get sa</span></span><br><span class="line">NAME      SECRETS   AGE</span><br><span class="line">default   1         128d</span><br><span class="line">[root@master ~]<span class="comment"># kubectl get secrets default-token-wqbdz</span></span><br><span class="line">NAME                  TYPE                                  DATA   AGE</span><br><span class="line">default-token-wqbdz   kubernetes.io/service-account-token   3      128d</span><br></pre></td></tr></table></figure><p>而默认的service account 仅仅只能获取当前Pod自身的相关属性，无法观察到其他名称空间Pod的相关属性信息。</p><p>从上可以得知，SA是会挂载到POD里面的，那具体是哪个位置呢？如下：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl exec -it nginx-geekbang-76579f4dfd-54d85 -- bash</span></span><br><span class="line">root@nginx-geekbang-76579f4dfd-54d85:/<span class="comment"># ls -l /run/secrets/kubernetes.io/serviceaccount/</span></span><br><span class="line">total 0</span><br><span class="line">lrwxrwxrwx 1 root root 13 Jun  7 08:14 ca.crt -&gt; ..data/ca.crt</span><br><span class="line">lrwxrwxrwx 1 root root 16 Jun  7 08:14 namespace -&gt; ..data/namespace</span><br><span class="line">lrwxrwxrwx 1 root root 12 Jun  7 08:14 token -&gt; ..data/token</span><br><span class="line">root@nginx-geekbang-76579f4dfd-54d85:/<span class="comment"># cat /run/secrets/kubernetes.io/serviceaccount/namespace |xargs</span></span><br><span class="line">default</span><br></pre></td></tr></table></figure><h2 id="创建方法"><a href="#创建方法" class="headerlink" title="创建方法"></a>创建方法</h2><p>SA的创建方法非常简单，直接 <code>kubectl create serviceaccount admin</code> 即可。我们在POD创建时可以使用<code>spec.serviceAccountName</code> 字段中将name设置为您想要用的 service account 名字即可。在 pod 创建之初 service account 就必须已经存在，否则创建将被拒绝。需要注意的是不能更新已创建的 pod 的 service account。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl create serviceaccount admin</span></span><br><span class="line">serviceaccount/admin created</span><br><span class="line">[root@master ~]<span class="comment"># kubectl get sa</span></span><br><span class="line">NAME      SECRETS   AGE</span><br><span class="line">admin     1         1s</span><br><span class="line">default   1         128d</span><br><span class="line">[root@master ~]<span class="comment"># kubectl describe sa admin</span></span><br><span class="line">Name:                admin</span><br><span class="line">Namespace:           default</span><br><span class="line">Labels:              &lt;none&gt;</span><br><span class="line">Annotations:         &lt;none&gt;</span><br><span class="line">Image pull secrets:  &lt;none&gt;</span><br><span class="line">Mountable secrets:   admin-token-5lpxh</span><br><span class="line">Tokens:              admin-token-5lpxh</span><br><span class="line">Events:              &lt;none&gt;</span><br></pre></td></tr></table></figure><h1 id="RBAC"><a href="#RBAC" class="headerlink" title="RBAC"></a>RBAC</h1><p>RBAC 引入了 4 个新的顶级资源对象：Role、ClusterRole、RoleBinding、ClusterRoleBinding，4 种对象类型均可以通过 kubectl 与 API 操作。</p><p><img src="/assets/1591533330072.png" alt></p><p><strong>在RBAC API中，一个角色包含了一套表示一组权限的规则。 权限以纯粹的累加形式累积（没有”否定”的规则）。 角色可以由命名空间（namespace）内的Role对象定义，而整个Kubernetes集群范围内有效的角色则通过ClusterRole对象实现。</strong></p><h2 id="Role与ClusterRole"><a href="#Role与ClusterRole" class="headerlink" title="Role与ClusterRole"></a>Role与ClusterRole</h2><p>在RBAC API中，一个角色包含了一套表示一组权限的规则。 权限以纯粹的累加形式累积（没有”否定”的规则）。 角色可以由名字空间（namespace）内的Role对象定义，而整个Kubernetes集群范围内有效的角色则通过ClusterRole对象实现。</p><p><code>一个Role对象只能用于授予对某一单一名字空间中资源的访问权限。</code></p><p>以下示例描述了”default”名字空间中的一个<code>Role</code>对象的定义，用于授予对pod的读访问权限：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">default</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">pod-reader</span></span><br><span class="line"><span class="attr">rules:</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">[""]</span> <span class="comment"># "" indicates the core API group</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["pods"]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"watch"</span><span class="string">,</span> <span class="string">"list"</span><span class="string">]</span></span><br></pre></td></tr></table></figure><p><code>ClusterRole对象可以授予与Role对象相同的权限，但属于集群级别的。</code>可以用于：</p><ul><li>集群级别的资源控制，如node的访问权限</li><li>非资源型的访问权限，如endpoints “/healthz”</li><li>所有命名空间的资源控制。即控制所有的namespace</li></ul><p>下面示例中的<code>ClusterRole</code>定义可用于授予用户对某一特定名字空间，或者所有名字空间中的secret的读访问权限：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">ClusterRole</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line">  <span class="comment"># 鉴于ClusterRole是集群范围对象，所以这里不需要定义"namespace"字段</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">secret-reader</span></span><br><span class="line"><span class="attr">rules:</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">[""]</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["secrets"]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"watch"</span><span class="string">,</span> <span class="string">"list"</span><span class="string">]</span></span><br></pre></td></tr></table></figure><h2 id="RoleBinding与ClusterRoleBinding"><a href="#RoleBinding与ClusterRoleBinding" class="headerlink" title="RoleBinding与ClusterRoleBinding"></a>RoleBinding与ClusterRoleBinding</h2><p><code>角色绑定将一个角色中定义的各种权限授予一个或者一组用户。</code>角色绑定包含了一组相关主体（即subject, 包括用户——User、用户组——Group、或者服务账户——Service Account）以及对被授予角色的引用。 在名字空间中可以通过RoleBinding对象授予权限，而集群范围的权限授予则通过ClusterRoleBinding对象完成。</p><p><code>RoleBinding可以引用在同一名字空间内定义的</code>Role对象。下面示例中定义的RoleBinding对象在”default”名字空间中将”pod-reader”角色授予用户”jane”。 这一授权将允许用户”jane”从”default”名字空间中读取pod。</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 以下角色绑定定义将允许用户"jane"，绑定pod-reader这个role</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">RoleBinding</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1beta1</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">read-pods</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">default</span></span><br><span class="line"><span class="attr">subjects:</span></span><br><span class="line"><span class="attr">- kind:</span> <span class="string">User</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">jane</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br><span class="line"><span class="attr">roleRef:</span></span><br><span class="line"><span class="attr">  kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">pod-reader</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br></pre></td></tr></table></figure><p><code>RoleBinding</code>对象也可以绑定一个<code>ClusterRole</code>对象。例如，尽管下面示例中的<code>RoleBinding</code>引用的是一个<code>ClusterRole</code>对象，但是用户”dave”（即角色绑定主体）还是只能读取”development” 名字空间中的secret（即<code>RoleBinding</code>所在的名字空间）。</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 以下角色绑定允许用户"dave"读取"development"名字空间中的secret。</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">RoleBinding</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1beta1</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">read-secrets</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">development</span> <span class="comment"># 这里表明仅授权读取"development"名字空间中的资源。</span></span><br><span class="line"><span class="attr">subjects:</span></span><br><span class="line"><span class="attr">- kind:</span> <span class="string">User</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">dave</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br><span class="line"><span class="attr">roleRef:</span></span><br><span class="line"><span class="attr">  kind:</span> <span class="string">ClusterRole</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">secret-reader</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br></pre></td></tr></table></figure><p>如果dave想要控制整个集群，那就上面的namespace删除掉就有了整个集群 <code>secret-reader</code>这个clusterRole的权限。</p><h2 id="Subject角色绑定主体"><a href="#Subject角色绑定主体" class="headerlink" title="Subject角色绑定主体"></a>Subject角色绑定主体</h2><p><code>RoleBinding</code>或者<code>ClusterRoleBinding</code>将角色绑定到角色绑定主体（Subject）。 角色绑定主体可以是用户组（Group）、用户（User）或者服务账户（Service Accounts）。</p><ul><li>用户名由字符串表示。可以是纯粹的用户名，例如”alice”、电子邮件风格的名字，如 “bob@example.com” 或者是用字符串表示的数字id。对于用户名，RBAC授权系统不要求任何特定的格式。然而，前缀<code>system:</code>是 为Kubernetes系统使用而保留的，所以管理员应该确保用户名不会意外地包含这个前缀。</li><li>Kubernetes中的用户组信息由授权模块提供。用户组与用户一样由字符串表示。Kubernetes对用户组 字符串没有格式要求，但前缀<code>system:</code>同样是被系统保留的。</li><li>Service Accounts拥有包含 <code>system:serviceaccount:</code>前缀的用户名，并属于拥有<code>system:serviceaccounts:</code>前缀的用户组。</li></ul><p>一个名为”alice@example.com”的用户：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">subjects:</span></span><br><span class="line"><span class="attr">- kind:</span> <span class="string">User</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">"alice@example.com"</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br></pre></td></tr></table></figure><h2 id="对资源的控制引用"><a href="#对资源的控制引用" class="headerlink" title="对资源的控制引用"></a>对资源的控制引用</h2><p>大多数资源由代表其名字的字符串表示，例如”pods”，就像它们出现在相关API endpoint的URL中一样。然而，有一些Kubernetes API还 包含了”子资源”，比如pod的logs。在Kubernetes中，pod logs endpoint的URL格式为：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">GET /api/v1/namespaces/&#123;namespace&#125;/pods/&#123;name&#125;/log</span><br></pre></td></tr></table></figure><p>在这种情况下，”pods”是名字空间资源，而”log”是pods的子资源。为了在RBAC角色中表示出这一点，我们需要使用斜线来划分资源 与子资源。如果需要角色绑定主体读取pods以及pod log，您需要定义以下角色：</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1beta1</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">default</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">pod-and-pod-logs-reader</span></span><br><span class="line"><span class="attr">rules:</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">[""]</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["pods",</span> <span class="string">"pods/log"</span><span class="string">]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"list"</span><span class="string">]</span></span><br></pre></td></tr></table></figure><p>这里的意思是rules的规则设置。</p><ul><li><code>apiGroups</code>定义可以使用api的版本，为空表示核心版本，如 <code>[&quot;extensions&quot;, &quot;apps&quot;]</code>允许读写在”extensions”和”apps”的API。</li><li><code>resources</code>表示对哪些资源进行控制，可以为pods、deployments、nodes、secrets。。。。</li><li><code>verbs</code>表示允许的动作。如<code>&quot;get&quot;, &quot;list&quot;, &quot;watch&quot;, &quot;create&quot;, &quot;update&quot;, &quot;patch&quot;, &quot;delete&quot;</code>等。</li><li><code>resourceNames</code>表示一个资源的实例名，如一个pod的名字为abc，那加上这个，那这个rule就表示只对这个pod生效。</li></ul><h1 id="实例"><a href="#实例" class="headerlink" title="实例"></a>实例</h1><h2 id="普通用户管理集群"><a href="#普通用户管理集群" class="headerlink" title="普通用户管理集群"></a>普通用户管理集群</h2><p>在实际生产环境中，有很可能是多个部门共用同一套K8S的环境，这就需要做权限的隔离，以下以devuser为例，创建一个普通用户只能管理namespace=dev这个环境。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 下载证书生成工具</span></span><br><span class="line">wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64</span><br><span class="line">mv cfssl_linux-amd64 /usr/<span class="built_in">local</span>/bin/cfssl</span><br><span class="line">wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64</span><br><span class="line">mv cfssljson_linux-amd64 /usr/<span class="built_in">local</span>/bin/cfssljson</span><br><span class="line">wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64</span><br><span class="line">mv cfssl-certinfo_linux-amd64 /usr/<span class="built_in">local</span>/bin/cfssl-certinfo</span><br><span class="line"></span><br><span class="line">cat &gt; /root/devuser-csr.json &lt;&lt;EOF</span><br><span class="line">&#123;</span><br><span class="line">  <span class="string">"CN"</span>: <span class="string">"devuser"</span>,</span><br><span class="line">  <span class="string">"hosts"</span>: [],</span><br><span class="line">  <span class="string">"key"</span>: &#123;</span><br><span class="line">     <span class="string">"algo"</span>: <span class="string">"rsa"</span>,</span><br><span class="line">     <span class="string">"size"</span>: 2048</span><br><span class="line">  &#125;,</span><br><span class="line">  <span class="string">"names"</span>: [</span><br><span class="line">  &#123;</span><br><span class="line">     <span class="string">"C"</span>: <span class="string">"CN"</span>,</span><br><span class="line">     <span class="string">"ST"</span>: <span class="string">"BeiJing"</span>,</span><br><span class="line">     <span class="string">"L"</span>: <span class="string">"BeiJing"</span>,</span><br><span class="line">     <span class="string">"O"</span>: <span class="string">"k8s"</span>,</span><br><span class="line">     <span class="string">"OU"</span>: <span class="string">"System"</span></span><br><span class="line">  &#125;</span><br><span class="line">  ]</span><br><span class="line">&#125;</span><br><span class="line">EOF</span><br><span class="line"></span><br><span class="line"><span class="built_in">cd</span> /etc/kubernetes/pki/</span><br><span class="line">cfssl gencert -ca=ca.crt -ca-key=ca.key -profile=kubernetes /root/devuser-csr.json | cfssljson -bare devuser</span><br><span class="line"></span><br><span class="line"><span class="comment"># 设置集群参数</span></span><br><span class="line"><span class="built_in">export</span> KUBE_APISERVER=<span class="string">"https://192.168.1.60:6443"</span></span><br><span class="line">kubectl config <span class="built_in">set</span>-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=<span class="literal">true</span> --server=<span class="variable">$&#123;KUBE_APISERVER&#125;</span> --kubeconfig=devuser.kubeconfig</span><br><span class="line"></span><br><span class="line"><span class="comment"># 设置客户端认证参数</span></span><br><span class="line">kubectl config <span class="built_in">set</span>-credentials devuser --client-certificate=/etc/kubernetes/pki/devuser.pem --client-key=/etc/kubernetes/pki/devuser-key.pem --embed-certs=<span class="literal">true</span> --kubeconfig=devuser.kubeconfig</span><br><span class="line"></span><br><span class="line"><span class="comment"># 设置上下文参数</span></span><br><span class="line">kubectl create namespace dev</span><br><span class="line">kubectl config <span class="built_in">set</span>-context kubernetes --cluster=kubernetes --user=devuser --namespace=dev --kubeconfig=devuser.kubeconfig</span><br><span class="line"></span><br><span class="line"><span class="comment"># 设置默认上下文</span></span><br><span class="line">kubectl create rolebinding devuser-admin-binding --clusterrole=admin --user=devuser --namespace=dev</span><br><span class="line"></span><br><span class="line"><span class="comment"># 创建用户</span></span><br><span class="line">useradd devuser</span><br><span class="line">cp -f ./devuser.kubeconfig /home/devuser/.kube/config</span><br><span class="line">chown devuser:devuser /home/devuser/.kube/config</span><br><span class="line"></span><br><span class="line"><span class="comment">#使用devuser用户登陆系统</span></span><br><span class="line">kubectl config use-context kubernetes</span><br></pre></td></tr></table></figure><p>配置完成之后，查看默认的配置：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[devuser@master ~]$ kubectl config view</span><br><span class="line">apiVersion: v1</span><br><span class="line">clusters:</span><br><span class="line">- cluster:</span><br><span class="line">    certificate-authority-data: DATA+OMITTED</span><br><span class="line">    server: https://192.168.1.60:6443</span><br><span class="line">  name: kubernetes</span><br><span class="line">contexts:</span><br><span class="line">- context:</span><br><span class="line">    cluster: kubernetes</span><br><span class="line">    namespace: dev</span><br><span class="line">    user: devuser</span><br><span class="line">  name: kubernetes</span><br><span class="line">current-context: kubernetes</span><br><span class="line">kind: Config</span><br><span class="line">preferences: &#123;&#125;</span><br><span class="line">users:</span><br><span class="line">- name: devuser</span><br><span class="line">  user:</span><br><span class="line">    client-certificate-data: REDACTED</span><br><span class="line">    client-key-data: REDACTED</span><br><span class="line">[devuser@master ~]$ kubectl get pods</span><br><span class="line">No resources found <span class="keyword">in</span> dev namespace.</span><br></pre></td></tr></table></figure><h2 id="创建一个只能访问某个namespace的用户"><a href="#创建一个只能访问某个namespace的用户" class="headerlink" title="创建一个只能访问某个namespace的用户"></a>创建一个只能访问某个namespace的用户</h2><h3 id="第1步：创建用户凭证"><a href="#第1步：创建用户凭证" class="headerlink" title="第1步：创建用户凭证"></a>第1步：创建用户凭证</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 给用户 fdm 创建一个私钥</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># openssl genrsa -out fdm.key 2048</span></span><br><span class="line">Generating RSA private key, 2048 bit long modulus</span><br><span class="line">...........................................................................................................................................................................+++</span><br><span class="line">.........................................................................................................................................................+++</span><br><span class="line">e is 65537 (0x10001)</span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># ll</span></span><br><span class="line">total 4</span><br><span class="line">-rw-r--r-- 1 root root 1675 Feb 29 23:13 fdm.key</span><br><span class="line"></span><br><span class="line"><span class="comment"># 使用我们刚刚创建的私钥创建一个证书签名请求文件fdm.csr。</span></span><br><span class="line"><span class="comment"># -subj参数中指定用户名和组，CN表示用户名，O表示组</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># openssl req -new -key fdm.key -out fdm.csr -subj "/CN=fdm/O=wangsu"</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># ll</span></span><br><span class="line">total 8</span><br><span class="line">-rw-r--r-- 1 root root  907 Feb 29 23:15 fdm.csr</span><br><span class="line">-rw-r--r-- 1 root root 1675 Feb 29 23:13 fdm.key</span><br><span class="line"></span><br><span class="line"><span class="comment"># 使用k8s的CA证书，生成最终的证书文件fdm.crt，我们这里设置证书的有效期为3650天</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># openssl x509 -req -in fdm.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out fdm.crt -days 3650</span></span><br><span class="line">Signature ok</span><br><span class="line">subject=/CN=fdm/O=wangsu</span><br><span class="line">Getting CA Private Key</span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># ll</span></span><br><span class="line">total 12</span><br><span class="line">-rw-r--r-- 1 root root  993 Feb 29 23:16 fdm.crt</span><br><span class="line">-rw-r--r-- 1 root root  907 Feb 29 23:15 fdm.csr</span><br><span class="line">-rw-r--r-- 1 root root 1675 Feb 29 23:13 fdm.key</span><br><span class="line"></span><br><span class="line"><span class="comment"># 查看证书的有效期</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># openssl x509 -in fdm.crt -text |grep Not</span></span><br><span class="line">            Not Before: Feb 29 15:16:49 2020 GMT</span><br><span class="line">            Not After : Feb 26 15:16:49 2030 GMT</span><br></pre></td></tr></table></figure><p>现在我们可以使用刚刚创建的证书文件和私钥文件在集群中创建新的凭证和上下文(Context)</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@master user_login_kube-systecm]<span class="comment"># kubectl config set-credentials fdm --client-certificate=fdm.crt --client-key=fdm.key</span></span><br><span class="line">User <span class="string">"fdm"</span> <span class="built_in">set</span>.</span><br><span class="line"><span class="comment"># 创建context</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># kubectl config set-context fdm-context --cluster=kubernetes --namespace kube-system --user=fdm</span></span><br><span class="line">Context <span class="string">"fdm-context"</span> created.</span><br><span class="line"></span><br><span class="line"><span class="comment"># 测试报错，因为没有为该用户定义任何操作的权限</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># kubectl get pods --context=fdm-context</span></span><br><span class="line">Error from server (Forbidden): pods is forbidden: User <span class="string">"fdm"</span> cannot list resource <span class="string">"pods"</span> <span class="keyword">in</span> API group <span class="string">""</span> <span class="keyword">in</span> the namespace <span class="string">"kube-system"</span></span><br></pre></td></tr></table></figure><h3 id="第2步：创建角色"><a href="#第2步：创建角色" class="headerlink" title="第2步：创建角色"></a>第2步：创建角色</h3><p>用户创建完成后，接下来就需要给该用户添加操作权限，我们来定义一个<code>YAML</code>文件，创建一个允许用户操作 Deployment、Pod、ReplicaSets 的角色。</p><figure class="highlight yaml"><figcaption><span>role.yaml</span></figcaption><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-role</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">kube-system</span></span><br><span class="line"><span class="attr">rules:</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">["",</span> <span class="string">"extensions"</span><span class="string">,</span> <span class="string">"apps"</span><span class="string">]</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["deployments",</span> <span class="string">"replicasets"</span><span class="string">,</span> <span class="string">"pods"</span><span class="string">]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"list"</span><span class="string">,</span> <span class="string">"watch"</span><span class="string">,</span> <span class="string">"create"</span><span class="string">,</span> <span class="string">"update"</span><span class="string">,</span> <span class="string">"patch"</span><span class="string">,</span> <span class="string">"delete"</span><span class="string">]</span> <span class="comment"># 也可以使用['*']</span></span><br></pre></td></tr></table></figure><p>其中<code>Pod</code>属于 core 这个 API Group，在<code>YAML</code>中用空字符就可以，而<code>Deployment</code>属于 apps 这个 API Group，<code>ReplicaSets</code>属于<code>extensions</code>。</p><h3 id="第3步：创建角色权限绑定"><a href="#第3步：创建角色权限绑定" class="headerlink" title="第3步：创建角色权限绑定"></a>第3步：创建角色权限绑定</h3><figure class="highlight yaml"><figcaption><span>RoleBinding.yaml</span></figcaption><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">RoleBinding</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-rolebinding</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">kube-system</span></span><br><span class="line"><span class="attr">subjects:</span></span><br><span class="line"><span class="attr">- kind:</span> <span class="string">User</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">""</span></span><br><span class="line"><span class="attr">roleRef:</span></span><br><span class="line"><span class="attr">  kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-role</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">""</span></span><br></pre></td></tr></table></figure><h3 id="第4步-测试"><a href="#第4步-测试" class="headerlink" title="第4步. 测试"></a>第4步. 测试</h3><p>现在我们应该可以上面的<code>haimaxy-context</code>上下文来操作集群了：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 默认查看的是kube-system的这个namespace空间</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># kubectl get pods --context=fdm-context</span></span><br><span class="line">NAME                             READY   STATUS    RESTARTS   AGE</span><br><span class="line">coredns-6955765f44-2nq69         1/1     Running   3          29d</span><br><span class="line">coredns-6955765f44-x9ztm         1/1     Running   3          29d</span><br><span class="line">etcd-master                      1/1     Running   3          29d</span><br><span class="line">kube-apiserver-master            1/1     Running   3          29d</span><br><span class="line">kube-controller-manager-master   1/1     Running   3          29d</span><br><span class="line">kube-proxy-46vfx                 1/1     Running   5          29d</span><br><span class="line">kube-proxy-b2tpn                 1/1     Running   0          9h</span><br><span class="line">kube-proxy-qfch9                 1/1     Running   3          29d</span><br><span class="line">kube-scheduler-master            1/1     Running   3          29d</span><br><span class="line">weave-net-7mtqr                  2/2     Running   0          9h</span><br><span class="line">weave-net-hkqx7                  2/2     Running   27         29d</span><br><span class="line">weave-net-mg9sz                  2/2     Running   9          29d</span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment">#</span></span><br><span class="line"><span class="comment"># 其他namespace空间是没有权限的</span></span><br><span class="line">[root@master user_login_kube-systecm]<span class="comment"># kubectl get pods --context=fdm-context -n mynamespace</span></span><br><span class="line">Error from server (Forbidden): pods is forbidden: User <span class="string">"fdm"</span> cannot list resource <span class="string">"pods"</span> <span class="keyword">in</span> API group <span class="string">""</span> <span class="keyword">in</span> the namespace <span class="string">"mynamespace"</span></span><br></pre></td></tr></table></figure><h2 id="创建一个只能访问某个namespace的ServiceAccount"><a href="#创建一个只能访问某个namespace的ServiceAccount" class="headerlink" title="创建一个只能访问某个namespace的ServiceAccount"></a>创建一个只能访问某个namespace的ServiceAccount</h2><h3 id="创建sa"><a href="#创建sa" class="headerlink" title="创建sa"></a>创建sa</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl create sa fdm-sa -n kube-system</span></span><br><span class="line">serviceaccount/fdm-sa created</span><br><span class="line">[root@master ~]<span class="comment"># kubectl get sa -n kube-system fdm-sa</span></span><br><span class="line">NAME     SECRETS   AGE</span><br><span class="line">fdm-sa   1         18s</span><br></pre></td></tr></table></figure><h3 id="创建role以及rolebinding"><a href="#创建role以及rolebinding" class="headerlink" title="创建role以及rolebinding"></a>创建role以及rolebinding</h3><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-sa-role</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">kube-system</span></span><br><span class="line"><span class="attr">rules:</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">[""]</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["pods"]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"watch"</span><span class="string">,</span> <span class="string">"list"</span><span class="string">]</span></span><br><span class="line"><span class="attr">- apiGroups:</span> <span class="string">["apps"]</span></span><br><span class="line"><span class="attr">  resources:</span> <span class="string">["deployments"]</span></span><br><span class="line"><span class="attr">  verbs:</span> <span class="string">["get",</span> <span class="string">"list"</span><span class="string">,</span> <span class="string">"watch"</span><span class="string">,</span> <span class="string">"create"</span><span class="string">,</span> <span class="string">"update"</span><span class="string">,</span> <span class="string">"patch"</span><span class="string">,</span> <span class="string">"delete"</span><span class="string">]</span></span><br><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">rbac.authorization.k8s.io/v1beta1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">RoleBinding</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-sa-rolebinding</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">kube-system</span></span><br><span class="line"><span class="attr">roleRef:</span></span><br><span class="line"><span class="attr">  apiGroup:</span> <span class="string">rbac.authorization.k8s.io</span></span><br><span class="line"><span class="attr">  kind:</span> <span class="string">Role</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-sa-role</span></span><br><span class="line"><span class="attr">subjects:</span></span><br><span class="line"><span class="attr">- kind:</span> <span class="string">ServiceAccount</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">fdm-sa</span></span><br><span class="line"><span class="attr">  namespace:</span> <span class="string">kube-system</span></span><br></pre></td></tr></table></figure><h3 id="登陆验证测试"><a href="#登陆验证测试" class="headerlink" title="登陆验证测试"></a>登陆验证测试</h3><p>先找到fdm-sa这个sa的token。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># kubectl get secret -n kube-system fdm-sa-token-f6vpc</span></span><br><span class="line">NAME                 TYPE                                  DATA   AGE</span><br><span class="line">fdm-sa-token-f6vpc   kubernetes.io/service-account-token   3      5m9s</span><br><span class="line">[root@master ~]<span class="comment">#</span></span><br><span class="line">[root@master ~]<span class="comment"># kubectl describe secret -n kube-system fdm-sa-token-f6vpc</span></span><br><span class="line">Name:         fdm-sa-token-f6vpc</span><br><span class="line">Namespace:    kube-system</span><br><span class="line">Labels:       &lt;none&gt;</span><br><span class="line">Annotations:  kubernetes.io/service-account.name: fdm-sa</span><br><span class="line">              kubernetes.io/service-account.uid: 5857f81b-a918-4fe8-b74f-21a3ba0022db</span><br><span class="line"></span><br><span class="line">Type:  kubernetes.io/service-account-token</span><br><span class="line"></span><br><span class="line">Data</span><br><span class="line">====</span><br><span class="line">ca.crt:     1025 bytes</span><br><span class="line">namespace:  11 bytes</span><br><span class="line">token:      eyJhbGciOiJSUzI1NiIsImtpZCI6Ino0bHA4S2FtengxV01SREJLTUUtNkc3TFd4N25HbTFoWTBDLXZGUWQ5VTgifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJmZG0tc2EtdG9rZW4tZjZ2cGMiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZmRtLXNhIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNTg1N2Y4MWItYTkxOC00ZmU4LWI3NGYtMjFhM2JhMDAyMmRiIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmZkbS1zYSJ9.b67MInBbcldQJivH_b0FyBtlFbfBSn0m1r6WbU6qVRPBlesk-ixnBjuxRttFHUQIlwx04IkQFVq7Bhgr_8OCOm7CfzZ5sPhCQUFHg6Z6TYQWkDeMYlA8Kgy_758sC53r4sYLUu0c4xepS54kC94gElSQ1Y92AZCzIHxV6eGXOTSrVR3gNoH3kQKlfsNaRo1LC1sZm74_zXoVq4d3kHd9B3Q6McY_zspGiCdQNzb13ZD5QnC4S0blvUT50ED2HgZZQLyOM2YuPEKBkLFm6AtC5rOao4pNRR-vxFEMPVPv-3VYDy3UmxVnBUbcDV1-2nOFEkZx0rAjAQ_zQ7JouW_ktA</span><br></pre></td></tr></table></figure><p>用这个token去登陆Dashboard，会发现很多异常报错，这是因为当前登录用只被授权了访问 pod 和 deployment 的权限</p><p><img src="/assets/image-20200301091354174.png" alt>、</p><h1 id="参考资料"><a href="#参考资料" class="headerlink" title="参考资料"></a>参考资料</h1><blockquote><p> <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/" target="_blank" rel="noopener">https://kubernetes.io/docs/reference/access-authn-authz/rbac/</a></p><p><a href="https://k8smeetup.github.io/docs/admin/authorization/rbac/" target="_blank" rel="noopener">https://k8smeetup.github.io/docs/admin/authorization/rbac/</a></p><p><a href="https://www.qikqiak.com/post/use-rbac-in-k8s/" target="_blank" rel="noopener">Kubernetes RBAC详解–阳明</a></p><p><a href="https://www.cnblogs.com/linuxk/p/9772117.html" target="_blank" rel="noopener">Kubernetes学习之路（十八）之认证、授权和准入控制</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;简介&quot;&gt;&lt;a href=&quot;#简介&quot; class=&quot;headerlink&quot; title=&quot;简介&quot;&gt;&lt;/a&gt;简介&lt;/h1&gt;&lt;p&gt;Kubernetes 作为一个分布式集群的管理工具，保证集群的安全性是其一个重要的任务。&lt;strong&gt;API Server 是集群内部各个组件通信的中介，也是外部控制的入口。&lt;/strong&gt;所以 Kubernetes 的安全机制基本就是围绕保护 API Server 来设计的。&lt;strong&gt;Kubernetes 使用了认证（Authentication）、鉴权（Authorization）、准入控制（Admission Control）三步来保证API Server的安全。&lt;/strong&gt;&lt;/p&gt;
    
    </summary>
    
      <category term="Docker&amp;k8s" scheme="https://www.wumingx.com/categories/k8s/"/>
    
    
      <category term="kubernetes" scheme="https://www.wumingx.com/tags/kubernetes/"/>
    
  </entry>
  
  <entry>
    <title>k8s入门教程十三:statefulset控制器</title>
    <link href="https://www.wumingx.com/k8s/kubernetes-statefulset.html"/>
    <id>https://www.wumingx.com/k8s/kubernetes-statefulset.html</id>
    <published>2020-06-06T12:48:28.000Z</published>
    <updated>2020-06-07T14:00:11.564Z</updated>
    
    <content type="html"><![CDATA[<h1 id="statefulset简介"><a href="#statefulset简介" class="headerlink" title="statefulset简介"></a>statefulset简介</h1><p>statefulset也是一种POD控制器，那为什么要放在PV/PVC之后再简介呢？这是因为statefulset是必须也有持久化数据，每个POD所对应的PV都是不一样的。相对于Deployment所创建的POD是无状态的，那statefulset是属于有状态的，即可以保留POD的状态信息。其特点有：</p><ul><li>1、稳定的持久化存储，即Pod重新调度后还是能访问到相同的持久化数据，基于PVC来实现</li><li>2、稳定的网络标志，即Pod重新调度后其PodName和HostName不变，<strong>基于Headless Service（即没有Cluster IP的Service）</strong>来实现</li><li>3、有序部署，有序扩展，即Pod是有顺序的，在部署或者扩展的时候要依据定义的顺序依次依次进行（即从0到N-1，在下一个Pod运行之前所有之前的Pod必须都是Running和Ready状态），基于init containers来实现</li><li>4、有序收缩，有序删除（即从N-1到0）</li><li>5、有序的滚动更新</li></ul><a id="more"></a><p>在  <a href="/k8s/kubernetes-Serivce.html">k8s入门教程九:Service之服务发现</a> 的最后有提出了一个疑问，那就是headless service这东西有什么用？这是因为statefulset的特性所决定的：在statefulset中是要求有序的，每一个pod的名称必须是固定的。当节点挂了，重建之后的标识符是不变的，每一个节点的节点名称是不能改变的。pod名称是作为pod识别的唯一标识符，必须保证其标识符的稳定并且唯一。<code>为了实现标识符的稳定，这时候就需要一个headless service 解析直达到pod，还需要给pod配置一个唯一的名称。</code></p><p><em>Pod名称、PVC和PV关系图如下：</em><br><img src="/assets/1349539-20190307152625497-1629030475.png" alt></p><h2 id="NFS实例演示"><a href="#NFS实例演示" class="headerlink" title="NFS实例演示"></a>NFS实例演示</h2><h2 id="NFS前期准备"><a href="#NFS前期准备" class="headerlink" title="NFS前期准备"></a>NFS前期准备</h2><p>在一台机器上面安装NFS</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">yum install -y nfs-common nfs-utils rpcbind</span><br><span class="line">mkdir /data/nfs&#123;0..3&#125;</span><br><span class="line">chmod 755 /data/nfs&#123;0..3&#125;</span><br><span class="line">chown nfsnobody /data/nfs&#123;0..3&#125;</span><br><span class="line">cat &gt;/etc/exports &lt;&lt;EOF</span><br><span class="line">/data/nfs0 *(rw,no_root_squash,no_all_squash,sync)</span><br><span class="line">/data/nfs1 *(rw,no_root_squash,no_all_squash,sync)</span><br><span class="line">/data/nfs2 *(rw,no_root_squash,no_all_squash,sync)</span><br><span class="line">/data/nfs3 *(rw,no_root_squash,no_all_squash,sync)</span><br><span class="line">EOF</span><br><span class="line">systemctl start rpcbind &amp;&amp; systemctl <span class="built_in">enable</span> rpcbind</span><br><span class="line">systemctl start nfs &amp;&amp; systemctl <span class="built_in">enable</span> nfs</span><br></pre></td></tr></table></figure><ul><li>/data/nfs0：共享的数据目录</li><li><code>*</code>：表示任何人都有权限连接，当然也可以是一个网段，一个 IP，也可以是域名</li><li>rw：读写的权限</li><li>sync：表示文件同时写入硬盘和内存</li><li>no_root_squash：当登录 NFS 主机使用共享目录的使用者是 root 时，其权限将被转换成为匿名使用者，通常它的 UID 与 GID，都会变成 nobody 身份</li></ul><p>在另一台机器可以使用showmount测试下。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@master ~]<span class="comment"># showmount -e 192.168.1.61</span></span><br><span class="line">Export list <span class="keyword">for</span> 192.168.1.61:</span><br><span class="line">/data/nfs3 *</span><br><span class="line">/data/nfs2 *</span><br><span class="line">/data/nfs1 *</span><br><span class="line">/data/nfs0 *</span><br></pre></td></tr></table></figure><p>再实际挂载测试下：</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">mkdir /nfs&#123;0..3&#125;</span><br><span class="line">mount -t nfs 192.168.1.61:/data/nfs0 /nfs0</span><br><span class="line">date &gt;/nfs0/index.html</span><br><span class="line">umount /nfs0/</span><br></pre></td></tr></table></figure><p>没有出现异常的话，说明NFS工作是正常的了。</p><p>或者使用docker来实现NFS：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker run -d --net=host --privileged --name nfs-server katacoda/contained-nfs-server:centos7 /exports/data-0001 /exports/data-0002</span><br></pre></td></tr></table></figure><p>这时以/etc/exports这个配置，就相当于在<code>192.168.1.61</code>创建了一个共享目录，分别为：<code>/data/nfs0 /data/nfs1 /data/nfs2 /data/nfs3</code>，这几个以下需要使用到。</p><h2 id="statefulset实例"><a href="#statefulset实例" class="headerlink" title="statefulset实例"></a>statefulset实例</h2><p>创建一个NFS共享的PV，以下配置申明了一个名为pv-volume0的5G空间，挂载的目录为 <code>192.168.1.61:/data/nfs0</code>。</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">PersistentVolume</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">pv-volume0</span></span><br><span class="line"><span class="attr">spec:</span></span><br><span class="line"><span class="attr">  capacity:</span></span><br><span class="line"><span class="attr">    storage:</span> <span class="number">5</span><span class="string">Gi</span></span><br><span class="line"><span class="attr">  volumeMode:</span> <span class="string">Filesystem</span></span><br><span class="line"><span class="attr">  accessModes:</span></span><br><span class="line"><span class="bullet">    -</span> <span class="string">ReadWriteOnce</span></span><br><span class="line"><span class="attr">  persistentVolumeReclaimPolicy:</span> <span class="string">Recycle</span></span><br><span class="line"><span class="attr">  storageClassName:</span> <span class="string">slow</span></span><br><span class="line"><span class="attr">  mountOptions:</span></span><br><span class="line"><span class="bullet">    -</span> <span class="string">hard</span></span><br><span class="line"><span class="bullet">    -</span> <span class="string">nfsvers=4.1</span></span><br><span class="line"><span class="attr">  nfs:</span></span><br><span class="line"><span class="attr">    path:</span> <span class="string">/data/nfs0</span></span><br><span class="line"><span class="attr">    server:</span> <span class="number">192.168</span><span class="number">.1</span><span class="number">.61</span></span><br></pre></td></tr></table></figure><p>再创建一个StatefulSet</p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 创建一个名为nginx的headless svc，端口80为内部使用，注意跟nodePort的区别</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">Service</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">nginx</span></span><br><span class="line"><span class="attr">  labels:</span></span><br><span class="line"><span class="attr">    app:</span> <span class="string">nginx</span></span><br><span class="line"><span class="attr">spec:</span></span><br><span class="line"><span class="attr">  ports:</span></span><br><span class="line"><span class="attr">  - port:</span> <span class="number">80</span></span><br><span class="line"><span class="attr">    name:</span> <span class="string">web</span></span><br><span class="line"><span class="attr">  clusterIP:</span> <span class="string">None</span></span><br><span class="line"><span class="attr">  selector:</span></span><br><span class="line"><span class="attr">    app:</span> <span class="string">nginx</span></span><br><span class="line"><span class="meta">---</span></span><br><span class="line"><span class="comment"># 副本为3的StatefulSet，serviceName名为nginx</span></span><br><span class="line"><span class="attr">apiVersion:</span> <span class="string">apps/v1</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">StatefulSet</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">web</span></span><br><span class="line"><span class="attr">spec:</span></span><br><span class="line"><span class="attr">  selector:</span></span><br><span class="line"><span class="attr">    matchLabels:</span></span><br><span class="line"><span class="attr">      app:</span> <span class="string">nginx</span></span><br><span class="line"><span class="attr">  serviceName:</span> <span class="string">"nginx"</span></span><br><span class="line"><span class="attr">  replicas:</span> <span class="number">3</span></span><br><span class="line"><span class="attr">  template:</span></span><br><span class="line"><span class="attr">    metadata:</span></span><br><span class="line"><span class="attr">      labels:</span></span><br><span class="line"><span class="attr">        app:</span> <span class="string">nginx</span></span><br><span class="line"><span class="attr">    spec:</span></span><br><span class="line"><span class="attr">      containers:</span></span><br><span class="line"><span class="attr">      - name:</span> <span class="string">nginx</span></span><br><span class="line"><span class="attr">        image:</span> <span class="attr">nginx:1.9.1</span></span><br><span class="line"><span class="attr">        ports:</span></span><br><span class="line"><span class="attr">        - containerPort:</span> <span class="number">80</span></span><br><span class="line"><span class="attr">          name:</span> <span class="string">web</span></span><br><span class="line"><span class="attr">        volumeMounts:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">www</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/usr/share/nginx/html</span></span><br><span class="line">  <span class="comment"># volumeClainTemplate作用：当在使用statefulset创建pod时，会自动生成一个PVC，从而请求绑定一个PV，从而有自己专用的存储卷。</span></span><br><span class="line"><span class="attr">  volumeClaimTemplates:</span></span><br><span class="line"><span class="attr">  - metadata:</span></span><br><span class="line"><span class="attr">      name:</span> <span class="string">www</span></span><br><span class="line"><span class="attr">    spec:</span></span><br><span class="line"><span class="attr">      accessModes:</span> <span class="string">[</span> <span class="string">"ReadWriteOnce"</span> <span class="string">]</span></span><br><span class="line"><span class="attr">      storageClassName:</span> <span class="string">"slow"</span></span><br><span class="line"><span class="attr">      resources:</span></span><br><span class="line"><span class="attr">        requests:</span></span><br><span class="line"><span class="attr">          storage:</span> <span class="number">1</span><span class="string">Gi</span></span><br></pre></td></tr></table></figure><p>由于StatefulSet是3个副本，但是看到只创建了web-0。在创建web-1时出错了。这是由于没有可用的PV了。<code>所以一个POD是绑定一个PVC的，每个POD所对应的存储空间是不一样的。</code></p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[root@master nfs]<span class="comment"># kubectl get po -o wide</span></span><br><span class="line">NAME          READY   STATUS    RESTARTS   AGE     IP          NODE     NOMINATED NODE   READINESS GATES</span><br><span class="line">web-0         1/1     Running   0          3m44s   10.32.0.5   master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">web-1         0/1     Pending   0          42s     &lt;none&gt;      &lt;none&gt;   &lt;none&gt;           &lt;none&gt;</span><br></pre></td></tr></table></figure><p>按上面的创建PV的模板修改一下，再创建2个PV。之后就可以看到正常运行了。</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">[root@master nfs]<span class="comment"># kubectl get po -o wide</span></span><br><span class="line">NAME          READY   STATUS    RESTARTS   AGE     IP          NODE     NOMINATED NODE   READINESS GATES</span><br><span class="line">web-0         1/1     Running   0          4m2s    10.32.0.5   master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">web-1         1/1     Running   0          3m56s   10.44.0.5   node1    &lt;none&gt;           &lt;none&gt;</span><br><span class="line">web-2         1/1     Running   0          60s     10.32.0.7   master   &lt;none&gt;           &lt;none&gt;</span><br><span class="line">[root@master nfs]<span class="comment"># curl 10.32.0.5</span></span><br><span class="line">/data/nfs0 ---&gt; web-0</span><br><span class="line">[root@master nfs]<span class="comment"># curl 10.44.0.5</span></span><br><span class="line">/data/nfs1 ---&gt; web-1</span><br><span class="line">[root@master nfs]<span class="comment"># curl 10.32.0.7</span></span><br><span class="line">/data/nfs2 ---&gt; web-2</span><br></pre></td></tr></table></figure><p>查看一下PV以及PVC</p><figure class="highlight sh"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[root@master nfs]<span class="comment"># kubectl get pv</span></span><br><span class="line">NAME         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM               STORAGECLASS   REASON   AGE</span><br><span class="line">pv-volume0   5Gi        RWO            Recycle          Bound    default/www-web-0   slow                    26m</span><br><span class="line">pv-volume1   8Gi        RWO            Retain           Bound    default/www-web-1   slow                    8m10s</span><br><span class="line">pv-volume2   10Gi       RWO            Retain           Bound    default/www-web-2   slow                    7m25s</span><br><span class="line">[root@master nfs]<span class="comment"># kubectl get pvc</span></span><br><span class="line">NAME        STATUS   VOLUME       CAPACITY   ACCESS MODES   STORAGECLASS   AGE</span><br><span class="line">www-web-0   Bound    pv-volume0   5Gi        RWO            slow           11m</span><br><span class="line">www-web-1   Bound    pv-volume1   8Gi        RWO            slow           10m</span><br><span class="line">www-web-2   Bound    pv-volume2   10Gi       RWO            slow           8m3s</span><br></pre></td></tr></table></figure><p>删除pvc时，回收策略为Recycle，会自动删除文件，而Retain，会保存文件。</p><h2 id="statefulset总结"><a href="#statefulset总结" class="headerlink" title="statefulset总结"></a>statefulset总结</h2><ul><li>匹配 Pod name ( 网络标识 ) 的模式为：<code>$(statefulset名称)-$(序号)</code>，从零开始，比如上面的示例：web-0，web-1， web-2</li><li>StatefulSet 为每个 Pod 副本创建了一个 DNS 域名，这个域名的格式为： <code>$(podname).(headless server name)</code>，也就意味着服务间是通过Pod域名来通信而非 Pod IP，因为当Pod所在Node发生故障时， Pod 会 被飘移到其它 Node 上，Pod IP 会发生变化，但是 Pod 域名不会有变化</li><li>StatefulSet 使用 Headless 服务来控制 Pod 的域名，这个域名的 FQDN 为：<code>$(service name).$(namespace).svc.cluster.local</code></li><li>根据 volumeClaimTemplates，为每个 Pod 创建一个 pvc，pvc 的命名规则匹配模式： <code>(volumeClaimTemplates.name)-(pod_name)</code>，比如上面的 volumeMounts.name=www， Pod name=web-[0-2]，因此创建出来的 PVC 是 www-web-0、www-web-1、www-web-2</li><li>删除 Pod 不会删除其 pvc，手动删除 pvc 将自动释放 pv</li></ul><p>Statefulset的启停顺序：</p><ul><li>有序部署：部署StatefulSet时，如果有多个Pod副本，<code>它们会被顺序地创建（从0到N-1）</code>，并且，在下一个 Pod运行之前所有之前的Pod必须都是Running和Ready状态。</li><li>有序删除：当Pod被删除时，它们<code>被终止的顺序是从N-1到0</code>。</li><li>有序扩展：当对Pod执行扩展操作时，与部署一样，它前面的Pod必须都处于Running和Ready状态。</li></ul><h1 id="mysql实例"><a href="#mysql实例" class="headerlink" title="mysql实例"></a>mysql实例</h1><p>以下是一个Mysql的实例化的StatefulSet的ymal，原链接为： <a href="https://github.com/oracle/kubernetes-website/blob/master/docs/tasks/run-application/mysql-statefulset.yaml" target="_blank" rel="noopener">https://github.com/oracle/kubernetes-website/blob/master/docs/tasks/run-application/mysql-statefulset.yaml</a></p><figure class="highlight yaml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">apiVersion:</span> <span class="string">apps/v1beta2</span></span><br><span class="line"><span class="attr">kind:</span> <span class="string">StatefulSet</span></span><br><span class="line"><span class="attr">metadata:</span></span><br><span class="line"><span class="attr">  name:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">spec:</span></span><br><span class="line"><span class="attr">  selector:</span></span><br><span class="line"><span class="attr">    matchLabels:</span></span><br><span class="line"><span class="attr">      app:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">  serviceName:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">  replicas:</span> <span class="number">3</span></span><br><span class="line"><span class="attr">  template:</span></span><br><span class="line"><span class="attr">    metadata:</span></span><br><span class="line"><span class="attr">      labels:</span></span><br><span class="line"><span class="attr">        app:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">    spec:</span></span><br><span class="line"><span class="attr">      initContainers:</span></span><br><span class="line"><span class="attr">      - name:</span> <span class="string">init-mysql</span></span><br><span class="line"><span class="attr">        image:</span> <span class="attr">mysql:5.7</span></span><br><span class="line"><span class="attr">        command:</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">bash</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">"-c"</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">|</span></span><br><span class="line"><span class="string">          set -ex</span></span><br><span class="line"><span class="string">          # Generate mysql server-id from pod ordinal index.</span></span><br><span class="line"><span class="string">          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1</span></span><br><span class="line"><span class="string">          ordinal=$&#123;BASH_REMATCH[1]&#125;</span></span><br><span class="line"><span class="string">          echo [mysqld] &gt; /mnt/conf.d/server-id.cnf</span></span><br><span class="line"><span class="string">          # Add an offset to avoid reserved server-id=0 value.</span></span><br><span class="line"><span class="string">          echo server-id=$((100 + $ordinal)) &gt;&gt; /mnt/conf.d/server-id.cnf</span></span><br><span class="line"><span class="string">          # Copy appropriate conf.d files from config-map to emptyDir.</span></span><br><span class="line"><span class="string">          if [[ $ordinal -eq 0 ]]; then</span></span><br><span class="line"><span class="string">            cp /mnt/config-map/master.cnf /mnt/conf.d/</span></span><br><span class="line"><span class="string">          else</span></span><br><span class="line"><span class="string">            cp /mnt/config-map/slave.cnf /mnt/conf.d/</span></span><br><span class="line"><span class="string">          fi</span></span><br><span class="line"><span class="string"></span><span class="attr">        volumeMounts:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">conf</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/mnt/conf.d</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">config-map</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/mnt/config-map</span></span><br><span class="line">          </span><br><span class="line"><span class="attr">      - name:</span> <span class="string">clone-mysql</span></span><br><span class="line"><span class="attr">        image:</span> <span class="string">gcr.io/google-samples/xtrabackup:1.0</span></span><br><span class="line"><span class="attr">        command:</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">bash</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">"-c"</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">|</span></span><br><span class="line"><span class="string">          set -ex</span></span><br><span class="line"><span class="string">          # Skip the clone if data already exists.</span></span><br><span class="line"><span class="string">          [[ -d /var/lib/mysql/mysql ]] &amp;&amp; exit 0</span></span><br><span class="line"><span class="string">          # Skip the clone on master (ordinal index 0).</span></span><br><span class="line"><span class="string">          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1</span></span><br><span class="line"><span class="string">          ordinal=$&#123;BASH_REMATCH[1]&#125;</span></span><br><span class="line"><span class="string">          [[ $ordinal -eq 0 ]] &amp;&amp; exit 0</span></span><br><span class="line"><span class="string">          # Clone data from previous peer.</span></span><br><span class="line"><span class="string">          ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql</span></span><br><span class="line"><span class="string">          # Prepare the backup.</span></span><br><span class="line"><span class="string">          xtrabackup --prepare --target-dir=/var/lib/mysql</span></span><br><span class="line"><span class="string"></span><span class="attr">        volumeMounts:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">data</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/var/lib/mysql</span></span><br><span class="line"><span class="attr">          subPath:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">conf</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/etc/mysql/conf.d</span></span><br><span class="line">          </span><br><span class="line"><span class="attr">      containers:</span></span><br><span class="line"><span class="attr">      - name:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">        image:</span> <span class="attr">mysql:5.7</span></span><br><span class="line"><span class="attr">        env:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">MYSQL_ALLOW_EMPTY_PASSWORD</span></span><br><span class="line"><span class="attr">          value:</span> <span class="string">"1"</span></span><br><span class="line"><span class="attr">        ports:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">          containerPort:</span> <span class="number">3306</span></span><br><span class="line"><span class="attr">        volumeMounts:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">data</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/var/lib/mysql</span></span><br><span class="line"><span class="attr">          subPath:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">conf</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/etc/mysql/conf.d</span></span><br><span class="line"><span class="attr">        resources:</span></span><br><span class="line"><span class="attr">          requests:</span></span><br><span class="line"><span class="attr">            cpu:</span> <span class="number">500</span><span class="string">m</span></span><br><span class="line"><span class="attr">            memory:</span> <span class="number">1</span><span class="string">Gi</span></span><br><span class="line"><span class="attr">        livenessProbe:</span></span><br><span class="line"><span class="attr">          exec:</span></span><br><span class="line"><span class="attr">            command:</span> <span class="string">["mysqladmin",</span> <span class="string">"ping"</span><span class="string">]</span></span><br><span class="line"><span class="attr">          initialDelaySeconds:</span> <span class="number">30</span></span><br><span class="line"><span class="attr">          periodSeconds:</span> <span class="number">10</span></span><br><span class="line"><span class="attr">          timeoutSeconds:</span> <span class="number">5</span></span><br><span class="line"><span class="attr">        readinessProbe:</span></span><br><span class="line"><span class="attr">          exec:</span></span><br><span class="line">            <span class="comment"># Check we can execute queries over TCP (skip-networking is off).</span></span><br><span class="line"><span class="attr">            command:</span> <span class="string">["mysql",</span> <span class="string">"-h"</span><span class="string">,</span> <span class="string">"127.0.0.1"</span><span class="string">,</span> <span class="string">"-e"</span><span class="string">,</span> <span class="string">"SELECT 1"</span><span class="string">]</span></span><br><span class="line"><span class="attr">          initialDelaySeconds:</span> <span class="number">5</span></span><br><span class="line"><span class="attr">          periodSeconds:</span> <span class="number">2</span></span><br><span class="line"><span class="attr">          timeoutSeconds:</span> <span class="number">1</span></span><br><span class="line">          </span><br><span class="line"><span class="attr">      - name:</span> <span class="string">xtrabackup</span></span><br><span class="line"><span class="attr">        image:</span> <span class="string">gcr.io/google-samples/xtrabackup:1.0</span></span><br><span class="line"><span class="attr">        ports:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">xtrabackup</span></span><br><span class="line"><span class="attr">          containerPort:</span> <span class="number">3307</span></span><br><span class="line"><span class="attr">        command:</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">bash</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">"-c"</span></span><br><span class="line"><span class="bullet">        -</span> <span class="string">|</span></span><br><span class="line"><span class="string">          set -ex</span></span><br><span class="line"><span class="string">          cd /var/lib/mysql</span></span><br><span class="line"><span class="string">          # 从备份信息文件里读取MASTER_LOG_FILEM和MASTER_LOG_POS这两个字段的值，用来拼装集群初始化SQL</span></span><br><span class="line"><span class="string">          if [[ -f xtrabackup_slave_info ]]; then</span></span><br><span class="line"><span class="string">            # 如果xtrabackup_slave_info文件存在，说明这个备份数据来自于另一个Slave节点。这种情况下，</span></span><br><span class="line"><span class="string">            #XtraBackup工具在备份的时候，就已经在这个文件里自动生成了"CHANGE MASTER TO" SQL语句。</span></span><br><span class="line"><span class="string">            # 所以，我们只需要把这个文件重命名为change_master_to.sql.in，后面直接使用即可</span></span><br><span class="line"><span class="string">            mv xtrabackup_slave_info change_master_to.sql.in</span></span><br><span class="line"><span class="string">            # 所以，也就用不着xtrabackup_binlog_info了</span></span><br><span class="line"><span class="string">            rm -f xtrabackup_binlog_info</span></span><br><span class="line"><span class="string">          elif [[ -f xtrabackup_binlog_info ]]; then</span></span><br><span class="line"><span class="string">            # 如果只存在xtrabackup_binlog_inf文件，那说明备份来自于Master节点，我们就需要解析这个备份信息文件，读取所需的两个字段的值</span></span><br><span class="line"><span class="string">            [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1</span></span><br><span class="line"><span class="string">            rm xtrabackup_binlog_info</span></span><br><span class="line"><span class="string">            echo "CHANGE MASTER TO MASTER_LOG_FILE='$&#123;BASH_REMATCH[1]&#125;',\</span></span><br><span class="line"><span class="string">                  MASTER_LOG_POS=$&#123;BASH_REMATCH[2]&#125;" &gt; change_master_to.sql.in</span></span><br><span class="line"><span class="string">          fi</span></span><br><span class="line"><span class="string">          #如果只存在xtrabackup_binlog_inf文件，那说明备份来自于Master节点，我们就需要解析这个备份信息文件，读取所需的两个字段的值</span></span><br><span class="line"><span class="string">          if [[ -f change_master_to.sql.in ]]; then</span></span><br><span class="line"><span class="string">            echo "Waiting for mysqld to be ready (accepting connections)"</span></span><br><span class="line"><span class="string">            until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done</span></span><br><span class="line"><span class="string">            echo "Initializing replication from clone position"</span></span><br><span class="line"><span class="string">            # 使用change_master_to.sql.orig的内容，也是就是前面拼装的SQL，组成一个完整的初始化和启动Slave的SQL语句</span></span><br><span class="line"><span class="string">            mv change_master_to.sql.in change_master_to.sql.orig</span></span><br><span class="line"><span class="string">            mysql -h 127.0.0.1 &lt;&lt;EOF</span></span><br><span class="line"><span class="string">          $(&lt;change_master_to.sql.orig),</span></span><br><span class="line"><span class="string">            MASTER_HOST='mysql-0.mysql',</span></span><br><span class="line"><span class="string">            MASTER_USER='root',</span></span><br><span class="line"><span class="string">            MASTER_PASSWORD='',</span></span><br><span class="line"><span class="string">            MASTER_CONNECT_RETRY=10;</span></span><br><span class="line"><span class="string">          START SLAVE;</span></span><br><span class="line"><span class="string">          EOF</span></span><br><span class="line"><span class="string">          fi</span></span><br><span class="line"><span class="string">          # 使用ncat监听3307端口。它的作用是，在收到传输请求的时候，直接执行"xtrabackup --backup"命令，备份MySQL的数据并发送给请求者</span></span><br><span class="line"><span class="string">          exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \</span></span><br><span class="line"><span class="string">            "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"</span></span><br><span class="line"><span class="string"></span><span class="attr">        volumeMounts:</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">data</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/var/lib/mysql</span></span><br><span class="line"><span class="attr">          subPath:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">        - name:</span> <span class="string">conf</span></span><br><span class="line"><span class="attr">          mountPath:</span> <span class="string">/etc/mysql/conf.d</span></span><br><span class="line"><span class="attr">        resources:</span></span><br><span class="line"><span class="attr">          requests:</span></span><br><span class="line"><span class="attr">            cpu:</span> <span class="number">100</span><span class="string">m</span></span><br><span class="line"><span class="attr">            memory:</span> <span class="number">100</span><span class="string">Mi</span></span><br><span class="line"><span class="attr">      volumes:</span></span><br><span class="line"><span class="attr">      - name:</span> <span class="string">conf</span></span><br><span class="line"><span class="attr">        emptyDir:</span> <span class="string">&#123;&#125;</span></span><br><span class="line"><span class="attr">      - name:</span> <span class="string">config-map</span></span><br><span class="line"><span class="attr">        configMap:</span></span><br><span class="line"><span class="attr">          name:</span> <span class="string">mysql</span></span><br><span class="line"><span class="attr">  volumeClaimTemplates:</span></span><br><span class="line"><span class="attr">  - metadata:</span></span><br><span class="line"><span class="attr">      name:</span> <span class="string">data</span></span><br><span class="line"><span class="attr">    spec:</span></span><br><span class="line"><span class="attr">      accessModes:</span> <span class="string">["ReadWriteOnce"]</span></span><br><span class="line"><span class="attr">      resources:</span></span><br><span class="line"><span class="attr">        requests:</span></span><br><span class="line"><span class="attr">          storage:</span> <span class="number">10</span><span class="string">Gi</span></span><br></pre></td></tr></table></figure><h1 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h1><blockquote><p><a href="https://www.cnblogs.com/linuxk/p/9767736.html" target="_blank" rel="noopener">Kubernetes学习之路（十七）之statefulset控制器</a></p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;h1 id=&quot;statefulset简介&quot;&gt;&lt;a href=&quot;#statefulset简介&quot; class=&quot;headerlink&quot; title=&quot;statefulset简介&quot;&gt;&lt;/a&gt;statefulset简介&lt;/h1&gt;&lt;p&gt;statefulset也是一种POD控制器，那为什么要放在PV/PVC之后再简介呢？这是因为statefulset是必须也有持久化数据，每个POD所对应的PV都是不一样的。相对于Deployment所创建的POD是无状态的，那statefulset是属于有状态的，即可以保留POD的状态信息。其特点有：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;1、稳定的持久化存储，即Pod重新调度后还是能访问到相同的持久化数据，基于PVC来实现&lt;/li&gt;
&lt;li&gt;2、稳定的网络标志，即Pod重新调度后其PodName和HostName不变，&lt;strong&gt;基于Headless Service（即没有Cluster IP的Service）&lt;/strong&gt;来实现&lt;/li&gt;
&lt;li&gt;3、有序部署，有序扩展，即Pod是有顺序的，在部署或者扩展的时候要依据定义的顺序依次依次进行（即从0到N-1，在下一个Pod运行之前所有之前的Pod必须都是Running和Ready状态），基于init containers来实现&lt;/li&gt;
&lt;li&gt;4、有序收缩，有序删除（即从N-1到0）&lt;/li&gt;
&lt;li&gt;5、有序的滚动更新&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="Docker&amp;k8s" scheme="https://www.wumingx.com/categories/k8s/"/>
    
    
      <category term="kubernetes" scheme="https://www.wumingx.com/tags/kubernetes/"/>
    
  </entry>
  
</feed>
