<h2 style="text-align:start">一、集群机器准备</h2>    <h3 style="text-align:start">1、本文采用了4台服务器</h3>    <p style="text-align:start">安装swarm集群对服务器没要求，如果只是做练习虚拟机也行。</p>    <h3 style="text-align:start">2、ip地址规划</h3>    <p style="text-align:start">192.168.56.101(主节点) <br> 192.168.56.102 <br> 192.168.56.103 <br> 192.168.56.104</p>    <h2 style="text-align:start">3、系统要求</h2>    <p style="text-align:start">本文采用CentOS 7.2（CentOS 6估计要升级内核），其他发行版也可以。</p>    <h2 style="text-align:start">三、配置docker</h2>    <p style="text-align:start">在使用 swarm 管理集群前，需要让集群中所有的节点的 docker 服务支持远程访问。</p>    <blockquote>     <p>修改/usr/lib/systemd/system/docker.service <br> 在ExecStart=/usr/bin/docker daemon后边加上</p>     <p>-H 0.0.0.0:2375 -H unix:///var/run/docker.sock –bip=172.11.0.1/24’</p>    </blockquote>    <p style="text-align:start"><strong>注意：</strong> <br> 每台docker节点的–bip要不一样。</p>    <h2 style="text-align:start">四、安装与配置Consul</h2>    <p style="text-align:start">Consul是一个支持多数据中心分布式高可用的服务发现和配置共享的服务软件，本文用于保存集群中的一些配置信息。</p>    <h3 style="text-align:start">1、主节点启动Consul服务</h3>    <pre style="text-align:start">  <code class="language-bash">docker run -d \      -p <span style="color:rgb(174, 129, 255)">8300</span>:<span style="color:rgb(174, 129, 255)">8300</span> \      -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span> \      -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span>/udp \      -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span> \      -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span>/udp \      -p <span style="color:rgb(174, 129, 255)">8400</span>:<span style="color:rgb(174, 129, 255)">8400</span> \      -p <span style="color:rgb(174, 129, 255)">8500</span>:<span style="color:rgb(174, 129, 255)">8500</span> \      -p <span style="color:rgb(174, 129, 255)">53</span>:<span style="color:rgb(174, 129, 255)">53</span> \      -p <span style="color:rgb(174, 129, 255)">53</span>:<span style="color:rgb(174, 129, 255)">53</span>/udp \      -h consul-server \      --restart=always \      --name=consul_server progrium/consul -server -bootstrap -ui-dir=/ui -advertise <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.101</span> -client <span style="color:rgb(174, 129, 255)">0.0</span><span style="color:rgb(174, 129, 255)">.0</span><span style="color:rgb(174, 129, 255)">.0</span>  </code></pre>    <p style="text-align:start"><strong>参数解释：</strong></p>    <blockquote>     <p><strong>-p 8400:8400</strong> 映射 consul的 rpc 端口 <br> <strong>-p 8500:8500</strong> 映射consul的 UI管理端口，这样方便我们使用 UI 界面. <br> <strong>-p 53:53/udp</strong> 绑定udp 端口(53是默认 DNS端口)在 docker0 bridge 地址上. <br> <strong>-advertise</strong> 192.168.56.101 服务对外公布的 IP，必须设置为主机ip <br> <strong>-client</strong> 0.0.0.0 consul 监听的地址，缺省是localhost.</p>    </blockquote>    <h3 style="text-align:start">2、从节点启动Consul</h3>    <p style="text-align:start">每个从节点执行以下命令 <br> <strong>注意：</strong> <br> 每个主机仅1条（ip地址不同）</p>    <pre style="text-align:start">  <code class="language-bash">docker run -d \     -p <span style="color:rgb(174, 129, 255)">8300</span>:<span style="color:rgb(174, 129, 255)">8300</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span> \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8400</span>:<span style="color:rgb(174, 129, 255)">8400</span> \     -p <span style="color:rgb(174, 129, 255)">8500</span>:<span style="color:rgb(174, 129, 255)">8500</span> \     -h consul-node-<span style="color:rgb(174, 129, 255)">102</span> \      --restart=always \     --name=consul progrium/consul -server -join <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.101</span> -advertise <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.102</span> -client <span style="color:rgb(174, 129, 255)">0.0</span><span style="color:rgb(174, 129, 255)">.0</span><span style="color:rgb(174, 129, 255)">.0</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d \     -p <span style="color:rgb(174, 129, 255)">8300</span>:<span style="color:rgb(174, 129, 255)">8300</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span> \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8400</span>:<span style="color:rgb(174, 129, 255)">8400</span> \     -p <span style="color:rgb(174, 129, 255)">8500</span>:<span style="color:rgb(174, 129, 255)">8500</span> \     -h consul-node-<span style="color:rgb(174, 129, 255)">103</span> \      --restart=always \     --name=consul progrium/consul -server -join <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.101</span> -advertise <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.103</span> -client <span style="color:rgb(174, 129, 255)">0.0</span><span style="color:rgb(174, 129, 255)">.0</span><span style="color:rgb(174, 129, 255)">.0</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d \     -p <span style="color:rgb(174, 129, 255)">8300</span>:<span style="color:rgb(174, 129, 255)">8300</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span> \     -p <span style="color:rgb(174, 129, 255)">8301</span>:<span style="color:rgb(174, 129, 255)">8301</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span> \     -p <span style="color:rgb(174, 129, 255)">8302</span>:<span style="color:rgb(174, 129, 255)">8302</span>/udp \     -p <span style="color:rgb(174, 129, 255)">8400</span>:<span style="color:rgb(174, 129, 255)">8400</span> \     -p <span style="color:rgb(174, 129, 255)">8500</span>:<span style="color:rgb(174, 129, 255)">8500</span> \     -h consul-node-<span style="color:rgb(174, 129, 255)">104</span> \      --restart=always \     --name=consul progrium/consul -server -join <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.101</span> -advertise <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.104</span> -client <span style="color:rgb(174, 129, 255)">0.0</span><span style="color:rgb(174, 129, 255)">.0</span><span style="color:rgb(174, 129, 255)">.0</span>  </code></pre>    <h2 style="text-align:start">五、安装与配置docker swarm集群</h2>    <h3 style="text-align:start">1、主节点启动swarm manage</h3>    <pre style="text-align:start">  <code class="language-bash">docker run -d -p <span style="color:rgb(174, 129, 255)">2376</span>:<span style="color:rgb(174, 129, 255)">2375</span> --restart=always --name swarm-manager swarm:latest manage consul:<span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <h3 style="text-align:start">2、在每台需要加入swarm集群的docker主机中运行swarm agent</h3>    <p style="text-align:start">每个从节点执行以下命令 <br> <strong>注意：</strong> <br> 每个主机仅1条（ip地址不同）</p>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --name swarm-agent swarm:latest join --addr <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.102</span>:<span style="color:rgb(174, 129, 255)">2375</span>  consul:<span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --name swarm-agent swarm:latest join --addr <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.103</span>:<span style="color:rgb(174, 129, 255)">2375</span>  consul:<span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --name swarm-agent swarm:latest join --addr <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.104</span>:<span style="color:rgb(174, 129, 255)">2375</span>  consul:<span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <h3 style="text-align:start">3、查看集群信息</h3>    <p style="text-align:start">docker -H 192.168.56.101:2376 info <br> 以下是示例数据，每个集群的信息会不一样。</p>    <pre style="text-align:start">  <code class="language-bash">Containers: <span style="color:rgb(174, 129, 255)">22</span>   Running: <span style="color:rgb(174, 129, 255)">11</span>   Paused: <span style="color:rgb(174, 129, 255)">0</span>   Stopped: <span style="color:rgb(174, 129, 255)">11</span>  Images: <span style="color:rgb(174, 129, 255)">21</span>  Server Version: swarm/<span style="color:rgb(174, 129, 255)">1.2</span><span style="color:rgb(174, 129, 255)">.2</span>  Role: primary  Strategy: spread  Filters: health, port, containerslots, dependency, affinity, constraint  Nodes: <span style="color:rgb(174, 129, 255)">3</span>   centos72-<span style="color:rgb(174, 129, 255)">2</span>: <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.102</span>:<span style="color:rgb(174, 129, 255)">2375</span>    └ ID: HV4M:QDFT:IMN2:KJWG:B64L:V5H3:F53E:DM2N:<span style="color:rgb(174, 129, 255)">32</span>AD:KU4M:<span style="color:rgb(174, 129, 255)">2</span>SRX:ZNKG    └ Status: Healthy    └ Containers: <span style="color:rgb(174, 129, 255)">4</span>    └ Reserved CPUs: <span style="color:rgb(174, 129, 255)">0</span> / <span style="color:rgb(174, 129, 255)">1</span>    └ Reserved Memory: <span style="color:rgb(174, 129, 255)">0</span> B / <span style="color:rgb(174, 129, 255)">1.017</span> GiB    └ Labels: executiondriver=, kernelversion=<span style="color:rgb(174, 129, 255)">4.4</span><span style="color:rgb(174, 129, 255)">.10</span>-<span style="color:rgb(174, 129, 255)">1.</span>el7.elrepo.x86_64, operatingsystem=CentOS Linux <span style="color:rgb(174, 129, 255)">7</span> (Core), storagedriver=devicemapper    └ Error: (none)    └ UpdatedAt: <span style="color:rgb(174, 129, 255)">2016</span>-<span style="color:rgb(174, 129, 255)">09</span>-<span style="color:rgb(174, 129, 255)">21</span>T01:<span style="color:rgb(174, 129, 255)">58</span>:<span style="color:rgb(174, 129, 255)">50</span>Z    └ ServerVersion: <span style="color:rgb(174, 129, 255)">1.11</span><span style="color:rgb(174, 129, 255)">.1</span>   centos72-<span style="color:rgb(174, 129, 255)">3</span>: <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.103</span>:<span style="color:rgb(174, 129, 255)">2375</span>    └ ID: OI3A:J7D3:<span style="color:rgb(174, 129, 255)">5</span>RBL:ELDG:PK6T:BDYV:G76M:WTVI:XCG5:D27F:<span style="color:rgb(174, 129, 255)">3</span>MT5:MLGS    └ Status: Healthy    └ Containers: <span style="color:rgb(174, 129, 255)">8</span>    └ Reserved CPUs: <span style="color:rgb(174, 129, 255)">0</span> / <span style="color:rgb(174, 129, 255)">1</span>    └ Reserved Memory: <span style="color:rgb(174, 129, 255)">0</span> B / <span style="color:rgb(174, 129, 255)">1.017</span> GiB    └ Labels: executiondriver=, kernelversion=<span style="color:rgb(174, 129, 255)">4.4</span><span style="color:rgb(174, 129, 255)">.10</span>-<span style="color:rgb(174, 129, 255)">1.</span>el7.elrepo.x86_64, operatingsystem=CentOS Linux <span style="color:rgb(174, 129, 255)">7</span> (Core), storagedriver=devicemapper    └ Error: (none)    └ UpdatedAt: <span style="color:rgb(174, 129, 255)">2016</span>-<span style="color:rgb(174, 129, 255)">09</span>-<span style="color:rgb(174, 129, 255)">21</span>T01:<span style="color:rgb(174, 129, 255)">59</span>:<span style="color:rgb(174, 129, 255)">10</span>Z    └ ServerVersion: <span style="color:rgb(174, 129, 255)">1.11</span><span style="color:rgb(174, 129, 255)">.1</span>   centos72-<span style="color:rgb(174, 129, 255)">4</span>: <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.104</span>:<span style="color:rgb(174, 129, 255)">2375</span>    └ ID: RZXT:JDJM:NZJC:QVCP:MTWQ:JMXX:<span style="color:rgb(174, 129, 255)">2F</span>D6:SZBO:WNYK:AWNY:YRHY:ITCV    └ Status: Healthy    └ Containers: <span style="color:rgb(174, 129, 255)">10</span>    └ Reserved CPUs: <span style="color:rgb(174, 129, 255)">0</span> / <span style="color:rgb(174, 129, 255)">1</span>    └ Reserved Memory: <span style="color:rgb(174, 129, 255)">0</span> B / <span style="color:rgb(174, 129, 255)">1.017</span> GiB    └ Labels: executiondriver=, kernelversion=<span style="color:rgb(174, 129, 255)">4.4</span><span style="color:rgb(174, 129, 255)">.10</span>-<span style="color:rgb(174, 129, 255)">1.</span>el7.elrepo.x86_64, operatingsystem=CentOS Linux <span style="color:rgb(174, 129, 255)">7</span> (Core), storagedriver=devicemapper    └ Error: (none)    └ UpdatedAt: <span style="color:rgb(174, 129, 255)">2016</span>-<span style="color:rgb(174, 129, 255)">09</span>-<span style="color:rgb(174, 129, 255)">21</span>T01:<span style="color:rgb(174, 129, 255)">58</span>:<span style="color:rgb(174, 129, 255)">57</span>Z    └ ServerVersion: <span style="color:rgb(174, 129, 255)">1.11</span><span style="color:rgb(174, 129, 255)">.1</span>  Plugins:    Volume:    Network:   Kernel Version: <span style="color:rgb(174, 129, 255)">4.4</span><span style="color:rgb(174, 129, 255)">.10</span>-<span style="color:rgb(174, 129, 255)">1.</span>el7.elrepo.x86_64  Operating System: linux  Architecture: amd64  Number of Docker Hooks: <span style="color:rgb(174, 129, 255)">0</span>  CPUs: <span style="color:rgb(174, 129, 255)">3</span>  Total Memory: <span style="color:rgb(174, 129, 255)">3.052</span> GiB  Name: <span style="color:rgb(174, 129, 255)">2f</span>1bb563a4de  Registries:   </code></pre>    <h3 style="text-align:start">4、如果docker info 出现Error: ID duplicated.</h3>    <p style="text-align:start">删掉/etc/docker/key.json文件，然后重启docker服务。虚拟机里面做测试，如果虚拟机镜像是复制的，一般就会出现这个问题</p>    <h3 style="text-align:start">5、查看集群内的容器</h3>    <p style="text-align:start">docker -H 192.168.56.101:2376 ps -a</p>    <h2 style="text-align:start">六、安装shipyard（可选）</h2>    <p style="text-align:start">shipyard是一个开源的docker管理平台，支持docker集群管理</p>    <h3 style="text-align:start">1、主节点启动rethinkdb</h3>    <pre style="text-align:start">  <code class="language-bash">docker run -d <span style="color:rgb(117, 113, 94)">--restart=always --name shipyard-rethinkdb rethinkdb</span>  </code></pre>    <h3 style="text-align:start">2、主节点启动shipyard</h3>    <pre style="text-align:start">  <code class="language-bash"><span style="color:rgb(117, 113, 94)">docker</span> <span style="color:rgb(117, 113, 94)">run</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">d</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">restart=always</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">name</span> <span style="color:rgb(117, 113, 94)">shipyard</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">link</span> <span style="color:rgb(117, 113, 94)">shipyard</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">rethinkdb:rethinkdb</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">link</span> <span style="color:rgb(117, 113, 94)">swarm</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">manager:swarm</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">p</span> <span style="color:rgb(117, 113, 94)">8080:8080</span> <span style="color:rgb(117, 113, 94)">shipyard/shipyard:latest</span> <span style="color:rgb(117, 113, 94)">server</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">rethinkdb</span><span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">addr</span> <span style="color:rgb(117, 113, 94)">rethinkdb:28015</span> <span style="color:rgb(174, 129, 255)">-</span><span style="color:rgb(117, 113, 94)">d</span> <span style="color:rgb(117, 113, 94)">tcp://swarm:2375</span>  </code></pre>    <h3 style="text-align:start">3、默认账户：admin, 密码：shipyard</h3>    <h2 style="text-align:start">七、安装Registrator（可选）</h2>    <p style="text-align:start">registrator 是基于 docker 的 sock 文件来做服务发现的一个第三方的方案</p>    <p style="text-align:start">每个从节点执行以下命令 <br> <strong>注意：</strong> <br> 每个主机仅1条（ip地址不同）</p>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --privileged=<span style="color:rgb(174, 129, 255)">true</span> --name=registrator --net=host -v <span style="color:rgb(174, 129, 255)">/var/</span>run<span style="color:rgb(174, 129, 255)">/docker.sock:/</span>tmp<span style="color:rgb(174, 129, 255)">/docker.sock gliderlabs/</span>registrator -ip <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.102</span> <span style="color:rgb(230, 219, 116)">consul:</span><span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --privileged=<span style="color:rgb(174, 129, 255)">true</span> --name=registrator --net=host -v <span style="color:rgb(174, 129, 255)">/var/</span>run<span style="color:rgb(174, 129, 255)">/docker.sock:/</span>tmp<span style="color:rgb(174, 129, 255)">/docker.sock gliderlabs/</span>registrator -ip <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.103</span> <span style="color:rgb(230, 219, 116)">consul:</span><span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <pre style="text-align:start">  <code class="language-bash">docker run -d --restart=always --privileged=<span style="color:rgb(174, 129, 255)">true</span> --name=registrator --net=host -v <span style="color:rgb(174, 129, 255)">/var/</span>run<span style="color:rgb(174, 129, 255)">/docker.sock:/</span>tmp<span style="color:rgb(174, 129, 255)">/docker.sock gliderlabs/</span>registrator -ip <span style="color:rgb(174, 129, 255)">192.168</span><span style="color:rgb(174, 129, 255)">.56</span><span style="color:rgb(174, 129, 255)">.104</span> <span style="color:rgb(230, 219, 116)">consul:</span><span style="color:rgb(117, 113, 94)">//192.168.56.101:8500</span>  </code></pre>    <p style="text-align:start"><strong>参数解释：</strong></p>    <blockquote>     <p><strong>-v /var/run/docker.sock:/tmp/docker.sock</strong> 映射 docker 的 socket 到 container 中的对应位置, 这样 registration 就能监控有哪些服务启动了。 <br> <strong>-ip</strong> registration 所属的主机 IP, 一定要设置此属性, 否则服务IP会显示为127.0.0.1 <br> <strong>consul://192.168.56.101:8500</strong> consul 服务的地址</p>    </blockquote>    <p>本站原创，转载时保留以下信息：</p>    <p>本文转自：深度开源（open-open.com）</p>    <p>原文标题：<a href="http://www.open-open.com/lib/view/open1474425365773.html">Docker swarm 集群快速安装</a></p>    <p>原文地址：<a href="http://www.open-open.com/lib/view/open1474425365773.html">http://www.open-open.com/lib/view/open1474425365773.html</a></p>