正文
docker镜像中文件丢失
小程序：扫一扫查出行【扫一扫了解最新限行尾号】
复制小程序
背景介绍
笔者创建了一个镜像，然后在不同的主机上启动，发现有的能启动，有的却不行，报错信息为找不到文件。
犹记得当初有人介绍，只要docker镜像做好了，拿到任何地方都可以用，此处好像不成呢，好诡异的事。
初步怀疑docker存储驱动导致的问题。
制作镜像步骤还原
1、在某一台主机做dockerfile，该主机环境如下
[root@ ~]# docker info
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 46
Server Version: 1.13.1
Storage Driver: btrfs
Build Version: Btrfs v4.9.1
Library Version: 102
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Init Binary: /usr/libexec/docker/docker-init-current
containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: 9c3c5f853ebf0ffac0d087e94daef462133b69c7 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
seccomp
WARNING: You're not using the default seccomp profile
Profile: /etc/docker/seccomp.json
Kernel Version: 3.10.0-514.16.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 40
Total Memory: 125.7 GiB
Name: cndh1325-10-12.ptmind.com
ID: 2AHA:F2JK:ECEX:U2WY:OAET:BFOS:CY77:MNAK:2Y2P:7DRS:KU72:6VI5
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)
[root@test ~]# xfs_info /
meta-data=/dev/sda5 isize=256 agcount=4, agsize=116621952 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0 spinodes=0
data = bsize=4096 blocks=466487808, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal bsize=4096 blocks=227777, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
可以看到，制作镜像的主机存储驱动为 Storage Driver: btrfs ，文件系统xfs不支持d_type。
将镜像上传镜像库，已提供使用。
2、在A主机拉取镜像，镜像不能启动
笔者是创建的elasticsearch:6.2.4版本镜像，从官方拉取，官方镜像默认包含ingest-geoip插件,在dockerfile中已经将插件删除
dockfile如下
FROM docker.elastic.co/elasticsearch/elasticsearch:6.2.4 COPY elasticsearch.yml /usr/share/elasticsearch/config/
COPY jvm.options /usr/share/elasticsearch/config/ RUN /usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-geoip --purge \
&& /usr/share/elasticsearch/bin/elasticsearch-plugin remove ingest-user-agent \
&& chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/config \
报错信息如下：
Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config/ingest-geoip
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config/ingest-geoip
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
A主机环境如下
[root@A ~]# docker info
Containers: 20
Running: 20
Paused: 0
Stopped: 0
Images: 21
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: false
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Init Binary: /usr/libexec/docker/docker-init-current
containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: 5eda6f6fd0c2884c2c8e78a6e7119e8d0ecedb77 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
seccomp
WARNING: You're not using the default seccomp profile
Profile: /etc/docker/seccomp.json
Kernel Version: 3.10.0-862.9.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 16
Total Memory: 62.74 GiB
Name: jp33e505-10-14.ptengine.com
ID: PDSY:4YPU:AUSW:HA4V:223D:YBSG:E67E:TUJF:3WUJ:EL7J:VNGH:LG3O
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)
[root@A ~]# xfs_info /
meta-data=/dev/sdc2 isize=256 agcount=4, agsize=6553600 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0 finobt=0 spinodes=0
data = bsize=4096 blocks=26214400, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal bsize=4096 blocks=12800, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
检查发现存储驱动为overlay2，文件系统xfs不支持d_type。
3、在B主机拉取镜像，镜像能启动
B主机环境如下
[root@B ~]# docker info
Containers: 21
Running: 21
Paused: 0
Stopped: 0
Images: 22
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: xfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: docker-runc runc
Default Runtime: docker-runc
Init Binary: /usr/libexec/docker/docker-init-current
containerd version: (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: df5c38a9167e87f53a9894d77c0950e178a745e7 (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: fec3683b971d9c3ef73f284f176672c44b448662 (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
seccomp
WARNING: You're not using the default seccomp profile
Profile: /etc/docker/seccomp.json
Kernel Version: 3.10.0-957.10.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 16
Total Memory: 62.75 GiB
Name: jp33e506-10-19.ptengine.com
ID: OFVX:ELT3:RNT4:OM6W:L7GC:OC47:R5QJ:XBVL:T5HW:UGIL:PJUJ:66ZO
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)
[root@B ~]# xfs_info /
meta-data=/dev/md126p3 isize=512 agcount=4, agsize=7169408 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=0 spinodes=0
data = bsize=4096 blocks=28677632, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=14002, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
现在已经很明显了，A和B唯一不同的就是文件系统xfs是否支持d_type，而官方已明确说明如果使用xfs文件系统，需要d_type的支持，即 ftype =1，具体参考 https://docs.docker.com/storage/storagedriver/overlayfs-driver/
修改A主机的d_type
mkfs.xfs -f -n ftype=1 /dev/sdg5
重启docker，一切正常。
