ARM 容器问题
前段时间需要验证某客户的 32 位 ARM Linux 程序，由于我们只有现成的 ARM64 设备和系统，而 ARM64 兼容 32 位 armel 和 armhf，因此想到直接在 ARM64（aarch64）Linux 系统中运行 32 位 ARM（armhf）容器来进行测试。
客户程序使用的是 CentOS 7 32 位 ARM 开发环境，这里我使用 LXC 官方镜像来创建一个完整的 CentOS 7 容器系统：
~ # lxc-create -n arm -t download --dir=/var/lib/lxc/arm/rootfs -- --server mirrors.tuna.tsinghua.edu.cn/lxc-images -d centos -r 7 -a armhf
32 位 ARM 容器可以正常启动，但使用 yum 安装任何软件包会报错：
bash-4.2# yum install openssh-server
Loaded plugins: fastestmirror
Determining fastest mirrors
One of the configured repositories failed (Unknown),
and yum doesn't have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work "fix" this:
1. Contact the upstream for the repository and get them to fix the problem.
2. Reconfigure the baseurl/etc. for the repository, to point to a working
upstream. This is most often useful if you are using a newer
distribution release than is supported by the repository (and the
packages for the previous distribution release still work).
3. Run the command with the repository temporarily disabled
yum --disablerepo=<repoid> ...
4. Disable the repository permanently, so yum won't use it by default. Yum
will then just ignore the repository until you permanently enable it
again or use --enablerepo for temporary usage:
yum-config-manager --disable <repoid>
or
subscription-manager repos --disable=<repoid>
5. Configure the failing repository to be skipped, if it is unavailable.
Note that yum will try to contact the repo. when it runs most commands,
so will have to try and fail each time (and thus. yum will be be much
slower). If it is a very temporary problem though, this is often a nice
compromise:
yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
Cannot find a valid baseurl for repo: base/7/armv8l
yum 修改
上面 yum 命令的报错提示找不到 repo 为 base/7/armv8l 的地址，我们可以在容器中使用 uname 命令确认当前的系统架构为 armv8l（ARM64 内核运行 32 位 ARM 程序）：
bash-4.2# uname -m armv8l
而在主机端则可以看到真正的系统架构为 aarch64：
~ # uname -m aarch64
正是由于 yum 不能正确识别 armv8l这个特殊的系统架构，导致命令执行失败。
我们可以修改 /usr/lib/python2.7/site-packages/rpmUtils/arch.py 文件，使 yum 可以正确处理 armv8l 架构，以下为 patch 修改内容：
diff -rNp a/arch.py b/arch.py
*** a/arch.py Thu Oct 29 23:46:21 2020
--- b/arch.py Thu Oct 29 23:48:55 2020
*************** arches = {
*** 73,78 ****
--- 73,79 ----
"armv5tel": "noarch",
#arm hardware floating point
+ "armv8l": "armv7hl",
"armv7hnl": "armv7hl",
"armv7hl": "noarch",
*************** def getBaseArch(myarch=None):
*** 442,447 ****
--- 443,450 ----
return "ppc"
elif myarch.startswith("arm64"):
return "arm64"
+ elif myarch == "armv8l":
+ return "armhfp"
elif myarch.startswith("armv7h"):
return "armhfp"
elif myarch.startswith("arm"):
此时再运行 yum 就可以正确找到软件包了：
bash-4.2# yum install openssh-server Loaded plugins: fastestmirror Determining fastest mirrors * base: mirrors.bfsu.edu.cn * extras: mirrors.bfsu.edu.cn * updates: mirrors.bfsu.edu.cn base | 3.6 kB 00:00:00 extras | 2.9 kB 00:00:00 updates | 2.9 kB 00:00:00 (1/4): base/7/armhfp/group_gz | 153 kB 00:00:00 (2/4): extras/7/armhfp/primary_db | 170 kB 00:00:00 (3/4): updates/7/armhfp/primary_db | 562 kB 00:00:00 (4/4): base/7/armhfp/primary_db | 4.1 MB 00:00:00 Resolving Dependencies --> Running transaction check ---> Package openssh-server.armv7hl 0:7.4p1-21.el7 will be installed --> Processing Dependency: libwrap.so.0 for package: openssh-server-7.4p1-21.el7.armv7hl --> Running transaction check ---> Package tcp_wrappers-libs.armv7hl 0:7.6-77.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================================================================= Package Arch Version Repository Size ========================================================================================================================================================================================================================= Installing: openssh-server armv7hl 7.4p1-21.el7 base 446 k Installing for dependencies: tcp_wrappers-libs armv7hl 7.6-77.el7 base 64 k Transaction Summary ========================================================================================================================================================================================================================= Install 1 Package (+1 Dependent package) Total download size: 510 k Installed size: 1.0 M Is this ok [y/d/N]: y Downloading packages: (1/2): tcp_wrappers-libs-7.6-77.el7.armv7hl.rpm | 64 kB 00:00:00 (2/2): openssh-server-7.4p1-21.el7.armv7hl.rpm | 446 kB 00:00:00 ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.7 MB/s | 510 kB 00:00:00 Running transaction check Running transaction test Transaction check error: package tcp_wrappers-libs-7.6-77.el7.armv7hl is intended for a different architecture package openssh-server-7.4p1-21.el7.armv7hl is intended for a different architecture
不过这次轮到 rpm 报错了，提示 RPM 包的架构不相符。
rpm 修改
我们先试着手工安装 yum 下载下来但安装失败的 RPM 包，可以看到还是相同的报错：
bash-4.2# rpm -ivh /var/cache/yum/armhfp/7/base/packages/tcp_wrappers-libs-7.6-77.el7.armv7hl.rpm
Preparing... ################################# [100%]
package tcp_wrappers-libs-7.6-77.el7.armv7hl is intended for a different architecture
这也是由于 rpm 命令不能正确处理 armv8l 架构，其实对于这种 RPM 包的架构不相符的情况，我们可以为 rpm 命令增加 --ignorearch 参数强制安装：
bash-4.2# rpm -ivh --ignorearch /var/cache/yum/armhfp/7/base/packages/tcp_wrappers-libs-7.6-77.el7.armv7hl.rpm Preparing... ################################# [100%] Updating / installing... 1:tcp_wrappers-libs-7.6-77.el7 ################################# [100%]
当然我们的最终目的是要让 yum 能够自动安装软件包，因此还需要让 rpm 命令也能正确识别 armv8l 架构。
为此我们需要修改 /usr/lib/rpm/rpmrc 文件，将 armv8l 视为 armv7hnl：
diff -rNp a/rpmrc b/rpmrc *** a/rpmrc Fri Oct 30 00:15:17 2020 --- b/rpmrc Fri Oct 30 00:16:32 2020 *************** arch_compat: armv5tel: armv4tl *** 398,403 **** --- 398,404 ---- arch_compat: armv4tl: armv4l arch_compat: armv4l: armv3l arch_compat: armv3l: noarch + arch_compat: armv8l: armv7hnl arch_compat: armv7hnl: armv7hl arch_compat: armv7hl: noarch
此时我们再使用 yum 命令就可以正确下载和安装软件包了：
bash-4.2# yum install openssh-server Loaded plugins: fastestmirror Loading mirror speeds from cached hostfile * base: mirrors.bfsu.edu.cn * extras: mirrors.bfsu.edu.cn * updates: mirrors.bfsu.edu.cn Resolving Dependencies --> Running transaction check ---> Package openssh-server.armv7hl 0:7.4p1-21.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================================================================= Package Arch Version Repository Size ========================================================================================================================================================================================================================= Installing: openssh-server armv7hl 7.4p1-21.el7 base 446 k Transaction Summary ========================================================================================================================================================================================================================= Install 1 Package Total size: 446 k Installed size: 921 k Is this ok [y/d/N]: y Downloading packages: Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum. Installing : openssh-server-7.4p1-21.el7.armv7hl 1/1 Verifying : openssh-server-7.4p1-21.el7.armv7hl 1/1 Installed: openssh-server.armv7hl 0:7.4p1-21.el7 Complete!
后续安装其它软件包也都没有问题了，例如安装基础开发包：
bash-4.2# yum -y install gcc make openssl-devel glibc-static openssl-static
不用 setarch 试试么？
这个原来想过，试了一下
setarch --list显示只有 linux32 和 linux64 可配置，使用 linux64，
uname -m显示 ARM64 自己的 aarch64 了，使用 linux32，
uname -m显示现在的 armv8l，如果能直接设置或者返回一个正常表示 32 位的 armv7l 的话，那就不用改 yum 和 rpm 了。
咦，你的 setarch 不支持 armv7l 啥的啊……
看了下 CentOS 7 自带的 util-linux 太老不支持，最新 2.36 版本中的 setarch 才支持 armv7l，也是最新的改动：
setarch: add arm and aarch64 architectures to transition rules
升级成 setarch 2.36 版本确实有了：
bash-4.2# setarch --list
uname26
linux32
linux64
armv7l
armv8l
armh
arm
arm64
aarch64
然而 setarch 指定使用 armv7l 返回的仍然是 armv8l：
bash-4.2# setarch armv7l uname -m
armv8l