在 QEMU 上运行 RISC-V 32 位版本的 Linux

前言

最近中科院软件所PLCT实验室要开始基于 OpenJDK11 进行 RISC-V 32G 的移植工作，需要搭建好 RISC-V 32 位的 Linux 环境，本文就是介绍 RISC-V 32 位 Yocto版本的 Linux 环境的搭建过程。

1. 环境准备

1.1 宿主机

本次部署基于 Ubuntu 18.04.5 LTS 进行操作。

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:    18.04
Codename:   bionic

1.2 交叉工具链

构建 32-bit RV32GC 的 riscv-gnu-toolchain，以下两种方法均可以完成构建，本文采用的是第二种。

基于官方的指导构建

https://github.com/riscv/riscv-gnu-toolchain

直接下载构建好的工具链

http://plct.zdlgv5.com/riscv32_extlib.tar.gzplct.zdlgv5.com

$ wget http://plct.zdlgv5.com/riscv32_extlib.tar.gz
$ tar -xzvf riscv32_extlib.tar.gz

设置环境变量

$ export PATH=$PATH:./riscv32/bin

注：

上面的环境变量只在当前终端有效，也可以 ~/.bashrc 文件中永久生效。
测试构建是否成功可以执行：

$ riscv32-unknown-linux-gnu-gcc -v

出现类似输出即表示安装成功。

$ riscv32-unknown-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=riscv32-unknown-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/home/jiayou/riscv32/bin/../libexec/gcc/riscv32-unknown-linux-gnu/9.2.0/lto-wrapper
Target: riscv32-unknown-linux-gnu
Configured with: /home/zhangdingli/toolchain/riscv-gnu-toolchain/riscv-gcc/configure --target=riscv32-unknown-linux-gnu --prefix=/home/zhangdingli/toolchain/riscv32 --with-sysroot=/home/zhangdingli/toolchain/riscv32/sysroot --with-system-zlib --enable-shared --enable-tls --enable-languages=c,c++,fortran --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libsanitizer --disable-nls --disable-bootstrap --src=.././riscv-gcc --disable-multilib --with-abi=ilp32d --with-arch=rv32gc --with-tune=rocket 'CFLAGS_FOR_TARGET=-O2   -mcmodel=medlow' 'CXXFLAGS_FOR_TARGET=-O2   -mcmodel=medlow'
Thread model: posix
gcc version 9.2.0 (GCC)

2. 部署 Yocto

2.1 Create workspace

$ mkdir riscv-yocto && cd riscv-yocto
$ repo init -u git://github.com/riscv/meta-riscv  -b master -m tools/manifests/riscv-yocto.xml
$ repo sync
$ repo start work --all

注：

这里可能会遇到下面的错误：

Traceback (most recent call last):File "/home/jiayou/riscv-sifive/.repo/repo/main.py", line 49, in <module>import event_logFile "/home/jiayou/riscv-sifive/.repo/repo/event_log.py", line 167, in <module>_EVENT_ID = multiprocessing.Value('i', 1)File "/snap/git-repo/18/usr/lib/python2.7/multiprocessing/__init__.py", line 253, in Valuereturn Value(typecode_or_type, *args, **kwds)File "/snap/git-repo/18/usr/lib/python2.7/multiprocessing/sharedctypes.py", line 108, in Valuelock = RLock()File "/snap/git-repo/18/usr/lib/python2.7/multiprocessing/__init__.py", line 183, in RLockreturn RLock()File "/snap/git-repo/18/usr/lib/python2.7/multiprocessing/synchronize.py", line 172, in __init__SemLock.__init__(self, RECURSIVE_MUTEX, 1, 1)File "/snap/git-repo/18/usr/lib/python2.7/multiprocessing/synchronize.py", line 75, in __init__sl = self._semlock = _multiprocessing.SemLock(kind, value, maxvalue)
OSError: [Errno 13] Permission denied

解决方法如下：

$ mkdir -p ~/.bin
$ PATH="${HOME}/.bin:${PATH}"
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo

2.2 Update existing workspace

$ cd riscv-yocto
$ repo sync
$ repo rebase

2.3 Setup Build Environment

. ./meta-riscv/setup.sh

2.4 Build Images

To build a console-only image for the 64-bit QEMU machine

$ MACHINE=qemuriscv32 bitbake core-image-full-cmdline

注：第一次执行这步需要较长的执行过程。

2.5 Running in QEMU

MACHINE=qemuriscv32 runqemu nographic

成功启动后终端测试如下：

...
OpenEmbedded nodistro.0 qemuriscv32 ttyS0qemuriscv32 login: root
root@qemuriscv32:~# uname -a
Linux qemuriscv32 5.8.18-yocto-standard #1 SMP PREEMPT Tue Jan 5 10:15:33 UTC 2021 riscv32 riscv32 riscv32 GNU/Linux
root@qemuriscv32:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       123M   88M   27M  78% /
devtmpfs        115M     0  115M   0% /dev
tmpfs           115M     0  115M   0% /dev/shm
tmpfs            46M  8.4M   38M  19% /run
tmpfs           4.0M     0  4.0M   0% /sys/fs/cgroup
tmpfs           115M     0  115M   0% /tmp
tmpfs           115M   72K  115M   1% /var/volatile
tmpfs            23M     0   23M   0% /run/user/0
root@qemuriscv32:~# cat /proc/cpuinfo
processor   : 0
hart        : 0
isa     : rv32imafdcsu
mmu     : sv32root@qemuriscv32:~# free -mhtotal        used        free      shared  buff/cache   available
Mem:          228Mi        28Mi       157Mi       8.0Mi        42Mi       184Mi
Swap:            0B          0B          0B
root@qemuriscv32:~#

3. 修改虚机配置

上面启动的虚机本身存储空间较小，且不能使用 SSH 和 SCP 功能，以下操作是对应的解决办法。

3.1 开启 SSH 和 SCP

修改虚机配置文件 riscv-yocto/openembedded-core/meta/conf/machine/include/riscv/qemuriscv.inc

QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no"

为

QB_TAP_OPT = "-netdev tap,id=net0,ifname=@TAP@,script=no,downscript=no,hostfwd=tcp::2222-:22"

重新编译镜像

$ MACHINE=qemuriscv32 bitbake core-image-full-cmdline

启动虚拟机

$ MACHINE=qemuriscv32 runqemu nographic slirp

测试
SCP

$ scp -P 2222 ./jdk32.tar.gz root@localhost:~/
The authenticity of host '[localhost]:2222 ([127.0.0.1]:2222)' can't be established.
ECDSA key fingerprint is SHA256:qnd5gnS289OoComcMnUqAn/JC24ibc+GUNDiUghshcU.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[localhost]:2222' (ECDSA) to the list of known hosts.
jdk32.tar.gz                                                                                                                                                                                                  100%  160MB   8.8MB/s   00:18
jiayou@jiayou-ubuntu-1804-01:~$

SSH

$ ssh -p 2222 root@localhost
Last login: Thu Dec 10 11:45:03 2020
root@qemuriscv32:~#

注：

启动 SSH 和 SCP 后启动虚机命令变为 " MACHINE=qemuriscv32 runqemu nographic slirp "
参考： https://github.com/riscv/meta-riscv/issues/195

3.2 修改虚机存储

编译的镜像存储在 riscv-yocto/build/tmp-glibc/deploy/images/qemuriscv32 文件下，可以进去该目录下进行相关操作

进入镜像存储位置

$ cd riscv-yocto/build/tmp-glibc/deploy/images/qemuriscv32
$ ls -lh
total 289M
lrwxrwxrwx 2 jiayou jiayou   74 Jan  5 10:19 Image -> Image--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin
-rw-r--r-- 2 jiayou jiayou  21M Jan  5 10:19 Image--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin
lrwxrwxrwx 2 jiayou jiayou   74 Jan  5 10:19 Image-qemuriscv32.bin -> Image--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin
-rw-rw-r-- 2 jiayou jiayou 1.5K Jan  5 11:18 core-image-full-cmdline-qemuriscv32-20210105094513.qemuboot.conf
-rw-r--r-- 2 jiayou jiayou 132M Jan  5 12:04 core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4
-rw-r--r-- 2 jiayou jiayou  15K Jan  5 11:18 core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.manifest
-rw-rw-r-- 2 jiayou jiayou  29M Jan  5 11:18 core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.tar.bz2
-rw-r--r-- 2 jiayou jiayou 108M Jan  5 11:18 core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.wic.qcow2
-rw-r--r-- 2 jiayou jiayou 214K Jan  5 11:18 core-image-full-cmdline-qemuriscv32-20210105094513.testdata.json
lrwxrwxrwx 2 jiayou jiayou   62 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.ext4 -> core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4
lrwxrwxrwx 2 jiayou jiayou   66 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.manifest -> core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.manifest
lrwxrwxrwx 2 jiayou jiayou   64 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.qemuboot.conf -> core-image-full-cmdline-qemuriscv32-20210105094513.qemuboot.conf
lrwxrwxrwx 2 jiayou jiayou   65 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.tar.bz2 -> core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.tar.bz2
lrwxrwxrwx 2 jiayou jiayou   64 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.testdata.json -> core-image-full-cmdline-qemuriscv32-20210105094513.testdata.json
lrwxrwxrwx 2 jiayou jiayou   67 Jan  5 11:18 core-image-full-cmdline-qemuriscv32.wic.qcow2 -> core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.wic.qcow2
-rw-rw-r-- 2 jiayou jiayou 4.1K Jan  5 11:18 core-image-full-cmdline.env
-rwxr-xr-x 2 jiayou jiayou  61K Jan  5 11:02 fw_dynamic.bin
-rwxr-xr-x 2 jiayou jiayou 555K Jan  5 11:02 fw_dynamic.elf
-rwxr-xr-x 2 jiayou jiayou  61K Jan  5 11:02 fw_jump.bin
-rwxr-xr-x 2 jiayou jiayou 555K Jan  5 11:02 fw_jump.elf
-rwxr-xr-x 2 jiayou jiayou  25M Jan  5 11:02 fw_payload.bin
-rwxr-xr-x 2 jiayou jiayou  22M Jan  5 11:02 fw_payload.elf
-rw-rw-r-- 2 jiayou jiayou 2.4M Jan  5 10:19 modules--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.tgz
lrwxrwxrwx 2 jiayou jiayou   76 Jan  5 10:19 modules-qemuriscv32.tgz -> modules--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.tgz
lrwxrwxrwx 2 jiayou jiayou   75 Jan  5 10:19 uImage -> uImage--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin
-rw-r--r-- 2 jiayou jiayou 5.8M Jan  5 10:19 uImage--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin
lrwxrwxrwx 2 jiayou jiayou   75 Jan  5 10:19 uImage-qemuriscv32.bin -> uImage--5.8.18+git0+b976de4f41_3c5d210805-r0-qemuriscv32-20210105094513.bin

修改镜像大小

# 创建名字为new_img.img，大小为：10G 的Image
$ dd if=/dev/zero of=new_img.img bs=1G count=10
# 格式化新建的Image ext4
$ mkfs.ext4 new_img.img
# 创建新旧Image的挂在目录
$ sudo mkdir newImage
$ sudo mkdir oldImage
# 挂载Image
$ sudo mount new_img.img ./newImage/
$ sudo mount core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4 ./oldImage/
# 将旧image对应的目录页拷贝到新image对应的目录
$ sudo cp -r ./oldImage/* ./newImage/
# 卸载目录
sudo umount ./newImage/
sudo umount ./oldImage/
# 将新的image文件修改为旧image文件名
$ sudo mv core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4 core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4.backup
$ sudo mv new_img.img core-image-full-cmdline-qemuriscv32-20210105094513.rootfs.ext4

再启动虚机查看存储确实变为 10 G

qemuriscv32 login: root
root@qemuriscv32:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       9.8G  144M  9.2G   2% /
devtmpfs        115M     0  115M   0% /dev
tmpfs           115M     0  115M   0% /dev/shm
tmpfs            46M  8.4M   38M  19% /run
tmpfs           4.0M     0  4.0M   0% /sys/fs/cgroup
tmpfs           115M     0  115M   0% /tmp
tmpfs           115M   72K  115M   1% /var/volatile
tmpfs            23M     0   23M   0% /run/user/0
root@qemuriscv32:~#

注：
参考：https://www.cnblogs.com/grandblogs/p/12216955.html