M-Labs/nix-servo
3
1
Fork 2
Code Issues 1 Pull Requests Packages Projects Releases Wiki Activity

add not-os disk image builder and qemu #10

Merged
sb10q merged 1 commits from fsagbuya/nix-servo:sdimage into master 2024-08-17 17:37:24 +08:00
Conversation 9 Commits 1 Files Changed 2 +13 -3
fsagbuya commented 2023-12-15 15:49:09 +08:00
Collaborator
Copy Link

Description

This patch aims to:

  1. Add full disk image builder for not-os, built with minimal size using the sd-image builder of nixpkgs as reference.
    See: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/sd-card/sd-image.nix#L173-L256
  • postBootCommand to resize partition on first boot to be added on a separate PR.
  1. Add basic qemu script to simulate the system for booting and testing.
  2. Enable networking. Need to rollback to qemu 8.1.13 for now, since the latest 8.2.0 in nixpkgs unstable branch are causing network errors.
[    6.691226] macb e000b000.ethernet eth0: validation of rgmii-id with support 00000000,00000000,00006280 and advertisement 00000000,00000000,00004280 failed: -EINVAL
[    6.691851] macb e000b000.ethernet eth0: Could not attach PHY (-22)
RTNETLINK answers: Invalid argument
Error: Device for nexthop is not up.
Error: Nexthop has invalid gateway.
Error resolving 0.nixos.pool.ntp.org: System error (-11)
 1 Jan 00:00:06 ntpdate[147]: Can't find host 0.nixos.pool.ntp.org: System error (-11)
### Description This patch aims to: 1. Add full disk image builder for `not-os`, built with minimal size using the sd-image builder of nixpkgs as reference. See: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/installer/sd-card/sd-image.nix#L173-L256 - `postBootCommand` to resize partition on first boot to be added on a separate PR. 2. Add basic qemu script to simulate the system for booting and testing. 3. Enable networking. Need to rollback to qemu 8.1.13 for now, since the latest 8.2.0 in nixpkgs unstable branch are causing network errors. ``` [ 6.691226] macb e000b000.ethernet eth0: validation of rgmii-id with support 00000000,00000000,00006280 and advertisement 00000000,00000000,00004280 failed: -EINVAL [ 6.691851] macb e000b000.ethernet eth0: Could not attach PHY (-22) RTNETLINK answers: Invalid argument Error: Device for nexthop is not up. Error: Nexthop has invalid gateway. Error resolving 0.nixos.pool.ntp.org: System error (-11) 1 Jan 00:00:06 ntpdate[147]: Can't find host 0.nixos.pool.ntp.org: System error (-11) ```
sb10q commented 2023-12-15 16:45:35 +08:00
Owner
Copy Link

Do we really need a 2GB output?
It's smarter to make a minimally-sized disk image and resize the partition on first boot like NixOS on Raspberry Pi does.

Do we really need a 2GB output? It's smarter to make a minimally-sized disk image and resize the partition on first boot like NixOS on Raspberry Pi does.
👍 1
sb10q commented 2023-12-15 16:46:10 +08:00
Owner
Copy Link

I also doubt FSBL+uboot+kernel+stage1 needs 200MB.

I also doubt FSBL+uboot+kernel+stage1 needs 200MB.
sb10q reviewed 2023-12-15 16:47:39 +08:00
flake.nix Outdated
@ -235,0 +270,4 @@
cp ${not-os-cfg.build.uRamdisk}/initrd /mnt/uramdisk.image.gz
cp ${not-os-cfg.build.kernel}/dtbs/zynq-zc706.dtb /mnt/devicetree.dtb
cat ${rootfs} > /dev/vda2
fsck -f -y /dev/vda2
sb10q commented 2023-12-15 16:47:39 +08:00
Owner
Copy Link

Not needed.

Not needed.
👍 1
fsagbuya marked this conversation as resolved
sb10q reviewed 2023-12-15 16:47:48 +08:00
flake.nix Outdated
@ -235,0 +260,4 @@
1 : size=${toString (2048 * 200)}, type=c
2 : type=83
EOF
lsblk
sb10q commented 2023-12-15 16:47:48 +08:00
Owner
Copy Link

Not needed.

Not needed.
👍 1
fsagbuya marked this conversation as resolved
sb10q reviewed 2023-12-15 16:48:09 +08:00
flake.nix Outdated
@ -235,0 +239,4 @@
storePaths = [ not-os-cfg.build.toplevel ];
volumeLabel = "root";
};
in pkgs.vmTools.runInLinuxVM (pkgs.runCommand "disk_image" {
sb10q commented 2023-12-15 16:48:09 +08:00
Owner
Copy Link

I don't think a VM is required to create a few filesystems and copy some files.

I don't think a VM is required to create a few filesystems and copy some files.
sb10q commented 2023-12-15 16:48:57 +08:00
Owner
Copy Link

Where is this idea from?

Where is this idea from?
fsagbuya commented 2023-12-15 17:04:54 +08:00
Author
Collaborator
Copy Link

The VM method is mainly from make-disk-image.nix and some examples in Github. This can be helpful to run disk formatting commands that require root privileges.

The VM method is mainly from [make-disk-image.nix](https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-disk-image.nix#L525) and some examples in Github. This can be helpful to run disk formatting commands that require root privileges.
sb10q commented 2023-12-15 17:22:55 +08:00
Owner
Copy Link

I suspect you can use e2fsprogs and mtools instead to modify the disk image directly without this little circus. Try it.

Also parted is more amenable to command line scripting than sfdisk. See the NixOS installation manual which gives you almost exactly the parted commands you need here.

I suspect you can use e2fsprogs and mtools instead to modify the disk image directly without this little circus. Try it. Also parted is more amenable to command line scripting than sfdisk. See the NixOS installation manual which gives you almost exactly the parted commands you need here.
fsagbuya commented 2023-12-15 17:30:24 +08:00
Author
Collaborator
Copy Link

Understood. Will do as suggested.

Understood. Will do as suggested.
sb10q commented 2023-12-15 17:31:44 +08:00
Owner
Copy Link

You may have to create raw filesystem images for each partition, and then dd them at the correct offsets into the final image with the partitions, unless those tools can deal with the partition tables already. Do your research.

You may have to create raw filesystem images for each partition, and then dd them at the correct offsets into the final image with the partitions, unless those tools can deal with the partition tables already. Do your research.
fsagbuya marked this conversation as resolved
sb10q reviewed 2023-12-15 16:49:30 +08:00
flake.nix Outdated
@ -235,0 +271,4 @@
cp ${not-os-cfg.build.kernel}/dtbs/zynq-zc706.dtb /mnt/devicetree.dtb
cat ${rootfs} > /dev/vda2
fsck -f -y /dev/vda2
resize2fs /dev/vda2
sb10q commented 2023-12-15 16:49:30 +08:00
Owner
Copy Link

Why?

Why?
fsagbuya marked this conversation as resolved
sb10q commented 2023-12-15 16:50:32 +08:00
Owner
Copy Link

57K -r--r--r-- 1 root root 57K nix-path-registration

What is that for? NixOS systems don't have a /nix-path-registration file and work fine.

> 57K -r--r--r-- 1 root root 57K nix-path-registration What is that for? NixOS systems don't have a `/nix-path-registration` file and work fine.
sb10q commented 2023-12-15 16:50:57 +08:00
Owner
Copy Link

Does it boot?

Does it boot?
fsagbuya commented 2023-12-15 17:07:06 +08:00
Author
Collaborator
Copy Link

Does it boot?

In QEMU yes. Still need to test in the physical board.

> Does it boot? In QEMU yes. Still need to test in the physical board.
fsagbuya changed title from flake: add not-os disk image builder to WIP: flake: add not-os disk image builder 2023-12-20 15:21:39 +08:00
fsagbuya changed title from WIP: flake: add not-os disk image builder to WIP: add not-os disk image builder 2023-12-20 15:22:08 +08:00
fsagbuya force-pushed sdimage from 12e0f79931 to 94c7cce5de 2023-12-22 15:37:08 +08:00 Compare
fsagbuya changed title from WIP: add not-os disk image builder to add not-os disk image builder 2023-12-22 15:37:27 +08:00
fsagbuya changed title from add not-os disk image builder to add not-os disk image builder and qemu 2023-12-22 15:37:39 +08:00
fsagbuya commented 2023-12-22 15:38:58 +08:00
Author
Collaborator
Copy Link

Updated. See new Description for the changes.

Updated. See new Description for the changes.
fsagbuya force-pushed sdimage from 94c7cce5de to 0c349356f5 2023-12-22 15:43:10 +08:00 Compare
sb10q reviewed 2023-12-22 17:14:12 +08:00
flake.nix Outdated
@ -250,1 +339,3 @@
zc706-not-os = not-os-configured.config.system.build.zynq_image;
zc706-not-os = not-os-cfg.build.zynq_image;
zc706-sd-image = sd-image;
zc706-qemu = not-os-qemu;
sb10q commented 2023-12-22 17:14:12 +08:00
Owner
Copy Link

What is the difference with the QEMU launcher in zc706-not-os?

What is the difference with the QEMU launcher in zc706-not-os?
fsagbuya commented 2023-12-22 17:53:46 +08:00
Author
Collaborator
Copy Link

zc706-not-os now only builds dtb, kernel and initrd (qemu exluded) while the qemu launcher includes everything, also the sd-image.img for booting in root-fs. It would be cleaner to include these in the zc706-not-os though. However zc706-not-os builds on modules, wasn't able to find a way to include a package in it for now.

zc706-not-os now only builds dtb, kernel and initrd (qemu exluded) while the qemu launcher includes everything, also the sd-image.img for booting in root-fs. It would be cleaner to include these in the zc706-not-os though. However zc706-not-os builds on modules, wasn't able to find a way to include a package in it for now.
fsagbuya marked this conversation as resolved
sb10q commented 2023-12-22 17:15:47 +08:00
Owner
Copy Link

Update the not-os patch to add the fix for Missing privilege separation directory: /var/empty. Closes #6.

It's rather messy to change the not-os PR and update the hash here. Probably better to copy the patches to this repos.

> Update the not-os patch to add the fix for Missing privilege separation directory: /var/empty. Closes #6. It's rather messy to change the not-os PR and update the hash here. Probably better to copy the patches to this repos.
👍 1
sb10q reviewed 2023-12-22 17:17:16 +08:00
flake.nix Outdated
@ -242,0 +250,4 @@
cp $base/sd-image.img /tmp/
chmod +w /tmp/sd-image.img
truncate -s 512m /tmp/sd-image.img
sb10q commented 2023-12-22 17:17:16 +08:00
Owner
Copy Link

What is this about?

What is this about?
sb10q commented 2023-12-22 17:18:10 +08:00
Owner
Copy Link

Just use QEMU copy-on-write overlays, duh.

Just use QEMU copy-on-write overlays, duh.
👍 1
fsagbuya marked this conversation as resolved
sb10q reviewed 2023-12-22 17:19:02 +08:00
flake.nix Outdated
@ -242,0 +254,4 @@
qemu-system-arm \
-M xilinx-zynq-a9 \
-serial /dev/null \
sb10q commented 2023-12-22 17:19:02 +08:00
Owner
Copy Link

Needed? What does this do?

Needed? What does this do?
fsagbuya commented 2023-12-22 18:10:20 +08:00
Author
Collaborator
Copy Link

Got it from an example qemu command in https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/821854273/Running+Bare+Metal+Applications+on+QEMU. It seems when this is omitted, the qemu monitor hangs.

Got it from an example qemu command in https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/821854273/Running+Bare+Metal+Applications+on+QEMU. It seems when this is omitted, the qemu monitor hangs.
sb10q commented 2023-12-23 08:52:59 +08:00
Owner
Copy Link

Please add code comment and link to source.

Please add code comment and link to source.
fsagbuya marked this conversation as resolved
fsagbuya force-pushed sdimage from 0c349356f5 to e718a865ec 2023-12-22 19:12:37 +08:00 Compare
sb10q reviewed 2023-12-23 08:54:35 +08:00
flake.nix Outdated
@ -249,2 +335,3 @@
zc706-bootimage = bootimage { board = "zc706"; };
zc706-not-os = not-os-configured.config.system.build.zynq_image;
zc706-not-os = not-os-cfg.build.zynq_image;
zc706-sd-image = sd-image;
sb10q commented 2023-12-23 08:54:35 +08:00
Owner
Copy Link

There's a blatant problem here since you pass board=zc706 to the functions above and then suddenly stop.

There's a blatant problem here since you pass board=zc706 to the functions above and then suddenly stop.
fsagbuya force-pushed sdimage from e718a865ec to 34a6d026fd 2023-12-23 13:26:51 +08:00 Compare
sb10q reviewed 2023-12-23 15:21:32 +08:00
flake.nix Outdated
@ -242,0 +319,4 @@
faketime "1970-01-01 00:00:00" mkfs.vfat -n BOOT firmware_part.img
mkdir firmware
cp ${bootimage { board = "zc706"; }}/boot.bin firmware/
sb10q commented 2023-12-23 15:21:32 +08:00
Owner
Copy Link

Please read your code again. The problem is only blatant at those lines of code at the end, but runs deeper.

Please read your code again. The problem is only blatant at those lines of code at the end, but runs deeper.
👍 1
sb10q reviewed 2023-12-23 15:22:32 +08:00
flake.nix Outdated
@ -242,0 +247,4 @@
export PATH=${pkgs.qemu}/bin:$PATH
set -x
base=$(dirname $0)
qemu-img create -f qcow2 -b $base/sd-image.img -F raw /tmp/sd-overlay.qcow2 512M
sb10q commented 2023-12-23 15:22:31 +08:00
Owner
Copy Link

Why -F raw?
And you can use mktemp.

Why -F raw? And you can use mktemp.
fsagbuya commented 2023-12-27 11:31:00 +08:00
Author
Collaborator
Copy Link

-F raw is the detected backing format for the img file

qemu-img: /tmp/tmp.4H8j6kjlht/sd-overlay.qcow2: Backing file specified without backing format
Detected format of raw.
-F raw is the detected backing format for the `img` file ``` qemu-img: /tmp/tmp.4H8j6kjlht/sd-overlay.qcow2: Backing file specified without backing format Detected format of raw. ```
sb10q commented 2023-12-27 12:01:22 +08:00
Owner
Copy Link

I can read the man page and figure out what -F does. The point is I'm questioning the use of the raw format over the other options that QEMU offers.

I can read the man page and figure out what -F does. The point is I'm questioning the use of the raw format over the other options that QEMU offers.
fsagbuya commented 2023-12-27 14:02:45 +08:00
Author
Collaborator
Copy Link

Usage here is only for direct backing of the current img format, otherwise we can convert the raw image first to qcow2 (or other format recommended) then apply overlay to it.

Usage here is only for direct backing of the current `img` format, otherwise we can convert the raw image first to `qcow2` (or other format recommended) then apply overlay to it.
sb10q commented 2023-12-27 17:51:28 +08:00
Owner
Copy Link

Okay. It's rather unclear from the command line that this applies only to the base SD image. Can you reorder the arguments to make it less confusing, or is this a fundamental issue with the CLI of qemu-img?

Okay. It's rather unclear from the command line that this applies only to the base SD image. Can you reorder the arguments to make it less confusing, or is this a fundamental issue with the CLI of qemu-img?
sb10q commented 2023-12-27 17:52:22 +08:00
Owner
Copy Link

And I would avoid reconverting images since those copies are quite large and waste storage or memory.

And I would avoid reconverting images since those copies are quite large and waste storage or memory.
👍 1
fsagbuya commented 2023-12-27 21:04:10 +08:00
Author
Collaborator
Copy Link

Rearranged the arguments for clarity while maintaining their intended function.

Rearranged the arguments for clarity while maintaining their intended function.
fsagbuya marked this conversation as resolved
fsagbuya force-pushed sdimage from 34a6d026fd to e00f47f5bb 2023-12-27 11:25:10 +08:00 Compare
fsagbuya force-pushed sdimage from e00f47f5bb to c10d81fb3b 2023-12-27 11:27:03 +08:00 Compare
fsagbuya force-pushed sdimage from c10d81fb3b to d14d965e81 2023-12-27 20:59:23 +08:00 Compare
sb10q reviewed 2023-12-27 21:09:50 +08:00
flake.nix Outdated
@ -242,0 +274,4 @@
cd $out
cp -s ${not-os-cfg.build.kernel}/uImage .
cp -s ${not-os-cfg.build.uRamdisk}/initrd uramdisk.image.gz
cp -s ${not-os-cfg.build.kernel}/dtbs/zynq-zc706.dtb devicetree.dtb
sb10q commented 2023-12-27 21:09:49 +08:00
Owner
Copy Link

Might be OK to merge since this is already broken all over the place, but some deep cleanup of multiple board management will be required at some point.

Might be OK to merge since this is already broken all over the place, but some deep cleanup of multiple board management will be required at some point.
fsagbuya commented 2024-01-02 10:47:35 +08:00
Author
Collaborator
Copy Link

Will add multiple board management in the next PR. I'm seeing zc702 and zedboard from Xilinx, may I confirm if these are the ones to be added?

Will add multiple board management in the next PR. I'm seeing zc702 and zedboard from Xilinx, may I confirm if these are the ones to be added?
sb10q commented 2024-01-02 10:48:45 +08:00
Owner
Copy Link

I don't care about those. The goal is to support fast-servo and nothing else. zc706 is only for testing.

I don't care about those. The goal is to support fast-servo and nothing else. zc706 is only for testing.
fsagbuya commented 2024-01-02 10:51:58 +08:00
Author
Collaborator
Copy Link

Understood. Will do some cleanup once testing is done and add the support for fast-servo.

Understood. Will do some cleanup once testing is done and add the support for fast-servo.
sb10q reviewed 2023-12-27 21:12:31 +08:00
flake.nix Outdated
@ -242,0 +322,4 @@
mkdir firmware
cp ${bootimage { inherit board; }}/boot.bin firmware/
cp ${not-os-cfg.build.kernel}/uImage firmware/
cp ${not-os-cfg.build.uRamdisk}/initrd firmware/uramdisk.image.gz
sb10q commented 2023-12-27 21:12:31 +08:00
Owner
Copy Link

Inconsistent capitalization.

Inconsistent capitalization.
fsagbuya commented 2024-01-02 10:43:33 +08:00
Author
Collaborator
Copy Link

Will add this on a separate PR since editing the case of uRamdisk will require not-os hash update, or are you pointing to uramdisk.image.gz that is inconsistent?

Will add this on a separate PR since editing the case of `uRamdisk` will require not-os hash update, or are you pointing to `uramdisk.image.gz` that is inconsistent?
sb10q commented 2024-01-02 10:45:24 +08:00
Owner
Copy Link

Yes the names should be exactly the same everywhere, including upper/lowercase letters.

Yes the names should be exactly the same everywhere, including upper/lowercase letters.
👍 1
fsagbuya marked this conversation as resolved
sb10q reviewed 2023-12-27 21:17:52 +08:00
flake.nix Outdated
@ -242,0 +246,4 @@
#!/bin/bash
export PATH=${pkgs.qemu}/bin:$PATH
set -x
IMGDIR=$(mktemp -d)
sb10q commented 2023-12-27 21:17:52 +08:00
Owner
Copy Link

Shouldn't the script delete this at the end?

Shouldn't the script delete this at the end?
fsagbuya marked this conversation as resolved
fsagbuya force-pushed sdimage from d14d965e81 to 6fd302b8a5 2024-01-02 10:39:38 +08:00 Compare
fsagbuya force-pushed sdimage from 6fd302b8a5 to 3aa5b425d6 2024-01-02 10:54:59 +08:00 Compare
fsagbuya changed title from add not-os disk image builder and qemu to WIP: add not-os disk image builder and qemu 2024-01-02 13:06:50 +08:00
fsagbuya force-pushed sdimage from 3aa5b425d6 to 18146ae313 2024-01-02 13:44:06 +08:00 Compare
fsagbuya changed title from WIP: add not-os disk image builder and qemu to add not-os disk image builder and qemu 2024-01-02 13:59:40 +08:00
fsagbuya force-pushed sdimage from 18146ae313 to c5623e0529 2024-01-02 17:22:08 +08:00 Compare
sb10q commented 2024-01-02 17:26:05 +08:00
Owner
Copy Link

Add a SD card image download link to hydra (nix-support build product) - see the mcu stuff for examples.

Add a SD card image download link to hydra (nix-support build product) - see the mcu stuff for examples.
👍 1
fsagbuya force-pushed sdimage from c5623e0529 to f8fc8a6f99 2024-01-03 10:44:02 +08:00 Compare
fsagbuya force-pushed sdimage from f8fc8a6f99 to d7d627ae42 2024-01-05 12:44:06 +08:00 Compare
fsagbuya changed title from add not-os disk image builder and qemu to WIP: add not-os disk image builder and qemu 2024-01-05 14:09:27 +08:00
fsagbuya force-pushed sdimage from d7d627ae42 to 1f62e6a35e 2024-01-08 19:47:42 +08:00 Compare
fsagbuya changed title from WIP: add not-os disk image builder and qemu to add not-os disk image builder and qemu 2024-01-08 19:48:22 +08:00
fsagbuya commented 2024-01-08 19:51:33 +08:00
Author
Collaborator
Copy Link

Updates: Added hydra build products and enable network support.

Updates: Added hydra build products and enable network support.
sb10q merged commit 1f62e6a35e into master 2024-01-09 08:09:12 +08:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-servo#10
No description provided.
Delete Branch "fsagbuya/nix-servo:sdimage"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?