Add offline installer #12
sb10q
Reference in New Issue
Block a user
Delete Branch "esavkin/artiq-extrapkg:2289-offline-installer"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
@@ -538,0 +597,4 @@msys2.wine-msys2];};installer = pkgs.mkShell {msys2-
Or just merge it with the above one.
@@ -0,0 +445,4 @@})(pkgs.fetchurl {url = "https://msys2.m-labs.hk/artiq-beta/mingw-w64-x86_64-python-sipyco-1.6-1-any.pkg.tar.zst";I told you at least four times that this had to use the packages built here in the same flake and not download them from Hydra.
@@ -0,0 +1,102 @@#My script didn't need another copy of this script which is mostly downloaded from GitHub. Why does yours do?
@@ -0,0 +7,4 @@mkdir -p $MSYS2DIR/var/lib/pacman $MSYS2DIR/msys/etccurl -L https://repo.msys2.org/msys/x86_64/pacman-mirrors-20240210-1-any.pkg.tar.zst | tar xvf - -C $MSYS2DIR --zstdcurl -L https://raw.githubusercontent.com/msys2/MSYS2-packages/master/pacman/pacman.conf | sed -e "s|/etc/pacman.d|$MSYS2DIR/etc/pacman.d|g" -e "s|SigLevel = Required|SigLevel = Never|g" > $MSYS2DIR/etc/pacman.confWhy do you need another copy of that file if you're downloading and editing it here?
@@ -526,0 +571,4 @@-p "qt-ifw/packages" \-c "qt-ifw/config/config.xml" \--offline-only \"msys2-x86_64.exe" \Give it a better name to differentiate it from the original MSYS2 installer.
@@ -526,0 +562,4 @@export HOME=`mktemp -d`cp -r $src/installer/qt-ifw qt-ifwchmod -R 700 qt-ifwfind qt-ifw \( -name "package.xml" -or -name "config.xml" \) -exec sed -i "s|@DATE@|$(date +'%Y-%m-%d')|g" "{}" \;find? There are only two files and the paths are known, no?
@@ -526,0 +561,4 @@buildPhase = ''export HOME=`mktemp -d`cp -r $src/installer/qt-ifw qt-ifwchmod -R 700 qt-ifwWhy?
So that 7z can place the archive in this directory
Needs detailed explanation in comments.
@@ -8,0 +14,4 @@nix build .#hydraJobs.msys2-offline-installer```Update packages if needed:This should only update the dependencies of the packages that are built here.
The packages built here should be injected into the list as another step so they can be updated as part of
nix flake update, and also so this repos can work without Hydra.@@ -0,0 +1,37 @@#!/usr/bin/env bashCan the code duplication with my script be avoided?
@@ -526,0 +536,4 @@installPhase = (pkgs.lib.strings.concatStringsSep "\n" (["mkdir $out"] ++ (map (p: "tar xvf ${p} -C $out") srcs)));};msys2_image = pkgs.stdenvNoCC.mkDerivation rec {image?
@@ -538,0 +601,4 @@fakerootp7zipmsys2.wine-msys2wine64Is there a difference between wine64 and wineWowPackages.stable that was used everywhere else before your patch?
In any case you need to be consistent.
@@ -0,0 +37,4 @@component.addOperation("CreateShortcut", "@TargetDir@/mingw32.exe", "@StartMenuDir@/MSYS2 MINGW32.lnk", "iconPath=@TargetDir@/mingw32.exe");component.addOperation("CreateShortcut", "@TargetDir@/mingw64.exe", "@StartMenuDir@/MSYS2 MINGW64.lnk", "iconPath=@TargetDir@/mingw64.exe");component.addOperation("CreateShortcut", "@TargetDir@/ucrt64.exe", "@StartMenuDir@/MSYS2 UCRT64.lnk", "iconPath=@TargetDir@/ucrt64.exe");component.addOperation("CreateShortcut", "@TargetDir@/clang64.exe", "@StartMenuDir@/MSYS2 CLANG64.lnk", "iconPath=@TargetDir@/clang64.exe");Can the non-mingw64 stuff be removed entirely?
As far as I can tell in the majority of cases it just causes software bloat and user confusion.
@@ -24,1 +20,3 @@echo "]" >> msys2_packages.nixpacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf \--cachedir $MSYS2DIR/msys/cache --sync --print-format "%l %n" \mingw-w64-x86_64-python\Obviously the code duplication with https://git.m-labs.hk/M-Labs/artiq-extrapkg/src/branch/master/flake.nix#L445-L465 has to be addressed.
I have already told you many times that code duplication is bad. Please pay attention.
Put this list into a separate file and have Nix and the shell script read it.
https://nixos.org/manual/nix/stable/language/builtins.html#builtins-readFile
@@ -25,0 +37,4 @@mingw-w64-x86_64-lld \mingw-w64-x86_64-python-aiohttp \mingw-w64-x86_64-python-idna \mingw-w64-x86_64-python-yarl \Where are idna and yarl coming from?
Pacman should be able to find such dependencies by itself, no?
ce6a1e8975to14ecd12286@@ -19,0 +22,4 @@openocdDependenciesFile = ./msys2/openocd_deps.txt;pythonparserDependenciesFile = ./msys2/pythonparser_deps.txt;# reads filename and return list of non-empty lines from itnonEmptyLinesFileReader = (filename: builtins.filter (x: builtins.lessThan 0 (builtins.length (pkgs.lib.strings.stringToCharacters x)))Silly name.
You removed the .lnk files for the non-mingw64 stuff. But, is the corresponding data also removed, i.e. is the bloat issue resolved?
cacd9e685ctof49d816ea2@@ -0,0 +1,2 @@mingw-w64-x86_64-openocdpythonparser_deps does not contain pythonparser
artiq_deps does not contain artiq
openocd_deps contains openocd
Why?
Maybe it should just not be called _deps ?
Or rather, just merge it into artiq_deps.txt. They are always used together.
@@ -19,0 +22,4 @@openocdDependenciesFile = ./msys2/openocd_deps.txt;pythonparserDependenciesFile = ./msys2/pythonparser_deps.txt;# reads filename and return list of non-empty lines from itreadPackagesFromFile = (filename: builtins.filter (x: builtins.lessThan 0 (builtins.length (pkgs.lib.strings.stringToCharacters x)))readStringListFromFile ? Nothing tied to packages.
@@ -526,0 +527,4 @@installPhase = (pkgs.lib.strings.concatStringsSep "\n" (["mkdir $out"] ++ (map (p: "tar xvf ${p} -C $out") srcs)));};msys2_artiq_image = pkgs.stdenvNoCC.mkDerivation rec {For the second time, you need to rename this. It's not an image and I can't even find a single person using "image" in this context on the web.
Maybe "environment" ?
@@ -4,21 +4,40 @@ set -ecd $(dirname $0)MSYS2DIR=`pwd`/msys2rm -rf $MSYS2DIRRemove. It's not in my script because leaving it there is very useful when debugging.
@@ -4,21 +4,40 @@ set -ecd $(dirname $0)MSYS2DIR=`pwd`/msys2rm -rf $MSYS2DIRRemove. It's not in my script because leaving it there is very useful when debugging.
@@ -13,1 +15,3 @@pacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf --cachedir $MSYS2DIR/msys/cache -Sp mingw-w64-x86_64-cc mingw-w64-x86_64-python3.11 mingw-w64-x86_64-python-numpy mingw-w64-x86_64-python-setuptools > $MSYS2DIR/packages.txtpacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf \--cachedir $MSYS2DIR/msys/cache --sync --print-format "%l %n" \mingw-w64-x86_64-cc mingw-w64-x86_64-python3.11 \mingw-w64-x86_64-python3.11 vs. mingw-w64-x86_64-python in your files should make alarm bells go off.
@@ -12,2 +14,3 @@fakeroot pacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf -Syypacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf --cachedir $MSYS2DIR/msys/cache -Sp mingw-w64-x86_64-cc mingw-w64-x86_64-python3.11 mingw-w64-x86_64-python-numpy mingw-w64-x86_64-python-setuptools > $MSYS2DIR/packages.txtpacman --root $MSYS2DIR --config $MSYS2DIR/etc/pacman.conf \--cachedir $MSYS2DIR/msys/cache --sync --print-format "%l %n" \I never needed --sync or --print-format. Why did you add them?
--sync is same as -S https://archlinux.org/pacman/pacman.8.html, print format is used to separate package name from its full name, which may contain tilde.
@@ -526,0 +517,4 @@msys2-offline-installer = pkgs.stdenv.mkDerivation {name = "msys2-installer";buildInputs = with pkgs; [ curl pacman fakeroot p7zip wine64 ];Please pay attention.
There's a bunch of messages like:
Are they harmless?
Why did the binary size increase from 200-something MB to 381MB?
Include the ARTIQ version in the file name.
I added LLVM as a dependency, otherwise artiq_run/compile fails because there are no LLVM tools
This needs to be in a separate PR which is independent from this one, as I have already told you several times.
f49d816ea2to9a818cc70b03c9f705catob65402644cb65402644cto471027d336Yes, they are not producing errors afterwards and do not influence the artiq.
Though it would be advised to run
pacman -Syu(upgrade) two times after install.This defeats the purpose of an offline installer.
Not really, it works totally fine without these updates. Just the base image is slightly outdated, and there is no good way to install base packages from repos under Linux/nix environment.
471027d336to196f0b163a