M-Labs/it-infra
M-Labs
/
it-infra
8
0
Fork 6
Code Issues 10 Pull Requests Releases Wiki Activity

Deploy web2019 to the intl domain #40

Merged
sb10q merged 3 commits from 134-deploy into master 2024-08-14 10:54:53 +08:00
Conversation 1 Commits 3 Files Changed 1 +14 -8
1 changed files with 14 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files
Showing only changes of commit 3d4ec9f431 - Show all commits

22
nixbld-etc-nixos/configuration.nix
View File

@ -649,7 +649,19 @@ in
notificationSender = "hydra@m-labs.hk";
minimumDiskFree = 15; # in GB
minimumDiskFreeEvaluator = 1;
extraConfig =
extraConfig = let
deployWebIntl = pkgs.writeShellScript "deployWebIntl"
''
#!${pkgs.bash}/bin/bash
sb10q commented 2024-07-29 15:33:50 +08:00
Outdated
Review
Copy Link

cp -a or similar?

Such a command would be shorter and avoid any time window during which the file is world-readable.

``cp -a`` or similar? Such a command would be shorter and avoid any time window during which the file is world-readable.
[ $(jq '.buildStatus' < $HYDRA_JSON) = 0 ]
export "TMPSSH=`mktemp -d`"
sb10q commented 2024-07-18 15:36:23 +08:00
Outdated
Review
Copy Link

Does this work as intended when used in RunCommand?
May be safer to use writeShellScript as I mentioned earlier (and which you ignored, as often).

Does this work as intended when used in RunCommand? May be safer to use writeShellScript as I mentioned earlier (and which you ignored, as often).
esavkin commented 2024-07-18 16:46:47 +08:00
Outdated
Review
Copy Link

They forgot to specify anything useful in the docs, but looking at the source code they eventually just run it with IPC::Run3: d7986226f0/src/lib/Hydra/Plugin/RunCommand.pm (L259) . Which under the hood runs the subprocess with perl's system https://metacpan.org/pod/perlfunc#system , which runs it with default shell (/bin/sh).

Also code for checking that trap actually works:

use strict;
use warnings;
use IPC::Run3;

my $output;
open(my $pipe, "| cat");
my $command = "export TMPSSH=\`mktemp -d\` && \\\necho \$TMPSSH && \\\ntrap \"rm -rf \$TMPSSH\" EXIT";
run3 $command, undef, \$output, undef;
run3 "ls -al ${output}";

Will fail with ls: cannot access '/tmp/tmp.bOSgsRnGNW': No such file or directory

They forgot to specify anything useful in the docs, but looking at the source code they eventually just run it with IPC::Run3: https://github.com/NixOS/hydra/blob/d7986226f0666d5aa0032fdcdb9f38eef6a91dd3/src/lib/Hydra/Plugin/RunCommand.pm#L259 . Which under the hood runs the subprocess with perl's `system` https://metacpan.org/pod/perlfunc#system , which runs it with default shell (`/bin/sh`). Also code for checking that `trap` actually works: ```perl use strict; use warnings; use IPC::Run3; my $output; open(my $pipe, "| cat"); my $command = "export TMPSSH=\`mktemp -d\` && \\\necho \$TMPSSH && \\\ntrap \"rm -rf \$TMPSSH\" EXIT"; run3 $command, undef, \$output, undef; run3 "ls -al ${output}"; ``` Will fail with `ls: cannot access '/tmp/tmp.bOSgsRnGNW': No such file or directory`
sb10q commented 2024-07-18 16:47:40 +08:00
Outdated
Review
Copy Link

Right. It's not specified and might change.

Right. It's not specified and might change.
trap "rm -rf '$TMPSSH'" EXIT
sb10q commented 2024-07-18 15:38:23 +08:00
Outdated
Review
Copy Link

Why do you need that .ssh directory if you're overriding each file in the ssh invokation using -o ?

Why do you need that .ssh directory if you're overriding each file in the ssh invokation using -o ?
cp --preserve=mode /opt/hydra_id_ed25519 "$TMPSSH/id_ed25519"
cp --preserve=mode /opt/hydra_id_ed25519.pub "$TMPSSH/id_ed25519.pub"
echo "5.78.86.156 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMbV69aqkHdQ1T5lMuALyHjNowU1rottZtEV4OhKQ6Y" > "$TMPSSH/known_hosts"
${pkgs.rsync}/bin/rsync -r -e "${pkgs.openssh}/bin/ssh -i '$TMPSSH/id_ed25519' -o 'UserKnownHostsFile=$TMPSSH/known_hosts' -o IdentitiesOnly=yes" -c $(jq -r '.outputs[0].path' < $HYDRA_JSON)/ zolaupd@5.78.86.156:/var/www/m-labs-intl.com/html/
sb10q commented 2024-06-20 17:46:08 +08:00
Outdated
Review
Copy Link

Is this tested? I doubt rsync is in scope.
You may also want to use writeShellScript.

Is this tested? I doubt rsync is in scope. You may also want to use writeShellScript.
'';
sb10q commented 2024-07-18 12:04:50 +08:00
Outdated
Review
Copy Link

Is this still executed if a command fails?

If you had paid attention to my other commits in this repos, you would have noticed that I use trap to handle situations like this.

Is this still executed if a command fails? If you had paid attention to my other commits in this repos, you would have noticed that I use ``trap`` to handle situations like this.
sb10q commented 2024-07-18 12:05:16 +08:00
Outdated
Review
Copy Link

Also rm -rf $HOME is bad form in any shell script.

Also ``rm -rf $HOME`` is bad form in any shell script.
in
''
binary_cache_secret_key_file = /etc/nixos/secret/nixbld.m-labs.hk-1
max_output_size = 10000000000
@ -660,13 +672,7 @@ in
</runcommand>
<runcommand>
job = web:web:web-intl
command = [ $(jq '.buildStatus' < $HYDRA_JSON) = 0 ] && export HOME=`mktemp -d` && \
mkdir $HOME/.ssh && \
cp /opt/hydra_id_ed25519 $HOME/.ssh/id_ed25519 && \
cp /opt/hydra_id_ed25519.pub $HOME/.ssh/id_ed25519.pub && \
echo "5.78.86.156 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMbV69aqkHdQ1T5lMuALyHjNowU1rottZtEV4OhKQ6Y" > $HOME/.ssh/known_hosts && \
chmod 600 $HOME/.ssh/id_ed25519 && \
${pkgs.rsync}/bin/rsync -r -c $(jq -r '.outputs[0].path' < $HYDRA_JSON) zolaupd@5.78.86.156:/var/www/m-labs-intl.com/html/
command = ${deployWebIntl}
</runcommand>
<runcommand>
job = web:web:nmigen-docs