M-Labs/it-infra
M-Labs
/
it-infra
8
0
Fork 6
Code Issues 10 Pull Requests Releases Wiki Activity

Deploy web2019 to the intl domain #40

Merged
sb10q merged 3 commits from 134-deploy into master 2024-08-14 10:54:53 +08:00
Conversation 1 Commits 3 Files Changed 1 +4
esavkin commented 2024-06-20 17:44:10 +08:00
Owner
Copy Link
No description provided.
sb10q reviewed 2024-06-20 17:46:08 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -654,0 +659,4 @@
cp /opt/hydra_id_ed25519.pub $HOME/.ssh/id_ed25519.pub && \
echo "5.78.86.156 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMbV69aqkHdQ1T5lMuALyHjNowU1rottZtEV4OhKQ6Y" > $HOME/.ssh/known_hosts && \
chmod 600 $HOME/.ssh/id_ed25519 && \
rsync -r -c $(jq -r '.outputs[0].path' < $HYDRA_JSON) zolaupd@5.78.86.156:/var/www/m-labs-intl.com/html/
sb10q commented 2024-06-20 17:46:08 +08:00
Owner
Copy Link

Is this tested? I doubt rsync is in scope.
You may also want to use writeShellScript.

Is this tested? I doubt rsync is in scope. You may also want to use writeShellScript.
esavkin force-pushed 134-deploy from be2aa53220 to 7455367033 2024-06-21 10:53:24 +08:00 Compare
sb10q commented 2024-06-24 10:21:08 +08:00
Owner
Copy Link

Temporary files created inside Nix derivations are automatically cleaned up after build, but is it the case here?

Temporary files created inside Nix derivations are automatically cleaned up after build, but is it the case here?
esavkin added 1 commit 2024-06-24 10:58:12 +08:00
1b3efaa5dd Remove temporary files after uploading website 
Signed-off-by: Egor Savkin <es@m-labs.hk>
sb10q reviewed 2024-07-18 12:04:50 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -654,0 +660,4 @@
echo "5.78.86.156 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMbV69aqkHdQ1T5lMuALyHjNowU1rottZtEV4OhKQ6Y" > $HOME/.ssh/known_hosts && \
chmod 600 $HOME/.ssh/id_ed25519 && \
${pkgs.rsync}/bin/rsync -r -c $(jq -r '.outputs[0].path' < $HYDRA_JSON)/ zolaupd@5.78.86.156:/var/www/m-labs-intl.com/html/
rm -rf $HOME
sb10q commented 2024-07-18 12:04:50 +08:00
Owner
Copy Link

Is this still executed if a command fails?

If you had paid attention to my other commits in this repos, you would have noticed that I use trap to handle situations like this.

Is this still executed if a command fails? If you had paid attention to my other commits in this repos, you would have noticed that I use ``trap`` to handle situations like this.
sb10q commented 2024-07-18 12:05:16 +08:00
Owner
Copy Link

Also rm -rf $HOME is bad form in any shell script.

Also ``rm -rf $HOME`` is bad form in any shell script.
esavkin force-pushed 134-deploy from 1b3efaa5dd to 3868eb8919 2024-07-18 15:35:19 +08:00 Compare
sb10q reviewed 2024-07-18 15:36:23 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -654,0 +654,4 @@
<runcommand>
job = web:web:web-intl
command = [ $(jq '.buildStatus' < $HYDRA_JSON) = 0 ] && export TMPSSH=`mktemp -d` && \
trap "rm -rf ${TMPSSH@Q}" EXIT && \
sb10q commented 2024-07-18 15:36:23 +08:00
Owner
Copy Link

Does this work as intended when used in RunCommand?
May be safer to use writeShellScript as I mentioned earlier (and which you ignored, as often).

Does this work as intended when used in RunCommand? May be safer to use writeShellScript as I mentioned earlier (and which you ignored, as often).
esavkin commented 2024-07-18 16:46:47 +08:00
Author
Owner
Copy Link

They forgot to specify anything useful in the docs, but looking at the source code they eventually just run it with IPC::Run3: d7986226f0/src/lib/Hydra/Plugin/RunCommand.pm (L259) . Which under the hood runs the subprocess with perl's system https://metacpan.org/pod/perlfunc#system , which runs it with default shell (/bin/sh).

Also code for checking that trap actually works:

use strict;
use warnings;
use IPC::Run3;

my $output;
open(my $pipe, "| cat");
my $command = "export TMPSSH=\`mktemp -d\` && \\\necho \$TMPSSH && \\\ntrap \"rm -rf \$TMPSSH\" EXIT";
run3 $command, undef, \$output, undef;
run3 "ls -al ${output}";

Will fail with ls: cannot access '/tmp/tmp.bOSgsRnGNW': No such file or directory

They forgot to specify anything useful in the docs, but looking at the source code they eventually just run it with IPC::Run3: https://github.com/NixOS/hydra/blob/d7986226f0666d5aa0032fdcdb9f38eef6a91dd3/src/lib/Hydra/Plugin/RunCommand.pm#L259 . Which under the hood runs the subprocess with perl's `system` https://metacpan.org/pod/perlfunc#system , which runs it with default shell (`/bin/sh`). Also code for checking that `trap` actually works: ```perl use strict; use warnings; use IPC::Run3; my $output; open(my $pipe, "| cat"); my $command = "export TMPSSH=\`mktemp -d\` && \\\necho \$TMPSSH && \\\ntrap \"rm -rf \$TMPSSH\" EXIT"; run3 $command, undef, \$output, undef; run3 "ls -al ${output}"; ``` Will fail with `ls: cannot access '/tmp/tmp.bOSgsRnGNW': No such file or directory`
sb10q commented 2024-07-18 16:47:40 +08:00
Owner
Copy Link

Right. It's not specified and might change.

Right. It's not specified and might change.
sb10q reviewed 2024-07-18 15:38:23 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -654,0 +655,4 @@
job = web:web:web-intl
command = [ $(jq '.buildStatus' < $HYDRA_JSON) = 0 ] && export TMPSSH=`mktemp -d` && \
trap "rm -rf ${TMPSSH@Q}" EXIT && \
mkdir $TMPSSH/.ssh && \
sb10q commented 2024-07-18 15:38:23 +08:00
Owner
Copy Link

Why do you need that .ssh directory if you're overriding each file in the ssh invokation using -o ?

Why do you need that .ssh directory if you're overriding each file in the ssh invokation using -o ?
esavkin force-pushed 134-deploy from 3868eb8919 to d498d6330b 2024-07-18 17:44:41 +08:00 Compare
sb10q reviewed 2024-07-18 18:50:52 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -10,0 +11,4 @@
#!${pkgs.bash}/bin/bash
[ $(jq '.buildStatus' < $HYDRA_JSON) = 0 ]
export TMPSSH=`mktemp -d`
trap "rm -rf ${TMPSSH@Q}" EXIT
sb10q commented 2024-07-18 18:50:52 +08:00
Owner
Copy Link

Why is @Q needed? Any reason other than it was suggested by ChatGPT?

And you can use a local let. That derivation is only used once.

Why is ``@Q`` needed? Any reason other than it was suggested by ChatGPT? And you can use a local let. That derivation is only used once.
esavkin force-pushed 134-deploy from d498d6330b to d3c830f4ef 2024-07-19 10:35:28 +08:00 Compare
esavkin force-pushed 134-deploy from d3c830f4ef to 0cfc234187 2024-07-29 15:30:02 +08:00 Compare
sb10q reviewed 2024-07-29 15:33:50 +08:00
nixbld-etc-nixos/configuration.nix Outdated
@ -646,0 +652,4 @@
cp /opt/hydra_id_ed25519 "$TMPSSH/id_ed25519"
cp /opt/hydra_id_ed25519.pub "$TMPSSH/id_ed25519.pub"
echo "5.78.86.156 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEMbV69aqkHdQ1T5lMuALyHjNowU1rottZtEV4OhKQ6Y" > "$TMPSSH/known_hosts"
chmod 600 "$TMPSSH/id_ed25519"
sb10q commented 2024-07-29 15:33:50 +08:00
Owner
Copy Link

cp -a or similar?

Such a command would be shorter and avoid any time window during which the file is world-readable.

``cp -a`` or similar? Such a command would be shorter and avoid any time window during which the file is world-readable.
esavkin force-pushed 134-deploy from 0cfc234187 to d749f20cc6 2024-07-29 16:11:23 +08:00 Compare
esavkin force-pushed 134-deploy from d749f20cc6 to 9da1ab6707 2024-08-14 10:53:34 +08:00 Compare
sb10q merged commit 18194be5c3 into master 2024-08-14 10:54:53 +08:00
sb10q deleted branch 134-deploy 2024-08-14 10:54:53 +08:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/it-infra#40
No description provided.
Delete Branch "134-deploy"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?