M-Labs
/
nix-scripts
9
0
Fork 7
Code Issues 6 Pull Requests 3 Releases Wiki Activity

nicer URLs for ARTIQ manuals #12

New Issue
Closed
opened 2019-05-10 11:05:53 +08:00 by sb10q · 11 comments
sb10q commented 2019-05-10 11:05:53 +08:00
Owner

Right now the page https://m-labs.hk/artiq/resources.html points to the Hydra links that then redirect to the latest builds for ARTIQ-5.
Those URLs are a bit too long and also, after Hydra has made the redirection, the resulting URL points to a particular version of the manual which isn't updated and can even be garbage-collected, breaking user bookmarks.

I suggest instead having those URLs:

which then can access any file within the corresponding HTML manual builds.

One way we could do this is by adding a <RunCommand> section in the Hydra configuration which symlinks the /nix/store output of the latest manual builds to something like /var/www/m-labs.hk/artiq/manual-beta (or another location where hydra-queue-runner can write, with corresponding nginx settings to fix the URL).

jq can be a useful program to parse the JSON emitted by Hydra. IIRC the environment used by <RunCommand> is defined by the hydraPath attribute in /pkgs/development/tools/misc/hydra/default.nix.

Right now the page https://m-labs.hk/artiq/resources.html points to the Hydra links that then redirect to the latest builds for ARTIQ-5. Those URLs are a bit too long and also, after Hydra has made the redirection, the resulting URL points to a particular version of the manual which isn't updated and can even be garbage-collected, breaking user bookmarks. I suggest instead having those URLs: * https://m-labs.hk/artiq/manual-beta and https://m-labs.hk/artiq/manual-beta.pdf (for the development version) * https://m-labs.hk/artiq/manual and https://m-labs.hk/artiq/manual.pdf (for the latest release - to be done later) which then can access any file within the corresponding HTML manual builds. One way we could do this is by adding a ``<RunCommand>`` section in the Hydra configuration which symlinks the ``/nix/store`` output of the latest manual builds to something like ``/var/www/m-labs.hk/artiq/manual-beta`` (or another location where ``hydra-queue-runner`` can write, with corresponding nginx settings to fix the URL). ``jq`` can be a useful program to parse the JSON emitted by Hydra. IIRC the environment used by ``<RunCommand>`` is defined by the ``hydraPath`` attribute in ``/pkgs/development/tools/misc/hydra/default.nix``.
astro commented 2019-05-14 00:56:54 +08:00

Commit 25de5790cd was a bit too eager, still playing with <RunCommand> locally.

Commit 25de5790cda9fb5ddf01145cffb6ac3a8f424117 was a bit too eager, still playing with `<RunCommand>` locally.
sb10q commented 2019-05-20 10:00:07 +08:00
Poster
Owner

I've fixed the issue with jq being not found, and also made it print the path to the docs.

I've fixed the issue with jq being not found, and also made it print the path to the docs.
astro commented 2019-05-21 01:05:01 +08:00

Please review 71d631d. I have tested the changes in a virtual NixOS machine with Hydra and Nginx.

Any desires regarding code structure? As hydraWwwOutputs affects a few already existing configurations, splitting it up into multiple files may not be good for readability. YMMV.

Please review 71d631d. I have tested the changes in a virtual NixOS machine with Hydra and Nginx. Any desires regarding code structure? As `hydraWwwOutputs` affects a few already existing configurations, splitting it up into multiple files may not be good for readability. YMMV.
sb10q commented 2019-05-21 09:18:23 +08:00
Poster
Owner

Looks good, thanks!

Looks good, thanks!
sb10q closed this issue 2019-05-21 09:18:23 +08:00
sb10q commented 2019-05-21 13:50:45 +08:00
Poster
Owner

Might need "Correct Caching when Serving Static Files from /nix/store"
from https://nixos.wiki/wiki/Nginx

Might need "Correct Caching when Serving Static Files from /nix/store" from https://nixos.wiki/wiki/Nginx
sb10q reopened this issue 2019-05-21 14:12:00 +08:00
astro commented 2019-05-22 03:45:56 +08:00

As my solution to that Etags problems required much more code, I have refactored out into a separate module that is configured through services.hydraWwwOutputs. I hope that NixOS imports merging will always do the right thing.

As my solution to that Etags problems required much more code, I have refactored out into a separate module that is configured through `services.hydraWwwOutputs`. I hope that NixOS imports merging will always do the right thing.
sb10q commented 2019-05-22 09:08:40 +08:00
Poster
Owner

Uh, that's quite complicated. Can't we just disable etags completely when serving those paths?

Uh, that's quite complicated. Can't we just disable etags completely when serving those paths?
sb10q commented 2019-05-22 09:12:56 +08:00
Poster
Owner

Alternatively, set the date of the manual files in /nix/store to the date of the commit when building the manual. I'm not sure if Nix enforces timestamps in the store, or if this enforcement can be disabled in a simple way.

Alternatively, set the date of the manual files in /nix/store to the date of the commit when building the manual. I'm not sure if Nix enforces timestamps in the store, or if this enforcement can be disabled in a simple way.
sb10q commented 2019-05-22 09:24:36 +08:00
Poster
Owner

Changing the dates in /nix/store doesn't seem easy, so the alternative option (to disabling etags) would be to copy the files and fix their dates (e.g. just set the current time from RunCommand) instead of symlinking.

Changing the dates in /nix/store doesn't seem easy, so the alternative option (to disabling etags) would be to copy the files and fix their dates (e.g. just set the current time from RunCommand) instead of symlinking.
sb10q commented 2019-05-22 19:25:07 +08:00
Poster
Owner

I've disabled caching for now. After looking more closely at what browsers do, the "last-modified header" system would have been a problem as well.

I've disabled caching for now. After looking more closely at what browsers do, the "last-modified header" system would have been a problem as well.
sb10q closed this issue 2019-05-22 19:25:07 +08:00
astro commented 2019-05-23 05:25:50 +08:00

so the alternative option (to disabling etags) would be to copy the files and fix their dates (e.g. just set the current time from RunCommand) instead of symlinking.

If copying is an option, I agree: this is the simplest solution.

I will take care of other issues until you choose to reopen this one. Readers of the manual will appreciate cachability for sure.

> so the alternative option (to disabling etags) would be to copy the files and fix their dates (e.g. just set the current time from RunCommand) instead of symlinking. If copying is an option, I agree: this is the simplest solution. I will take care of other issues until you choose to reopen this one. Readers of the manual will appreciate cachability for sure.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
master
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/nix-scripts#12
There is no content yet.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may exist for a short time before cleaning up, in most cases it CANNOT be undone. Continue?