srenblad
/
artiq
forked from M-Labs/artiq
1
0
Fork 0
Code Pull Requests Activity
7,520 Commits 63 Branches 34 Tags 19 MiB
Commit Graph

6040 Commits

Author SHA1 Message Date
Robert Jördens 3e036e365a phaser: nco, settings and init tweaks 2020-09-22 09:52:49 +00:00
Robert Jördens fdb2867757 phaser: fewer iotest patterns 2020-09-21 17:06:26 +02:00
Robert Jördens d730851397 phaser: elaborate init sequence, more tests 2020-09-21 15:05:29 +00:00
Robert Jördens f0959fb871 phaser: iotest early, check_alarms 2020-09-17 14:13:58 +00:00
Robert Jördens b15e388b5f ad53xx: distinguish errors 2020-09-17 14:13:10 +00:00
Sebastien Bourdeauducq 29c940f4e3 kasli2: forward sma_clkin to si5324 2020-09-17 16:53:43 +08:00
Robert Jördens 868a9a1f0c phaser: new multidds 2020-09-16 14:06:38 +00:00
Robert Jördens c18f515bf9 phaser: rework rtio channels, sync_dly, init() 2020-09-16 12:23:07 +00:00
Robert Jördens f3b0398720 phaser: n=2, m=16, sync_dly 2020-09-16 09:19:15 +00:00
Robert Jördens 9b58b712a6 phaser: doc tweaks 2020-09-15 12:35:26 +00:00
Robert Jördens ff57813a9c phaser: init [wip] 2020-09-15 08:46:47 +00:00
Robert Jördens 07418258ae phaser: init [wip] 2020-09-15 08:46:10 +00:00
Robert Jördens 3a79ef740b phaser: work around integer size 2020-09-15 08:46:10 +00:00
Robert Jördens b449e7202b phaser: rework docs 2020-09-15 08:46:10 +00:00
Robert Jördens b619f657b9 phaser: doc tweaks 2020-09-12 19:59:49 +02:00
Robert Jördens c3728678d6 phaser: document, elaborate comments, some fixes 2020-09-12 17:35:14 +00:00
Robert Jördens e505dfed5b phaser: refactor coredevice driver 2020-09-12 14:17:40 +00:00
Robert Jördens fdd2d6f2fb phaser: SI methods 2020-09-12 11:02:37 +00:00
Sebastien Bourdeauducq bff611a888 test: relax test_dma_playback_time on Zynq 2020-09-11 11:21:45 +08:00
Robert Jördens 4e24700205 phaser: spelling 2020-09-09 16:52:52 +00:00
Robert Jördens 8aaeaa604e phaser: share_lut 2020-09-07 16:06:35 +00:00
Robert Jördens e69bb0aeb3 phaser: add comment about get_dac_data 2020-09-07 16:06:16 +00:00
pca006132 6195b1d3a0 rpc: fixed _write_bool 
Closes #1519
 2020-09-04 13:49:22 +08:00
Robert Jördens 56aa22caeb fastino: document/cleanup 
* added documentation on `update`/`hold` mechanism
* mask machine unit values
* cleanup coredevice driver

close #1518
 2020-09-03 17:44:26 +02:00
Astro 1b475bdac4 build_soc: remove assertion that was used for test runs 2020-09-03 20:24:18 +08:00
Harry Ho 458a411320
metlino_sayma_ttl: Fix RTIO frequency & demo code (#1516) 2020-09-03 15:08:31 +08:00
Sebastien Bourdeauducq 47e88dfcbe Revert "test: temporarily disable test_async_throughput" 
This reverts commit f0289d49ab.
 2020-09-03 14:19:55 +08:00
Astro 002a71dd8d build_soc: rename identifier_str to gateware_identifier_str 2020-09-02 00:00:57 +08:00
Sebastien Bourdeauducq 4398a2d5fa test: relax loopback gate timing 2020-09-01 17:50:09 +08:00
Sebastien Bourdeauducq f0289d49ab test: temporarily disable test_async_throughput 
M-Labs/artiq-zynq#104
 2020-09-01 17:49:40 +08:00
Sebastien Bourdeauducq 8d5dc0ad2a test: relax test_pulse_rate on Zynq 2020-09-01 17:08:26 +08:00
Sebastien Bourdeauducq f294d039b3 test: skip NonexistentI2CBus if I2C is not supported 2020-09-01 16:47:04 +08:00
Astro 91df3d7290 build_soc: override identifier_str only for gateware 2020-09-01 10:46:39 +08:00
Harry Ho 3d84135810 examples: add Metlino master, Sayma satellite with TTLOuts via FMC 2020-08-31 16:21:45 +08:00
Harry Ho dfbf3311cb sayma_amc: add support for 4x DIO output channels via FMC 2020-08-31 16:21:45 +08:00
Harry Ho 1ad9deaf91 fmcdio_vhdci_eem: fix pin naming 2020-08-31 16:21:45 +08:00
Astro 45ae6202c0 build_soc: add identifier_str override option 
Signed-off-by: Stephan Maka <stephan@spaceboyz.net>
 2020-08-31 11:48:58 +08:00
Robert Jördens 272dc5d36a phaser: documentation 2020-08-28 16:36:44 +00:00
pca006132 b2572003ac RPC: optimization by caching 
This reduced the calls needed for socket send/recv.
 2020-08-28 14:58:34 +08:00
pca006132 69f0699ebd test: improved test_performance 
1. Added tests for small payload.
2. Added statistics.
 2020-08-28 14:58:34 +08:00
Sebastien Bourdeauducq 7cf974a6a7 comm_kernel: fix typo 2020-08-28 12:25:23 +08:00
Robert Jördens 68bfa04abb phaser: trf readback strobe spi changes 2020-08-27 15:31:42 +00:00
Robert Jördens 96fc248d7c phaser: synchronize multidds to frame 2020-08-27 14:28:19 +00:00
Robert Jördens c10ac2c92a phaser: add trf, duc, interfaces, redo body assembly, use more natrual iq ordering (i lsb) 2020-08-27 14:26:09 +00:00
Robert Jördens e5e2392240 phaser: wire up multidds 2020-08-26 17:12:41 +00:00
Robert Jördens d1be1212ab phaser: coredevice shim, dds [wip] 2020-08-26 15:10:50 +00:00
pca006132 aac2194759 Ported rpc changes to or1k 2020-08-26 14:17:06 +08:00
pca006132 7181ff66a6 compiler: improved rpc performance for list and array 
1. Removed duplicated tags before each elements.
2. Use numpy functions to speedup parsing.
 2020-08-26 14:17:06 +08:00
pca006132 cfddc13294 test: fixed test_performance 
Added more tests and use normal rpc instead of async rpc.

Async RPC does not represent the real throughput which is limited by the
hardware and the network. Normal RPC which requires a response from the
remote is closer to real usecases.
 2020-08-26 14:17:06 +08:00
Robert Jördens 20fcfd95e9 phaser: coredevice shim, readback fix 2020-08-24 15:46:31 +00:00
Robert Jördens bcefb06e19 phaser: ddb template, split crc 2020-08-24 14:51:50 +00:00
Robert Jördens 11c9def589 phaser: readback delay, test fastlink 2020-08-24 14:49:36 +00:00
Paweł Kulik eb350c3459 Drive SFP0 TX_DISABLE low during startup (as was in Kasli v1.1). Fixes Ethernet on SFP modules with pullup on this line. 
Signed-off-by: Paweł Kulik <pawel.kulik@creotech.pl>
 2020-08-24 21:39:53 +08:00
Robert Jördens 63e4b95325 fastlink: rework crc injection 2020-08-23 19:41:13 +00:00
Robert Jördens a27a03ab3c fastlink: fix crc vs data width 2020-08-23 19:02:50 +00:00
Robert Jördens 7e584d0da1 fastino: use fastlink 2020-08-22 11:56:23 +00:00
Robert Jördens 3e99f1ce5a phaser: refactor link 2020-08-22 11:56:23 +00:00
Robert Jördens a34a647ec4 phaser: refactor fastlink 2020-08-22 11:56:23 +00:00
Robert Jördens aa0154d8e2 phaser: initial 2020-08-22 11:56:23 +00:00
Sebastien Bourdeauducq 5f6aa02b61 gui: unbreak background 2020-08-14 13:14:45 +08:00
David Nadlinger 69718fca90 gui: Improve fuzzy-select heuristics 
Even though the code already used non-greedy wildcards before,
it would not find the shortest match, as earlier match starts
would still take precedence.

This could possibly be sped up a bit in CPython by doing
everything inside re using lookahead-assertion trickery, but the
current code is already imperceptibly fast for hundreds of
choices.
 2020-08-14 02:13:45 +01:00
pca006132 a46573e97a Revert "test: set uart log level to INFO for DMA tests" 
This reverts commit b05cbcbc24.
 2020-08-13 12:44:33 +08:00
pca006132 b05cbcbc24 test: set uart log level to INFO for DMA tests 2020-08-13 12:24:57 +08:00
Sebastien Bourdeauducq 48008eaf5f test: omit unavailable math functions on OR1K 2020-08-12 15:01:13 +08:00
Sebastien Bourdeauducq d8cd5023f6 runtime: expose more libm functions 2020-08-12 13:36:06 +08:00
David Nadlinger c6f0c4dca4 test/coredevice: Ignore jagged 2D array embedding test for now 2020-08-10 00:23:38 +01:00
David Nadlinger daf57969b2 compiler: Do not expand strings into TInt(8)s in array() 2020-08-09 23:46:45 +01:00
David Nadlinger 778f2cf905 compiler: Fix numpy.full, implement for >1D 2020-08-09 23:46:45 +01:00
David Nadlinger 53d64d08a8 compiler: Fix multi-dim slice error message test, tweak wording 2020-08-09 23:14:56 +01:00
David Nadlinger d35f659d25 compiler: Add additional math fns available from Rust libm 2020-08-09 20:09:43 +01:00
David Nadlinger a39bd69ca4 compiler: Implement numpy.rint() using llvm.round() 2020-08-09 19:44:58 +01:00
David Nadlinger ae47d4c0ec test/coredevice: Add host/device consistency checks for NumPy math 2020-08-09 19:15:43 +01:00
David Nadlinger 8e262acd1e compiler: Slight array op implementation cleanup [nfc] 
array_unaryop_funcs was never used; since the mangled names
are unique, a single dictionary would be nicer for overrides
anyway.s
 2020-08-09 18:58:01 +01:00
David Nadlinger 33d931a5b7 compiler: Implement multi-dimensional indexing of arrays 
This generates rather more code than necessary, but has
the advantage of automatically handling incomplete
multi-dimensional subscripts which still leave arrays
behind.
 2020-08-09 17:08:43 +01:00
David Nadlinger b00ba5ece1 compiler: Support explicit array(…, dtype=…) syntax 2020-08-09 17:08:43 +01:00
David Nadlinger ad34df3de1 compiler: Support numpy.float 
This would previously crash the compiler.
 2020-08-09 17:08:43 +01:00
David Nadlinger 8783ba2072 compiler/firmware: RPCs for ndarrays 2020-08-09 17:08:43 +01:00
David Nadlinger 5472e830f6 compiler: Assume array()s are always rectangular 2020-08-09 03:54:42 +01:00
David Nadlinger 8eddb9194a test/lit: Add smoke test for math function broadcasting 2020-08-09 03:54:42 +01:00
David Nadlinger 1c645d8857 compiler: Unbreak quoting of 1D ndarrays 
Lists and arrays no longer have the same representation all
the way through codegen, as used to be the case.

This could/should be made more efficient later, eliding the
temporary copies.
 2020-08-09 03:54:42 +01:00
David Nadlinger df8f1c5c5a compiler: Annotate math functions nounwind/nowrite 2020-08-09 03:54:42 +01:00
David Nadlinger cc00ae9580 compiler: Implement broadcasting of math functions 2020-08-09 03:54:42 +01:00
David Nadlinger be7d78253f compiler: Implement 1D-/2D- array transpose 
Left generic transpose (shape order inversion) for now, as that
would be less ugly if we implement forwarding to Python function
bodies for array function implementations.

Needs a runtime test case.
 2020-08-09 03:54:42 +01:00
David Nadlinger faea886c44 compiler: Implement array vs. scalar broadcasting 2020-08-09 03:54:42 +01:00
David Nadlinger 56a872ccc0 compiler: Insert array binop shape check in caller for location information 2020-08-09 03:54:42 +01:00
David Nadlinger ef260adca8 compiler: Implement matrix multiplication 
LLVM will take care of optimising the loops. This was still
unnecessarily painful; implementing generics and implementing
this in ARTIQ Python looks very attractive right now.
 2020-08-09 03:54:42 +01:00
David Nadlinger 0da4a61d99 compiler: Fix method name typo [nfc] 2020-08-09 03:54:42 +01:00
David Nadlinger 78afa2ea8e compiler: Support MatMult in inferencer 
Still needs actual codegen support.
 2020-08-09 03:54:42 +01:00
David Nadlinger 4d48470320 compiler: Support common numpy.* math functions 
Relies on the runtime to provide the necessary
(libm-compatible) functions.

The test is nifty, but a bit brittle; if this breaks in the
future because of optimizer changes, do not hesitate to convert
this into a more pedestrian test case.
 2020-08-09 03:54:41 +01:00
David Nadlinger d37503f21d compiler: T{C -> External}Function, clarify docs [nfc] 2020-08-09 03:54:41 +01:00
David Nadlinger da255bee1b compiler: Implement element type coercion for arrays 
So far, this is not exposed to the user beyond implicit conversions.

Note that all the implicit conversions, such as triggered by adding
arrays of mismatching types, or dividing integer arrays, are currently
emitted in a maximally inefficient way, where a temporary copy is first
made for the type conversion. The conversions would more sensibly be
implemented during the per-element operations to save on the extra
copies, but the current behaviour fell out of the rest of the IR
generator structure without extra changes.
 2020-08-09 03:54:41 +01:00
David Nadlinger 4426e4144f compiler: Implement unary plus/minus for arrays 
Implementation is needlessly generic to anticipate
coercion/transcendental functions.
 2020-08-09 03:54:41 +01:00
David Nadlinger 0d8fbd4f19 test/lit: Add a test for matrix binary operations 
No reason to believe other operations won't work the same.
(More exhaustive tests to follow using embedding for comparison
against NumPy.)
 2020-08-09 03:54:41 +01:00
David Nadlinger 7bdd6785b7 test/lit: Basic ndarray smoke tests for all binops 2020-08-09 03:54:41 +01:00
David Nadlinger 4d002c7934 compiler: Explain use of rpc_tag() in array ops, formatting [nfc] 2020-08-09 03:54:41 +01:00
David Nadlinger a7e855b319 compiler.types: Change invalid default value [nfc] 
This wasn't actually ever used, but was a dict instead of a set.
 2020-08-09 03:54:41 +01:00
David Nadlinger 48fb80017f compiler: Implement basic element-wise array operations 2020-08-09 03:54:41 +01:00
David Nadlinger 9af6e5747d compiler: Factor rpc_tag() out of llvm_ir_generator 2020-08-09 03:54:41 +01:00
David Nadlinger e77c7d1c39 compiler: Add inferencer support for array operations 2020-08-09 03:54:41 +01:00
David Nadlinger ef57cad1a3 compiler: Test ndarray element assignment 2020-08-09 03:54:41 +01:00
David Nadlinger a9a975e5d4 language: Allow instantating TArray using bare ints 2020-08-09 03:54:41 +01:00
David Nadlinger 504b8f0148 language: Export TArray 2020-08-09 03:54:41 +01:00
David Nadlinger dea3c0c572 compiler: Don't store redundant ndarray buffer length, match list layout 
This adds `elt` to _TPointer and the ir.Offset IR instruction,
which is like GetElem but without the final load.
 2020-08-09 03:54:41 +01:00
David Nadlinger e82357d180 compiler: Fix inferencer tests after adding TArray.num_dims 2020-08-09 03:54:41 +01:00
David Nadlinger cb1cadb46a compiler: Fix/test 1D array construction from generic iterables 2020-08-09 03:54:41 +01:00
David Nadlinger 38c17622cc compiler: Axis-wise iteration of ndarrays 
Matches NumPy. Slicing a TList reallocates, this doesn't; offsetting
couldn't be handled in the IR without introducing new semantics
(the Alloc kludge; could/should be made its own IR type).
 2020-08-09 03:54:41 +01:00
David Nadlinger c95a978ab6 compiler: Iteration for 1D ndarrays 2020-08-09 03:54:41 +01:00
David Nadlinger bc17bb4d1a compiler: Parametrize TArray in number of dimensions 2020-08-09 03:54:41 +01:00
David Nadlinger 632c5bc937 compiler: Add ndarray .shape access 2020-08-09 03:54:41 +01:00
David Nadlinger 40f59561f2 compiler: Add test for length of empty arrays [nfc] 
This makes sure we are actually emitting this as an 1D array
(like NumPy does).
 2020-08-09 03:54:41 +01:00
David Nadlinger d882f8a3f0 compiler: Implement len() for ndarrays 2020-08-09 03:54:41 +01:00
David Nadlinger 575be2aeca compiler: Basic support for creation of multidimensional arrays 
Breaks all uses of array(), as indexing is not yet implemented.
 2020-08-09 03:54:41 +01:00
David Nadlinger 56010c49fb compiler/inferencer: Detect rectangular array()s 
Still needs support through all the rest of the compiler, and
support for higher-dimensional arrays.

Alternatively, we could always assume ndarrays of ndarrays
are rectangular (i.e. ban array/list element types), and
detect mismatch at runtime. This might turn out to be
preferrable to be able to construct matrices from rows/columns.

`array()` is disallowed for no particularly good reason but
numpy API compatibility.
 2020-08-09 03:54:41 +01:00
David Nadlinger 6ea836183d test/lit: Move some list tests to appropriate module [nfc] 2020-08-09 03:54:41 +01:00
pmldrmota 1df62862cd
AD9910: Write correct number of bits to POW register (#1498) 
* coredevice.ad9910: Add return type hints to conversion functions

* coredevice.ad9910: Make set_pow write correct number of bits
The AD9910 expects 16 bits. Thus, if writing 32 bits to the POW register, the chip would likely enter a locked-up state.

* coredevice.ad9910: Correct data alignment in write_16

Co-authored-by: Robert Jördens <rj@quartiq.de>

* coredevice.ad9910: Add function to read from 16 bit registers

Co-authored-by: drmota <peter.drmota@physics.ox.ac.uk>
Co-authored-by: Robert Jördens <rj@quartiq.de>
 2020-08-07 10:10:44 +02:00
Sebastien Bourdeauducq 504f72a02c rtio: remove legacy i_overflow_reset CSR 2020-08-06 17:52:32 +08:00
Sebastien Bourdeauducq 5f36e49f91 test_rtio: make DMA test generic wrt TTL channel 2020-08-06 16:36:14 +08:00
pca006132 3bfd372c20 compiler: linker discard local symbols. 
Fixes exception backtrace problem for ARM.
 2020-08-06 16:07:28 +08:00
Sebastien Bourdeauducq e3c5775584 test: skip CacheTest.test_borrow on Zynq 2020-08-06 10:54:30 +08:00
David Nadlinger ae999db8f6 compiler: Revert function call lifetime tracking fix 
This reverts commits f8d1506922
and cf19c9512d.

While the commit just fixes a clear typo in the implementation,
it turns out the original algorithm isn't flexible enough to
capture functions that transitively return references to
long-lived data. For instance, while cache_get() is special-cased
in the compiler to be recognised as returning a value of Global()
lifetime, a function just forwarding to it (as seen in the
embedding tests) isn't anymore.

A separate issue is also that this makes implementing functions
that take lists and return references to global data in user code
impossible, which central parts of the Oxford codebase rely on.

Just reverting for now to unblock master; a fix is easily designed,
but needs testing.
 2020-07-30 16:40:39 +01:00
Sebastien Bourdeauducq 709026d945 test: relax device_to_host_rate 2020-07-30 17:46:22 +08:00
Sebastien Bourdeauducq 455e4859b7 simplify versioneer 
Original version is very complex and still has a number of problems.
 2020-07-30 00:54:07 +08:00
Sebastien Bourdeauducq 5fd0d0bbb6 gui: work around quamash bug with python 3.8 2020-07-28 12:08:47 +08:00
David Nadlinger f8d1506922 compiler: Fix lifetime tracking for function call return values 
GitHub: Fixes #1497.
 2020-07-28 00:33:28 +01:00
cw-mlabs e4b16428f5 wrpll: fix run signal 2020-07-27 13:02:02 +08:00
cw-mlabs 8dd9a6d024 wrpll: fix scl signal 2020-07-27 12:59:32 +08:00
Charles Baynham 9b44ec7bc6 parameters: Allow forcing a NumberValue to return a float 
Signed-off-by: Charles Baynham <charles.baynham@npl.co.uk>
 2020-07-27 12:25:51 +08:00
David Nadlinger 1c72585c1b compiler: Handle None-returning function calls used as values 
GitHub: Fixes #1493.
 2020-07-25 02:20:53 +01:00
David Nadlinger 57e759a1ed compiler: Consistently use llunit through llvm_ir_generator [nfc] 2020-07-25 02:20:52 +01:00
Sebastien Bourdeauducq 2a2f5c4d58 comm_analyzer: make header error flag more general 2020-07-20 19:39:19 +08:00
Sebastien Bourdeauducq 553a49e194 test_moninj: set loop_out as output 2020-07-19 17:59:43 +08:00
Sebastien Bourdeauducq 8510bf4e55 test_analyzer: configure loop_out as output 2020-07-16 19:28:58 +08:00
pca006132 eb28d7be3a firmware/rpc: fixed typo 2020-07-16 15:15:47 +08:00
pca006132 f78d673079 firmware/rpc: added `#[repr(C)]` for structs. 
Previously the structs are in repr(Rust) which has no layout guarantee.
 2020-07-16 15:11:17 +08:00
Robert Jördens e31ee1f0b3 firmware/i2c: rewrite I2C implementation 
* Never drive SDL or SDA high. They are specified to be open
  collector/drain and pulled up by resistive pullups. Driving
  high fails miserably in a multi-master topology (e.g. with
  a USB I2C interface). It would only ever be implemented to
  speed up the bus actively but that's tricky and completely
  unnecessary here.
* Make the handover states between the I2C protocol phases (start, stop,
  restart, write, read) well defined. Add comments stressing those
  pre/postconditions.
* Add checks for SDA arbitration failures and stuck SCL.
* Remove wrong, misleading or redundant comments.
 2020-07-15 16:43:07 +08:00
Sebastien Bourdeauducq 4340a5cfc1 rtio/dma: fix previous commit 2020-07-12 10:14:22 +08:00
Sebastien Bourdeauducq f2e0d27334 rtio/dma: remove dead/broken code 2020-07-12 10:13:18 +08:00
Sebastien Bourdeauducq 901be75ba4 sayma_rtm: fix Si5324 reset 
Closes #1483
 2020-07-11 09:51:01 +08:00
Sebastien Bourdeauducq 8719bab726 Revert "i2c: duplicate TCA9548 control byte" 
This reverts commit f265976df6.
 2020-07-08 19:02:02 +08:00
Sebastien Bourdeauducq f273a9aacc artiq_ddb_template: remove SFP LEDs on hw 2.0+ 2020-07-08 18:15:36 +08:00
Sebastien Bourdeauducq 2d1f1fff7f kasli_generic: do not attempt to use SFP LED for RTIO on 2.0+ 2020-07-08 18:14:44 +08:00
Sebastien Bourdeauducq 85b5a04acf test: print transfer rates in MiB/s 2020-07-07 17:28:47 +08:00
Sebastien Bourdeauducq 13501115f6 test: remove watchdog test (#1458) 2020-07-07 17:28:47 +08:00
Donald Sebastian Leung f265976df6 i2c: duplicate TCA9548 control byte 2020-07-03 16:45:05 +08:00
David Nadlinger 3f0cf6e683 runtime: Stop kernel CPU before restarting comms CPU on panic 
Before, the system would enter a boot loop when a panic occurred
while the kernel CPU was active (and panic_reset == 1), as
kernel::start() for the startup kernel would panic.
 2020-07-01 17:29:05 +08:00
Sebastien Bourdeauducq 95807234d9 compiler: use binutils for ARM 
This is mostly due to Windoze, where installing anything is a PITA and the LLVM tools won't be available soon.
 2020-06-28 17:33:03 +08:00
Sebastien Bourdeauducq 89c53c35e8 dashboard: style 2020-06-26 10:12:03 +08:00
David Nadlinger f36692638c dashboard: Add "Quick Open" dialog for experiments on global shortcut 
This is similar to functionality in Sublime Text, VS Code, etc.
 2020-06-26 10:11:33 +08:00
David Nadlinger 966ed5d013 master/scheduler: Fix priority/due date precedence order when waiting to prepare 
See test case – previously, the highest-priority pending run would
be used to calculate the timeout, rather than the earliest one.

This probably managed to go undetected for that long as any unrelated
changes to the pipeline (e.g. new submissions, or experiments pausing)
would also cause _get_run() to be re-evaluated.
 2020-06-19 23:45:52 +01:00
David Nadlinger 7955b63b00 master: Always write results to HDF5 once run stage is reached 
Previously, a significant risk of losing experimental results would
be associated with long-running experiments, as any stray exceptions
while run()ing the experiment – for instance, due to infrequent
network glitches or hardware reliability issue – would cause no
HDF5 file to be written. This was especially troublesome as long
experiments would suffer from a higher probability of unanticipated
failures, while at the same time being more costly to re-take in
terms of wall-clock time.

Unanticipated uncaught exceptions like that were enough of an issue
that several Oxford codebases had come up with their own half-baked
mitigation strategies, from swallowing all exceptions in run() by
convention, to always broadcasting all results to uniquely named
datasets such that the partial results could be recovered and written
to HDF5 by manually run recovery experiments.

This commit addresses the problem at its source, changing the worker
behaviour such that an HDF5 file is always written as soon as run()
starts.
 2020-06-18 17:47:26 +01:00