785691ab98
fix indentation
2016-02-29 21:32:48 +08:00
72a993afe0
master: cache last RID. Closes #234
2016-02-15 18:20:50 +01:00
cc6b808bf8
master: finer control of worker exception reporting. Closes #233
2016-01-23 21:23:02 -05:00
whitequark
be560dbc63
Commit missing parts of 13e65c2a.
2016-01-16 03:00:17 +00:00
whitequark
13e65c2a0a
scheduler: make sure worker exceptions are not unexpectedly hidden.
2016-01-16 02:20:32 +00:00
5e14afde3e
scheduler: use current (last scanned) repo revision instead of HEAD
2015-12-06 19:00:41 +08:00
2c77c80b4f
master: expose more scheduler APIs to the experiments
2015-10-30 13:41:18 +08:00
828b48ad89
master/scheduler: reduce logging severity of worker exception backtraces to debug
2015-10-28 17:48:50 +08:00
9f04af63e6
scheduler: raise logging severity of errors
2015-10-14 16:02:22 +08:00
139072d402
Graceful experiment termination. Closes #76
2015-10-06 13:50:00 +08:00
f552d62b69
use Python 3.5 coroutines
2015-10-03 19:28:57 +08:00
cd3107ba75
do not use deprecated asyncio.JoinableQueue
2015-10-03 13:59:18 +08:00
06badd1dc1
scheduler: refactor, fix pipeline hazards
2015-08-10 21:58:11 +08:00
54d85efc2a
master,gui: show Git commit messages in schedule
2015-08-08 11:08:04 +08:00
7ed8fe57fa
Git support
2015-08-07 15:51:56 +08:00
96a5d73c81
worker: split build stage from prepare
2015-07-09 13:18:12 +02:00
9f9079589e
gui: send monitor requests to core device
2015-06-05 14:52:41 +08:00
aa242f7c66
scheduler: simplify priority policy
...
Remove overdueness. User must submit calibration experiments with higher priority values for them to take precedence.
2015-05-28 18:24:45 +08:00
b0f8141018
scheduler: cancel flush when run is cancelled
2015-05-28 17:48:33 +08:00
e752e57fa5
scheduler: do not duplicate 'run terminated' information
2015-05-28 17:37:08 +08:00
737f6d4485
scheduler: support pipeline flush
2015-05-28 17:20:58 +08:00
fc449509b8
scheduler: pass priority to experiments
2015-05-24 20:37:47 +08:00
a21373841c
scheduler: catch worker exceptions in prepare and analyze stages
2015-05-24 20:23:49 +08:00
d6ced1c780
scheduler: support priorities
2015-05-24 01:09:22 +08:00
b74b8d5826
Scheduling TNG
2015-05-17 16:11:00 +08:00
43a05c783d
worker: split write_results action
2015-03-11 19:06:46 +01:00
d5795fd619
master: watchdog support
...
Introduces a watchdog context manager to use in the experiment code that
terminates the process with an error if it times out. The syntax is:
with self.scheduler.watchdog(20*s):
...
Watchdogs timers are implemented by the master process (and the worker
communicates the necessary information about them) so that they can be
enforced even if the worker crashes. They can be nested arbitrarily.
During yields, all watchdog timers for the yielding worker are
suspended [TODO]. Setting up watchdogs is not supported in kernels,
however, a kernel can be called within watchdog contexts (and terminating
the worker will terminate the kernel [TODO]).
It is possible to implement a heartbeat mechanism using a watchdog, e.g.:
for i in range(...):
with self.scheduler.watchdog(...):
....
Crashes/freezes within the iterator or the loop management would not be
detected, but they should be rare enough.
2015-03-11 16:43:14 +01:00
f2134fa4b2
master,worker: split prepare/run/analyze
2015-03-09 23:34:09 +01:00
4c280d5fcc
master: use a new worker process for each experiment
2015-03-09 16:22:41 +01:00
6601bebcfe
master: make RIDs unique across restarts
2015-02-21 18:41:07 -07:00
cc172699ea
master: use RID + unit class name for HDF5 filenames
2015-02-20 14:11:55 -07:00
4d21b78314
master,client,gui: factor timeout into run_params
2015-02-19 20:03:55 -07:00
c69c4d5ce9
master: expose scheduler API to experiments
2015-02-19 12:09:11 -07:00
2fbe22e15e
master,client: support changing real-time results group
2015-01-30 20:36:54 +08:00
47aa634ab5
replace periodic schedule with timed schedule
2015-01-23 00:23:00 +08:00
3e22fe86b5
reorganize files as per discussion with Robert
2015-01-17 19:38:20 +08:00
5df8ffe08e
master: cleaner shutdown
2015-01-15 11:07:55 +08:00
070788a680
separate master modules
2015-01-14 12:16:49 +08:00
