Sebastien Bourdeauducq
4707aef45c
split out artiq-comtools
2019-11-14 15:21:51 +08:00
Sebastien Bourdeauducq
3fd6962bd2
use sipyco ( #585 )
2019-11-10 15:55:17 +08:00
Charles Baynham
e50a6d5aaf
worker_impy: ignore newline at start of experiment docstring
2019-09-20 22:10:49 +08:00
David Nadlinger
84b91ee8bd
master/scheduler: Document Deleter semantics [nfc]
...
From looking at the code, it wasn't obvious to me that this is
supposed to handle multiple calls to delete(). This is the case,
however, when for instance Scheduler.delete()ing a run, which
will then also be deleted again from AnalyzeStage.
2019-05-14 22:37:16 +01:00
Sebastien Bourdeauducq
93f4f31f45
devices.ctlmgr -> master.ctlmgr
2019-04-20 00:25:44 +08:00
Chris Ballance
8659c769cb
master/language: add methods to set experiment pipeline/priority/flush defaults
2019-03-12 10:54:15 +01:00
David Nadlinger
01c3000ef3
master: Print offending key on HDF5 dataset type error
...
This helps debugging the cause of TypeErrors arising from types
not handled by the HDF5 serializer, as the backtrace doesn't
otherwise include any useful information.
2019-02-09 20:50:38 +00:00
David Nadlinger
bf84226c7d
language: Support appending to datasets
2019-02-09 20:50:38 +00:00
David Nadlinger
0dab7ecd73
master: Include RID in worker exception messages
...
This helps when debugging unexpected shutdown problems
after the fact.
2019-01-20 19:45:50 +00:00
David Nadlinger
e24e893303
master/scheduler: Fix misleading indentation [nfc]
2019-01-20 19:45:47 +00:00
David Nadlinger
c213ab13ba
sync_struct: Notifier.{read -> raw_view}, factor out common dict update code [nfc]
2019-01-19 20:19:17 +00:00
Sebastien Bourdeauducq
9793632282
enviromnment: rename 'save' in set_dataset to 'archive'. Closes #1171
2018-10-21 12:08:34 +08:00
David Nadlinger
e3cfbfed06
master: Add minimal docstring to worker_impl [nfc]
2018-10-14 10:41:32 +08:00
David Nadlinger
64b9a377da
master: Factor RIDCounter out into own module, explain worker_db module [nfc]
...
The docstrings are quite minimal still, but should already
help with navigating the different layers when getting
accustomed with the code base.
RIDCounter was moved to its own module, as it isn't really
related to the other classes (used from master only).
2018-10-14 10:41:32 +08:00
David Nadlinger
4641ddf002
master: Remove unused import [nfc]
2018-10-14 10:41:32 +08:00
Sebastien Bourdeauducq
ea7f925852
Revert "worker_db: Only warn on repeated archive read if dataset changed"
...
Breaks numpy arrays.
This reverts commit 141fcaaa8a
.
2018-07-13 10:41:06 +08:00
David Nadligner
141fcaaa8a
worker_db: Only warn on repeated archive read if dataset changed
...
In larger experiments, it is quite natural for the same dataset
to be read from multiple unrelated components. The only situation
where multiple reads from an archived dataset are problematic is
when the valeu actually changes between reads. Hence, this commit
restricts the warning to the latter situation.
2018-07-12 10:15:42 +08:00
Sebastien Bourdeauducq
9153c4d8a3
use tokenize.open() to open Python source files
...
Fixes encoding issues especially with device databases modified in obscure editors.
2018-07-07 17:04:56 +08:00
Sebastien Bourdeauducq
0b4d06c7a9
worker: keep sys.modules untouched until the end of examine()
2018-05-02 12:50:37 +08:00
Sebastien Bourdeauducq
8079aa6d20
worker: python docs recommend not replacing sys.modules
2018-05-02 12:48:50 +08:00
Sebastien Bourdeauducq
8c69d939fb
worker: restore sys.modules in examine() ( #976 )
2018-05-02 12:32:35 +08:00
Robert Jördens
dd6c48fed2
Merge branch 'master' into epoch_time
2017-08-03 12:55:01 +02:00
Chris Ballance
cc289dd3a0
master: store run_time and start_time as doubles
2017-08-03 10:41:57 +01:00
Chris Ballance
223501f811
master: use epoch time for timestamps ( closes #726 )
2017-08-03 10:30:31 +01:00
Chris Ballance
eabca1f311
master: correct example datestring in help
2017-08-03 10:12:52 +01:00
Chris Ballance
810bb69989
master: rotate logs at midnight, rather than on log size
2017-08-03 00:31:04 +01:00
Sebastien Bourdeauducq
e4a631a3d7
scheduler: consider the pipeline flushed if everything has a lower priority than us. Closes #640
2017-05-22 18:43:59 +08:00
Sebastien Bourdeauducq
cd757c0f16
generate device database from executable python file
2017-05-18 23:14:55 +08:00
Chris Ballance
8ebb33c05c
master: record time run() is called
2017-04-26 23:36:19 +08:00
Sebastien Bourdeauducq
432c6b99e2
master: still save results when analyze fails. Closes #684
2017-03-27 17:57:02 +08:00
Chris Ballance
639066c6d8
Add tooltips to experiment arguments
2017-02-03 17:53:40 +08:00
Sebastien Bourdeauducq
780d6d152c
worker: fix handling of archive parameter during dataset get in examine
2017-01-07 16:20:17 +01:00
Sebastien Bourdeauducq
6aa13fbf25
master/worker_db: set default value for archive
2016-10-19 20:12:16 +08:00
Sebastien Bourdeauducq
5d184f8061
master: keep dataset manager consistent when set_dataset is called with contradictory attributes
2016-10-18 17:11:07 +08:00
Sebastien Bourdeauducq
69d96b0158
master: archive input datasets. Closes #587
2016-10-18 17:11:07 +08:00
Sebastien Bourdeauducq
ed2624545f
master: ensure same dataset is in broadcast and local when mutating
2016-10-18 17:11:07 +08:00
Sebastien Bourdeauducq
1908339d4e
scheduler: default submission arguments, closes #577
2016-10-18 17:11:06 +08:00
Sebastien Bourdeauducq
69099691f7
doc: clarify usage of pause/check_pause, closes #571
2016-10-17 20:08:15 +08:00
Sebastien Bourdeauducq
387688354c
master: optimize repository scan, closes #546
2016-09-09 19:19:01 +08:00
Sebastien Bourdeauducq
e45c089428
master, dashboard: support applet requests from experiments
2016-09-05 00:53:44 +08:00
Sebastien Bourdeauducq
84f4725015
cache source on import of modules that may contain kernels. Closes #416
2016-08-06 12:01:49 +08:00
Robert Jördens
9ca27e6d7f
worker_impl: style
2016-07-09 16:58:19 +02:00
Robert Jördens
cfb9fb808c
worker: also return DummyDevice from ExamineDeviceMgr
2016-07-09 16:53:28 +02:00
Robert Jördens
2a5a1f320f
browser, worker: feed experiments dummy devices, closes #454
...
* just returning `None` as dummy device (like ExamineDeviceMgr)
is not explicit enough, certainly hard to debug
* introducing a special flag for the `build` action does not
seem the right place
2016-07-08 01:23:28 +02:00
Sebastien Bourdeauducq
fdc25777da
master/dataset_db: support keeping old persist flag
2016-07-03 12:19:01 +08:00
Sebastien Bourdeauducq
4c8a8357b0
worker: increase send_timeout (Windows can be really slow)
2016-07-03 12:18:34 +08:00
Sebastien Bourdeauducq
03a69ec5b7
scheduler: add check_pause function
2016-06-27 14:37:29 +08:00
Sebastien Bourdeauducq
7dff446932
language: support setting slices of data in mutate_dataset
2016-06-12 13:08:29 +08:00
Sebastien Bourdeauducq
eb359ce50b
Revert "master/worker: pause/resume devices"
...
This reverts commit 437b37b158
.
2016-06-11 10:11:11 -06:00
Sebastien Bourdeauducq
44724197c5
Revert "master/worker_db: add pause_devices and resume_devices"
...
This reverts commit 3c70bc40a4
.
2016-06-11 10:11:07 -06:00
Sebastien Bourdeauducq
10267f39c9
log: use broadcast instead of sync_struct, filter on new messages only ( #411 )
2016-05-27 23:43:29 -05:00
Sebastien Bourdeauducq
597d7c389e
dashboard: support reloading arguments from HDF5
2016-05-05 00:51:30 +08:00
Sebastien Bourdeauducq
c50555e11c
worker: use unix time for HDF5 start_time
2016-05-03 21:29:54 +08:00
Sebastien Bourdeauducq
0cf6df1933
master/experiments: log more details about experiment name conflicts
2016-04-16 21:36:21 +08:00
Sebastien Bourdeauducq
b74aa12351
master/worker_db: rethrow DeviceError exception with device name on device creation failure. Closes #396
2016-04-16 19:55:08 +08:00
Sebastien Bourdeauducq
caf774579a
environment: refactor
2016-04-16 19:31:07 +08:00
Sebastien Bourdeauducq
12a8c76df9
environment,worker: remove enable_processors
2016-04-16 14:21:01 +08:00
Sebastien Bourdeauducq
848c86e8a4
master: group results by hour, not by minute
2016-04-15 01:13:53 +08:00
Sebastien Bourdeauducq
437b37b158
master/worker: pause/resume devices
2016-04-12 15:45:53 +08:00
Sebastien Bourdeauducq
3c70bc40a4
master/worker_db: add pause_devices and resume_devices
2016-04-12 15:45:30 +08:00
Robert Jördens
30d557a4f8
worker: run experiment in output directory
2016-04-08 10:53:13 +08:00
Robert Jördens
1ae8b8ffd1
worker_impl: save expid, rid, start_time
2016-04-08 10:21:53 +08:00
Robert Jördens
de002ac811
worker_db: factor get_output_prefix()
2016-04-08 10:21:53 +08:00
Robert Jördens
4759ea337b
worker_impl: style
2016-04-05 17:22:16 +08:00
Robert Jördens
05c61465e0
worker: trust that h5py encodes strings
2016-04-05 17:18:15 +08:00
Robert Jördens
587a0f4565
worker: trust that h5py maps all types as we want
2016-04-05 17:18:15 +08:00
Robert Jördens
4e7b004a4e
worker, hdf5: move datasets to subgroup
2016-04-05 17:17:18 +08:00
Sebastien Bourdeauducq
aa61c29efb
transfer Python builtin exceptions over pc_rpc and master/worker
2016-04-04 22:02:42 +08:00
Sebastien Bourdeauducq
774ecceaba
master/worker_impl: use ParentDatasetDB in examine mode. Closes #368
2016-04-02 23:06:14 +08:00
Sebastien Bourdeauducq
a53d32c24e
master: always expose full set of worker handlers ( #368 )
2016-04-02 23:05:16 +08:00
Sebastien Bourdeauducq
84d50c4caf
environment,worker_db: mutate datasets from experiments via dedicated method instead of Notifier. Closes #345
2016-03-29 16:26:14 +08:00
Sebastien Bourdeauducq
ca4e6e03fa
master/worker: use only first line in short_exc_info
2016-03-25 19:43:12 +08:00
Sebastien Bourdeauducq
0d7d584f08
master/worker_db: add controller_aux_target device_db entry type
2016-03-24 00:48:12 +08:00
Sebastien Bourdeauducq
4bf0db32be
master: publish repository status
2016-03-18 00:40:17 +08:00
Sebastien Bourdeauducq
790269eee7
master/worker_db: make arguments optional in DDB entries
2016-03-05 00:17:08 +08:00
Sebastien Bourdeauducq
785691ab98
fix indentation
2016-02-29 21:32:48 +08:00
Sebastien Bourdeauducq
dda0e0846c
worker: print core device tracebacks. Closes #292
2016-02-25 01:25:26 +08:00
Sebastien Bourdeauducq
251aed047f
worker: do not print redundant exception information
2016-02-25 01:10:31 +08:00
Robert Jördens
fef72506e4
ctlmgr/gui/master: start subprocesses in new pgroup
...
This only makes a difference on POSIX. It prevents subprocesses
from receiving the signals that the parent receives. For ctlmgr
and master is cuts down on spam on the console (KeyboardInterrupt
tracebacks from all controllers) and enforces that proper
termination is followed.
This does not help if the parent gets SIGKILL (subprocesses
may linger).
2016-02-18 23:51:12 +01:00
Sebastien Bourdeauducq
155c2ec2ef
ctlmgr,worker: set PYTHONUNBUFFERED for subprocesses
2016-02-18 12:41:08 +01:00
Sebastien Bourdeauducq
6196aaf2f5
master/worker: increase timeouts. Windows VMs can be really slow.
2016-02-16 09:44:50 +01:00
Sebastien Bourdeauducq
72a993afe0
master: cache last RID. Closes #234
2016-02-15 18:20:50 +01:00
Robert Jördens
6434a9cd5f
Merge branch 'master' into subprocess-termination
...
* master: (44 commits)
Revert "conda: restrict binutils-or1k-linux dependency to linux."
manual/installing: refresh
use https for m-labs.hk
gui/log: top cell alignment
master/log: do not break lines
conda: fix pyqt package name
gui/applets: log warning if IPC address not in command
applets: make sure pyqtgraph imports qt5
applets: avoid argparse subparser mess
examples/histogram: artiq -> artiq.experiment
gui/applets: save dock UID in state
setup.py: give up trying to check for PyQt
setup.py: fix PyQt5 package name
Use Qt5
applets: fix error message text
applets: handle dataset mutations
applets: properly name docks to support state save/restore
applets: clean shutdown
protocols/pyon: set support
protocols/pyon: remove FlatFileDB
...
2016-02-11 09:24:45 +01:00
Sebastien Bourdeauducq
2ce1e6a1e9
master/log: do not break lines
2016-02-10 06:33:41 +01:00
Robert Jördens
53e5d0a7bb
worker: flake8 style cleanup
2016-02-02 15:32:40 -07:00
Robert Jördens
55006119c8
subprocesses: unify termination logic
2016-02-02 15:32:36 -07:00
Sebastien Bourdeauducq
5076c85ed6
worker: Windows VMs are slow, increase send_timeout
2016-01-27 21:15:22 +01:00
Sebastien Bourdeauducq
be5162d60f
worker: restore short exception info in first line of log
2016-01-26 22:07:54 +01:00
Sebastien Bourdeauducq
3cf67afeb1
generalize multiline logging to remote logs and controllers
2016-01-26 21:59:37 +01:00
Sebastien Bourdeauducq
1fed38a8dc
worker: use MultilineFormatter
2016-01-26 21:30:28 +01:00
Sebastien Bourdeauducq
5aa4de8e89
refactor logging and implement in worker
2016-01-26 20:31:42 +01:00
Sebastien Bourdeauducq
a583a923d8
worker: use pipe_ipc (no log)
2016-01-26 14:59:36 +01:00
Sebastien Bourdeauducq
ae19f1c75d
master: add filename in worker log entries. Closes #226
2016-01-23 21:43:24 -05:00
Sebastien Bourdeauducq
cc6b808bf8
master: finer control of worker exception reporting. Closes #233
2016-01-23 21:23:02 -05:00
Sebastien Bourdeauducq
f9323c3286
master/worker_db/get_last_rid: ignore improperly named files
2016-01-16 10:56:01 -07:00
Sebastien Bourdeauducq
a808d26fc2
style
2016-01-16 10:48:09 -07:00
whitequark
be560dbc63
Commit missing parts of 13e65c2a
.
2016-01-16 03:00:17 +00:00
whitequark
13e65c2a0a
scheduler: make sure worker exceptions are not unexpectedly hidden.
2016-01-16 02:20:32 +00:00
whitequark
6bf48e60ba
worker: make parent errors readable in log.
2016-01-16 02:06:40 +00:00
whitequark
67d2e7a828
worker: display compile warnings and errors nicely ( #227 ).
2016-01-16 01:28:26 +00:00