Commit Graph

272 Commits

Author SHA1 Message Date
4707aef45c split out artiq-comtools 2019-11-14 15:21:51 +08:00
3fd6962bd2 use sipyco (#585) 2019-11-10 15:55:17 +08:00
Charles Baynham
e50a6d5aaf worker_impy: ignore newline at start of experiment docstring 2019-09-20 22:10:49 +08:00
David Nadlinger
84b91ee8bd master/scheduler: Document Deleter semantics [nfc]
From looking at the code, it wasn't obvious to me that this is
supposed to handle multiple calls to delete(). This is the case,
however, when for instance Scheduler.delete()ing a run, which
will then also be deleted again from AnalyzeStage.
2019-05-14 22:37:16 +01:00
93f4f31f45 devices.ctlmgr -> master.ctlmgr 2019-04-20 00:25:44 +08:00
8659c769cb master/language: add methods to set experiment pipeline/priority/flush defaults 2019-03-12 10:54:15 +01:00
David Nadlinger
01c3000ef3 master: Print offending key on HDF5 dataset type error
This helps debugging the cause of TypeErrors arising from types
not handled by the HDF5 serializer, as the backtrace doesn't
otherwise include any useful information.
2019-02-09 20:50:38 +00:00
David Nadlinger
bf84226c7d language: Support appending to datasets 2019-02-09 20:50:38 +00:00
David Nadlinger
0dab7ecd73 master: Include RID in worker exception messages
This helps when debugging unexpected shutdown problems
after the fact.
2019-01-20 19:45:50 +00:00
David Nadlinger
e24e893303 master/scheduler: Fix misleading indentation [nfc] 2019-01-20 19:45:47 +00:00
David Nadlinger
c213ab13ba sync_struct: Notifier.{read -> raw_view}, factor out common dict update code [nfc] 2019-01-19 20:19:17 +00:00
9793632282 enviromnment: rename 'save' in set_dataset to 'archive'. Closes #1171 2018-10-21 12:08:34 +08:00
David Nadlinger
e3cfbfed06 master: Add minimal docstring to worker_impl [nfc] 2018-10-14 10:41:32 +08:00
David Nadlinger
64b9a377da master: Factor RIDCounter out into own module, explain worker_db module [nfc]
The docstrings are quite minimal still, but should already
help with navigating the different layers when getting
accustomed with the code base.

RIDCounter was moved to its own module, as it isn't really
related to the other classes (used from master only).
2018-10-14 10:41:32 +08:00
David Nadlinger
4641ddf002 master: Remove unused import [nfc] 2018-10-14 10:41:32 +08:00
ea7f925852 Revert "worker_db: Only warn on repeated archive read if dataset changed"
Breaks numpy arrays.

This reverts commit 141fcaaa8a.
2018-07-13 10:41:06 +08:00
David Nadligner
141fcaaa8a worker_db: Only warn on repeated archive read if dataset changed
In larger experiments, it is quite natural for the same dataset
to be read from multiple unrelated components. The only situation
where multiple reads from an archived dataset are problematic is
when the valeu actually changes between reads. Hence, this commit
restricts the warning to the latter situation.
2018-07-12 10:15:42 +08:00
9153c4d8a3 use tokenize.open() to open Python source files
Fixes encoding issues especially with device databases modified in obscure editors.
2018-07-07 17:04:56 +08:00
0b4d06c7a9 worker: keep sys.modules untouched until the end of examine() 2018-05-02 12:50:37 +08:00
8079aa6d20 worker: python docs recommend not replacing sys.modules 2018-05-02 12:48:50 +08:00
8c69d939fb worker: restore sys.modules in examine() (#976) 2018-05-02 12:32:35 +08:00
dd6c48fed2 Merge branch 'master' into epoch_time 2017-08-03 12:55:01 +02:00
cc289dd3a0 master: store run_time and start_time as doubles 2017-08-03 10:41:57 +01:00
223501f811 master: use epoch time for timestamps (closes #726) 2017-08-03 10:30:31 +01:00
eabca1f311 master: correct example datestring in help 2017-08-03 10:12:52 +01:00
810bb69989 master: rotate logs at midnight, rather than on log size 2017-08-03 00:31:04 +01:00
e4a631a3d7 scheduler: consider the pipeline flushed if everything has a lower priority than us. Closes #640 2017-05-22 18:43:59 +08:00
cd757c0f16 generate device database from executable python file 2017-05-18 23:14:55 +08:00
8ebb33c05c master: record time run() is called 2017-04-26 23:36:19 +08:00
432c6b99e2 master: still save results when analyze fails. Closes #684 2017-03-27 17:57:02 +08:00
639066c6d8 Add tooltips to experiment arguments 2017-02-03 17:53:40 +08:00
780d6d152c worker: fix handling of archive parameter during dataset get in examine 2017-01-07 16:20:17 +01:00
6aa13fbf25 master/worker_db: set default value for archive 2016-10-19 20:12:16 +08:00
5d184f8061 master: keep dataset manager consistent when set_dataset is called with contradictory attributes 2016-10-18 17:11:07 +08:00
69d96b0158 master: archive input datasets. Closes #587 2016-10-18 17:11:07 +08:00
ed2624545f master: ensure same dataset is in broadcast and local when mutating 2016-10-18 17:11:07 +08:00
1908339d4e scheduler: default submission arguments, closes #577 2016-10-18 17:11:06 +08:00
69099691f7 doc: clarify usage of pause/check_pause, closes #571 2016-10-17 20:08:15 +08:00
387688354c master: optimize repository scan, closes #546 2016-09-09 19:19:01 +08:00
e45c089428 master, dashboard: support applet requests from experiments 2016-09-05 00:53:44 +08:00
84f4725015 cache source on import of modules that may contain kernels. Closes #416 2016-08-06 12:01:49 +08:00
9ca27e6d7f worker_impl: style 2016-07-09 16:58:19 +02:00
cfb9fb808c worker: also return DummyDevice from ExamineDeviceMgr 2016-07-09 16:53:28 +02:00
2a5a1f320f browser, worker: feed experiments dummy devices, closes #454
* just returning `None` as dummy device (like ExamineDeviceMgr)
is not explicit enough, certainly hard to debug
* introducing a special flag for the `build` action does not
seem the right place
2016-07-08 01:23:28 +02:00
fdc25777da master/dataset_db: support keeping old persist flag 2016-07-03 12:19:01 +08:00
4c8a8357b0 worker: increase send_timeout (Windows can be really slow) 2016-07-03 12:18:34 +08:00
03a69ec5b7 scheduler: add check_pause function 2016-06-27 14:37:29 +08:00
7dff446932 language: support setting slices of data in mutate_dataset 2016-06-12 13:08:29 +08:00
eb359ce50b Revert "master/worker: pause/resume devices"
This reverts commit 437b37b158.
2016-06-11 10:11:11 -06:00
44724197c5 Revert "master/worker_db: add pause_devices and resume_devices"
This reverts commit 3c70bc40a4.
2016-06-11 10:11:07 -06:00
10267f39c9 log: use broadcast instead of sync_struct, filter on new messages only (#411) 2016-05-27 23:43:29 -05:00
597d7c389e dashboard: support reloading arguments from HDF5 2016-05-05 00:51:30 +08:00
c50555e11c worker: use unix time for HDF5 start_time 2016-05-03 21:29:54 +08:00
0cf6df1933 master/experiments: log more details about experiment name conflicts 2016-04-16 21:36:21 +08:00
b74aa12351 master/worker_db: rethrow DeviceError exception with device name on device creation failure. Closes #396 2016-04-16 19:55:08 +08:00
caf774579a environment: refactor 2016-04-16 19:31:07 +08:00
12a8c76df9 environment,worker: remove enable_processors 2016-04-16 14:21:01 +08:00
848c86e8a4 master: group results by hour, not by minute 2016-04-15 01:13:53 +08:00
437b37b158 master/worker: pause/resume devices 2016-04-12 15:45:53 +08:00
3c70bc40a4 master/worker_db: add pause_devices and resume_devices 2016-04-12 15:45:30 +08:00
30d557a4f8 worker: run experiment in output directory 2016-04-08 10:53:13 +08:00
1ae8b8ffd1 worker_impl: save expid, rid, start_time 2016-04-08 10:21:53 +08:00
de002ac811 worker_db: factor get_output_prefix() 2016-04-08 10:21:53 +08:00
4759ea337b worker_impl: style 2016-04-05 17:22:16 +08:00
05c61465e0 worker: trust that h5py encodes strings 2016-04-05 17:18:15 +08:00
587a0f4565 worker: trust that h5py maps all types as we want 2016-04-05 17:18:15 +08:00
4e7b004a4e worker, hdf5: move datasets to subgroup 2016-04-05 17:17:18 +08:00
aa61c29efb transfer Python builtin exceptions over pc_rpc and master/worker 2016-04-04 22:02:42 +08:00
774ecceaba master/worker_impl: use ParentDatasetDB in examine mode. Closes #368 2016-04-02 23:06:14 +08:00
a53d32c24e master: always expose full set of worker handlers (#368) 2016-04-02 23:05:16 +08:00
84d50c4caf environment,worker_db: mutate datasets from experiments via dedicated method instead of Notifier. Closes #345 2016-03-29 16:26:14 +08:00
ca4e6e03fa master/worker: use only first line in short_exc_info 2016-03-25 19:43:12 +08:00
0d7d584f08 master/worker_db: add controller_aux_target device_db entry type 2016-03-24 00:48:12 +08:00
4bf0db32be master: publish repository status 2016-03-18 00:40:17 +08:00
790269eee7 master/worker_db: make arguments optional in DDB entries 2016-03-05 00:17:08 +08:00
785691ab98 fix indentation 2016-02-29 21:32:48 +08:00
dda0e0846c worker: print core device tracebacks. Closes #292 2016-02-25 01:25:26 +08:00
251aed047f worker: do not print redundant exception information 2016-02-25 01:10:31 +08:00
fef72506e4 ctlmgr/gui/master: start subprocesses in new pgroup
This only makes a difference on POSIX. It prevents subprocesses
from receiving the signals that the parent receives. For ctlmgr
and master is cuts down on spam on the console (KeyboardInterrupt
tracebacks from all controllers) and enforces that proper
termination is followed.

This does not help if the parent gets SIGKILL (subprocesses
may linger).
2016-02-18 23:51:12 +01:00
155c2ec2ef ctlmgr,worker: set PYTHONUNBUFFERED for subprocesses 2016-02-18 12:41:08 +01:00
6196aaf2f5 master/worker: increase timeouts. Windows VMs can be really slow. 2016-02-16 09:44:50 +01:00
72a993afe0 master: cache last RID. Closes #234 2016-02-15 18:20:50 +01:00
6434a9cd5f Merge branch 'master' into subprocess-termination
* master: (44 commits)
  Revert "conda: restrict binutils-or1k-linux dependency to linux."
  manual/installing: refresh
  use https for m-labs.hk
  gui/log: top cell alignment
  master/log: do not break lines
  conda: fix pyqt package name
  gui/applets: log warning if IPC address not in command
  applets: make sure pyqtgraph imports qt5
  applets: avoid argparse subparser mess
  examples/histogram: artiq -> artiq.experiment
  gui/applets: save dock UID in state
  setup.py: give up trying to check for PyQt
  setup.py: fix PyQt5 package name
  Use Qt5
  applets: fix error message text
  applets: handle dataset mutations
  applets: properly name docks to support state save/restore
  applets: clean shutdown
  protocols/pyon: set support
  protocols/pyon: remove FlatFileDB
  ...
2016-02-11 09:24:45 +01:00
2ce1e6a1e9 master/log: do not break lines 2016-02-10 06:33:41 +01:00
53e5d0a7bb worker: flake8 style cleanup 2016-02-02 15:32:40 -07:00
55006119c8 subprocesses: unify termination logic 2016-02-02 15:32:36 -07:00
5076c85ed6 worker: Windows VMs are slow, increase send_timeout 2016-01-27 21:15:22 +01:00
be5162d60f worker: restore short exception info in first line of log 2016-01-26 22:07:54 +01:00
3cf67afeb1 generalize multiline logging to remote logs and controllers 2016-01-26 21:59:37 +01:00
1fed38a8dc worker: use MultilineFormatter 2016-01-26 21:30:28 +01:00
5aa4de8e89 refactor logging and implement in worker 2016-01-26 20:31:42 +01:00
a583a923d8 worker: use pipe_ipc (no log) 2016-01-26 14:59:36 +01:00
ae19f1c75d master: add filename in worker log entries. Closes #226 2016-01-23 21:43:24 -05:00
cc6b808bf8 master: finer control of worker exception reporting. Closes #233 2016-01-23 21:23:02 -05:00
f9323c3286 master/worker_db/get_last_rid: ignore improperly named files 2016-01-16 10:56:01 -07:00
a808d26fc2 style 2016-01-16 10:48:09 -07:00
whitequark
be560dbc63 Commit missing parts of 13e65c2a. 2016-01-16 03:00:17 +00:00
whitequark
13e65c2a0a scheduler: make sure worker exceptions are not unexpectedly hidden. 2016-01-16 02:20:32 +00:00
whitequark
6bf48e60ba worker: make parent errors readable in log. 2016-01-16 02:06:40 +00:00
whitequark
67d2e7a828 worker: display compile warnings and errors nicely (#227). 2016-01-16 01:28:26 +00:00