Commit Graph

57 Commits (master)

Author SHA1 Message Date
Sebastien Bourdeauducq 8dd8cfa6b0 master: implement devarg_override 2023-12-18 12:11:40 +08:00
Sebastien Bourdeauducq ede0b37c6e devices: introduce notify_run_end API 2023-12-13 14:27:04 +08:00
Sebastien Bourdeauducq 795c4372fa DeviceManager: fix close exception error message 2023-12-13 14:06:53 +08:00
Simon Renblad bfbe13e51b worker_db: write hdf5 dataset metadata 2023-07-10 02:33:59 +00:00
Egor Savkin 5199bea353
master: emit warning if datasets will not be stored 2023-04-30 15:22:21 +08:00
hartytp 715bff3ebf
Revert "Merge pull request #1544 from airwoodix/dataset-compression" (#1838)
* Revert "Merge pull request #1544 from airwoodix/dataset-compression"

This reverts commit 311a818a49, reversing
changes made to 7ffe4dc2e3.

* fix accidental revert of f42bea06a8
2022-01-25 10:02:15 +08:00
Leon Riesebos f42bea06a8 worker_db: removed warning for writing a dataset that is also in the archive
Signed-off-by: Leon Riesebos <leon.riesebos@duke.edu>
2022-01-08 11:48:18 +08:00
Etienne Wodey d8b1e59538 datasets: allow passing options to HDF5 backend (e.g. compression)
This breaks the internal dataset representation used by applets
and when saving to disk (``dataset_db.pyon``).

See ``test/test_dataset_db.py`` and ``test/test_datasets.py``
for examples.

Signed-off-by: Etienne Wodey <wodey@iqo.uni-hannover.de>
2021-06-17 12:04:16 +02:00
Sebastien Bourdeauducq db13747279 fix device_db alias corner case bugs. Closes #1140 2019-11-14 16:22:45 +08:00
Sebastien Bourdeauducq 3fd6962bd2 use sipyco (#585) 2019-11-10 15:55:17 +08:00
David Nadlinger 01c3000ef3 master: Print offending key on HDF5 dataset type error
This helps debugging the cause of TypeErrors arising from types
not handled by the HDF5 serializer, as the backtrace doesn't
otherwise include any useful information.
2019-02-09 20:50:38 +00:00
David Nadlinger bf84226c7d language: Support appending to datasets 2019-02-09 20:50:38 +00:00
David Nadlinger c213ab13ba sync_struct: Notifier.{read -> raw_view}, factor out common dict update code [nfc] 2019-01-19 20:19:17 +00:00
Sebastien Bourdeauducq 9793632282 enviromnment: rename 'save' in set_dataset to 'archive'. Closes #1171 2018-10-21 12:08:34 +08:00
David Nadlinger 64b9a377da master: Factor RIDCounter out into own module, explain worker_db module [nfc]
The docstrings are quite minimal still, but should already
help with navigating the different layers when getting
accustomed with the code base.

RIDCounter was moved to its own module, as it isn't really
related to the other classes (used from master only).
2018-10-14 10:41:32 +08:00
Sebastien Bourdeauducq ea7f925852 Revert "worker_db: Only warn on repeated archive read if dataset changed"
Breaks numpy arrays.

This reverts commit 141fcaaa8a.
2018-07-13 10:41:06 +08:00
David Nadligner 141fcaaa8a worker_db: Only warn on repeated archive read if dataset changed
In larger experiments, it is quite natural for the same dataset
to be read from multiple unrelated components. The only situation
where multiple reads from an archived dataset are problematic is
when the valeu actually changes between reads. Hence, this commit
restricts the warning to the latter situation.
2018-07-12 10:15:42 +08:00
Sebastien Bourdeauducq 6aa13fbf25 master/worker_db: set default value for archive 2016-10-19 20:12:16 +08:00
Sebastien Bourdeauducq 5d184f8061 master: keep dataset manager consistent when set_dataset is called with contradictory attributes 2016-10-18 17:11:07 +08:00
Sebastien Bourdeauducq 69d96b0158 master: archive input datasets. Closes #587 2016-10-18 17:11:07 +08:00
Sebastien Bourdeauducq ed2624545f master: ensure same dataset is in broadcast and local when mutating 2016-10-18 17:11:07 +08:00
Robert Jördens 2a5a1f320f browser, worker: feed experiments dummy devices, closes #454
* just returning `None` as dummy device (like ExamineDeviceMgr)
is not explicit enough, certainly hard to debug
* introducing a special flag for the `build` action does not
seem the right place
2016-07-08 01:23:28 +02:00
Sebastien Bourdeauducq 7dff446932 language: support setting slices of data in mutate_dataset 2016-06-12 13:08:29 +08:00
Sebastien Bourdeauducq 44724197c5 Revert "master/worker_db: add pause_devices and resume_devices"
This reverts commit 3c70bc40a4.
2016-06-11 10:11:07 -06:00
Sebastien Bourdeauducq b74aa12351 master/worker_db: rethrow DeviceError exception with device name on device creation failure. Closes #396 2016-04-16 19:55:08 +08:00
Sebastien Bourdeauducq 848c86e8a4 master: group results by hour, not by minute 2016-04-15 01:13:53 +08:00
Sebastien Bourdeauducq 3c70bc40a4 master/worker_db: add pause_devices and resume_devices 2016-04-12 15:45:30 +08:00
Robert Jördens 30d557a4f8 worker: run experiment in output directory 2016-04-08 10:53:13 +08:00
Robert Jördens 1ae8b8ffd1 worker_impl: save expid, rid, start_time 2016-04-08 10:21:53 +08:00
Robert Jördens de002ac811 worker_db: factor get_output_prefix() 2016-04-08 10:21:53 +08:00
Robert Jördens 587a0f4565 worker: trust that h5py maps all types as we want 2016-04-05 17:18:15 +08:00
Robert Jördens 4e7b004a4e worker, hdf5: move datasets to subgroup 2016-04-05 17:17:18 +08:00
Sebastien Bourdeauducq 84d50c4caf environment,worker_db: mutate datasets from experiments via dedicated method instead of Notifier. Closes #345 2016-03-29 16:26:14 +08:00
Sebastien Bourdeauducq 0d7d584f08 master/worker_db: add controller_aux_target device_db entry type 2016-03-24 00:48:12 +08:00
Sebastien Bourdeauducq 790269eee7 master/worker_db: make arguments optional in DDB entries 2016-03-05 00:17:08 +08:00
Sebastien Bourdeauducq 72a993afe0 master: cache last RID. Closes #234 2016-02-15 18:20:50 +01:00
Sebastien Bourdeauducq f9323c3286 master/worker_db/get_last_rid: ignore improperly named files 2016-01-16 10:56:01 -07:00
Sebastien Bourdeauducq 40b4129c65 Only support scalars and numpy arrays in HDF5 output. Update documentation. Closes #145 2015-10-28 18:33:42 +08:00
Sebastien Bourdeauducq ec328cf5e1 master/worker_db: reduce exception backtrace clutter of get_dataset. Closes #168 2015-10-28 17:50:48 +08:00
Sebastien Bourdeauducq fcd29492f9 worker_db: support more types in HDF5 output. Closes #144. Closes #121 2015-10-24 10:54:59 +08:00
Sebastien Bourdeauducq 03e317780b device_db: make best_effort parameter optional for controllers 2015-10-18 14:37:08 +08:00
Sebastien Bourdeauducq a5606768ad device_db: make target_name parameter optional for controllers 2015-10-18 14:35:53 +08:00
Sebastien Bourdeauducq 3cec176318 get_ddb -> get_device_db 2015-10-12 19:46:31 +08:00
Sebastien Bourdeauducq 97accd2540 merge parameters and results into datasets 2015-10-12 17:18:23 +08:00
Sebastien Bourdeauducq b3584bc190 language,master,run: support raw access to DDB from experiments. Closes #123 2015-10-04 18:29:39 +08:00
Sebastien Bourdeauducq 9261254653 language/environment: support non-stored results 2015-08-06 18:43:27 +08:00
Sebastien Bourdeauducq 39c6bc940c coreconfig: use new database API (closes #75) 2015-07-17 10:49:55 +02:00
Sebastien Bourdeauducq 32d141f5ac refactor ddb/pdb/rdb 2015-07-13 22:21:32 +02:00
Yann Sionneau 4bf7875b87 flash_storage: refactor + unit tests + artiq_coreconfig.py CLI + doc 2015-05-27 18:06:12 +08:00
Sebastien Bourdeauducq ca89b6d0ed worker_db: support read-only mode 2015-04-07 15:40:57 +08:00