M-Labs/saymapp
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

from sinara-hw issue 601 prior to m-labs requested changes

Browse Source
This commit is contained in:
Joe Britton 2019-01-13 18:41:39 -05:00
parent 553456acda
commit fe1060ea4d
1 changed files with 324 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

324
pp.txt Normal file
View File

@ -0,0 +1,324 @@
Here's a coherent project plan for Sayma v2. It is informed by extensive discussion on github Issue system over the past year and recent offline discussion with the Sayma v1 developers. This is posted here so everyone knows what the contracts call for in development of Sayma v2. This plan will be updated if contracts differ from the present plan. Please use existing or new Issues for further discussion of technical topics.
This plan defines concrete performance goals, specifies which parties are responsible for specific deliverables and defines a project management structure. The aim is to provide sufficient specification and structure at the outset that all parties feel comfortable with their responsibilities and minimizes the role of ARL/UMD in project management.
The plan includes design choices that simplify Sayma and lower costs. The github Sayma Issue tracking contains a long list of bug fixes and modifications to v1 (https://github.com/sinara-hw/sinara). I call out explicit specifications below in cases where there is not a clear github consensus on how to proceed with Sayma v2. The plan is not meant to substitute for the Issue system except where which-path clarity is required in which case this plan is authoritative. I will annotate relevant github Issues with a link to this Issue so there is clarity about which-path.
# Definitions
Let __TS-MTCA__ be a micro TCA crate configured as follows:
- 1 NAT NATIVE-R5 (or -R9)
- 1 (or 2) NAT AC 600D
- 1 NAT MCH-Basic v3.5
Let __TS1__ be a Sayma test setup consisting of the following
- 1 Sayma AMC+RTM
- Powered by ATX connector from benchtop supply
Let __TS2__ be a Sayma test setup consisting of the following
- Single uTCA chassis configured as TS-MTCA
- 2 Sayma AMC+RTM
- 1 Metlino
- All RF channels producing 400 MHz sinusoid
- Cards in adjacent slots
Let __TS7__ be a Sayma test setup consisting of the following
- Single uTCA chassis configured as TS-MTCA
- 6 Sayma AMC+RTM
- 1 Metlino
- All RF channels producing 400 MHz sinusoid
- Cards in adjacent slots
# Sayma v2 Diffs
When diff also applies for [Metlino](https://github.com/sinara-hw/Metlino/issues) it is noted.
## RTM Synthesizer
- Layout both ADF4356 and HMC830, change capacitors to reroute clocks.
- Use HMC830 as primary synthesizer.
- problem: random lockups at power cycling, possible fix: possibly due to power sequencing
- problem: unsynchronised dividers, fix: Make FPGA measure the output phase and reset the HMC830 divider until synchronised.
- HMC830 lock status exposed on front-panel LED.
- ADF4356 is backup option. No development of ADF4356 until operation with HMC830 is fully tested and succeeds, or HMC830 bug clearly blocks forward progress.
## AMC clock recovery
Discussed in [meta/issues/15](https://github.com/sinara-hw/meta/issues/15). Also applies to [Metlino](https://github.com/sinara-hw/Metlino/issues).
- Use Si5324 as primary approach for Sayma v2.
- Layout both Si5324 and Tom’s WR PLL scheme that uses Si549 in addition to Si5324
- Change capacitors to reroute clocks.
- Confirm that DCXO and the Si5324 can both be shutdown in software (using SFOUTx_REG), otherwise add a FET on power lines.
- Clear delineation on schematics between Si5324 and WR PLL schemes.
- No funding to develop WR PLL support in this contract.
## DAC-FPGA Synchronization
Deterministic phase alignment between DACs on separate Sayma PCBs between power cycles is an important design criterion and should be considered at all stages of design. This implies a) SYSREF is aligned with RTIO clock b) DAC clocks are synchronized to local SYSREF edges.
- Follow approach demonstrated by Tom's prototype (Issue #583) using low jitter FF to retime AMC-FPGA SYNC pulse to f_dac_ref. Summarized in Issue #594.
- Remove HMC7043, add components needed to generate MGTREFCLK.
- Test conditions
- SAWG output on all channels on two Sayma boards, 400 MHz continuous RF
- Sayma-local TTL output at 100 MHz
- laboratory temperature stability +/- 0.5 C
- Cycle crate power 100 times
- Specification
- __ST1__: SAWG-SAWG phase alignment error across
all power cycles: < 1/f_dac_clk
- __ST2__: SAWG-TTL alignment error across all power cycles: < 0.5 ns
## DAC Performance
### DAC Phase Stability
Phase stability between DACs on separate Sayma PCBs is an important design criterion and should be considered at all stages of the design. This assumes that DAC-FPGA synchronization.
- Test conditions
- SAWG output on all channels on two Sayma boards, 400 MHz continuous RF
- Assume laboratory temperature stability +/- 0.5 C
- Duration 24 hours
- Target performance
- __ST3__: SAWG-SAWG phase alignment variation < +/- 0.5 deg
- __ST4__: SAWG-TTL alignment variation < +/-250 ps
### DAC Noise
Low cross talk between adjacent analog channels on same AFE is an important design criterion and should be considered at all stages of the design (particularly layout).
- Goal: at 400 MHz < -80 dBc.
Phase noise at 400 MHz (3 dB margin on AD9154 specification):
- 10 Hz offset, < -94 dBc
- 100 Hz offset, < -103 dBc
- 1000 Hz offset, < -120 dBc
- 10,000 Hz offset, < -126 dBc
## Reduce complexity of Sayma v2
- Remove fast JESD ADCs from RTM layout
- Reduce complexity of clocking options
- Remove clock mezzanine
- User must supply RTM with DAC reference clock (f_dac) via back-panel SMA
- f_rtio is obtained from front-panel SMA (if master) or fiber+CDR (if satellite)
- User must ensure that the following are phase locked: f_dac_ref and f_rtio
- Synth on RTM generates f_dac from f_dac_ref
- No support for analog backplane (LLRFBP, https://github.com/sinara-hw/sinara/issues/498).
- Unambiguous mapping between control pin and IC naming semantics in schematics and corresponding ARTIQ source.
- Remove 24-bit multi-channel ADC from Sayma RTM layout.
## AFE Interface to RTM
- Use FMC connector for RF, digital and power. This approach is validated by tests in https://github.com/sinara-hw/sinara/issues/386.
- Reduce number of AFE's to 2.
## BaseMod AFE v2
- New form factor to accommodate switch to FMC and 4 DAC channels per AFE.
- FMC-type front panels, with connector holes determined by choice of AFE.
- AFE name and version number on FMC front panel.
- No RF power detector.
- Add LVDS ADC. Copy design from Sampler. No development support for ADC in this contract.
- Hardware Developer shall specify PCB material assuming maximum RF frequency is 1 GHz.
## Smart Agile Waveform Generator
Use SAWG v1 with minimal modification to demonstrate Sayma v2 hardware. SAWG v2 will be pursued once Sayma v2 hardware is working.
### SAWG v1
- f_rtio = f_dac_ref = 150 MHz
- dac data rate 0.6 GSPS
- f_dac_clk = 1.2 GHz (2X interpolation)
### SAWG v2
- ARTIQ Python interface to SAWG v2 shall not make implicit modifications to RTIO timeline.
- f_rtio = f_dac_ref = 125 MHz (same as for Kasli)
- dac data rate 1.0 GSPS
- f_dac_clk = 2.0 GHz (2X interpolation) with following expected implications:
- Carrier bandwidth (f0): 500 MHz 1st Nyquist
- Baseband bandwidth (f1, f2): +/- 75 MHz 1st Nyquist
- M-Labs Comment: “To use Sayma with 1 GHz data rate, the parameters of SAWG, the JESD core as well as the clocking setup and DAC configuration code need to be adjusted. This is likely achievable as most of the parts are designed for and were individually already tested at 1 GHz. But it's not straight-forward.”
- __action__: Software and Gateware Developer does study
- Reduce resource consumption
- comment: resource utilization at 1 GSPS may already be acceptable ([link](https://github.com/m-labs/artiq/issues/1144#issuecomment-433447687))
- comment: consider decreasing resolution of f0
- comment: resource consumption of moninj not clear #675
- comment: https://github.com/m-labs/artiq/issues/801 Jordens “imagines several opportunities to improve and simplify SAWG (different DUC scheme as suggested elsewhere, improving and shrinking the spline coefficient representation/serialization, transaction-based SAWG RTIO events which would cut down the number of RTIO channels, raw sample channels, revisiting phase and frequency resolution, study required bit depth/resource usage…”
- feature priority: 1 GSPS data rate, AM for noise eating
- __action__: Software and Gateware Developer does study
## Ethernet
- Replace MAX24287 by M88E1512. This is same family IC as KC705 which is already supported by ARTIQ.
- Configuration of M88E1512 is by static pin strapping: 1 Gbps, GMII, 1000BASE-X.
- RGMII is connected only with FPGA -- no interface to MMC.
- Layout so that uTCA Port0 and SFP can be driven by M88E1512 or by GTH transceiver on FPGA. Use coupling capacitors to choose which.
- Also applies to Metlino.
---------------
It is anticipated that there will be three developers working together. Responsibilities and interfaces between developers are discussed in this section. UMD/ARL will serve as Technical Point of Contact (TPOC).
- Hardware Developer
- Integration and Timing Developer
- Software and Gateware Developer
# Hardware Developer
The Hardware Developer is responsible for hardware design and manufacturing. Testing responsibilities of the Hardware Developer are those detailed in HT3. All deliverables include bi-weekly written progress reports. The Hardware Developer has four deliverables HT1 to HT5.
## __HT1__ high level design
High-level schematics for all PCBs to include component part numbers, high-level component interconnection, power budget, high-level layout of all PCBs including location of ICs, board-to-board headers, RF shielding, heat sinks, mechanical support and front panel design. Post design files to github and tag. Participate in design review on github.
Layout-related considerations:
- silk screen #589
- __action__: Follow Tom's suggestions to label ICs, test points (power, gnd, signals, I2C, SPI), wide pitch for slow signals, push down pull up on logic lines.
- heat sinking #592
- **action**: Follow Greg's advice and add more vias.
HT1 delivery is complete when the Software and Gateware Developer, Integration and Timing Developer and UMD/ARL sign off on the drawings.
## __HT2__ detailed design
Detailed schematics for all PCBs, finished component interconnections, signal integrity analysis, thermal analysis, power analysis, final PCB routing. Post design files to github and tag. Participate in final design review, accept delegated responsibilities from Software and Gateware Developer Design. Output is layout ready for manufacture.
Hardware Developer shall lead HT2 design review. This design review is in collaboration with scientists and other developers. The review will not be comprehensive. It will focus on differences between v1 and v2 of the hardware. Shall include validation of FPGA pinout.
HT2 delivery is complete when Software and Gateware Developer and Integration and Timing Developer sign off on the drawings.
## __HT3__ manufacturing, baseline testing
Fabricate PCBs in the following quantities. TPOC provides sign off prior to commencement of manufacturing.
- Metlino AMC: print 10 PCBs, stuff 4
- Sayma AMC v2: print 10 PCBs, stuff 6
- Sayma RTM v2: print 10 PCBs, stuff 6
- BaseMod v2: print 20 PCBs, stuff 12
- TestMod v2: print 6 PCBs, stuff 6 (DAC outputs to SMAs via baluns)
Expectations for testing of all stuffed PCBs.
- Atomic documentation of tests and bugs using github Issue tracking system.
- https://github.com/sinara-hw/Sayma_AMC
- https://github.com/sinara-hw/Sayma_RTM
- https://github.com/sinara-hw/Metlino
- https://github.com/sinara-hw/BaseMod
- thermal test in TS7
- Maximum temperature of any IC 65 C
- Monitor temperature of several representative ICs using MMC IPMI
- crate power-on with all boards and with arbitrary RTM unplugged in TS7
- hot swap arbitrary AMC and RTM boards in TS7
- power-on with or without RTM unplugged in TS1
- Signal integrity testing on isolated chips, acceptable bit error rate is
- use any suitable IP (eg Xilinx)
- Ethernet, 1 Gb/s
- JESD, 6 Gb/s per lane
- SDRAM
- I2C, SPI, address all ICs and transmit data at full speed
- MMC configuration including power supply sequencing and IPMI
- Hardware Developer shall test MMC firmware on TS7 system prior to distribution.
- If errors arise after initial distribution Hardware Developer shall debug and test on TS2 system prior to distribution of the source.
- Publish MCH configuration configuration file used for these tests with TS-MTCA and NATIVE-R5.
Distribute hardware to system integrators:
- Software and Gateware Developer: 2 AMC, 2 RTM, 4 AFE, 1 Metlino
- Integration and Timing Developer: 2 AMC, 2 RTM, 4 AFE, 1 Metlino
- The Hardware Developer retains the following for testing and as backups in case of system integrator hardware failure: 2 AMC, 2 RTM, 4 AFE, 1 Metlino
HT3 delivery is complete when the Software and Gateware Developer and Integration and Timing Developer confirm receipt of hardware.
## __HT4__ documentation and MMC
- Support system integrators by github Issue system.
- Develop and document fixes for hardware bugs.
- If bug fixes are substantial (eg replacing multi-pin SMD), system integrators may at their option return boards to The Hardware Developer for repair. System integrator pays shipping inbound, The Hardware Developer pays shipping outbound.
- Each board should have a serial number, and a wiki page where everyone who reworks the board has to document what they did. Other notes about that particular board and its specific bugs can be added there.
- MMC and Exar configuration
- Publish source on github within sinara-hw domain.
- Explicit instructions for building and flashing.
- Mimic ARTIQ release structure: tag official releases, change log.
- Formal hardware documentation maintained in github
- Follow ARTIQ model using Sphinx.
- Setup automatic rendering on readthedocs with webhook to git pushes.
- Embed link in github that links to readthedocs.
- scope: expected content is superset of existing Sayma v1 documentation
- scope: documentation to also include the following github Issues
https://github.com/sinara-hw/sinara/issues?q=is%3Aissue+documentation+label%3Aarea%3Adocumentation+is%3Aopen
HT4 delivery is complete when documentation and source is committed and accepted by TPOC.
## __HT5__ delivery to UMD
Build demonstration system that is fully assembled and tested. The system shall consist of the following components.
- TS-MTCA
- 1 Metlino serving as ARTIQ Master
- 2 Sayma v2 (AMC + RTM)
- 4 BaseMod v2
- Test jigs, clock hardware, documentation and ARTIQ Python code needed to demonstrate the following:
- Passage of SAWG-tests written by TPOC; hosted on hithub https://github.com/jbqubit/sinara-testing/tree/master/sayma/repository/sawg-tests
- ST1, ST2, ST3 and ST4
- Ship demonstration system to UMD.
- UMD/ARL will return TS-MTCA and clock hardware to the Hardware Developer 1 month after delivery. All other parts will remain property of UMD.
HT5 delivery is complete when UMD confirms receipt of the demonstration system /and/ TPOC successfully runs test code. TPOC has 7 business days to conduct tests.
## __HT6__ round up
- Update schematics and layout to remedy bugs.
- Publish to github list of factory acceptance tests including expected results and failure criteria.
# Integration and Timing Developer
An Integration and Timing Developer is responsible for hardware integration and testing. This developer has one deliverable OT1. Bi-weekly written progress reports.
## __OT1__ integration and timing
- __O11__ implement DAC-FPGA synchronization with 2 GHz DAC clock (1 GSPS data rate).
- __O12__ participate in HT1 design review
- __O13__ participate in HT2 design review
- __O14__ Based on O11, demonstrate ST4 for a pair of AD9154 DACs on a single Sayma v2 board
in configuration TS1.
- Write and document code. Generate pull request.
- Re-test the ARTIQ-integrated version of the code and debug any issues that arise.
- Dependency: Software and Gateware Developer is responsible for supplying AD9154 support at the level of M25 and M32. Integration and Timing Developer will handle debugging of ADF4356 and AD9154 as relates to ST4.
- __O17__ in support of a future clock distribution scheme modeled after White Rabbit, specify layout of Si549 and related components
- __O18__ write and test ADF4356 driver
OT1 delivery is complete when the Integration and Timing Developer completes and documents all tests O11 to O18.
# Software and Gateware Developer
The Software and Gateware Developer is responsible for gateware and software needed to provide ARTIQ support and to implement SAWG. All software and gateware development items include:
- Test benches or unit tests for gateware and software
- Examples for runtime and kernel APIs
- Documentation for exposed APIs
- Documentation of design choices
- Documentation of code functionality
The developer’s work consists of three deliverables MT1 to MT3. An asterisk(*) indicates previously funded work. All deliverables include bi-weekly written progress reports.
## __MT1__ Sayma v2 Planning
- __M11__ linear ramp problem on Sayma v1
- __M12__ design study to support 1 GSPS data rate on Sayma v2 (SAWG v1)
- __M13__ design study to investigate SAWG v2 (eg limit f_f0 resolution, reduce RTIO channel number, Moninj)
- __M14__ design study to explore reduced resource consumption options in SAWG v2
- __M15__ participate in HT1 design review
- __M17__ participate in HT2 design review
- __M17__ Replace serwb with pure DRTIO.
- __RTM DRTIO__ -- already funded, can be prototyped on Sayma v1 or Kasli
- __distributed DMA__ -- already funded, can be prototyped on Sayma v1 or Kasli
## __MT2__ Sayma v2 ARTIQ Support
- __M21__ code review and trouble shooting in support of OT1
- __M22__ update clock tree and do code review for O14
- __M23__ implement, test and document ST1 and ST2 which depend on DRTIO; add to continuous integration
- __M24__ test AFE BaseMod v2 (excluding ADC)
- __M25__ responsible for testing SAWG v1 on Sayma v2
- __M26__ support Hardware Developer’s work on HT5
- __M27__ write ARTIQ-based Sayma v2 test bench to validate for continuous integration and document so it can be reproduced by Hardware Developer.
- __M28__ extend RTIO support to FPGA on RTM
## __MT3__ Sayma v2 Extension and Other
- __M31__ support code developed in __O18__
- __M32__ build and test SAWG v2 assuming features are 1 GSPS data rate, AM for noise eating
- __M34__ removed
- __M35__ removed
- __M36__ build infrastructure
- Support build for bitstream building for Sayma v1 and SAWG v1 until 6 months after HT3 delivery.
- Support for bitstream building for SAWG v1 on Sayma v2 until 6 months after SAWG v2 delivery.
- Support for bitstream building of SAWG v2 for Sayma v2 for 3 years after SAWG v2 delivery.
----------
# Project Organization
ARL/UMD will fund and manage contracts with @jbqubit serving as technical point of contact (TPOC). In the event of a dispute about interpretation of a contract or about contract execution, the TPOC will have project-level decision making. Note that the legal language in the officially executed contract documents has final authority.
New github repositories were setup for each component of the Sayma system.
- https://github.com/sinara-hw/Sayma_AMC
- https://github.com/sinara-hw/Sayma_RTM
- https://github.com/sinara-hw/Metlino
- https://github.com/sinara-hw/BaseMod
Github Issue tracking will be the authoritative location for test results and bugs. Keep discussion via other channels (email, IRC) to a minimum but when it happens post a summary to the relevant github Issue. Link back to the old sinara-hw/sinara Issues as needed but don’t conduct new discussions in sinara-hw/sinara.
Weekly discussions via google hangout during period of performance. Attendance of at least one team member for each collaborator is expected. Discussion limited to 30 minutes per week. Software and Gateware Developer, Hardware Developer and Integration and Timing Developer. UMD/ARL involvement if requested by MC. Responsibilities:
- Strict cycling of responsibilities between teams for each week
- Master of ceremonies (MC): publishes meeting agenda at least 4 hours prior to meeting start, enforces time limits, schedules next weekly meeting (advertised in UTC)
- Clerk: takes meeting minutes, posts to github