M-Labs
/
saymapp
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
saymapp/pp.md

23 KiB
Raw Permalink Blame History

This plan defines concrete performance goals, specifies which parties are responsible for specific deliverables and defines a project management structure. The aim is to provide sufficient specification and structure at the outset that all parties feel comfortable with their responsibilities and minimizes conflict.

The plan includes design choices that simplify Sayma and lower costs. The github Sayma Issue tracking contains a long list of bug fixes and modifications to v1 (https://github.com/sinara-hw/sinara). The plan calls out explicit specifications in cases where there is not a clear consensus in the Issues.

Performers

Three developers will work together on Sayma v2.

  • Hardware Developer
  • Integration and Timing Developer
  • Software and Gateware Developer ARL will serve as Technical Point of Contact (TPOC) for the contracts.

Tasks

Developer tasks are labeled HTj, Oj.k and Mj.k (jk integers) and are not divisible. If a task or feature is not assigned to a Developer then that Developer is not responsible for it.

Design Review

The design review consists of three steps: HT1, HT1.5 and HT2. The Hardware Developer has primary responsibility for the design review. Certain review tasks are delegated to other Developers as detailed below. Tracking of design review tasks is in a google docs spreadsheet communicated under separate cover.

Handling Hardware Flaws

The Software and Gateware Developer and Integration and Timing Developer have tasks which depend on proper functionality of the Sayma v2 hardware. The Hardware Developer testing and reporting expectations are detailed in HT3.

The Hardware Developer shall establish a financial reserve sufficient to stuff up to two replacement PCBs in the case of catastrophic damage or shipping loss in the course of collaboration with other Developers.

Flaw found in HT3

If the HT3 report indicates a failing component or subsystem, a Developer working on a dependent task may at its option pursue the following options - cancel dependent task(s) - renegotiate contract terms for dependent task(s) with TPOC

Flaw found after HT3

If a component or subsystem receives passing marks on the HT3 report but a Developer finds a hardware problem after receiving the Sayma v2 hardware the following applies. Said Developer shall - report the problem and work with the Hardware Developer to identify a patch - budget 24 hours for hardware-flaw related debugging - apply white wire patches
- budget for two shipments of PCBs back to Hardware Developer for rework If shipment budget or hardware-flaw budget is exhausted the Developer may - cancel dependent task(s) and receive payment for up to 50% task cost - renegotiate contract terms for dependent task(s) with TPOC

Hardware Developer Dependency Resolution

Hardware Developer task HT5 depends on deliverables from Software and Gateware Developer and Integration and Timing Developer. Two provisions are forseen to account for slow or failed delivery of dependencies by other Developers.

  • If critical dependencies for HT5 are delayed beyond April 20, the scope of work for HT5 can be renegotiated with TPOC.
  • If other Developers provide notice that HT5 dependencies will not be supplied, Hardware Developer may ship demonstration system to UMD without successful completion of a subset of tests ST1, ST2, ST3 and ST4.

Definitions

Let TS-MTCA be a micro TCA crate configured as follows:

  • 1 NAT NATIVE-R5 (or -R9)
  • 1 (or 2) NAT AC 600D
  • 1 NAT MCH-Basic v3.5

Let TS1 be a Sayma test setup consisting of the following

  • 1 Sayma AMC+RTM
  • Powered by ATX connector from benchtop supply

Let TS2 be a Sayma test setup consisting of the following

  • Single uTCA chassis configured as TS-MTCA
  • 2 Sayma AMC+RTM
  • 1 Metlino
  • All RF channels producing 400 MHz sinusoid
  • Cards in adjacent slots

Let TS7 be a Sayma test setup consisting of the following

  • Single uTCA chassis configured as TS-MTCA
  • 6 Sayma AMC+RTM
  • 1 Metlino
  • All RF channels producing 400 MHz sinusoid
  • Cards in adjacent slots

SAWG v1.0 milestone

The original SAWG was developed by M-Labs for Sayma v1.0. Here, "v1" refers to SAWG with the same feature set but targeted at Sayma v2.

  • f_rtio = f_dac_ref = 150 MHz
  • dac data rate 0.6 GSPS
  • f_dac_clk = 2.4 GHz (4X interpolation)
  • synchronization: none
  • functionality in 1st Nyquist zone
    • 240 MHz is max RF out
    • f1, f2: 150 MHz modulation, +/- 60 MHz anti-aliasing filter bandwidth

_SAWG v2.0 milestone

This is an extension of SAWG v1.0 with a 1 GSPS data rate. This milestone also includes board-board synchronization.

  • f_rtio = f_dac_ref = 125 MHz (same as for Kasli)
  • dac data rate 1.0 GSPS
  • f_dac_clk = 2.0 GHz (2X interpolation)
  • synchronization: new Sayma v2 sysref scheme, ST3
  • support 2 RF channels per DAC (!)
  • functionality in 1st Nyquist
    • 400 MHz is max RF out
    • f1, f2: 125 MHz modulation, +/- 50 MHz anti-aliasing filter bandwidth

Sayma v2 Diffs

When diff also applies for Metlino it is noted.

RTM Synthesizer

  • Use HMC830

DDMTD

Implement DDMTD (White Rabbit) clock recovery on both AMC and RTM.

DAC-FPGA Synchronization

Deterministic phase alignment between DACs on separate Sayma PCBs between power cycles is an important design criterion and should be considered at all stages of design. This implies a) SYSREF is aligned with RTIO clock b) DAC clocks are synchronized to local SYSREF edges.

  • Follow approach demonstrated in Issue #583 and summarized in Issue #594.
  • Remove HMC7043, add components needed to generate MGTREFCLK.
  • Test conditions
    • SAWG output on all channels on two Sayma boards, 400 MHz continuous RF
    • Sayma-local TTL output at 62.5 MHz
    • using existing RTIO infrastructure and TTL PHY
    • laboratory temperature stability +/- 0.5 C
    • Cycle crate power 100 times
  • Specification
    • ST1: SAWG-SAWG phase alignment error across all power cycles: < 1/f_dac_clk
    • ST2: SAWG to TTL alignment error across all power cycles: < 0.5 ns

DAC Performance

DAC Phase Stability

Phase stability between DACs on separate Sayma PCBs is an important design criterion and should be considered at all stages of the design. This assumes that DAC-FPGA synchronization.

  • Test conditions
    • SAWG output on all channels on two Sayma boards, 400 MHz continuous RF
    • Assume laboratory temperature stability +/- 0.5 C
    • Duration 24 hours
  • Target performance
    • ST3: SAWG-SAWG phase alignment variation < +/- 0.5 deg
    • ST4: SAWG-TTL alignment variation < +/-250 ps

DAC Noise

Low cross talk between adjacent analog channels on same AFE is an important design criterion and should be considered at all stages of the design (particularly layout).

  • cross talk goal: at 400 MHz < -80 dBc.

Phase noise at 400 MHz (3 dB margin on AD9154 specification):

  • 10 Hz offset, < -94 dBc
  • 100 Hz offset, < -103 dBc
  • 1000 Hz offset, < -120 dBc
  • 10,000 Hz offset, < -126 dBc

spurs: < -60 dBc SFDR across operating range, max amplitude

Reduce complexity of Sayma v2

  • Remove fast JESD ADCs from RTM layout
  • Reduce complexity of clocking options
    • Remove clock mezzanine
    • User must supply RTM with DAC reference clock (f_dac_ref) via back-panel SMA
    • f_rtio is obtained from front-panel SMA (if master) or fiber+CDR (if satellite)
    • User must ensure that the following are phase locked: f_dac_ref and f_rtio
    • Synth on RTM generates f_dac from f_dac_ref
    • No support for analog backplane (LLRFBP, https://github.com/sinara-hw/sinara/issues/498).
  • Unambiguous mapping between control pin and IC naming semantics in schematics and corresponding ARTIQ source.
  • Remove 24-bit multi-channel ADC from Sayma RTM layout.

AFE Interface to RTM

BaseMod AFE v2

  • New form factor to accommodate switch to FMC and 4 DAC channels per AFE.
  • FMC-type front panels, with connector holes determined by choice of AFE.
  • AFE name and version number on FMC front panel.
  • No RF power detector.
  • Add LVDS ADC. Copy design from Sampler. No development support for ADC in this contract.
  • Hardware Developer shall specify PCB material assuming maximum RF frequency is 1 GHz.

Smart Agile Waveform Generator

Use SAWG v1 with minimal modification to demonstrate Sayma v2 hardware. SAWG v2 will be pursued once Sayma v2 hardware is working.

Ethernet

  • Replace MAX24287 by M88E1512. This is same family IC as KC705 which is already supported by ARTIQ.
  • Configuration of M88E1512 is by static pin strapping: 1 Gbps, GMII, 1000BASE-X.
  • RGMII is connected only with FPGA -- no interface to MMC.
  • Layout so that uTCA Port0 and SFP can be driven by M88E1512 or by GTH transceiver on FPGA. Use coupling capacitors to choose which.
  • Also applies to Metlino.

Hardware Developer

The Hardware Developer is responsible for hardware design and manufacturing. Testing responsibilities of the Hardware Developer are those detailed in HT3.

The Hardware Developer has deliverables designated HTn. Written progress reports by the 1st of the month.

HT1 high level design

High-level schematics for all PCBs to include component part numbers, high-level component interconnection, power budget, high-level layout of all PCBs including location of ICs, board-to-board headers, RF shielding, heat sinks, mechanical support and front panel design. Post design files to github and tag. Participate in design review on github.

HT1 delivery is complete when the Software and Gateware Developer, Integration and Timing Developer and UMD/ARL sign off on the drawings.

HT1.5 high level design review of interconnections

HT1.5 is a review of connections between components. It consists of the following checks.

  • Are design change clearly articulated? (eg IC choice)? Confirm that schematic satisfies Sayma v2 Project Plan section Sayma v2 Diffs.

    • Clocks
    • GTx, Ethernet
    • RTM to AMC connectivity requirements
    • qty of LVDS links to AFE for future physicist needs
    • Are resources sufficient for WR implementation?
    • Front/rear panel connectivity
    • power consumption

  • Check lead time for key ICs.

  • Are any BOM items near end of life?

  • Check of newly created symbols in layout software. Reviewer checks all pins against schematic. https://github.com/sinara-hw/sinara/issues/605

HT2 detailed design

Detailed schematics for all PCBs, finished component interconnections, signal integrity analysis, thermal analysis, power analysis, final PCB routing. Post design files to github and tag. Participate in final design review, accept delegated responsibilities from Software and Gateware Developer Design. Output is layout ready for manufacture.

Hardware Developer shall lead HT2 design review. This design review is in collaboration with scientists and other developers. The review will not be comprehensive. It will focus on differences between v1 and v2 of the hardware. Shall include validation of FPGA pinout.

Interaction with Software and Gateware Developer:

  • Hardware Developer delivers FPGA netlist(s) Software and Gateware Developer.

HT2 delivery is complete when Software and Gateware Developer and Integration and Timing Developer sign off on the drawings.

HT3 manufacturing, baseline testing

Fabricate PCBs in the following quantities. TPOC provides sign off prior to commencement of manufacturing.

  • Metlino AMC: print 10 PCBs, stuff 4
  • Sayma AMC v2: print 10 PCBs, stuff 6
  • Sayma RTM v2: print 10 PCBs, stuff 6
  • BaseMod v2: print 20 PCBs, stuff 12
  • TestMod v2: print 6 PCBs, stuff 6 (DAC outputs to SMAs via baluns)

UMD shall loan the following parts to Hardware Developer at the start of HT3. They will be returned to UMD at the close of HT5.

  • 1 NAT NATIVE-R5
  • 1 NAT AC 600D
  • 1 NAT MCH-Basic v3.5

Expectations for testing of all stuffed PCBs.

  • Atomic documentation of tests and bugs using github Issue tracking system.
  • thermal test in TS7
    • goal: maximum temperature of measured ICs < 65 C
    • Monitor temperature of a subset of ICs using MMC IPMI.
  • crate power-on with all boards and with arbitrary RTM unplugged in TS7
  • hot swap arbitrary AMC and RTM boards in TS7
  • power-on with or without RTM unplugged in TS1
  • Signal integrity testing using any suitable IP (eg Xilinx) for all ICs. Tested individually.
  • Design and implement test to simulate heavy loading of a system configured as TS2. The test is expected to simultaneously exercise the following subsystems
    • I2C, SPI, address all ICs round robin at full speed
    • 2 TestMod installed dissipating representative power
    • AD built-in DAC JESD PRBS test at 10 Gb/s lane for both DACs
    • SDRAM PRBS write-then-read at [TODO ____ data rate]
    • AMC backplane ethernet PRBS at 1 GSPS
    • SFP loop-back PRBS at 6 Gb/s
    • AMC backplane PRBS at 6 Gb/s
    • FMC loop-back
    • Realistic FPGA fabric load and clock activity
  • MMC configuration including power supply sequencing and IPMI
    • Hardware Developer shall test MMC firmware on TS7 system prior to distribution.
    • If errors arise after initial distribution Hardware Developer shall debug and test on TS2.
  • Publish MCH configuration configuration file used for these tests with TS-MTCA and NATIVE-R5.

Distribute hardware to system integrators:

  • Software and Gateware Developer: 2 AMC, 2 RTM, 4 AFE, 1 Metlino
  • Integration and Timing Developer: 2 AMC, 2 RTM, 4 AFE, 1 Metlino
  • The Hardware Developer retains the following for testing and as backups in case of system integrator hardware failure: 2 AMC, 2 RTM, 4 AFE, 1 Metlino

HT3 delivery is complete when the Software and Gateware Developer and Integration and Timing Developer confirm receipt of hardware.

HT4 documentation and MMC

  • Support system integrators by github Issue system.
  • Develop and document fixes for hardware bugs.
    • If bug fixes are substantial (eg replacing multi-pin SMD), system integrators may at their option return boards to the Hardware Developer for repair. System integrator pays shipping inbound, Hardware Developer pays shipping outbound. Limit to 10 shipping round trips.
    • Each board should have a serial number, and a wiki page where everyone who reworks the board has to document what they did. Other notes about that particular board and its specific bugs can be added there.
  • MMC and Exar configuration
    • Publish source on github within sinara-hw domain.
    • Explicit instructions for building and flashing.
    • Mimic ARTIQ release structure: tag official releases, change log.
  • Formal hardware documentation maintained in github

HT4 delivery is complete when documentation and source is committed and accepted by TPOC.

HT5 delivery to UMD

Build demonstration system that is fully assembled and tested. Collect code and test results from other Developers for ST1 to ST4. Reproduce test results on Hardware Developer's system. Add test results to Sayma v2 documentation. The system shall consist of the following components.

  • TS-MTCA
  • 1 Metlino serving as ARTIQ Master
  • 2 Sayma v2 (AMC + RTM)
  • 4 BaseMod v2
  • Test jigs, clock hardware, documentation and ARTIQ Python code needed to demonstrate the following:
    • ST1, ST2, ST3 and ST4
    • Use SAWG >= 2.0
  • Ship demonstration system to UMD.
  • UMD/ARL will return clock hardware to the Hardware Developer 1 month after delivery. All other parts will remain property of UMD.

HT5 delivery is complete when UMD confirms receipt of the demonstration system /and/ TPOC successfully runs test code. TPOC has 7 business days to conduct tests.

HT6 round up

  • Update schematics and layout to remedy bugs.
  • Publish to github list of factory acceptance tests including expected results and failure criteria.

Integration and Timing Developer

An Integration and Timing Developer is responsible for certain aspects of hardware integration and testing as detailed in this section.

This developer has one deliverable OT1. Written progress reports by the 1st of the month.

OT1 integration and timing

Dependency: Software and Gateware Developer is responsible for supplying AD9154 support at the level of M3.04 and M3.06. Integration and Timing Developer will handle debugging of ADF4356 and AD9154 as relates to ST4.

Code submitted for inclusion in ARTIQ shall comply with CONTRIBUTING.rst (https://github.com/m-labs/artiq/blob/master/CONTRIBUTING.rst).

  • O1.01 implement DAC-FPGA synchronization with 2 GHz DAC clock (1 GSPS data rate) with ST3 in mind

  • O1.02 participate in HT1 and HT1.5 design reviews

  • O1.03 participate in HT2 design review

  • _O1.04 based on O1.01, ensure passage of ST3 for a pair of AD9154 DACs on a single Sayma v2 board in configuration TS1. Note that this test is short of the full ST3 test which specifies a pair of Sayma v2 boards.

    • Write and document code. Generate pull request.

  • O1.05 ADF4356

    • Measure phase determinism of ADF4356.
    • Write and test ADF4356 driver.
    • Write and document code. Generate pull request.

  • O1.06 in support of clock distribution modeled after White Rabbit (WR, DDMTD), specify layout of Si549 and related components.

OT1 delivery is complete when the Integration and Timing Developer completes and documents the above tasks and returns 1 Metlino, 1 Sayma AMC, 1 Sayma RTM, 2 Basemods to the Hardware Developer.

Software and Gateware Developer

The Software and Gateware Developer is responsible for gateware and software needed to provide ARTIQ support and to implement SAWG. All software and gateware development items include:

  • Test benches or unit tests for gateware and software
  • Examples for runtime and kernel APIs
  • Documentation for exposed APIs
  • Documentation of design choices
  • Documentation of code functionality

The developer’s work consists of deliverables MTk below. Written progress reports are due by the 1st of the month.

MT1 SAWG

MT1 is a port of SAWG v1.0 designed for Sayma v1 to Sayma v2.

  • M1.01 Support SAWG v2.0
    • Develop code for Sayma v2 SAWG+JESD+fpga clock tree at 1 GSPS.
    • Implement new Sayma v2 sysref synchronization scheme.

MT2 Sayma v2 Planning

  • M2.01 Develop fixed test pattern generator (12-point cosine on odd channels and ramps with major carry toggles on even channels) as a compile time alternative to SAWG.

  • M2.02 Review and verify Sayma v2 design from the FPGA perspective. Do this by building a stub ARTIQ target and test compilation -- depends on obtaining netlist for FPGAs from Hardware Developer (HT2). This will verify IO assignments and usage patterns around the following subsystems to the level currently used in ARTIQ

    • ethernet
    • transceivers
    • clocking
    • SDRAM
    • flash
    • RTM interface
    • AFE ports
    • FMC ports

  • M2.03 Review of the scheme for synchronization between pairs of Sayma boards and complete clocking scheme.

MT3 Sayma v2 ARTIQ Support

  • M3.01 Review the code emerging from O14 and O15. Support merge into ARTIQ. Aspects included in review: DRTIO-on-RTM, DRTIO clock recovery on RTM, JESD204B deterministic latency/synchronization and DDMTD.

  • M3.02 Work with TPOC to develop test cases for SAWG v2.0 with ST1 and ST2 in mind. Split into manual test cases and tests amendable to continuous integration (CI). Procure hardware needed for hardware-in-the-loop tests.

  • M3.03 Support for Sayma_AMC to include the following

    • power up
    • boot tasks including JTAG, loading flash & FPGA
    • UARTs
    • SDRAM
    • ethernet
    • I2C EEPROM (MAC address and board serial numbers) (through USB-FTDI?)
    • clocking (including DRTIO clock recovery)
    • RTIO transcievers
    • RTIO SERDES TTL
    • FMC connectors
    • build infrastructure and packaging

  • M3.04 Support for Sayma_RTM to include the following

    • power up
    • boot tasks including slave serial loading of FPGA
    • RTM clocking
    • develop DRTIO for AMC FPGA to RTM FPGA link
    • SPI to HMC830
    • AFE interfaces
    • build infrastructure and packaging
    • dependency: M3.03

  • M3.05 Support for BaseMod to include the following

    • attenuator shift register
    • DRTIO control of RF switches using TTL PHY
    • dependency: M3.04

  • M3.06 Validate RF output using SAWG v1.0. This tests operation of SAWG as developed for Sayma v1 works on Sayma v2. It tests the following elements.

    • clocking of AD9154
    • SYSREF syncing of DAC and JESD core
    • JESD204B to AD9154
    • SAWG v1.0 integration
    • dependency: M3.04

  • M3.07 SAWG v2.0

    • build and document tests ST1, ST2, ST3 and ST4, ensure that they pass on SAWG v2.0
    • dependencies: M3.06, O1.04

  • M3.11 Support Hardware Developer on HT5

  • M3.13 Port of existing Metlino port

    • A Migen port for Metlino based on Sayma AMC v1 exists.
    • Adapt the existing Migen port to Metlino based on Sayma AMC v2.

Project Organization

ARL/UMD will fund and manage contracts with @jbqubit serving as technical point of contact (TPOC).

New github repositories were setup for each component of the Sayma system.

Use github Issue tracking for test results and bug tracking. Link back to the old sinara-hw/sinara Issues as needed but don’t conduct new discussions in sinara-hw/sinara.

Weekly discussions via google hangout during period of performance. Attendance of at least one team member for each Developer team is expected. Discussion limited to 30 minutes per week. UMD/ARL involvement if requested by MC.

Master of Ceremonies (MC) expectations:

  • Solicit agenda items from group. Select agenda topics that can be completed in 30 minutes.
  • Regulate discussion so that planned agenda items are covered. For technical matters that prove contentious or detailed, shift discussion to github Issue.
  • Publish agenda > 4 hours prior to meeting start.
  • Launch google hangout > 5 minutes in advance of meeting start.
  • Schedule next meeting time using doodle; advertise meeting time in UTC
  • Each week rotate responsibilities for MC and minutes-taking between Developers and UMD/ARL Agenda items must include:
    • vote on acceptance of last meeting's minutes
    • designate next meeting's MC
    • designate next meeting minutes-taker
    • ask participants to reflect on how the meeting went and what to do better next week