RTIO/SYS Clock merge #212

mwojcik · 2023-01-11T14:59:10+08:00

mwojcik commented

2023-01-11 14:59:10 +08:00

Marked as WIP to keep track of the progress, but don't merge yet.

At this point there's no bootstrap clock provided - that required minor reordering (e.g. set up clock before init_gateware).

For now tested with Kasli-SoC on Standalone (seems OK).

Marked as WIP to keep track of the progress, but don't merge yet. At this point there's no bootstrap clock provided - that required minor reordering (e.g. set up clock before ``init_gateware``). For now tested with Kasli-SoC on Standalone (seems OK).

mwojcik commented

2023-01-16 18:25:21 +08:00

Turns out I can't really go without a bootstrap clock - standalone may work (although zc706 doesn't want to cooperate fully yet - maybe same power cycle issue?), but for DRTIO the transceivers need to know when the clock is stable, to make sure the transceivers don't glitch up.

Besides that, bootstrap clock is used for GT[XP] tx_init.

However at this point I'm getting:

INFO: [Route 35-580] Design has 833 pins with tight setup and hold constraints.

the next part takes forever with some truly abstract unmet timing:

------------------------------------------------------------------------------------------------
| Design Timing Summary
| --------------------- ------------------------------------------------------------------------------------------------
    WNS(ns)      TNS(ns)  TNS Failing Endpoints  TNS Total Endpoints      WHS(ns)      THS(ns)  THS Failing Endpoints  THS Total Endpoints     WPWS(ns)     TPWS(ns)  TPWS Failing Endpoints  TPWS Total Endpoints
    -------      -------  ---------------------  -------------------      -------      -------  ---------------------  -------------------     --------     --------  ----------------------  --------------------
     -3.413    -3303.899                   2354                43182        0.003        0.000                      0                43180        0.650        0.000                       0                 20205
Timing constraints are not met.

That was solved with additional timing constraints in Kasli/KC705, maybe I'm missing something, open to suggestions.

Turns out I can't really go without a bootstrap clock - standalone *may* work (although zc706 doesn't want to cooperate fully yet - maybe same power cycle issue?), but for DRTIO the transceivers need to know when the clock is stable, to make sure the transceivers don't glitch up. Besides that, bootstrap clock is used for GT\[XP] ``tx_init``. However at this point I'm getting: ``` INFO: [Route 35-580] Design has 833 pins with tight setup and hold constraints. ``` the next part takes forever with some truly abstract unmet timing: ``` ------------------------------------------------------------------------------------------------ | Design Timing Summary | --------------------- ------------------------------------------------------------------------------------------------ WNS(ns) TNS(ns) TNS Failing Endpoints TNS Total Endpoints WHS(ns) THS(ns) THS Failing Endpoints THS Total Endpoints WPWS(ns) TPWS(ns) TPWS Failing Endpoints TPWS Total Endpoints ------- ------- --------------------- ------------------- ------- ------- --------------------- ------------------- -------- -------- ---------------------- -------------------- -3.413 -3303.899 2354 43182 0.003 0.000 0 43180 0.650 0.000 0 20205 Timing constraints are not met. ``` That was solved with additional timing constraints in Kasli/KC705, maybe I'm missing something, open to suggestions.

mwojcik commented

2023-01-20 17:31:44 +08:00

As of now the status is:

Kasli-SoC seems to be happy on Standalone
zc706 will not lock PLL (I suspect same thing as with kc705, need powercycle)
DRTIO on Zynq is done almost the same way as on Kasli and it looks fine code-wise, but PLL won't lock - and GTP clock is not visible on timing reports in clock summary... but I can trace it all within top.v. I suspect some Vivado shenanigans, again.

As of now the status is: - Kasli-SoC seems to be happy on Standalone - zc706 will not lock PLL (I suspect same thing as with kc705, need powercycle) - DRTIO on Zynq is done almost the same way as on Kasli and it looks fine code-wise, but PLL won't lock - and GTP clock is not visible on timing reports in clock summary... but I can trace it all within ``top.v``. I suspect some Vivado shenanigans, again.

mwojcik force-pushed rtiosys_clk_merge from af657dec74 to 91d7248df4

2023-01-20 17:33:55 +08:00

Compare

mwojcik commented

2023-02-09 17:20:04 +08:00

At this point:

there's a bootstrap clock just like in mainline ARTIQ
That is for reasons DRTIO related. One is making sure GTX does not get glitchy clock output from Si5324 (stable_clkin) - that has to be somehow passed to CSR, and that requires working clock; second is tx_init done in bootstrap domain, to survive sys change.

PLL locks now fine if we base it on tx_init.done, rather do it from the PS - just like on mainline.

standalone Kasli-SoC and zc706 seem to work fine (incl. RTIO, running kernels)
master Kasli-SoC and zc706 seem to boot normally and switches clocks - local RTIO works fine, but there's no DRTIO (link not detected)
satellite also boots and seems to switch clock, but haven't tested it further yet.

At this point: - there's a bootstrap clock just like in mainline ARTIQ That is for reasons DRTIO related. One is making sure GTX does not get glitchy clock output from Si5324 (``stable_clkin``) - that has to be somehow passed to CSR, and that requires working clock; second is tx_init done in bootstrap domain, to survive sys change. PLL locks now fine if we base it on tx_init.done, rather do it from the PS - just like on mainline. - standalone Kasli-SoC and zc706 seem to work fine (incl. RTIO, running kernels) - master Kasli-SoC and zc706 seem to boot normally and switches clocks - local RTIO works fine, but there's no DRTIO (link not detected) - satellite also boots and seems to switch clock, but haven't tested it further yet.

mwojcik commented

2023-02-14 16:09:11 +08:00

Now it switches clocks - as it actually didn't before, that's why clk switch was exported. For whatever reason, stable_clkin needs a bit of a delay, and FCLK needed to be configured. Before that, PLL VCO frequency would be below specifications.

Master can see an uplink but the ping will fail (and no activity is detected on the tested satellite). With copper cable.

Due to FCLK's configurable/resettable nature I'm thinking that it's not a good source for bootstrap clock. ZC706 has clk200 that could be used, Kasli-SoC has clk_125gtp instead. I will start with that to see if that helps DRTIO - given GTX's delicate nature, where looking at it wrong can make it fail.

Now it switches clocks - as it actually didn't before, that's why clk switch was exported. For whatever reason, stable_clkin needs a bit of a delay, and FCLK needed to be configured. Before that, PLL VCO frequency would be below specifications. Master can see an uplink but the ping will fail (and no activity is detected on the tested satellite). With copper cable. Due to FCLK's configurable/resettable nature I'm thinking that it's not a good source for bootstrap clock. ZC706 has ``clk200`` that could be used, Kasli-SoC has ``clk_125gtp`` instead. I will start with that to see if that helps DRTIO - given GTX's delicate nature, where looking at it wrong can make it fail.

mwojcik force-pushed rtiosys_clk_merge from dc19b36064 to cff2caa88f

2023-02-16 15:56:08 +08:00

Compare

mwojcik changed title from ~~[WIP] RTIO/SYS Clock merge~~ to RTIO/SYS Clock merge

2023-02-17 13:03:03 +08:00

mwojcik commented

2023-02-17 13:13:35 +08:00

Tested on zc706 - sometimes clock won't switch but when it does everything works (DRTIO master/satellite). Kasli-SoC... seems like the board I've been using has issues that I lost some time on (need more investigation!). With another board everything was good, every time. Master connected to Kasli 1.1 satellite and exchanged data (including some TTL tests); SoC satellite connected to 2.0 master received DRTIO data too.

In hindsight it seems that using external clock source rather than FCLK was not necessary, but I think it's a more elegant method, even if it's more board specific.

Either way ready for full review and going forward.

Tested on zc706 - sometimes clock won't switch but when it does everything works (DRTIO master/satellite). Kasli-SoC... seems like the board I've been using has issues that I lost some time on (need more investigation!). With another board everything was good, every time. Master connected to Kasli 1.1 satellite and exchanged data (including some TTL tests); SoC satellite connected to 2.0 master received DRTIO data too. In hindsight it seems that using external clock source rather than FCLK was not necessary, but I think it's a more elegant method, even if it's more board specific. Either way ready for full review and going forward.

mwojcik force-pushed rtiosys_clk_merge from 4f410188e5 to 5f10387684

2023-02-17 15:30:01 +08:00

Compare

mwojcik added 2 commits 2023-02-17 15:48:44 +08:00

libboard_artiq: fix warnings 02903503c6

kasli_soc: minor cleanup 65678fb4c2

mwojcik force-pushed rtiosys_clk_merge from 4d6edef142 to f7f956fc34

2023-02-17 15:49:34 +08:00

Compare

mwojcik added 1 commit 2023-02-17 15:51:46 +08:00

remove out-of-scope error LED support for satellite f3ae8660d0

sb10q merged commit 46b2687d70 into master