M-Labs/artiq-zynq
10
1
Fork 20
Code Issues 13 Pull Requests Releases Wiki Activity

corruption of kernel RPC values #8

New Issue
Closed
opened 2020-06-09 12:53:57 +08:00 by sb10q · 10 comments
sb10q commented 2020-06-09 12:53:57 +08:00
Owner
Copy Link

This incorrectly prints 0.0:

from artiq.experiment import *
from artiq.language.types import *

class Corruption(EnvExperiment):
    def build(self):
        self.setattr_device("core")
        self.data = b"\x00"*(10**5)

    @rpc(flags={"async"})
    def sink(self, data):
        assert data == self.data

    @kernel
    def test(self):
        t0 = self.core.get_rtio_counter_mu()
        self.sink(self.data)
        t1 = self.core.get_rtio_counter_mu()
        return self.core.mu_to_seconds(343705672)

    def run(self):
        print(self.test())

Making changes to the code e.g. removing the t0 or t1 statement or reducing the size of the data buffer makes the issue disappear.

Playing with stack/heap sizes does not help.

This incorrectly prints ``0.0``: ```python from artiq.experiment import * from artiq.language.types import * class Corruption(EnvExperiment): def build(self): self.setattr_device("core") self.data = b"\x00"*(10**5) @rpc(flags={"async"}) def sink(self, data): assert data == self.data @kernel def test(self): t0 = self.core.get_rtio_counter_mu() self.sink(self.data) t1 = self.core.get_rtio_counter_mu() return self.core.mu_to_seconds(343705672) def run(self): print(self.test()) ``` Making changes to the code e.g. removing the t0 or t1 statement or reducing the size of the data buffer makes the issue disappear. Playing with stack/heap sizes does not help.
sb10q commented 2020-06-09 13:00:15 +08:00
Author
Owner
Copy Link

This seems to be the RPC buffer being written where it should not, changing the contents to 0xff instead of 0x00 makes the program print nan.

This seems to be the RPC buffer being written where it should not, changing the contents to ``0xff`` instead of ``0x00`` makes the program print ``nan``.
astro commented 2020-06-30 06:44:26 +08:00
Contributor
Copy Link

My first approach is tightening permissions in the page tables (branch mem_protect) so that some bogus memory access immediately causes a CPU exception at the instruction that caused it.

My first approach is tightening permissions in the page tables (branch `mem_protect`) so that some bogus memory access immediately causes a CPU exception at the instruction that caused it.
sb10q commented 2020-06-30 11:26:20 +08:00
Author
Owner
Copy Link

It seems that both the intended destination and the victim of corruption are in the same readable+writable section, so that might not turn up anything - am I wrong?

It seems that both the intended destination and the victim of corruption are in the same readable+writable section, so that might not turn up anything - am I wrong?
sb10q changed title from corruption of kernel values to corruption of kernel RPC values 2020-07-05 23:57:39 +08:00
pca006132 commented 2020-07-06 15:09:33 +08:00
Contributor
Copy Link

In the latest master, this would not print an incorrect value, instead it will exhibit some other weird behaviors...

  1. Wrong address passed to rpc_send_common: 
    [     6.249298s]  INFO(runtime::kernel): kernel starting
[     6.254251s] DEBUG(runtime::kernel): *Data = 0x10157e10
[     6.261017s] DEBUG(runtime::kernel): *Data = 0x10157df8
[     6.270079s] DEBUG(runtime::kernel): *Data = 0xea03fffe
panic at runtime/src/rpc.rs:212:13: tag without a return separator
...
  2. Data Abort.
  3. Core1 Restarted: 
    [     5.569364s]  INFO(runtime::comms): received connection
[     5.613024s] DEBUG(runtime::kernel): Core1 started
[     5.617804s] DEBUG(runtime::kernel): FPU enabled on Core1
[     5.625722s] DEBUG(dyld): ELF target: 168020 bytes, align to 10000, allocated at 00180000
[     5.635669s] DEBUG(dyld): Relocating 0 rela, 96 rel, 40 pltrel
[     5.641514s] DEBUG(runtime::kernel): Library image: 0x180000
[     5.647160s] DEBUG(runtime::kernel): Library image length: 0x29054
[     5.653251s] DEBUG(runtime::kernel): kernel loaded
[     5.659316s]  INFO(runtime::kernel): kernel starting
[     5.664271s] DEBUG(runtime::kernel): Data addr: 0x10157e10
[     5.671143s] DEBUG(runtime::kernel): Data addr: 0x10157df8
[     5.680319s] DEBUG(runtime::kernel): Core1 started
[     5.685091s] DEBUG(runtime::kernel): FPU enabled on Core1
    And then the next kernel load would cause a BorrowMutError as we did not correctly terminated the kernel.
In the latest master, this would not print an incorrect value, instead it will exhibit some other weird behaviors... 1. Wrong address passed to `rpc_send_common`: ``` [ 6.249298s] INFO(runtime::kernel): kernel starting [ 6.254251s] DEBUG(runtime::kernel): *Data = 0x10157e10 [ 6.261017s] DEBUG(runtime::kernel): *Data = 0x10157df8 [ 6.270079s] DEBUG(runtime::kernel): *Data = 0xea03fffe panic at runtime/src/rpc.rs:212:13: tag without a return separator ... ``` 2. Data Abort. 3. Core1 Restarted: ``` [ 5.569364s] INFO(runtime::comms): received connection [ 5.613024s] DEBUG(runtime::kernel): Core1 started [ 5.617804s] DEBUG(runtime::kernel): FPU enabled on Core1 [ 5.625722s] DEBUG(dyld): ELF target: 168020 bytes, align to 10000, allocated at 00180000 [ 5.635669s] DEBUG(dyld): Relocating 0 rela, 96 rel, 40 pltrel [ 5.641514s] DEBUG(runtime::kernel): Library image: 0x180000 [ 5.647160s] DEBUG(runtime::kernel): Library image length: 0x29054 [ 5.653251s] DEBUG(runtime::kernel): kernel loaded [ 5.659316s] INFO(runtime::kernel): kernel starting [ 5.664271s] DEBUG(runtime::kernel): Data addr: 0x10157e10 [ 5.671143s] DEBUG(runtime::kernel): Data addr: 0x10157df8 [ 5.680319s] DEBUG(runtime::kernel): Core1 started [ 5.685091s] DEBUG(runtime::kernel): FPU enabled on Core1 ``` And then the next kernel load would cause a `BorrowMutError` as we did not correctly terminated the kernel.
pca006132 commented 2020-07-06 15:12:40 +08:00
Contributor
Copy Link

More information regarding the data abort:

[    42.912680s]  INFO(runtime::comms): received connection
[    42.954501s] DEBUG(runtime::kernel): Core1 started
[    42.959199s] DEBUG(runtime::kernel): FPU enabled on Core1
[    42.967196s] DEBUG(dyld): ELF target: 168020 bytes, align to 10000, allocated at 00180000
[    42.977137s] DEBUG(dyld): Relocating 0 rela, 96 rel, 40 pltrel
[    42.982988s] DEBUG(runtime::kernel): Library image: 0x180000
[    42.988633s] DEBUG(runtime::kernel): Library image length: 0x29054
[    42.994803s] DEBUG(runtime::kernel): kernel loaded
[    43.207994s]  INFO(runtime::kernel): kernel starting
[    43.212945s] DEBUG(runtime::kernel): Data addr: 0x10157e10
[    43.219763s] DEBUG(runtime::kernel): Data addr: 0x10157df8
[    43.228943s] DEBUG(runtime::kernel): Core1 started
[    43.233722s] DEBUG(runtime::kernel): FPU enabled on Core1
DataAbort on core 0
DFSR: 001
More information regarding the data abort: ``` [ 42.912680s] INFO(runtime::comms): received connection [ 42.954501s] DEBUG(runtime::kernel): Core1 started [ 42.959199s] DEBUG(runtime::kernel): FPU enabled on Core1 [ 42.967196s] DEBUG(dyld): ELF target: 168020 bytes, align to 10000, allocated at 00180000 [ 42.977137s] DEBUG(dyld): Relocating 0 rela, 96 rel, 40 pltrel [ 42.982988s] DEBUG(runtime::kernel): Library image: 0x180000 [ 42.988633s] DEBUG(runtime::kernel): Library image length: 0x29054 [ 42.994803s] DEBUG(runtime::kernel): kernel loaded [ 43.207994s] INFO(runtime::kernel): kernel starting [ 43.212945s] DEBUG(runtime::kernel): Data addr: 0x10157e10 [ 43.219763s] DEBUG(runtime::kernel): Data addr: 0x10157df8 [ 43.228943s] DEBUG(runtime::kernel): Core1 started [ 43.233722s] DEBUG(runtime::kernel): FPU enabled on Core1 DataAbort on core 0 DFSR: 001 ```
astro commented 2020-07-07 07:11:51 +08:00
Contributor
Copy Link

Next approach: dumping/diffing the kernel image before start and after crash.

Next approach: dumping/diffing the kernel image before start and after crash.
pca006132 commented 2020-07-07 15:11:07 +08:00
Contributor
Copy Link

After dumping the image into SD card, it seems that the corruption is caused by both rpc::send_args and core1_tx.send in rpc_send_common.

It seems that the RPC buffer is written into the image.

Before send_args and after send_args:

--- BEFORE.txt	2020-07-07 15:08:58.212253487 +0800
+++ AFTER1.txt	2020-07-07 15:08:58.193253104 +0800
@@ -1,76 +1,76 @@
-0000000 457f 464c 0101 0001 0000 0000 0000 0000
-0000010 0003 0028 0001 0000 03b0 0000 0034 0000
-0000020 94f0 0001 0400 0500 0034 0020 0006 0028
-0000030 0015 0014 0001 7000 8b5c 0001 8b5c 0001
-0000040 8b5c 0001 0010 0000 0010 0000 0004 0000
-0000050 0004 0000 0001 0000 0000 0000 0000 0000
-0000060 0000 0000 8b6c 0001 8b6c 0001 0005 0000
-0000070 0000 0001 0001 0000 8f40 0001 8f40 0002
-0000080 8f40 0002 0114 0000 0114 0000 0006 0000
-0000090 0000 0001 0002 0000 8f68 0001 8f68 0002
-00000a0 8f68 0002 0098 0000 0098 0000 0006 0000
-00000b0 0004 0000 e551 6474 0000 0000 0000 0000
-00000c0 0000 0000 0000 0000 0000 0000 0006 0000
-00000d0 0010 0000 e552 6474 8f40 0001 8f40 0002
-00000e0 8f40 0002 00c0 0000 00c0 0000 0004 0000
-00000f0 0001 0000 0003 0000 000f 0000 0005 0000
-0000100 0009 0000 0004 0000 0000 0000 0000 0000
-0000110 0000 0000 0006 0000 000e 0000 0008 0000
-0000120 0000 0000 000d 0000 0007 0000 000b 0000
-0000130 0000 0000 000a 0000 0003 0000 0000 0000
-0000140 000c 0000 0003 0000 0006 0000 0002 0000
-0000150 0006 0000 0089 31a8 c400 8862 0006 0000
-0000160 000a 0000 000d 0000 af00 e834 5dd2 f86f
-0000170 4542 ecd5 e3bb 7c92 cbf4 00f3 dabe 1b57
-0000180 71d9 1c58 5476 943c ef59 7997 0000 0000
-0000190 0000 0000 0000 0000 0000 0000 0000 0000
-00001a0 03b0 0000 0000 0000 0003 0008 0000 0000
-00001b0 9020 0002 0000 0000 0003 000f 0030 0000
-00001c0 0000 0000 0000 0000 0010 0000 0001 0000
-00001d0 0000 0000 0000 0000 0010 0000 0021 0000
-00001e0 0000 0000 0000 0000 0010 0000 006c 0000
-00001f0 9054 0002 0000 0000 0010 000f 0041 0000
-0000200 904c 0002 0008 0000 0011 000f 004a 0000
-0000210 9054 0002 0000 0000 0010 000f 007f 0000
-0000220 9054 0002 0000 0000 0010 000f 0015 0000
-0000230 03b0 0000 00c4 0000 0012 0008 006b 0000
-0000240 9054 0002 0000 0000 0010 000f 0051 0000
-0000250 9054 0002 0000 0000 0010 000f 005d 0000
-0000260 9054 0002 0000 0000 0010 000f 0077 0000
-0000270 9054 0002 0000 0000 0010 000f 5f00 615f
-0000280 7472 7169 705f 7265 6f73 616e 696c 7974
-0000290 5f00 6d5f 646f 6e69 7469 5f5f 7200 6370
-00002a0 735f 6e65 5f64 7361 6e79 0063 7472 6f69
-00002b0 675f 7465 635f 756f 746e 7265 7400 7079
-00002c0 6965 666e 006f 655f 6164 6174 5f00 625f
-00002d0 7373 735f 6174 7472 5f00 625f 7373 735f
-00002e0 6174 7472 5f5f 5f00 625f 7373 655f 646e
-00002f0 5f5f 5f00 655f 646e 5f5f 5f00 6e65 0064
-0000300 8f44 0002 0017 0000 8f4c 0002 0017 0000
-0000310 8f54 0002 0017 0000 8f58 0002 0017 0000
-0000320 8f60 0002 0017 0000 8f64 0002 0017 0000
-0000330 9024 0002 0017 0000 9028 0002 0017 0000
-0000340 9034 0002 0017 0000 903c 0002 0017 0000
-0000350 9044 0002 0017 0000 904c 0002 0017 0000
-0000360 900c 0002 0316 0000 9010 0002 0416 0000
-0000370 9014 0002 0516 0000 e004 e52d e004 e59f
-0000380 e00e e08f f008 e5be 8c78 0002 c600 e28f
-0000390 ca28 e28c fc78 e5bc c600 e28f ca28 e28c
-00003a0 fc70 e5bc c600 e28f ca28 e28c fc68 e5bc
-00003b0 4830 e92d d030 e24d 40a8 e59f 4004 e08f
-00003c0 5004 e594 fff0 ebff 009c e59f 200c e594
-00003d0 0000 e08f 0028 e58d 0004 e3a0 1008 e594
-00003e0 002c e58d 001c e28d 0020 e58d 0014 e28d
-00003f0 0016 e880 0014 e28d 1028 e28d 2020 e28d
-0000400 0024 e58d 0002 e3a0 400d e1a0 ffe4 ebff
-0000410 d004 e1a0 ffdc ebff 0b10 eddf 1003 e3a0
-0000420 1b02 edd5 2008 e28d 0040 e59f 0ba0 ee61
-0000430 1010 e58d 0000 e08f 000c e58d 000d e1a0
-0000440 100c e28d 0008 e58d 0001 e3a0 0b00 edcd
-0000450 ffd3 ebff d030 e28d 8830 e8bd f000 e320
-0000460 0000 4800 7c88 41b4 8c5c 0002 876c 0001
-0000470 86f4 0001 0000 0000 0000 0000 0000 0000
-0000480 0006 0000 0000 0000 d695 e826 2e0b 3e11
+0000000 ffff ffff ffff ffff ffff ffff ffff ffff
+0000010 ffff ffff ffff ffff ffff ffff ffff ffff
+0000020 ffff ffff ffff ffff ffff ffff ffff ffff
+0000030 ffff ffff ffff ffff ffff ffff ffff ffff
+0000040 ffff ffff ffff ffff ffff ffff ffff ffff
+0000050 ffff ffff ffff ffff ffff ffff ffff ffff
+0000060 ffff ffff ffff ffff ffff ffff ffff ffff
+0000070 ffff ffff ffff ffff ffff ffff ffff ffff
+0000080 ffff ffff ffff ffff ffff ffff ffff ffff
+0000090 ffff ffff ffff ffff ffff ffff ffff ffff
+00000a0 ffff ffff ffff ffff ffff ffff ffff ffff
+00000b0 ffff ffff ffff ffff ffff ffff ffff ffff
+00000c0 ffff ffff ffff ffff ffff ffff ffff ffff
+00000d0 ffff ffff ffff ffff ffff ffff ffff ffff
+00000e0 ffff ffff ffff ffff ffff ffff ffff ffff
+00000f0 ffff ffff ffff ffff ffff ffff ffff ffff
+0000100 ffff ffff ffff ffff ffff ffff ffff ffff
+0000110 ffff ffff ffff ffff ffff ffff ffff ffff
+0000120 ffff ffff ffff ffff ffff ffff ffff ffff
+0000130 ffff ffff ffff ffff ffff ffff ffff ffff
+0000140 ffff ffff ffff ffff ffff ffff ffff ffff
+0000150 ffff ffff ffff ffff ffff ffff ffff ffff
+0000160 ffff ffff ffff ffff ffff ffff ffff ffff
+0000170 ffff ffff ffff ffff ffff ffff ffff ffff
+0000180 ffff ffff ffff ffff ffff ffff ffff ffff
+0000190 ffff ffff ffff ffff ffff ffff ffff ffff
+00001a0 ffff ffff ffff ffff ffff ffff ffff ffff
+00001b0 ffff ffff ffff ffff ffff ffff ffff ffff
+00001c0 ffff ffff ffff ffff ffff ffff ffff ffff
+00001d0 ffff ffff ffff ffff ffff ffff ffff ffff
+00001e0 ffff ffff ffff ffff ffff ffff ffff ffff
+00001f0 ffff ffff ffff ffff ffff ffff ffff ffff
+0000200 ffff ffff ffff ffff ffff ffff ffff ffff
+0000210 ffff ffff ffff ffff ffff ffff ffff ffff
+0000220 ffff ffff ffff ffff ffff ffff ffff ffff
+0000230 ffff ffff ffff ffff ffff ffff ffff ffff
+0000240 ffff ffff ffff ffff ffff ffff ffff ffff
+0000250 ffff ffff ffff ffff ffff ffff ffff ffff
+0000260 ffff ffff ffff ffff ffff ffff ffff ffff
+0000270 ffff ffff ffff ffff ffff ffff ffff ffff
+0000280 ffff ffff ffff ffff ffff ffff ffff ffff
+0000290 ffff ffff ffff ffff ffff ffff ffff ffff
+00002a0 ffff ffff ffff ffff ffff ffff ffff ffff
+00002b0 ffff ffff ffff ffff ffff ffff ffff ffff
+00002c0 ffff ffff ffff ffff ffff ffff ffff ffff
+00002d0 ffff ffff ffff ffff ffff ffff ffff ffff
+00002e0 ffff ffff ffff ffff ffff ffff ffff ffff
+00002f0 ffff ffff ffff ffff ffff ffff ffff ffff
+0000300 ffff ffff ffff ffff ffff ffff ffff ffff
+0000310 ffff ffff ffff ffff ffff ffff ffff ffff
+0000320 ffff ffff ffff ffff ffff ffff ffff ffff
+0000330 ffff ffff ffff ffff ffff ffff ffff ffff
+0000340 ffff ffff ffff ffff ffff ffff ffff ffff
+0000350 ffff ffff ffff ffff ffff ffff ffff ffff
+0000360 ffff ffff ffff ffff ffff ffff ffff ffff
+0000370 ffff ffff ffff ffff ffff ffff ffff ffff
+0000380 ffff ffff ffff ffff ffff ffff ffff ffff
+0000390 ffff ffff ffff ffff ffff ffff ffff ffff
+00003a0 ffff ffff ffff ffff ffff ffff ffff ffff
+00003b0 ffff ffff ffff ffff ffff ffff ffff ffff
+00003c0 ffff ffff ffff ffff ffff ffff ffff ffff
+00003d0 ffff ffff ffff ffff ffff ffff ffff ffff
+00003e0 ffff ffff ffff ffff ffff ffff ffff ffff
+00003f0 ffff ffff ffff ffff ffff ffff ffff ffff
+0000400 ffff ffff ffff ffff ffff ffff ffff ffff
+0000410 ffff ffff ffff ffff ffff ffff ffff ffff
+0000420 ffff ffff ffff ffff ffff ffff ffff ffff
+0000430 ffff ffff ffff ffff ffff ffff ffff ffff
+0000440 ffff ffff ffff ffff ffff ffff ffff ffff
+0000450 ffff ffff ffff ffff ffff ffff ffff ffff
+0000460 ffff ffff ffff ffff ffff ffff ffff ffff
+0000470 ffff ffff ffff ffff ffff ffff ffff ffff
+0000480 ffff ffff ffff ffff ffff ffff ffff ffff
 0000490 ffff ffff ffff ffff ffff ffff ffff ffff
 00004a0 ffff ffff ffff ffff ffff ffff ffff ffff
 00004b0 ffff ffff ffff ffff ffff ffff ffff ffff
@@ -2246,8 +2246,8 @@
 0008c50 ffff ffff ffff ffff ffff ffff ffff ffff
 0008c60 ffff ffff ffff ffff ffff ffff ffff ffff
 0008c70 ffff ffff ffff ffff ffff ffff ffff ffff
-0008c80 ffff ffff ffff ffff ffff ffff ffff ffff
-0008c90 ffff ffff ffff ffff ffff ffff ffff ffff
+0008c80 ffff ffff ffff ffff ffff ffff ffff 0000
+0008c90 0000 6e01 ffff ffff ffff ffff ffff ffff
 0008ca0 ffff ffff ffff ffff ffff ffff ffff ffff
 0008cb0 ffff ffff ffff ffff ffff ffff ffff ffff
 0008cc0 ffff ffff ffff ffff ffff ffff ffff ffff
@@ -8497,8 +8497,8 @@
 0021300 0000 0000 0000 0000 0000 0000 0000 0000
 0021310 0000 0000 0000 0000 0000 0000 0000 0000
 0021320 0000 0000 0000 0000 0000 0000 0000 0000
-0021330 0000 0000 0000 0000 0000 0000 0000 0000
-0021340 0000 0000 0000 0000 0000 0000 0000 0000
+0021330 0000 0000 0000 0000 0000 0000 ecc4 0004
+0021340 0000 0040 0000 0000 0000 0000 0000 0000
 0021350 0000 0000 0000 0000 0000 0000 0000 0000
 0021360 0000 0000 0000 0000 0000 0000 0000 0000
 0021370 0000 0000 0000 0000 0000 0000 0000 0000

Before core0_tx.send and after:

--- AFTER1.txt	2020-07-07 15:08:58.193253104 +0800
+++ AFTER2.txt	2020-07-07 15:08:58.202253285 +0800
@@ -8498,7 +8498,7 @@
 0021310 0000 0000 0000 0000 0000 0000 0000 0000
 0021320 0000 0000 0000 0000 0000 0000 0000 0000
 0021330 0000 0000 0000 0000 0000 0000 ecc4 0004
-0021340 0000 0040 0000 0000 0000 0000 0000 0000
+0021340 0000 0000 0000 0000 0000 0000 0000 0000
 0021350 0000 0000 0000 0000 0000 0000 0000 0000
 0021360 0000 0000 0000 0000 0000 0000 0000 0000
 0021370 0000 0000 0000 0000 0000 0000 0000 0000

Code: https://git.m-labs.hk/pca006132/artiq-zynq/src/branch/corruption_test

After dumping the image into SD card, it seems that the corruption is caused by both `rpc::send_args` and `core1_tx.send` in `rpc_send_common`. It seems that the RPC buffer is written into the image. Before `send_args` and after `send_args`: ```diff --- BEFORE.txt 2020-07-07 15:08:58.212253487 +0800 +++ AFTER1.txt 2020-07-07 15:08:58.193253104 +0800 @@ -1,76 +1,76 @@ -0000000 457f 464c 0101 0001 0000 0000 0000 0000 -0000010 0003 0028 0001 0000 03b0 0000 0034 0000 -0000020 94f0 0001 0400 0500 0034 0020 0006 0028 -0000030 0015 0014 0001 7000 8b5c 0001 8b5c 0001 -0000040 8b5c 0001 0010 0000 0010 0000 0004 0000 -0000050 0004 0000 0001 0000 0000 0000 0000 0000 -0000060 0000 0000 8b6c 0001 8b6c 0001 0005 0000 -0000070 0000 0001 0001 0000 8f40 0001 8f40 0002 -0000080 8f40 0002 0114 0000 0114 0000 0006 0000 -0000090 0000 0001 0002 0000 8f68 0001 8f68 0002 -00000a0 8f68 0002 0098 0000 0098 0000 0006 0000 -00000b0 0004 0000 e551 6474 0000 0000 0000 0000 -00000c0 0000 0000 0000 0000 0000 0000 0006 0000 -00000d0 0010 0000 e552 6474 8f40 0001 8f40 0002 -00000e0 8f40 0002 00c0 0000 00c0 0000 0004 0000 -00000f0 0001 0000 0003 0000 000f 0000 0005 0000 -0000100 0009 0000 0004 0000 0000 0000 0000 0000 -0000110 0000 0000 0006 0000 000e 0000 0008 0000 -0000120 0000 0000 000d 0000 0007 0000 000b 0000 -0000130 0000 0000 000a 0000 0003 0000 0000 0000 -0000140 000c 0000 0003 0000 0006 0000 0002 0000 -0000150 0006 0000 0089 31a8 c400 8862 0006 0000 -0000160 000a 0000 000d 0000 af00 e834 5dd2 f86f -0000170 4542 ecd5 e3bb 7c92 cbf4 00f3 dabe 1b57 -0000180 71d9 1c58 5476 943c ef59 7997 0000 0000 -0000190 0000 0000 0000 0000 0000 0000 0000 0000 -00001a0 03b0 0000 0000 0000 0003 0008 0000 0000 -00001b0 9020 0002 0000 0000 0003 000f 0030 0000 -00001c0 0000 0000 0000 0000 0010 0000 0001 0000 -00001d0 0000 0000 0000 0000 0010 0000 0021 0000 -00001e0 0000 0000 0000 0000 0010 0000 006c 0000 -00001f0 9054 0002 0000 0000 0010 000f 0041 0000 -0000200 904c 0002 0008 0000 0011 000f 004a 0000 -0000210 9054 0002 0000 0000 0010 000f 007f 0000 -0000220 9054 0002 0000 0000 0010 000f 0015 0000 -0000230 03b0 0000 00c4 0000 0012 0008 006b 0000 -0000240 9054 0002 0000 0000 0010 000f 0051 0000 -0000250 9054 0002 0000 0000 0010 000f 005d 0000 -0000260 9054 0002 0000 0000 0010 000f 0077 0000 -0000270 9054 0002 0000 0000 0010 000f 5f00 615f -0000280 7472 7169 705f 7265 6f73 616e 696c 7974 -0000290 5f00 6d5f 646f 6e69 7469 5f5f 7200 6370 -00002a0 735f 6e65 5f64 7361 6e79 0063 7472 6f69 -00002b0 675f 7465 635f 756f 746e 7265 7400 7079 -00002c0 6965 666e 006f 655f 6164 6174 5f00 625f -00002d0 7373 735f 6174 7472 5f00 625f 7373 735f -00002e0 6174 7472 5f5f 5f00 625f 7373 655f 646e -00002f0 5f5f 5f00 655f 646e 5f5f 5f00 6e65 0064 -0000300 8f44 0002 0017 0000 8f4c 0002 0017 0000 -0000310 8f54 0002 0017 0000 8f58 0002 0017 0000 -0000320 8f60 0002 0017 0000 8f64 0002 0017 0000 -0000330 9024 0002 0017 0000 9028 0002 0017 0000 -0000340 9034 0002 0017 0000 903c 0002 0017 0000 -0000350 9044 0002 0017 0000 904c 0002 0017 0000 -0000360 900c 0002 0316 0000 9010 0002 0416 0000 -0000370 9014 0002 0516 0000 e004 e52d e004 e59f -0000380 e00e e08f f008 e5be 8c78 0002 c600 e28f -0000390 ca28 e28c fc78 e5bc c600 e28f ca28 e28c -00003a0 fc70 e5bc c600 e28f ca28 e28c fc68 e5bc -00003b0 4830 e92d d030 e24d 40a8 e59f 4004 e08f -00003c0 5004 e594 fff0 ebff 009c e59f 200c e594 -00003d0 0000 e08f 0028 e58d 0004 e3a0 1008 e594 -00003e0 002c e58d 001c e28d 0020 e58d 0014 e28d -00003f0 0016 e880 0014 e28d 1028 e28d 2020 e28d -0000400 0024 e58d 0002 e3a0 400d e1a0 ffe4 ebff -0000410 d004 e1a0 ffdc ebff 0b10 eddf 1003 e3a0 -0000420 1b02 edd5 2008 e28d 0040 e59f 0ba0 ee61 -0000430 1010 e58d 0000 e08f 000c e58d 000d e1a0 -0000440 100c e28d 0008 e58d 0001 e3a0 0b00 edcd -0000450 ffd3 ebff d030 e28d 8830 e8bd f000 e320 -0000460 0000 4800 7c88 41b4 8c5c 0002 876c 0001 -0000470 86f4 0001 0000 0000 0000 0000 0000 0000 -0000480 0006 0000 0000 0000 d695 e826 2e0b 3e11 +0000000 ffff ffff ffff ffff ffff ffff ffff ffff +0000010 ffff ffff ffff ffff ffff ffff ffff ffff +0000020 ffff ffff ffff ffff ffff ffff ffff ffff +0000030 ffff ffff ffff ffff ffff ffff ffff ffff +0000040 ffff ffff ffff ffff ffff ffff ffff ffff +0000050 ffff ffff ffff ffff ffff ffff ffff ffff +0000060 ffff ffff ffff ffff ffff ffff ffff ffff +0000070 ffff ffff ffff ffff ffff ffff ffff ffff +0000080 ffff ffff ffff ffff ffff ffff ffff ffff +0000090 ffff ffff ffff ffff ffff ffff ffff ffff +00000a0 ffff ffff ffff ffff ffff ffff ffff ffff +00000b0 ffff ffff ffff ffff ffff ffff ffff ffff +00000c0 ffff ffff ffff ffff ffff ffff ffff ffff +00000d0 ffff ffff ffff ffff ffff ffff ffff ffff +00000e0 ffff ffff ffff ffff ffff ffff ffff ffff +00000f0 ffff ffff ffff ffff ffff ffff ffff ffff +0000100 ffff ffff ffff ffff ffff ffff ffff ffff +0000110 ffff ffff ffff ffff ffff ffff ffff ffff +0000120 ffff ffff ffff ffff ffff ffff ffff ffff +0000130 ffff ffff ffff ffff ffff ffff ffff ffff +0000140 ffff ffff ffff ffff ffff ffff ffff ffff +0000150 ffff ffff ffff ffff ffff ffff ffff ffff +0000160 ffff ffff ffff ffff ffff ffff ffff ffff +0000170 ffff ffff ffff ffff ffff ffff ffff ffff +0000180 ffff ffff ffff ffff ffff ffff ffff ffff +0000190 ffff ffff ffff ffff ffff ffff ffff ffff +00001a0 ffff ffff ffff ffff ffff ffff ffff ffff +00001b0 ffff ffff ffff ffff ffff ffff ffff ffff +00001c0 ffff ffff ffff ffff ffff ffff ffff ffff +00001d0 ffff ffff ffff ffff ffff ffff ffff ffff +00001e0 ffff ffff ffff ffff ffff ffff ffff ffff +00001f0 ffff ffff ffff ffff ffff ffff ffff ffff +0000200 ffff ffff ffff ffff ffff ffff ffff ffff +0000210 ffff ffff ffff ffff ffff ffff ffff ffff +0000220 ffff ffff ffff ffff ffff ffff ffff ffff +0000230 ffff ffff ffff ffff ffff ffff ffff ffff +0000240 ffff ffff ffff ffff ffff ffff ffff ffff +0000250 ffff ffff ffff ffff ffff ffff ffff ffff +0000260 ffff ffff ffff ffff ffff ffff ffff ffff +0000270 ffff ffff ffff ffff ffff ffff ffff ffff +0000280 ffff ffff ffff ffff ffff ffff ffff ffff +0000290 ffff ffff ffff ffff ffff ffff ffff ffff +00002a0 ffff ffff ffff ffff ffff ffff ffff ffff +00002b0 ffff ffff ffff ffff ffff ffff ffff ffff +00002c0 ffff ffff ffff ffff ffff ffff ffff ffff +00002d0 ffff ffff ffff ffff ffff ffff ffff ffff +00002e0 ffff ffff ffff ffff ffff ffff ffff ffff +00002f0 ffff ffff ffff ffff ffff ffff ffff ffff +0000300 ffff ffff ffff ffff ffff ffff ffff ffff +0000310 ffff ffff ffff ffff ffff ffff ffff ffff +0000320 ffff ffff ffff ffff ffff ffff ffff ffff +0000330 ffff ffff ffff ffff ffff ffff ffff ffff +0000340 ffff ffff ffff ffff ffff ffff ffff ffff +0000350 ffff ffff ffff ffff ffff ffff ffff ffff +0000360 ffff ffff ffff ffff ffff ffff ffff ffff +0000370 ffff ffff ffff ffff ffff ffff ffff ffff +0000380 ffff ffff ffff ffff ffff ffff ffff ffff +0000390 ffff ffff ffff ffff ffff ffff ffff ffff +00003a0 ffff ffff ffff ffff ffff ffff ffff ffff +00003b0 ffff ffff ffff ffff ffff ffff ffff ffff +00003c0 ffff ffff ffff ffff ffff ffff ffff ffff +00003d0 ffff ffff ffff ffff ffff ffff ffff ffff +00003e0 ffff ffff ffff ffff ffff ffff ffff ffff +00003f0 ffff ffff ffff ffff ffff ffff ffff ffff +0000400 ffff ffff ffff ffff ffff ffff ffff ffff +0000410 ffff ffff ffff ffff ffff ffff ffff ffff +0000420 ffff ffff ffff ffff ffff ffff ffff ffff +0000430 ffff ffff ffff ffff ffff ffff ffff ffff +0000440 ffff ffff ffff ffff ffff ffff ffff ffff +0000450 ffff ffff ffff ffff ffff ffff ffff ffff +0000460 ffff ffff ffff ffff ffff ffff ffff ffff +0000470 ffff ffff ffff ffff ffff ffff ffff ffff +0000480 ffff ffff ffff ffff ffff ffff ffff ffff 0000490 ffff ffff ffff ffff ffff ffff ffff ffff 00004a0 ffff ffff ffff ffff ffff ffff ffff ffff 00004b0 ffff ffff ffff ffff ffff ffff ffff ffff @@ -2246,8 +2246,8 @@ 0008c50 ffff ffff ffff ffff ffff ffff ffff ffff 0008c60 ffff ffff ffff ffff ffff ffff ffff ffff 0008c70 ffff ffff ffff ffff ffff ffff ffff ffff -0008c80 ffff ffff ffff ffff ffff ffff ffff ffff -0008c90 ffff ffff ffff ffff ffff ffff ffff ffff +0008c80 ffff ffff ffff ffff ffff ffff ffff 0000 +0008c90 0000 6e01 ffff ffff ffff ffff ffff ffff 0008ca0 ffff ffff ffff ffff ffff ffff ffff ffff 0008cb0 ffff ffff ffff ffff ffff ffff ffff ffff 0008cc0 ffff ffff ffff ffff ffff ffff ffff ffff @@ -8497,8 +8497,8 @@ 0021300 0000 0000 0000 0000 0000 0000 0000 0000 0021310 0000 0000 0000 0000 0000 0000 0000 0000 0021320 0000 0000 0000 0000 0000 0000 0000 0000 -0021330 0000 0000 0000 0000 0000 0000 0000 0000 -0021340 0000 0000 0000 0000 0000 0000 0000 0000 +0021330 0000 0000 0000 0000 0000 0000 ecc4 0004 +0021340 0000 0040 0000 0000 0000 0000 0000 0000 0021350 0000 0000 0000 0000 0000 0000 0000 0000 0021360 0000 0000 0000 0000 0000 0000 0000 0000 0021370 0000 0000 0000 0000 0000 0000 0000 0000 ``` Before `core0_tx.send` and after: ```diff --- AFTER1.txt 2020-07-07 15:08:58.193253104 +0800 +++ AFTER2.txt 2020-07-07 15:08:58.202253285 +0800 @@ -8498,7 +8498,7 @@ 0021310 0000 0000 0000 0000 0000 0000 0000 0000 0021320 0000 0000 0000 0000 0000 0000 0000 0000 0021330 0000 0000 0000 0000 0000 0000 ecc4 0004 -0021340 0000 0040 0000 0000 0000 0000 0000 0000 +0021340 0000 0000 0000 0000 0000 0000 0000 0000 0021350 0000 0000 0000 0000 0000 0000 0000 0000 0021360 0000 0000 0000 0000 0000 0000 0000 0000 0021370 0000 0000 0000 0000 0000 0000 0000 0000 ``` Code: https://git.m-labs.hk/pca006132/artiq-zynq/src/branch/corruption_test
sb10q commented 2020-07-07 16:11:21 +08:00
Author
Owner
Copy Link

Confirmed this works, well done @pca006132 !

Confirmed this works, well done @pca006132 !
sb10q closed this issue 2020-07-07 16:11:24 +08:00
astro commented 2020-07-08 05:33:52 +08:00
Contributor
Copy Link

Oh wow. Very nice find. I am very happy that this bug hunt is over.

Oh wow. Very nice find. I am very happy that this bug hunt is over.
sb10q commented 2020-07-14 16:59:44 +08:00
Author
Owner
Copy Link

Famous last words... there's still plenty of corruption and obscure crashes and some seems triggered by RPCs.

Famous last words... there's still plenty of corruption and obscure crashes and some seems triggered by RPCs.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
priority:critical

   
priority:high

   
priority:low

   
priority:medium
No Label
priority:critical
priority:high
priority:low
priority:medium
Milestone
Clear milestone
No items
No Milestone
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/artiq-zynq#8
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?