intermittent transmit problem #65

New Issue

pca006132 · 2020-08-25T11:46:12+08:00

pca006132 commented

2020-08-25 11:46:12 +08:00

It seems that there is still occasional packet drop for the ethernet driver. The other end would just wait forever for the packet.

The probablity is pretty low, but I do get it once or twice this morning when I try to test the performance for RPC.

It seems that there is still occasional packet drop for the ethernet driver. The other end would just wait forever for the packet. The probablity is pretty low, but I do get it once or twice this morning when I try to test the performance for RPC.

pca006132 commented

2020-08-26 14:18:53 +08:00

Poster

Also when I try to modify the socket options with tcp_nodelay, our code would fail, perhaps some packets are dropped. This makes me a bit worried.

Shall we try to do some fault injection into the system and see if our ethernet driver is robust enough? Like packet re-ordering, packet loss, etc.

Also when I try to modify the socket options with `tcp_nodelay`, our code would fail, perhaps some packets are dropped. This makes me a bit worried. Shall we try to do some fault injection into the system and see if our ethernet driver is robust enough? Like packet re-ordering, packet loss, etc.

sb10q commented

2020-08-27 20:10:09 +08:00

Do you mean the TCP/IP stack (not quite the driver)?

pca006132 commented

2020-08-28 09:31:45 +08:00

Poster

Do you mean the TCP/IP stack (not quite the driver)?

I'm not sure, the behavior is that the next kernel would not be started until I try another connection such as artiq_coremgmt log. Perhaps this is not due to packet drop but some weird bug like M-Labs/artiq-zynq#93

Note: this is very rare.

> Do you mean the TCP/IP stack (not quite the driver)? I'm not sure, the behavior is that the next kernel would not be started until I try another connection such as `artiq_coremgmt log`. Perhaps this is not due to packet drop but some weird bug like https://git.m-labs.hk/M-Labs/artiq-zynq/issues/93 Note: this is very rare.

sb10q commented

2020-08-28 09:34:12 +08:00

Does it have to be an actual TCP connection, or does e.g. a ping packet unblock the kernel?

pca006132 commented

2020-09-02 11:00:34 +08:00

Poster

Does it have to be an actual TCP connection, or does e.g. a ping packet unblock the kernel?

yes, a ping packet would do.

> Does it have to be an actual TCP connection, or does e.g. a ping packet unblock the kernel? yes, a ping packet would do.

sb10q commented

2020-09-02 16:53:38 +08:00

OK, so it looks like it could be in the driver indeed.

pca006132 commented

2020-09-03 10:46:37 +08:00

Poster

Running the test_performance in a loop could reproduce this issue consistently. It seems that the packet is not really transmitted until other packet comes. I am not sure what is happening there.

Running the `test_performance` in a loop could reproduce this issue consistently. It seems that the packet is not really transmitted until other packet comes. I am not sure what is happening there.

pca006132 changed title from ~~intermittent packet drop~~ to intermittent transmit problem

2020-09-03 14:59:32 +08:00

pca006132 commented

2020-09-04 09:31:08 +08:00

Poster

Not occurring after fixing the memory leak in aritq-zyqn. Possibly issue with timing, and the response time for the firmware increases as the heap is fragmented.

Closing now.

Not occurring after fixing the memory leak in aritq-zyqn. Possibly issue with timing, and the response time for the firmware increases as the heap is fragmented. Closing now.

pca006132 closed this issue

2020-09-04 09:31:10 +08:00

Sign in to join this conversation.

No Label

No Milestone

No Assignees

2 Participants

Notifications

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: M-Labs/zynq-rs#65