[DRBD-user] Warning: Data Corruption Issue Discovered in DRBD 8.4 and 9.0

Eric Robinson eric.robinson at psmnv.com
Tue Sep 26 05:46:12 CEST 2017

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


> I think the conclusion you've arrived at is not quite accurate. It could be
> described more accurately as a possible data corruption issue specific to drbd
> and/or the kernel, and trim commands as issued by the TrimTester software. It
> appears TrimTester was written to debug a very specific SSD trim issue when
> accessing SSDs directly. It would be necessary to look at exactly what
> TrimTester is doing to determine if this is a real bug or some anomaly due to
> what exactly the software is doing.  

I anticipated this objection. That's why I also ran the tests without the part of TrimTester that issues TRIM commands. There are three parts to TrimTester. The main executable just writes tons of data to the device.  A separate shell script called trim_periodic.sh issues TRIM commands. And the third part called drop_cache.sh periodically drops the system caches. I ran some of my tests using only the TrimTester executable without the TRIM management pieces, and the results were the same.


> If there is an issue it is possible it could be
> either in in drbd or the kernel. Just because the issue is present when drbd is
> used does not automatically exclude the problem being in kernel, it may be an
> issue that only arises in the kernel when drbd is used. What if you try with a 
> newer kernel such as 4.9, would the same corruption occur?  


I can't disagree with that. What I do know is that the corruption only occurred when DRBD was in the stack, and never when it isn't. I can test with a different kernel, but if it still fails (as seems likely) then we would not have learned anything


> Also, would
> the specific conditions created by TrimTester ever be reproduced in a real
> world scenario, or is it something that will only ever show up when running
> TrimTester?

I don't know. The source code for TrimTester is in the download, but I don't know C or Linux system calls well enough to answer those questions. According to the TrimTester author, all it does is write a buttload of data to the device and work it hard. 

--
Eric Robinson




More information about the drbd-user mailing list