[DRBD-user] Sun Fire x4240 Benchmarks (was: Re: Ideas for splitting data between A and C protocol devices - safe or not?)

Casey Allen Shobe cshobe at bepress.com
Mon Aug 25 21:32:12 CEST 2008

Note: "permalinks" may not be as permanent as we would like,
direct links of old sources may well be a few messages off.


iostat -k -x monitoring

This test has the advantage of being measurable on a heavily-loaded  
production resource.  I was able to get good data from times when our  
production server (2 15kRPM disk RAID1) was under 100% disk  
utilization load.  On new Sun Fire x4240 servers with DRBD, I created  
sustained load with pgbench -c 32 -t 50000.  An odd thing is that  
pgbench returns better benchmarks on the current production server, if  
our historical records are accurate, anyways.  But when watching how  
much is actually being sent to the disk, the newer configuration with  
DRBD seems about ten times better than current production hardware,  
and I think this is a more reliable test.  This is also consistent  
with other tests which show a performance improvement of about 2x when  
the DRBD secondary is disconnected.

* CURRENT PRODUCTION:
Average kB/s write speed: 1686.92

Device:           rrqm/s wrqm/s r/s     w/s     rsec/s    wsec/s     
rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.20   123.22 350.10  55.60  5593.48   1534.83   
2796.74   767.41    17.57     4.98   12.35   2.49 100.94
sda               4.67    30.02 440.57  22.92  9989.45    524.14   
4994.73   262.07    22.68     2.31    4.98   2.06  95.66
sda               0.00    43.70 363.62  28.86  5913.82    734.96   
2956.91   367.48    16.94     3.76    9.59   2.48  97.32
sda               0.00    45.25 324.24  26.87  5207.27    698.18   
2603.64   349.09    16.82     2.62    7.46   2.75  96.57
sda               0.00    53.81 361.03  36.91  5972.78    852.78   
2986.39   426.39    17.15     3.21    8.03   2.56 101.86
sda               0.20    42.54 342.94  30.06  5603.27    705.11   
2801.64   352.56    16.91     4.42   11.83   2.74 102.33
sda               1.22    35.85 355.19  50.51  6030.14    975.97   
3015.07   487.98    17.27     3.88    9.56   2.51 101.83
sda               0.00    66.13 356.80  50.30  6206.90   1049.90   
3103.45   524.95    17.83     5.55   13.63   2.49 101.50
sda               0.20    41.37 345.78  26.51  5518.07    650.60   
2759.04   325.30    16.57     5.43   14.57   2.68  99.68
sda               0.81  1329.70 283.03  78.59  4543.03  11338.99   
2271.52  5669.49    43.92    15.82   43.72   2.76  99.88
sda               0.00    35.28 305.85  17.74  4962.90    566.13   
2481.45   283.06    17.09     3.29   10.15   3.11 100.48
sda               0.00    43.93 321.05  25.30  5148.18    584.62   
2574.09   292.31    16.55     4.88   14.16   2.92 101.30
sda               0.00    28.34 335.11  20.33  5409.45    504.31   
2704.72   252.16    16.64     4.44   12.51   2.89 102.75
sda               0.21   150.31 327.01  57.53  5283.30   1916.70   
2641.65   958.35    18.72     4.36   11.34   2.67 102.60
sda               0.21  1298.54 292.69  76.62  4733.19  11213.36   
2366.60  5606.68    43.18     8.22   22.16   2.81 103.88
sda               1.02   381.97 255.12  67.83  4260.66   3962.30   
2130.33  1981.15    25.46     8.75   27.14   3.16 102.13
sda               0.83    73.65 274.90  46.89  4426.56   1201.66   
2213.28   600.83    17.49     5.96   18.47   3.16 101.66
sda               1.65    50.62 369.63  32.64  7033.06    928.93   
3516.53   464.46    19.79     5.12   12.74   2.48  99.83
sda              53.78    44.38 895.50  57.26  26368.92  1092.84  
13184.46   546.42    28.82     8.63    9.06   1.06 100.86
sda              73.77  1303.28 1156.97 85.45  33654.10 11359.02  
16827.05  5679.51    36.23    18.45   14.82   0.82 101.89
sda              39.96    53.89 742.83  58.40  19800.00  1065.57   
9900.00   532.79    26.04     7.81    9.70   1.27 101.97
sda               0.20  2305.47 417.41  176.52 7336.03  20089.07   
3668.02 10044.53    46.18    22.34   37.73   1.71 101.30
sda             292.09   145.64 436.71  124.34 89179.72  2560.65  
44589.86  1280.32   163.51    11.29   19.83   1.80 101.18
sda             138.54   445.64 460.65  217.65 83165.92  5879.11  
41582.96  2939.55   131.28    12.61   18.58   1.50 101.42
sda             135.63    90.28 556.68  68.02  97361.94  1387.85  
48680.97   693.93   158.08    23.54   37.41   1.62 101.38
sda             126.52    44.33 541.09  19.03  92034.01   568.42  
46017.00   284.21   165.33    12.67   22.60   1.81 101.21
sda              94.53  1055.67 427.13  38.06  65488.26  2636.44  
32744.13  1318.22   146.45    49.56   69.77   2.18 101.30
sda              92.48   228.86 321.95  57.11  48443.90  8609.76  
24221.95  4304.88   150.51    30.10  124.90   2.68 101.71
sda             134.62   111.34 395.34  49.60  69872.06  1472.06  
34936.03   736.03   160.35    16.49   37.11   2.28 101.30
sda             161.05    25.35 443.41  20.89  90185.80   520.89  
45092.90   260.45   195.36     7.29   15.68   2.19 101.50
sda             107.51   140.16 446.04  53.75  92692.90  1585.40  
46346.45   792.70   188.63    11.38   22.70   2.03 101.50
sda             122.52    32.86 560.45  19.27  106283.16  464.10  
53141.58   232.05   184.14     6.88   11.93   1.75 101.50
sda              64.50  1301.42 460.85  68.97  46052.74 10959.84  
23026.37  5479.92   107.61     9.70   15.73   1.91 101.42
sda               7.52   141.87 380.89  96.54  8173.98   2308.94   
4086.99  1154.47    21.96     4.97   13.28   2.01  95.77
sda               7.47   120.00 292.53  95.56  6280.40   1963.64   
3140.20   981.82    21.24    10.16   26.20   2.60 100.93
sda              17.89   670.73 317.68  40.45  14978.86   886.18   
7489.43   443.09    44.30     7.23   16.02   2.59  92.60
sda               4.05  1856.68 234.21  122.47 4058.30  20965.18   
2029.15 10482.59    70.16    14.69   45.35   2.81 100.40
sda               0.20    45.16 305.24  44.56  4874.19    829.03   
2437.10   414.52    16.30     2.88    8.26   2.84  99.44
sda               0.00     1.62 282.15  6.69   4478.70    149.29   
2239.35    74.65    16.02     2.46    8.52   3.49 100.77
sda               0.81    63.31 316.13  45.16  5193.55    996.77   
2596.77   498.39    17.13     7.83   21.72   2.79 100.89
sda               0.00  1270.85 282.19  70.85  4618.62  10863.16   
2309.31  5431.58    43.85    17.12   48.46   2.86 101.05
sda               0.20    34.28 337.32  26.57  5645.44    563.08   
2822.72   281.54    17.06     4.91   13.52   2.78 101.34
sda               0.20    65.99 319.23  54.45  5078.54   1204.86   
2539.27   602.43    16.81     9.63   25.75   2.71 101.13
sda               0.00  1258.27 317.74  71.17  5151.61  10741.94   
2575.81  5370.97    40.87    12.69   32.63   2.59 100.89
sda               0.40    43.72 341.30  21.46  5519.03    602.43   
2759.51   301.21    16.88     5.04   13.75   2.79 101.30
sda               0.61    72.47 326.72  33.20  5510.93    963.56   
2755.47   481.78    17.99    10.83   30.06   2.81 101.30
sda               2.03    30.28 392.48  13.41  6980.49    408.13   
3490.24   204.07    18.20     4.30   10.74   2.49 100.98
sda               6.46    38.79 388.28  26.06  8305.45    620.61   
4152.73   310.30    21.54     4.58   11.09   2.43 100.69
sda               0.00   127.59 362.07  62.27  6325.35   1887.22   
3162.68   943.61    19.35     5.91   13.81   2.39 101.50
sda               0.20    32.80 381.49  18.11  6675.25    571.43   
3337.63   285.71    18.13     3.09    7.77   2.50  99.80
sda               0.00   155.56 350.91  50.10  5747.07   1819.80   
2873.54   909.90    18.87     7.74   19.35   2.49  99.88
sda               0.20    48.79 370.24  28.95  6116.60    822.67   
3058.30   411.34    17.38     5.09   12.67   2.54 101.30
sda               0.00    32.39 345.75  25.10  5515.79    728.74   
2757.89   364.37    16.84     4.46   12.07   2.71 100.57
sda               0.40    71.72 344.24  55.56  5536.97   1239.60   
2768.48   619.80    16.95     5.73   14.33   2.51 100.20
sda               0.00    40.28 362.75  35.83  5844.53    719.03   
2922.27   359.51    16.47     5.11   12.85   2.54 101.21
sda               4.66  1254.25 368.62  70.65  8662.35  10735.22   
4331.17  5367.61    44.16    11.48   26.13   2.27  99.84
sda               0.20    51.42 329.15  43.32  5246.96    983.00   
2623.48   491.50    16.73     6.31   16.94   2.70 100.73
sda               0.20  1277.96 333.67  86.17  5548.70  11031.66   
2774.35  5515.83    39.49     9.80   23.29   2.38  99.80
sda               0.40  1266.93 363.33  70.54  6010.42  10866.53   
3005.21  5433.27    38.90     9.84   22.69   2.30  99.96
sda               0.00    63.11 339.14  48.57  5544.26   1059.02   
2772.13   529.51    17.03     7.46   19.16   2.64 102.46
sda               0.40    90.51 352.93  38.99  6073.54   1118.38   
3036.77   559.19    18.35     4.59   11.81   2.55  99.80
sda               1.42   106.10 331.71  48.58  5302.44   1334.96   
2651.22   667.48    17.45     3.87   10.17   2.64 100.57

* NEW SUN FIRE x4240 CLUSTER (DRBD running with protocol C):
Average kB/s write speed: 18280.49

Device:         rrqm/s   wrqm/s  r/s  w/s        rkB/s    wkB/s avgrq- 
sz avgqu-sz   await  svctm  %util
drbd0             0.00     0.00  0.00 1381.17     0.00 21610.24     
31.29   228.21  223.86   0.55  75.69
drbd0             0.00     0.00  0.00 801.28      0.00 14240.59     
35.54    71.23   50.26   1.06  85.19
drbd0             0.00     0.00  0.00 927.80      0.00 11899.25     
25.65   123.10  166.90   0.97  89.55
drbd0             0.00     0.00  0.00 393.57      0.00  9052.21     
46.00     4.91   12.53   2.23  87.68
drbd0             0.00     0.00  0.00 392.19      0.00  8993.31     
45.86     4.73   12.11   2.22  87.17
drbd0             0.00     0.00  0.00 2549.33     0.00 27588.53     
21.64   427.99  134.36   0.37  93.88
drbd0             0.00     0.00  0.00 2625.44     0.00 18898.62     
14.40   978.93  406.77   0.38  98.62
drbd0             0.00     0.00  0.00 1686.80     0.00 22007.23     
26.09   219.14  130.11   0.45  75.95
drbd0             0.00     0.00  0.18 1066.84     0.71 21749.02     
40.77     2.66    2.51   0.67  71.12
drbd0             0.00     0.00  0.00 3090.12     0.00 38816.28     
25.12   233.81   62.17   0.25  78.49
drbd0             0.00     0.00  0.00 2596.43     0.00 18497.74     
14.25   898.75  356.70   0.36  93.98
drbd0             0.00     0.00  0.00 1484.43     0.00 16417.56     
22.12   391.39  273.83   0.64  95.21
drbd0             0.00     0.00  0.00 413.59      0.00 10113.04     
48.90     5.70   13.79   2.08  86.05
drbd0             0.00     0.00  0.00 518.07      0.00 10294.81     
39.74    11.67   22.57   1.63  84.44
drbd0             0.00     0.00  0.00 562.08      0.00 12738.87     
45.33     4.48    7.98   1.56  87.55
drbd0             0.00     0.00  0.00 477.88      0.00  9833.46     
41.15    10.47   21.91   1.84  87.92
drbd0             0.00     0.00  0.00 3826.10     0.00 38017.67     
19.87   744.32  190.20   0.25  95.18

* NEW SUN FIRE x4240 CLUSTER (with DRBD secondary disconnected):
Average kB/s write speed: 38543.34

Device:         rrqm/s   wrqm/s  r/s  w/s        rkB/s    wkB/s avgrq- 
sz avgqu-sz   await  svctm  %util
drbd0             0.00     0.00  0.00 4856.09     0.00 47280.46     
19.47    16.84    3.65   0.15  71.18
drbd0             0.00     0.00  0.00 3207.39     0.00 32246.69     
20.11     4.52    1.72   0.13  42.22
drbd0             0.00     0.00  0.00 2675.51     0.00 39217.94     
29.32     0.50    0.19   0.10  26.17
drbd0             0.00     0.00  0.20 5036.02     0.79 52669.29     
20.92    52.49   10.57   0.14  69.69
drbd0             0.00     0.00  0.00 3403.19     0.00 33696.41     
19.80     4.99    1.70   0.21  70.52
drbd0             0.00     0.00  0.00 2015.66     0.00 28346.77     
28.13     0.42    0.21   0.09  18.40
drbd0             0.00     0.00  0.00 4527.74     0.00 53557.74     
23.66     7.52    1.74   0.07  30.75
drbd0             0.00     0.00  0.00 3388.49     0.00 26375.85     
15.57     9.69    3.19   0.24  81.70
drbd0             0.00     0.00  0.00 2289.02     0.00 28259.73     
24.69     3.56    1.66   0.18  42.00
drbd0             0.00     0.00  0.00 2662.98     0.00 37339.53     
28.04     0.61    0.24   0.09  23.84
drbd0             0.00     0.00  0.00 4568.43     0.00 47341.96     
20.73    34.62    7.77   0.16  70.98
drbd0             0.00     0.00  0.00 3160.08     0.00 28546.77     
18.07     8.46    3.03   0.19  58.75
drbd0             0.00     0.00  0.20 2766.60     0.79 42462.08     
30.69     0.39    0.14   0.09  24.17
drbd0             0.00     0.00  0.00 4017.09     0.00 47566.60     
23.68    14.12    3.39   0.13  52.23
drbd0             0.00     0.00  0.00 3600.19     0.00 29346.80     
16.30     9.25    3.17   0.23  83.88
drbd0             0.00     0.00  0.00 2738.56     0.00 32945.94     
24.06     2.92    1.20   0.18  50.09
drbd0             0.00     0.00  0.00 2489.88     0.00 33831.13     
27.17     0.48    0.19   0.08  21.01
drbd0             0.00     0.00  0.00 5128.38     0.00 51743.63     
20.18    23.32    4.72   0.17  84.75
drbd0             0.00     0.00  0.00 3077.88     0.00 30346.31     
19.72     6.41    2.33   0.23  71.64
drbd0             0.00     0.00  0.00 2597.11     0.00 38963.39     
30.01     0.47    0.19   0.11  28.13
drbd0             0.00     0.00  0.00 5163.50     0.00 56120.68     
21.74    21.68    4.32   0.14  72.36
drbd0             0.00     0.00  0.00 3379.74     0.00 27410.41     
16.22    11.97    3.92   0.23  78.81
drbd0             0.00     0.00  0.00 2516.51     0.00 35833.78     
28.48     0.97    0.39   0.11  27.32
drbd0             0.00     0.00  0.39 4557.76     1.57 54898.23     
24.09    11.79    2.59   0.07  34.18
drbd0             0.00     0.00  0.00 2849.34     0.00 25360.30     
17.80    16.19    6.15   0.25  70.70
drbd0             0.00     0.00  0.00 3261.96     0.00 33731.76     
20.68     4.89    1.78   0.16  51.76
drbd0             0.00     0.00  0.00 2043.52     0.00 30833.33     
30.18     0.50    0.26   0.09  18.89
drbd0             0.00     0.00  0.00 4215.49     0.00 48196.08     
22.87    13.53    3.03   0.12  51.37
drbd0             0.00     0.00  0.00 3605.38     0.00 28974.50     
16.07     8.83    3.13   0.22  77.89
drbd0             0.00     0.00  0.00 2308.95     0.00 29350.97     
25.42     2.99    1.38   0.17  38.72
drbd0             0.00     0.00  0.00 2628.49     0.00 35123.02     
26.72     0.54    0.22   0.09  23.40
drbd0             0.00     0.00  0.00 5380.86     0.00 54895.31     
20.40    17.00    3.33   0.12  66.99
drbd0             0.00     0.00  0.00 3178.64     0.00 28961.55     
18.22     8.13    2.85   0.26  82.33
drbd0             0.00     0.00  0.00 2804.06     0.00 40173.31     
28.65     0.34    0.12   0.07  19.73
drbd0             0.00     0.00  0.00 4493.29     0.00 49576.33     
22.07    15.04    3.27   0.14  63.12
drbd0             0.00     0.00  0.00 3310.85     0.00 27405.43     
16.55     9.72    3.44   0.25  82.17
drbd0             0.00     0.00  0.20 2472.67     0.79 28863.37     
23.34     2.79    1.34   0.14  35.84
drbd0             0.00     0.00  0.00 2304.39     0.00 32481.68     
28.19     0.58    0.25   0.10  22.14
drbd0             0.00     0.00  0.00 5572.21     0.00 59488.03     
21.35    12.52    2.36   0.11  62.68
drbd0             0.00     0.00  0.00 3450.09     0.00 27968.56     
16.21     8.35    2.73   0.24  81.90
drbd0             0.00     0.00  0.19 2627.34     0.76 36135.37     
27.51     0.68    0.26   0.10  27.53
drbd0             0.00     0.00  0.20 5086.27     0.78 61538.04     
24.20    10.81    2.13   0.08  41.57
drbd0             0.00     0.00  0.00 3414.10     0.00 27194.67     
15.93    12.23    3.86   0.25  87.05
drbd0             0.00     0.00  0.00 2621.64     0.00 30576.00     
23.33     6.37    2.71   0.19  49.64
drbd0             0.00     0.00  0.00 2312.89     0.00 32974.22     
28.51     0.70    0.30   0.12  27.15
drbd0             0.00     0.00  0.00 4520.85     0.00 50033.20     
22.13    20.79    4.71   0.13  59.85
drbd0             0.00     0.00  0.40 4137.52     1.60 37271.06     
18.02     9.56    2.62   0.21  86.43
drbd0             0.00     0.00  0.00 2460.08     0.00 30514.17     
24.81     2.42    1.11   0.21  50.70
drbd0             0.00     0.00  0.20 3989.17     0.79 48795.28     
24.46    12.05    2.90   0.10  39.37
drbd0             0.00     0.00  0.00 4282.18     0.00 39978.61     
18.67    14.16    3.70   0.21  90.69
drbd0             0.00     0.00  0.00 3468.18     0.00 39852.17     
22.98     8.66    2.70   0.23  78.46
drbd0             0.00     0.00  0.00 2629.47     0.00 38732.42     
29.46     0.38    0.15   0.07  19.45
drbd0             0.00     0.00  0.00 4976.03     0.00 55009.82     
22.11    13.02    2.79   0.11  56.19
drbd0             0.00     0.00  0.00 3465.76     0.00 28721.40     
16.57    11.06    3.45   0.23  79.18
drbd0             0.00     0.00  0.00 3433.27     0.00 39198.44     
22.83     3.31    1.14   0.13  44.75
drbd0             0.00     0.00  0.00 4003.50     0.00 47852.92     
23.91     9.46    2.39   0.07  27.43
drbd0             0.00     0.00  0.00 4438.28     0.00 37784.38     
17.03    14.62    3.56   0.19  82.62
drbd0             0.00     0.00  0.00 3758.57     0.00 35619.92     
18.95     5.58    1.75   0.19  71.91

Cheers,
-- 
Casey Allen Shobe
Database Architect, The Berkeley Electronic Press
cshobe at bepress.com (email/jabber/aim/msn)
http://www.bepress.com | +1 (510) 665-1200 x163



More information about the drbd-user mailing list