<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><title>RE: [DRBD-user] Dell Server Question</title><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>You are correct, it’s only 12+2 drives on the R515. I was confusing it with another server (non Dell).<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>You would think that 2.5” drives would be faster. Maybe it’s just the particular drives I have tested with. Have you done any benchmarks, especially random I/O? It was only 10K on the 2.5 drives vs. 15K on the 3.5 with much higher capacities. For my needs, I am mostly interested in random I/O than sequential.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Roof, Morey R. [mailto:MRoof@admin.nmt.edu] <br><b>Sent:</b> Friday, December 09, 2011 9:34 PM<br><b>To:</b> John Lauro; drbd-user@lists.linbit.com<br><b>Subject:</b> RE: [DRBD-user] Dell Server Question<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div id=idOWAReplyText44697><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif";color:black'>The R515 only does 12 drives, not 16 drives and the two internal mean a server shutdown for replacement which is slight annoying.</span><o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>One of the IBM servers (which also has an LSI card) holds 28 2.5" drives. It is the model x3630 M3 and it is another I have been looking at.</span><o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Very strange that you would say that 3.5 are faster than 2.5. With the higher densities of the 2.5 drives in the 10k range the larger bit densities per cm^2 means a much higher data transfer speed. Also, if you look at the seek times they are pretty much identical and a full drive sweep is a bit faster on the 2.5 than the 3.5 since the drive is smaller.</span><o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div></div><div><p class=MsoNormal><o:p> </o:p></p><div class=MsoNormal align=center style='text-align:center'><hr size=2 width="100%" align=center></div><p class=MsoNormal style='margin-bottom:12.0pt'><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> John Lauro [<a href="mailto:john.lauro@covenanteyes.com">mailto:john.lauro@covenanteyes.com</a>]<br><b>Sent:</b> Fri 12/9/2011 5:21 PM<br><b>To:</b> Roof, Morey R.; 'Kushnir, Michael (NIH/NLM/LHC) [C]'; <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br><b>Subject:</b> RE: [DRBD-user] Dell Server Question</span><o:p></o:p></p></div><div><p><span style='font-size:10.0pt'>Supermicro might be your best option, but for comparisons a more standard<br>model for a Dell is the R515 (or 510) instead of the C-series, with 16<br>3.5" drives and 2 2.5" drives all in 2U. Unless you count SSD, despite<br>the higher density (24 vs 16) I haven't found 2.5" drives that really<br>match the performance of 3.5" drives, although they are getting closer.<br>Much easier to find trays/drives for the R515 than the C-series.<br><br>PS: Often you can get Dell to match prices on systems if you give them a<br>quote to compete with. Given the shortage of hard drives, it might be<br>harder than normal right now...<br><br><br>My biggest annoyance with Dell (and LSI as they make the controllers), is<br>they don't properly do Raid 1/10 despite their configuration info implying<br>they do. They really only read from 1 drive, so performance-wise you are<br>better off not utilizing the hardware RAID and making a md raid 1/10 on<br>top of the individual drives. That said, their RAID 5 performance is<br>excellent.<br><br><br><br>> -----Original Message-----<br>> From: <a href="mailto:drbd-user-bounces@lists.linbit.com">drbd-user-bounces@lists.linbit.com</a> [<a href="mailto:drbd-user-">mailto:drbd-user-</a><br>> <a href="mailto:bounces@lists.linbit.com">bounces@lists.linbit.com</a>] On Behalf Of Roof, Morey R.<br>> Sent: Friday, December 09, 2011 2:54 PM<br>> To: Kushnir, Michael (NIH/NLM/LHC) [C]; <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>> Subject: Re: [DRBD-user] Dell Server Question<br>><br>> Hi Michael,<br>><br>> Thanks a bunch for the information. I was looking at the Supermicro<br>> boxes (used quite a few of them) and was a bit curious about the Dell<br>> offering. I'm glad to know I shouldn't use them and I believe I will<br>> just go with the Supermicro setup as I was planning.<br>><br>> One big issue I wanted to know was the backplane for the HDDs. Having<br>> it split like that it is really stupid. With the supermicro chassis<br>> they can be ordered with dual expander chips so cards like the Megaraid<br>> can load balance the IO to both ports on the HDDs.<br>><br>> Pretty sad that Dell would act like that but it is their loss.<br>><br>> Thanks again,<br>> Morey<br>><br>> -----Original Message-----<br>> From: Kushnir, Michael (NIH/NLM/LHC) [C]<br>> [<a href="mailto:michael.kushnir@nih.gov">mailto:michael.kushnir@nih.gov</a>]<br>> Sent: Friday, December 09, 2011 11:53 AM<br>> To: Roof, Morey R.; <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>> Subject: RE: Dell Server Question<br>><br>> Hi Morey,<br>><br>> I am using two C2100 servers with 2.5" inch chassis with DRBD. To save<br>> you the long read, my quick and short recommendation is: AVOID THEM LIKE<br>> THE PLAGUE!<br>><br>> For the money, get<br>> <a href="http://www.supermicro.com/products/chassis/4U/417/SC417E16-R1400U.cfm">http://www.supermicro.com/products/chassis/4U/417/SC417E16-R1400U.cfm</a><br>> (72-bay - 4U) or the<br>> <a href="http://www.supermicro.com/products/chassis/2U/216/SC216E16-R1200U.cfm">http://www.supermicro.com/products/chassis/2U/216/SC216E16-R1200U.cfm</a><br>> (24 bay - 2U) instead.<br>><br>> If you want more detailed info, read on! :)<br>><br>> My servers came with the LSI 9260-8i RAID cards, single E5620 CPU, 12GB<br>> RAM, and the Intel based "mezzanine" dual-port 10GbE card. The backplane<br>> is split with 12 slots connected to one port on the RAID card, and 12<br>> slots on the other.<br>><br>> CPU and RAM:<br>> I've found the E5620 is more than capable of handling the load from DRBD<br>> and IET (software iscsi target). 12GB RAM is also ample for my needs.<br>><br>> Network:<br>> I've found the Intel 10GbE card to be a good performer. I am using it<br>> with Finisar SFP+ modules and have it connected to a Cisco 4900 switch<br>> with Cisco optics on the switch end. I have the two ports in an LACP<br>> (mode 4) bond. I am getting ping latencies of about 0.23-0.3ms. I'm<br>> happy with that performance.<br>><br>> RAID:<br>> I have my OS (RHEL 5) and swap on a pair of 500GB SATA2 disks in RAID1,<br>> and my DRBD on a RAID10 made up of 10 Constellation.2 1TB disks.<br>> Metadata is set to internal. The max replication rate (for the initial<br>> sync) was ~3.2Gbits/s (measured by iftop).<br>><br>><br>> Issues:<br>><br>> 1.<br>><br>> This server is not actually made by Dell. It is completely outsourced to<br>> another manufacturer. I was not able to get any useful support from Dell<br>> with any issues that I had. Their support people simply don't know crap<br>> about this server, how to use it, or how to manage it. Considering that<br>> I'm with the Federal government, and go through a dedicated support<br>> team, that should say something. Dell was not even able to tell me what<br>> software to use to manage the RAID array and failure notifications.<br>><br>> Dell kept telling me that the LSI command line utility is the only way<br>> to go, and that the server, "is intended for hyper scale environments<br>> where the expectation is that a failed server would just fail and be<br>> replaced by another, and where administrators should know how to manage<br>> such an environment."<br>><br>> Thankfully, LSI support people pointed me to their GUI which unlocks<br>> great features like SNMP and SMTP notifications, SSD caching, and SSD<br>> protector for SSD RAID.<br>><br>> 2.<br>><br>> Because the hardware is not actually "made" by Dell, this server is not<br>> compatible with Open Manage and you can't update your drivers firmware<br>> from Dell's yum repositories firmware.<br>><br>> 3.<br>><br>> Drive trays--- wow was this a surprise for me!!! Dell has a new policy<br>> where they do not sell drive trays without hard drives in them. As you<br>> might know, Dell's drives are highly overpriced. They will also not sell<br>> you a server fully populated with trays unless you get it fully<br>> populated with drives. We bought over $100k of Dell equipment this year,<br>> and they would not make an exception and give us the drive trays.<br>><br>> So, finally, we relented, and we bought 16 (8 for each server) Crucial<br>> M4 512GB SSDs from Dell for about $13k. Once we bought the drives, we<br>> asked for trays. Dell told us to go... The policy is that trays are only<br>> supplied with Dell BRAND drives, not any drives purchased from Dell.<br>><br>> We relented again, and asked Dell to sell us their cheapest hard drives<br>> just to get trays. Dell told us that they cannot, because they are out<br>> of hard drives due to flooding in Thailand. And they cannot just charge<br>> us for drives and ship only the trays. GRR!!! Bastards!!!<br>><br>> I've tried finding the trays online. It took me months to find some at<br>> codemicro.com. Then they suddenly sold out of all 400+ units. So, the<br>> last batch, I had to buy on eBay. I thank God that Ivan from New Jersey<br>> was able to find some to put them on eBay.<br>><br>><br>> So, my recommendation is: STAY AWAY! Get the SuperMicro chassis (it<br>> comes with all 72 drive trays and plenty of expansion capacity. Get ASUS<br>> server motherboards (I use them in my private lab), Myri 10GbE NICs (I<br>> use them in other production servers), and LSI 9260-8i (or better) RAID<br>> cards. You will be in a good place in terms of hardware and money spent.<br>><br>><br>><br>> Best,<br>> Michael Kushnir<br>><br>> Lead VMware and UNIX Systems Administrator<br>> Communications Engineering Branch,<br>> Lister Hill National Center for Biomedical Communications,<br>> National Library of Medicine,<br>> National Institutes of Health<br>><br>> -----Original Message-----<br>> From: Roof, Morey R. [<a href="mailto:MRoof@admin.nmt.edu">mailto:MRoof@admin.nmt.edu</a>]<br>> Sent: Thursday, December 08, 2011 1:30 PM<br>> To: <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>> Subject: [DRBD-user] Dell Server Question<br>><br>> Hi Everyone,<br>><br>> Anyone out there using a Dell Poweredge C2100 with the 24 2.5" HDD<br>> backplane? I'm curious to know a little about the backplane, mainly if<br>> it is a dual port backplane where the RAID controller can use both ports<br>> on the SAS drives to load balance the IO operations. Dell's website<br>> isn't very specific on this point. Also, some information about how the<br>> server seems work with DRBD would be very helpful to me.<br>><br>> Thanks,<br>> Morey<br>> _______________________________________________<br>> drbd-user mailing list<br>> <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>> <a href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a><br>> _______________________________________________<br>> drbd-user mailing list<br>> <a href="mailto:drbd-user@lists.linbit.com">drbd-user@lists.linbit.com</a><br>> <a href="http://lists.linbit.com/mailman/listinfo/drbd-user">http://lists.linbit.com/mailman/listinfo/drbd-user</a></span><o:p></o:p></p></div></div></div></body></html>