Some Tips about running a Dell PowerEdge Tower Server as your workstation
Some use workstations as servers.
I'm using servers as workstations.
Over the years, I've changed computing gear on quite a few occasions. I've been using Tower Servers for the past 5 years and would like to share some tips to help others.
- But why would anyone want to do that??
- Servers are well integrated systems and are usually seriously designed and tested.
- They offer greater expandability (6x3.5" hotswap bays in my previous T410, now 8x3.5" in my T430).
- They usually include some kind of Remote Access Card (RAC) which is great for remote'ing in when all else has failed.
- I can get tons of server equipment on ebay that will be compatible with that system.
- Where else can I get 192Gb of ECC DDR4 RDIMM, dual 6-core Xeons and 8 hotswap bays?
- Tip #1 : Choose your chassis with care.
- Rack servers are usually thin and noisy (those 8k rpm fans have the job of cool that 2U enclosure). It is not uncommon for them to be in the 60-70dBA range.
- Tower servers are much bigger and less noisy. The are also more expensive -but- you get an electricity bill that's lower than a comparable Rack server so the price difference will shrink after a few months. And having a server that makes less noise and draws less power is more environment-friendly!
- Most pre-2011 tower servers from Dell and HP (before Dell 11th Gen and before HP's Gen8) are less quiet than their modern counter-parts.
In 2016, I'd recommend getting a 12th or 13th Gen from Dell.. If you are into HP Gear, get a Gen8 or a Gen9. I've never done Lenovo or Cisco gear, so I can't help here.
- Most modern towers from Dell feature a single 120mm PWM fan to cool the entire chassis. That's the T410, T420 or T430. I assume the T310, T320 and T320 are similar since they feature the same chassis.
- The environmental ratings for current and past servers can usually be obtained from the manufacturer. Check the specs carefully. I found the spec for most recent Dell Tower servers here:
Dell-13G PowerEdge Acoustical Performance and Dependencies
- Tip #2 : Choose your components carefully.
Now that you've selected the system, let's pick the components.- CPUs
- Most recent tower servers feature PWM (4-pin) fans that are controlled by the iDrac/iLo controller. The sensors on these systems feed the former with information which they use to drive the speed of the fans.- Consequently, even if you want enough Xeon cores, you probably don't want one of their 145W 12-core monsters. Such a chip (or a pair of them) will increase thermal response under load in your system which will result in increased fan speed. On the other hand, lower Wattage Xeons usually have a low core frequency that might make the user experience in interactive sessions oh-not-so-great.
I usually pick Xeons in the 65W-85W range. These typically feature decent punch while keeping heat (and noise) tolerable.
Wikipedia has a great list of all Xeon processors with Wattage, Cores, etc.. here:
List of Intel Xeon Microprocessors
- Graphics!
The bundled graphics adapter in your server will not let you run much else than a 2D environment. This can be solved by adding a PCI-E GPU which will give you decent 3D performance.Forget about the latest Radeon or NVidia monster, it's not going to work at all.
When I tried my NVidia Quadro K2000 (a 65W card) in my Dell PowerEdge T130, the system simply refused to boot and told me that the card was drawing too much power to power on all components.
GPUs can usually work fine if they are in the 45W or below range. I've used with great success NVidia Quadro K620 and K600 cards in my Poweredge. The Passive Geforce cards from the previous gens (GT730, etc...) can also be used successfully.
Here's my Poweredge T130:
# lspci |grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GF108 [GeForce GT 620] (rev a1)That card was replaced by a GT 730:
# lspci |grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 730] (rev a1)
And here's the Poweredge T430:
# lspci |grep VGA
03:00.0 VGA compatible controller: NVIDIA Corporation GF108GL [Quadro 600] (rev a1)
- Sound
Servers don't have sound cards.. but I've used with much success USB audio adapters to get sound from Videos and Games on my Linux Servers/Workstations.These can usually be obtain for about USD10 on amazon or e-bay:
- Tip #3 : Use the right settings
Disable PCI-E 3rd Party thermal reponse (can also be done from the iDrac submenu of the BIOS GUI):
Here's a 13th Gen server. I hightlighted the most important fields.
/admin1-> racadm get System.ThermalSettings
[Key=System.Embedded.1#ThermalSettings.1]
#FanSpeedHighOffsetVal=75
#FanSpeedLowOffsetVal=15
#FanSpeedMaxOffsetVal=100
#FanSpeedMediumOffsetVal=45
FanSpeedOffset=Off
#MFSMaximumLimit=100
#MFSMinimumLimit=5
MinimumFanSpeed=255
ThermalProfile=Minimum Power
ThirdPartyPCIFanResponse=Disabled
To be continued...
#FanSpeedHighOffsetVal=75
#FanSpeedLowOffsetVal=15
#FanSpeedMaxOffsetVal=100
#FanSpeedMediumOffsetVal=45
FanSpeedOffset=Off
#MFSMaximumLimit=100
#MFSMinimumLimit=5
MinimumFanSpeed=255
ThermalProfile=Minimum Power
ThirdPartyPCIFanResponse=Disabled
Some of these can be modified by using the iDrac CLI:
/admin1-> racadm set System.ThermalSettings.FanSpeedOffset Off
/admin1-> racadm set System.ThermalSettings.ThirdPartyPCIFanResponse 0
[Key=System.Embedded.1#ThermalSettings.1]
Object value modified successfully
To be continued...
Hi,
ReplyDeleteI'm trying to add a graphics card to my T130. I bought the same model as you indicated: MSI NVidia GeForce GT 730. Unfortunately when I try to disable the built-in graphics card, the system does not boot and says "No boot device available".
What BIOS options have you configured to boot with the onboard graphics card disabled and the new one activated?
Thank you,
Miguel.
Hi Miguel,
DeleteI don't think I configured anything specifically. Here are my BIOS settings:
racadm>>get BIOS.IntegratedDevices
racadm get BIOS.IntegratedDevices
[Key=BIOS.Setup.1-1#IntegratedDevices]
#CurrentEmbVideoState=Disabled
EmbNic1Nic2=DisabledOs
EmbVideo=Disabled
InternalUsb=On
MmioAbove4Gb=Enabled
OsWatchdogTimer=Enabled
UsbPorts=AllOn
I cannot remember if I disabled EmbVideo manually but perhaps that's something you could try.
Regards,
Vincent
Fixed: I updated the BIOS to version 2.0.8
ReplyDeleteLife saver! I was pulling out my hair trying to get the fan to return to normal after installing a third party add-on card.
ReplyDelete"racadm set System.ThermalSettings.ThirdPartyPCIFanResponse 0" did the trick!
Installed a Mellanox ConnectX-3 card and the system fan got so loud I couldn't stand it.
ReplyDelete"racadm set System.ThermalSettings.ThirdPartyPCIFanResponse 0" worked for me too!
Thank you for restoring my sanity!
Hi,
DeleteGlad to hear it helped but please be aware of the following: Mellanox CX-3 are known to run very hot and overheat. Please make sure you used the sensors on the card to monitor its temperature. I recall having CX-3 cards run too hot because they were in a PCI-E slot that was getting a little less airflow.
When this happens, the card will shut itself down, drop off the PCI-E. The server will then reset and an event will be logged to the iDrac's log.
Good luck,
Vincent
I'm new to this. I was unable to find the ThirdPartyPCIEFanResponce setting in the Bios. Any help would be greatly appreciated.
DeleteYou need to SSH into the iDrac and run the commands there
DeleteI´m also new to it.
ReplyDeleteDid the steps you mentioned and it got more quiet. But still like a hover.
PWM on 31% and 1920 rpms and the system is idle :(
Perhaps it's the H710 controller that triggers the fan?!?
Great tips! You say you've used NVidia Quadro K600 in your PowerEdge with great success. I'm having a very bad time with the K600 on my T130... Even after updating to the last BIOS version (2.5.0), during bootup I keep getting the error
ReplyDeleteUEFI0067: A PCIe link training failure is observed in PCIe Slot 2 and the link is disabled
I set the thermal parameters just the same as yours, right before inserting the K600.
What could I be doing wrong?
Thank you very much
Any suggested shopping links to buy MSI NVidia GeForce GT 730 for my PowerEdge T130?.
ReplyDeleteWhen I find the specs at GeForce I found these:
1. GT 730 DDR3, 128-bit
2. GT 730 DDR3, 64-bit
3. GT 730 GDDR5.
https://www.geforce.com/hardware/desktop-gpus/geforce-gt-730/specifications
Which one is the best fit for PowerEdge T130?
Thank you very much in advance
Hi, First of all, I'd go for DDR5.
DeleteNow, I've replaced the GT730 with a GT1030 from MSI with Displayport and it's been working very fine in my T130:
https://www.newegg.ca/Product/Product.aspx?Item=N82E16814137139
Now need to go for a 730 now, the 1030 is now the same price than the 730 used to be.
I hope this helps,
Hi, we are an it company (Italy)
ReplyDeleteWe try your guide for insert an external graphic card in a Dell T130
We failed. Can you help us?
write me at info@micaweb.it
We pay for this work
Thank you for the guide
Hi, I sent you an e-mail a couple days ago but got no reply. Please post here with aditionnal details.
Deletethese commands dont work anymore on idrac9
ReplyDeleteThese noisy fans go quiet with LFF command set.
I spent hours until it succeded with just one command to Disable the LFFMode fro Auto to disabled.
Hi,
DeleteThis is what I had described in another article in early 2019 here:
https://vcojot.blogspot.com/2019/03/some-tips-about-poweredge-as.html
Thanks,
Vincent
you have to set the
ReplyDeleteSystem.PCIeSlotLFM.LFMMode parameter with set command to 1 which is disabled. By default its 0 which means Auto and fans go crazy when you install a video card.
I have a PowerEdge T410 that I am trying to put a GT 730 graphics card in. I disabled the onboard graphics card, but Server 2008R2 keeps telling me that I only have a standard VGA adaptor. Is there some special magic words that I have to say to get the GT 730 to work?
ReplyDeleteHi Danny, I had a T410 in which I used a GPU (might have been a GT610) and it was working fine (tested on Linux). Have you tested your GPU in another system to confirm that it works? Also, under Windows you might have to install some NVidia drivers.
Deleteyes, I pulled the card from a working system. I tried downloading the drivers and it said it could not locate the hardware so it didnt install the driver.
DeleteSomeone told me that I had to install this x8 card into the x16 slot in order for the card to work. Is this true?
DeleteMaybe. Check your T410 manual/docs. sometimes x8 slots are cabled as x4.
DeleteGreat post! I was able to slap in a Quatro T600 in a T430 with 750w PSUs. I had to disable the onboard graphics in order for the display to output through the card. After that, I ssh'd into the drac, ran the fan commands, and everything's working.
ReplyDelete