[tech] [Conversation Starter]: GPU server

Sat Oct 26 11:36:45 AWST 2019

More relevant stuff from discord. I think this direction may be more promising. It may seem a bit odd to just be copy-pasting from discord, but I think it's good to keep some record of the discussion all in one place.______________________________________________________________________________
tecYesterday at 21:03Anyone got any thoughts on the GPU server email? I gotta admit just one response is a bit disapointing. ______________________________________________________________________________
coxyYesterday at 21:10here's my thoughts, please don't take this as some kind of NO 1. It is, in the whole, a good idea 2. Not sure motsugo can take a card, heathred can (it was specifically built to) 3. Would be nice to see some more member demand for GPGPU before buying anything? 4. As a corollary, anyone who would want it would already have a card in their PC better than one UCC would purchase(edited) hmm where's my markdown numbers ______________________________________________________________________________
tecYesterday at 21:11Thanks for that @coxy. In response to some of your points it may be helpful to explain why I am personally interested in this 1. I don't want my gpu to be stuck on ML training for hours on end 2. My distro has poor cuda stuff (haven't been able to get it to work yet), so I currently can't use my card 2.1 The 20XX series has much better low-precision compute (orders of mangitude) ______________________________________________________________________________
benYesterday at 21:14Is there something that could farm out the gaming GPU's when they aren't in use? ______________________________________________________________________________
tecYesterday at 21:14Hmmm. That does sound like an ideal solution ______________________________________________________________________________
coxyYesterday at 21:14I don't think any of the windows machines have anything that spec either ______________________________________________________________________________
benYesterday at 21:15Buying new generation GPU's for gaming machines is usually a pretty easy sell. And they're probably idle 12 hours a day. ______________________________________________________________________________
tecYesterday at 21:21the only thing that comes to mind to me is a drastic shift in the computing model in UCC to one where there are no true desktops, we just have servers with desktop-performance-grade VMs, and moniters. When a desktop session is started some resources are monopolised, however unused 'desktops' can simply have their VMs halted and their resources freed to a pool. I imagine that is a nice idea, but not feasable. ______________________________________________________________________________
benYesterday at 21:23VM's seem to have pretty inescapable overheads. CPU access gets mushy when you need responsiveness and disk I/O is always pretty bad compared to raw hardware. ______________________________________________________________________________
tecYesterday at 21:24Hmm. I'll put it down as a 'nice idea' then. ______________________________________________________________________________
FireGreyYesterday at 21:24Just use KVM ______________________________________________________________________________
tecYesterday at 21:24Interestingly we now do have things like https://azure.microsoft.com/en-au/services/virtual-desktop/ and stadia Windows Virtual Desktop | Microsoft AzureDeliver a virtualised modern desktop or apps to any device. Windows Virtual Desktop combines the scale, security and cost benefits of Azure and Microsoft 365.I wonder how bad VM Host → Windows → RDP would be could it be worth a test for fun? ______________________________________________________________________________
BonkersYesterday at 21:29I vote kvm ______________________________________________________________________________
spookydonut🐘Yesterday at 21:29from experience it is not great kvm is great qemu-kvm ______________________________________________________________________________
BonkersYesterday at 21:29based and red pilled ______________________________________________________________________________
spookydonut🐘Yesterday at 21:29paravirt ______________________________________________________________________________
FireGreyYesterday at 21:29I prefer cemu to qemu ______________________________________________________________________________
tecYesterday at 21:29So bad for windows, ok for linux? ______________________________________________________________________________
spookydonut🐘Yesterday at 21:30desktop windows editions just assume they are running bare metal and in general are not pleasant to use ______________________________________________________________________________
FireGreyYesterday at 21:30windows in general are not pleasant to use ______________________________________________________________________________
tecYesterday at 21:30 at FireGrey true I think after exams I may just do some testing to see what the latency/performance difference si *is ______________________________________________________________________________
Dread PirateYesterday at 21:32If you did IOMMU passthrough for GPU to VM performance might be ok but the system won't expect it's PCI display device to suddenly disappear when you deem it 'not in use' and want to do some hash cracking ______________________________________________________________________________
FireGreyYesterday at 21:32Why can't all OSs be like macOS, the XNU microkernel approach is really the only way to go moving forward ______________________________________________________________________________
tecYesterday at 21:32I'd be supprised if it worked well … but if it does, the more I think about it the better the idea seems ______________________________________________________________________________
Dread PirateYesterday at 21:33You could absolutely have a cluster of GPUs 1:1 paired to VMs ______________________________________________________________________________
tecYesterday at 21:33That's what I was thinking ______________________________________________________________________________
Dread PirateYesterday at 21:33but yeah, not sure how useable it'd be ______________________________________________________________________________
FireGreyYesterday at 21:33or mine bitcoin ______________________________________________________________________________
Dread PirateYesterday at 21:33That's a good idea Tell the uni you're testing clustered VMs with GPUs and then just use them to mine bitcoins ______________________________________________________________________________
spookydonut🐘Yesterday at 21:34if you want to gpu hash dont use vms ______________________________________________________________________________
coxyYesterday at 21:35does it really make a difference? ______________________________________________________________________________
tecYesterday at 21:35It you want to mine bitcoin use FGPAs *cough plebs ______________________________________________________________________________
coxyYesterday at 21:35would have though you're just pushing a few instructions to the GPUs and leaving them ______________________________________________________________________________
spookydonut🐘Yesterday at 21:35the key to mining is lowest running cost ______________________________________________________________________________
Dread PirateYesterday at 21:35I wonder how comparable IOMMU cuda hashing would be vs bare metal ______________________________________________________________________________
FireGreyYesterday at 21:35Sorry I meant <insert current crypto>coin ______________________________________________________________________________
spookydonut🐘Yesterday at 21:35vms means overhead meaning more poweruse ______________________________________________________________________________
coxyYesterday at 21:36well AU is fucked for power so a few % ain't gonna matter ______________________________________________________________________________
Dread PirateYesterday at 21:36I don't think that's really a concern in this scenario but otherwise yes ______________________________________________________________________________
tecYesterday at 21:36… back to the use case of Desktop sessions + ML ______________________________________________________________________________
Dread PirateYesterday at 21:36Yeah EC2 is like twice the price of US here ______________________________________________________________________________
FireGreyYesterday at 21:36You could offer remote sessions to people across campus, depending on how good the network is :o) ______________________________________________________________________________
Dread PirateYesterday at 21:36^ ______________________________________________________________________________
tecYesterday at 21:38To me the main benefit is better utilisation of storage/compute resources ______________________________________________________________________________
FireGreyYesterday at 21:39GPUs are going to be the main constraint for gaming and there's not much you can do to improve their utilisation across multiple devices ______________________________________________________________________________
tecYesterday at 21:39I think gpu passthgrough for VMs would be the only real way to go ______________________________________________________________________________
coxyYesterday at 21:40if you actually care more about running some load, as opposed to improving the capabilities of UCC shell servers, I can grant you a few hundred hours on a Azure GPU https://azure.microsoft.com/en-au/pricing/details/virtual-machines/linux/ Pricing – Linux Virtual Machines | Microsoft AzureAzure offers many pricing options for Linux Virtual Machines. Choose from many different licensing categories to get started.______________________________________________________________________________
Dread PirateYesterday at 21:41where is the asic farm ༼ つ ◕◕ ༽つ༼ つ ◕◕ ༽つ ______________________________________________________________________________
tecYesterday at 21:42That's very kind of you @coxy. There is an initial use case that prompted me thinking about all this (GPT-2), however at this stage I'm guenuinely interested in whether this is possible, because it seems like it could be a good idea ______________________________________________________________________________
coxyYesterday at 21:42yes, I agree ______________________________________________________________________________
tecYesterday at 21:45Hmm. This looks promising https://www.reddit.com/r/pcmasterrace/comments/3lno0t/gpu_passthrough_revisited_an_updated_guide_on_how/ redditGPU Passthrough Revisited - An Updated Guide On How To Game In A V...An easier to read version of this tutorial can be found [here](http://bufferoverflow.io/?p=1) New [Proof](http://www.3dmark.com/3dm/8633319?) &... ______________________________________________________________________________
mshBOTYesterday at 22:13my experience is that gpgpu is less stable than normal computing, so don't put it in an important server but a beefy gpu for people to play with sounds like a good idea

On Thursday, October 24, 2019 18:49 AWST, "tec" <tec at ucc.gu.uwa.edu.au> wrote:
  That does sound better. I just assumed that the obvious way to do thing was acess via VMs.
Does motsugo have any x16/x8 expansion slots?

On Thursday, October 24, 2019 15:47 AWST, Lyndon White <oxinabox at ucc.asn.au> wrote:
 Why would you put it on VM?Why not a the general user server (motsugo)It's designed for compute,It has your home directory on a local disk and a ton of RAM. Putting it on a VM is just making more work. For no reason.You have to ensure everything passes through without overhead. [*OX] On Oct 24, 2019 7:19 AM, tec <tec at ucc.gu.uwa.edu.au> wrote:Hi All,

With the general rise in GPU-accelerated compute tasks, and particularly ML I think it could be a good idea to be able to have VMs with access to GPU resources.

Give me your thoughts!

Also, here's some relevant copy-pasta from discord:______________________________________________________________________________
 tec28/09/2019  I'm looking to try to install a bunch of stuff to try out GPT2, and hopefully not break anything. Have we got any GPUs I can access via a VM?   I would install on my desktop, but my distro doesn't have the nvidia cuda stuff packaged ______________________________________________________________________________
 coxy28/09/2019  loveday has some shitty nvidia card   bah, can't get the dvb tuner working on windows10 ______________________________________________________________________________
 tec28/09/2019   Just wanna check gpu ______________________________________________________________________________
 coxy28/09/2019  think you need to ssh in as root, check uccpass   when we get the new proxmox hosts we should put something in heathred ______________________________________________________________________________
 NickBOT28/09/2019  @tec @DylanH[333]👌: If you're up for it, that would be pretty handy and I'm sure there would be club funding available for that sort of thing (and/or donations?). "HowToGPGPU at UCC" ______________________________________________________________________________
 tec28/09/2019  I don't know how yet, but I'd like to ______________________________________________________________________________
 DylanH[333]👌28/09/2019  I'd attend a session on something like that, but I'   m not knowledgeable enough to run one   I could potentially help with GPU passthrough in ProxMox though   I've done PCIe passthrough before for a NIC ______________________________________________________________________________
 NickBOT28/09/2019  @DylanH[333]👌: Yep, step one is expanding the cluster and hardware installation - happy to help experiment ______________________________________________________________________________
 DylanH[333]👌28/09/2019  Yeah, I need to get back onto that. I still haven't taken the time to ask committee about ordering SSDs   I'm now of the mind: didge the SD cards and reinstall Proxmox on 500GB+ SSDs, and dedicate an LVM volume for Ceph   I just haven't had the time/motivation ______________________________________________________________________________
 NickBOT28/09/2019  As for the OpenCL? software setup, @tec will no doubt help? ::-) ______________________________________________________________________________
 DylanH[333]👌28/09/2019  Also, we'd have to use one of our existing hosts in the cluster for a GPU, unless we have one lying around that's no thicker than one slot ______________________________________________________________________________
 NickBOT28/09/2019  Sounds good to me, one big drive per machine now that we have >3 hosts? though the SD boot is neat, too. ______________________________________________________________________________
 DylanH[333]👌28/09/2019  The 1RU HPs have a full-sized PCIe slot, but it's pretty thin ______________________________________________________________________________
 tec28/09/2019  PCIe extension ribbon? ______________________________________________________________________________
 DylanH[333]👌28/09/2019  Still doesn't solve where to actually fit the GPU ______________________________________________________________________________
 NickBOT28/09/2019  Loveday should be good for physical space. ______________________________________________________________________________
 DylanH[333]👌28/09/2019  @tec This is the sort of clearance the HPs have to work with ______________________________________________________________________________
 TRS-8028/09/2019  also how do you get power to the gpu? ______________________________________________________________________________
 DylanH[333]👌28/09/2019  PCIe slot - so it'd have to be 75W or less iirc   I suppose most GPUs that fit that power limit will probably fit that size limit as well 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ucc.gu.uwa.edu.au/pipermail/tech/attachments/20191026/220a2a7f/attachment-0001.htm 