Victor C.
dehnhaide
·
AI & ML interests
None yet
Recent Activity
reacted
to
csabakecskemeti's
post
with 🔥
about 12 hours ago
Just sharing a result of a homelab infrastructure experiment:
I've managed to setup a distributed inference infra at home using a DGX Spark (128GB unified gddr6) and a linux workstation with an RTX 6000 Pro (96GB gddr7) connected via 100Gbps RoCEv2. The model I've used (https://lnkd.in/gx6J7YuB) is about 140GB so could not fit either of the GPU. Full setup and tutorial soon on devquasar.com
Screen recording:
https://lnkd.in/gKM9H5GJ
new activity
about 14 hours ago
mratsim/MiniMax-M2.1-FP8-INT4-AWQ:Looking forward to trying this!
new activity
4 days ago
mratsim/GLM-4.7-EXL3:Excellent work (2.57bpw-tuned) ... and a small kind request
Organizations
None yet