Local LLM Challenge | Speed vs Efficiency

Post Content

​ I put three systems to the local LLM test.

Gear Links
* K9 Mini with 32GB RAM: https://amzn.to/3ZiKjcp
* 96GB RAM kit: https://amzn.to/3ZhQ4qR
* Mac Mini M2 Pro: https://amzn.to/4fbgmzY
* RTX 4090: https://amzn.to/3YvvHpg
* Mini PC with Oculink: https://amzn.to/3UgLNAK
* My gear: https://www.amazon.com/shop/alexziskind

Related Videos
* Cheap mini runs a 70B LLM – https://youtu.be/xyKEQjUzfAk
* It’s over…my new LLM Rig – https://youtu.be/IXixbu7Kkd8
* RAM torture test on Mac – https://youtu.be/l3zIwPgan7M
* FREE Local LLMs on Apple Silicon | FAST! – https://youtu.be/bp2eev21Qfo
* Set up Conda – https://youtu.be/2Acht_5_HTo
* INSANE Machine Learning on Neural Engine – https://youtu.be/Y2FOUg_jo7k

* Developer productivity Playlist – https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX
AI for Coding Playlist: – https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw

— — — — — — — — —

SUBSCRIBE TO MY YOUTUBE CHANNEL
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

— — — — — — — — —

Join this channel to get access to perks:
https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join

— — — — — — — — —

ALEX ON X: https://twitter.com/digitalix

#machinelearning #llm #softwaredevelopment   Read More Alex Ziskind