Local LLM Challenge | Speed vs Efficiency

Estimated read time 2 min read

Post Content

โ€‹ย I put three systems to the local LLM test.

๐Ÿ›’ Gear Links ๐Ÿ›’
* ๐Ÿ’ป๐Ÿ”„ K9 Mini with 32GB RAM: https://amzn.to/3ZiKjcp
* ๐Ÿ› ๏ธ๐Ÿš€ 96GB RAM kit: https://amzn.to/3ZhQ4qR
* ๐Ÿ๐Ÿ’ฅ Mac Mini M2 Pro: https://amzn.to/4fbgmzY
* ๐ŸŽงโšก RTX 4090: https://amzn.to/3YvvHpg
* ๐Ÿ› ๏ธ๐Ÿš€ Mini PC with Oculink: https://amzn.to/3UgLNAK
* ๐Ÿ“ฆ๐ŸŽฎ My gear: https://www.amazon.com/shop/alexziskind

๐ŸŽฅ Related Videos ๐ŸŽฅ
* ๐Ÿคฏ Cheap mini runs a 70B LLM – https://youtu.be/xyKEQjUzfAk
* ๐Ÿค– Itโ€™s overโ€ฆmy new LLM Rig – https://youtu.be/IXixbu7Kkd8
* ๐ŸŒ— RAM torture test on Mac – https://youtu.be/l3zIwPgan7M
* ๐Ÿ› ๏ธ FREE Local LLMs on Apple Silicon | FAST! – https://youtu.be/bp2eev21Qfo
* ๐Ÿ› ๏ธ Set up Conda – https://youtu.be/2Acht_5_HTo
* ๐Ÿค– INSANE Machine Learning on Neural Engine – https://youtu.be/Y2FOUg_jo7k

* ๐Ÿ› ๏ธ Developer productivity Playlist – https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX
๐Ÿ”— AI for Coding Playlist: ๐Ÿ“š – https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

โค๏ธ SUBSCRIBE TO MY YOUTUBE CHANNEL ๐Ÿ“บ
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

Join this channel to get access to perks:
https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

๐Ÿ“ฑ ALEX ON X: https://twitter.com/digitalix

#machinelearning #llm #softwaredevelopmentย ย ย Read Moreย Alex Ziskindย 

You May Also Like

More From Author