Your local LLM is 10x slower than it should be

Estimated read time 2 min read

Post Content

​ Here’s the one change that took mine from ~120 tok/s to 1,200+ without a new GPU.
TryHackMe just launched Cyber Security 101 (SEC1) — and for a limited time you can get 40% off with my link: https://tryhackme.com/alexsec1
Use code ALEXSEC1 for 40% off the exam fee + 3 months of Premium access.

🛒 Gear Links 🛒

🪛🪛Highly rated precision driver kit: https://amzn.to/4fkMVfg
💻☕ Favorite 15″ display with magnet: https://amzn.to/3zD1DhQ
🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo
📦🎮 My gear: https://www.amazon.com/shop/alexziskind

🎥 Related Videos 🎥

🧳🧰 Mini PC portable setup – https://youtu.be/4RYmsrarOSw
🍎💻 Dev setup on Mac – https://youtu.be/KiKUN4i1SeU
💸🧠 Cheap mini runs a 70B LLM 🤯 – https://youtu.be/xyKEQjUzfAk
🧪🔥 RAM torture test on Mac – https://youtu.be/l3zIwPgan7M
🍏⚡ FREE Local LLMs on Apple Silicon | FAST! – https://youtu.be/bp2eev21Qfo
🧠📉 REALITY vs Apple’s Memory Claims | vs RTX4090m – https://youtu.be/fdvzQAWXU7A
🧬🐍 Set up Conda – https://youtu.be/2Acht_5_HTo
⚡💥 Thunderbolt 5 BREAKS Apple’s Upcharge – https://youtu.be/nHqrvxcRc7o
🧠🚀 INSANE Machine Learning on Neural Engine – https://youtu.be/Y2FOUg_jo7k
🧱🖥️ Mac Mini Cluster – https://youtu.be/GBR6pHZ68Ho

* 🛠️ Developer productivity Playlist – https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX
🔗 AI for Coding Playlist: 📚 – https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw

— — — — — — — — —

❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

— — — — — — — — —

Join this channel to get access to perks:
https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join

— — — — — — — — —
📱 ALEX on X: https://x.com/digitalix

#macstudio #gdxspark #llamacpp   Read More Alex Ziskind 

You May Also Like

More From Author