100 Tokens per Second from a Lunchbox

Estimated read time 2 min read

Post Content

โ€‹ย I turn a mini PC and a GPU dock nto a 100-tokens/sec LLM rig and stress-test its limits.
๐Ÿ‘‰ Try TryHackMe โ€” many rooms are free, and use code ALEX25 for 25% off the annual plan: https://tryhackme.com/

๐Ÿ›’ Gear Links ๐Ÿ›’
๐Ÿ Beelink Mini PC: https://amzn.to/4nXrIfh
๐Ÿ Beelink GPU Dock: https://amzn.to/42fxSyX
๐Ÿช›๐Ÿช›Highly rated precision driver kit: https://amzn.to/4fkMVfg
โŒจ๏ธ๐Ÿข Keyboard for office (heavy): https://www.keychron.com/products/keychron-q1-max-qmk-via-wireless-custom-mechanical-keyboard?ref=azisk
โŒจ๏ธโ˜• Keyboard for cafe (light): https://www.keychron.com/products/keychron-k7-max-qmk-via-wireless-custom-mechanical-keyboard?ref=azisk
๐Ÿ’ปโ˜• Favorite 15″ display with magnet: https://amzn.to/3zD1DhQ
๐ŸŽงโšก Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
๐Ÿ› ๏ธ๐Ÿš€ My nvme ssd: https://amzn.to/3YLEySo
๐Ÿ“ฆ๐ŸŽฎ My gear: https://www.amazon.com/shop/alexziskind

๐ŸŽฅ Related Videos ๐ŸŽฅ

* ๐Ÿ› ๏ธ Mini PC portable setup –
https://youtu.be/4RYmsrarOSw
* ๐Ÿ› ๏ธ Dev setup on Mac –
https://youtu.be/KiKUN4i1SeU
* ๐ŸŒ— RAM torture test on Mac – https://youtu.be/l3zIwPgan7M
* ๐Ÿ› ๏ธ FREE Local LLMs on Apple Silicon | FAST! – https://youtu.be/bp2eev21Qfo
* ๐ŸŒ— REALITY vs Appleโ€™s Memory Claims | vs RTX4090m – https://youtu.be/fdvzQAWXU7A
* ๐Ÿ› ๏ธ Set up Conda – https://youtu.be/2Acht_5_HTo
* ๐Ÿค– INSANE Machine Learning on Neural Engine – https://youtu.be/Y2FOUg_jo7k

* ๐Ÿ› ๏ธ Developer productivity Playlist – https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX
๐Ÿ”— AI for Coding Playlist: ๐Ÿ“š – https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

โค๏ธ SUBSCRIBE TO MY YOUTUBE CHANNEL ๐Ÿ“บ
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

Join this channel to get access to perks:
https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

๐Ÿ“ฑ ALEX ON X: https://twitter.com/digitalix

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

โฑ๏ธ Chapters
0:00 First impressions
1:48 Power, scores, Speedometer
3:06 Python Mandelbrot test
4:04 Sponsor: TryHackMe demo
5:29 .NET mega build test
6:36 Switching to LLMs
7:53 Qwen 34B: GPU vs CPU
8:32 70B loads, very slow
13:09 RTX 5060 dock: big boost

#minipcs #llm #beelinkย ย ย Read Moreย Alex Ziskindย 

You May Also Like

More From Author