There are many open-source tools for hosting open weights LLMs locally for inference: from the command line (CLI) tools to full GUI desktop applications. Here I’ll outline some popular options and provide my own recommendations. I have split this post into the following sections:
https://docs.google.com/spreadsheets/d/1Xv38p90V3GiJXjq0a3qc24056Vicn1I5MG6QiFE6nVE/edit#gid=0
read more on https://medium.com/thedeephub/50-open-source-options-for-running-llms-locally-db1ec6f5a54f
