Appreciation post: Open source models over corporate ones every time

I keep seeing people rave about the latest GPT or Claude update like it's the only game in town. But I've been running Mistral and Llama locally for the last 4 months on my home server and I honestly prefer them. I spent $0 on API fees after I got the hardware set up, and I can tweak the temperature and context window however I want. No worrying about my data being scraped or a model getting shut down because of a policy change. The tradeoff is you need a decent GPU and a little patience to get things installed right. But once it's running, I get better results for my specific use case than I ever did paying for cloud access. Has anyone else moved to local models and found they work better for niche tasks?

2 comments

2 Comments

ivancoleman4d ago

I was fully on the "just use ChatGPT for everything" train until I tried running Llama 3 locally a couple months ago and it honestly changed how I think about this whole space.

barbara4294d ago

Oh man, that's exactly what I've been noticing everywhere lately. It's like when everyone jumped on the meal kit delivery bandwagon a few years ago then slowly realized cooking from scratch with stuff from the farmers market just felt better somehow. There's something about doing it yourself that changes the whole experience, even if it's not as polished or fast. Makes you appreciate the process more and actually understand what's going on under the hood. Glad I'm not the only one who's been thinking this lately.