I have avoided AI (LLMS) because it didn’t seem nearly useful enough to warrant supporting these tech assholes. Is there anyway to use it locally on a normal device (16 Gb ram, 8 year old Geforce graphics, etc?). Can I just slow it down and get it to work locally?
I’ll just leave this here. https://sleepingrobots.com/dreams/stop-using-ollama/
So… I’m with you/them about the vendor lock in but I think they’re slightly underestimating what ollama brings to the table. For example, ollama bundles CUDA libraries so I don’t need to worry about that as I wrangle with my GPU. It’s also well supported by other ecosystems like home assistant in a way that raw openai compatible API server isn’t.
Overall I don’t know if I find these arguments compelling at this time.
Reading through it I see some good points but… What’s the alternative? Isn’t Ollama still the least of evils (by far)?
Just use llama-server. I quickly found that all wrappers just make life harder and they made some great progress to improve usability with stuff like router-mode