without knowing what you are running and your hardware, it’s hard to say. The symptoms sound similar to when I was running AI on a small server with 32gb ram only, and I hot oom, I’d be able to ping it but nothing else would work and would require a hard reset.
Matrix 👉 @danny:radiantfig.fyi
- 0 Posts
- 7 Comments
fozid@lem.radiantfig.fyito
LocalLLaMA@sh.itjust.works•I have a RTX A1000 (8GB) with 64GB DRR5, what would you recommend for a newbie?English
4·20 days agoAsking an ai to do something for you is essentially agentic usage. All an llm can do is generate text or images or audio or video. The key part is generate. They can’t do anything but generate. As soon as you want an llm to do something, you need to give it a tool to do that thing with. It then needs to be capable of working out how to effectively carry out that task.
The most basic llm will chat to you based on the knowledge it was trained with. Generally, the bigger the llm the more knowledge it has. This is a basic chat bot, the bigger it is the more knowledge it has.
After this came reasoning and thinking, where the chat bots got smarter. They still had the same amount of knowledge, but they were able to reason, work stuff out, think about things, before answering your question and prompt.
Then to do stuff, like create docs, search the web, carry out an action, the llm needed to be given tool access and actually be able to do tool calls to use those tools. Anything like create, edit and delete a file, view a folders contents, create a diary entry, search the internet, all required tools. In fact anything except for basic chat response requires a tool. This is where MCP servers and agentic harnesses come in. But you also need an llm capable of tool calling. Giving an llm access to a coding sandbox enables it to be able to calculate answers for you.
As an example say you want to get an llm to create a diary event for 30 days before Xmas to buy a present for a friend. The llm needs to first work out what day today is, work when Xmas is, then work out what day 30 days before Xmas is, llm’s are terrible at maths, but with a coding sandbox, it can create a basic calculation to work out the date to create the diary event for. It would then call a tool to create the diary event on that day.
Adding voice recognition is another model that takes your voice commands, converts it to text, then passes it to the next llm that does stuff.
Ai isn’t magic. You have to build and create the stack that does what you want. But ai can help you do that.
fozid@lem.radiantfig.fyito
LocalLLaMA@sh.itjust.works•I have a RTX A1000 (8GB) with 64GB DRR5, what would you recommend for a newbie?English
5·20 days agoOk for image generation, I recommend flux 2 klein 4b and run it with stable diffusion which provides both a cli and a web UI. You can also run it through comfyui instead, but after trying it out I wasn’t a massive fan.
For the rest I’m confused, you said no to agentic and coding, but then mentioned task logging and summarising documents, which depending on the tasks and documents, can benefit from agentic and coding? You won’t be running a sota model, so the llm won’t know everything. It will need to be honest about what it does and doesn’t know, and be able to research using the internet, then if you want it to produce a document in a folder summarising or reporting for you, this is all agentic stuff, you need to setup MCP tooling.
Genuinely if you want a good general chat bot that can use the internet, if go with lfm2.5 8b a1b on a q4 k m as it will be really fast and good quality. If you want something stronger with the agentic tool calling then orinth 1.5 35b a3b q4 k m.
fozid@lem.radiantfig.fyito
LocalLLaMA@sh.itjust.works•I have a RTX A1000 (8GB) with 64GB DRR5, what would you recommend for a newbie?English
11·20 days agoDepends what workload you expect to use it for. Different models for different tasks. Personally for agentic coding on an 8gb GPU I currently use either Groot 19b a3b, cyber tiel 35b a3b or orinth1.5 35b a3b, all in q4_k_m running with llama.cpp with context of 130k. Get about 30 tokens per second. My own benchmarking puts Groot just slightly ahead of sonnet 5, not quite 5.5 though.
fozid@lem.radiantfig.fyito
Selfhosted@lemmy.world•How much pain should I expect upgrading my server from Debian 12 to 13?English
6·23 days agoOn a normal setup without leads or crazy customisation, it’ll be a breeze 👍
fozid@lem.radiantfig.fyito
Selfhosted@lemmy.world•Any good file sync server with web interface other than Nextcloud?English
0·3 months agoInstead of installing a massive all in one solution, why not install individual services specifically to cover your requirements? Navidrome for music consumption, synching to actually sync files, WebDAV and OnlyOffice for office docs, VLC, Kodi or jellyfin for video consumption, immich for photos, etc.
It will end up being much lighter and more specialised

If it’s tor like, then it’ll be slow in the same way tor is slow, not due to the small network but due to the anonymisation logic, the multi hop obfuscation, that is inherintly slow.
I feel the issue with a multi location search function being built into a client makes said client a defacto pirating tool, this a target for take down orders, whereas a torrential client without that feature can be used for genuine uses and thus can’t be subject to take downs in the same way.
Years ago I used a client that had the search in, but it disappeared long ago… Talking 15+ years ago though