I have a decent CPU and GPU with 12GB VRam - this should let me run the 7B at least, from what I have seen in the sticky post.

Beside downloading the model, what kind of UI should I start with? Are there good tutorials around, that you are aware of?

Thanks for the input!

I recently built a new PC to handle Stable Diffusion, that gives me 12GB of VRAM to work with. I also started to self-host a few things on a VPS recently, so I have a bit of a basis there.

As for Stable Diffusion integration: I do storytelling/worldbuilding as a hobby and find LLM’s to be an amazing tool to “brainstorm with myself”. It would be amazing if I could tell the LLM to “make a picture of the new character” and it would connect to and prompt SD accordingly. I assume that this is out of scope of what’s currently possible, but something like that would be my goal. I will certainly have a look at langchain as you proposed. Also that’s the context of me asking about cross referencing and context length. I’ve been working with ChatGPT, and while it is an amazing tool it had me bummed out when it couldn’t reference a character that was developed a couple hours earlier (even in the same thread). The cross referencing solution that you sketched above might work for me, but I guess it’ll take a while to learn how to do it.

Given this as a bit of context: where should I start? Downloading Llama 2 as another reply suggests and go from there?

Yes, the final setup would be to run Adguard on a docker container and have this container be in a VPN. I’m not sure yet, how I would do that without messing up the other things already running on that VPS. Maybe I will go for Raspberry to run adguard at home.

Edit: I have set the second DNS in Windows to the same IP, so it shouldn’t have a fallback.

thank you, that clears things up a bit. Now it’s to play around with it, until I get it up and running :)

Could you have a look at my answer to the poster above - would multiplexing mean, that I configure my internal IP 0.0.0.0:XXXA for one service and 0.0.0.0:XXXB for another?