I’m thinking about starting a self hosting setup, and my first thought was to install k8s (k3s probably) and containerise everything.

But I see most people on here seem to recommend virtualizing everything with proxmox.

What are the benefits of using VMs/proxmox over containers/k8s?

Or really I’m more interested in the reverse, are there reasons not to just run everything with k8s as the base layer? Since it’s more relevant to my actual job, I’d lean towards ramping up on k8s unless there’s a compelling reason not to.

  • donalonzo@lemmy.worldEnglish
    5·
    2 years ago

    VMs are often imperative and can be quite easy and familiar to setup for most people, but can be harder or more time-consuming to reproduce, depending on the type of update or error to be fixed. They have their own kernel and can have window managers and graphical interfaces, and can therefore also be a bit resource heavy.

    Containers are declarative and are quite easy to reproduce, but can be harder to setup, as you’ll have to work by trial-and-error from the CLI. They also run on your computers kernel and can be extremely slimmed down.

    They are both powerful, depends how you want to maintain and interface with them, how resource efficient you want them to be, and how much you’re willing to learn if necessary.

  • SilentMobius@lemmy.worldEnglish
    4·
    2 years ago

    Personally I always use containers unless there is a good reason to use a VM, and those reasons do exist. Sometime you want a whole, fully functional OS complete with custom kernel, in that situation a VM is a good idea, sometimes a utility only comes packaged as a VM.

    But absent of a good reason, containers are just better in the majority of cases

  • LetMeThinkAboutIt@programming.devEnglish
    3·
    2 years ago

    Why not do both ? As I understand it, to do kubernetes clusters, you must have at least 3 hosts. They don’t need to be 3 different physical hosts: they could be VM (hosted on Proxmox).

    Proxmox also having a very strong implementation of ZFS, then it could be used as the storage « host », and it gives you also the option to do snapshots of the VM (and the storage pool), as well as replication/etc.

    • scottyjoe9@sh.itjust.worksEnglish
      2·
      2 years ago

      A k8s cluster can run on a single host if that’s what you want. I’m not sure if it would be worth the virtualisation cost to run it on VMs in the middle as well. If you were only ever going to run on a single host I probably wouldn’t use k8s though, I would just run containers. 🤷‍♂️

  • InverseParallax@lemmy.worldEnglish
    1·
    2 years ago

    Por que los dos?

    Debian hypervisor with raidz2 hosting vms, the main ones being 1 main freebsd host with 20 jails containing 1-2 apps each, and 1 main debian vm hosting things that are too much of a pain in the ass to get running on freebsd, so it hosts 5 docker containers.

    Rocks my world.

  • Zardoz@lemmy.worldEnglish
    1·
    2 years ago

    Not a proxmox pro by any means, but it can do both VMs and containers. I have a few VMs for various Linux distros to play around with. I also have one dedicated VM for all my security related tools.

    Stuff like PI hole, jellyfin, logstash, etc. dont really have any need for a full OS, so a container works perfectly. Plus having a full OS with several things running on it makes it more difficult if you just need to restart one service

    I started doing everything in VMs but over time realized some things were better to maintain as containers