Many LLM operations rely on fast memory and gpus seem to have that. Even though their memory is soldered and vbios is practically a black box that is tightly controlled. Nothing on a GPU is modular or repairable without soldering skills(and tools).
What’s even more silly about this is that you never bothered to cook it yourself to experience better cooked food and the reason is? Idk for me it was because I am lame and too shy to ask to change the established way of life. On the other hand I have adjusted to eat food of all sorts even though it is displeasing. Except foods that have capsaicin or or peppers, I’m allergic to them.
I think the biggest limiting factor for your mini PC will always be the VRAM and any workload that enjoys that fast RAM speed. Really, I think this mini PC from framework is only sensible for certain workloads. It was poised as a mobile chip and certainly is majorly power efficient. On the other hand I don’t think it is for large scaling but more for testing at home or working at home on the cheap. It isn’t something I expected from framework though as I expected them to maintain modularity and the only modularity here is the little USB cards and the 3D printed front panel designs lol
Edit
Personally I am in that niche market of high RAM speed. Also, access to high VRAM for occasional LLM testing. Though it is an AMD and I don’t know if am comfortable switching from Nvidia for that workload just yet. Renting a GPU is just barely cheap enough.
Ah the chaos theory reasoning. There can’t be anything perfect when thinking about it as a whole but you can find sections in a diversified area that are mostly perfect at some point in time but nothing ever lasts.