They’d be competing with a bajillion other case makers. And I’m pretty sure there are already cases with what you ask (such as 5.25 bay mounted IO running off USB headers, at least).

Like… I don’t really see what framework can bring making a case. Maybe it could be a super SFF mobo with a GPU bay, but that’s close to what they did here.

There’s little point in framework selling a conventional desktop.

I guess they could have made another laptop size with the the dev time, but… I dunno, this seems like a niche that needs to be filled.

On my 7800, it’s static. The 2GB I allocate is not usable for the CPU, and compute apps don’t like it “overflowing” past that.

This is on Linux, on a desktop, ASRock mobo. YMMV.

Most CUDA or PyTorch apps can be run through ROCM. Your performance/experience may vary. ZLUDA is also being revived as an alternate route to CUDA compat, as the vast majority of development/intertia is with CUDA.

Vulkan has become a popular “community” GPU agnostic API, all but supplanting OpenCL, even though it’s not built for that at all. Hardware support is just so much better, I suppose.

There are some other efforts trying to take off, like MLIR-based frameworks (with Mojo being a popular example), Apache TVM (with MLC-LLM being a prominent user), XLA or whatever Google is calling it now, but honestly getting away from CUDA is really hard. It doesn’t help that Intel’s unification effort is kinda failing because they keep dropping the ball on the hardware side.

I suspect this machine will be popular with hobbyists for running really large open weight LLMs.

Yeah.

It will probably spur a lot of development! I’ve seen a lot of bs=1 speedup “hacks” shelved because GPUs are fast enough, and memory efficiency is the real bottleneck. But suddenly all these devs are going to have a 48GB-96GB pool that’s significantly slower than a 3090. And multimodal becomes much more viable.

Not to speak of better ROCM compatibility. AMD should have done this ages ago…

And even better, “testing” it. Maybe I’m sloppy, but I have failed runs, errors, hacks, hours of “tinkering,” optimizing, or just trying to get something to launch that feels like an utter waste of an A100 mostly sitting idle… Hence I often don’t do it at all.

One thing you should keep in mind is that the compute power of this thing is not like an A/H100, especially if you get a big slowdown with rocm, so what could take you 2-3 days could take over a week. It’d be nice if framework sold a cheap MI300A, but… shrug.

shrug

Playing devil’s advocate, someone has to moderate replies on the mirrors though.

The APU/RAM is one unit, everything else is modular and repairable. They aren’t price gouging the RAM either.

They’re one upping Apple, big PC OEMs and Chinese Mini PC makers with a more repairable, consumer friendly product. That sounds like Framework to me.

I think you are clinging to the idea that RAM will be separate and upgradable from the CPU for a long time… Physics dictates that it will not, especially in the laptop space where wasted power is so critical. Hacks needed to even make that work now with DDR5 are kinda crazy and inefficient (just look at the voltage/speeds/timings ddr5 sodimms run at). LPCAMM is supposedly a good stopgap, but even that is having teething issues

Not exactly. OpenCL as a compute framework is kinda dead.

“VRAM” has to be allocated to the integrated GPU in the BIOS, and reports (and previous platforms) suggest the max one can allocate is 96GB, or 3/4 of it.

Any that needs a lot of VRAM and good CPU performance on a budget, but not necessarily the real time performance of a W7900 or whatever.

Tesla is currently four times as big as Toyota by market capitalization.

…But that number is total BS, just gambler’s hype, it doesn’t actually represent how “big” a company is. Like, I’m more “pro investment” than most of Lemmy and subscribe to the Buffet ideology that stocks aren’t a total gamble, but Tesla’s stock price has been untethered from reality for years. The idea that they would somehow take over the entire auto market (as the price suggests) makes zero sense.

Fair. True.

If your workload/test fits in 24GB, that’s already a “solved” problem. If it fits in 48GB, it’s possibly solved with your institution’s workstation or whatever.

But if it takes 80GB, as many projects seem to require these days since the A100 is such a common baseline, you are likely using very expensive cloud GPU time. I really love the idea of being able to tinker with a “full” 80GB+ workload (even having to deal with ROCM) without having to pay per hour.

I guess I’m not sure what you want Framework to due instead. Just not launch this at all? What alternative are you advocating for?

Honestly CPUs are bad for AI, especially in this case where there’s a GPU on the same bus anyway.

Of the top of my head, video encoding and compression/decompression really likes raw memory bandwidth. Maybe some games? Basically, wherever the M Pro/Max CPUs are really strong compared to the base M, these will excel in the same way.

I’d argue not. It’s as modular/repairable as the platform can be (with them outright stating the problematic soldered RAM), and not exorbitantly priced for what it is.

But what I think is most “Framework” is shooting for a niche big OEMs have completely flubbed or enshittified. There’s a market (like me) that wants precisely this, not like a framework-branded gaming tower or whatever else a desktop would look like.

And the “base” of this is physically more like a cut down M4 Pro than a regular M4.

The AI max chips are a completely different platform, more than double the physical silicon size of most minipc chips.

Yeah.

But that’s AMD’s fault, as they gimped the GPU so much on the lower end. There should be a “cheap” 8-core, 1-CCD part with close to the full 40 CUs… But there is not.