How does he use IG?

Assuming its not a parody.

I feel like they are dropping the ball in the GPU space though, both on desktop and in servers.

They’renot really leveraging it. They killed the steam deck line of “small core count, GPU heavy APUs” which is why Valve hasn’t updated it and competitors seem so power hungry. They all but killed server APUs, making them mega expensive and HPC only. They’re finally coming out with a M-Pro like consumer APU, but it took until 2025, and pricing will probably be a joke just like their Radeon Pro GPUs…

And I don’t even wanna get into the AI space. They get like 99% there and then go “nah, we don’t really care about this market, let Nvidia have their monopoly and screw everyone over.” It makes me want to pull my hair out.

With the only saving grace being that he’s actually like a hesitant coyote not big enough to take a combined Europe, but with a bomb strapped to its chest.

My take is they should be fighting the corporate API vs open source models war, instead of just “screw all AI” which really means “screw open source AI and let Sam Altman enshittify everything”

Especially on Lemmy.

It’d be like blanket railing against social media and ultimately getting the Fediverse banned, while Facebook and X walk away.

Perplexity (or open source equivalents) are much better for this.

Yeah, well, this is what everyone thought in Georgia, and Ukraine in 2014.

He’s not going to change. His chosen successor probably won’t either.

We also knew exactly who Trump is. We have a very long history.

I particularly love stuff about him before he was in politics, like the Motley Fool podcast on how he duped public investors for his private company through pumping up real estate values. They went to his office, saw this weird array of gaudy decoration and oddly attractive employees, sat down with him, and saw through his lie. Then made the only short in their firm’s entire history… and it paid off.

There’s no excuse of bias. You can’t blame any politicians. It’s just him. And while not perfect by any means, you have to squint hard to see Kamala in the same light.

Last time bothering you! I used to be really into the GAN space myself, but the newer diffusion models really blow them away. Check this out: https://github.com/mit-han-lab/nunchaku

This can squeeze Flux 1D onto your 3080, and (with the right pipeline/settings) it should blow anything else away at “enhancing” a low res image with img2img. It should also work with batching and torch.compile so you can get quite a lot of throughput from your 3080. Of course, there’s no temporal consistency yet (or it may be, it’s hard to keep up with all the adapter releases), but I’m sure its coming… And you can kinda hack some in with 2D models anyway.

Pick up a 3090 if you can!

Then you can combine it with your 3080 and squeeze Qwen 72B in, and straight up beat GPT-4 in some use cases.

Also, TabbyAPI can be tricky to set up, ping me if you need help.

And never watch them?

For a while, yes…

If you like Castlevania, try DOTA Dragon’s Blood.

And both Avatar series, if you somehow haven’t seen them.

Just saying, 3rd party databases are great: https://reelgood.com/

It’s incredible how much more useful they are than the stupid rec algorithms.

I don’t think anyone wants a hot war in NK, and I’m not sure what good it would do.

Europe needs to (and should have) get off their butts and send every piece of hardware they have to Ukraine though, even cutting edge ones. Maybe even enforce a no-fly zone. As I keep asking, what are they waiting for… Spain to invade France? No, they built all this stuff to deter Soviet aggression, and its just sitting there, rotting instead of doing its job. If Ukraine would have stayed secure, they basically would never have to worry about this again.

Now they have no excuse. Russia clearly has no shame. And it’s almost (but not quite) too late.

Bitnet is theoretical now and unsupported by NPUs anyway.

Basically they are useless for large models :P

The IGPs on the newest AMD/Intel IGPs are OK for hosting models up to like 14B though. Maybe 32B if with the right BIOS, if you don’t mind very slow output.

If I were you, on a 3080, if you keep desktop vram usage VERY minimal, I would run TabbyAPI and a 4bpw exl2 quantization of Qwen 2.5 14B coder, instruct, and RP finetune… pick your flavor. I’d recommend this one in particular.

https://huggingface.co/bartowski/SuperNova-Medius-exl2/tree/4_25

Run it with Q6 cache and set the context to like 16K, or whatever you can fit in your vram.

I guarantee this will blow away whatever llama (8b) setup you have.

What’s ironic is that the local llm/diffusion communities will not touch these. They’re just too slow, and impossibley finicky to set up with models big enough for people to actually want.

AMD’s next gen could change that, but they’ve already poisoned the branding. Good job.

NPUs are basically useless for LLMs because no software supports them. They also can’t allocate much memory, and they don’t support the exotic quantization schemes modern runtimes use very well.

And speed wise, they are rather limited by their slow memory busses they’re attached to.

Even on Apple, where there is a little support for running LLMs on NPUs, everyone just does the compute on the GPU anyway because its so much faster and more flexible.

This MIGHT change if bitnet llms take off, or if Inte/AMD start regularly shipping quad channel designs.

Apple is also much faster because the integrated graphics are actually usable for LLMs.

The base M is just a big faster than an Intel/AMD laptop if you can get their graphics working. The M Pro is 2x is fast (as its memory bus is 2x as wide). The M Max is 4x as fast.

AMD is coming out with something more competitive in 2025 though, Strix Halo.

Unless you have an Nvidia card.

I’ve been on linux for years, I work the Nvidia libraries all the time, I alternate booting wayland and X… I even use my AMD IGP as output these days, instead of the Nvidia card.

And I STILL hold my breath wondering if I’m going to get a blackscreen, and have to go into tty mode or boot from a usb stick to investigate and fix it.

Honestly, hardly anyone I know IRL knows specifics about Israel and Palestine.

Imagine if it found its way into Musk’s feed!

I would not be against this. I absolutely know how.

We can rebuild him. We have the technology…