What’s ironic is that the local llm/diffusion communities will not touch these. They’re just too slow, and impossibley finicky to set up with models big enough for people to actually want.
AMD’s next gen could change that, but they’ve already poisoned the branding. Good job.
What the hell do I need a laptop with an NPU for? I can write my own emails and documents. I don’t give a crap about the paint and notepad AI additions. I seriously cant think of any use case for copilot plus.
What is an NPU? Nonsense Processing Unit?
Neural, I think?
It is now!
This is such a hilarious bit of branding nonsense. There is no such thing as “AI PCs”.
I mean, I technically own one, in that the branding says I do and it has a Copilot button, but… well, that’s definitely not why I purchased it and I don’t think I’ve used an “AI feature” on it. I’m not even opinionated against them, I have run local LLMs in my other computers, it’s just not a good application for the device I own that is specifically branded for “AI”.
The stupidity of it is that my “AI device” is an ARM device, and I absolutely love the things ARM Windows does that are actually useful. I pulled up my old x64 device that I used before I got this and man, the speed of Windows Hello, how much better it handles video streams, the efficiency… I’d never go back for a portable device at this point.
But the marketing says it’s “AI”, so once people start telling each other that “AI PCs” are bad and new AMD and Intel “AI” CPUs are released it’s anybody’s guess how the actually useful newer Windows ARM devices will fare.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides so we can start talking about real features now, because man, this has been a frustrating generation to parse for portable Windows devices, and we still have Android, iOS and Mac devices coming down the pipe with similar branding nonsense.
I think this is referring to machines that come with APUs that have enough tensor (or whatever the equivalent is) cores in order to provide a certain baseline of “AI” operations per second, so Microsoft et al can rely on that for Recall etc without users thinking it slow.
Amusingly, the first generation of these didn’t really have enough, so they’re truly unwanted.
Yeah, my problem is that this was made to coincide with the Snapdragon Windows PCs, which are really good at a bunch of stuff and specifically not good at NPU performance, so the result of the “AI” branding ends up being really disappointing.
We could talk about all the other growing pains and the ways those devices were covered, but the obsessive focus on “AI” certainly didn’t help, as demonstrated by the bizarre reporting linked in the OP.
I’m still hoping that the somewhat irrational anger towards “AI” stuff subsides
I think this anger is linked to the irrational exuberance for “AI”.
Personally, I kinda hate AI. Not because of any sort of fear of job loss or anything like that. It’s because “AI” has been rolled out heavily in the Cybersecurity space, making my work life hell because of it. Models are only as good as their training and this means that any AI model which is going to spot anomalies in a network needs to spend a good amount of time being trained. However, what the vendors sell are touted as unsupervised models. They just need to spend a while on your network and they can automagically learn what “normal” is and then alert you on “abnormal”. This ignores the fact that you still need your analysts chasing false positives constantly from this black box. And that “black box” aspect is a major problem. You’ll get an AI/ML based alert with exactly fuck all in detail on why the alert triggered. If you’re lucky, you might get a couple log entries along with the alert, but nothing saying why those entries are suspicious.
I will grant that, there are a few cases where the “AI” in a product has worked. Mostly, it’s been in language processing. Heck, having an AI half-write a function for you in a tool you don’t use very often is quite nice. You almost always need to rework the results a bit, but it can get you started. But, my first question for any vendor talking about “AI Detections” is “how do we tune false positives?”. It’s just too big of a headache. And most of them try to downplay the need or dodge the question. Or, you have to babysit the model, effectively making it a supervised model. Which, fine. Just stop telling me how much time it’s going to save me, when I’m going to spend more time supervising the model than searching for threats in my environment. And, for fucks sake, design that shit to explain itself.
As for putting AI in my system. I can see a use case for language processing. Heck, I’d love to have the Star Trek style, “hello computer…” type stuff actually work worth a damn. Google and Siri are pretty close, though even those can be shit on toast when trying to do anything slightly complex. And having all that done locally, without having to send data “to the cloud” sounds great for privacy and security (until MS adds a keylogger as part of the OS). But, given how much time my GPU sits at or very near idle, I do wonder if the extra chip is worth the silicon or space.
In the end, I’m expecting this to go much the way TPM has. We’ll all end up with it in our systems, whether or not we know, care or use it. All because manufacturers just start soldering it on to everything. Maybe someone will find a good use for it eventually, distributed AI porn, maybe? But, like a lot of AI, it seems like a solution in search of a problem.
I’ve been extremely annoyed with all partisanship on this issue from the go. The absolutely insane range of promises, hyperbole and circlejerking among the AI techbros is absolutely insufferable. Meanwhile, the other side has been entirely content with getting a chance to repeat the dogpile they (deservedly) gave cryptobros and entirely unwilling to acknowledge any sort of nuance, often to the point of deliberate misinformation.
The only reason I’m not angrier at this is that, honestly, this whole thing is way less momentous than either side has claimed and I have bigger things to get mad about.
What are you referring to about Windows Hello?
Windows Hello existed long before AI became a trendy thing.
The instructions supported by NPU processors are designed for tasks like image recognition.
So, theoretically Hello might be quite a bit faster or more reliable (for a given time or energy budget) with an NPU.
The marketing is 100% drivel though.
Nothing to do with NPUs, as far as I know, it’s down to the dedicated video stream acceleration that ARM chips have carried for a while.
In practical terms, Windows Hello unlocks noticeably faster, just like a phone with facial recognition does. It also disproportionately handles video playback more efficiently, given the relative overall performance. Nothing new, either, it’s all stuff Apple got out of their own ARM transition, but way underrated in how much of a moment-to-moment difference it makes, particularly for stuff like the Surface Pro and other hybrids, where you want to handle it on battery and use it more like a tablet sometimes.
It’s a double whammy. No one wants AI. No one likes Windows 11.
I expect that a decent amount of sales of these are just people who don’t care replacing their PC.
I’d be shocked if there’s in any way a statistically significant amount of these purchases driven by the “ai features”.
Don’t forget corporations that lease new computers every few years.
It isn’t that surprising… they are shipping AI chips, but there is almost no software that makes use of them yet. Unless people require them for a certain use case, why bother?
Reminds me a bit of “5G ready” phone contracts, just meaningless marketing. Don’t need to chase every hype.
5g shot nobody asked for.
LTE is more secure anyway if that shit can be secure at all 🤡
I will block the AI on any computer I use.
What kind of AI workloads are these NPUs good at? I mean it can’t be most of generative AI like LLMs, since that’s mainly limited by the memory bandwith and at this point it doesn’t really matter if you have a NPU, GPU or CPU… You first need lots of fast RAM and a wide interface to it.
That’s why NPU will have high bandwidth memory on chip. They’re also low precision to save power but massively parallel. A GPU and CPU can do it too, but less optimized.
That was my question… How much on-chip memory do they have? And what are applications for that amount of memory? I think an image generator needs like 4-5GB and a LLM that’s smart enough as a general porpose chatbot needs like 8-10GB. More will be better. And at that point you’d better make it unified memory like with the M-series Macs or other APUs? Or this isn’t targeted at generative AI but some other applications. Hence my question.
Last I heard this is for onboard speech recognition and basic image recognition/OCR so these things can more intelligently listen, see and store what you’re doing without sending it to a server. Not creepy at all.
Have they tried forcing people to upgrade to AI PCs in order to receive security updates by checking to see if your PC is an AI PC? You know just to prove people really want AI PCs?
/s
Marketing ruining it for themselves. Good job.
Can someone point me to technical/learning resources about NPUs? So far all I have seen is superficial marketing talk and ads. And on top of that, everything existing in the AI/ML sector still seems to require beefy server hardware. So is there any real point to NPUs at all?
Because AI still sucks. It’s incredibly unuseful for day to day operations, and only useful in very specific scenarios that I, personally, only need access to it like 2-3 times a year. It’s not worth having it on my personal devices when I can just go to chatgpt website or something when I want to give it a shot.
I got one for a good price a few months ago to replace my dying Surface Pro 3.
No regrets, but I love it for the speed and the screen, not all the AI bullshit.
No issues no my end with ARM; it doesn’t play games well but I have a gaming PC for that.
They are complete ass.
So, I might build a new machine next year. Does that mean motherboards will have an extra NPU chip on them? Or are new CPUs supposed to come out with the extra functionality?
I won’t be using it in any case, as 1. I’m not running Windows, and 2. there most likely wouldn’t be any purpose to it even if I was, but this will probably contribute to making hardware more expensive for no gain (yay).
The newer mobile chips have some sort of npu type silicon included. I don’t think it’s very widespread on desktop yet.
