“Where are the engineers” to manage this on prem hardware?
Well, you laid them off, you fuck.
They’re going to use AI to install the AI.
Recursive Self Installation
“I used the stones to destroy the stones.”
IBMs a bit weird. They fired a shitload of them a while back.
Then they said they’re only replacing junior roles.
Then I think they’re hiring a lot more juniors because they need to build the next seniors.
I didn’t really follow too closely but that last statement is at least something I can respect.
[edit] I think mushroommunk is absolutely on point [/edit]
It’s not really something you should respect. It’s PR shell games. They fired senior staff to replace them with cheaper juniors. Not because they actually think they’re training the future but because they think they can get the same work out of them+AI
Of course IBM has a hardware solution for you
Krishna also predicted demand for on-prem AI will spur sales of Big Blue’s Spyre mainframe accelerator cards.
“It’s a simple question for any enterprise. Would you like something that’s 10 percent of the cost if it’s 90 percent as good as the best, or do you want to pay that 10 times higher cost?”
That really depends…
90% is disasterous. It may sound good because it’s “just” 10% off 100%, but it’s awful. Two iterations reduces that to ~0.01%< but that’s still pretty bad. Obviously the more complex, the higher the risk and the more demand and time per iteration. The machine also has no concept of criticality, so how much is being saved with three or even four iterations to assure failure is basically zero?
“Then the question is: how many tasks require that? I would tell you some do, but the vast majority, summarizing an email, examining a contract, answering a customer call, composing an email, all of those can be done as well with the much cheaper answer.”
Imagine 10,000 emails an hour through your enterprise and 1,000 times It fucks up. Anything from missing info to misinterpreting to just straight up modifying/deleting. No concept of criticality per email. 8,000 in a 9–5 office. 24,000 a day for a global enterprise. That’s what 90% looks like.
Then you got “examining contracts” and all the other stuff he mentioned…
BTW, IBM/RHEL is hoping you might go for some red hat, IBM solutions for local distributed AI stuff :
- https://www.redhat.com/en/artificial-intelligence/inference#distributed-inference
- https://developers.redhat.com/products/rhel-ai/overview
- https://www.redhat.com/en/products/ai/enterprise-linux-ai
But you know, the CEO isn’t biased. /s
At least they’re working on letting you have access to the open source (ish) local way of working with AIs instead of just the subscription model that the others are pushing.
I did some digging into this before giving the fuck up since my computer is a potato, and I have too many fucking things to do with my time.
If I could pick a company that has fucked up more than Microsoft or EA, it would be IBM. For the past 20 years, the whole company has been on a wild bender of buying good products, and shoving them headfirst into the ground.
For all their faults, IBM has treated Red Hat and Hashicorp better than Broadcom has treated VMware.
I read your article and still think my statement stands. In fact, your article shows that the behavior the author didn’t like from Red Hat existed long before the IBM acquisition. If you see the Red Hat treatment of its paying customers equal to what Broadcom is doing to its VMware customers, then I don’t believe you have a good grasp of the Broadcoms current behavior.
Maybe I’m just sleep deprived, cynical, or thinking too deep into it, but I don’t even like how they are using the words “on-prem” throughout the article instead of simply saying they’re running local LLMs. Seems like such a weird choice of wording that is meant to subtly steer less informed people away from looking into alternatives to the subscription based LLM services, as most casual people wont associate it as the same thing and ignore it and move on. Even if that’s the correct terminology for that specific field, I feel like most people would just naturally say “Yes, we are running the LLM servers locally here” as that’s the only way I’ve ever heard people say it even online.
For a company “on-prem” might not mean the same as “local”. You might run them in your company data center or in a colocated server (rented space in a data center). In either of those cases, the data center might be physically close to you, but it could be on a different continent.
“On-prem” implies that the company has physical access/control of the server.
