Sometimes you’re not even allowed to even suggest that AI might be a thing that exists

It’s healthy to change mind in light of new evidence. AI has improved since GPT 3 days - in particular in coding tasks.

“This is a BIG merge window,” he wrote in his weekly state of the kernel update, as it contains almost 20,000 commits compared to the usual ~15,000 that developers submit for a new cut of the kernel.

Great to see so much momentum!

Anti-intellectuals think science is like in the movies: write gibberish on a chalkboard and time travel is suddenly invented. “Who needs experiments? My equation is correct!”

Wake me up when their AI can propose and conduct novel scientific experiments.

Maths is an “easy” target for AI because it’s self contained. It’s easier to brute force training data in mathematics when there’s no external forces.

Can’t have that cheat code in science, unless they let their models study the behavior of real ant colonies as part of their reinforcement learning process.

90% of brits probably don’t know about Ubuntu.

I prefer Java style:

entity.position = entity.position.add(entity.velocity.scale(dt));

Not hard to read.

I think I’ve only encountered clearly anti AI people twice IRL. What I found kinda ironic is how they both parroted the same anti AI talking points I’ve already seen online multiple times.

Operator overloads is something that sounds like a good idea until you experience them firsthand. Just properly name your functions for fuck’s sake.

There’s a difference between pre-training and reinforcement learning.

Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.

Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.

As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.

Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.

There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.

That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.

They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.

Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).

They’re going to use AI to install the AI.

Recursive Self Installation

Wordpress is still commonly used for e-commerce. You’re probably visiting more PHP sites than you realize.

There might be “better” languages, but PHP is still used by ~70% of the web.

四