andallthat

the point I was trying to make is that the reason both pro and anti-AI sentiments are blind is because "AI" companies are purposely mixing up things that don't belong together, in order to attract investments.

If you wrote "cruise ships are generating a lot of pollution" and someone answered "but it Magellan or Columbus hadn't had ships, our knowledge of the World wouldn't have advanced", you'd think they are gaslighting you, right? You wouldn't say "this blind anti-ship sentiment is going to hurt geography"

Machine Learning Models have existed for a long time. They are at their core predictors: you give them data, you carefully tweak the model's parameters for a long time and you can finally train a model that can make predictions in a specific domain. That way you can have a model trained specifically to identify patterns that look like cancer on medical imaging or another one (like in your example) to predict a protein's structure.

LLMs are ML models too, but they are trained on language. They learn to identify patterns in human language and predict long pieces of text that are similar to those language patterns. They also accept input in natural language.

The hype consists in slapping a new "AI" marketing label onto all of Machine Learning, mixing LLMs and other types of models, and creating the delusion that predicting a protein's structure was done by people at Google casually throwing prompts at Gemini.

And as these LLMs are exceptionally power-hungry and super expensive (turns out that predicting human language based on a whole internet's worth of training requires incredibly complex models), that hype is to gather all the needed trillions of investment. GenAI is not the whole of Machine Learning and saying "Copilot is not worth the cost of the energy that's needed to power it" doesn't mean creating obstacles to ML used for cancer research.

In other news: AI is a better human than Duolingo CEO

You are right. Bunch of incel 19-year-olds... This is probably more about hiding their browser history from their moms

" Under the mighty gaze of our Beloved Supreme Leader, steel folded and the Great Warship itself bowed. Cower and tremble, enemies of our Powerful State!"

Easy, we just give AI access to all our files and personal information and it will know our age!

Look up stuff where? Some things are verifiable more or less directly: the Moon is not 80% made of cheese,adding glue to pizza is not healthy, the average human hand does not have seven fingers. A "reasoning" model might do better with those than current LLMs.

But for a lot of our knowledge, verifying means "I say X because here are two reputable sources that say X". For that, having AI-generated text creeping up everywhere (including peer-reviewed scientific papers, that tend to be considered reputable) is blurring the line between truth and "hallucination" for both LLMs and humans

Basically, model collapse happens when the training data no longer matches real-world data

I'm more concerned about LLMs collaping the whole idea of "real-world".

I'm not a machine learning expert but I do get the basic concept of training a model and then evaluating its output against real data. But the whole thing rests on the idea that you have a model trained with relatively small samples of the real world and a big, clearly distinct "real world" to check the model's performance.

If LLMs have already ingested basically the entire information in the "real world" and their output is so pervasive that you can't easily tell what's true and what's AI-generated slop "how do we train our models now" is not my main concern.

As an example, take the judges who found made-up cases because lawyers used a LLM. What happens if made-up cases are referenced in several other places, including some legal textbooks used in Law Schools? Don't they become part of the "real world"?

I tried reading the paper. There is a free preprint version on arxiv. This page (from the article linked by OP) also links the code they used and the data they tried compressing, in the end.

While most of the theory is above my head, the basic intuition is that compression improves if you have some level of "understanding" or higher-level context of the data you are compressing. And LLMs are generally better at doing that than numeric algorithms.

As an example if you recognize a sequence of letters as the first chapter of the book Moby-Dick you'll probably transmit that information more efficiently than a compression algorithm. "The first chapter of Moby-Dick"; there .. I just did it.

I was not blaming your country at all, you're more than doing your part. It's just frustrating.

Thinking of the families of the victims, I hope that knowing they are not forgotten and people are trying to uncover the truth about what happened will at least provide some closure.

Especially right now, I'm feeling lots of things but "lucky" ain't one...

The Netherlands and Australia want the ICAO Council to order Russia to enter into talks on possible reparations

"enter into talks on possible reparations". Absolutely brutal, I wouldn't want to be Russia right now...