The A.I. Thread

[activeStick]

Originally Posted by Fuzz

There is also no reason the other models can't adopt this technique, and still leverage their extra power and model sizes. Will be interesting to see if that is another step improvement.

Yup.

https://twitter.com/user/status/1883912157768913377

https://twitter.com/user/status/1883912485335629865

https://twitter.com/user/status/1883924181118771519

https://twitter.com/user/status/1883924872499462164

[Shazam]

Originally Posted by Fuzz

Thanks for the link. Sounds like it is a bit novel, as they used a different feedback mechanism that doesn't appear to have caused any major issues. This will save a lot of training/money and I'd expect it to get refined further.

I dunno. Next thing we'll find out a bunch of political dissidents were the labour behind the reinforcement learning.

[Fuzz]

That would obviously be bad, but it sounds like it's all handled with code.

1. Start with a smart normal model, like DeepSeek-V3, and perform the following reinforcement-learning loop
2. Ask that model to solve a mathematical problem, with a prompt that pushes it to think step-by-step
3. Verify the answer in code (i.e. not with a model, but by directly parsing the answer and checking it)
4. If correct, reward the model; if wrong, punish the model
5. Repeat for a long time

The asking the model part is probably more manual, as they'd need to create a list of problems, though I suspect a lot of this is done and grabbed from elsewhere.

[indes]

How come ChatGPT wasn't able to create an AI like this? Doesn't it seem like a black mark to the entire "AI" community if this small group of people with 6.5 million dollars just smoked these corporations with billions of dollars worth of processing power/AI development?

[Fuzz]

Because they thought about the problem differently and came up with a different solution. It's how science works, and it's a good validator that money and resources aren't everything.
I hope someone thinks differently and realizes LLM's are kind of a dead end, and finds a better way. You can use enough tricks to make people believe an LLM is thinking(and it may be convincing and good enough that it doesn't' matter), but an LLM will never be developed into a thinking machine.

[Jason14h]

Because in reality they probably had the entire force of the Chinese government behind them !

Wait until you read about the amazing things going on in North Korea !

In more semi seriousness - their founder is a genius Quant and used AI trading to get rich , so probably knows about as much about this stuff as anyone on the planet

[Fuzz]

OpenAI says it has found evidence that Chinese artificial intelligence start-up DeepSeek used the US company’s proprietary models to train its own open-source competitor, as concerns grow over a potential breach of intellectual property.

The San Francisco-based ChatGPT maker told the Financial Times it had seen some evidence of “distillation”, which it suspects to be from DeepSeek.

The technique is used by developers to obtain better performance on smaller models by using outputs from larger, more capable ones, allowing them to achieve similar results on specific tasks at a much lower cost.

https://www.ft.com/content/a0dfedd1-...c-1fe01de87ea6

How'd you get that Ferrari, anyway? Doesn't make sense.

Oh, well, you see, I stole it.

China gonna China.

[GoFlamesGo89]

Originally Posted by Fuzz

https://www.ft.com/content/a0dfedd1-...c-1fe01de87ea6

How'd you get that Ferrari, anyway? Doesn't make sense.

Oh, well, you see, I stole it.

China gonna China.

Take one more step— where does OpenAI’s training data come from?

Scraped from across the internet— corporation stealing from individuals. America going to America.

[Shazam]

[Shazam]

Originally Posted by GoFlamesGo89

Take one more step— where does OpenAI’s training data come from?

Scraped from across the internet— corporation stealing from individuals. America going to America.

I mean search engines wouldn't work otherwise either.

[OldDutch]

Originally Posted by Fuzz

https://www.ft.com/content/a0dfedd1-...c-1fe01de87ea6

How'd you get that Ferrari, anyway? Doesn't make sense.

Oh, well, you see, I stole it.

China gonna China.

Super wild and surprising I guess. CCP funded hackers stole the data, passed it to deep seek. Voila 6M AI.

[Fuzz]

They didn't steal the data, they used the OpenAI model to train their own.

[Shazam]

Now I am going to train my AI using Deepseek.

[GoFlamesGo89]

Originally Posted by Shazam

I mean search engines wouldn't work otherwise either.

Search engines (by and large) respect copyright and have agreements. The OpenAI training data uses substantial copywrite information.
https://www.reddit.com/r/technews/co...in_generative/


I'm not taking one side over another here-- just that there is no moral high ground.

[Shazam]

Originally Posted by GoFlamesGo89

Search engines (by and large) respect copyright and have agreements. The OpenAI training data uses substantial copywrite information.
https://www.reddit.com/r/technews/co...in_generative/


I'm not taking one side over another here-- just that there is no moral high ground.

Search engines make money from copyright content. It's just not done directly.

[Russic]

Originally Posted by indes

How come ChatGPT wasn't able to create an AI like this? Doesn't it seem like a black mark to the entire "AI" community if this small group of people with 6.5 million dollars just smoked these corporations with billions of dollars worth of processing power/AI development?

I find one of the more interesting thought experiments with anything is "what would happen if we had to do this twice as fast with half the resources?" That's basically the position China was put into. Creativity thrives within constraints sometimes, and the big companies in the US didn't have many. They kept gaining billions of dollars, so they didn't need to whoa back and re-evaluate. It's guerilla warfare.

[Jason14h]

https://www.reuters.com/technology/a...v3-2025-01-29/

Alibaba now #1

[Jason14h]

Originally Posted by Russic

I find one of the more interesting thought experiments with anything is "what would happen if we had to do this twice as fast with half the resources?" That's basically the position China was put into. Creativity thrives within constraints sometimes, and the big companies in the US didn't have many. They kept gaining billions of dollars, so they didn't need to whoa back and re-evaluate. It's guerilla warfare.

Its called IP theft. It's easier to steal then create.

China has much more resources then any tech company in the States if need be.....

Anyone believing this was created for 6 million needs to give their head a shake.

They have all but admitted they have 50,000 illegal Nvidia Chips. These chips are $30-40 K each AND China would be buying on the black market.

The chips alone are 2.5 billion if bought at $50K each (Black Market prices?)

[Shazam]

https://arstechnica.com/tech-policy/...ore-robots-txt

[Fuzz]

Originally Posted by Jason14h

https://www.reuters.com/technology/a...v3-2025-01-29/

Alibaba now #1

That sounds like a self-claim, and they are also claiming to beat Deepseek-V3, not Deepseek R-1. So at best, even their self claim is for #2, not #1.