Miscellaneous News
- Thread starter bd popeye
- Start date
If you have EPYC/Xeon along with around 700 GB RAM (671 GB for the FP8 model + room for context) or a similar amount of VRAM on GPUs, then yeah, you can run R1 locally. If you have the same rig as above, then you can also run LLaMA 3.1 405B locally - albeit, if you run on CPU, then it would work painfully slow unlike R1 because 405B is a dense model while R1 is MoE with 37B activated parameters at a time. However, if it's for your personal usage, then it does not make any sense to spend so much money when the DeepSeek API is dirt cheap. Plus, you can use stuff like web search through their Web UI.
$850M Capex is for the datacentre, DeepSeek also was not trained on a desktop, lol, they had 2048xH800 iirc. It is still impressive because typically the models of that class require at least the same number of cards as this big LLaMA - i.e. 16k H100, so DeepSeek pulling this off with only 2k H800s, which are gimped versions of H100, is huge.
Distilled versions are not really comparable to the full R1 though, not even in the same league. Most are not even the best in their parameter size category, mainly because these distilled versions did not go through the same training process as R1 with RL and all of that, only fine-tuned on the R1 outputs.
I wonder what the Qwen team will deliver this time; their Qwen 2.5 models are still the best or close to the top in their respective parameter sizes. For example, Qwen-2.5-32b-Coder is the best coding model at this size, no contest.
Last edited:
$1T is not big amount when you consider dollar devaluation and population size. these numbers population adjusted even Russia achieved much before. Russia has reached such a stage that it no longer need to sell anything based on high tech. it can just sell other stuff and still create trade surplus. This is not Abu Dhabi that is far richer and they are not going to compromise on real estate prices at higher end to attract any one. they can just boost salaries unofficially.
plawolf
Lieutenant General
It would be a mistake to dismiss Trump out of hand as an idiot. If we forget about who is championing the policy and just look at the strategy behind the moves, it’s actually not without merit.
Firstly, Trump is someone who cares deeply about his legacy and how he will be remembered. That is easily evident from his desperate pathological need to literally put his name on everything. As such, he would not be all that interested in spamming loads of orders that then get systematically undone even before he leaves office. As such, this blitz of EOs is not just a real life DOS attack. While that is certainly part of the method, it’s more of a means to an end rather than the end in itself.
I think the strategy behind this move is simple. It’s about momentum and critical mass. America needs a revolution, it’s as simple as that. Trump is trying to give them enough of the wholesale societal change of a revolution to head off the actual revolution and do away with the oligarchy getting guillotined part, since his head will also roll in that case. Worst case, all of his EOs does get overturned eventually, but in the months or even years until then, he has already changed reality on the ground. The hope is that while legally speaking he hasn’t got a leg to stand on, but if he can achieve results fast enough, the legislature and people might just decide they actually liked what he was selling once the courts overturns his EOs and demand those changes gets made back into actual laws. Even if the people don’t like the methods enough to enshrine them as law, they might like the results enough to not seek to undo what Trump has achieved. Because if there is one thing Trump loves more than putting his name on buildings, it’s the idea of people saying he was right.
AssassinsMace
Lieutenant General
The US thinking everyone else wastes time spinning as much as they do. Slavery isn’t slavery if you just don’t call it slavery. Don’t call Marco Rubio… Marco Rubio so that China can save face and still talk the US Secretary of State even though he’s under China’s sanction list.
plawolf
Lieutenant General
What good are those machines when they come preloaded with kill switches? Hard pass.
It's so funny that speaker was so confused that she directly asked in English.
An article I read earlier on Zhihu perfectly answers this question.
I fear they’re taking the wrong lesson from this XHS situation; rather than banding together into a workers revolt and a new armed revolution t9 usher in a new socialist America, I fear their response is to try to become an illegal immigrant in China or worse, thailand style sexpat in Asia. It’s bad enough that Asia has to deal with harassment from creepy Indians but an enclave of colonisers in Asia is the last thing China nor the rest of east Asia needs.
What a pitiful state. How the mighty have fallen, Denmark the ardent champion of globohomo is now to be devoured to perpetuate white Anglo supremacy. If the Danes do not fight, they may as well castrate themselves and give over their women to men of superior genes.
This is where we should cosplay as Anglos and other westerners by tut tutting about “how can we trust the U.S. Government’s numbers”?! ”funny numbers”
It’s racism. It’s always been racism, because now every caucasoid and wannabe caucasoid realises they have nothing that they can Base their superiority complex on with regards to China, plus only a few months ago westerners viewed Chinese as lower than them. XHS showed them the truth, and 6th gen fighters and Zhuhai reminded them of their mortality, if you were say a WASP elite, Jewish cabalist or Mormon CIA worker, all of whom depend and perpetuate Anglo American whites supremacy, China is the single greatest threat and challenge to your ability to exploit the entire globe.
You just know it isn’t just normal sex workers but Epstein level crap as well.
You cannot run the full Deep Seek model in a Rasperry Pi. It is mathematically impossible to run a 671 billion parameters model on anything less than a cluster of work station GPUs.
What you're talking about is running the distilled versions of the model, which is significantly weaker than the full R1, and while still impressive, it's more for hobby developers than enterprise users.
The real benefit of Deep Seek is the ~20x reduction in enterprise API costs (according to benchmarks - Deep Seek generates more thinking tokens so costs a bit more than it might seem just looking at cost/token vs. O1). That is a consequence of several factors, of which cheaper training costs is only a small contributor.
It took Meta ~$60 million to train Llama 3.1. But how much money do you think they spent on engineering resources within the company? A team of ~200 researchers/engineers costs >$250 million a year for a Silicon Valley company to keep around between compensation and benefits. Add that on top of all the GPUs they had to buy to build up infrastructure, the support staff, the building costs, and we're talking a billion dollars a year for a generative AI team. It is well known that Open AI pays its researchers $1 million a year, so they're probably paying even more.
That's the thing that Deep Seek was able to get around, not just from significantly reducing model training costs, but from superior value/currency spent, which is shared by all Chinese companies relative to the West. The cost to train a model is just a fraction of the cost it takes to maintain a viable LLM product. It's everything else where the cost difference makes the most impact, and that's also why I think, longer term, China's real advantage won't be from algorithmic innovations - which will be quickly copied - it'll be from structural advantages, which are far more sustainable.
Last edited: