I was jst curios. Bcoz she "re-signed" an almost like "bri" doc w CN, just wthout the "bri" label right?
Artificial Intelligence thread
- Thread starter 9dashline
- Start date
Kimi seems to be in sync with DeepSeek. Last time, he released Kimi 1.5 at the same time as DeepSeek. After DeepSeek released the NSA technology to improve the efficiency of long text processing yesterday, Kimi also released a similar technology MOBA a few hours later. And at the same time, the code for training and reasoning was released.
github.com/MoonshotAI/MoBA
Similarities between the two:
1. Both use a sparse attention mechanism, that is, not all tokens need to pay attention to each other, but selectively pay attention to some tokens to reduce the amount of calculation.
2. Both operate based on the concept of blocks, dividing long texts into multiple blocks, and performing sparse selection or calculation in blocks.
3. Both are compatible with the Transformer architecture and can be used as a substitute for the standard attention mechanism and embedded in the existing Transformer model.
github.com/MoonshotAI/MoBA
Similarities between the two:
1. Both use a sparse attention mechanism, that is, not all tokens need to pay attention to each other, but selectively pay attention to some tokens to reduce the amount of calculation.
2. Both operate based on the concept of blocks, dividing long texts into multiple blocks, and performing sparse selection or calculation in blocks.
3. Both are compatible with the Transformer architecture and can be used as a substitute for the standard attention mechanism and embedded in the existing Transformer model.
Attachments
-
82c654dfly1hypm532kytj21lj0r7apx.jpg
Aside from saying DeepSeek force them to raise their game. Baidu said 22% of search right now includes AI generated content. They are doing real changes to basic search content. So yeah, they need to deeply integrate DeepSeek is their platform
Tencent came out with their own Hunyuan-T1 model which can also do deep search
Looks like in Tencent Yuanbao, now you can choose between the 2 for web searching.
This is bad for Baidu.
Silicon Flow had several hundred million RMB pre-A round of funding.
Zhejiang university has started full blooded R1 version. It uses the 西湖之光 compute power.
Looks like Tshinghua university fine tuned a version of R1 for learning.
Tomorrow marks One Month since DeepSeek R1 was released and I think its fair to say that at least for Chinese domestic AI its been the second coming of the Cambrian Explosion... I don't in the history of mankind and certainly in China's own 5000+ year history has anything been adopted so quickly
Turns out this is the real "AI Diffusion" (not Biden's) and the fact that they choose to release this on the morning of Trumps inauguration, right around the "StarGate" reveal, and their recent breakthrough paper underlined it was taking on the "NSA" lol (intentional play on words) and probably for the massive DDoS attacks to try and suppress DeekSeek worldwide usage etc .... I don't think any of this was coincidental....
Well, there was also the android moment when all the Chinese OEMs quickly adopted to using android to commoditize the market.
That was very impressive for that time.
Grok 3 pulled a Volkswagen moment. They hit the benchmark then gave users a more crappy version. Although this could also be a face saving made up reason.
AI CUDA is a good idea. Need AI MindSpore to match them.
AI CUDA is a good idea. Need AI MindSpore to match them.
Last edited:
According to their own blogpost they sort of gamed the benchmark by using cons@64 (consensus result after 64 tries) while the other models are only given a single try. AIME benchmark is only 30 questions so it basically only gets about 1 more question correct than r1 and does about 1 question worse than o3 mini high.
Musk used about 120x more GPUs than DeepSeek yet their Grok is only marginally better than R1..... wtf?!?
Seems like raw brute forcing hit a hard wall of diminishing returns
Probably also why we hadnt had a new GPT version since 2023. And Claude 3.5 OPUS is still nowhere to be found