Why do we even bother with this when we all know where Western chatbots stand on the Palestinian genocide.
Artificial Intelligence thread
- Thread starter 9dashline
- Start date
Why do we even bother with this when we all know where Western chatbots stand on the Palestinian genocide.
English is an interesting language:
1. Regarding its vocabulary, it is the most indo-european language of all because has large amount of words from other European languages: by origin is a Germanic language, but half of its vocabulary is from Latin or strictly derived languages (French/Spanish/etc) {f.i. in the sentence I have just written above, these are the words with Latin origin: vocabulary, language, large, amount, Germanic, strictly, derived, sentence]. It has also many words from Greek: Europe, music, cemetery, galaxy, democracy, etc. For historical reason, it has much less from the other big branch of the European family, that is Slavic languages.
2. Instead regarding it's grammar, it is totally different from the others. It is much, much simplified. The original proto-indo-european language was quite a complex one, with 9 cases and a lot of stuff. Today the slavic languages keep a grammar similar to the orginal one, in the ancient times also Latin and Greek had a similar complex grammar, the other modern European languages simplifed it somehow...but English language is the only one to have totally trivialized it. This is definitely a bonus for people that has to learn it: I knew a dutch man that told me that New York was a ducth town in origin and that only by chance US choose English as their language and not Dutch....well it was definetly a big luck for all the world
ducth is way harder.3. Finally let me add what is probably the most peculiar aspect of the language: the pronunciation/spell of the words. for some unfathomable reason, somehow during middle age period, English people decided that they had to pronounce words differently from how they are written. It is the only European language that took this apparently uncomfortable road. In all the other languages you can just read the words as they are written, mainly because the "written form" was invented just for that....but in Great Britain they begged to differ.
Last edited:
I'm not involved in the research side of things, Will have to ask Steve Hsu on this.
I would assume OpenAI has been scraping Chinese internet for as much as data as they can access.
That just confirms the server which hosts "chat.deepseek.com" is located inside China
DeepSeek guys used some low level assembly "Shifts Registers" tinkering to highly optimize the training of their models.
Does this imply that cutting-edge LLM development no longer needs large-scale GPU clusters? Were the massive computing investments by Google, OpenAI, Meta, and xAI ultimately futile?
The prevailing consensus among AI developers is that this is not the case. However, it is clear that there is still much to be gained through data and algorithms, and many new optimization methods are expected to emerge in the future. Since DeepSeek’s V3 model was released as open source, the technical report on V3 has been described in great detail. This report documents the extent of low-level optimizations performed by DeepSeek. In simple terms, the level of optimization could be summed up as “it seems like they rebuilt everything from the ground up.” For example, when training V3 with NVIDIA’s H800 GPUs, DeepSeek customized parts of the GPU’s core computational units, called SMs (Streaming Multiprocessors), to suit their needs. Out of 132 SMs, they allocated 20 exclusively for server-to-server communication tasks instead of computational tasks.
This customization was carried out at the PTX (Parallel Thread Execution) level, a low-level instruction set for NVIDIA GPUs. PTX operates at a level close to assembly language, allowing for fine-grained optimizations such as register allocation and thread/warp-level adjustments. However, such detailed control is highly complex and difficult to maintain. This is why higher-level programming languages like CUDA are typically used, as they generally provide sufficient performance optimization for most parallel programming tasks without requiring lower-level modifications. Nevertheless, in cases where GPU resources need to be utilized to their absolute limit and special optimizations are necessary, developers turn to PTX. This highlights the extraordinary level of engineering undertaken by DeepSeek and demonstrates how the “GPU shortage crisis,” exacerbated by U.S. sanctions on China, has spurred both urgency and creativity.
That means that may also bypassed CUDA which could open the door for other GPUs makers like Huawei Ascend or Moore Thread to be used in the training process.
????? Ascend has been used in training by quite a few companies, what are you talking about?
I mean, they can tune Ascend chips that they acquire in the same way. There is nothing magical about this.
Kimi 1.5 by Moonshot is out and it's sad that they are probably not going to get much recognition for this because DeepSeek is sucking up all the oxygen.
so Steve says that using Chinese language probably does help, but he is not sure it actually helps all that much.
My NVIDIA investment is cooked but I’ve never been so happy about it.
Feel bad for Kimi. They keep reminding us that their model is out, it's a great model, but got completely overshadowed by DeepSeek. Shows that in life a lot is just timing/luck. Hopefully they get the recognition they deserve in the West eventually.
-----
CS professor complains that DeepSeek is not open enough to be called truly open source.
OK, but they are still sharing more information than most frontier model papers from closed labs. Also, they still need some amount of moat, no? I think they strike a nice balance.
-----
CS professor complains that DeepSeek is not open enough to be called truly open source.
OK, but they are still sharing more information than most frontier model papers from closed labs. Also, they still need some amount of moat, no? I think they strike a nice balance.
This is actually pretty big deal. This is first time an open weight model reached 1 million context length window, all prior max out at 128k
This means enterprises can do local inferencing with a much more use case etc and coupled with workflows of Qwen 1million + DS R1, it can now top and surpass the Chatgpt o1 Pro in many meaningful ways (see "Claude + R1" > o1 pro, but Claude isnt open weight etc)
While Qwen had this for a while, they kept it behind gated API... with this release its a new game changing for long context workload and use cases... will probably get installed in a lot of enterprised worldwide
Coordinated or not, feels like China is doing a double tap rugpull on US ClosedAI etc
Dang that was quick...
Last edited: