News on China's scientific and technological development.

luminary said:

I'm convinced that ChatGPT is less of an immediate product and more of a media stunt to prop up public confidence and investment in flagging tech companies. Right now Silicon Valley is still hemorrhaging layoffs with no end in sight.

MSM has been going on about "paradigm shift" and "replace all human programmers" but there are dedicated AI services like GitHub Copilot that have been around since 2021...

Click to expand...

I'm a human programmer

I have been hearing this from managers since like 2015 and it has never manifested. Writing the code for 99% of the companies is not the issue or hardest part of the job. For 99% of the companies the hardest part of the job is getting management to sign off certain projects, refactoring code bases or complete rewrites and off course office politics.

Github copilot is actually a good example of using AI to boost programmers productivity, I imaging if ChatGPT can be used in such a way it would actually add to western productivity. But i'm more convinced it will be used to write web nonsense and produce a shit load more garbage data.

tacoburger said:

If that's the case, why didn't chinese researchers create the first usable large deep learning models? Maybe because research papers don't cost much but training A.I models with billions of parameters cost a lot.

Click to expand...

You insinuated that China does not innovate in AI and I proved that it is false, why change the topic to cost, lol? ResNet is a transformative work that allowed models like ChatGPT to exist in the first place, same as Transformer. They are as important as the fundamental physics behind e.g. semiconductors, i.e. the latter exists because the former was discovered. I don't disagree with increasing funding but that argument is bogus.

How do you define "transformative" btw? From what I see, you define it by the amount of hype and buzz generated by the media and laymen rather than innovativeness or the like because both examples provided by you are essentially wrappers around the existing models, GPT-3 and CLIP (for DALL-E) where the former was already surpassed by WuDao in scale. Btw, the aforementioned ERNIE-ViLG is not second "DALL-E", it is stable diffusion (good example of innovation rather than "large model go brrr") model and introduces a novel end-to-end training framework.

tacoburger said:

If this was the case, than China won't be the runner up in most of this technologies, only a few months behind in some cases, beating out Europe. It tells me that China has the capabilities to actually be first, but won't actually put in resources until it's proven to be viable. If development and wealth accumulation was all that mattered, than european countries will be the runner up, with China a distant 3rd.

Click to expand...

China's runner up status fits the ranking of global R&D spending

tonyget said:

Why doesn't Chinese companies come out with some novel algorithms？Google come out with the Transformer，OpenAI come out with GPT，where is Chinese companies algorithms？

Click to expand...

BAAI's Wudao - introduced a new training method for large scale NLP models, which was also the main point behind GPT, it is larger than GPT-3 btw; Baidu's ERNiE-ViLG - new training regimen and bidirectiobal generator, which allowed it to greatly surpass DALL-E in zero-shot learning in terms of FID. So yeah, the issue is not that Chinese companies don't come up with novelties, you just don't hear about them much from the English-language media. Overall, true - Chinese companies need to catch up in producing not just production-ready models.

BlackWindMnt said:

What does it usually produce is more data in the metaverse but is chatgpt used to boost real economic productive forces because what i can follow from China is that they usually use tech improvement to boost productive forces and care less about boosting the metaverse.

Click to expand...

If someone use to write a long article in a day，now able to do it under an hour with the help of ChatGPT，that's productivity increase

tonyget said:

If someone use to write a long article in a day，now able to do it under an hour with the help of ChatGPT，that's productivity increase

Click to expand...

I think you pretty much have to be a subject matter expert before you can really start exploiting ChatGPT well. If you're not well versed in the material you most likely need to double or triple check everything ChatGPT produces for you. I can see experienced writers use ChatGPT to create the outlines for their initial draft. Just like i expect artist to use AI as some sort of initial source of inspiration or template.

It's kind of the same thing with programming you already need to know what you need to search for and what keywords to use.
Also given the tooling we programmers have at our disposal we pretty much get instant feedback on what ChatGPT produced for you if it can be used or not.

BlackWindMnt said:

I think you pretty much have to be a subject matter expert before you can really start exploiting ChatGPT well. If you're not well versed in the material you most likely need to double or triple check everything ChatGPT produces for you. I can see experienced writers use ChatGPT to create the outlines for their initial draft. Just like i expect artist to use AI as some sort of initial source of inspiration or template.

It's kind of the same thing with programming you already need to know what you need to search for and what keywords to use.
Also given the tooling we programmers have at our disposal we pretty much get instant feedback on what ChatGPT produced for you if it can be used or not.

Click to expand...

Exactly that. ChatGPT and AI art have severe flaws that are quickly apparent to SMEs and are more useful as drafting/inspiration tools than as actual production work. Those flaws are also fundamental to the way they work so its not like simply more time and effort will resolve them.

Perhaps there will be some research on a new model / approach that will overcome those issues, but right now they can't replace anyone but the least skillful workers and function, like you said, as augments.

tonyget said:

Why doesn't Chinese companies come out with some novel algorithms？Google come out with the Transformer，OpenAI come out with GPT，where is Chinese companies algorithms？

Click to expand...

From @xypher:

And large natural language/multi-modal models like GPT/DALL-E would not be possible without Chinese image model ResNet which allowed to build such deep models in the first place by introducing the idea of residual/skip connections to effectively propagate gradients to deeper layers. This finally allowed to match the theory to practice because previous models all suffered from decaying performance with model growth despite the theory stating that deeper models should be at least as good as shallow models. It is the most cited DL research paper, far ahead of transformers - the basis of GPT and other modern NLP models (also uses residual connections btw).

Click to expand...

xypher said:

And large natural language/multi-modal models like GPT/DALL-E would not be possible without Chinese image model ResNet which allowed to build such deep models in the first place by introducing the idea of residual/skip connections to effectively propagate gradients to deeper layers. This finally allowed to match the theory to practice because previous models all suffered from decaying performance with model growth despite the theory stating that deeper models should be at least as good as shallow models. It is the most cited DL research paper, far ahead of transformers - the basis of GPT and other modern NLP models (also uses residual connections btw).

Click to expand...

The problem, as I said earlier, is that while China has plenty of AI researchers, it doesn't have the same energy or money in its AI start up scene, compared to the West. Or at least, it didn't until recently.

Companies like Baidu, Tencent, Alibaba are similar to Google, Facebook, and Microsoft in that they are highly hierarchical, political, and bureaucratic corporations, where a lot of nothing happens until a disruptive trend from a start-up forces them to either buy out the start-up, or be a "fast follower" in the technology. You can also throw in, in this group, South Korean companies like Samsung and Japanese companies like Sony.

That's why they were all beaten by Open AI, a company of merely 350 people. Microsoft at least had the foresight to invest in Open AI, but even then, why couldn't they build it themselves? Whey did Open AI and Stability AI beat them to market on ChatGPT, StableDiffusion, and Dall-E? It's the same reason Chinese companies like Baidu, Tencent, and Alibaba fail to be first to market. Corporations of that size just can't innovate at the same speed. Too much internal bureaucracy and corporate paralysis.

China needs more companies in the AI space like Byte Dance, DJI, and Mihoyo - unicorns that can move quickly and without the usual regulatory and bureaucratic bull **** that paralyze the big corporations. It must NOT become like Japan and South Korea where the economy is monopolized by such corporations.

Large corporations and consolidation are needed in sectors like chips manufacturing and heavy industries because of the massive over head costs and generally closed technology systems. But in AI and software, start ups rule. So what I'll be looking for from the Chinese government is the ability to distinguish between these two sectors. Xi is trying to bring more of the economy under state control; that could be a positive for advanced manufacturing industries, but it could be a negative for software and AI companies, which benefit from being less centralized.

Chinese Academy of Sciences: The gap between China and ChatGPT is only a few months, and the lagging behind is the large model data