Artificial Intelligence thread

[9dashline]

might have believed you if ChatGTP didn't completely set the room on fire and send dozens of startup and companies looking for a ChatGTP clone, all at the same time and in a matter of months. PaLM was impressive. Lamda was so good that it made a google researcher claim that it was sentient. GTP-3 made big waves. Deepmind/OpenA.I were openly saying that trillion parameter scale A.I models were how you got to AGI for 2 years now.

So why was the only A.I model that finally shook the chinese A.I sector the one that got so much mainstream media attention that my 63 year old mother who can barely use her smartphone was asking me about it? It's like they only got their A.I news from mainstream media. I'm not working in A.I fields and even I took note of every openA.I/google/deepmind A.I model release or showcase.

Oh please. Just because ChatGPT has garnered media attention and has companies scrambling for a piece of the action, it doesn't mean it's the holy grail of AGI. Fads and hypes are a dime a dozen in the tech world, and ChatGPT is no exception.

The Chinese AI sector isn't simply reacting to mainstream media; they're strategic and savvy in their approach. You're conveniently ignoring the fact that numerous AI developments have been taking place in China, some of which may not be as flashy as ChatGPT but are equally important.

And as for those claims about sentience and AGI from researchers, consider the possibility of exaggeration or even misinformation. It's crucial to separate the wheat from the chaff when dealing with such a complex field as AI.

Just because you've been keeping an eye on AI model releases doesn't mean you have a complete understanding of the state of the field or the path to AGI. So, let's not get carried away with the hype and maintain a healthy dose of skepticism.

 

[tacoburger]

Anyway, A.I news

Please, Log in or Register to view URLs content!

Baidu's ERNIE. Most developed LLM in china right now. Can generate images and video. Meant to be integrated into Baidu's digital products, ecosystem and search engine. Baidu's CEO has stated that there's thousands of potential customers waiting for this A.I.

Please, Log in or Register to view URLs content!

Please, Log in or Register to view URLs content!

New startup by the famous Kai-Fu Lee, aims to produce their own LLM better than ChatGTP or GTP-4. Not much data otherwise.

Please, Log in or Register to view URLs content!

Please, Log in or Register to view URLs content!

A 130B LLM model from Tsinghua University

Please, Log in or Register to view URLs content!

A open-source multimodal LLM by SenseTime, who usually develops A.I vision systems.

Please, Log in or Register to view URLs content!

Meituan cofounder Wang Huiwen has created a startup made to create a LLM.

Please, Log in or Register to view URLs content!

Another new A.I company meant to develop large multimodal AI models

Please, Log in or Register to view URLs content!

Tencent joins the ChatGTP race.

Please, Log in or Register to view URLs content!

Same for Alibaba.

Please, Log in or Register to view URLs content!

Here's ByteDance attempt.

Please, Log in or Register to view URLs content!

Here's JD with a another LLM. "an industrial version of ChatGPT focused on the vertical industrial fields of retail and finance."

Please, Log in or Register to view URLs content!

An LLM A.I meant primarely for use in education . Developed by youdao

Please, Log in or Register to view URLs content!

Netease developing a LLM for use in generating dialogue in games.

Please, Log in or Register to view URLs content!

iFlytek, which offers voice recognition and automatic translation services, said the company has prioritized pushing to develop generative AI pre-training large-scale model in December. Chinese artificial intelligence pioneer iFlytek said ChatGPT-related technology will be first used in its products, such as learning machines, and it will hold a product launch event in May.

Please, Log in or Register to view URLs content!

As you can see, most of there are very new.... Companies like Alibaba, tencent or ByteDance are flush with cash, data, A.I talent and computing power, again, it pisses me off that they didn't see the trend in 2020/2021 and seized it with both fists. Alibaba literally has one of the biggest A.I lab in China and ??? What were doing in the last 2 years?

 

[9dashline]

Anyway, A.I news

Please, Log in or Register to view URLs content!

Baidu's ERNIE. Most developed LLM in china right now. Can generate images and video. Meant to be integrated into Baidu's digital products, ecosystem and search engine. Baidu's CEO has stated that there's thousands of potential customers waiting for this A.I.

Please, Log in or Register to view URLs content!

Please, Log in or Register to view URLs content!

New startup by the famous Kai-Fu Lee, aims to produce their own LLM better than ChatGTP or GTP-4. Not much data otherwise.

Please, Log in or Register to view URLs content!

Please, Log in or Register to view URLs content!

A 130B LLM model from Tsinghua University

Please, Log in or Register to view URLs content!

A open-source multimodal LLM by SenseTime, who usually develops A.I vision systems.

Please, Log in or Register to view URLs content!

Meituan cofounder Wang Huiwen has created a startup made to create a LLM.

Please, Log in or Register to view URLs content!

Another new A.I company meant to develop large multimodal AI models

Please, Log in or Register to view URLs content!

Tencent joins the ChatGTP race.

Please, Log in or Register to view URLs content!

Same for Alibaba.

Please, Log in or Register to view URLs content!

Here's ByteDance attempt.

Please, Log in or Register to view URLs content!

Here's JD with a another LLM. "an industrial version of ChatGPT focused on the vertical industrial fields of retail and finance."

Please, Log in or Register to view URLs content!

An LLM A.I meant primarely for use in education . Developed by youdao

Please, Log in or Register to view URLs content!

Netease developing a LLM for use in generating dialogue in games.

Please, Log in or Register to view URLs content!

iFlytek, which offers voice recognition and automatic translation services, said the company has prioritized pushing to develop generative AI pre-training large-scale model in December. Chinese artificial intelligence pioneer iFlytek said ChatGPT-related technology will be first used in its products, such as learning machines, and it will hold a product launch event in May.

Please, Log in or Register to view URLs content!

Protein Representation Learning via Knowledge Enhanced Primary Structure Modeling​

As you can see, most of there are very new.... Companies like Alibaba, tencent or ByteDance are flush with cash, data, A.I talent and computing power, again, it pisses me off that they didn't see the trend in 2020/2021 and seized it with both fists. Alibaba literally has one of the biggest A.I lab in China and ???

OpenAI used the very same deep learning technique that Google originally published in order to beat them at their own game...ChatGPT is based on Google AI research

How did Deepmind get so far behind when they were king of the world in the AlphaGo era? To the point that they had to go "code red" to catch up to Microsoft?

The AI war is just starting, if you get pissed off by the slightest wind blowing maybe its best to go do something else for a while and later come back to see China take the win...

 

[tacoburger]

The AI war is just starting, if you get pissed off by the slightest wind blowing maybe its best to go do something else for a while and later come back to see China take the win...

It one thing to be behind. It's another when they couldn't even recognize the potential of LLM until it was so mainstream that it couldn't be ignored. Pretty much only Baidu, Huawei and Tsinghua University were making their LLM before ChatGTP came out. Than it turned into more than two dozen companies joining the race after ChatGTP. They slept though GTP-2, GTP-3, Chinchilla, PaLM, Lamda, Megatron, google pathways all without waking up.

And this is LLM. China still doesn't have much in the way of text to video A.I, text to image, text to voice, text to 3D model A.I, deepfake generation development. America beat China to all of those and still has better models to this day. It's pretty clear there some sort of sector wide issue here if America beat China to the punch in all of those and major tech companies had to wait 2 full years before even dipping their toes into LLM development.

Also, the AI war started 2 years with GTP-3, or 4 years ago, with GTP-2. It didn't just start. But it seems like China only just started to race....

How did Deepmind get so far behind when they were king of the world in the AlphaGo era? To the point that they had to go "code red" to catch up to Microsoft?

At least they have something, they did develop Chinchilla, even if it was not as good as GTP series. They also have Gato, which is pretty unique to this day.

OpenAI used the very same deep learning technique that Google originally published in order to beat them at their own game...ChatGPT is based on Google AI research

And google has PaLM, lamda, pathways etc etc. At least they tried. Not not like PaLM or Lamda is bad, they're still really impressive, they beat out GTP-3, ChatGTP is just a different beast. Better than tech giants like Alibaba/tencent/Bytedance with dedicated A.I research arms sleeping on the job for the last 2 years.

 

[9dashline]

It one thing to be behind. It's another when they couldn't even recognize the potential of LLM until it was so mainstream that it couldn't be ignored. Pretty much only Baidu, Huawei and Tsinghua University were making their LLM before ChatGTP came out. Than it turned into more than two dozen companies joining the race after ChatGTP. They slept though GTP-2, GTP-3, Chinchilla, PaLM, Lamda, Megatron, google pathways all without waking up.

And this is LLM. China still doesn't have much in the way of text to video A.I, text to image, text to voice, text to 3D model A.I, deepfake generation development. America beat China to all of those and still has better models to this day. It's pretty clear there some sort of sector wide issue here if America beat China to the punch in all of those and major tech companies had to wait 2 full years before even dipping their toes into LLM development.

Also, the AI war started 2 years with GTP-3, or 4 years ago, with GTP-2. It didn't just start. But it seems like China only just started to race....

At least they have something, they did develop Chinchilla, even if it was not as good as GTP series. They also have Gato, which is pretty unique to this day.

And google has PaLM, lamda, pathways etc etc. At least they tried. Not not like PaLM or Lamda is bad, they're still really impressive, they beat out GTP-3, ChatGTP is just a different beast. Better than tech giants like Alibaba/tencent/Bytedance with dedicated A.I research arms sleeping on the job for the last 2 years.

I already told you for the Nth time that WuDao 2.0 was already way ahead of GPT3

Stop repeating factually false bs about how China is behind and didnt even catch up to ChatGPT 2... you have zero credibility and making an open fool of yourself

 

[tacoburger]

but let's not overlook the fact that human intelligence goes far beyond language

Yes, the brain matter, aka the model and parameter count itself. I'm just saying that a language model is going to be our current best shot at developing AGI, unless some new novel model comes out overnight.

Large language models like Chat-GPT are, quite frankly, limited in their scope. They can spit out human-like responses, but when it comes to genuine reasoning and understanding, they fall short. These models are just glorified pattern matchers, lacking the depth of understanding that defines real intelligence.

Didn't we just go though this? The larger the model starts, the more and more emergent abilities we start to see. GTP-4 could easily have a internal worldview and logic, just not enough for true sentience. No real way to check. GTP-4 does seem to have some sense of reason, it can explain jokes, brand new jokes at that, something that even humans have trouble doing without any context. This are new jokes, how can they pattern match data on new jokes that needs context to find funny? This suggests that GTP-4 has some internal logic and reasoning at play here. Not even near human level, but enough to "understand" some of the meaning and logic behind the words it generates, it does bode well for improved scaling laws, the improvements it has over ChatGTP.

LLM could reach AGI via scaling alone, the only way find out is to develop them and try it.

These models are just glorified pattern matchers, lacking the depth of understanding that defines real intelligence.

You can easily say that for a animal, or a baby. What the difference between them and adult humans? Just the scale of the number and density of neurons in our brains. Didn't we just go though this? It's not like LLM don't spontaneously develop emergent abilities that weren't specifically trained for at certain sizes.

This is impossible to talk about, we legit don't understand our own intelligence, consciousness or brains enough, it's impossible to say how or what an A.I is thinking. It's entirely possible that our own intelligence is nothing but really good pattern recognition that's predicating and selecting our thought based on subconscious factors and knowledge that we have no control over, and that given the same conditions, we will forever act the same and make the same choices. In that case just scale the LLM up and you got a human level intelligence. Or it might not be. We do know that subconscious factors do play a role in thoughts and decision making, but to what degree nobody knows yet.

It's the chinese room issue again. Is consciousness an singular interior experience of understanding, or is consciousness as the emergent result of merely functional non-introspective subconscious components. Which does the A.I experience? Does it matter as long as it acts like it's conscious? Hell whose method do humans experience, if you really boil it down, I'm just a few billion neurons talking to each other fast enough and enough times to create a conscious experience, not that much different from a neural network.

What if an AGI is truly conscious, intelligence and logical but makes mistake after mistake due to lousy training data? It's not like humans are infaillible, far from it, we make mistake after mistake but are still considered super intelligent.

Emotional, social, and spatial intelligence also play a significant role. So even if an LLM can mimic human language, it doesn't mean it has achieved true intelligence.

No A.I is going to think like a human. You don't need emotions, social awareness or whatnot to be intelligent. Any A.I developed, no matter how closely we try to align it to thinking like a human, will be very alien in it's thinking. But all that matters for use is that it's intelligence enough to do shit for humans.

I already told you for the Nth time that WuDao 2.0 was already way ahead of GPT3

You got any other detail on it? How it does on a benchmark? And why did development stop?

 

[Skorf]

I am a final year medical student in a Western country. Most of my later years were focussed on clinical placements rather than studying neuroscience so I am by no means an expert in the field.

I think (and this may be complete garbage) that while dramatically upscaling LLM is unlikely to produce AGI, I am not sure if that means it is not worth attempting. The harm is probably opportunity cost in that the funding for this project could have been used for something else in AI research.

If it fails, you could probably learn something from it, the question is how expensive it will be.

 

[solarz]

Oh please. Just because ChatGPT has garnered media attention and has companies scrambling for a piece of the action, it doesn't mean it's the holy grail of AGI. Fads and hypes are a dime a dozen in the tech world, and ChatGPT is no exception.

The Chinese AI sector isn't simply reacting to mainstream media; they're strategic and savvy in their approach. You're conveniently ignoring the fact that numerous AI developments have been taking place in China, some of which may not be as flashy as ChatGPT but are equally important.

And as for those claims about sentience and AGI from researchers, consider the possibility of exaggeration or even misinformation. It's crucial to separate the wheat from the chaff when dealing with such a complex field as AI.

Just because you've been keeping an eye on AI model releases doesn't mean you have a complete understanding of the state of the field or the path to AGI. So, let's not get carried away with the hype and maintain a healthy dose of skepticism.

Dumb or uneducated people have always mistaken articulation for wisdom. ChatGPT is actually dumb as bricks, but perfectly articulate, which makes it perfect for generating content that doesn't require accuracy or accountability. I.E. marketing.

 

[tacoburger]

I am a final year medical student in a Western country. Most of my later years were focussed on clinical placements rather than studying neuroscience so I am by no means an expert in the field.

I think (and this may be complete garbage) that while dramatically upscaling LLM is unlikely to produce AGI, I am not sure if that means it is not worth attempting. The harm is probably opportunity cost in that the funding for this project could have been used for something else in AI research.

If it fails, you could probably learn something from it, the question is how expensive it will be.

Well said.

Dumb or uneducated people have always mistaken articulation for wisdom. ChatGPT is actually dumb as bricks, but perfectly articulate, which makes it perfect for generating content that doesn't require accuracy or accountability. I.E. marketing.

Are you keeping up here? Yes, ChatGTP isn't AGI, not even close. But it could have some internal form of understanding, reasoning or logic, not enough to be human level. GTP-4 does seem to have some sense of reason or understanding, it can explain jokes, brand new jokes at that, something that even humans have trouble doing without any context. This are new jokes, how can they pattern match data on new jokes that needs context to find funny? This suggests that GTP-4 has some internal logic and reasoning at play here. Not even near human level, but enough to "understand" some of the meaning and logic behind the words it generates.

The true answer is that we have no idea how consciousness or intelligence works, not even in humans, not even in simpler animals. How does a few billion neurons talking to each other produce consciousness, intelligence, memories, self correcting logic or a internal state of mind?

What we do know is that scaling laws do work in nature. A few hundred neurons=Insects. A hundred billion neuron=human. And the whole range of intelligence in between. It's no secret that the great apes all have some of the highest neuron count in the animal kingdom other than us. That's the best theory that we have for biological intelligence. So at some point, adding neurons and synapse turns something that can barely be considered sentient, to a sentient creature but one that doesn't have a theory of mind or intelligence , to full sapient creature with sense of self, logic, intelligence etc etc. Do note that it's entirely possible, even likely that human intelligence is nothing more than very very good pattern recognition and extrapolation, no difference from LLM...only a thousand times better.

It's emergent properties, we don't have special neurons or a vastly different brain structure with other animals or our great ape cousins. If you were some kind of energy being, with no experience with biology, you wouldn't think that a higher neuron count can magically produce humans from great apes, or great apes from rats, or rats from whatever fish crawled out of the oceans. It appears that our brains hit critical mass, the right number of neurons to form the emergent properties that make us intelligence enough to do what we do.

And that's what we see in LLM. The larger they get, not only do they get better at their trained jobs, they gain new abilities that were never trained for, almost like some kind of emergent properties... Like real brains...

Please, Log in or Register to view URLs content!

In addition to these steady quantitative improvements, the scaling process also leads to interesting qualitative behavior. As LLMs are scaled they hit a series of critical scales at which new abilities are suddenly “unlocked”. LLMs are not directly trained to have these abilities, and they appear in rapid and unpredictable ways as if emerging out of thin air. These emergent abilities include performing arithmetic, answering questions, summarizing passages, and more, which LLMs learn simply by observing natural language.

As it turns out, if you scale the Language Model into a Large Language Model, it is capable of performing these tasks without any architectural modifications or task-specific training. LLMs are capable of performing these tasks, sometimes better than specialized, fine-tuned networks, just by phrasing them in terms of natural language.

While the fact that LLMs gain these abilities as they scale is remarkable, it is the manner in which they appear that is especially interesting. In particular, many abilities of Large Language Models appear to be emergent. That is, as LLMs grow in size, they increase from near-zero performance to sometimes state-of-the-art performance at incredibly rapid paces and at unpredictable scales.

So yeah, while unlikely, it's entirely possible that the larger you train a LLM, it could hit critical mass at some point and gain true self awareness or some form of true intelligence. Current models already look like they have level of real understanding of the language that they are using. It will look nothing like a human intelligence, which is going to make it hard to figure out exactly what's going on under the hood. It's not impossible is what I'm trying to said.

LLM are the most likely A.I model to get us AGI right now. It's not like we have a dozen vastly different A.I architecture and transformers and all have a equally good chance at AGI, LLM or a heavily modified version of LLM is the only real game in town for now, unless you come up with new racial architecture or transformer overnight, or even a vastly different method than neural networks.

 

[bobsagget]

As far as i was aware i thought china had the largest model at 175 trillion parameters

Please, Log in or Register to view URLs content!