Check back with me on this in Sept 2026 lol, we shall see who was right.
Artificial Intelligence thread
- Thread starter 9dashline
- Start date
Just to rant a bit on this topic:
Putting it very simply, LLMs, fundamentally, simply predicts what is the most likely/suitable response from it's in-built knowledge trained from vast, vast datasets scoured from the Internet. Modern complex LLMs are engineered to be able to learn patterns and relationships within the data, so for example:
1. "The cat sat on the mat in the living room."
2. "The CEO sat on the board in the company."
Despite being different, the LLM would recognize the pattern "[Subject] + [verb in past tense] + on the [noun] + in the [location]" and would be able to answer correctly the question "What is sitting on what?", despite probably never actually seeing that exact sentence and answer in it's dataset.
However, the LLM certainly is not aware and definitely does not understand what I am asking it. The most obvious example is asking any LLMs the question "How many r's are there in 'Strawberry'?". The AI will answer '2', which is wrong!
Why? Partially because the nature of LLMs require the question to be broken into tokens, which could and would warp the context that it is processing on (breaking "Strawberry" into "straw" and "berry", for example), but more importantly it is because LLMs fundamentally does not understand the question! LLMs does not possess the ability to "step back" and examine a word letter by letter, or truly understand the concept of counting letters. They're simply regurgitating what they have calculated to be the most probable response based on their training data and the patterns they've learned.
LLMs assume things, but they never question whether their assumptions are correct, because they don't understand what is correct or wrong in different contexts. Thus, hallucinations and garbage responses.
This limitation extends to many other types of questions and tasks. While LLMs can often produce impressively human-like responses, they're fundamentally pattern-matching and prediction engines, completely incapable of genuine comprehension or reasoning - therefore never ever replacing humanity in software development or other fields.
Now, newer models like ClosedAI's o1 can somewhat emulate higher order reasoning to improve it's coding effectiveness, but it isn't and will never be capable of independently recognize that despite following the logical steps it generated to the letter, what it produces is rubbish and useless in real life. (Just tried asking the preview model to solve some coding issues of mine, and it managed to distill the problem correctly into steps which needs to be solved, then proceeded to produce code that doesn't work at all. Does it realize that the code doesn't work? Does it understand what the problem actually is? No, of course not.)
Honestly, the best argument against "AI will replace X" is actually trying to do X with AI for a period of time.
Putting it very simply, LLMs, fundamentally, simply predicts what is the most likely/suitable response from it's in-built knowledge trained from vast, vast datasets scoured from the Internet. Modern complex LLMs are engineered to be able to learn patterns and relationships within the data, so for example:
1. "The cat sat on the mat in the living room."
2. "The CEO sat on the board in the company."
Despite being different, the LLM would recognize the pattern "[Subject] + [verb in past tense] + on the [noun] + in the [location]" and would be able to answer correctly the question "What is sitting on what?", despite probably never actually seeing that exact sentence and answer in it's dataset.
However, the LLM certainly is not aware and definitely does not understand what I am asking it. The most obvious example is asking any LLMs the question "How many r's are there in 'Strawberry'?". The AI will answer '2', which is wrong!
Why? Partially because the nature of LLMs require the question to be broken into tokens, which could and would warp the context that it is processing on (breaking "Strawberry" into "straw" and "berry", for example), but more importantly it is because LLMs fundamentally does not understand the question! LLMs does not possess the ability to "step back" and examine a word letter by letter, or truly understand the concept of counting letters. They're simply regurgitating what they have calculated to be the most probable response based on their training data and the patterns they've learned.
LLMs assume things, but they never question whether their assumptions are correct, because they don't understand what is correct or wrong in different contexts. Thus, hallucinations and garbage responses.
This limitation extends to many other types of questions and tasks. While LLMs can often produce impressively human-like responses, they're fundamentally pattern-matching and prediction engines, completely incapable of genuine comprehension or reasoning - therefore never ever replacing humanity in software development or other fields.
Now, newer models like ClosedAI's o1 can somewhat emulate higher order reasoning to improve it's coding effectiveness, but it isn't and will never be capable of independently recognize that despite following the logical steps it generated to the letter, what it produces is rubbish and useless in real life. (Just tried asking the preview model to solve some coding issues of mine, and it managed to distill the problem correctly into steps which needs to be solved, then proceeded to produce code that doesn't work at all. Does it realize that the code doesn't work? Does it understand what the problem actually is? No, of course not.)
Honestly, the best argument against "AI will replace X" is actually trying to do X with AI for a period of time.
Last edited:
IMHO when they will be able to fix open issues, that will be a big day.
I'm not talking of single models, like GPT4, but of dedicated systems designed just for code engineering. One example is Delvin. If you look at it, it is way more than a wrapper to call a LLM. They are still not there, but in the last few months they have improved a lot, and they are not the only ones, many startups work on it.
When Delvin or someone of its sibling will be able to read a well written open issue (with clear bug description and steps to reproduce) and will be able to open a corresponding pull-request, that will be very useful. Debugging is a long and tedious process, actually is the biggest waste of time for developers.
Maybe it will be able to successfully manage only 50% of tickets, maybe only 30%....that is already huge and worth its price.
P.S: If a bug is reproducible, of course not only is easier to fix, but Delvin will have a powerful feedback signal, when runs the patched code.
P.P.S Someone will still have to write the bug report and approve the patch though
....
What happened to checking back with you at the end of August 2022 when you were going to leave the forum if the US didn't launch a war on China?
Miscellaneous News
Special Operation to retrieve stolen goods from the British Museum. Technically the PLAN have operated in the Thames before, when relations hadn't turned sour:
www.sinodefenceforum.com
AGI is a loaded word. Something like GPT5 may appear to user as AGI and do better on IQ test, but at the end of the day, it is just a really advanced prediction model.
I agree with this tweet
Understand what it is and you can apply it in powerful usage cases.
right now, learning how to prompt LLM is one of the hardest things to do.
Let's say you have developed model capable of developing advanced C++ program. How would you be able to tell model to develop the program exactly how you want to do it and then to build unit tests, debug code and hunt for memory leaks without actually having a deep knowledge of how to write complicated program in the first place?
And how do you develop and retain that knowledge without spending time programming?
I think you can use Test Driven Development. You, and the client and the AI get together and write the tests first, the idea is at the beginning all the test fail, because there is no code, and at the end all the tests pass.
Once the tests have been defined the AI writes the code so that the unit tests pass, once they all do you have a working product.
Real test driven development is seldom used as far as I know and off course anecdotal experience.... At least I don't know anyone who did this for real.
Once tried to use behavior based development with cucumber and gherkin that was pretty close to prompting for analist. We did that for a couple a month and said fuck this. Only good thing from that experiment was jira tickets being written in a gherkin like way.
TDD doesn't work unless you know exactly what your end result is going to look like, technically speaking, and exactly how it's going to work. Which is way too rigid in practice, there's always ambiguities and scope creep. So you wind up rewriting all your tests to fit the actual end result, and in that case why bother.
I've not seen anyone approach the "LLMs can't fully replace humans" from this angle, but the halting problem guarantees (at least with regards to anything that's a computer [no brains in jars]) that no program (which LLMs are) will be able to read any code and decide if the code works as intended.
Granted, it might be able to come close to deciding (it might give one of those '99% probability' claims) or be able to operate on a subset of possible code or inputs, but this is all still quite the ways away (as evidenced by LLMs producing code that doesn't even run, let alone run as intended). By the time it does come to pass, the only people producing 'meaningful' work will be mathematicians.
One of the big problems with LLMs 'coding' is that they're trained on the gibberish code you find online about "Learn how to make animated buttons in HTML" or "Rainbow in CSS". I'll be impressed once someone builds an LLM that can reliably produce meaningful contributions to the Linux kernel or produce a driver for new graphics cards.
Granted, it might be able to come close to deciding (it might give one of those '99% probability' claims) or be able to operate on a subset of possible code or inputs, but this is all still quite the ways away (as evidenced by LLMs producing code that doesn't even run, let alone run as intended). By the time it does come to pass, the only people producing 'meaningful' work will be mathematicians.
One of the big problems with LLMs 'coding' is that they're trained on the gibberish code you find online about "Learn how to make animated buttons in HTML" or "Rainbow in CSS". I'll be impressed once someone builds an LLM that can reliably produce meaningful contributions to the Linux kernel or produce a driver for new graphics cards.
Guys just admit it, and give it up, AGI will be here within 18 months then ASI will be right around the corner. Debating whether AI can replace a human coder is already a moot point. Coding itself is but a human machine abstraction interface...a middle man tier that itself is not too much longer for this world.