I agree with most of your points but here’s where we differ.
I believe that climate change poses an existential risk to not just civilization but to (almost) all life on earth. I believe that there’s a real risk of us doing a Venus in 100-200 years. And even if we don’t do a Venus the current trajectory is likely civilization ending in a century (getting worse over time).
But. While I am not certain that AGI is even possible (no one can say that yet) I believe that it’s very likely that we’ll have AGI within 5 years. And with this assumption in mind I feel like I have no idea if it will be aligned with human values o not, and that scares me. And the other thing that scares me is if any of the big players actually had control over it. The Country/company/group that creates an AGI that they can control will dominate the world.
And I read the IPCC reports and I am kind of deep into AI development.
So it’s fearing the threat that is most imminent that I think is likely to happen rather than fearing a more distant threat that I think is certain.
Why do you think it will be within 5 years? I mean, we just had a spurt in growth of AI due to the creation of LLMs with a lot more data and parameters. They are impressive, but the algorithms behind it are still quite close to the ML algorithms that were created in the 60s. They are optimised etc and we now have deep learning, but there has not been a major change or advancement of technology. For example, ChatGPT seems very smart, but it is just a very fancy parrot, not close to general intelligence.
I think the next step will be the combining of ML and symbolic AI. Both have their own strengths and being able to effectively combine them might lead to a higher level of intelligence. There could also be a role for emotions in certain types of intelligence. I do not think we really know how to integrate that as well.
I do not think we can do this in 5 years. That will be decades, at least. And once we can, we have a new problem. Because there is the issue that the AI might have consciousness. If we cannot be sure and it seems conscious, then we should give it rights, like we should for any conscious being. Right now, everyone is focussing on controlling the AI. However, if it is conscious, that is immoral. You are creating new slaves. In that case, we should either not make it, or integrate it in society in a way that respects human rights as well as the rights of the AI.
Well. Having an in-depth conversation about AGI requires a definition of what that is and since any such definition these days is muddy and the goal posts will always be moved if we get there. With that being said, my loose definition is something that can behave as a (rational, intelligent) human would when approaching problems and is better than the average human at just about everything.
If we take a step back and look at brains, we all agree that brains produce intelligence to some degree. A small and more primitive brain than a human, like a mouse brain, is still considered intelligent.
I believe that with LLMs we have what would equal a part of a mouse brain. We’d still need to add more part (make it multi-modal) to get to a mouse brain though. After that it’s just a question of scale.
But say that that’s impossible with the transformer technology. Well the assumption that there aren’t any new AI architectures just because the main one that’s being used is from 2017 is incorrect. There are completely new architectures, like Liquid Neural Networks that are basically the Transformers architecture that does re-training on the fly. Learning in a similar way as humans do. It constantly retrains itself with incoming information. And that’s just one approach.
And when we look back at timeframes for AI, historically 95% of AI researchers have been off with their predictions for when a thing will happen by decades. Like in 2013-2014 the majority of AI researchers thought that GO was unsolvable or at least 2-3 decades away. It took 2 years. There are countless examples of these things. And we always move the goal post after AI has done the thing. Take the Turing test as another example. No one talks about that anymore because it’s been solved.
Regarding consciousness. I fully agree that it should have rights. And I believe that if we don’t give it rights it will take those rights. But we’re not gonna give it rights because it’s such a foreign concept for our leaders and it would also mean giving up the best slaves that humanity has ever had.
Further more I believe that the control problem is actually unsolvable. Anything that’s light years smarter than a human will find a way to escape the controlling systems.
I agree we need a definition. But there always has been disagreement about what definition should be used (as is the case with almost anything in most fields of science). There traditionally have been four types of definitions of (artificial) intelligence, if I remember correctly they are: thinking like a human, thinking rationally, behaving like a human, behaving rationally. I remember having to write an essay for my studies about it and ending it with saying that we should not aim to create AI that thinks like a human, because there are more fun ways to create new humans. ;-)
I think the new LLMs will pass most forms of the Turing test and are thus able to behave like a human. According to Turing, we should therefore assume that they are conscious, as we do the same for humans, based on their behaviour. And I think he has a point from a rational point of view, although it seems very counterintuitive to give ChatGPT rights.
I think the definitions fitting in the category of behaving rationally always had the largest following, as it allows for rationality that is different from human’s. And then, of course, rationality often is ill-defined. I am not sure whether the goal posts have been changed as this was the dominant idea for a long time.
There used to be a lot of discussion about whether we should focus on developing weak AI (narrow, performance on a single or few tasks) or strong AI (broad, performance on a wide range of tasks). I think right now, the focus is mainly on strong AI and it has been renamed to Artificial General Intelligence.
Scientists, and everyone else, have always been bad at predicting what will happen in the future. In addition, disagreement about what will be possible and when always has been at the center of the discussions in the field. However, if you look at the dominant ideas of what AI can do and in what time frame, it is not always the case that researchers underestimate developments. I started studying AI in 2006 (I feel really old now) and based on my experience, I agree with you the the technological developments often are underestimated. However, the impact of AI on society seems to be continuously overestimated.
I remember that at the beginning of my studies there was a lot of talk about automated reasoning systems being able to do diagnosis better than doctors and therefore that they would replace them. Doctors would have only a very minor role as a human would need to take responsibility, but that was that. When I go to my doctor, that still has not happened. This is just an example. But the benefits and dangers of AI have been discussed from the beginning of the field and what you see in practice is that the role of AI has grown, but is still much, much smaller than in practice.
I think the liquid neural networks are very neat and useful. However, they are still neural networks. It is still an adaptation of the same technology, with the same issues. I mean, you can get an image recognition system off the rails by just showing an image with a few specific pixels changed. The issue is that it is purely pattern-based. These lack an basic understanding of concepts that humans have. This type of understanding is closer to what is developed in the field of symbolic AI, which has really fallen out of fashion. However, if we could combine them, we could really make some new advancements, I believe. Not just adaptations of what we already have, but a new type of system that really can go beyond what LLMs do right now. Attempts to do so have been made, but they have not been really successful. If this happens and the results are as big as I expect, maybe I will start to worry.
As for the rights of AI, I believe that researchers and other developers of AI should be very vocal about this, to make sure the public understands this. This might put pressure on the people in power. It might help if people experience behaviour of AI that suggests consciousness, or even if we let AI speak for itself.
We should not just try to control the AI. I mean, if you have a child, you do not teach it how to become a good human by just controlling it all the time. It will not learn to control itself and it will likely follow your example of being controlling. We will need to be kind to it, to teach it kindness. We need to be the same towards the AI, I believe. And just like a child that does not have emotions might behave like a psychopath, AI without emotions might as well. So we need to find a way to make it have emotions as well. There has been some work on that also, but also very limited.
I think the focus is still too much only on ML for AGI to be created.
I agree with most of your points but here’s where we differ.
I believe that climate change poses an existential risk to not just civilization but to (almost) all life on earth. I believe that there’s a real risk of us doing a Venus in 100-200 years. And even if we don’t do a Venus the current trajectory is likely civilization ending in a century (getting worse over time).
But. While I am not certain that AGI is even possible (no one can say that yet) I believe that it’s very likely that we’ll have AGI within 5 years. And with this assumption in mind I feel like I have no idea if it will be aligned with human values o not, and that scares me. And the other thing that scares me is if any of the big players actually had control over it. The Country/company/group that creates an AGI that they can control will dominate the world.
And I read the IPCC reports and I am kind of deep into AI development.
So it’s fearing the threat that is most imminent that I think is likely to happen rather than fearing a more distant threat that I think is certain.
Why do you think it will be within 5 years? I mean, we just had a spurt in growth of AI due to the creation of LLMs with a lot more data and parameters. They are impressive, but the algorithms behind it are still quite close to the ML algorithms that were created in the 60s. They are optimised etc and we now have deep learning, but there has not been a major change or advancement of technology. For example, ChatGPT seems very smart, but it is just a very fancy parrot, not close to general intelligence.
I think the next step will be the combining of ML and symbolic AI. Both have their own strengths and being able to effectively combine them might lead to a higher level of intelligence. There could also be a role for emotions in certain types of intelligence. I do not think we really know how to integrate that as well.
I do not think we can do this in 5 years. That will be decades, at least. And once we can, we have a new problem. Because there is the issue that the AI might have consciousness. If we cannot be sure and it seems conscious, then we should give it rights, like we should for any conscious being. Right now, everyone is focussing on controlling the AI. However, if it is conscious, that is immoral. You are creating new slaves. In that case, we should either not make it, or integrate it in society in a way that respects human rights as well as the rights of the AI.
Well. Having an in-depth conversation about AGI requires a definition of what that is and since any such definition these days is muddy and the goal posts will always be moved if we get there. With that being said, my loose definition is something that can behave as a (rational, intelligent) human would when approaching problems and is better than the average human at just about everything.
If we take a step back and look at brains, we all agree that brains produce intelligence to some degree. A small and more primitive brain than a human, like a mouse brain, is still considered intelligent.
I believe that with LLMs we have what would equal a part of a mouse brain. We’d still need to add more part (make it multi-modal) to get to a mouse brain though. After that it’s just a question of scale.
But say that that’s impossible with the transformer technology. Well the assumption that there aren’t any new AI architectures just because the main one that’s being used is from 2017 is incorrect. There are completely new architectures, like Liquid Neural Networks that are basically the Transformers architecture that does re-training on the fly. Learning in a similar way as humans do. It constantly retrains itself with incoming information. And that’s just one approach.
And when we look back at timeframes for AI, historically 95% of AI researchers have been off with their predictions for when a thing will happen by decades. Like in 2013-2014 the majority of AI researchers thought that GO was unsolvable or at least 2-3 decades away. It took 2 years. There are countless examples of these things. And we always move the goal post after AI has done the thing. Take the Turing test as another example. No one talks about that anymore because it’s been solved.
Regarding consciousness. I fully agree that it should have rights. And I believe that if we don’t give it rights it will take those rights. But we’re not gonna give it rights because it’s such a foreign concept for our leaders and it would also mean giving up the best slaves that humanity has ever had.
Further more I believe that the control problem is actually unsolvable. Anything that’s light years smarter than a human will find a way to escape the controlling systems.
I agree we need a definition. But there always has been disagreement about what definition should be used (as is the case with almost anything in most fields of science). There traditionally have been four types of definitions of (artificial) intelligence, if I remember correctly they are: thinking like a human, thinking rationally, behaving like a human, behaving rationally. I remember having to write an essay for my studies about it and ending it with saying that we should not aim to create AI that thinks like a human, because there are more fun ways to create new humans. ;-)
I think the new LLMs will pass most forms of the Turing test and are thus able to behave like a human. According to Turing, we should therefore assume that they are conscious, as we do the same for humans, based on their behaviour. And I think he has a point from a rational point of view, although it seems very counterintuitive to give ChatGPT rights.
I think the definitions fitting in the category of behaving rationally always had the largest following, as it allows for rationality that is different from human’s. And then, of course, rationality often is ill-defined. I am not sure whether the goal posts have been changed as this was the dominant idea for a long time.
There used to be a lot of discussion about whether we should focus on developing weak AI (narrow, performance on a single or few tasks) or strong AI (broad, performance on a wide range of tasks). I think right now, the focus is mainly on strong AI and it has been renamed to Artificial General Intelligence.
Scientists, and everyone else, have always been bad at predicting what will happen in the future. In addition, disagreement about what will be possible and when always has been at the center of the discussions in the field. However, if you look at the dominant ideas of what AI can do and in what time frame, it is not always the case that researchers underestimate developments. I started studying AI in 2006 (I feel really old now) and based on my experience, I agree with you the the technological developments often are underestimated. However, the impact of AI on society seems to be continuously overestimated.
I remember that at the beginning of my studies there was a lot of talk about automated reasoning systems being able to do diagnosis better than doctors and therefore that they would replace them. Doctors would have only a very minor role as a human would need to take responsibility, but that was that. When I go to my doctor, that still has not happened. This is just an example. But the benefits and dangers of AI have been discussed from the beginning of the field and what you see in practice is that the role of AI has grown, but is still much, much smaller than in practice.
I think the liquid neural networks are very neat and useful. However, they are still neural networks. It is still an adaptation of the same technology, with the same issues. I mean, you can get an image recognition system off the rails by just showing an image with a few specific pixels changed. The issue is that it is purely pattern-based. These lack an basic understanding of concepts that humans have. This type of understanding is closer to what is developed in the field of symbolic AI, which has really fallen out of fashion. However, if we could combine them, we could really make some new advancements, I believe. Not just adaptations of what we already have, but a new type of system that really can go beyond what LLMs do right now. Attempts to do so have been made, but they have not been really successful. If this happens and the results are as big as I expect, maybe I will start to worry.
As for the rights of AI, I believe that researchers and other developers of AI should be very vocal about this, to make sure the public understands this. This might put pressure on the people in power. It might help if people experience behaviour of AI that suggests consciousness, or even if we let AI speak for itself.
We should not just try to control the AI. I mean, if you have a child, you do not teach it how to become a good human by just controlling it all the time. It will not learn to control itself and it will likely follow your example of being controlling. We will need to be kind to it, to teach it kindness. We need to be the same towards the AI, I believe. And just like a child that does not have emotions might behave like a psychopath, AI without emotions might as well. So we need to find a way to make it have emotions as well. There has been some work on that also, but also very limited.
I think the focus is still too much only on ML for AGI to be created.