People when the sound like a person machine sounds like a person

https://gfycat.com/corruptmetallicitalianbrownbear

one for the boomers

look I don’t want to shock you but that’s basically what they get paid to do. and (perverse) incentives apply - of course goog isn’t just going to spend a couple decabillion then go “oh shit, hmm, we’ve reached the limits of what this can do. okay everyone, pack it in, we’re done with this one!”, they’re gonna keep trying to milk it to make some of those decabillions back. and there’s plenty of useful suckers out there

a lot of corporations involved with AI are doing their damndest to damage our relationship with the scientific process by releasing as much fluff disguised as research as they can manage, and I really feel like it’s a trick they learned from watching cryptocurrency projects release an interminable amount of whitepapers (which, itself, damaged our relationship with and expectations from the engineering process)

Just I think to summarize your beliefs: rationalists are wrong about a lot of things and assholes. And also the singularity (which predates yuds existence) is not in fact possible by the mechanism I outlined.

I think this is a big crux here. It’s one thing if its a cult around a false belief. It’s kind of a problem to sneer at a cult if the core S of it happens to be a true law of nature.

Or an analogy. I think gpt-4 is like the data from the Chicago pile. That data was enough to convince the domain experts then a nuke was going to work to the point they didn’t test Fat Man, you believe not. Clearly machine generality is possible, clearly it can solve every problem you named including, with the help of humans, ordering every part off digikey and loading the pick and place and inspecting the boards and building the wire harnesses and so on.

take some time and read this

I read it. I appreciated the point that human perception of current AI performance can scam us, though this is nothing new. People were fooled by Eliza.

It’s a weak argument though. For causing an AI singularity, functional intelligence is the relevant parameter. Functional intelligence just means “if the machine is given a task, what is the probability it completes the task successfully”. Theoretically an infinite chinese room can have functional intelligence (the machine just looks up the sequence of steps for any given task).

People have benchmarked GPT-4 and it’s got general functional intelligence at tasks that can be done on a computer. You can also just go pay up $20 a month and try it. It’s below human level overall I think, but still surprisingly strong given it’s emergent behavior from computing tokens.

tbh I read the statement about epistemic status as ironic. I was disabused of this notion rapidly.

A bad habit rationalism teaches is to treat a stock verbiage of polite and open discussion as on the one hand (a) integral to productive conversation, and (b) automatically generative of productive conversation. But people aren’t like that, because people are in general really smart listeners (and readers) when it comes to figuring out what is stock verbiage and what is meant in earnest.

Thanks for articulating this.

we held off for a bit cos we don’t want to be actively unkind to the recovering rationalists, and he was our first ardent debate bro actually on the instance, but he rapidly also became our first 24 hour ban of a local account rather than a federated one. perhaps his posting will improve a day hence!

I appreciated this post because it never occurred to me that the “thumb might be on the scales” for the “rules for discourse” that seems to be the norm around the rat forms. I personally ignore most of it, however, the “ES” rat phrase is simply saying, “I know we humans are biased observers, this is where I’m coming from”. If the topic were renewable energy and I was the ‘head of extraction at BP’, you can expect that whatever I have to say is probably biased against renewable energy.

My other thought reading this was : what about the truth. Maybe the mainstream is correct about everything. “Sneer club” seems to be mostly mainstream opinions. That’s fine I guess but the mainstream is sometimes wrong about issues that have been poorly examined or near future events. The collective opinions of everyone don’t really price in things that are about to happen, even if it’s obvious to experts. For example, the mainstream opinion on covid was usually lagging several weeks behind Zvi’s posts on lesswrong.

Where I am going with this is you can point out bad arguments on my part, but I mean in the end, does truth matter? Like are we here to score points on each other or share what we think reality is or will in the very near future be?

Well, that at least convinces me that a robot will replace you.

i am sincerely convinced that VCs who fearmonger about AI are worried that GPT-3 would already do twitter better than them

1, 2 : since you claim you can’t measure this even as a thought experiment, there’s nothing to discuss 3. I meant complex robotic systems able to mine minerals, truck the minerals to processing plants, maintain and operate the processing plants, load the next set of trucks, the trucks go to part assembly plants, inside the plant robots unload the trucks and feed the materials into CNC machines and mill the parts and robots inspect the output and pack it and more trucks…culminating in robots assembling new robots.

It is totally fine if some human labor hours are still required, this cheapens the cost of robots by a lot.

4. This is deeply coupled to (3). If you have cheap robots, if an AI system can control a robot well enough to do the task as well as a human, obviously it’s cheaper to have robots do the task than a human in most situations.

Regarding (3) : the specific mechanism would be AI that works like this:

Millions of hours of video of human workers doing tasks in the above domain + all video accessible to the AI company -> tokenized compressed description of the human actions -> llm like model. The llm like model thus is predicting “what would a human do”. You then need a model to transform the what to robotic hardware that is built differently than humans, and this is called the “foundation model”: you use reinforcement learning where actual or simulated robots let the AI system learn from millions of hours of practice to improve on the foundation model.

The long story short of all these tech bro terms is robotic generality - the model will be able to control a robot to do every easy or medium difficulty task, the same way it can solve every easy or medium homework problem. This is what lets you automate (3), because you don’t need to do a lot of engineering work for a robot to do a million different jobs.

Multiple startups and deepmind are working on this.

Having trouble with quotes here **I do not find likely that 25% of currently existing occupations are going to be effectively automated in this decade and I don’t think generative machine learning models like LLMs or stable diffusion are going to be the sole major driver of that automation. **

1. I meant 25% of the tasks, not 25% of the jobs. So some combination of jobs where AI systems can do 90% of some jobs, and 10% of others. I also implicitly was weighting by labor hour, so if 10% of all the labor hours done by US citizens are driving, and AI can drive, that would be 10% automation. Does this change anything in your response?

No. Even if Skynet had full control of a robot factory, heck, all the robot factories, and staffed them with a bunch of sleepless foodless always motivated droids, it would still face many of the constraints we do. Physical constraints (a conveyor belt can only go so fast without breaking), economic constraints (Where do the robot parts and the money to buy them come from? Expect robotics IC shortages when semiconductor fabs’ backlogs are full of AI accelerators), even basic motivational constraints (who the hell programmed Skynet to be a paperclip C3PO maximizer?)

2. I didn’t mean ‘skynet’. I meant, AI systems. chatGPT and all the other LLMs are an AI system. So is midjourney with controlnet. So humans want things. They want robots to make the things. They order robots to make more robots (initially using a lot of human factory workers to kick it off). Eventually robots get really cheap, making the things humans want cheaper and that’s where you get the limited form of Singularity I mentioned.

At all points humans are ordering all these robots, and using all the things the robots make. An AI system is many parts. It has device drivers and hardware and cloud services and many neural networks and simulators and so on. One thing that might slow it all down is that the enormous list of IP needed to make even 1 robot work and all the owners of all the software packages will still demand a cut even if the robot hardware is being built by factories with almost all robots working in it.

**I just think the threat model of autonomous robot factories making superhuman android workers and replicas of itself at an exponential rate is pure science fiction. **

3. So again that’s a detail I didn’t give. Obviously there are many kinds of robotic hardware, specialized for whatever task they do, and the only reason to make a robot humanoid is if it’s a sexbot or otherwise used as a ‘face’ for humans. None of the hardware has to be superhuman, though obviously industrial robot arms have greater lifting capacity than humans. Just to give a detail what the real stuff would look like : most robots will be in no way superhuman in that they will lack sensors where they don’t need it, won’t be armored, won’t even have onboard batteries or compute hardware, will miss entire modalities of human sense, cannot replicate themselves, and so on. It’s just hardware that does a task, made in factory, and it takes many factories with these machines in it to make all the parts used.

think:

I don’t really see much likelihood in a singularity though, there’s probably a bunch of useful shit you could work out if you analysed the right extant data in the right way but there’s huge amounts of garbage data that it’s not obvious is garbage.

My experience in research indicates to me that figuring shit out is hard and time consuming, and “intelligence” whatever that is has a lot less to do with it than having enough resources and luck. I’m not sure why some super smart digital mind would be able to do science much faster than humans.

Physics is a bitch and there are just sort of limits on how awesome technology can be. Maybe I’m wrong but it seems like digital intelligence would be more useful for stuff like finding new antibiotics than making flying nanomagic fabricator paperclip drones.

You know, I thought that moving sneerclub onto lemmy meant we probably would not get that familiar mix of rationalists, heterodox rationalists, and just-left-but-still-mired-in-the-mindset ex-rationalists that swing by and want to quiz sneerclub. Maybe we’re just that irresistible.

from 2011-2013 i was getting these guys email me directly about roko’s basilisk because lesswrong had banned discussion and rationalwiki was the only place even mentioning it

now they work hard to seek us out even here

i hope the esteemed gentleposter realises that there are no recoverable good parts and it’s dumbassery all the way down sooner rather than later, preferably before posting again

And just to be clear, for one to be “lost in the AI religion”, the claims have to be false, correct? We will not see the things I mentioned within the timeframe I gave (7 years, 17 years, and implicitly if there is not immediate progress towards the nearer deadline within 1 year it’s not going to happen).

Google’s Gemini will not be multimodal, be capable of learning to do tasks by reinforcement learning to human level, right? Robotics foundation models will not work.

Consider a flying saucer cult. Clearly a cult, great leader, mothership coming to pick everyone up, things will be great.

…What if telescopes show a large object decelerating into the solar system, the flaw from the matter annihilation engine clearly visible. You can go pay $20 a month and rent a telescope and see the flare.

The cult uh points out their “sequences” of writings by the Great Leader and some stuff is lining up with the imminent arrival of this interstellar vehicle.

My point is that lesswrong knew about GPT-3 years before the mainstream found it, many OpenAI employees post there etc. If the imminent arrival of AI is fake - like the hyped idea of bitcoin going to infinity or replacing real currency, or NFTs - that would be one thing. But I mean, pay $20 a month and man this tool seems to be smart, what could it do if it could learn from it’s mistakes and had the vision module deployed…

Oh and I guess the other plot twist in this analogy : the Great Leader is saying the incoming alien vehicle will kill everyone, tearing up his own Sequences of rants, and that’s actually not a totally unreasonable outcome if you could see an alien spacecraft approaching earth.

And he’s saying to do stupid stuff like nuke each other so the aliens will go away and other unhinged rants, and his followers are eating it up.

And just briefly, because the default answer to this point is “yes but we’ll eventually do it”: once we do come up with a complex problem solver, why would we actually get it to start up the singularity? Nobody needs infinite computing power forever, except for Nick Bostrom’s ridiculous future humans and they aren’t alive to be sad about it so I’m not giving them anything. A robot strip mining the moon to build a big computer doesn’t really do that much for us here on Earth.

The counter argument is GPT-4. For the domains this machine has been trained on it has a large amount of generality - a large amount of capturing that real world complexity and dirtiness. Reinforcement learning can make it better.

Or in essence, if you collect colossal amounts of information, yes pirated from humans, and then choose what to do next by ‘what would a human do’, this does seem to solve the generality problem. You then fix your mistakes with RL updates when the machine fails on a real world task.

I actually really liked this reply purely on the fact that it walked a different avenue of response

Because yeah indeed, under the lens of raw naïve implementation, the utter breadth of scope involved in basically anything is so significantly beyond useful (or even tenuous) human comprehension it’s staggering

We are, notably, remarkably competent at abstraction[0], and this goes a hell of a long way in affordance but it’s also not an answer

I’ll probably edit this later to flesh the post out a bit, because I’m feeling bad at words rn

[0] - this ties in with the “lossy at scale” post I need to get to writing (soon.gif)

Current the global economy doubles every 23 years. Robots building robots and robot making equipment can probably double faster than that. It won’t be in a week or a month, energy requirements alone limit how fast it can happen.

Suppose the doubling time is 5 years, just to put a number on it. So the economy would be growing a bit over 16 times faster than it was previously. This continues until the solar system runs out of matter.

Is this a relevant event? Does it qualify as a singularity? Genuinely asking, how have you “priced in” this possibility in your world view?

Oh yeah, I fucking wrote a snarky blog post about this a few days ago

The SFnal idea of the Singularity is when technological progress goes faster and faster until it disappears up the hockey stick curve of pure unknoweabilty. What’s happening now in actuality is that hype cycles are crashing faster and faster. Blockchain! Self driving! LLM!

Any takeoffs are going to run into the iron cloud cover of climate change anyway.

OP putting the Large L in LLM

What I’m trying to get at is that the practicalities of improving technology are generally skated over by aingularatians in favor of imagining technology as a magic number that you can just throw “intelligence” at to make it go up.

this is where the singularity always lost me. like, imagine, you build an AI and it maxes out the compute in its server farm (a known and extremely easy to calculate quantity) so it decides to spread onto the internet where it’ll have infinite compute! well congrats, now the AI is extremely slow cause the actual internet isn’t magic, it’s a network where latency and reliability are gigantic issues, and there isn’t really any way for an AI to work around that. so singulatarians just handwave it away

or like when they reach for nanomachines as a “scientific” reason why the AI would be able to exert godlike influence on the real world. but nanomachines don’t work like that at all, it’s just a lazy soft sci-fi idea that gets taken way too seriously by folks who are mediocre at best at understanding science