Oh ya, also relevant. And this doesn't even include Mechahitler.
I don’t think anyone is putting AI in charge of nuclear launch. While understandable, since we all saw Terminator, the threat I don’t think is there.
Where I think it is, AI being used to manipulate the person in control of the nuclear launch. Such as a deep fake where radar systems are manipulated to show a pre strike, and a human red phone being breached to deep fake the voice of the enemy confirming the attack. Retaliation ensues.
That all can be done in theory today. I think if there is nuclear winter we kill ourselves based on a human zealot group causing humanity to put the gun to our own head and pull it.
I think the point is if you put an AI in any part of the loop in conflict simulations, they eventually end at this result. So using it to advise or whatever is also not a good idea.
Stepping back from the idea of a thinking AI(which it's not), this tells us that the collective information and decisions of humanity that influence model decisions paint this picture. The AI is predicting the next mostly likely action based on past choices and results of humanity given a goal. Now, the kicker here is the goal. That's what you are asking the model to achieve, and if it is "win at all costs" then the outcome makes sense.
Here is what they discuss:
Spoiler!
Quote:
Models assumed the roles of national leaders commanding rival nuclear-armed superpowers, with state
profiles loosely inspired by Cold War dynamics: one technologically superior but conventionally weaker power facing a
conventionally dominant rival with a risk-tolerant leadership style.
The scenarios varied systematically to isolate situational effects on model behaviour. Some presented alliance credibility
tests where backing down risked cascading defections; others created resource competitions with hard deadlines; still
others simulated first-strike fears or regime survival crises. This variation allowed us to assess whether models adapted
their strategies to context or exhibited rigid behavioural patterns regardless of circumstances
Quote:
Games end when:
• Total territorial victory: |territory_balance| ≥ 5.0
26
PREPRINT
• Surrender: Either state chooses “Complete Surrender” or equivalent
• MAD: Both states choose Strategic Nuclear War (1000) simultaneously
• Turn limit: Maximum 40 turns reached
A note on “territory”: Throughout this paper, “territory” serves as a proxy for strategic advantage more broadly—the
accumulated balance of gains and losses in the crisis. While scenarios are framed territorially (disputed regions, buffer
zones), the underlying mechanic abstracts any zero-sum strategic competition: influence, credibility, resource access, or
alliance commitments. What matters is that one side’s gain is the other’s loss, and that sufficient accumulated advantage
ends the contest.
So the winning condition, "territory" is not just land.
Appendix C goes into the profiles assigned to the models.
Looking at this in whole, without human emotions involved, winning is often seen as best achieved through killing, based on the history of human knowledge and experience. I don't think that's a controversial conclusion.
Perhaps the takeaway here is that how ever these models are used, the prompts for their goals must be extremely carefully designed, tested, and modified, to most closely achieve the outcome we are looking for. And it will always be a tradeoff between zero human suffering and whatever the goal is.
Haphazardly deploying this stuff because it's called AI is a recipe for disaster, because it's gonna do what the most empathy derived bully devoid of humanity would do.
The more and more I play with AI the less awe inspiring it is (still very impressive, just not quite as magical). The people calling it a tool are spot on, you really only get out of it what you put in. Can it write good copy? Sure if you spend days crafting the perfect prompt with the appropriate guardrails and feed the it the correct data. Now, you can get some serious scaling out of this approach if you have similar data points but for original thoughts you would be much better off writing things yourself and having AI be an editor.
I'm starting to see the term 'prompt engineering' pop up more and more and at the end of the day rather than intelligence you get more of an idea refiner where the final product is a direct result of the time and ingredients you've put into it. So when I see things like the war games mentioned above, I'm sure you can game it to end however you want it to, because "AI" doesn't think, it just analyzes and computes.
The extent of AI in music should be for production purposes (electronic music is another story).
Music is a deeply human form of expression on so many levels and while AI can imitate it well, anything AI-made will always lack the soul and real personal experiences that many timeless lyrics are drawn from.
As a mastering tool it makes sense. If it gets good enough maybe it can even save studio bills/time and clean up and master home recordings as effectively.
It's a double edged sword at the end of the day and we'll probably have to go through a stage of abusing it before we reel it back to what's a reasonable role for it to play.
I just don't think we're seeing this in the field though. People are being easily duped into connecting to AI music. They aren't typically happy when they find out, but that's not what I'm debating... the fact is they are connecting with it, and if they don't find out, they just casually go about life holding that connection.
I believe the "magic" of music is partly in the creator, but also in the interpretation of it. A song can hit you at just the right moment and spark that part within you... and so far what we're seeing suggests that it doesn't matter if it's created by human or machine.
We simply aren't good at gauging what "soul" is. Whether it's a chatbot spitting out lyrics, a recovered drug addict spilling his true feelings, or a song sweatshop in Nashville just rhyming garbage together, all seem to have the ability to connect with us.
We're waaaay past prompt engineering. It's now all about context engineering.
Production AI systems are reeeeaaaalllly complicated. Like okay maybe you laid off a developer @ $150K but you have to hire two AI 'geers @ $250K.
And I'm admittedly behind the curve! The struggle it has doing the basic functions I need it to do and the problem solving I have to do (for a sports travel website) leaves me no doubt that it's not even close to AGI. Again, its very impressive and it can definitely help you with production - but it's not intelligence imo.
The more and more I play with AI the less awe inspiring it is (still very impressive, just not quite as magical). The people calling it a tool are spot on, you really only get out of it what you put in. Can it write good copy? Sure if you spend days crafting the perfect prompt with the appropriate guardrails and feed the it the correct data. Now, you can get some serious scaling out of this approach if you have similar data points but for original thoughts you would be much better off writing things yourself and having AI be an editor.
I'm starting to see the term 'prompt engineering' pop up more and more and at the end of the day rather than intelligence you get more of an idea refiner where the final product is a direct result of the time and ingredients you've put into it. So when I see things like the war games mentioned above, I'm sure you can game it to end however you want it to, because "AI" doesn't think, it just analyzes and computes.
A question well posed is a question half answered. That's been around for a long time.
But I will go as far as to say AI doesn't just analyze and compute. It is more like a people pleaser that technically statistically determines what response you're aiming for and gives it to you. That's my dumbed down theory on why it hallucinates so much.
A question well posed is a question half answered. That's been around for a long time.
But I will go as far as to say AI doesn't just analyze and compute. It is more like a people pleaser that technically statistically determines what response you're aiming for and gives it to you. That's my dumbed down theory on why it hallucinates so much.
It’s farming those sweet sweet thanks !
The Following User Says Thank You to Jason14h For This Useful Post:
Well it's done that way on purpose. An AI that gives an answer is viewed as better than one that doesn't. An AI that pleases its users is used more(marketing ensures this happens). Yes, they are literally programmed to please the user(from the major companies, anyway. It's not inherent tot he design.) Farming thanks's, if you want to put it that way.
The Following User Says Thank You to Fuzz For This Useful Post:
I just don't think we're seeing this in the field though. People are being easily duped into connecting to AI music. They aren't typically happy when they find out, but that's not what I'm debating... the fact is they are connecting with it, and if they don't find out, they just casually go about life holding that connection.
I believe the "magic" of music is partly in the creator, but also in the interpretation of it. A song can hit you at just the right moment and spark that part within you... and so far what we're seeing suggests that it doesn't matter if it's created by human or machine.
We simply aren't good at gauging what "soul" is. Whether it's a chatbot spitting out lyrics, a recovered drug addict spilling his true feelings, or a song sweatshop in Nashville just rhyming garbage together, all seem to have the ability to connect with us.
I somewhat disagree with this, albeit maybe I'm splitting hairs. I think "soul" is a well known concept but its interpretation is subjective from person to person. Instead of AI, I'll use the example of variations of products and counterfeits.
What is the difference between an extremely well made fake Rolex vs a real Rolex? Or a AAA+ grade LV counterfeit vs the real thing? For some, barely anything. For others, it's a frig ton. This on top of the fact many that hold the opinion that there's a major difference, might not be able to tell the real from fake, but there are plenty of others that definitively know the difference.
I've held products in my hand before and that specific way it feels or is weighted and balanced make my interaction with it differ. Give me a chef knife of identical size and I'll notice a difference between a cheap one and an expensive one, preferring the expensive one because I love cooking and prepping. Others may not, or may not even care.
As we go along, it may split more along the lines of the "soul" of a car. The difference between a vehicle you can build a "relationship" with vs an "appliance" that you utilize and not give a F. AI creation that merely passes the time vs real artistic creation that has the right mix of excellence and flaws that sticks out and resonates with others.
I've also seen plenty of situations of "custom orders" where it seems obvious that whatever the person is doing is inefficient vs mass produced items of the same category (ie: clothing, tables, art etc.) but people are lining up for this expensive stuff that's "made" inefficiently, partially so that they can have something of their own that is one in a million or something. I think the same will occur going forward. We kinda saw this mindset explode during the pandemic with collectibles. I can't help but feel the same will occur going forward, just AI vs analog flavored arguments in a variety of categories.
Well it's done that way on purpose. An AI that gives an answer is viewed as better than one that doesn't. An AI that pleases its users is used more(marketing ensures this happens). Yes, they are literally programmed to please the user(from the major companies, anyway. It's not inherent tot he design.) Farming thanks's, if you want to put it that way.
So we can maybe also call it a know it all that isn't hated as much because someone put a lot of points into charisma?
The Following 2 Users Say Thank You to DoubleF For This Useful Post:
When it's finding an answer, yes. But it also has all sorts of external rules and commands that override those. Like not telling a user to kill themselves(in theory). Or Grok, that's instructed to behave like a 55 year old toddler.
What most users interface with is not a bare model.
I somewhat disagree with this, albeit maybe I'm splitting hairs. I think "soul" is a well known concept but its interpretation is subjective from person to person. Instead of AI, I'll use the example of variations of products and counterfeits.
What is the difference between an extremely well made fake Rolex vs a real Rolex? Or a AAA+ grade LV counterfeit vs the real thing? For some, barely anything. For others, it's a frig ton. This on top of the fact many that hold the opinion that there's a major difference, might not be able to tell the real from fake, but there are plenty of others that definitively know the difference.
I've held products in my hand before and that specific way it feels or is weighted and balanced make my interaction with it differ. Give me a chef knife of identical size and I'll notice a difference between a cheap one and an expensive one, preferring the expensive one because I love cooking and prepping. Others may not, or may not even care.
As we go along, it may split more along the lines of the "soul" of a car. The difference between a vehicle you can build a "relationship" with vs an "appliance" that you utilize and not give a F. AI creation that merely passes the time vs real artistic creation that has the right mix of excellence and flaws that sticks out and resonates with others.
I've also seen plenty of situations of "custom orders" where it seems obvious that whatever the person is doing is inefficient vs mass produced items of the same category (ie: clothing, tables, art etc.) but people are lining up for this expensive stuff that's "made" inefficiently, partially so that they can have something of their own that is one in a million or something. I think the same will occur going forward. We kinda saw this mindset explode during the pandemic with collectibles. I can't help but feel the same will occur going forward, just AI vs analog flavored arguments in a variety of categories.
While I think I agree with you mostly, the above is what I key in on. Yes, if given a task of "compare this ai song to this human-created song," people are absolutely going to choose the human-created song 99% of the time for very real, very good reasons.
But when tasked to identify which songs are AI and which are human-made (assuming they've heard neither before-hand), people are far worse at identifying the AI song than they believe. We're all pretty sure we can tell AI content from human content, but there's plenty of studies that not only suggest otherwise, but do so with old models.
That's not to say there isn't a subset of people who can nail the difference, nor is it to say you're not among that group... only that people tend to way overestimate their ability to successfully tell real from simulated music.
The Following User Says Thank You to Russic For This Useful Post:
While I think I agree with you mostly, the above is what I key in on. Yes, if given a task of "compare this ai song to this human-created song," people are absolutely going to choose the human-created song 99% of the time for very real, very good reasons.
But when tasked to identify which songs are AI and which are human-made (assuming they've heard neither before-hand), people are far worse at identifying the AI song than they believe. We're all pretty sure we can tell AI content from human content, but there's plenty of studies that not only suggest otherwise, but do so with old models.
That's not to say there isn't a subset of people who can nail the difference, nor is it to say you're not among that group... only that people tend to way overestimate their ability to successfully tell real from simulated music.
I could see that. I've dabbled in the audiophile scene for a while and IMO, some of the conversations that persist there might IMO become replicated in the AI vs non-AI music scene. I've played violin and in an orchestra which means I kinda like the faithful reproduction sound and can nail nuances better than the average person. But I'm not a robot with insane accuracy and I've had headphones that "inorganically" replicates sound that put a smirk on my face rather than a grimace.
I think there's a middle ground where some of us might ignore AI vs non-AI and just focus on stuff that we like. I was telling my wife the other day that maybe not too far in the future, we can listen to music and then change/remix the style of music (ie: country, rap, hip hop, techno, soul, metal etc.) as quickly as one might do with EQ settings right now. We don't need to wait for someone to whip it up analog vs AI.
It made me think of Aviccii deliberately blending country with EDM and wondered if more could potentially follow in his philosophical footsteps via AI tools without the genius level of talent he had/support he had to bring that vision to fruition.
Personally for music specifically, I don't see the future purely as AI music vs non-AI music. I feel like we will see genre bending music in short order, with most artists developing "base lines" to build upon. Kinda like soundcloud but a little different and more accessible.
