As an addon one of the oldest debates around AI in combination with something like Asimov's rules of robotics.
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.
Lets say that they finally built the Ed 209 from Robocop, and it came across a hostage situation where a human with a gun was threatening a hostage.
Would the AI be able to rewrite the three rules above to save the hostage from the hostage taker?
Nope, it would either have to retreat from the situation, find a compromise solution or blow its own head off.