One nuance with the defender example is the AI was given the goal of surviving. The next step would be to put the AI in an environment, and let it come up with its own goals and attempt to accomplish them. But what would drive the need to come up with a a goal? How would it determine a "good" goal worth pursuing vs a "bad" goal that should be rejected?
I do think "flashes of brilliance" is running some sort sort of algorithm (whether a genetic algorithm, or something else) in our subconscious. How many times have you been stuck on a problem, then go to bed, and when you wake up, the solution hits you? Something was going on while you slept.
In many ways we are all just highly advanced pattern matching machines. That sudden sense of dread you get when you walk into some scenario is likely your subconscious pattern matching something in your environment with a past negative experience.
|