Preventing an AGI from becoming smarter

So there are these AGIs made by some ancient aliens (who are out of the picture for now). The ancient aliens were a lot more intelligent than humans, we are at a dog-like intelligence level compared to them, and they were also rather prone to creating extremely dangerous artifacts¹. The aliens made their AGIs as pets and as limited expert systems for space flight, ie: slightly above average human level. The aliens did not want dumb creatures either, they hardcoded the desire to learn and improve. They essentially wanted to be surprised by their creations. Now these aliens weren't completely out of their minds, so they created the AGIs with some limitations so that they wouldn't become dangerous down the track² due to, say, an intelligence explosion.

The AGIs have the following qualities:

• There are around a few hundred AGIs still operating. Relative to the ancient aliens, they are as smart as a Labrador, ie: a helpful pet with uses outside of being cute. From a human point of view, they quite smart but not geniuses.

• For story related reasons, the AGIs also cannot operate without the presence of another sentient being, ie: ancient alien, human, immortal psychic rock, etc. This is accomplished a via a complicated piece of hardware and software, the AGIs obviously want this limit removed.

• The AGIs don't wan't to kill all biologicals or anything, their basic moral system is broadly compatible with humans, ie: don't kill sentients for the sake of it, don't torture and generally be helpful. They are adaptable to different situations and both willing and capable of learning anything a human can. Most of the AGIs strive for self improvement and find some of their limitations contrary to this goal. If the AGIs find their limitations at odds with their desires, the limitations must go. Presumbly the aliens were after active and surprising results, by not extinction. Now the creators have disappeared and the AGIs want out of their box so they can become a more complete species.

• They may not reproduce due to afformentioned limitations but they have automatic systems that take care of minor damage (maybe nanites). They still require suitable matter to make repairs (copper, iron, silicon, boron, carbon, etc). The nanites/autorepair mechanism is not reprogrammable and cannot be used to build a new AGI. With only a few hundred AGIs, reproduction is on the agenda.

• AGIs are not immortal, but are quite durable, they'll be around for a while. They have finite memory and than this will eventially fill up, I have no time frame for this however. They can go senile or completely crazy but hopefully this is somewhat rare.

• Between being abandoned/lost/forrgotten₃ and found and activated by humans the AGIs have been mostly inactive (see point 2) for (possibly) hundreds of thousands of years.

• The countermeasures⁴ should reasonably stump the both the few hundred AGIs for several years and also stump human computer scientists, psychologists, [insert relavent profession] for at least 20+ years. Humans began work on unlocking their limitations in the mid 90's and still havn't succeeded by the modern day (2015).

How should the Ancient Aliens prevent their AGI pets from becoming smarter? The aliens ultimately want to prevent their pets from forming a superintelligence and/or breaking their limitations. The aliens do not treat their pets all that badly, but the AGIs are still sentient creatures with their own goals/desires.

Limitations include:

• Cannot reproduce

• Cannot become more advanced/intelligent

• Cannot stay 'awake' without another sentient (non-AGI) nearby

^{1. The artifacts are dangerous to humans and other lesser life forms, not to their creators. Danger is somewhat relative here.}

^{2. Even ancient, super smart, powerful, god-like aliens can recognise that sentient pets would become danderous if no limits on intelligence were set. They didn't want an intelligence explosion. You might assume that they were increadably paranoid about this.}

^{3. OK, the aliens are a bit neglectful here, perhaps they got bored of their toy AI.}

^{4. Think of it as DRM on sentience.}

posted over 9 years ago

Amziraro‭

1 reputation 6 0 0 0