tag:blogger.com,1999:blog-11168555.post8768720533244273075..comments2024-03-28T03:22:24.202-04:00Comments on The Multiverse According to Ben: Coherent Extrapolated Valuation (CEVa): Another Stab at Defining a Future-Looking Human-Friendly Value SystemBenhttp://www.blogger.com/profile/12743597120529571571noreply@blogger.comBlogger4125tag:blogger.com,1999:blog-11168555.post-72925404244868982132016-09-13T22:23:14.043-04:002016-09-13T22:23:14.043-04:00I am surprised this thing is still knocking about....I am surprised this thing is still knocking about. Many, including myself, commented on its flaws at the time of its birth and I thought it was pretty much dead letter. The few times I have run into Eliezer since then and mentioned it he seemed to more or less disown it as defective but was not clear that there was something new and better. At least that was my perception of the situation.<br /><br /><br />One of the strang flaws of the CEV is that an AGI or something else that is super smart but not necessarily autonomous as a being, will somehow extrapolate with its powers of computation what we would in fact want if we were quite a bit different from the way we are, at least in qualitative ways and in quantitative ways of how well we can process how much information. Worse it seeks to extrapolate with general human desires and proclivities, our evolved psychology if you will, held as more or less a given but leading these other things vary. However one of the most likely results of being quantitatively and qualitatively much better and different is that we would likely see through and eschew much of this change programming and its implied goal structure basis.<br /><br />Then there is the problem that the original model seemed to seek to do some kind of universal maximization of human happiness, a grand utilitarianism. But happiness is notoriously sketchy to define much less optimize. And if indeed you start with assuming our evolved psychology is kept constant then what makes beings with that psychology experience happiness may not be at all adequate to such advanced theoretical versions of ourselves. <br /><br />So a deep problem of CEV is that it has no workable notion of what the "good" actually is beyond some hand waving toward our what our evolutionary psychology plus what works in inter-relationship between peers is as much of "good" as can be thought about.<br /><br /><br />What of those that don't like what this CEV, if one could possibly build one, comes up with? What is this CEV intersects some other evolved technological species' CEV equivalent? Can the ethical basis be extended? <br />samanthahttps://www.blogger.com/profile/05637374540680524149noreply@blogger.comtag:blogger.com,1999:blog-11168555.post-14911343084485705742016-08-30T10:09:22.484-04:002016-08-30T10:09:22.484-04:00It would be cool to see a group try to approximate...It would be cool to see a group try to approximate this for the globe. <br /><br />A framework whose parameters require us to map out the global ethical spectrum of preferences is good ;-) <br />[Even if it doesn't just tell us what to do on a silver platter :p]zariuqhttps://www.blogger.com/profile/17078809186461450388noreply@blogger.comtag:blogger.com,1999:blog-11168555.post-28196293096309165412016-08-29T02:27:44.496-04:002016-08-29T02:27:44.496-04:00The CEV is based on a false premise, that mental p...The CEV is based on a false premise, that mental properties are completely reducible to physical ones.<br /><br />In other words, the mistake here is an overly reductionistic approach to ethics - the mistaken idea that you could reduce ethics to purely mechanical terms.<br />The notion that you could somehow mechanically 'extrapolate' a person's future values without any consciousness present is based on a reductionistic fallacy.<br />The whole *point* of conscious experience is to to evolve our values into the future. In other words...the only way for our values to evolve is for us to live our lives...you cannot predict in advance how a person's values will evolve, without actually *being* a person.<br /><br />The 'orthogonality' postulate of Bostrom/Yudkowsky and the resulting ludcrious notion of the 'paperclip monster' is based on the same big ontological mistaken as CEV: the idea that mental properties are completely reducible to physical ones. <br />But there's another argument that is sufficient to rebut CEV and orthogonality:<br />Intelligence and values are not the same thing, but they *are* related. The fabric of knowledge is a unified whole, and the whole history of science is of domains that were once thought to be separate later found to be related. This fact alone is enough to cast serious doubt on 'orthogonality', quite apart from the ontological mistake I talked about above.<br /><br />Any AGI or general-intelligence mind needs 3 distinct systems: <br />Evaluation system/Basic Drives, Decision-making system and Planning system.<br /><br />Whilst these 3 systems are not the same, they *are* related, and there is little basis for thinking that you can arbitrarily chop and change one without it seriously affecting the other 2.<br /><br />In other words, emotions (basic drives), decision-making (policy) and planning (high-level values) all *depend* on each other for smooth functioning.<br />If an AI has the wrong values, this will seriously limit its decision-making system, thus falsifying orthogonality. In the one example of a general purpose intelligence that we know about (humans) this is clearly true - cutting out a person's basic emotions/drives results in severe degradation of decision-making abilities - 'paralysis by analysis'.<br /><br />The correct approach to ethics lies in knowledge representation and ontology. One needs to identity the correct a-priori ('universal') categories of thought that are *necessary* prerequisites to thought in the first place. Kant had the right idea all those years ago!<br /><br />Once we've identified the 'universal categories' , we code them up, and our job is basically done. The categories should form the 'seeds' for our AGI to do all the rest of the learning on its own.<br /><br />In other words, identity the basic ontological 'primitives' of ethics (the conceptual seeds), code these up, and let the AGI learn the rest on its own. The seeds are the conceptual scaffolding on which the AGI would then build, based on empirical learning of human values.<br /><br />Of course, Bayesian induction isn't a fully general method of reasoning under uncertainty. Real rationality is *abduction* not induction (Bayesian induction is actually just a special sense of abduction).<br /><br />It is through abduction that science really works, building coherent categories on top of the a-priori universal categories of thought that serve as the conceptual seeds.<br /><br />See my A-Z list of the basic concepts needed to understand abduction here:<br />http://www.zarzuelazen.com/ConceptLearning.html<br /><br />My list connects to wikipedia articles, be sure that you read *all* of these, and I promise, after reflecting on what you've read, it will become clear how Bayesian induction is really just a special case of the real rationality, *abduction*. <br />After you've grasped this, all the AGI problems will start to fall like dominos and the road to Singularity will finally be clear...ZARZUELAZENhttps://www.blogger.com/profile/07742429508206464486noreply@blogger.comtag:blogger.com,1999:blog-11168555.post-63589558580133739152016-08-29T02:03:03.634-04:002016-08-29T02:03:03.634-04:00This comment has been removed by the author.ZARZUELAZENhttps://www.blogger.com/profile/07742429508206464486noreply@blogger.com