Wednesday, March 21, 2012

More on Kurzweil's Predictions

After I wrote my last blog post reacting to Alex Knapp's critique of Ray Kurzweil's predictive accuracy, Ray Kurzweil wrote his own rebuttal of Alex's argument.

Ray then emailed me, thanking me for my defense of his predictions, but questioning my criticism of his penchant for focusing on precise predictions about future technology. I'm copying my reply to Ray here, as it may be of general interest...

Hi Ray,

I wrote that blog post in a hurry and in hindsight wish I had framed things more carefully there....  But of course, it was just a personal blog post not a journalistic article, and in that context a bit of sloppiness is OK I guess...

Whether YOU should emphasize precise predictions less is a complex question, and I don't have a clear idea about that.  As a maverick myself, I don't like telling others what to do!  You're passionate about predictions and pretty good at making them, so maybe making predictions is what you should do ;-) ....  And you've been wonderfully successful at publicizing the Singularity idea, so obviously there's something major that's right about your approach, in terms of appealing to the mass human psyche.

 I do have a clear feeling that the making of temporally precise predictions should play a smaller role in discussion of the Singularity than it now does.   But this outcome might be better achieved via the emergence of additional, vocal Singularity pundits alongside you, with approaches complementing your prediction-based approach -- rather than via you toning down your emphasis on precise prediction, which after all is what comes naturally to you...

One thing that worries me about your precise predictions is that in some cases they  may serve to slow progress down.  For example, you predict human-level AGI around 2029 -- and to the extent that your views are influential, this may dissuade investors from funding AGI projects now ... because it seems too far away!  Whereas if potential AGI investors more fully embraced the uncertainty in the timeline to human-level AGI, they might be more eager for current investment.

Thinking more about the nature of your predictions ... one thing that these discussions of your predictive accuracy highlights is that the assessment of partial fulfillment of a prediction is extremely qualitative.  For instance, consider a prediction like “The majority of text is created using continuous speech recognition.”   You rate this as partially correct, because of voice recognition on smartphones.  Alex Knapp rates this as "not even close."   But really -- what percentage of text do you think is created using continuous speech recognition, right now?  If we count on a per character basis, I'm sure it's well below 1%.  So on a mathematical basis, it's hard to justify "1%" as a partially correct estimate of ">50%.   Yet in some sense, your prediction *is* qualitatively partially correct.  If the prediction had been "Significant subsets of text production will be conducted using continuous speech recognition", then the prediction would have to be judged valid or almost valid.

One problem with counting partial fulfillment of predictions, and not specifying the criteria for partial fulfillment, is that assessment of predictive accuracy then becomes very theory-dependent.  Your assessment of your accuracy is driven by your theoretical view, and Alex Knapp's is driven by his own theoretical view. 

Another problem with partial fulfillment is that the criteria for it, are usually determined *after the fact*.   To the extent that one is attempting scientific prediction rather than qualitative, evocative prediction, it would be better to rigorously specify the criteria for partial fulfillment, at least to some degree, in advance, along with the predictions.

So all in all, if one allows partial fulfillment, then precise predictions become not much different from highly imprecise, explicitly hand-wavy predictions.   Once one allows partial matching via criteria defined subjectively on the fly, “The majority of text will be created using continuous speech recognition in 2009” becomes not that different from just saying something qualitative like "In the next decade or so, continuous speech recognition will become a lot more prevalent."  So precise predictions with undefined partial matching, are basically just a precise-looking way of making rough qualitative predictions ;)

If one wishes to avoid this problem, my suggestion is to explicitly supply more precise criteria for partial fulfillment along with each prediction.  Of course this shouldn't be done in the body of a book, because it would make the book boring.  But it could be offered in endnotes or online supplementary material.  Obviously this would not eliminate the theory-dependence of partial fulfillment assessment -- but it might diminish it considerably.

For example the prediction “The majority of text is created using continuous speech recognition.” could have been accompanied with information such as "I will consider this prediction strongly partially validated if, for example, more than 25% of the text produced in some population comprising more than 25% of people is produced by continuous speech recognition; or if more than 25% of text in some socially important text production domain is produced by continuous speech recognition."   This would make assessment of the prediction's partial match to current reality a lot easier.

I'm very clear on the value of qualitative predictions like "In the next decade or so, continuous speech recognition will become a lot more prevalent."  I'm much less clear on the value of trying to make predictions more precisely than this.   But maybe most of your readers actually, implicitly interpret your precise predictions as qualitative predictions... in which case the precise/qualitative distinction is largely stylistic rather than substantive


Interesting stuff to think about ;)

Tuesday, March 20, 2012

Ray Kurzweil's (Sometimes) Wrong Predictions

Note: there was a followup blog post to this one, presenting some complementary views that I also hold, and linking to some more recent comments by Ray Kurzweil on the  matter.

Forbes blogger Alex Knapp, who often covers advanced technology and futurist topics, recently wrote a post titled Ray Kurzweil's Predictions for 2009 Were Mostly Inaccurate ...

Some of Knapp's posts are annoyingly opinionated and closed-minded, but this one was well-put together, and I made a lengthy comment there, which I repeat here.  You should read his post first to get the context...

And also, once you read his post, you might want to read Ray's rebuttal to Michael Anissimov's earlier critique of his predictions. 

Ray rates himself as 90% right out of 100+ predictions; Michael looks at only a handful of Ray's predictions and finds most of them unfulfilled.

Looking at the "90% right" that Ray claims, it seems to me about half of these are strong wins, and the other half are places where the technologies Ray has forecast DO now exist, but aren't as good or as prevalent as he had envisioned.

On the other hand, Alex Knapp in Forbes took Ray's top 10 predictions rather than the full 100+, and found a lower accuracy for these.

An excerpt from my comment to Alex's post on the Forbes site (with light edits) is:



One thing that should be clarified for the general readership is that the vast majority of those of us in the "Singularitarian" community do not, and never did, buy into all of Ray Kurzweil's temporally-specific predictions.  We love Ray dearly and respect him immensely -- and I think the world owes Ray a great debt for all he's done, not only as an inventor, but to bring the world's attention to the Singularity and related themes.  However, nearly all of us who believe a technological Singularity is a likely event this century, prefer to shy away from the extreme specificity of Ray's predictions.

Predicting a Singularity in 2045 makes headlines, and is evocative.  Predicting exactly which technologies will succeed by 2009 or 2019 makes headlines, and is evocative.  But most Singularitarians understand that predictions with this level of predictions aren't plausible to make.

The main problem with specific technology forecasts, is highlighted by thinking about multiple kinds of predictions one could make in reference to any technology X:

1) How long would it take to develop X if a number of moderately large, well-organized, well-funded teams of really smart people were working on it continuously?

2) How long would it take to develop X if a large, well-funded, bloated, inefficient government or corporate bureaucracy were working on it continuously?

3) How long would it take to develop X if there were almost no $$ put into the development of X, so X had to be developed by ragtag groups of mavericks working largely in their spare time?

4) How long would it take to develop X if a handful of well-run but closed-minded large companies dominated the X industry with moderately-functional tools, making it nearly impossible to get funding for alternate, radical approaches to X with more medium-term potential

When thinking about the future of a technology one loves or wants, it's easy to fall into making predictions based on Case 1.  But in reality what we often have is Case 2 or 3 or 4.

Predicting the future of a technology is not just about what is "on the horizon" in terms of science and technology, but also about how society will "choose" to handle that technology.   That's what's hard to predict.

For example a lot of Ray's failed top predictions had to do with speech technology.  As that is pretty close to my own research area, I can say pretty confidently that we COULD have had great text to speech technology by now.  But instead we've had Case 4 above -- a few large companies have dominated the market with mediocre HMM-based text to speech systems.  These work well enough that it's hard to make something better, using a deeper and more ultimately promising approach, without a couple years effort by a dedicated team of professionals.  But nobody wants to fund that couple years effort commercially, because the competition from HMM based systems seems too steep.  And it's not the kind of work that is effectively done in universities, as it requires a combination of engineering and research.

Medical research, unfortunately, is Case 2.  Pharma firms are commonly bloated and inefficient and shut off to new ideas, partly because of their co-dependent relationship with the FDA.  Radical new approaches to medicine have terrible trouble getting funded lately.  You can't get VC $$ for a new therapeutic approach until you've shown it to work in mouse trials or preferably human trials -- so how do you get the $$ to fund the research leading up to those trials?

Artificial General Intelligence, my  main research area, is of course Case 3.  There's essentially no direct funding for AGI on the planet, so we need to get AGI research done via getting funding for other sorts of  projects and cleverly working AGI into these projects....  A massive efficiency drain!!

If speech-to-text, longevity therapy or AGI had been worked on in the last 10 years with the efficiency that Apple put into building the iPad, or Google put into building its search and ad engines, then we'd be a heck of a lot further advanced on all three.

Ray's predictive methodology tries to incorporate all these social and funding related factors into its extrapolations, but ultimately that's too hard to do, because the time series being extrapolated aren't that long and depend on so many factors.

However, the failure of many of his specific predictions, does not remotely imply he got the big picture wrong.  Lots of things have developed faster than he or anyone thought they would in 2009, just as some developed more slowly.

To my mind, the broad scope of exponential technological acceleration is very clear and obvious, and predicting the specifics is futile and unnecessary -- except, say, for marketing purposes, or for trying to assess the viability of a particular business in a particular area.

The nearness of the Singularity does not depend on whether text-to-speech matures in 2009 or 2019 -- nor on whether AGI or longevity pills emerge in 2020 or 2040.  

To me, as a 45 year old guy, it matters a lot personally whether the Singularity happens in 2025, 2045 or 2095.  But in the grand scope of human history, it may not matter at all....

The overall scope and trend of technology development is harder to capsulize in sound bites and blog posts than specific predictions -- hence we have phenomena like Ray's book with its overly specific predictions, and your acute blog post refuting them. 

Anyway, anyone who is reading this and not familiar with the issues involved, I encourage you to read Ray's book the Singularity is Near -- and also Damien  Broderick's book "The Spike."  

Broderick's book made very similar points around a decade earlier, -- but it didn't get famous.  Why?  Because "Spike" sounds less funky than "Singularity", because the time wasn't quite ripe then, and because Broderick restricted himself to pointing out the very clear general trends rather than trying and failing to make overly precise predictions!

Ben Goertzel

P.S. Regarding Ray's prediction that "“The neo-Luddite movement is growing.” -- I think that the influence of the Taliban possibly should push this into the "Prediction Met" or "Partially Met" category.  The prediction was wrong if restricted to the US, but scarily correct globally...

Sunday, March 11, 2012

Will Corporations Prevent the Singularity?

It occurred to me yesterday that the world possesses some very powerful intelligent organisms that are directly and clearly opposed to the Singularity -- corporations.

Human beings are confused and confusing creatures -- we don't have very clear goal systems, and are quite willing and able to adapt our top-level goals to the circumstances.  I have little doubt that most humans will go with the flow as Singularity approaches.

But corporations are a different matter.  Corporations are entities/organisms unto themselves these days, with wills and cognitive structures quite distinct from the people that comprise them.   Public corporations have much clearer goal systems than humans: To maximize shareholder value.

And rather clearly, a Singularity is not a good way to maximize shareholder value.  It introduces way too much uncertainty.  Abolishing money and scarcity is not a good route to maximizing shareholder value -- and nor is abolishing shareholders via uploading them into radical transhuman forms!

So one can expect corporations -- as emergent, self-organizing, coherent minds of their own -- to act against the emergence of a true Singularity, and act in favor of some kind of future in which money and shareholding still has meaning.

Sure, corporations may adapt to the changes as Singularity approaches.  But my point is that corporations may be inherently less pliant than individual humans, because their goals are more precisely defined and less nebulous.  The relative inflexibility of large corporations is certainly well known.

Charles Stross, in his wonderful novel Accelerando, presents an alternate view, in which corporations themselves become superintelligent self-modifying systems -- and leave Earth to populate space-based computer systems where they communicate using sophisticated forms of auctioning.   This is not wholly implausible.   Yet my own intuition is that notions of money and economic exchange will become less relevant as intelligence exceeds the human level.  I suspect the importance of money and economic exchange is an artifact of the current domain of relative material scarcity in which we find ourselves, and that once advanced technology (nanotech, femtotech, etc.) radically diminishes material scarcity, the importance of economic thinking will drastically decrease.  So that far from becoming dominant as in Accelerando, corporations will become increasingly irrelevant post-Singularity.  But if they are smart enough to foresee this, they will probably try to prevent it.

Ultimately corporations are composed of people (until AGI advances a lot more at any rate), so maybe this issue will be resolved as Singularity comes nearer, by people choosing to abandon corporations in favor of other structures guided by their ever-changing value systems.   But one can be sure that corporations will fight to stop this from happening.

One might expect large corporations to push hard for some variety of "AI Nanny" type scenario, in which truly radical change would be forestalled and their own existence persisted, as part of the AI Nanny's global bureaucratic infrastructure.  M&A with the AI Nanny may be seen as preferable to the utter uncertainty of Singularity.

The details are hard to foresee, but the interplay between individuals and corporations as Singularity approaches should be fascinating to watch.

Are Prediction and Reward Relevant to Superintelligences?

In response to some conversation on an AGI mailing list today, I started musing about the relationship between prediction, reward and intelligence.

Obviously, in everyday human and animal life, there's a fairly close relationship between prediction, reward and intelligence.  Many intelligent acts boil down to predicting the future; and smarter people tend to be better at prediction.  And much of life is about seeking rewards of one kind or another.  To the extent that intelligence is about choosing actions that are likely to achieve one's goals given one's current context, prediction and reward are extremely useful for intelligence.

But some mathematics-based interpretations of "intelligence" extend the relation between intelligence and prediction/reward far beyond  human and animal life.  This is something that I question.

Solomonoff induction is a mathematical theory of agents that predict the future of a computational system at least as well as any other possible computational agents.  Hutter's "Universal AI" theory is a mathematical theory of agents that achieve (computably predictable) reward at least as well as any other possible computational agents acting in a computable environment.   Shane Legg and Marcus Hutter have defined intelligence in these terms, essentially positing intelligence as generality of predictive power, or degree of approximation to the optimally predictive computational reward-seeking agent AIXI.   I have done some work in this direction as well, modifying Legg and Hutter's definition into something more realistic -- conceiving intelligence as (roughly speaking) the degree to which a system can be modeled as efficiently using its resources to help it achieve computably predictable rewards across some relevant probability distribution of computable environments.  Indeed, way back in 1993 before knowing about Marcus Hutter, I posited something similar to his approach to intelligence as part of my first book The Structure of Intelligence (though with much less mathematical rigor).

I think this general line of thinking about intelligence is useful, to an extent.  But I shrink back a bit from taking it as a general foundational understanding of intelligence.

It is becoming more and more common, in parts of the AGI community, to interpret these mathematical theories as positing that general intelligence, far above the human level, is well characterized in terms of prediction capability and reward maximization.  But this isn't very clear to me (which is the main point of this blog post).  To me this seems rather presumptuous regarding the nature of massively superhuman minds!

It may well be that, once one gets into domains of vastly greater than human intelligence, other concepts besides prediction and reward start to seem more relevant to intelligence, and prediction and reward start to seem less relevant.

Why might this be the case?

Regarding prediction: Consider the possibility that superintelligent minds might perceive time very differently than we do.  If superintelligent minds' experience goes beyond the sense of a linear flow of time, then maybe prediction becomes only semi-relevant to them.  Maybe other concepts we don't now know become more relevant.  So that thinking about superintelligent minds in terms of prediction may be a non-sequitur.

It's similarly quite quite unclear that it makes sense to model superintelligences in terms of reward.  One thinks about the "intelligent" ocean in Lem's Solaris.  Maybe a fixation on maximizing reward is an artifact of early-stage minds living in a primitive condition of material scarcity.

Matt Mahoney made the following relevant comment, regarding an earlier version of this post: "I can think of 3 existing examples of systems that already exceed the human brain in both knowledge and computing power: evolution, humanity, and the internet.  It does not seem to me that any of these can be modeled as reinforcement learners (except maybe evolution), or that their intelligence is related to prediction in any of them."

All these are  speculative thoughts, of course... but please bear in mind that the relation of Solomonoff induction and "Universal AI" to real-world general intelligence of any kind is also rather wildly speculative...  This stuff is beautiful math, but does it really have anything to do with real-world intelligence?  These theories have little to say about human intelligence, and they're not directly useful as foundations for building AGI systems (though, admittedly, a handful of scientists are working on "scaling them down" to make them realistic; so far this only works for very simple toy problems, and it's hard to see how to extend the approach broadly to yield anything near human-level AGI).  And it's not clear they will be applicable to future superintelligent minds either, as these minds may be best conceived using radically different concepts.

So by all means enjoy the nice math, but please take it with the appropriate fuzzy number of grains of salt ;-) ...

It's fun to think about various kinds of highly powerful hypothetical computational systems, and fun to speculate about the nature of incredibly smart superintelligences.  But fortunately it's not necessary to resolve these matters -- or even think about them much -- to design and build human-level AGI systems.