Bird Perspectives and Outside Views
Very powerful, even though outside views can be misapplied
My previous post involved a comparison between its topic under investigation and a canonical concept (Egan’s law) of the rationalist community on LessWrong. In this spirit, let’s compare the concept of bird perspective, after which this Substack newsletter is named, and the concept of outside view on LessWrong. After all, that was already the plan here more than two years ago!
Back then, for the term “bird perspective”, I referenced a commentary piece by Max Tegmark, 2007 in Nature. Tegmark uses the term for a view at theoretical-physics equations from above, a view he prefers to the “frog perspective” of an observer living and perceiving inside the equations. As I recently learned, credit should be extended to the late Freeman Dyson, another theoretical physicist, who, in a similar spirit and no later than 2007 also, divides mathematicians into “frogs” and “birds”.
Tegmark adopts the bird perspective to argue for the Everettian or many-worlds interpretation of quantum physics, but I don’t want multiverses to derail this post; so here’s a substitute example. Consider the following passage from Sean Carroll’s popular book on quantum physics, Something Deeply Hidden, presenting an anecdote that, Carroll says, “is a favourite among Everettians”:
One sunny day in Cambridge, England, Elizabeth Anscombe ran into her teacher, Ludwig Wittgenstein. “Why do people say,” Wittgenstein opened in his inimitable fashion, “that it was natural to think that the sun went round the earth, rather than that the earth turned on its axis?” Anscombe gave the obvious answer, that it just looks like the sun goes around the Earth. “Well,” Wittgenstein replied, “what would it have looked like if the Earth had turned on its axis?”
The common thread to Wittgenstein’s reply and Tegmark’s preference seems to be to rise from one’s particular, narrow, observer point of view and look at a broader model that includes the observer, but now seen from outside. One can then think about what the observer in the model would see, and perhaps arrive at the epiphany “oh wow, that’s exactly what I see — that model is what reality might in fact be like!”
Now, in the previous paragraph, the word “outside” popped up very naturally: so why would I want to speak of bird and frog perspective as opposed to of outside and inside view, a standard couple of terms on LessWrong that has a tag page there (like Egan’s law)? And that is so natural indeed that it appears in Tegmark’s paper, too?
Well, for one thing, bird perspective clearly connotes something more desirable than frog perspective does. Whereas being an outsider sounds less desirable than being an insider — because the outsider tends to lack “inside information”, and the latter concept will play a role below. Still, I have to admit that in Wittgenstein’s reply the philosophical point is not for the observer entity to rise as high as possible, to some maximum altitude. The point is that it looks at itself, from the outside.
But here is another example: the discovery of random coincidence. Imagine someone is hit by a falling tree branch and badly hurt. As far as I know, in most cultures throughout history a random accident wouldn’t have been among the interpretations entertained. As opposed to witchcraft; or perhaps it looked like divine punishment for some transgression. Now think about it . . . it would seem that randomness can only be discovered from a bird perspective. One has to look down at one’s world from a bird perspective and think about how rotten branches fall off occasionally and people walk underneath occasionally, so very occasionally the two types of events are going to happen simultaneously, if there is no force at work that would keep them separate. Such a view from above could then lead to the bird epiphany “oh wow, my injury could be of this type! No witchcraft or anything, just one of these . . . how to call them?” Co-incidences.
If we said here that the observer “adopts an outside view”, as opposed to “ascends to a bird perspective”, our language would not reflect the fact that the bird epiphany includes the outside part. When they start thinking in the way described here, people do not know yet that they are about to look at themselves from outside.
In any case, the term “outside view” is taken already — as we will learn when we now look closer at the LessWrong tag page, opening a can of worms by doing so.
It turns out that Wittgenstein’s reply, such a nice apparent example of an outside view, does not actually fall under that term on LessWrong. It turns out that, to summarise:
The term “outside view” denotes there an application of a specific kind of bird perspective, an application called reference class forecasting; and there has been controversy, quite recently regarding the terminology, but also virtually from the beginning regarding reference class forecasting itself, a technique that has been invoked in dubious ways, notably with regard to artificial intelligence.
Let’s unravel this. First, what is reference class forecasting?
We can use for illustration the 1993 example by Daniel Kahneman and Dan Lovallo that is cited on LessWrong as the original inspiration. Kahneman was involved in the design of a certain new high-school curriculum. About a year in, he asked each team member to write down in private an estimate of how much longer it would take to finish. Answers ranged from 18 to 30 months. But when Kahneman hit on the idea of asking one of the members, an expert in curriculum development, about other, comparable projects that he knew about, the embarrassed expert had to admit that it actually took those between seven and ten years! About 40 percent of projects never finished at all.
To quote some original words that follow this story in Kahneman & Lovallo’s 1993 paper, words that are also part of an extensive quotation by Robin Hanson in his July 2007 post Beware the Inside View, apparently the earliest rationalist writing on the issue:
The inside view of the problem is the one that all participants in the meeting spontaneously adopted. An inside view forecast is generated by focusing on the case at hand [...] The outside view is the one that the curriculum expert was encouraged to adopt. It essentially ignores the details of the case at hand. Instead, it focuses on the statistics of a class of cases chosen to be similar [...]
The class of similar cases is the reference class, and the technique is reference class forecasting, called outside view on LessWrong, following Kahneman & Lovallo. It provided a much more pessimistic prediction than the inside view that everyone took originally. It was also a bird perspective, with the bird epiphany “our own case is not really different from these other ones, so the project might take us much longer than we thought!”
Wait a moment. How can we say that our own case is not different? Perhaps it is actually different in some relevant respect, so that our chances are better? But no, in the Kahneman story it wasn’t. The curriculum expert was asked by Kahneman about that and had to admit (to quote again from the paper) “I cannot think of any relevant factor that distinguishes us favorably from the teams I have been thinking about”.
But then it is not really true that they “ignore[d] the details of the case at hand”, that the outsiders lacked inside information, right? And in Wittgenstein’s reply no details were ignored either; the loss of inside information is not an issue that would affect bird perspectives in general. Even from high above you can see everything, as long as your resolution is good enough. Thus, regarding the outside view, we need to distinguish between
the bird perspective that looks at the broader picture, without losing any inside information;
and reference class forecasting as the application of that perspective, an application that does ignore inside information and is only valid if consideration of the inside information did not reveal relevant differences between the case at hand and the other cases in the reference class.
And it is the application that is called the outside view. Rather confusing. Now let’s move on to the controversies.
There is Daniel Kokotajlo’s post Taboo “Outside View”, the top-ranked linked post on the LessWrong tag page. It complains that usage of the term has expanded beyond reference class forecasting, to various other, vaguely similar, heuristics, for which it provides examples (they are forecasting heuristics, so none of them appears similar to Wittgenstein’s reply). This creates confusion and can also provide cover for sloppy reasoning. Better explain exactly what heuristic you are using, says Kokotajlo.
I find his admonition reasonable — and when I was thinking about what categories of bird perspectives could be distinguished, it hit a particular nerve. Those with reference classes or reference items appear to form quite a principled category, whereas the “outside” property that I emphasised as a key feature turns out to be more vague. To illustrate, is that property present when an observer simply takes a broader view after previously missing the forest for the trees? Say, when a conflict can be better understood and judged by considering its whole history than by just following the day-to-day tit-for-tat happenings? I would not want to withhold the term “bird perspective” from such cases, it seems too apt. And yes, I suppose we could press this example into my scheme by telling a story of a bird epiphany revealing a surprising model of reality that explains in a new way why the observer sees these day-to-day happenings. But ultimately there is hardly an inversion of reality like in Wittgenstein’s reply. The observer simply takes note of helpful context and as a result gains a better understanding.
So we have to be aware that the domain of bird perspectives ends up rather fuzzy. Still, I think the concept is useful as long as we mean only perspectives, not arguments or heuristics. Perhaps if the LessWrong meaning of outside viewing did not extend to an actual application, an actual argument, then Kokotajlo might not have had to write his post.
The post made the 2021 best-of list on LessWrong. But about reference class forecasting itself, there has been controversy virtually from the beginning. Eliezer Yudkowsky’s 2008 post The Outside View’s domain already indicates that something has gone wrong; and here is the post of his that I would recommend most, from 2010, called “Outside View!” as Conversation-Halter. This is also his highest-ranked linked post on the LessWrong tag page.
What’s the problem, then? And how does the outside view halt conversations?
Basically, there are two sides to the coin. On the one hand:
In his aforementioned 2007 post that kicked it all off, Hanson writes: “Most, perhaps all, ways to overcome bias seem like [allowing] an outside view to overrule an inside view” — and while I don’t think other biases don’t exist, I absolutely can relate to the sentiment. I think that bird perspectives and outside views are great! I named this Substack newsletter after them! I think they should be taught in school and adopted in class there at every opportunity! They help to undergird the basic achievements of civilisation! As an illustration of the ambitious claim at the end, take the right to free speech. Admittedly, many people who profess support for that will have in mind mainly their own right to speak. But a more principled support could arise from the bird epiphany “there are all those people who have this intuitive certainty that they’re right and others wrong, but they can’t all be right — and I am one of them! Not that I’m likely to ever overcome my own intuitive certainty, but intellectually, this realisation should propel me to support a right for everyone to contribute to the debate, including for those with very different opinions from mine.”
So far so good. Yet on the other hand:
Reference class forecasting no longer works if there is a relevant difference between the case at hand and the others in the reference class. Doesn’t mean ascending to the bird perspective and looking at the reference class wasn’t a source of valuable inspiration — but no, we can’t take the reference class forecast as our answer.
In the Kahneman example, clearly the outside view should override the team’s own inside-view estimates. Between the case at hand and those in the reference class, there is no relevant difference (not in the right direction, anyway). At least if the other teams also would have said, at the same stage, that they would finish in 30 months at most. Could it be that they actually correctly estimated it would take more than seven years? But this would be rather weird. The discrepancy between inside and outside view in Kahneman’s team is much better explained by a human bias towards over-optimistic planning estimates.
Consider the following train of thought. The other teams probably were similarly over-optimistic, but they did not have a Kahneman in their ranks who managed to alert them to the planning fallacy. Might this warning put Kahneman’s team in a better position going forward? If it constitutes a relevant difference between the case at hand and the others in the reference class, the applicability of the outside view becomes shaky. And yet clearly the inside estimate of 18 to 30 months remains unrealistic, so let’s take something inbetween inside and outside view? Perhaps the midpoint?
Probably not quite the midpoint. In general, artfully combining different techniques makes a good prediction-market forecaster, I guess — but with our question “what does the outside view do to conversations” in mind, notice that essentially the decision how to combine is arbitrary, determined by intuition. In order to go beyond this, we would need to analyse how exactly the bias works, and what effect, if any, the warning is going to have in that light. Which means we use, not an outside/inside combination, but an inside view again — after a bird perspective alerted us to a problem with our first forecasting attempt.
Inside viewing, in Yudkowsky’s words, from the 2008 post, allows “two people [to] focus on the internal structure and argue about what happens and their dispute will be commensurable”. But clearly, overall, the Kahneman planning example is one where the outside view shines. Let us now switch to a very different example.
Yudkowsky’s 2010 post is written in reply partly to a 2008 post by Robin Hanson, titled Outside View of Singularity (almost a contradiction in the title already, but Hanson is serious). A notable motivation for both posts is artificial intelligence (AI), which I will use as my example, too. Though in a slightly different way; and this is not meant to be a summary of Yudkowsky’s opinions, or even something I’m confident he would endorse. With that said, suppose now, for the purpose of discussion, that we are certain about the arrival of powerful AI (say, superhuman at advancing every scientific field), and that we want to think about the risk. We might set out to estimate an upper bound for catastrophic risk by assembling previous major leaps in technology into a reference class . . .
But here we have technology with agent potential. Intelligence and planning capabilities of its own. Unlike even nuclear weapons. Surely that’s a game-changing difference. And that looks like the end of this outside view. Any reassuring information to be gleaned from the reference class can be eclipsed by the novel risk, if the latter is high enough. Whether it is, the outside view cannot tell.
Well, there is one way the outside viewer could continue. We can consider “human-group technology”, such as corporations. This could be seen as a previous instance of agentic technology. Countless corporations failed to end the world, so should we be somewhat relaxed about AI? I think the comparison to corporations isn’t useful (for more on AI risk see here — though I’m not an expert), but at least I would see that as a legitimate way for the pure outside view to prolong a dispute for one turn. Whereas claiming that it should still, difference or not, retain some meaningful credence wouldn’t qualify as such a way for me.
Admittedly, merely saying “surely that’s a game-changing difference” might have been too lazy for Paul Christiano’s taste — but if your technology begins to think about how to take control from you, what else to say, really? It’s a different game now.
Then again, how sure are we that powerful AI will think about this? Or perhaps even if it crosses that line, perhaps it can be safely confined in a box? Certainly the outside viewer could argue that the AI may not cross that line, and that the outside view remains informative if it doesn’t. Perhaps a general schema for an outside viewer’s continuation against a game-changer claim could look something like this:
argue for a probability that a certain line won’t be crossed and deny the justification of the claim for that case;
repeat step 1 for all interesting lines;
proceed with the consultation of the outside view, but restricted by what you have constructed there (specifying the details here would require a more advanced quantitative reasoner than myself, I’m afraid).
Unlike the argument from corporations, this schema involves inside viewing. As does something we already talked about for the Kahneman project, namely taking inspiration from the underlying bird perspective and looking at why cases in the reference class turned out the way they did, then considering the implications for the case at hand. The latter method is certainly recommended (see also here), and Hanson employs it in his aforementioned 2008 post on how to think about the next singularity — whose final paragraph, however, begins like follows:
Excess inside viewing usually continues even after folks are warned that outside viewing works better; after all, inside viewing better show offs inside knowledge and abilities. People usually justify this via reasons why the current case is exceptional. (Remember how all the old rules didn’t apply to the new dotcom economy?)
If human-level AI and beyond is as imminent as many experts now think, it renders particularly unfortunate the timing of those who have convinced themselves, apparently, that the outside view is simply superior. That it is the view for sophisticated thinkers to aspire to, whereas “this time is different” is a sentiment for naive fellows. And the cry “outside view!” of the sophisticated becomes the “conversation halter” in the title of Yudkowsky’s 2010 post. Even if you’re happy to do all the talking yourself, if you do it from the inside view then you’re a showoff.
Let me conclude with two clarifications regarding the outside view and its domain.
First, Hanson is skeptical of inside views because he knows “how quickly errors accumulate when reasoning about details”, to quote again from his 2008 post. If one regards one’s interlocutor’s unyielding inside view as unproductive, not because one can point out a specific reasoning error, but because of an inevitable accumulation of such errors, could one then not defensibly halt the conversation? In theory, yes — but notice that it would have little to do with any outside views, which would have to be judged on their own merits. It would just be the case that the inside view is deemed useless or worse. It’s perhaps like we might want to throw out a map of Luxembourg city, when searching for the Philharmonie Luxembourg, if we deem that map utterly useless; but we wouldn’t throw it out because our abstracted knowledge of the maps of German and French cities serves us better.
Or French cities only? German cities? European cities? This is called “reference class tennis”, see Yudkowsky’s 2010 post — although my example is too neat; real-life scenarios may feature ambiguity about even which sorts of members a good reference class should contain.
Anyway, on to the second clarification, and it’s an important one. If we prefer to reject what an outside view teaches us, then, practically speaking, we can basically always provide some difference that, with some flimsy justification, we declare game-changing. Almost whatever the topic. There is hardly any way Hanson could ever “defeat” us in a dispute with his outside views alone if we are stubborn enough. We can frivolously force him to descend from his higher level and partake in some inside viewing.
But — and this ties back to Kokotajlo’s warning, or at least to my comment on it — that’s okay! Outside views aren’t tools to win disputes. When competing against inside views, they are, perhaps the term I’m looking for is: debiasing techniques. Similarly for bird perspectives more generally, “valuable inspiration” as I said earlier, debiasing inspiration, intuition pumps. Not normally arguments that you could rely on to win a dispute for you.
Actually, I can’t quite get over Hanson’s showoff remark. Is it really the inside view that is the showoff view? Check out Vasco Grilo’s recent outside-view posts on extinction risk, apparently well-received in the Effective Altruism Forum, where he uses an impressive statistical apparatus to arrive at, for example, a preferred estimate of 6.36 in 100000000000000 from his methodology for the annual probability of war causing human extinction.