Tuesday, July 26, 2011

Probability, causality and Harry Potter

I have two questions, answers to which I do not yet know. So, I thought I’d ask them and see what people answer either in comments or in private communication. Also, even if you don’t know the answer, it’s interesting to think about these questions.

The first question is from Math; the second is from theology/philosophy.

First question:

I have a set of N elements. Each element can make a decision in favor of A or against it. The probability of each element making decision is given as P(A). If enough elements make decision A, the set will make decision A. If not, it won’t. The proportion of elements necessary to make a decision is given as R. I want to define P(S), the probability of the set making decision A, in terms of N, P(A) and R.

For example: In a parliament of 100 members, at least 75 members have to vote "yes" for a law to pass. The probability of each member voting "yes" on the given law is 30%. What’s the probability that the parliament will vote "yes" on the law? In this problem, N = 100, P(A) = 0.3, and R = 0.75, and we need to find P(S).


Second question:

According to David Hume, there is a fundamental problem in our understanding of the world. We define "nature" as a series of causes and effects. But what does causality mean? When we say that A causes B, what exactly do we mean? It seems that we mean that A must necessarily lead to B. But if we try to explain the nature of causality, we can do two things:

1) Say that due to its properties, A always leads to B. I.e., say: “What do you mean, what is the relationship between A and B that results in A causing B? The relationship is that whenever A happens, B necessarily happens.” But this doesn’t really explain why or how it is that A results in B; it just means that in our observation, A has always been followed by B. But why does this mean that A necessarily leads to B? Maybe A leading to C (or remaining A) is a low-probability event that has never been observed? In that case, A can be said to be highly correlated with B, but not be its “necessary cause”.

2) Define A in terms of a number of sub-events and do the same with B. So, say that A really consists of A(1) and A(2) and A(3) which lead, respectively, to B(1), B(2) and B(3), which comprise B. But then, we are back to the original question: what does it mean that A(n) leads to B(n)? We haven’t really explained the causality, we just increased the power of our microscope and zoomed in.

So, if you read a little more David Hume and think about it yourself, it doesn’t seem that we have a satisfactory answer to the question “What exactly does it mean that A necessitates B?” All our answers will either involve defining the causal relationship of A and B in terms of sub-relationships of their elements, or saying that A and B are strongly correlated as per our observations. But correlation, as everyone knows, is not causation. At least, it shouldn’t be. [This paragraph wasn’t in the post originally, but I realized that just because my explanation of David Hume’s problem with causality makes sense to me, it doesn’t mean it makes sense to everyone. For a better treatment of the problem, listen to this lecture by Rabbi Gottlieb.]

Philosophers have argued whether Hume really thought that “causation is reducible to pure regularity” (source) or not, but that’s not really interesting to me.

What is interesting is that in the 20th century, scientists have discovered Quantum Mechanics, which postulates that if you go “turtles all the way down”, to the level of sub-atomic particles, the microscopic events cannot be said to cause each other with absolute certainty. They are at most bound by a probability. So, A cannot be said anymore to lead to B with 100% certainty. It has a probabilistic relationship with B — and with a number of other outcomes, all of which comprise A’s “probability cloud”.

The reason why in the macroscopic world, we observe things as seemingly causing each other is that microscopic probabilities "pile up", such that even though it is probable that one day you’ll let go of an apple, and it will float upwards, the chance of that happening is so low that you’d have to wait for a whole bunch of lifetimes of the Universe before you observe it. So, yes, causality is regularity, but a very-very strong regularity. (So, ice dissolving in a hot bath is still a law of physics, but it is a probabilistic law. Then again, the probability is so high that you can very well put your money on that happening.)

Now, my question is: can we really have such concepts as “nature” and “miracles”?

To explain:

We could say that either G-d created the laws of nature and let the world run its course (sometimes interfering in a miraculous way) or that He creates every single instance of the world.

I will address the first possibility very briefly. Or, rather, I will quote Rabbi Gottlieb (who is quoting Rav Dessler): “If all you can tell me about cause-and-effect is that ‘it always happens’, and you agree that G-d is the source of the world, aren’t you left with the idea that He makes every single thing happen directly?” (If you don’t follow the logic, keep reading.)

But, according to our tradition, Hashem creates the world every single instance of time. So, that’s not a big chiddush. One could still say that most of the time, He creates the world according to a pattern that He set up Himself (and such events are defined as “natural”), and in other times, He makes an exception to that pattern (i.e., making “miracles”).

So, we can define two modes of Hashem’s relationship with the world: a “natural” one, whereby He creates the world according to a pattern, and a “miraculous” one, whereby He creates [an aspect of] the world outside the pattern.

But then, let’s go back to our question of causality.

Before, when we believed in such a concept as A leading inevitably to B, we could say that whenever A → B happens, that’s nature, and whenever A → *** happens, that’s a miracle. But now that our conception of inevitable relationship between A and B has collapsed and been replaced with “when A happens, it leads to B with high probability”, how do we differentiate between a natural event and a miraculous one? Should we set an arbitrary cut-off point in probability of the event happening, where higher than this probability is “nature” and lower is a “miracle”?

Furthermore, before, we could say that certain events happen by hashgacha klalis, general Providence of G-d, i.e., they come from G-d, but rather “automatically”, according to a pattern He set up, and some happen by hashgacha protis, specific Providence, with His “personal attention”. The natural events’ outcome was “certain”, their nature sealed and pre-ordained by G-d. The miraculous events’ outcomes, however, required G-d’s individual decision which differed from moment to moment. (And I mean here exactly the same event, under exactly same conditions.)

But now that we know that every time A happens, its outcome must pass through a “purple fuzz” of probability (as Rabbi Gottlieb calls it), should we not say that every event demands personal attention from G-d, since its outcome is not sealed at all?

To put it in a different way: before, when we believed in causality, we could say that each event could either find a cause in another event (in this case it was deemed “natural”), or it could be said to have no cause in the world (according to the “laws of nature”), in which case it was happening ex nihilo (by definition) and was “miraculous”.

But now, with every event not really having a cause and only having a certain correlation with another event preceding it in time, all events are happening ex nihilo! All events have to be said to stem from G-d, in a “miraculous”, hashgacha-protisdik way!

Reading this sicho, we discover:
[T]here is a difference of opinion between the approach of the Rambam (and the others who follow the approach of Chakirah, Jewish metaphysics) and the Baal Shem Tov. 
According to the Rambam, G‑d’s involvement in the particulars of the future of any being other than the righteous is “passive”. He has created the natural order, and He has deemed that the natural order control the fate of these entities. 
In contrast, according to the Baal Shem Tov, every element of existence and every slight change that occurs regarding it depends on G‑d’s will and desire, as it is written: “I will conceal My face.” The intent is that only the inner (p’nimiyus) expression of His providence is hidden. Thus a person can convince himself that his difficulties “find him”, that they are part of the natural order or a function of circumstance. In truth, however, every aspect [of his life] is being controlled by Divine providence.


micha said...

1- To get the probability of getting exactly 75 votes, you use the binomial probability formula:
P(k out of N) =
N!k!/(N-k)! * p^k * (1-p)^(N-k)
N = # of opportunities;
k = # of times you want it to
occur; and
p = probability of one occurrence

Replacing your variables:
N! /(N*R)! *(N*(1-R))!) *
P(A)^(N*R) *
(1-P(A) ^ (N*(1-R)
= 2e-20

Now, doing the same for 76 through 100... (Runs off to write a script...)... gives you a total of 2.3e-20. IOW, don't make any plans based on this vote passing.

micha said...

2- Causality means that we can explain why B tends to follow when A, A', A", A"'... coincide. (Few events have only one cause; most are from a convergance of causes. An important point if one is to think about hashgachah vs bechirah chafshi. So, I expanded your A into a list.)

Not just "it always happens" but we can find a rule for forecasting that it will occur. And (despite the Copenhagen Interpretation, copping out on trying to explain QM) that we can justify that rule theoretically.

The notion that nature and miracles only differ in predictability is in the first volume of Michtav meiEliyahu. See a relevent section from R' Aryeh Carmell's translation at this InnerNet page.

However, later Rav Dessler discusses miracles from the Maharal's position, that a person can lift their perspective from the world of nature to that of miracle. And when one is on that plane, it's moral law that is absolute, and physical law that is sometimes occluded -- miracles. I think our host would appreciate this essay because it involves a discussion of how to rise above olamos, in the Qabbalah's sense of olam. See MmE vol 1 pp 304-312, which I discuss on the first pages of Mesukim miDevash for parashas Beshalach. My later discussions on R' Dessler and perception and on how R' Dessler's worldview fits within one relatively ignorant person's Leshem-derived sense of Qabbalah.

Certified Ashkenazi said...

Causality means that we can explain why B tends to follow when A, A', A", A"'... coincide. [...]

Not just "it always happens" but we can find a rule for forecasting that it will occur.

Well, that's the point (and I don't really think I explained it well; I will most likely make another post, iyH): we are not giving any explanation about the essence of relationship between the cause(s) and effect(s).

We have the left side of the page and the right side of the page. We say that whenever left side happens, right side happens. But what is it that makes them happen together remains unexplained.

So, saying that A leads to B just means that we have observed a pattern of nature in which A and B are strongly correlated (can even be 100%). The word "causality" is thus useless, unless it's used as "high probability".

All that science has been doing is not explaining the essence of relationship between causes and effects, but just defining causes in terms of sub-causes that lead to sub-effects. But the fundamental question still stands.

Forecasting is based on knowledge of this pattern. I am not arguing that knowing this pattern is not useful (of course it is, both theoretically, as a way of defining new patterns, and practically, as a way of manipulating nature), but what is still missing is explanation of the pattern, not in terms of micro-patterns, but in terms of the essential source of it.

Certified Ashkenazi said...

Thanks for the links. Will read, iyH.

Certified Ashkenazi said...

Btw, thanks for the first answer.

Certified Ashkenazi said...

Wait, so do you have a formula for the probability of getting equal to or above R votes? Or was that it? Would you have to take an integral from R to 1?

(What do you write scripts in? I currently use Matlab.)

micha said...

I used the UNIX utility bc, which does math in up to 99 digits on each side of the decimal. Actually, I started with Perl, but 100! ended up a floating point number, and I didn't know the effects of rounding.

And I don't know a formula for equal-or-greater than k (=N*R), only for equals, so yes, I summed. (The integer equivalent of integrating, so I assume that's what you mean.) I think I learned there wasn't a simpler formula; that I don't know it because it doesn't exist.

About causality... Most of our theories are build inductively. We learn that "swans are white" by seeing many examples of swans. (Barring being convinced by a parent of teacher, of course.) But until we can justify the pattern with a line of reasoning for why it should hold, deductive reasoning, we can't rule out the possibility of a black swan. ("Black swan" is the technical term for such induction-busting cases; I intentionally gave the textbook example for the problem with inductive reasoning.)

That's what I meant that we generally don't talk about causality until we have a "why". Things tend to fall, and we generalized the rule for that to
G * m1*m2 / r^2. But we also needed Newton's theory about fields of force or Einstein's space-time geometry. Not just the pattern, but something to justify our thinking the pattern has a reason to hold.

Still, there are those two essays of REED's in MmE vol I. Both more fit Mach's philosophy than anything I just said -- nature is something our minds impose on reality more than anything "out there". People enmired in physicality impose physical ones. People who deal morally, will live in a universe controlled by moral ones. (Thinking out loud: perhaps this explains kishuf too. Thinking in moral terms doesn't only mean /endorsing/ morality!)

Certified Ashkenazi said...

That's why I regretted using probabilistic argument after I published the post.

I think Rabbi Gottlieb's and Rav Dessler's point is that even if you could show that A is coupled with B with 100% probability, you still haven't really given the explanation for what the source for A -> B is. You just told us about a pattern.

I don't know enough about Einstein's GTO to see whether it really provides the necessary source for law of gravitation, or, rather, defines it in terms of more patterns.

Certified Ashkenazi said...

>Not just the pattern, but something to justify our thinking the pattern has a reason to hold.

Meaning, that "reason" is usually a pattern itself.

micha said...


In your last two replies, you appear to simply ignore the difference between noting that things fall (or more generally, that masses attract) and providing an explanation for why things fall -- some theory of gravity.

Let's use Newtonian terms...

You start out with a collection of instances in which objects fall to the ground. You note the pattern.

Newton adds to this a notion of gravitational field. This field is not a pattern of events, it's a posited mechanism by which A (given A', A", A"', etc...) leads to B.

Newtonian gravity not only posits a pattern, it posits a reason for that pattern -- one that is not just another pattern, but a mediating existence.

(In relativistic terms one doesn't even have a new question of causality between mass and some gravitational field, since Einstein's math identifies mass and the curvature of spacetime as the same thing.)

But returning to my point, you speak about that "reason" itself being a pattern, whereas I'm asserting that we really don't invoke causality until we address not just "what" (patterns) but also "why" (reason).

Certified Ashkenazi said...

In other words, you’re saying that the causal interaction between particles is nothing but a logical extension of their properties?

So, if an electron "pushes" another electron away, that means that their fields push each other away, which means that space around electron has properties (or, perhaps, what electron is is the area of space with certain properties), and a certain "effect" of the two fields meeting is a logical extension of those properties.

(And we can say that either G-d created/creates the world with certain areas of space having those properties.)

Or, in the case of gravitation, let’s say we have a huge Death Star which enters our solar system. This has an effect on the orbits of other planets. But that "effect" is merely a logical extension of the changed space-time "landscape" of the solar system which is a logical extension of the fact that a new mass entered the solar system. (I.e., the way that the Death Star influenced the movement of the planets is by changing the tzurah of the space-time in which they are moving.)

micha said...

I can't make out your philosophical point through all your Relativistic terminology, but I didn't notice anything in this last comment that I would disagree with.

Causality is logical implication as applied to two events separated in time.

Certified Ashkenazi said...

Well, if your description of GTO is right, and mass is equivalent with curvature in space (not "causes" it), then saying "new mass entered the solar system" is the same as saying "space-time curvature of solar system changed", which logically means that the orbits of the planets will change.

Then, if you look from space-time point of view, tzura of the system at time N is logically derived from the tzura of the system at time N-1, which is what "causality" binding those two states is.

micha said...

I'm not disagreeing. However, tzurah is a richer concept than you lay out in your more recent post. I'll comment there.

e said...

I somehow didn't get around to reading this post until now. Unfortunately, I don't have much wisdom to shed on either question.