Here's a thought.
What does it mean to say that a certain event is 'random'?
According to the universal definition, randomness boils down to the inability to predict future events.
But even though we cannot predict random events with certainty, we can in many cases assign a partial index of occurrence. This is known as probability.
An assigned probability gives us the likelihood (at average) that a certain event will occur. As a trivial example, we say a balanced coin tossed fairly has a 50% likelihood of landing heads.
But what does it really mean to say that an event has x% probability of occurrence? For practical purposes, it means that over a relatively large number of iterations, the event in question will occur roughly x% of the time.
BUT, for a single iteration, the figure x% is practically meaningless. Taking an extreme example, say you wagered you entire life savings on a casino game. You would either win, or lose. 50/50.
But any casino game has win probably much less than 50%. So what’s your real chance of winning? Well, if you play just once, it really is 50%. Only if you play many times does the probably change,
Meaning, theoretical probabilities only appear in the real (physical) world over a large number of repetitions or iterations.
So how can we test whether the actual occurrence of any random event really conforms to its theoretical probability?
The only way to do that is to execute the event repeatedly, and note the instances of occurrence as a ratio of the total. So say you took the time to toss a coin 50 times, you would roughly see 25 heads and 25 tails. You will also find that as you increase the number of tosses, the deviation from 50% becomes smaller.
This is where things become interesting to me. There seems to be something almost mystical about the whole process, as if some invisible force is making sure that the coin will always land heads some 50% of the time. Of course, its no more mystical than the force of gravity. But its still fascinating to me...partly because it carries a slight element of predicting the future, partly because it reveals an inherent order in the universe, and partly because it feels like magic.
As stated, the only way to verify whether theoretical probabilities actually manifest themselves in the real world is by physically executing the event a large number of times, then comparing the theoretical numbers to the actual results. But given that a large number of iterations are required for the probabilities to smooth out, and that I don't have the patience or inclination to toss a coin 500 times, I thought about writing a computer program to simulate the events, thereby using a computer to do what it does best...brute and monotonous logical labor.
Such a program would be fascinating in three ways...first, it would allow me to simulate an event over a large number of repetitions within seconds, and organize the results neatly and effortlessly. Second, it would allow me to compare theoretical probabilities against their actual physical manifestations. And third, it would test whether probabilities manifest exactly the same across isomorphic systems.
What I mean by the latter is this...simulating coin tosses within a computer is really playing with a bunch of abstract numbers, which in turn are the results of electric signals. The simulation is completely isomorphic (i.e. logically similar) to physically performing coin tosses, yet would theoretical probabilities really manifest themselves within the simulation? Intuitively and practically, they should. But might something get lost in translation? Would the law of probability really apply the same to virtual coins (made up of numbers made up of electric currents) as they do to real coins, causing the virtual coin to also toss heads 50% of the time? If someone tells me that a Rolls Royce convertible drives my house every day at exactly 2 pm, I may or may not believe them depending on the credibility of the person, the likelihood of the scenario etc. But the only way to verify the assertion is for me to actually go out and look for myself, and thereby establish the fact with certainty, via demonstration.
And that is exactly what I wanted to do with a computer simulation, for the manifestation of probabilities.
To begin with, I needed a source of randomness. Computers produce random numbers by using the so called pseudo random number generators (PRNG). These are algorithms of varying complexity which take a seed number, and run that through a convolution to output a sequence of seemingly random numbers. The output is 'psuedo' random, since it depends on both a fixed algorithm and a seed number. As such, someone who knows what the seed and algorithm is could completely predict the output. If fact, given enough processing power, the algorithm itself can be discerned just by observing the output for regularities.
Most PRNGs provided with compilers are of extremely poor quality (meaning the output is easily predictable) with a low period (meaning the output repeats itself after a short time). While searching the net, I came across the Mersenne Twister...a PRNG of very high quality and a very large period. Additionally, I also quite fortuitously discovered the QRBG. Developed at a Croatian university, it is an electronic device which generates 'truly random' numbers by using the only known source of true randomness in the universe, that being quantum indeterminacy. It uses a photoelectric diode to detect the spontaneous emission of photons from an LED. A single bit is assigned based on the duration of the interval between consecutive emissions, and the stream of output bits are translated into numbers. Since each photon's appearance/emission is completely random, hence the duration of the interval between consecutively emitted photons is completely unpredictable, and hence the translation leads to truly random numbers. The device is hooked up to an online server. Its website provides libraries in various languages, which users can download and use to stream the random numbers to their own PCs...for free. Check it out at http://qrbg.irb.hr/
I decided to simulate two separate randomness scenarios. The first being the toss of a pair of dice (a coin toss being too trivial), the other being the famous Monty Hall problem.
Twin dice toss: The probability distribution of the sum of the face of a pair of tossed dice pans out as the well-known bell curve:
The probability of getting a sum of 7 is the highest, because there are more number of ways that one can attain a seven than any other number (1+6, 2+5, 3+4...). Incidentally, this is also where the term 'lucky 7' came from, as twin dice are used in craps. Since it is known precisely the number of ways any total in the sample space can be attained, the theoretical probabilities can be calculated on paper. Once we have done that, we would expect the actual outcomes over a sufficiently large number of physical tosses of twin dice to tend closely to these theoretical probabilities.
After a dozen or so hours of coding and debugging, I finally ran the simulation for 10,000 tosses using the Mersenne Twister PRNG as the source of randomness. And behold
The beautiful bell curve! The first time I saw this, I was simply blown away. Also note the near symmetrical even/odd split. Its one thing to know the theoretical probabilities, and quite another to see them actually manifest exactly as such in real life.
Monty Hall: The Monty Hall problem was first made famous as a question to Marylin vos Savant (holder of the Guinness record for highest IQ, 238 by one count) in her column in Parade magazine. Without delving too deep into the interesting history of the problem and its variations, I'll simply state it in its simplest form:
Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?
Vos Savant answered correctly that one should switch doors, as doing so doubles the probability of winning. The answer seems rather counter-intuitive, most people would answer that it makes no difference whether the player switches or not. Indeed, from wikipedia:
When vos Savant gave the correct answer that switching would win two-thirds of the time, she estimates the magazine received 10,000 letters including close to 1,000 signed by Ph.D.s, many on letterheads of mathematics and science departments, declaring that her solution was wrong.
http://en.wikipedia.org/wiki/Monty_Hall_problem
Given its counter-intuitive solution, this scenario is the perfect candidate for a randomness simulation. I wanted to see via demonstration if Vos Savant's theoretical answer is correct, and whether switching really would double the probability of winning.
After neglecting the program for months, I finally got around to coding the scenario. Here are the results for several runs of 10,000 iterations each. For 3 doors, I ran the sim twice: Once where player chooses to switch for the first 5,000 iterations, and not switch for the last 5,000....and once where the player chooses to switch/not-switch randomly. In both cases, the number of wins almost exactly doubled when player chose to switch, as expected.
For 5 and 10 doors, the probability of wins went up to roughly 80% and 90% respectively (so switching improved the odds 4 times and 10 times respectively). These numbers are consistent with the general solution for the Monty Hall problem: P(win upon switch) = (D-1)/D(D-op-1), where D = number of doors, op = number of doors opened by Monty before switch offer. In my sim, Monty opens all doors except the switch offer door.
The simulations achieved their originally set goals perfectly. First, they allowed me to simulate two interesting scenarios within seconds and organize the results neatly. Second, they proved that given a random setting and enough iterations, actual manifestation of probabilities in real life tends nearly exactly to the theoretical numbers. Third, it proves that probabilities apply exactly the same across isomorphic systems. The last two points are intuitive, accepted, and well known facts. Yet as with the Rolls Royce driving by my house, there is a tangible difference between theorized knowledge and physical reality. This difference is not a difference of numbers or facts (since those remain the same), but rather a difference in perception and experience.
(I am yet to execute both simulations with QRBG as the randomness source, but I doubt there will be much difference in the results, given Mersenne Twister's high period and the relatively lower number of iterations [due to limited runtime memory] of the simulations).
Conclusion? It brings about the realization that random events are paradoxically the most predictable...in that truly random events are inescapably bound by their regular probability distributions, which is as immutable as any law of the universe. Thus, one can predict the number of times these events are likely to occur over a large number of iterations, with a precision that increases with the number of iterations. And that brings me back to the feeling I expressed earlier, that this precision/order seems almost mystical. If you reached this far without dozing off, and have any thoughts on this topic, I'd love to hear them. L
Addendum 7/2/2011:
Recently, I started thinking about this a little more. In trying to organize these concepts a little more, I came up with the following:
I) Probability: The common definition of this is the measure of the likelihood that an event will occur. That's straightforward. But lets examine it more closely, with a typical example. So the probability of a fair dice tossing a 4 is 1/6. What this really means is that over a large number of iterations, the number 4 will appear nearly 1/6th of the time. The major characteristics of probabilities (and some reasons) are as follows:
1) They are always < 100%: I believe that this is owing simply to lack of information. If an observer had complete information (velocity and position) about every object (quantum objects included) within a closed system, and the observer was outside of the system, then he should be able to predict the event in question with 100% accuracy. Of course, this is impossible at the quantum level, simply due to the uncertainty principle.
2) They only become apparent when the number of iterations is sufficiently larger than the sample space: This follows logically, since you need at least six tosses of a dice before you have enough iterations in order for each outcome to have the chance to manifest. However what I meant by this is highlighted by the words 'sufficiently larger'. Indeed, a mere 12 tosses will probably produce a highly uneven distribution. One would need a number of tosses 'sufficiently larger' than the sample space 6 in order for the distribution to approach reasonable close to the theoretical numbers. Lets hold off on the reasons for this until we discuss the next characteristic.
3) The % of actual occurrence always tends to the theoretical %s, and approaches closer (to the theoretical) as the number of iterations increase: Given enough iterations and given that there are no biases, we will find to a very high degree that the tosses of a dice will yield a probability close to 16.6% for each number. Moreover, we will find that the probability gets closer and closer to 1/6th as the number of iterations increase. This is where things start appearing mystical.
a) Firstly, what ensures that every number will toss roughly 1/6th of the time, as predicted by the theoretical number? The obvious answer is that each number has an equal chance of appearing since we granted the premise of 0 bias. But what exactly counts as 0 bias? I suppose a fair dice would be one that is manufactured as a cuboid with a very high degree of manufacturing accuracy. As such, it would be balanced in size and weight to a very high degree, though not perfectly of course (that would be impossible). So for our purposes, 0 bias would mean very close to 0 bias. Whatever biases do exist (such as minute differences in weight of sides), they interact with other variables which are unpredictable (such as the direction and intesity of the toss, the collisions with air molecules etc), and the sum total of the interactions results in either of the 6 outcomes. Now since the biases and variables are themselves random, hence each number has equal probability of occurrence. But then, what is randomness? We will discuss that in the second part of this post.
b) Secondly, why does it take a 'sufficiently larger' (than the sample space) number of iterations for the probabilities to appear sufficiently close to the theoretical numbers? What exactly makes this happen, and why?
c) Lastly, why do the actual occurrences tend closer to the theoretical one's as the number of iterations increase? It appears that a graph between the theoretical and actual %s would be an exponential curve, where the actual tends sharply towards the theoretical as soon as we have 'sufficiently larger' number of iterations, and very slowly thereafter. But why so?
The last two points, I don't have answers for.
II) Randomness: So what is it? As applied to probabilities, it would be defined as:
All outcomes being unpredictable and, in the ideal, equally probable. Lack of predictability, without any systematic pattern.
So it is unpredictability AND equal probability of occurrence. So the sequence of integers from 1 to 10 would not be random even though each number occurs only once, since it is entirely predictable.
As far as unpredictability goes, this can be attributed to lack of information (as in the case of all probabilities being < 100% in I) 1). Indeed, in a determinist universe, an external observer with complete information about a closed system and enough processing power would be able to predict all events, and hence nothing would be random for him insofar as randomness is defined as unpredictability.
As for equal probability of occurrence, this would stem from the absence of biases. Things get a tad circular here, since 'bias' itself would be defined as any condition that affects the outcome of events by influencing it towards any particular direction. But for our hypothetical omniscient observer, complete knowledge of the system would enable him to objectively decide whether or not the arrangement of variables encompass sufficient conditions to qualify as bias. Now, we know that at the molecular level (and below), there is a great deal of uniformity. Molecules are farily evenly distributed within matter (near uniform density) and have fairly even energy distribution. Things are even more orderly at the quantum level. Since in a uniform distribution, the attributes (e.g mass and energy for molecules) of all elements are very similar, thus uniformity translates to an absence of bias. And hence, the 'uniformity' at the the molecular (and lower levels) translates to 'equal probabilty of occurrence'.
Now since after removing macro sources of bias (such as by ensuring the sides of the dice are even and balanced to a very high degree) we reduce our variables to the molecular and quantum levels, and since things are farily uniform and unknown at those levels (at which we find the criteria of unpredictability AND equal probability of occurrence (synonymous with uniformity as established above) are both fulfilled), thus events that are bias free at the macro level qualify as 'random', with the source of randomness being the uniformity at the lower levels.
This brings to light most ironic characteristic of randomness...that it is most predictable! You know without a shred of doubt that every number on a dice will toss roughly 1/6th of the time. What you don't know however is which number will toss next, and what the exact %s will be for any number of iterations. Not unless you are the external omniscient observer.
So returning to the question in part I) a as to what ensures that the % of actual occurrence tends close to the % of theoretical occurrence, we established that it has to do with the randomness of the biases and variables affecting the tosses. And in part II), we established that this randomness stems is the inherent natural 'order' of things at levels molecular and below, manifested as the result of uniform arrangement and distribution of elements, coupled with the lack of complete information about the same.
Comments