Can one tell a lie to non-religious Jews to get them to keep Shabbos? Everything that I regard as central to Judaism tells me that the answer is "no". Not only do I, a halachic layperson, think this, but also rabbonim that I’ve consulted say unequivocally that something you even suspect may be false should not be used as part of outreach.
The Torah codes debate forces us to face this issue squarely. There are those who assert that it has been statistically proven that there are codes in the Torah so it can be used as a first step to get some non-religious Jews to start thinking seriously about yiddishkeit. But the virtually unanimous opinion of those professional mathematicians and statisticians [1] who have carefully examined the evidence is that there has been no scientific proof of the codes.
Those who remember my earlier article will note a change – in the six months [2] between my writing that piece [3] and this one, I have spent considerable time studying the evidence [4] and the replies of the proponents (both in Jewish Action and on the web) and I have gone from being skeptical about the arguments presented for the validity of the codes to being certain that all of the evidence presented so far has no legitimacy [5]. My first goal here will be to explain why I and many other scientists have reached this conclusion. Then I’ll address some of the other points raised in the articles of Rabbi Mechanic and Mr. Witztum [6]. Finally, I’ll turn to the key issue of what the scientific community has to say of what is, after all, claimed to be a scientific proof.
Basic Flaws
Recall that the codes are based on searching for equidistant letter sequences (ELSs), that is words occurring in the text of the Torah but with skips between the letters, for example in the 6th, 12th, 18th, … letter after an initial letter (the skip in this case would be said to be 6). Some, but not all of the time, the codes searchers look for minimal ELSs with the skip as small as possible [7].
The codes I’ll discuss here are of two distinct types: clusters of words [8] near each other, and the famous Rabbis experiment [9] which appeared in the journal Statistical Science [10]. There are three [11] fundamental problems with the codes:
- Any text of similar size will have similar clusters of words to those found in the Torah.
- The probabilities quoted for the word clusters are computed by methods contrary to the accepted laws of probability and used in situations where it is essentially impossible to assign meaningful probabilities.
- The famous Rabbis experiment has many flaws in both its data and in its method. The largest flaw, one that, by itself, already entirely explains what was found, is the subjective nature of the list of appellations used for the Rabbis.
I intend to address each of these points in turn and then the issues of whether there is a mesora for minimal ELSs, the significance of publication in Statistical Science, and finally, the view of the scientific community towards the issue of validity of the codes claims.
Problems with Word Clusters
The first two fundamental problems with the codes deal with word clusters. Let me explain what I mean by point 1, which says that any text of similar size will have similar clusters of words to those found in the Torah. Then I’ll present four illustrative examples. It is critical to note that the word used before "clusters" is "similar" and not "same". After finding something in the Torah, codes proponents are fond of pointing out that they looked for the same thing elsewhere and didn’t find it. That’s not surprising at all because it is only similar clusters that will be found elsewhere, not the exact same words. These phenomena invariably depend on what scientists call degrees of freedom and which I’ll refer to as wiggle room.
The point is that the searcher can make many choices in looking for codes. For example, if you are looking for holiday clusters, which holiday do you look for and once you pick the holiday, say Chanukah, which words do you look for? In a Chanukah cluster, you could look for names of the Maccabees and other historic figures, words connected with the Temple, words connected with miracles in general, words connected with candles or even words like latke, dreidl and sufganyiot. Moreover, the Hebrew language multiplies each possibility into many. Do I place a hey (for the definite article) in front? Do I include the extra yuds and vavs that can occur in variants of some words?
It is the wiggle room that allows an ELS cluster to be found in the Torah and it’s the same wiggle room that allows an ELS cluster to be found in other texts. But the very nature of wiggle room is that you have freedom in choices so the precise alternatives necessary to get the cluster in one text will be different from the alternatives needed to find something in another text. The clusters in the two texts will be similar but not the same. The quality of the clusters will be largely determined by the creativity of the searcher rather than by anything intrinsic to the underlying text.
As a first example of this phenomena, please consider the following figure, which is an impressive cluster of words associated with Chanukah.
The phrase that anchors the cluster is the ELS for Ner Chanukah (Chanukah candle) which starts from the lower right (last line) and swings up on a diagonal every fourth line [12]. It is a minimal ELS in the entire text from which this picture is a fragment. The other words are all minimal (in the entire text) ELSs of Chanukah related words, for example, Menorah. Chanukah clusters have been a popular topic with codes searchers and none has more words within a short space than this one [13].
This example is an impressive cluster that might lead you to suspect that it didn’t occur by chance. However, you should know that it is not a code in the Torah but rather a code in the Hebrew translation of War and Peace [14].
As a second example, consider one of the most charming of the codes. The names of 25 trees were found [15] in the section of Bereishit [16] where the Garden of Eden is described. They were all found as ELSs (although not minimal ELSs) in this section. Dr. Brendan McKay [17] took that same section and did 100 random permutations [18] of its letters and found that all 25 tree names still appeared as ELSs in 58 out of the 100. I did the same test on War and Peace using the section of the document that started at the exact same letter position and had the same length [19]. I found that all 25 tree names appear in this analogous section of War and Peace. I also found out of 100 random permutations that 59 had all the names (and each one of the hundred had at least 23 names.).
I’ve met many people who expressed amazement at the very existence of all these words in the one section of Bereishit. Our initial tests show that this fact alone has absolutely no significance. We then looked more closely at Prof. Michelson’s article and saw that he said that since the words are short, one shouldn’t be as surprised at the occurrence of all these words as by the fact that they are in ELSs with especially short skips. Indeed, he estimated the probability of the trees appearing as such short ELSs was 1 in 100,000.
So Dr. McKay and I decided to do some additional research on the subject. First we decided to measure the size of a set of ELSs by the total sum of the skips of the individual words. We did the test with the original Bereishit text and original War and Peace text and 1000 permutations of each. The actual Bereishit text scored in 8th place out of the 2002 sample texts. That implies that an estimate [20] for the probability is about 1 in 125, a far cry from the 1 in 100,000 but still something that appears to have some significance.
But before we could be sure of this significance, we needed to examine whether perhaps, the phenomenon could be explained by degrees of freedom that are present. There are two obvious places where one might think there are possible choices – exactly which trees to search for and how to spell them. Prof. Michelson explained that Prof. Rips took all of the trees found in the Torah according to the book The Fauna and Flora of the Torah by Yehuda Feliks. In addition to looking in the book, we decided to consult Prof. Feliks, who is, after all, a recognized expert on this subject.
A friend of mine met with Prof. Feliks in Jerusalem who informed us that first of all, there were three trees from the Torah missing from the list, and, secondly, that some of the spellings used differed from those in his book. For the three additional trees (Comiphora Africana, Astragalus, Cinnamon), we found that one doesn’t even occur as an ELS in that section of Bereishit and that the other two occur only as very long ELSs.
Dr. McKay then redid the experiment [21] with the spellings provided by Prof. Feliks (only using those trees that occur as ELSs) and out of 2002 permutations, the actual Bereishit text ranked 235. This 1 in 10 ranking is at a level that statisticians never regard as statistically significant. We concluded that wiggle room can explain the apparently significant result obtained by Prof. Rips and quoted by Prof. Michelson.
As a third example, I mention a challenge of Michael Drosnin [22] who stated in a Newsweek interview on June 9, 1997: "When my critics find a message about the assassination of a prime minister encrypted in Moby Dick, I'll believe them." So Brendan McKay looked there and found [23] clusters based on the names of Indira Ghandi, Rene Moawad, Leon Trotsky, Martin Luther King, Chancellor Engelbert Dollfuss, Sirhan Sirhan, Abraham Lincoln and Yitchak Rabin. For example, McKay found Abe, Lincoln and bang as ELSs near the plain text word killed. [24]
Sometimes people ask me about Rav Weissmandl’s examples. Here is Aish haTorah’s position on his ELSs [25] : "Only one or two of Rabbi Weissmandl’s word patterns are presented at Discovery, for the pedagogical purpose and process of showing that they are NOT Codes (i.e., word patterns that were deliberately placed in the Torah by G-d) and how one can be easily fooled into thinking that they are, and how referring to these accidental ‘patterns’ as Codes is sheer nonsense". It is unfortunate that by using Rav Weissmandl’s codes for something for which they were not intended, Aish has decided that it is necessary to kill them.
Mr. Witztum’s Closing Example
In my original article, I said: "Doubts that I may have had about publishing my conclusions are overridden, I feel, by a gemara in Shabbos, which says, ‘The seal of HaKadosh Baruch Hu is Truth.’". Mr. Witztum, in his reply, presents what he says is this concept encoded as in ELS in Bereishit. He found the Hebrew word for seal and variants as plain text near an ELS for Elokim Emes and quotes a very small probability for what he found [26] . As my fourth example of wiggle room in word clusters, I’ll consider this example [27].
The quote I used appears three times in the Babylonian Talmud with those exact words. I have no idea what Mr. Witztum did or didn’t look for, but it is clear if he was seeking that quote what he should have looked for are ELSs for seal, truth and HaKadosh Baruch Hu near each other. In fact "HaKadosh Baruch Hu" is 12 letters, too long to expect to be found as in ELS in Bereishit. Indeed, it is not found. That’s OK because the abbreviation HKBH is standard [28]. But searches for this abbreviation against variants of seal and truth yield no ELSs especially near each other.
Were Mr. Witztum a codes researcher as he claims, he’d have informed us that he tried that search and it failed. But he’s a code searcher rather than an impartial researcher. His method seems to be to keep searching until he finds something. With this attitude, the probability that he’ll find something to report on is one out of one, not one out of millions.
So instead of searching for my original quote, he searches for something which is conceptually related but with rather different words. You should note in passing that Elokim Emes is usually translated as saying that G-d, Himself, (not his seal) is True. You should also note that while words in most of Mr. Witztum’s clusters are ELSs, in this case he takes some of the words as plain text because the example doesn’t work if one looks for ELSs for each of the phrases. But most of all, you should note that once one allows oneself to use an alternate name to HKBH, there are many other possibilities – for example, he could have looked for Elokim Chayim, Melech Olam, or HaShem Elokim. You should now understand what wiggle room is.
No doubt, faced with this criticism, Mr. Witztum could (and perhaps will) produce a story explaining why it was reasonable to search for this particular combination although he didn’t do that in his Jewish Action article. Just as one could find a story for Elokim Chayim, Melech Olam, or HaShem Elokim. You can always find a story once you locate a cluster.
Problems with Calculations of Probabilities
Word clusters as presented by Mr. Witztum and other codes proponents are often accompanied by reports of fantastic probabilities. The second basic fact is that these probabilities are computed by methods contrary to the accepted laws of probability and used in situations where it is essentially impossible to assign meaningful probabilities. The other basic points require little mathematical sophistication to understand, but because the correct calculation of probabilities is a subtle science, I cannot explain this fully here since I don’t think you want to read a textbook on Probability Theory. So I’ll have to settle for trying to briefly explain the most serious error and give pointers to the mathematically knowledgeable reader of what some of the other issues are. This is one place where I’m going to have to ask you to rely on the judgement of those professionals whose job it is evaluate mathematical arguments.
A key issue involves the notion of independence. Flip a coin ten times and write down the results as a string of Hs and Ts. The probability you get the string HHHHHHHHHH is 1 in 1024, that is one half times one half times one half times… (ten times). Now flip a coin one time and write down the result of that single flip ten times, repeating the same letter. The probability of HHHHHHHHHH is exactly one half. The difference, of course, is that in the first case, the letters are independent of one another while in the other they are highly dependent. In real life, things are rarely totally independent nor totally dependent which make the calculations of probabilities exceedingly difficult.
Invariably, the large numbers in Mr. Witztum’s calculation rely on multiplying together lots of not so large numbers (as in the above case one gets 1024 by multiplying 2 times itself ten times) by assuming independence but in situations where independence is not a valid assumption.
Besides assuming independence in situations where it is not valid, the codes proponent’s calculations suffer from other problems [29], among them:
- they either ignore the "wiggle room" or consider a very, very small subset of possible wiggles
- they are a posteriori rather than a priori [30] (I discussed this in my Jewish Action article)
- they sometimes use as a base the minimal text that contains the cluster rather than the text actually searched.
But you needn’t take my word on the fact that clusters have no statistical significance. Consider these two quotes:
- "The only statistically significant ELSs (i.e., Codes) that have been discovered so far are the statistically verified Rabbis example"
- "Other examples are presented (Sadat, Diabetes, Gulf War, etc.) that employ the same methodology as Witztum's Famous Rabbis experiment but, for technical reasons, cannot be statistically measured. This limitation is made clear and they are presented only as a lead-up to, and an explanation of, the methodology used for the Famous Rabbis experiment."
These quotes are from Aish’s Discovery Seminar website [31].
The Nature of the Famous Rabbi's Experiment
Before turning to the most serious flaw in the Famous Rabbis experiment, I want to make sure the reader understands the nature of the evidence since it is really very different from the simple word clusters discussed so far [32]. In the article referenced in footnote 10 that I’ll call WRR, the authors looked at appellations and dates associated to 32 moderately famous Rabbis. By appellations, I mean some of the manifold ways that Rabbis are called. Consider, for example, the Rambam and the Chofetz Chaim, appellations for Moshe ben Maimon and Yisroel Meir Kagan and you’ll understand what is meant by an appellation. Most personalities have multiple appellations, for example, in WRR’s parallel very famous Rabbis list, the Rambam appears as HaRambam and Rabbi Moshe [33].
They took appellations of the Rabbis and their dates of birth and/or death and looked at the correlations between their ELSs and found a high statistical correlation for these lists in Bereishit but not in the test text of War and Peace.
You need to understand that the evidence is purely statistical. Many people I’ve talked to left codes presentations with the impression that all 32 Rabbis’ names were found close to their dates of birth or death. This is not so [34]. For example, taking into account multiple dates and appellations, there are 298 pairs. Of these 135 (45%) don’t even occur as ELS pairs.
WRR assign a number between 1/125 and 1 to each pair that they call the c-value with a small c-value indicating closeness and a large value less closeness. In essence, their evidence is that considerably more than a fifth of the remaining pairs have a c-value between 0 and 0.2, indicating closeness. But 8 of the pairs have a c-value between 0.8 and 1.0 so that it is certainly not true (even after throwing out the 45% of examples that don’t occur at all) that all the Rabbis’ appellations appear near their dates.
Moreover, Dr. McKay did an interesting test. He took the minimal ELS for HaRambam (the version of the name that WRR use) and looked for its distance (using WRR c-values) from all the dates of the year using the WRR form of dates. Of the 1064 possible dates [35], only 930 occurred as ELSs and he ranked these dates in c-value order against Rambam’s name. When he looked at distances for the four dates of Rambam’s birth and death that are among the 930, he found they ranked at positions 332, 686, 696 and 890. So Rambam’s name does not occur at all close to his dates of birth and death compared to other dates.
You should also know that the c-values are defined in a rather indirect way that doesn’t strictly correspond to our intuition of closeness. It is theoretically [36] possible for a pair of words with the minimal ELS for one totally in the parsha of Bereishit and the other in the parsha of Vayechi [37] to have a c-value of 1/125 (as close as possible according to c-values) and it is also theoretically possible to have two minimal ELSs which intersect and have a c-value of 1 (as far apart as possible according to c-values).
The definition of c-value is extremely complex. If I were attempting to assign a distance between the encodings of two words, I’d look for them as minimal ELSs and measure the number of letters between the centers of those ELSs. This simple-minded method involves computing a single distance in the text. In contrast, the WRR method of computing c-value, of say two 5 letter words typically involves not 1, not 2, not 10 or 20 but the computation of over 6 million distances. You read that right – over 6 million [38].
I’d emphasize that neither the purely statistical nature of the evidence nor the complex method of measuring closeness are central to the issue of the scientific validity of the experiment. But they raise two kinds of questions. The first is theological: if HaShem placed this evidence there, would he do it such an incredibly indirect and imperfect way? After all, the Torah represents Truth, not truth in some average statistical sense.
The second is that the complexity of the experiment suggests that the result may be sensitive to changes of the method of measuring distances and the statistical method used [39]. Indeed, Prof. Haralick (whom Rabbi Mechanic identifies as having positive leanings towards the codes) has redone the WRR experiment using a different method of measuring distances that he arrived at with his vast experience in pattern recognition. He also changed the method of statistical analysis but he used the same lists of dates and appellations. The probability dropped from the 1 in 62,500 quoted by WRR to only 1 in 400 [40]. This level is one that is at best marginal for tests of statistical significance. More importantly, it was arrived at by only varying the method. I’m about to claim that the data WRR use is seriously flawed. Prof. Haralick found a substantial decrease in probabilities by varying the method without even changing the most faulty part of the experiment.
The Fatal Flaw in the Famous Rabbis Experiment
Now I turn to the data in the famous Rabbis experiment and the most serious problem with the WRR paper. While there are legitimate questions about the choice of dates, the place of the most significant wiggle room, which as we’ll see can totally explain what was found, is the list of appellations that WRR used.
On its web site, Aish has put its finger on what is needed for a test to be scientifically valid [41]: " The words to be searched for must be derivable by a priori specified and repeatable methods". The list must be objective so that another researcher can replicate the choices in the list. It should not be based on subjective judgements.
Let’s see what WRR have to say about the choice of the list of appellations. Their main statement is that "The list of appellations for each personality was provided by Professor S. Z. Havlin, of the Department of Bibliography and Librarianship at Bar Ilan University, on the basis of a computer search of the Responsa database at that university". They also provide four spelling rules that they use but that is all they say about how the list was constructed.
When mathematicians began to seriously look at the details, many questions were raised – for example, it turns out that for 12 of the 32 Rabbis, Prof. Havlin didn’t use the Responsa database [42]. In response to a list of queries, Prof. Havlin provided a statement with various details about his construction of the list [43]. Prof. Havlin’s testimony makes it clear that the list is not objective. That is, two well-meaning scholars starting from scratch and working independently would arrive at very different lists. He calls a large section of his document "Professional Judgements" and says he had to use discretion in making choices. Not even in Alice in Wonderland would a list that required judgements on the part of the list maker be called objective. Indeed, since he says in several places that he can’t recall why he didn’t include certain names, it is clear that not only wouldn’t the list be the same if another scholar produced it, it wouldn’t be the same if the same scholar produced it ten years later.
Along the way, Prof. Havlin poses a large number of rules, many of which are clearly arbitrary. For example, he states that appellations must be pronounceable to be included? Why? Any Talmudic student seeing the letters beis-yud, would know that the Bais Yosef (Rabbi Yosef Caro) is intended. But Prof. Havlin says that this isn’t a legitimate appellation because it isn’t pronounceable. If HaShem placed Rabbis’ names in the written Torah, why would He fail to use the most common written nicknames for those Rabbis? The pronouncability rule is an arbitrary one that another scholar making the list might not use. In addition since the rules have only been made explicit years after the experiment was published, the lists also lack the property of being a priori.
It is critical to realize that the choice of rules in making the list is as much a part of making the list as then applying those rules. Once the list is subjective, the experiment becomes a test of the list, not of Bereishit. Rabbi Mechanic’s clear criteria, with which the scientific world would agree, that the method must be repeatable is violated. The experiment isn’t valid science.
You might wonder though if the wiggle room provided by the subjective nature of the list of appellations is enough to explain the striking results of WRR. And here the interesting paper of Bar-Natan and McKay [44] (henceforth BNMK) provides the answer. They produced another list of reasonable appellations, many of them in common with the WRR list [45].
Recall that WRR found that the WRR list had a statistical correlation with the dates when looked for in Bereishit but not in War and Peace. Similarly, BNMK found that the BNMK list had a statistical correlation with the dates when looked for in War and Peace but not when looked for in Bereishit. The correlation that BNMK found in War and Peace for their list was as statistically significant as what WRR found in Bereishit for their experiment. So BNMK show that the wiggle room in the list is indeed enough to totally explain the phenomenon observed by WRR.
Mr. Witztum has a long reply posted on the web trying to explain why the WRR list is valid and the BNMK list is not [46]. It is totally unconvincing because it again depends almost entirely on reference to arbitrary rules. One clear example of the issues concerns whether the name of Rav David Oppenheim when written in Hebrew characters has one yud or two yuds before the final mem. WWR use one yud while BNMK use two. The original WRR paper told us that the list was prepared by searching the Responsa database. So what does this database say about this variant spelling? The one yud version used by WRR appears twice in the Responsa database. The two yud version that BNMK use appears over 30 times including as a signature in some of the responsa written by Rav Oppenheim. How does Mr. Witztum defend this? He can’t use the four rules that appear in the original WWR paper nor the dozen plus rules that are implicit in Prof. Havlin’s letter since none of them applies to this case. So he invents a new rule about how one should transliterate words whose original was German. I ask you: If Rav Oppenheim’s name were encoded in a special way in Bereishit, do you think it would use the spelling Rav Oppenheim used in his own responsa or would it use a variant that Mr. Witztum decided was correct on the basis of some arbitrary rule he made up? [47]
On the Issue of Honesty
Mr. Witztum has said that I’m secretly accusing him of dishonesty and Rabbi Mechanic claims that I am calling their list "manipulated". That is not true. I have no need to consider the issue of honesty because it is irrelevant. Once the list is seen to be subjective, the case of whether this is scientific proof is closed (it is not) and the issue of whether the list was arrived at in an honest way or dishonest way, with or without unknown bias, becomes moot. Indeed, if I had firm evidence that they had cheated, I could not publicly present it on account of the laws of lashon hora. The experiment is seen to be invalid on the basis of Havlin’s own testimony that the list is subjective, so what purpose would there be in my exposing the evidence of dishonesty? [48]
It is important to realize that it is precisely because of this issue of "Are you accusing me of dishonesty or of unintentional bias?" that science demands repeatability in experiments. There are too many examples in the history of science of researchers fooling themselves and/or others when they do a subjective analysis. For this reason, science has to reject situations where the result depends entirely on presumed honesty of the research because independent researchers cannot repeat the entire experiment independently. Once researchers have to say "Don’t you believe we’re honest?" they’ve admitted that the research involves subjective elements and so admitted that their experiment is flawed.
There is a further element to consider. Is it really sound to approach someone who is struggling with the issue of faith that the Torah is divine with the notion that you have a proof of this cardinal point of faith if only they will have a different kind of faith -- that some researchers were honest?
A comment on the Nations Experiment
When I met with Professor Rips in Jerusalem, he agreed that there were lots of degrees of freedom in the famous Rabbis experiment, which could raise questions. He urged me to instead look at paper he gave me, also by Witztum, Rips and Rosenberg [49], which contained what he called the Nations experiment [50]. This paper looked for correlations in the sense of the original WRR paper between the names of the 70 nations from Parshat Noach and ELS for the phrase <attribute+nation> for certain attributes [51]. Their test claimed that the possibility of what they found occurring by chance is 1 out of an astounding 250 million.
There are a myriad of possible attributes you could look for – for instance language of Magog, King of Magog, leader of Magog, and religion of Magog but Rips said they picked the attributes mentioned in the writing of the Vilna Gaon so there were no degrees of freedom in the problem. Of course, this ignores the biggest source of wiggle room – why look at the writing of the Gaon and not some other authority (and why look at the particular writing of the Gaon that they picked – the Gaon wrote about attributes of the Nations at least four times). But let’s see if there is any other wiggle room. Here is an exact quote of what this paper says:
"Jewish tradition further tells us (Hagra, 1905), that a nation has the following four characteristics:
- its name,
- its country,
- its language,
- its script."
