This is available as part of a book offering that is free for community members of Cyb3rSynLabs. Please check here (https://www.cyb3rsynlabs.com/c/books/) for Second Order Cybernetics Essays for Silicon Valley. The e-book version is available here (https://www.cyb3rsyn.com/products/soc-book)
Stay safe and always keep on learning…
In case you missed it, my last post was Cybernetics of Kindness:
I have written a lot about the problem of induction before. This was explained very well by the great Scottish philosopher, David Hume. Hume looked at the basis of beliefs that we hold such as:
Hume noted that there is no uniformity in nature. In other words, it is not rational to believe that what has happened in the past will happen again in the future. Just because, we have seen the sun rise every single day of our lives, it does not guarantee that it will rise again tomorrow. We are using our experience of the sun rising to believe that it will rise again tomorrow. Even though, this might be irrational, Hume does not deny that we may see the belief of the sun rising as a sensible proposition. He notes:
None but a fool or madman will ever pretend to dispute the authority of experience, or to reject that great guide of human life.
It’s just that we cannot use logic to back this proposition up. We cannot conclude that the future is going to resemble the past, no matter how many examples of the past we have. We cannot simply use experience of the past because the only experience we have is of the past, and not of the future. Hume noted that to propose that the next future event will resemble the past because our most recent “future event” (the last experience event) resembled the past is circular:
All our experimental conclusions proceed upon the supposition that the future will be conformable to the past. To endeavor, therefore, the proof of this last supposition by probable arguments, or arguments regarding existence, must be evidently going in a circle, and taking that for granted, which is the very point in question.
Hume concluded that we fall prey to the problem of induction because we are creatures of habits:
For wherever the repetition of any act or operation produces a propensity to renew the same act or operation, without being impelled by any reasoning or process of the understanding, we always say, that this propensity is the effect of Custom. By employing this word, we pretend not to have given the ultimate reason of such a propensity. We only point out a principle of human nature, which is universally acknowledged, and which is well known by its effects.
In other words, it is our human nature to identify and seek patterns, use them to make predictions of the future. This is just how we are wired. We do this unconsciously. Our brains are prediction engines. We cannot help but do this. I will go further with this idea by utilizing a brilliant example from the wonderful American philosopher Charles Sanders Peirce. Peirce in 1868 wrote about an experiment to reveal the blind spot in the retina:
Does the reader know of the blind spot on the retina? Take a number of this journal, turn over the cover so as to expose the white paper, lay it sideways upon the table before which you must sit, and put two cents upon it, one near the left-hand edge, and the other to the right. Put your left hand over your left eye, and with the right eye look steadily at the left-hand cent. Then, with your right hand, move the right-hand cent (which is now plainly seen) towards the left hand. When it comes to a place near the middle of the page it will disappear—you cannot see it without turning your eye. Bring it nearer to the other cent, or carry it further away, and it will reappear; but at that particular spot it cannot be seen. Thus, it appears that there is a blind spot nearly in the middle of the retina; and this is confirmed by anatomy. It follows that the space we immediately see (when one eye is closed) is not, as we had imagined, a continuous oval, but is a ring, the filling up of which must be the work of the intellect. What more striking example could be desired of the impossibility of distinguishing intellectual results from intuitional data, by mere contemplation?
I highly encourage the reader to check this out, if they have not heard of this experiment. In fact, I welcome the reader to draw a line and then place the coin on the line. Doing so, the reader will see that the coin vanishes, however the line still remains visible in the periphery. This means that even though, our eye “sees” a ring, the brain actually fills it out and makes us see a “whole” picture. To add to this wonderful capability of our interpretative framework, the image that falls on our retina is actually upside-down. Yet, our brain makes it the “right-side” up. This would mean that newborn babies may actually see the world upside down and with voids, but at some point, the interpretative framework changes to correct it so that we see the world “correctly”.
How does our brain know to do this? The answer to this is that it was evolutionarily beneficial for our ancestors to do this, just like our custom to look for patterns. This is what Lila Gatlin would refer to as a D1 constraint. This is a context-free constraint that was evolutionarily passed down from generation to generation. This is a constraint that acts in any situation. In other words, to quote Alicia Juarrero, it is context free.
To go past this constraint, we have to use second order thinking. In other words, we have to think about thinking; we have to learn about learning; we have to look at understanding understanding. I welcome the reader to look at the posts I have written on this matter. I will finish with two quotes to further meditate on this:
Only when you realize you are blind, can you see. (Paraphrasing Heinz von Foerster)
The quieter you become, the more you can hear. – Ram Dass
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning…
In case you missed it, my last post was The Cybernetics of “Here & Now” and “There & Then”
I have had some good conversations recently about epistemology. Today’s post is influenced by those conversations. In today’s post, I am looking at Bayesian epistemology, something that I am very influenced by. As the readers of my blog may know, I am a student of Cybernetics. One of the main starting points in Cybernetics is that we are informationally closed. This means that information cannot enter into us from outside. This may be evident for any teachers in my viewership. You are not able to open up a student’s brain and pour information in as a commodity and then afterwards seal it back up. What happens instead is that the teacher perturbs the student and the student in turn generates meaning out of the perturbation. This would also mean that all knowledge is personal. This is something that was taught by Michael Polanyi.
How we know something is based on what we already know. The obvious question at this juncture is what about the first knowledge? Ross Ashby, one of the pioneers of Cybernetics, has written that there are two main forms of regulations. One is the gene pattern, something that was developed over generations through the evolutionary process. An example of this is the impulse of a baby to grab or to breastfeed without any training. The second is the ability to learn. The ability to learn amplifies the chance of survival of the organism. In our species, this allows us to literally reach for the celestial bodies.
If one accepts that we are informationally closed, then one has to also accept that we do not have direct access to the external reality. What we have access to is what we make sense of from experiencing the external perturbations. Cybernetics aligns with constructivism, the philosophy that we construct a reality from our experience. Heinz von Foerster, one of my favorite Cyberneticians, postulated that our nervous system as a whole is organized in such a way (organizes itself in such a way) that it computes a stable reality. All we have is what we can perceive through our perception framework. The famous philosopher, Immanuel Kant, referred to this as the noumena (the reality that we don’t have direct access to) and the phenomena (the perceived representation of the external reality). We compute a reality based on our interpretive framework. This is just a version of the reality, and each one of us computes such a reality that is unique to each one of us. The stability comes from repeat interactions with the external reality, as well as with interactions with others. We do not exist in isolation from others. The more interactions we have the more we have the chance to “calibrate” it against each other.
With this framework, one does not start from ontology, instead one starts from epistemology. Epistemology deals with the theory of knowledge and ontology deals with being (what is out there). What I can talk about is what I know about rather than what is really out there.
Bayesian epistemology is based on induction. Induction is a process of reasoning where one makes a generalization from a series of observations. For example, if all the swans you have seen so far in your life are white swans, then induction would direct you to generalize that all swans are white. Induction assumes uniformity of nature, to quote the famous Scottish philosopher David Hume. This means that you assume that the future will resemble the past. Hume pointed out that induction is faulty because no matter how many observations one makes, one cannot assume that the future will resemble the past. We seek patterns in the world, and we make generalizations from them. Hume pointed out that we do this out of habit. While many people have tried to solve the problem of induction, nobody has really solved it.
All of this discussion lays the background for Bayesian epistemology. I will not go into the math of Bayesian statistics in this post. I will provide a general explanation instead. Bayesian epistemology puts forth that probability is not a characteristic of a phenomenon, but a statement about our epistemology. The probabilities we assign are not for THE reality but for the constructed reality. It is a statement about OUR uncertainty, and not about the uncertainty associated with the phenomenon itself. The Bayesian approach requires that we start with what we know. We start with stating our prior belief, and based on the evidence presented, we will modify our belief. This is termed as the “posterior” in Bayesian terms. Today’s posterior becomes tomorrow’s prior because “what we know now” is the posterior.
Another important thing to keep in mind is that one does not assign a 0 or 100% for your belief. Even if you see a coin with 10,000 heads in a row, you should not assume that the coin is double headed. This would be jumping into the pit of the problem of induction. We can keep updating our prior based on evidence without reaching 100%.
I will write more on this topic. I wanted to start off with an introductory post and follow up with additional discussions. I will finish with some appealing points of Bayesian epistemology.
Bayesian epistemology is self-correcting – Bayesian statistics has the tendency to cut down your overconfidence or underconfidence. The new evidence presented over several iterations corrects your over or under reach into confidence.
Bayesian epistemology is observer dependent and context sensitive – As noted above, probability in Bayesian epistemology is a statement of the observer’s belief. The framework is entirely dependent on the observer and the context around sensemaking. You do not remove the observer out of the observation. In this regard, Bayesian framework is hermeneutical. We bring our biases to the equation, and we put money where our mouth is by assigning a probability value to it.
Circularity – There is an aspect of circularity in Bayesian framework in that today’s prior becomes tomorrow’s posterior as noted before.
Second Order Nature – The Bayesian framework requires that you be open to changing your beliefs. It requires you to challenge your assumptions and remain open to correcting your belief system. There is an aspect of error correction in this. You realize that you have cognitive blind spots. Knowing this allows us to better our sensemaking ability. We try to be “less wrong” than “more right”.
Conditionality – The Bayesian framework utilizes conditional probability. You see that phenomena or events do not exist in isolation. They are connected to each other and therefore require us to look at the holistic viewpoint.
Coherence not Correspondence – The use of priors forces us to use what we know. To use Willard Van Orman Quine’s phrase, we have a “web of belief”. Our priors must make sense with all the other beliefs we already have in place. This supports the coherence theory of truth instead of the realist’s favorite correspondence theory of truth. I welcome the reader to pursue this with this post.
Consistency not completeness – The idea of a consistency over completeness is quite fascinating. This is mainly due to the limitation of our nervous system to have a true representation of the reality. There is a common belief that we live with uncertainty, but our nervous system strives to provide us a stable version of reality, one that is devoid of uncertainties. This is a fascinating idea. We are able to think about this only from a second order standpoint. We are able to ponder about our cognitive blind spots because we are able to do second order cybernetics. We are able to think about thinking. We are able to put ourselves into the observed.
I will finish with an excellent quote from Albert Einstein:
“As far as the laws of mathematics refer to reality, they are not certain; as far as they are certain, they do not refer to reality”.
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning…
In case you missed it, my last post was Error Correction of Error Correction:
This is available as part of a book offering that is free for community members of Cyb3rSynLabs. Please check here (https://www.cyb3rsynlabs.com/c/books/) for Second Order Cybernetics Essays for Silicon Valley. The e-book version is available here (https://www.cyb3rsyn.com/products/soc-book)
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning… In case you missed it, my last post was The Open Concept of Systems:
In today’s post, I am looking at the famous American philosopher Morris Weitz’s Closed and Open Concepts. Weitz studied aesthetics, the branch of philosophy interested in beauty and taste. He looked at the simple or not so simple question of “how do you define art?” This might seem to be a simple question at first. As we try to answer this, we will soon find that this is not so easy to answer. This might remind you of Socrates and the Socratic method of asking questions. Socrates would ask questions such as what is virtue? For any answer he got, he would find a contradiction that would push the other person further and further into a corner. Weitz came out against this approach and said that the question “what is art?” is itself the wrong question. Instead, he said that you should ask “what sort of concept is art?” The general tendency amongst theorists is to use strict definitions about the essence of something. Weitz called this approach a “closed concept”. Weitz said:
If necessary and sufficient conditions for the application of a concept can be stated, the concept is a closed one. But this can happen only in logic or mathematics where concepts are constructed and completely defined. It cannot occur with empirically-descriptive and normative concepts unless we arbitrarily close them by stipulating the ranges of their uses.
In this fashion, Weitz noted that – Art, as the logic of the concept shows, has no set of necessary and sufficient properties, hence a theory of it is logically impossible and not merely factually difficult.
To contrast the closed concept with the open concept, Weitz stated:
A concept is open if its conditions of application are emendable and corrigible; i.e., if a situation or case can be imagined or secured which would call for some sort of decision on our part to extend the use of the concept to cover this, or to close the concept and invent a new one to deal with the new case and its new property.
Weitz had strong words against the theorists of Aesthetics wanting to confine the subject into a box:
Aesthetic theory is a logically vain attempt to define what cannot be defined, to state the necessary and sufficient properties of that which has no necessary and sufficient properties, to conceive the concept of art as closed when it’s very use reveals and demands its openness.
Weitz was a fan of Wittgenstein and seems to have been influenced by his idea of “what a game is?” In his posthumous book, Philosophical Investigations, Wittgenstein talked about how a concept such as a game can be defined. There are so many different games that you would be able to identify a game when you engage in it. They all have similarities but it is very hard to properly define a game in a closed concept sense. You know that Chess and Soccer (Football) are games, but also very different. Similarly, skating and polo are games, again of very different nature. They have family resemblances! Wittgenstein’s main point is that the meaning of a word is in its use. Weitz noted:
In his new work, Philosophical investigations, Wittgenstein raises as an illustrative question, What is a game? The traditional philosophical, theoretical answer would be in terms of some exhaustive set of properties common to an games. To this Wittgenstein says, let us consider what we call “games”: “I mean board-games, card-games, ball-games, Olympic games, and so on. What is common to them all?—Don’t say: ‘there must be something common, or they would not be called “games'” but look and see whether there is anything common to all.—For if you look at them you will not see something that is common to all, but similarities, relationships, and a whole series of them at that. … ” Card games are like board games in some respects but not in others. Not all games are amusing, nor is there always winning or losing or competition. Some games resemble others in some respects—that is all. What we find are no necessary and sufficient properties, only “a complicated network of similarities overlapping and crisscrossing,” such that we can say of games that they form a family with family resemblances and no common trait. If one asks what a game is, we pick out sample games, describe these, and add, “This and similar things are called ‘games.’ ” This is all we need to say and indeed all any of us knows about games. Knowing what a game is, is not knowing some real definition or theory but being able to recognize and explain games and to decide which among imaginary and new examples would or would not be called “games.”
In other words, a “game” is an open concept. How you define a game is specifically up to how you, as the observer, view the actual functioning of the concept. Weitz does note that it is possible to “close” an “open” concept in certain cases. The example he gives is that of “tragedy” and “Greek tragedy”. Tragedy is an open concept, whereas Greek tragedy is a closed concept. He notes:
Of course, there are legitimate and serviceable closed concepts in art. But these are always those whose boundaries of conditions have been drawn for a special purpose. Consider the difference, for example, between “tragedy” and “Greek tragedy. ” The first is open and must remain so to allow for the possibility of new conditions, e.g., a play in which the hero is not noble or fallen or in which there is no hero but other elements that are like those of plays we already call “tragedy.” The second is closed. The plays it can be applied to, the conditions under which it can be correctly used are all in, once the boundary, “Greek,” is drawn. Here the critic can work out a theory or real definition in which he lists the common properties at least of the extant Greek tragedies.
Systems:
I was fascinated with the idea of open and closed concepts. I think this has use in Systems Thinking. Often, systems are depicted as real entities in the world that one can change or fix. This is to me, the use of a closed concept in systems thinking. Systems, similar to art, should be viewed as an open concept. A system is entirely dependent upon who does the observation. If we have three observers, then there are at least three systems of the same phenomenon. To paraphrase Dominik Jarczewski, the question whether something is a system is not a factual problem. It is a decision problem. How you define your system is entirely contingent upon your worldview, your biases and your experiential realities. The knowledge of what is a system is not theoretical but practical. You can replace the word “art” in the previous section with “system”, and there will be no meaning lost.
Peter Checkland, the eminent Systems Thinker provides more light on this. He noted that there will be an observer who gives an account of the world, or part of it, in systems terms; the principle which makes them coherent entities; the means and mechanism by which they tend to maintain their integrity; their boundaries, inputs, outputs, and components; their structure. Finally their behavior may be described in terms of inputs and outputs via state descriptions.
If you are trying to understand a system, you must not view it as a closed concept. You must view it as an open concept, and this means that you have to try to understand where the other person is coming from, and how it is constructed by that person. In other words, how does the functioning of the coherent whole affect that person. It is easy to fall into the mindset that systems can be viewed as closed concepts, where the purpose, the whole, etc. are definable and understandable by everybody. You might be tempted to say that the whole is more important than the parts, as if your whole is accepted by everybody. You might think that holism is the way to do systems thinking, and that reductionism is a terrible idea. When you embrace systems as an open concept, you realize that holism can be as bad as reductionism and reductionism can be as good as holism. All you have are abstractions. Even the holism you look at, is a form of reductionism.
I will finish with some more food-for-thought idea from Weitz that systems thinking is a meta-discipline (replacing “art” with “system”):
If I may paraphrase Wittgenstein, we must not ask, What is the nature of any system x?, or even, according to the semanticist, What does “x” mean?, a transformation that leads to the disastrous interpretation of “system” as a name for some specifiable class of objects; but rather, What is the use or employment of “x”? What does “x” do in the language? This, I take it, is the initial question, the begin-all if not the end-all of any philosophical problem and solution. Thus, … our first problem is the elucidation of the actual employment of the concept of a system, to give a logical description of the actual functioning of the concept, including a description of the conditions under which we correctly use it or its correlates.
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning…
In case you missed it, my last post was Direct and Indirect Constraints:
[1] Art by Annie Jose
In today’s post, I am following on the theme of Lila Gatlin’s work on constraints and tying it up with cybernetics. Please refer to my previous posts here and here for additional background. As I discussed in the last post, Lila Gatlin used the analogy of language to explain the emergence of complexity in evolution. She postulated that lower complex organisms such as invertebrates focused on D1 constraints to ensure that the genetic material is passed on accurately over generations, while vertebrates maintained a constant level of D1 constraints and utilized D2 constraints to introduce novelty leading to complexification of the species. Gatlin noted that this is similar to Shannon’s second theorem which points out that if a message is encoded properly, then it can be sent over a noisy medium in a reliable manner. As Jeremy Campbell notes:
In Shannon’s theory, the essence of successful communication is that the message must be properly encoded before it is sent, so that it arrives at its destination just as it left the transmitter, intact and free from errors caused by the randomizing effects of noise. This means that a certain amount of redundancy must be built into the message at the source… In Gatlin’s new kind of natural selection, “second-theorem selection,” fitness is defined in terms very different and abstract than in classical theory of evolution. Fitness here is not a matter of strong bodies and prolific reproduction, but of genetic information coded according to Shannon’s principles.
The codes that made possible the so-called higher organisms, Gatlin suggests, were redundant enough to ensure transmission along the channel from DNA to protein without error, yet at the same time they possessed an entropy, in Shannon’s sense of “amount of potential information,” high enough to generate a large variety of possible messages.
Gatlin viewed that complexity arose from the ability to introduce more variety while at the same time maintaining accuracy in an optimal mix, similar to human language where there is always constant emergence of new and new ideas while the main grammar, syntax etc. are maintained. As Campbell continues:
In the course of evolution, certain living organisms acquired DNA messages which were coded in this optimum way, giving them a highly successful balance between variety and accuracy, a property also displayed by human languages. These winning creatures were the vertebrates, immensely innovative and versatile forms of life, whose arrival led to a speeding-up of evolution.
As Campbell puts it, vertebrates were agents of novelty. They were able to revolutionize their anatomy and body chemistry. They were able to evolve more rapidly and adapt to their surroundings. The first known vertebrate is a bottom-dwelling fish that lived over 350 million years ago. They had a heavy external skeleton that anchored them to the floor of the water-body. They evolved such that some of the spiny parts of the skeleton grew into fins. They also evolved such that they developed skull with openings for sense organs such as eyes, nose, ears etc. Later on, some of them developed limbs from the bony supports of fins, leading to the rise of amphibians.
What kind of error-correcting redundancy did he DNA of these evolutionary prize winners, the vertebrates, possess? It had to give them the freedom to be creative, to become something markedly different, for their emergence was made possible not merely by changes in the shape of a common skeleton, but rather by developing whole new parts and organs of the body. Yet this redundancy also had to provide them with the constraints needed to keep their genetic messages undistorted.
Gatlin defined the first type of redundancy, one that allows deviation from equiprobability as ‘D1 constraint’. This is also referred to as ‘governing constraint’. The second type of redundancy, one that allows deviation from independence was termed by Gatlin as ‘D2 constraint’, and this is also referred to as ‘enabling constraint’. Gatlin’s speculation was that vertebrates were able to use both D1 and D2 constraints to increase their complexification, ultimately leading to a high cognitive being such as our species, homo sapiens.
One of the pioneers in Cybernetics, Ross Ashby, also looked at a similar question. He was looking at the biological learning mechanisms of “advanced” organisms. Ashby identified that for lower complex organisms, the main source of regulation is their gene pattern. For Ashby, regulation is linked to their viability or survival. He noted that the lower complex organisms can rely just on their gene pattern to continue to survive in their environment. Ashby noted that they are adapted because their conditions have been constant over many generations. In other words, a low complex organism such as a hunting wasp can hunt and survive simply based on their genetic information. They do not need to learn to adapt, they can adapt with what they have. Ashby referred to this as direct regulation. With direct regulation, there is a limit to the adaptation. If the regularities of the environment change, the hunting wasp will not be able to survive. It relies on the regularities of the environment for its survival. Ashby contrasted this with indirect regulation. With indirect regulation, one is able to amplify adaptation. Indirect regulation is the learning mechanism that allows the organism to adapt. A great example for this is a kitten. As Ashby notes:
This (indirect regulation) is the learning mechanism. Its peculiarity is that the gene-pattern delegates part of its control over the organism to the environment. Thus, it does not specify in detail how a kitten shall catch a mouse, but provides a learning mechanism and a tendency to play, so that it is the mouse which teaches the kitten the finer points of how to catch mice.
The learning mechanism in its gene pattern does not directly teach the kitten to hunt for the mice. However, chasing the mice and interacting with it, trains the kitten how to catch the mice. As Ashby notes, the gene pattern is supplemented by the information supplied by the environment. Part of the regulation is delegated to the environment.
In the same way the gene-pattern, when it determines the growth of a learning animal, expends part of its resources in forming a brain that is adapted not only by details in the gene-pattern but also by details in the environment. The environment acts as the dictionary, while the hunting wasp, as it attacks its prey, is guided in detail by its genetic inheritance, the kitten is taught how to catch mice by the mice themselves. Thus, in the learning organism the information that comes to it by the gene-pattern is much supplemented by information supplied by the environment; so, the total adaptation possible, after learning, can exceed the quantity transmitted directly through the gene-pattern.
Ashby further notes:
As a channel of communication, it has a definite, finite capacity, Q say. If this capacity is used directly, then, by the law of requisite variety, the amount of regulation that the organism can use as defense against the environment cannot exceed Q. To this limit, the non-learning organisms must conform. If, however, the regulation is done indirectly, then the quantity Q, used appropriately, may enable the organism to achieve, against its environment, an amount of regulation much greater than Q. Thus, the learning organisms are no longer restricted by the limit.
In the same way the gene-pattern, when it determines the growth of a learning animal, expends part of its resources in forming a brain that is adapted not only by details in the gene-pattern but also by details in the environment. The environment acts as the dictionary, while the hunting wasp, as it attacks its prey, is guided in detail by its genetic inheritance, the kitten is taught how to catch mice by the mice themselves. Thus, in the learning organism the information that comes to it by the gene-pattern is much supplemented by information supplied by the environment; so the total adaptation possible, after learning, can exceed the quantity transmitted directly through the gene-pattern.
As I look at Ashby’s ideas, I cannot help but see similarities between the D1/D2 constraints and Direct/Indirect regulation respectively. Indirect regulation, similar to enabling constraints, helps the organism adapt to its environment by connecting things together. Indirect regulation has a second order nature to it such as learning how to learn. It works on being open to possibilities when interacting with the environment. It brings novelty into the situation. Similar to governing constraints, direct regulation focuses only on the accuracy of the ‘message’. Nothing additional or any form of amplification is not possible. Direct regulation is hardwired, whereas indirect regulation is enabling. Direct regulation is context-free, whereas indirect regulation is context-sensitive. What the hunting wasp does is entirely reliant on its gene pattern, no matter the situation, whereas, what a kitten does is entirely dependent on the context of the situation.
Final Words:
Cybernetics can be looked at as the study of possibilities, especially why out of all the possibilities only certain outcomes occur. There are strong undercurrents to information theory in Cybernetics. For example, in information theory entropy is a measure of how many messages might have been sent, but were not. In other words, if there are a lot of possible messages available, and only one message is selected, then it eliminates a lot of uncertainty. Therefore, this represents a high information scenario. Indirect regulation allows us to look at the different possibilities and adapt as needed. Additionally, indirect regulation allows retaining the successes and failures and the lessons learned from them.
I will finish with a great lesson from Ashby to explain the idea of the indirect regulation:
If a child wanted to discover the meanings of English words, and his father had only ten minutes available for instruction, the father would have two possible modes of action. One is to use the ten minutes in telling the child the meanings of as many words as can be described in that time. Clearly there is a limit to the number of words that can be so explained. This is the direct method. The indirect method is for the father to spend the ten minutes showing the child how to use a dictionary. At the end of the ten minutes the child is, in one sense, no better off; for not a single word has been added to his vocabulary. Nevertheless, the second method has a fundamental advantage; for in the future the number of words that the child can understand is no longer bounded by the limit imposed by the ten minutes. The reason is that if the information about meanings has to come through the father directly, it is limited to ten-minutes’ worth; in the indirect method the information comes partly through the father and partly through another channel (the dictionary) that the father’s ten-minute act has made available.
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning…
In case you missed it, my last post was D1 and D2 Constraints:
In today’s post, I am following up from my last post and looking further at the idea of constraints as proposed by Dr. Lila Gatlin. Gatlin was an American biophysicist, who used the idea of information theory to propose an information-processing aspect of life. In information theory, the ‘constraints’ are the ‘redundancies’ utilized for the transmission of the message. Gatlin’s use of this idea from an evolutionary standpoint is quite remarkable. I will explain the idea of redundancies in language using an example I have used before here. This is the famous idea that if a monkey had infinite time on its hands and a typewriter, it will at some point, type out the entire works of Shakespeare, just by randomly clicking on the typewriter keys. It is obviously highly unlikely that a monkey can actually do this. In fact, this was investigated further by William R. Bennett, Jr., a Yale professor of Engineering. As Jeremy Campbell, in his wonderful book, Grammatical Man, notes:
Bennett… using computers, has calculated that if a trillion monkeys were to type ten keys a second at random, it would take more thana trillion times as long as the universe has been in existence merely to produce the sentence “To be, or not to be: that is the question.”
This is mainly because the keyboard of a typewriter does not truly reflect the alphabet as they are used in English. The typewriter keyboard has only one key for each letter. This means that every letter has the same chance of being struck. From an information theory standpoint, this represents a maximum entropy scenario. Any letter can come next since they all have the same probability of being struck. In English, however, the distribution of letters is not the same. Some letters such as “E” are more likely to occur than say “Q”. This is a form of “redundancy” in language. Here redundancy refers to regularities, something that occurs on a regular basis. Gatlin referred to this redundancy as “D1”, which she described as divergence from equiprobability. Bennett used this redundancy next in his experiment. This will be like saying that some letters now had lot more keys on the typewriter so that they are more likely to be clicked. Campbell continues:
Bennett has shown that by applying certain quite simple rules of probability, so that the typewriter keys were not struck completely at random, imaginary monkeys could, in a matter of minutes, turn out passages which contain striking resemblances to lines from Shakespeare’s plays. He supplied his computers with the twenty-six letters of the alphabet, a space and an apostrophe. Then, using Act Three of Hamlet as his statistical model, Bennett wrote a program arranging for certain letters to appear more frequently than others, on the average, just as they do in the play, where the four most common letters are e, o, t, and a, and the four least common letters are j, n, q, and z. Given these instructions, the computer monkeys still wrote gibberish, but no it had a slight hint of structure.
The next type of redundancy in English is the divergence from independence. In English, we know that certain letters are more likely to come together. For example, “ing” or “qu” or “ion”. If we see an “i” and “o”, then there is high chance that the next letter is going to be an “n”. If we see a “q”, we can be fairly sure that the next letter is going to be a “u”. The occurrence of one letter makes the occurrence of another letter highly likely. In other words, this type of redundancy makes the letter interdependent rather than independent. Gatlin referred to this as “D2”. Bennett utilized this redundancy for his experiment:
Next, Bennett programmed in some statistical rules about which letters are likely to appear at the beginning and end of words, and which pairs of letters, such as th, he, qu, and ex, are used most often. This improved the monkey’s copy somewhat, although it still fell short of the Bard’s standards. At this second stage of programming, a large number of indelicate words and expletives appeared, leading Bennett to suspect that one-syllable obscenities are among the most probable sequences of letters used in normal language. Swearing has a low information content! When Bennett then programmed the computer to take into account triplets of letters, in which the probability of one letter is affected by the two letters which come before it, half the words were correct English ones and the proportion of obscenities increased. At a fourth level of programming, where groups of four letters were considered, only 10 percent of the words produced were gibberish and one sentence, the fruit of an all-night computer run, bore a certain ghostly resemblance to Hamlet’s soliloquy:
TO DEA NOW NAT TO BE WILL AND THEM BE DOES
DOESORNS CALAWROUTOULD
We can see that as Bennett’s experiment started using more and more redundancies found in English, a certain structure seems to emerge. With the use of redundancies, even though it might appear that the monkeys were free to choose any key, the program made it such that certain events were more likely to happen than others. This is the basic premise of constraints. Constraints make certain things more likely to happen than others. This is different than a cause-and-effect phenomenon like a billiard ball hitting another billiard ball. Gatlin’s brilliance was to use this analogy with evolution. She pondered why some species were able to evolve to be more complex than others. She concluded that this has to do with the two types of redundancies, D1 and D2. She considered the transmission of genetic material to be similar to how a message is transmitted from the source to the receiver. She determined that some species were able to evolve differently because they were able to use the two redundancies in an optimal fashion.
If we come back to the analogy with the language, and if we were to only use D1 redundancy, then we would have a very high success rate of repeating certain letters again and again. Eventually, the strings we would generate would become monotonous, without any variety. It would be something like EEEAAEEEAAAEEEO. Novelty is introduced when we utilize the second type of redundancy, D2. Using D2 introduces a more likelihood of emergence since there are more connections present. As Campbell explains the two redundancies further:
Both kinds lower the entropy, but not in the same way, and the distinction is a critical one. The first kind of redundancy, which she calls D1, is the statistical rule that some letters likely to appear more often than the others, on the average, in a passage of text. D1 which is context-free, measures the extent to which a sequence of symbols generated by a message source departs from the completely random state where each symbol is just as likely to appear as any other symbol. The second kind of redundancy, D2, which is context-sensitive, measures the extent to which the individual symbols have departed from a state of perfect independence from one another, departed from a state in which context does not exist. These two types of redundancy apply as much to a sequence of chemical bases strung out along a molecule of DNA as to the letters and words of a language.
Campbell suggests that D2 is a richer version of redundancy because it permits greater variety, while at the same time controlling errors. Campbell also notes that Bennett had to utilize the D1 constraint as a constant, whereas he had to keep on increasing the D2 constraints to the limit of his equipment until he saw something roughly similar to sensible English. Using this analogy to evolution, Gatlin notes:
Let us assume that the first DNA molecules assembled in the primordial soup were random sequences, that is, D2 was zero, and possibly also D1. One of the primary requisites of a living system is that it reproduces itself accurately. If this reproduction is highly inaccurate, the system has not survived. Therefore, any device for increasing the fidelity of information processing would be extremely valuable in the emergence of living forms, particularly higher forms… Lower organisms first attempted to increase the fidelity of the genetic message by increasing redundancy primarily by increasing D1, the divergence from equiprobability of the symbols. This is a very unsuccessful and naive technique because as D1 increases, the potential message variety, the number of different words that can be formed per unit message length, declines. Gatlin determined that this was the reason why invertebrates remained “lower organisms”.
A much more sophisticated technique for increasing the accuracy of the genetic message without paying such a high price for it was first achieved by vertebrates. First, they fixed D1. This is a fundamental prerequisite to the formulation of any language, particularly more complex languages… The vertebrates were the first living organisms to achieve the stabilization of D1, thus laying the foundation for the formulation of a genetic language. Then they increased D2 at relatively constant D1. Hence, they increased the reliability of the genetic message-without loss of potential message variety. They achieved a reduction in error probability without paying too great a price for it… It is possible’ within limits to increase the fidelity of the genetic message without loss of potential message variety provided that the entropy variables change in just the right way, namely, by increasing D2 at relatively constant D1. This is what the vertebrates have done. This is why we are “higher” organisms.
Final Words:
I have always wondered about the exponential advancement of technology and how we as a species were able to achieve it. Gatlin’s ideas made me wonder if they are applicable to our species’ tremendous technological advancement. We started off with stone tools and now we are on the brink of visiting Mars. It is quite likely that we first came across a sharp stone and cut ourselves on it and then thought of using it for cutting things. From there, we realized that we could sharpen certain stones to get the same result. Gatlin puts forth that during the initial stages, it is extremely important that errors are kept to a minimum. We had to first get better at the stone tools before we could proceed to higher and more complex tools. The complexification happened when we were able to make connections – by increasing D2 redundancy. As Gatlin states – D2 endows the structure, The more tools and ideas we could connect, the faster and better we could invent new technologies. The exponentiality only came by when we were able to connect more things to each other.
I was introduced to Gatlin’s ideas through Campbell and Alicia Juarrero. As far as I could tell, Gatlin did not use the terms “context-free” or “context-sensitive”. They seem to have been used by Campbell. Juarrero refers to “context-free constraints” as “governing constraints” and “context-sensitive constraints” as “enabling constraints”. I will be writing about these in a future post. I will finish with a neat observation about the ever-present redundancies in English language from Claude Shannon, the father of Information Theory.:
The redundancy of ordinary English, not considering statistical structure over greater distances than about eight letters, is roughly 50%. This means that when we write English half of what we write is determined by the structure of the language and half is chosen freely.
In other words, if you follow basic rules of English language, you could make sense at least 50% of what you have written, as long as you use short words!
Please maintain social distance, wear masks and take vaccination, if able. Stay safe and always keep on learning… In case you missed it, my last post was More Notes on Constraints in Cybernetics:
This is available as part of a book offering that is free for community members of Cyb3rSynLabs. Please check here (https://www.cyb3rsynlabs.com/c/books/) for Second Order Cybernetics Essays for Silicon Valley. The e-book version is available here (https://www.cyb3rsyn.com/products/soc-book)
This is available as part of a book offering that is free for community members of Cyb3rSynLabs. Please check here (https://www.cyb3rsynlabs.com/c/books/) for Second Order Cybernetics Essays for Silicon Valley. The e-book version is available here (https://www.cyb3rsyn.com/products/soc-book)
In case you missed it, my last post was The Reality of Informationally Closed Entities:
In today’s post, I am looking at the idea of “informationally closed”. The idea of informational closure was first proposed by Ross Ashby. Ashby defined Cybernetics as a study of systems that are informationally tight. Ashby wanted cyberneticians to look at all the possibilities that a system can be in. Here the system refers to a selection of variables that the observer has chosen. Ashby noted that we should not look at what individual act a system produces ‘here and now’, but at all the possible behaviors it can produce. For example, he asked why does the ovum grows into a rabbit, and not a dog or a fish? Ashby noted that this is strictly related to information, and not energy:
Growth of some form there will be; cybernetics asks “why should the changes be to the rabbit-form, and not to a dog-form, a fish-form or even to a teratoma-form?” Cybernetics envisages a set of possibilities much wider than the actual, and then asks why the particular case should conform to its usual particular restriction. In this discussion, questions of energy play almost no part – the energy is simply taken for granted. Even whether the system is closed to energy or open is often irrelevant; what is important is the extent to which the system is subject to determining and controlling factors. So, no information or signal or determining factor may pass from part to part without its being recorded as a significant event. Cybernetics might, in fact, be defined as the study of systems that are open to energy by closed to information and control – systems that are information-tight.
Ashby’s main point regarding this is that the machine or the system under observation selects its actions from a set of possible actions, and this will remain the same until there is a significant event that causes it to alter the set of possible actions. The action of the system is entirely based on its structure, and not because an external agent is choosing that action for the system. The external agent is only triggering or perturbing the system, and the system in turn reacts. This idea of informational closure was further taken up by Humberto Maturana and Francisco Varela. The idea of “informationally closed” is a strong premise for constructivism – the idea that all knowledge is constructed rather than perceived through senses. They noted that as cognizant beings, we are informationally closed. We do not have information enter us externally. We are instead perturbed by the environment, and we react in ways that we are accustomed to. Jonathan D. Raskin expands on this further:
People are informationally closed systems only in touch with their own processes. What an organism knows is personal and private. In adhering to such a view, constructivism does not conceptualize knowledge in the traditional manner, as something moving from “outside” to “inside” a person. Instead, what is outside sets off, triggers, or disrupts a person’s internal processes, which then generate experiences that the person treats as reflective of what is outside. Sensory data and what we make of it are indirect reflections of a presumed outside world. This is why different organisms experience things quite differently. How Jack’s backyard smells to his dog is different from how it smells to him because he and his dog have qualitatively different olfactory systems. Of course, how Jack’s backyard smells to him may also differ from how it smells to Sara because not only is each of them biologically distinct but each has a unique history that informs the things to which they attend and attribute meaning. The world does not dictate what it “smells” like; it merely triggers biological and psychological processes within organisms, which then react to these triggers in their own ways. The kinds of experiences an organism has depend on its structure and history. Therefore, what is known is always a private and personal product of one’s own processes.
Raskin gives an example of a toaster or a washing machine to provide more clarity on the informational closure.
Maturana asserts that from the point of view of a biologist living systems are informationally closed–that is, things don’t get in and they don’t get out. From the outside, you can trigger a change, but you cannot directly instruct. Think of it as having a toaster and a washing machine. And, the toaster is going to toast no matter what you do. And, the washing machine is going to wash no matter what you do. And they both can be triggered by electricity. But the electricity doesn’t tell the toaster what to do. The toaster’s structure tells the toaster what to do. So similarly, we trigger organisms, but what they do has to do with their internal structure–including their nervous system–and the way it responds to various perturbations.
The idea of informational closure forces us to bring a new perspective to how we view the world. How are we able to know about reality? From a constructivism standpoint, we do not have a direct access to the external reality. What we can truly say is how we experience the world, not how the world really is. We do not construct a representation of the external world. This is not possible, if we are informationally closed. What we do is actually construct how we experience the world. As Raskin points out, the world is not a construction; only our experience of it is. Distinguishing experiential reality from external reality (even a hypothetical, impossible-to-prove-for-sure external reality) is important in maintaining a coherent constructivist stance.
All knowledge from this standpoint is personal, and cannot be passed on as a commodity. In constructivism, there is an idea called as the myth of instructive interaction. This means that we cannot be directly instructed. A teacher cannot teach a student with a direct and exact impact. All the teacher can do is to perturb the student so that the student can construct their personal knowledge based on their internal structure. Raskin notes – once people’s internal systems are triggered, they organize their experiential responses into something meaningful and coherent. That is to say, they actively construe. Events alone do not dictate what people know; constructive processes play a central role as people impose meaning and order on sensory data.
The more interactions we have with a phenomenon, the better we can experience the phenomenon, and it aids in our construction of the stable experiential reality of that phenomenon. Repetition is an important ingredient for this. Ernst von Glasersfeld notes:
Without repetition there would be no reason to claim that a given experiential item has some kind of permanence. Only if we consider an experience to be the second instance of the self-same item we have experienced before, does the notion of permanence arise.
From this point, I will try to look at some questions that might help to further our understanding of constructivism.
What is the point of constructivism if it means that we cannot have an accurate representation of the real world? The ultimate point about constructivism is not about an ontological stance, it is about viability. It is about being able to continue to survive. All organisms are informationally closed, and they continue to stay viable. The goal is to fit into the real world. Raskin explains – the purpose of this knowledge is not to replicate a presumed outside world but to help the organism survive. In Cybernetics, we say that we need to have a model of what we are trying to manage or control. This “model” does not have to be an exact representation of the “system” we are trying to control. We can treat it as a black box where we have no idea about the inner workings of the system. As long as we are able to come up with a set of possibilities and possible triggers for possible outcomes, we can manage the system. A true representation is not needed.
How would one account for a social realm if we are informationally closed? If each of us are informationally closed, and our knowledge are personal, how we do account for the social realm, where we all acknowledge a version of stable social reality. Raskin provides some clarity on this. He notes:
Von Glasersfeld held that people create a subjective internal environment that they populate with “repeatable objects.” These repeatable objects are experienced as “external and independent” by the person constructing internal representations of them. Certain repeatable objects–those we identify as sentient, primarily other people–are treated as if they have the same active meaning-making abilities that we attribute to ourselves. Consequently, we are able to experience an intersubjective reality whenever other people respond to us in ways that we interpret as indicating they experience things the same way we do. Once again, this alleviates concerns about constructivism being solipsistic because people do relationally coordinate with one another in confirming and maintaining their constructions.
For von Glasersfeld, it means that people construe one another as active meaning makers and consequently treat their personal understandings as communally shared when others’ behavior is interpreted as affirming those understandings. As I stated elsewhere, “when experiencing sociality or an intersubjective reality, we come to experience our constructions as socially shared to the extent that they appear to be (and, for all functional purposes, can be treated as if they are) also held by others”.
Each one of us construct an experiential reality of the external world. This external world includes other people in it. Our ongoing interaction with these people enhances and updates our own experiential world. We come to see the external world as a social construction. Our personal construction gets triggered in a social setting resulting in a social version of that construction. The more frequent and diverse interactions we get, the more viable this construction becomes. The other people are part of this experiential reality and thereby become cocreators of the social reality. In many regards, what we construct are not representations of the external world, but more a domain of constraints and possibilities. Making sense of the external world is a question about viability. If it does not affect viability, one may very well believe in a God or think that the world is flat. The moment, the viability is impacted, the constructions of the reality will have to adjusted/modified.
The image I have chosen for the post is an artwork by the Japanese Zen master, Nakahara Nantenbō (1839 – 1925). The artwork is a depiction of ensō (circle). The caption reads:
Born within the ensō (circle) of the world, the human heart must also become an ensō (circle).
Please maintain social distance and wear masks. Please take vaccination, if able. Stay safe and Always keep on learning…
In case you missed it, my last post was The Ghost in the System:
This post is also available as a podcast – https://anchor.fm/harish-jose/episodes/The-Reality-of-Informationally-Closed-Entities-e16ke0d
References:
Harish's Notebook - My notes... Lean, Cybernetics, Quality & Data Science. 