The Dot- Connect Theory

[jananee]

The Dot- Connect Theory by Chitravina Ravikiran:

http://ravikiranmusic.blogspot.com/2011 ... heory.html

[vasanthakokilam]

Thanks jananee for the link. Fascinating insights and theory. I always had this notion that wisdom and creativity is about connecting the dots and the dots represents individual pieces of knowledge ( like Sri. Rajaji says in the introduction to MSS's Bajagovindam ).. That is, knowledge are the dots and wisdom is the meaningful connections among the dots.. Sort of similar to Ravikiran's theory which he develops from first principles. I will post something in his blog if I have anything useful to add.

[mahavishnu]

Jananee, thanks for posting this.

VK: It is definitely interesting. But with all due respect to Ravikiran, this sounds like pseudo-science to me, which is not dangerous in itself; but could lead to serious misinterpretations, some of which are already emerging on his blog post.

I found the development of issues, especially the jargon about divisible/indivisible brain etc to be weak. The hypothesis he puts forth is neither new nor original. The idea that subsymbolic elements combine in meaningful ways, has been expressed by everyone from Patanjali to David Hume. It is the basis of most of what we do in neural networks, especially in studies of associative memory. That knowledge can be built out of the interactions of "particulate" matter ( the dots in his "dot-connect" theory) is now a well-established idea in modern science.

I think Ravikiran would do well to focus on how musical creativity emerges from connecting the dots in his domain (swaras combining to create amazing flavors/patterns/ideas). Very little is known about this and his contributions to this area would be taken very seriously. A possible corollary of this could be things related to subjective experiences when listening to these patterns emerge, and how that involves some dot-connecting on the part of the listener.

Sorry to be so curmudgeonly humbug about this. The world does not need another self-appointed Castaneda.

[veeyens3]

Appears far fetched and controversial yet with for all we know with frog in well syndrome blinkers clouding our judgement, there may be some truth in it. and deserve benefit of doubt. Before Copernicus, people believed geo centric concept of planetary motions. , Otherwise how can we explain the authors own feat in identifying ragas as a toddler during a demonstration in Krishna Gana Sabha many decades ago, at the same time proving that he was an ordinary normal child by demanding Seedai a sweet made during Krishna Jayanthi celebrations after every correct answer. .Even the one and only Ramanujam might not have explained how he formulated such complicated theoroms which are baffling mathematicians even today. And how generations back our forefathers preserved many terabytes (I do not know the nomenclature of higher values) like vedas in memory and retrievd them at will with he help of appropriate nemonics called ganam jata etc.May Sri Rama bless Sri Ravikiran

[vasanthakokilam]

Mahavishnu: This gives me an opportunity to voice two topics that had been on my mind for a while. I would like to get your input, even if is means stepping outside of the strict scientific bounds in which you operate.

Yes, RK's hypothesis seems like a lakshaya top-down reverse engineered model than a lakshaNa model which is the play ground of scientists like yourself. Interestingly, as far as theorizing about the brain is concerned, there is more scope for dabblers to participate and improvise!! I do understand your point. Ravikiran could have positioned it better and remove any scientific notions about it.

Treating it at that level and even higher, and not necessarily specific to his article, it is commonly felt by pragmatic creative thinkers ( those who make things that are useful for the masses ) that creativity in terms of new ideas is about connecting the dots. It is not creativity as in artistic creativity but in the sense of seeing things that others do not see. In that sense, connecting the dots and creativity are treated as cause and effect ( though not the only cause ).

That always felt inadequate for me since the unfortunate use of the word 'dot' implies it is a model of the data. Two things are missing. a) The behavior part which is the animating force behind making those connections, a state machine b) Pragmatics. The dots have contextual meaning, there is a history for each dot, distilling some essential meaning out of it and applying it in a different context requires a much more complex reasoning ability and hence a more powerful model. This is before even one can attempt to connect the dots for something new which has its own pragmatics.

I know these are all being looked at from many dimensions scientifically by you and others, and you can write something about these things if applicable.

Second thing.. One reoccurring thought for me is this. Humans have created a lot of things outside of 'us' and have figured out a lot of things about the world out there. Let us call these 'brain output'. It can be manufacturing/assembly line, chemistry, computers and linear and nonlinear algorithms of incredible complexity, photovolatic cells, information transducers ( Microphone, speakers ) etc. The startling thing is, nature has used 'similar technologies' in manufacturing us, whether how it codes for meta data in the DNA, or how the senses work ( transduction ) or how the internal chemistry works for digestion, or in keeping the body parameters within bounds using feedback systems etc. I am wondering if our brain, again a product of similar strategies of nature, works the way to create macro things because of the very same nature's strategy/intelligence at the micro level. In many cases, nature's methods are much inefficient, complicated and round-about ( for example, look at the various stages of transducers for hearing ). it also reuses a lot of the same strategies, even though a new thing would probably be more efficient. But what matters is that it works. Efficiency and optimization for their own sake is not the "lakshayam" of evolution, but survival and propagation is. But the "intelligence/information content' behind the strategies are very reminiscent of how we think (i.e "how the brain operates at the macro outward looking level").

This is probably not a new thought, probably others have talked and written about it as well. And it is probably not a big deal either, since this 'coincidence' is probably only natural since both the nature and brain have to work with the same physics and chemistry of nature. But information and intelligence is not like physics and chemistry, they do not have to be same at the micro and macro level. But what if it is? Assuming there is some truth to this observation, one practical use for figuring out brain function at lower levels is to reverse what we are trying to do in neural network algorithms. Look around us to see what strategies brain has created for similar problems in the macro world and form a hypothesis as to how our internals may be operating and use that as a starting point to conduct experiments to prove/disprove it. I do realize that any hypothesis about the brain is a reflection of the brain's way of thinking, that is a problem with any kind of self-theories. That is why it can seek help by looking outward first to see what it created and use that as a way to understand how its internal parts are put together and operate, on the wild hypothesis/assumption that there is a relationship between the two.

Bringing this back to this topic, Ravikiran's thinking can be seen as a hypothesis born out of this kind of thinking, even if he did not explicitly state so.

[mahavishnu]

VK: If only Ravikiran had said what you have articulated so eloquently

Point taken about the fact that there is an "agency" missing in all this discourse. I use the term in the context of a whole that emerges when one pieces the dots or "data" together. There is a whole group of cognitive scientists that have been busily engaged in exactly that issue for about 40 years. Incidentally, today we sadly mark the death of one of the pioneers in this field, a guy named David Rumelhart (see NYT obit here: http://www.nytimes.com/2011/03/19/healt ... lhart.html).

David showed evidence for how these "dots" have ways of combining on the basis of physical principles, using a class of models that he developed called "connectionist" models. Such models work incredibly well to explain data from human perception, memory, language use and reasoning. Even music production (at a very basic level). For example, just as in thermodynamics, a disordered system of "dots" will try to achieve stability or order through optimal management of entropy or information.

both the nature and brain have to work with the same physics and chemistry of nature

Excellent point. The entire basis of the sciences of the mind is based on this conjecture.

any hypothesis about the brain is a reflection of the brain's way of thinking

Absolutely. Phenomenologists often say that you must have a theory about what "thought" is before we can talk about thought. We humans hypothesized that we were breathing something that sustained life long before Priestly's discovery of oxygen. Most geniuses when asked about their thought process (to address Veeyens' point) have no idea how they came up with creative solutions from their thoughts, or the building block "dots". Ramanujan attributed much of his genius to the Goddess Namagiri whispering equations into his head. Even us lesser mortals have great deal of trouble expressing things from procedural memory (like how to ride a bicycle) in declarative terms (as equations of motion of a two spinning discs kept in equilibrium through conservation of angular momentum). Lance Armstrong couldn't give us any better insight into this, right? Implicit knowledge is very hard to access through explicit knowledge.

So, where does this leave us? 1) Creativity, if it were to emerge from just these dots, will have to be explained by some basic principle like symmetry breaking or equivalent. 2) Yes, we arrive at the strategies that the brain uses to solve these problems through careful experiments and hypotheses about how the "wetware" supports these computations and 3) Ravikiran still needs to do a lot of reading.

[cmlover]

Beautiful VK/Ramesh
We need more such discussions and elaborations to generate new ideas for practical research. Some day we may be able to connect the micro model with the macro model of reality. Then creativity will be within the reach of all and sundry rather than in the blessed domain of the so-called 'gifted'.

Continue your fasicnating speculations ...

[vasanthakokilam]

Thanks CML..

Mahavishnu: Thanks for the follow up. I learned quite a bit there. I will have to check out Sri. David Rumelhart's work if it is layman accessible.

In the comments section of Ravikiran's blog, a Neuroscientist, Dr. Hari Subramanian, has summarized what we know about the brain. It is a succinct summary that took only 4 paragraphs, that too in gopucha yati, of what all has been learned by the world at large in the last 40 years!! It is not a lot compared to other disciplines but at the same time it is quite incredible how much we actually know about the brain. ( You can check it out and elaborate here if you feel the need )

What you state here is highly relatable to me at a gut level: "disordered system of "dots" will try to achieve stability or order through optimal management of entropy or information.". Dr. Hari also talks about the self-similarity of neurons, the neuronal pathways and higher level configurations are all based on 'survival'. I need to read up and understand that aspect.

Of course, I am dragging in things in bits and pieces that are aesthetically appealing to me, but the direction in which this leads me is with respect to the macro vs micro: I should not be thinking about individual brain's discoveries and inventions out there as a model for how the brain works inside since there is no single agency inside the brain. May be there is some pay dirt in the following approach: Look at what the 'society' ( million outward looking brain output ) has created through the usual complex system maneuvers ( of positive and negative feedback ) and see if a similar model is going on inside. That is, macro-micro correspondence is something like this:

Individual neurons and small clusters --> Individual people and the full brain
full brain................................ --> society
Emergent properties inside the brain --> Emergent properties in the society ("fashion", "economy", "politics")

There is always this lingering debate that the so called 'invisble hand' in the laissez-faire capitalistic model is nothing more than the entropy management that you refer to regarding creating local order out of an otherwise chaotic behavior of individuals operating on incomplete info. And a similar invisible hand in guiding the species evolution at the macro level. Usually, this is brought up by anti-capitalists to brand capitalism as the survival of the fittest and how inhuman it is while acknowledging implicitly that evolution is quite inhuman Here we have an example equivalence of emergent phenomenon and we understand to some extent the activities of the players in the macro system which causes these emergent macro effects. If a similar correspondence can be made for emergent phenomenon in the brain and hypothesize about what kind of behavior the players ( neuronic clusters ) should exhibit to cause such an emergent phenomenon, there is a possibility of pay dirt!!

My own reason for thinking about this is a frustration that is born out of impatience. As an outsider, it is frustrating to see the slow progress while the intellectual side of me fully relates to the enormous difficulty. But the nagging feeling is, there may not be enough scientists who come at this from multiple angles. One example of the frustration is in the slow pace in understanding everything there is to know about stem cells, given the enormity of the pay dirt. Now, it is an amazing thing, quite magical and fundamental to how we are all put together, as we all know. The scientists seem to be looking at the molecular and cell level to understand things. But there is an alternate approach. Whenever, the same darn thing becomes different things depending on its neighbors and its orientation in three dimension, the field to look for advice is Topology. Mathematicians have made incredible progress in Topology for decades. It will be great to have mathematicians attack this stem cell issue from a purely mathematical point of view. Talk to a structural engineer, they will tell you about simulation models they have built over the decades since a structure is made up of self-similar smaller structures in different orientations ( think of struts and beams to build large buildings, bridges etc. ). Imagine how sweet it will be that how cells become specific organ tissues are fully explained by topological manipulations. The underlying chemistry matters but what we are after is the higher level understanding of this magic.

Another area is to treat life ( organic functions ) as a software executing on an organic computer. The parallels to how we have designed out computer system to how we are all put together are well known. Now the thing is, the structure and behavior of a program like this browser can be understood by looking at the machine language instructions but it will be very complex. But it is much easier to understand the same if given the higher level source code in which it is written. The famous tongue-in-cheek quote from the person who wrote the first Fortran compiler when asked what motivated him was 'I am inherently lazy'!! Using this analogy, what is missing is that compiler that takes a life script and translates that to the actions of proteins and cellular behavior. The problem here is complex indeed, you do not get to decide the hardware, you do not get to decide the machine language and you do not get to write the higher level program...they are all there. Our task is to reverse-engineer a possible compiler or interpreter to provide mapping of the high level concepts to the low level concepts. The benefit is, once that is done, we do not have to worry about the lower level ones, we can operate at the virtual biological machine, even if completely made up, defined by such a compiler/interpreter, to understand bigger and better things about life while a separate set of teams work on optimizing the virtual machine.. This is how software production has scaled to match the demands, we would be in a standstill if we are still programming in machine-language. ( it is quite possible that such a biological virtual machine is being worked on to run life as we know it )

I am looking for that break through in biological computing, equivalent of the first FORTRAN compiler from IBM which is a watershed event in computing.

In a way this is another form connecting the dots. Stem cells and Topology are distant dots but connecting them for some useful stuff requires a multi-disciplinary team of experts. And I just took Topology as just one discipline out of thin air. Imagine the M x N matrix of such things and someone has to unearth those connections which has the potential to cause exponential progress.

[mahavishnu]

CML, thanks for the kind words. Today, we are a lot closer to linking the micro-macro worlds in our understanding of the sciences of complex systems.

Individual neurons and small clusters Individual people and the full brain
full brain................................ society
Emergent properties inside the brain Emergent properties in the society ("fashion", "economy", "politics")

VK. this is a very accurate analogy. There are two important phenomena to consider here. Reductionism and emergence. Reductionism is what we have been doing in the sciences from pre-Socratic days. The principle here is that Nature is reducible to a more exact level of explanation at a lower level. For example, all biological interactions can be reduced to their chemical components. So a neuron can be characterised by its ability to channel Na+/K+ ions and the lock and key behind the ion transport is controlled by chemicals such as neurotransmitters (glutamate, norepinephrine, dopamine etc). These neurotransmitters are able to open up synapses and change the post-synaptic potential of other neurons, thus affecting brain function albeit at a very local level. Now this interaction is reducible to its basic physical properties in the form of charge and induction, potentiation and de-potentiation. Now physicists can look at the mechanics of neural tranmission as a purely physical process. Now the reductionism has enabled us to study this phenomemon at a more micro level with greater explanatory power of synaptic information transmission. Essentially by "reducing" the neural interactions to their component physics we have gained a better "micro" understanding, but we are not anywhere near explaining how the input-output behavior of neurons is related to brain function. We could go even deeper and look at the quantum mechanical processes in the atomic structure of the elements that make up the neuron, but we will not be any closer to explaining brain function. Thus "reductionism" does not give us a more privileged view of nature. Scientists have made this mistake for several centuries. We have always believed that there will be a micro level explanation for all macro phenomena.

Now, enter emergentism. All of a sudden we find that there are "privileged" scales for observing nature. A population or collection of neurons behaves differently from an individual neuron. So we look for scientific explanations at this scale. Similarly, society behaves differently from an individual (although the physical principles of survival for an individual/society are equivalent). Thus there are emergent properties that seem to be scale based. The beauty in nature comes from the fact that there is something called scale-invariance. No matter which scale you look at, the laws of organization seem to be the same. What an amazing design concept, if you were the "creator" and decided to engineer the universe.

To use a very simple example from fractally based geometric systems, the coast of Britain looks the same, similar jaggedy edges, whether you look at the entire island or just a slice off the coast of Cornwall. Now why do we have self-similarity across these scales? Although scientists are quite divided on this, the consensus is that the similar physical principles are at work in organizing the parts with the whole. Thus the patterns of activity of the entire brain should (in an ideal world) be self-similar to the behavior of a single neuron.

Here I disagree with Hari Subramanian's Neural Darwinistic position. The principles we are looking for are more generic than just mechanisms of survival, but they should be universal principles like conservation or energy or the second law of thermodynamics or its Boltzmannian equivalent. We have also gained much more of an understanding by through the physical principles in the study of biological systems.

In the sciences of complexity, of which I believe cognitive science is one, we ask the following question. How does the interaction of one billion "dumb" neurons in the head produce intelligent thoughts, desires, beliefs and actions? The reductionist in us has always tried to approach this problem by looking at each neuron and each brain area and see how they compute together and in aggregate form. However, the "emergentists" have been quick to warn that breaking the problem down is not going to give us any further insight into understanding the "mind", which is an emergent property of the 100 billion neurons that work in concert. The analogy of machine language to high-level interactions of a sentient computing being is very apt here (even understanding fully well that all the high-level stuff will have to implemented in assembly). So far, I am agreeing with you and adding my perspective on the way you have expressed your words.

However, we are much closer to your desired understanding of biological computing. Neural networks that are biologically-inspired connectionist machines, are behind every piece of technology we own. Cruise control (moving set-point control systems), google search (categorization of complex information and searching through solution spaces) to trivial things like automatic movement tracking in face-time for the mac, all use "brain-like" algorithms. But these are all implemented on silicon based computing systems. Can we actually build organic, carbon based computers that will generate complexity as they go along at both micro and macro levels? The answer is not that far away. Work on transplanting satellite cells (type of stem cells) has shown that associative memories lost due to the development of degenerative amyloid plaques in the brain can be regained with "new" neurons that were not used in encoding the information that has been lost. This is suggesting the "micro" encoders are less important that the "macro" state that these neurons are part of.

In all of this, as you say, a very trans-disciplinary perspective is required. There is no more just "biology" or "neuroscience" or "computer science" or "physics".
Fortunately, our task is not to reverse engineer life itself. We would be competing with some very powerful characters in the universe if we were tasked with that

The most fascinating summer of my life was spent at the Santa Fe Institute in New Mexico (http://www.santafe.edu/), just around the bend from VK Viswanathan's Los Alamos labs. The mission of this institute is to put topologists, economists, physicists and computer scientists together to define the new sciences of complexity. Please browse through their website; you will see a lot of things to be excited about. They even have people studying the physics of "frustrated" systems (systems seeking to create order through dissipation of energy)

[vasanthakokilam]

Excellent Mahavishnu. Thanks for the detailed write up. I relate very much to what you wrote, especially the part about ""micro" encoders are less important that the "macro" state that these neurons are part of" which is very much the crux of the matter and show the utter inadequacy of the reductionists' approach. A rough analogy is how birds fly together in formations. You can take a few birds out and introduce a few new birds in, they will fit right into formation, becoming part of the macro state which is the overall shape of their formation. Such emergent phenomenon requires very little local intelligence on the part of each bird. It is a little back and forth adjustment among the few neighbors of each bird and the entire system adjusts to a shape. Even after some system wide disturbances which scatter them a bit, they can all re-form in to a new shape.

I tell my fellow computer science types that we have a much bigger role to play in understanding life than we realize, as the problems become more and more information-theoretic ones. In addition to, in fact more than, using what we know about how our system solves problems to solve external problems, the reverse, as you wrote, is also where computer science theorists, complex system experts and mathematicians can contribute in a big way.

I have heard of the Santa Fe Institute. That is indeed a privilege to spend a summer there. I can only imagine how fascinating it would have been. I will check out their web site you referenced. Thank you.

[VRV]

Ramesh (Mahavishnu)

Which summer did you spend in New Mexico at the Santa Fe Institute. I spent quite some exciting time there working with Dr. Peter Salamon.


Vinod

[mahavishnu]

Vinod, Small world. I mean that not in the context of "small-world" networks

I was there in the summer of 1998. Initially for the complex systems summer program followed by some time with the groups of Doyne Farmer, Murray Gel Mann, Stu Kauffman and others. All those fascinating late night conversations, amazing treks with Stu's group and the incredible southwestern yellow sunset from the top of the opera house. Truly exceptional location.

Hope you are enjoying your new sciences of complexity! Cleveland this spring?

[VRV]

Vinod, Small world. I mean that not in the context of "small-world" networks

Hope you are enjoying your new sciences of complexity! Cleveland this spring?

Ramesh,

Hoping to be in Cleveland the second weekend, provided my class schedules pan out. Unfortunately Spring Break for our campus is this week and Cleveland is in April.

Hope to see you

Vinod