Throughout the last 4 years I have been fortunate enough to have had the company of friends who are not only smart people but have an intelligent and curious outlook towards the world. I think that it is relatively easy to be good at something, anything if only you start early enough and work relentlessly towards it. Which is not to say that talent is something which I don't respect. I do, but what I respect more than talent is if it adds a perpective to how a person sees things. There are a lot of really really talented people in the world, much much better than me, and there is something to be said about that, but they only have to open their mouth for you to realize that the capacity for unification of concepts which can potentially come from the pursuit of passion has somehow missed them. As Feynman said, it's similar to the difference between knowing the name of something and knowing something. I have been fortunate to have friends who know things or at least have a healthy drive to know them. Anyway, during one insightful conversation with Rathina today, we started discussing about a mutual interest, chess. The starting point was again a Feynman observation where he is making analogies between science and chess. Specifically he says that finding the laws of nature is like figuring out the rules of the game of chess by looking every now and then at the snapshots of a game. These snapshots are the only information one is allowed to have and the challenge for human intelligence is to find order from this seeming chaos.
Our discussion veered off in the specific direction of trying to see if it's possible to figure out the rules of the game just by looking at it, and if it is possible, how many example moves would one require to completely figure out every rule? Now it's a complicated problem to even pose formally because if it's a human intelligence which is trying to figure out the rules then the answer is obviously indeterminable. It's because while I might take thousands of examples to figure things out, Bobby Fisher might do it in a few games. So we tried to pose it in a computational framework. How many examples would an algorithm need to figure out the rules? How do we define 'figuring out all the rules?' There are only a finite, albeit stupendously large, number of possible chess games. So we defined figuring out the rules as determining all the possible legal moves which would be sufficient to generate all possible chess games. Now it is not an impossible problem, at least hypothetically, to calculate the minimum number of example moves that one would need in order for an algorithm to figure out all the legal moves for each piece which in turn would generate all possible games. That's not the problem though. The problem is the following question, 'As far as the standards of human intelligence is concerned, does determining all possible legal moves equal determining the rules of chess?' For example, it is easy for an algorithm to make a list of all possible positions a bishop can go to but it's a stretch to say that it is equal to the statement 'a bishop always moves on a diagonal.' It is possible to conceive of an algorithm which is built such that it can distill, from all the data it has, seemingly intelligent statements like 'a pawn always moves one or two squares in a straight line unless it captures another piece in which case it moves diagonally' or 'the game is drawn when a set of moves is repeated three times.' It is possible to make an algorithm which can put in words or figure out, from a sufficiently large number of examples, most of the 'intelligent' statements about the rules of chess a human can make. But what about the rules the examples for which it never encounters? One example of such a rule is that a black piece never captures a black piece. Another is that a piece never jumps out of the board. The dilemma here is that while a finite number of examples demonstrate every possible legal move, not even a single example shows an illegal move! When you add to that the fact that an infinite number of possible illegal rules exist, it seems hopeless how an algorithm can ever figure out all that is there to know. It can definitely generate all possible legal games, which was our original intent, but while the human understanding of chess includes the knowledge that a piece never captures a piece of the same color or that the queen is not allowed to jump out of the board or that you cannot say 'abracadabra' and claim victory, how an algorithm would do it is beyond my current grasp.
But I understand that I am probably oversimplifying. The most obvious simplification is that I am talking about a game which I already know the rules for. On the human side I am probably subconsciously delving upon my prior knowledge of the game - something which I cannot do for an algorithm because I don't know how subconscious prior knowledge can be represented for a computer. This makes it intuitive for me to envisage what 'questions' about the rules a human would find worthy of asking. It helps because there are infinite questions one might ask in order to figure out things. But one has to ask the smart questions. And it is not fair to the algorithm because I, speaking for a human, already know which are the worthy questions as far as chess is concerned.
Asking the right questions requires a lot of creativity though. The chess problem is similar to the following physical problem: imagine you had information about every apple which ever fell to the ground. Now you could ask an infinite number of questions which can be verified or refuted by all the apples, but how does one go about figuring out gravity from them? This, I think, has a deep philosophical implication. The implication is that our theories are not empirical. They can be directly derived from experimentation only to the extent that gravity is a natural outcome of falling apples, which is not much. The more important observation is that their power is in their creative origins and explanatory powers which do not depend upon experiments all that much. It's satisfying to see how the little discussion indicates that the inductivist viewpoint is quite shallow.