From data based models to program based models.

 The most successful approach to modern AI has been the data-oriented version. Using statistical learning methods, we have created systems that can recognize, and generate information. While there are numerous benefits to these paradigms as can be seen obviously many new techniques that have become commonplace in daily life, from more accurate recommendation systems to text and image generation systems that make it easier to retrieve human knowledge in a compressed form, these methods are limited in the sense that they depend on human-generated data to be "intelligent".

The main issue arises from our understanding of what intelligence requires. Our educational systems have conditioned us to think that intelligence requires the absorption and retrieval of information, and so we have created systems that do these actions to a greater or lesser degree. 

If this is what we consider intelligence, then the  LLMs and image generation systems of today are the totality of what that kind of intelligence is. But as we can now see, this is not the approach that will lead to the creation of the human mind on a non-biological substrate, at least the part of it that could be to us a general intelligence just like a calculator was able to take our ability to calculate and perfect it.

The big problem here is not the execution of ideas about intelligence, but the definition of the term itself. But another approach is this, what if we forget about viewing the achievement of intelligence in some philosophical sense and just focus more on functionality. Meaning, that what if we focus on just creating tools, rather than focusing on fantasies about creating intelligent systems.

Intelligent systems are already here, they have been here since the creation of the Zuse, ENIAC, etc. computers. The power of intelligence is computation, our ability to take a part of our knowledge of the world and make systems that replicate that executive knowledge. 

Computation itself is the transformation of information using rules, everything can be represented in such a manner using rules. We learn to calculate or compute with numbers, by learning rules that transform some input to some output.

This power to store a computational procedure outside our bodies and mechanize, whether that be through, paper, an abacus, or an electronic computer is the encoding of intelligence outside the body, that is, intelligence in a none biological substrate.

So why are data-based methods limited? With data-based methods, we can solve problems for which we cannot develop rules using our own intelligence alone. We build a system that can generate rules through observation of input examples. 

Now these methods of searching for rules when we cannot invent one out of our own ingenuity is what we have termed artificial intelligence, and rightly so. However, the current models are not able to take advantage of the full power of computation that we have developed so far. 

We are limited to storing information in large arrays and using simple arithmetical computation to store generalized structures and retrieve them via the process of inference. 

What is being proposed here is that we can develop better systems for searching for rules that enable us to encode different computation structures by searching directly with computation structures as described in a particular computational language, rather than searching through raw numerical encodings as we currently do in recent statistical methods. 

What gave birth to the dominant statistical learning methods of today was the inability to keep inventing algorithms or rules to encode certain inputs to outputs. If we give a particular computation system the image of a cat and try to devise rules manually by traditional programming to determine if the image or any other similar images can be recognized as cats it would be difficult. In fact, this was tried in the early days before deep learning techniques became mainstream. the results from inventing such algorithms were poor despite the immense human ingenuity it took to invent such algorithms.

So with deep learning (aka a statistical machine learning method), we can learn parameters to a universal function that learns a general rule for any number pattern we provide, this number pattern in this case is the vector of the pixel values that make up the image.

In the future rather than searching through numerical parameter spaces to find the most general description for a class of input, we will be searching program space for programs that do the same, in a much more efficient way than current methods allow us to do.

This brings us to the main point of this post, we will need a programming language that enables us to search efficiently for program spaces to find appropriate programs that solve problems, the same way we are not searching through parameter space for generalized representation. 

After searching through for such a language, I initially chose LISP or at least the Scheme implementation of LISP, this is because early work in genetic programming by John Koza was done in this language and thus it made it easy to start using it right away because there was ready-made material on it as pertains to genetic programming. 

Recently I have decided to use Wolfram language because it cleans up a lot of difficult things with LISP-like languages and has a prebuilt infrastructure that enables faster innovation in this field of evolving solutions to problems as programs. 

In conclusion, we see that while deep learning methods have brought us this far, the future will require much more robust systems, and models based on programs rather than raw numerical parameter search.