Will Artificial Intelligence Ever be More than Fancy Curve Fitting ?

One of my favourite books from my late 20’s was Roger Penrose’s The Emperor’s New Mind, where he argues that human consciousness is non-algorithmic and non-computable. This is a view that is not shared by many in the AI community, who believe that the brain is a computer and that consciousness is an emergent property of the brain’s computation.

Penrose and Stuart Hameroff further arguee that consciousness arises from quantum effects in the brain, and that the brain is a quantum computer. Hameroff has been working on a theory of consciousness called Orchestrated Objective Reduction (Orch-OR) for many years, and has been trying to find experimental evidence for it. Many are sceptical of this (quite possibly righly so), and it is still a fringe theory.

Machine Learning and curve fitting

Machine learning is often described as fancy curve fitting, and in many ways it is. The most common machine learning algorithms are based on finding the best parameters for a model that fits the data. This is done by minimising a loss function, which measures the difference between the model’s predictions and the actual data. Arguably, this very well may be what the brain does as well, and that consciousness is an emergent property of this computation. However, there have been no convincing arguments which convincingly confirm this is the case, or if there is something a bit more mysterious going on.

When we say that machine learning is fancy curve fitting, we mean that we are trying to find a function that best fits the data. This function can be linear, polynomial, or any other type of function. The goal is to find the function that best describes the data, so that we can make predictions on new data. You’ve seen the recent explosion of Large Language Models and all of that, but at the end of the day, they are just trying to predict the next word in a sentence, and that prediction is nothing more than a form of fancy curve fitting - the LLM is not consciously aware of what it is doing, nor does it have any understanding of the text it is generating (in some circles saying this is controversial in itself!), it is instead doing some incredibly complex, hyper-dimensional curve fitting.

Here’s an example of curve and surface fitting using a polynomial function, to help with the intuition.

Show the code

import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
from mpl_toolkits.mplot3d import Axes3D


# Define the quadratic function for 2D fitting
def quadratic(x, a, b, c):
    return a * x**2 + b * x + c


# Generate example data for 2D fitting
np.random.seed(0)
x_data_2d = np.linspace(-10, 10, 100)
y_data_2d = 3.5 * x_data_2d**2 - 2.2 * x_data_2d + 1.3
noise_2d = np.random.normal(0, 15, size=x_data_2d.size)  # Adding noise
y_data_2d_noisy = y_data_2d + noise_2d

# Fit the 2D quadratic model to the noisy data
params_2d, _ = curve_fit(quadratic, x_data_2d, y_data_2d_noisy)
a_2d, b_2d, c_2d = params_2d
y_fit_2d = quadratic(x_data_2d, a_2d, b_2d, c_2d)


# Define the quadratic surface function for 3D fitting
def quadratic_surface(xy, a, b, c, d, e, f):
    x, y = xy
    return a * x**2 + b * y**2 + c * x * y + d * x + e * y + f


# Generate example data for 3D fitting
x_data_3d = np.linspace(-10, 10, 20)
y_data_3d = np.linspace(-10, 10, 20)
x_data_3d, y_data_3d = np.meshgrid(x_data_3d, y_data_3d)
z_data_3d = (
    3.5 * x_data_3d**2
    - 2.2 * y_data_3d**2
    + 1.0 * x_data_3d * y_data_3d
    + 2.5 * x_data_3d
    - 3.5 * y_data_3d
    + 1.0
)
noise_3d = np.random.normal(0, 10, size=x_data_3d.shape)  # Adding noise
z_data_3d_noisy = z_data_3d + noise_3d

# Flatten the x_data and y_data arrays for curve_fit
x_data_flat = x_data_3d.flatten()
y_data_flat = y_data_3d.flatten()
z_data_flat = z_data_3d_noisy.flatten()

# Fit the 3D quadratic model to the noisy data
params_3d, _ = curve_fit(quadratic_surface, (x_data_flat, y_data_flat), z_data_flat)
a_3d, b_3d, c_3d, d_3d, e_3d, f_3d = params_3d
z_fit_3d = quadratic_surface((x_data_3d, y_data_3d), a_3d, b_3d, c_3d, d_3d, e_3d, f_3d)

# Create a 1x2 grid plot
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 6))

# Plot the 2D data and fit
ax1.scatter(x_data_2d, y_data_2d_noisy, label="Data", color="blue")
ax1.plot(x_data_2d, y_fit_2d, color="red", label="Fitted curve")
ax1.set_xlabel("x")
ax1.set_ylabel("y")
ax1.set_title("2D Quadratic Curve Fitting")
ax1.legend()

# Plot the 3D data and fit
ax2 = fig.add_subplot(122, projection="3d")
ax2.scatter(
    x_data_3d, y_data_3d, z_data_3d_noisy, label="Data with noise", color="blue"
)
ax2.plot_surface(x_data_3d, y_data_3d, z_fit_3d, color="red", alpha=0.5)
ax2.set_xlabel("x")
ax2.set_ylabel("y")
ax2.set_zlabel("z")
ax2.set_title("3D Quadratic Surface Fitting")

plt.show()

# Print the fitting parameters for 2D and 3D
print(f"Fitted parameters (2D): a = {a_2d:.2f}, b = {b_2d:.2f}, c = {c_2d:.2f}")
print(
    f"Fitted parameters (3D): a = {a_3d:.2f}, b = {b_3d:.2f}, c = {c_3d:.2f}, d = {d_3d:.2f}, e = {e_3d:.2f}, f = {f_3d:.2f}"
)

Fitted parameters (2D): a = 3.69, b = -2.42, c = -4.41
Fitted parameters (3D): a = 3.51, b = -2.17, c = 1.01, d = 2.70, e = -3.59, f = -0.96

Of course one might think, but AI can now do so much more! How can this all just be a bit of maths ? Well, it is a bit more than that, but at the end of the day, it is not much more than mathematics, and lots of engineering. The question is, will it ever be more than that, and will models approach a state of embodied intelligence ?

What is the difference between curve fitting and AI ?

The difference between curve fitting (or machine learning) and AI is that the latter is a more general term that encompasses a wide range of techniques, including machine learning, deep learning, and reinforcement learning. Machine learning is a subset of AI that focuses on building models that can learn from data, while deep learning is a subset of machine learning that uses neural networks to learn from data. Reinforcement learning is a subset of machine learning that focuses on building models that can learn from feedback.

The goal of AI is to build systems that can perform tasks that would normally require human intelligence, such as understanding natural language, recognising objects in images, and playing games. These tasks are often very complex and require a lot of data to train the models. However, at the end of the day, these models are pretty much doing the same “stuff”, and much of the amazement we all feel comes from pure clever engineering and the sheer amount of data that is available to train these models.

The “controversial” recent paper

What is really interesting and which has until now been the source of debunking of Penrose’s and Hameroff’s views, is that there has been no strong evidence of quantum effects in the brain. However, a recent paper has pretty much now demonstrated that indeed there are observable quantum effects in brain cells.

About Quantum Effects

Remember that quantum computers need to be kept at very low temperatures to work, and that the brain is at body temperature. This is one of the reasons why the view that computational quantum effects in the brain are not possible has been so prevalent.

Without pretending to understand the paper to any degree of depth, here’s what I believe it concludes.

The article titled “Ultraviolet Superradiance from Mega-Networks of Tryptophan in Biological Architectures,” published in The Journal of Physical Chemistry B, explores a significant discovery in the field of quantum biology. Researchers found that tryptophan molecules, when organized in symmetrical networks, exhibit a collective behavior known as “superradiance.” This quantum effect allows the tryptophan networks to fluoresce more strongly and rapidly compared to individual molecules.

Two major implications arise from this discovery:

Neuroprotection: The study suggests that these large tryptophan networks, found in neurons, could protect against degenerative diseases such as Alzheimer’s. They can absorb harmful ultraviolet light and re-emit it at lower, safer energy levels, potentially mitigating oxidative stress linked to such diseases.
Signal Transmission: These networks might act as quantum fiber optics within the brain, enabling extremely fast signal transmission. This could revolutionize our understanding of neuronal communication, suggesting that quantum processes might play a role in how the brain processes information much faster than traditional chemical processes allow.

The research bridges quantum computing and biological systems, indicating that quantum effects can persist in the noisy, warm environment of biological cells, which has implications for both neuroscience and quantum technology development.

What this means for AI

Well, ultimately it might not mean much. However, it does give considerable weight to the idea that perhaps there’s more to true consciousness than just what can be achieved using classic computing. It might be that the brain is indeed a quantum computer, and that consciousness arises from quantum effects in the brain. Also there’s the embodiment argument, that high levels of intelligence emerge from the interaction of a body with the environment, which is a stage that current technology is still far from achieving.

All of this would mean that existing models will ultimately do nothing else but to roughly simulate intelligence (not that this actually makes any difference to their potential utility and impact). This allied with limits in available data, scalability of computation, as well as cost, points at an evolution curve that might not be as steep as some might think. Ultimately, the question is whether the current trajectory of AI will lead to trully ground-breaking changes in the way we live, work and use technology, or if it will be just a bit more of the same but with a lot more gimmicky capabilities which “simulate” human capabilities.

Certainly in the near future, progress is likely to be more related to clever engineering, and application of what we already know, rather than any fundamental breakthroughs which will bring a step change in the capabilities of AI. This is not to say that there won’t be any breakthroughs, but that they are likely to be incremental (and slow), rather than revolutionary and fast.

It could just be we are seing the early signs pointing to a conclusion that we already are in the curve of diminishing returns.

Reuse

This work is licensed under CC BY (View License)