Karl Popper should have been a software engineer

Theories should be falsifiable. That was Karl Popper’s (1902 - 1994) main point, and gave him worldwide acknowledgement as an influential philosopher. Besides philosophizing, he was a renowned academic, writer, and believer of the open society. All of that is great, but I think he would have made a teriffic software engineer. There are a few reasons why.

First of all, what do we mean when we say a theory should be falsibiable?

Does God exist

There are theories that are impossible to prove wrong. How would you refute the idea that God exists? Would you go look in every corner of the universe, looking for God? How would you even know what to look for? He or she could be moving around at the speed of light, or be all around us all the time in a microscopic gas form.

One approach I could think of is to try and refute all properties that people generally attribute to God. God is all knowing, but can he really be given Heisenberg’s uncertainty principle and quantum physics? But that only proves contradictions in God’s properties, and we could either come up with some other definition of God or even just accept that God is contradicting.

The point is that God as an entity is at best a loosely defined concept, which is unfeasible to base scientific theories on.

And that means “God exists” as a theory is not falsifiable. There is no conceivable experiment that allows us to prove that God doesn’t exist.

All swans are white

On the other hand “All swans are white” is easy to falsify - just find a swan that is non-white. It may be that there are no non-white swans in our universe, but at least we can think of a scenario where this theory is proven false, namely, we have found a non-white swan.

Back to Popper. If some theory is not falsifiable, it is to be considered “pseudo-science”. Therefore, scientists should strive to maximize the falsifiability of their theories. The point is not to find confirmations of a theory, but to actively trying to refute it.

Dependencies

The problem with this line of thinking is deciding what to do when a theory has been falsified.

Typically, a theory is part of a larger framework, with all kinds of assumptions and other theories. It could be that the error lies in one of these external dependencies. It is impossible to isolate a theory in such a way that you can be certain the theory itself is wrong, not any of its dependencies. This is also known as the Duhem-Quine thesis.

In practice, this means that it is impossible to falsify any theory.

Software Functions

In the digital world software engineers also come up with theories. They take in business requirements, and produce code that they think accurately fulfills these requirements. The code is grouped in functions, that each describe a part of the problem to be solved. Functions make code repeatable, easier to read, and most of all, easier to falsify.

Take for example the gradient descent algorithm, which is useful in machine learning. A dependency of the algorithm is the cost function, which can be coded as:

def cost_function(y_true, y_pred):
    """
    Compute the Cost with Mean Squared Error

    :param y_true: Array of actual values
    :param y_pred: Array of predicted values
    :return: Cost
    """
    n = len(y_true)
    total_error = 0

    for i in range(n):
        error = y_true[i] - y_pred[i]
        total_error += error ** 2

    return total_error / n

Consider this function to be a theory. Is it falsifiable? Yes, we can try any values (parameters) we want, and verify the cost value (return) that comes out of it.

The Unit Test for this function is quite trivial, and allows us to try and falsify it:

import unittest

class TestCostFunction(unittest.TestCase):

    def test_cost_function(self):
        y_true = [3, -0.5, 2, 7]
        y_pred = [2.5, 0.0, 2, 8]
        result = cost_function(y_true, y_pred)
        self.assertAlmostEqual(result, 0.375)

    def test_cost_function_with_zeros(self):
        y_true = [0, 0, 0, 0]
        y_pred = [0, 0, 0, 0]
        result = cost_function(y_true, y_pred)
        self.assertEqual(result, 0)

if __name__ == "__main__":
    unittest.main()

In this case, we have proven that the function works for these values. It is important to note the we have not systematically proven the algorithm to be correct for all possible values.

That doesn't matter, the point is that this is a falsifiable function and that all dependencies (parameters) are exactly known. There are no side effects and it is possible to examine the function in total isolation.

The difference with scientific theories of the natural world, is that they come with a set of assumptions. In the mathematical and software world, it is much easier to concisely define a domain in which to operate.

I can only wonder what Karl would have thought of this Python code and the unit test, but it would support his line of thinking.

Applicability

Are there functions that are harder to falsify? The cost_function is a well-defined example, but in practice software also interacts with messy, non-deterministic systems. If a function includes some form of randomness, or interacts with some global state or external system, it becomes harder to test it in isolation:

def hard_to_unittest_function(file_path):
    global global_counter 
    global_counter += 1
    
    with open(file_path, 'r') as file:
        content = file.readline().strip()
        
    random_number = random.randint(1, 100)
    
    return f"Counter: {global_counter}, Content: {content}, Random: {random_number}"

But still, it is possible mock external dependencies, or write integration tests that combine multiple systems.

Would Karl have liked the fact that falsifiability is applicable to most if not all software systems as long as the requirements are clear?

Iterative development

Software engineers also run, use and test their implementations. If it passes these preliminary tests, it may be deployed to production so other people can start using the software. Even then, end-users may find bugs in the application.

Bugs are gathered in some bug-tracking system, and prioritized. A software engineer looks at the bug description, and solves it by slightly changing the code.

In a similar fashion, when training a Machine Learning model, the model is only slightly altered when it has processed a single record of the training data.

This process of constantly evaluating running code aligns with the thinking that theories should be falsifiable. When code is falsified, we can try to improve it and come closer to a form of "truth", or better alignment with the business requirements.

In contrast, theories concerning the natural world could exist for ages before they are falsified. Experimental settings are often more difficult and expensive to create.

Concluding

Of course, computers as we know them today didn’t exist when Karl Popper started his career. His thoughts resonate with the digital world. It makes me think as a software developer what it would take to test and falsify software that I write. Based on constant feedback loops, we constantly strive to come closer to the "truth", if any such thing can be defined.

Whether Karl would have been a great software engineer is still open for debate, but his philosophy reminds us to test the code we produce in the ever challenging effort to align it with human requirements.