Old Gods
One of my favorite quotes on computers comes from Joseph Campbell :
Computers are like Old Testament gods; lots of rules and no mercy.
Through the levity there is more than a bit of truth in the statement. Computer programs are dumb, pedantic and utterly unforgiving. Most of the major innovations in software engineering over the past few decades seem to come from attempting in some way to protect us from ourselves. Whether it be commonplace efforts like test driven development or linting to arcane methods such as Hoare Logic see an excellent tutorial here . Software engineering seems to be a mental self-defense art.
It is beyond my knowledge and it would seem beyond any moral computer scientist to propose a sure fire method to guarantee correct code. Even when it's too expensive to fail like in space flight, we've seen almost comical errors such as NASA's 1999 metric mess up. Thus it would seem like a sisyphean endeavor to try get things perfect. Is the best we can hope for a mostly stable application to be quickly patched and eventually discarded when the technical debt mounts too high? Silicon Valley's move fast and break things philosophy seems to agree with this assessment
Yet code is law on face value is a mantra that demands perfection and like the gods of the old testament inflicts brutal punishment for failure. My central argument is that this interpretation isn't accurate. Rather it's a misunderstanding at best and a politically motivated strawman at worst.
Allegality
When Ethereum started the allegality movement, the tagline was code that can't care. My interpretation of the concept is that caring can't be at a protocol level. It is the responsibility of the author and the users as well as their respective jurisdictions to sort these things out. The protocol- like Bitcoin before it- would be an automated goliath dispassionately parsing line after line of logic until something meaningful is output.
This notion seems to violate our human sense of morality. We can imagine a modern case of the inquiring murderer where a terrorist connects a weapon of mass destruction to a timer smart contract. Should we censor the contract to prevent the bomb from going off? Is the code is law paradigm really worth human life? Further discussion can be read in Shi's excellent essay The Ring of Gyges.
The most likely source for this debate stems from a conflation of human with computer ethics. Cryptocurrency protocols, autonomous vehicles, drones and other systems that are in part or completely controlled by code aren't governed by human morality. They are absent of it. This reality also makes them perfectly predictable, immune to corruption and coercion. There is no ego to protect. There is no bias in judgement.
We aren't used to interacting with these types of systems. Humans are creatures of exceptions, biases and relationships. We build dense networks of contacts to protect ourselves from unforeseen issues. We leverage political connections to gain advantages. The more sociopathic manipulate emotions or use extortion to get what they want . These behaviors- for better or worse- are expected and entirely human.
When an actor is introduced so alien to our expectations, we initially could enjoy the benefits, but many will immediately try to change its nature the moment that they are exposed to consequences. The DAO incident isn't some dramatic black swan event; it's the rule. Thus we are left at a strange impasse. Is it ok for humans to be governed by systems that will not care about us?
One Sided Notions
Returning to the beginning, perhaps the hardest component of this paradigm lies in our inability to fully express our intention and will to computers. We simply don't speak the same language and cannot get enough precision to guarantee unintended things won't happen. It seems unfair to punish someone for a misinterpretation. Yet this notion seems awfully one sided.
We can easily see the flaws in computer interpretation, but what if the human side is defective? For example, the recent Tesla autopilot crash attracted international press. Reporters and consumer advocacy groups were quick to call for a dramatic slowdown in the proliferation of autopilot technology citing safety concerns; however, empirical evidence shows clearly that even in Tesla's semi-flawed state, autopilot is dramatically safer than driving drunk. How many are killed every year from distracted or impaired driving?
This nuance is what I think is lost in the arguments against code is law. We like to argue from a particular to a general case, yet fail to understand the long term positive benefits of the general case. Bitcoin's existence has forced a dramatic evolution in payments and banking. It has forced the issue of reducing remittance costs and broader inclusion into the world financial system. It has done all of this without protocol level regulation or accepting human ethics. We've had to engineer humanity in above the protocol.
Similarly Ethereum under a code is law paradigm forces the issue of correctness at a much higher level. It forces engineers to explicitly code exceptions, paths to upgrades or defenses to mistakes into their smart contracts. It forces better practices and punishes bad ones. It seems to be an evolutionary incentive towards a better standard of engineering. Furthermore, it forces humans to think much more carefully about intent and outcomes.
The key point is that code of law doesn't say that you can't have intervention and arbitration. It doesn't say that you have to abandon local laws and regulations. It says that you have to program them into your smart contract. Furthermore, where it's difficult to model intent programmatically, one can still develop a domain specific language to capture it for humans to arbitrate (like a legal markup if one wills).
These are necessary steps to safely interact with an old testament code god. And, they actually bring a great deal of benefit. You end up with much better contracts, cleaner expression of intent, open source libraries that are reusable and emerging government regulations customized to account for the new technology. None of these things are necessary when you permit human intervention at the protocol level.
Nosce Te Ipsum
Thus, I conclude it takes courage to know a new land. We cannot truly appreciate the road ahead nor the events that will hurt us, but we can acknowledge that we are flawed creatures.
The existence of old testament protocols are monuments to our intellect. We have created something that allows us to transcend our very nature and interact with something completely different.
While these interactions have consequences, they make us better and force us to think differently. To me, it is a new epoch in humanity that we must embrace. It's the 21st century's nosce te ipsum moment. Let's not squander it.