Relying on the unreliable.

When I was little I remember listening to old people talking about a time when there where no cars, the feeling of excitement and wonder when they saw their first one rattling and belching down the cobbled street, a feeling mixed with a little fear as the mechanical marvel seemed to take over every aspect of life. Where once they played safely in the road, now the car was the king and a ruthless one at that. Communities divided by a constant steam of deadly traffic.
Of course today we take the car for granted. It would be an over simplification to say we have moved away from the workers slums into suburbia and now rely on the car to support this freedom, but you get the idea.
We teach our children ‘road sense’ so they can cross the road safely. Most drivers are not deadly speed demons (although in town most people still speed, 40 in a 30 zone IS deadly), everyone works together to make the new situation work. Society adjusts and we move on.
Now it seems that its my turn to sound old because I remember a time when there were no PCs.
I remember the excitement of my first Sinclair ZX80, and the awe of seeing the colour ZX Spectrum. In fact looking back I can still feel a little of that excitement about those pioneering machines.
But now I feel the fear, a deep and profound fear.
Now don’t get me wrong here, I am a great believer in the usefulness of computers, I have a degree in computer systems engineering, I have made a career out of devising computer control systems for cars and I love gadgets.
But still, now I feel the fear.
When I was studying to become and engineer, every step of the way I was told of the importance of doing things properly, especially where safety or security were concerned. With a large computer program we were taught to exactly and correctly specify what it should do in every detail. We had to also specify what it must not do! Once the program is written then it must be tested against this specification, and every possible combination of circumstances must be tested. That is the only way to ensure there are no ‘bugs’ and unexpected side effects.
But life is not like that, it turns out.
The software (and also hardware now) on almost everything is so complex that it requires a computer program just to be able to test it.
No one programmer can understand the whole thing, its just too big, so we have teams which may be spread out across the world. This gets complicated in itself so now we have programs to help the teams work together without bits getting left out and to prevent miss interpretations etc.
But we live in a market driven society. Its not usually the engineers alone that create products, it’s the corporations. Many individuals with their own beliefs on how things should be done dictating the boundaries and detail of what the engineers can do, but far too often without a sound understanding of the technicalities.
Money has too be made (although notable exceptions include Linux and shareware (three cheers)) and so whole chunks of code from other programs are grafted in to new programs, the people producing this new program may not know the details of how this chunk was written and all its effects. Sometimes there may be a ‘surprise’ effect caused by the interaction of this chunk with the rest of the program, or with other chunks grafted in or indeed other programs running on the same machine or network.
Testing takes time and money and delays the launch date. Some things just can’t be tested completely due to their nature, for example if your program predicts the weather then how do you test every possible combination of weather across the whole world and still meet the project deadlines.
The hardware too is so complex that it is not commercially viable, or indeed possible, to test every single thing. With several million transistors on a single chip it is never going to get tested for the effects of every combination of individual transistor failures.
So that’s where we are today. Our systems are only partially tested and often a patchwork of other peoples work all stuck together with what boils down to little more than hope and optimism. Or indeed sometimes cynicism if the corporation concerned has little respect for the end user of its products.
Many consumer products are made by inexperienced teams and pushed out by unscrupulous corporations (particularly in countries where software standards are not enforced) and are largely unproven. There is also the modern phenomena of social networks, these are a great benefit to individuals and businesses alike and I use Twitter, Facebook, Google+ and blog sites to generate interest in my work. In fact I rely on these systems as a key part of my business, but these were never even designed to be mission critical business tools.
Many of us have experienced the result of this growing problem, such as the PC just locking up when you try a new program or simply getting slower and slower as time goes by. These bug and software faults are so common that many people think it is normal for computers to behave like this. It must be realised that it doesn’t have to be this way technically, but commercial pressures could continue to make the problem worse.
Complexity is a big problem and is the subject of many a professor’s career.
Now, the reason that I am writing this is not just to have a good grumble about my computer crashing or indeed to complain about commercial forces ruining good engineering. Those thing make me angry, but they are not the cause of my fear.
The fear stems from how we are using these systems as a society, how we are relying on the unreliable.
Computer systems are now increasingly being used as part of the law enforcement system, finance control, travel systems and even food production
Speed cameras always cause a good argument so I will stir things up a bit further. Now I know very well that excessive speed increases danger of injury and general twisting of machinery and limb, and putting a speed camera outside a school is no bad thing.
The issue for me comes from the fact that the picture generates an automatic fine for a person. There is no human judgement in the loop, bang, guilty until proven innocent. And that’s wrong.
A friend of mine suffered from a theft from his car, not from inside but from outside. The number plates were stolen. Persons of criminal persuasion had stolen a car, then went cruising round till they find an identical type of car to put an innocent chaps plates on their stolen car. Then they can generate speeding fines and parking tickets with impunity and even commit serious crime knowing full well that the system will point the finger at some one else. It even causes the police to waste time with the wrong chap during the investigation, keeping the heat of the criminals long enough for them to make their escape.
Guilty until proven innocent, not good, not very British.
Soon we may all have ID cards. This means that criminals only need to forge one item instead of a string of items as at present, thus making their life easier. Another classic case of the decision makers not understanding the technology.
The systems used for security on such cards are simply to complex to be testable, and driven down on price so the quality has to suffer. It’s simply not reliable.
If you want quality you have to pay for it because quality systems take more time to engineer, and crucially more time and resources to test, it all costs money.
In the near future there may be an attempt to make remote vehicle arrestors mandatory on all new cars. This system uses ABS systems that have full authority breaking and engine management systems to bring a car to a halt using a radio command that only police will have.
In a simplistic world this is great, you report your car stolen and the police can bring it to a halt when the conditions are safe. No more getaway cars. Well, unless criminals use older cars, but that loophole is easily solved by making classic cars illegal and crushing them all!
The problems include accidental stopping of the car (you cant prove the software completely due to its complexity and you cant prove the hardware completely because you cant test every failure and every type of possible radio interference etc), incorrect use by the police or other agencies, vehicle being stopped by criminals equipped with illicit stopping systems for the purpose of car jacking. Finally there is always a way to bypass the system, always a loop hole, a bug, a back door or an ‘unintentional feature’.
I was on a train in Germany a while back which suddenly stopped in the middle of nowhere without warning, brakes full on. Luckily I had finished my coffee so the cup was empty when it flew off the table. The cause of this potentially dangerous emergency stop was a software error in the very system that is supposed to protect the train from crashes.
Our corporate based society does not allow for well written systems to be made as profitably as quickly written ones.
This is a real problem, and it is getting worse as more systems are used which interact with each other in even more complex ways.
In my life I rely on a mobile phone, I rely on my car, my computer, email, bank direct debits, automatic payments, alarm clock, microwave, fridge, washing machine etc.
The power feeding my home is controlled by systems all linked together in a network across the country and even linked in to grids in other countries. Some years ago a large areas of the USA lost power as one network was hit by lightning which knocked out a power station, the systems automatically switched in power from other networks but this overloaded them, a domino effect then ensued as one grid after another went out.
Even the basic things in life are computer controlled, like the amount of chlorine in the water I drink. And to be fair most of the time all these systems do a fantastic job, but can we rely on it?
Aeroplanes are flown expertly by computers over my head, the air traffic is controlled by other computers. These systems have traditionally been tested to the highest standards and the track record is superb. But of course it is still a commercial enterprise, and as fuel prices creep ever upward there is increasing pressure on the technology to deliver more for less cost.
I use my switch card to pay for car tax, the little computer in the post office reads my details and talks to one of many networked computers at the bank, the figure in my account file is reduced and a message sent to the post office bank computer to tell it to increase the number in it’s account. Then a message is sent to a computer at DVLA and it changes the value of a variable in a file so that when another program does it’s daily check of who has tax it will not automatically send a message to another computer to send me a fine and automatically turn me into a criminal. I never see these computers and they never see me. But they can bankrupt me accidentally or send me to jail.
At the large scale these systems are not designed by engineers, they are created by politicians and sales executives who simply don’t understand every detail of the system.
When I was a child, I was proud to be British, a country that believed in tolerance, understanding and fair play. I was proud of my country.
Now I am scared of my country and the automatic systems that rule my life.
My bank local branch has just got rid of all its cashiers, you have to use the machines now. Signatures are being replaced with PINs.
These systems give us great ability as a society and as in individual, but if we are to rely on systems then they must be reliable. Particularly government systems and essential services must be run to the best standards we can think of.
Also, there must always be a human in the loop when ever civil liberty is at stake. This is fundamental to a fair and just society, we must have the right to explain and contest. After all it’s not like we are short of people in the world to help out.
And finally, there must always be a manual back up for those odd days when things don’t quite work the way they should, just in case.

Leave a Reply

Your email address will not be published. Required fields are marked *