Diagnostics

On most cars built in the last 14 years there is a little yellow warning light with a picture of an engine on. This ‘Check Engine light, sometimes called the MIL light (Malfunction Indicator Light), comes on when something is gone wrong with the engine, it might not be a big problem but the engine’s computer thinks it’s at least bad enough for the car to fail an emissions test.
The great thing about these cars is that you and I can read its mind.

It’s been a long time since On Board Diagnostics (OBD) became standard on cars. There have been a few variations on the theme, such as K line or CAN, but these days there are a respectable number of fairly cheap devices that can read fault codes, making looking after your car that bit easier. In fact I would encourage any car enthusiast to get one.

For instance I use an application for my Android phone, it’s called Torque (I reviewed it in Evo magazine last year) and it cost me the princely sum of £2.92. To be able to physically talk to the car I connect it using a Bluetooth OBD interface based on the ELM327 chip, which cost about £12 off eBay. This allows me to read fault codes from the engine, and when appropriate to clear them. It also allows me to look at the values from sensors such as coolant temperature, engine speed and the signals from the oxygen sensors.

My HTC running Torque, this window lets me set up a virtual dash showing any info I want from the car
My HTC running Torque, this window lets me set up a virtual dash showing any info I want from the car

This is very handy when you like playing with bargain bangers, which tend to be about ten years old and frequently come pre-equipped with a host of minor faults. In fact I used it on the last car I bought, I must confess that I did something that I always advise other people not to do and bid on a car on eBay without viewing it! So when I went to pick it up I plugged my phone in to the OBD port and listed off the current faults, then had a chat with the seller about their claim that the car was ‘faultless’. We came to an arrangement.
They say knowledge is power, and knowledge of what’s on a cars mind certainly does give you bargaining power.
And all power came for less than £15, not bad.
I use this kit for servicing and maintenance, it can indicate when a small exhaust leak has just started or when an air meter is dirty and is reducing performance and economy. But I also use it for tuning, I’ve tried different spark plugs and checked the knock reading as well as watching the fuel flow to see if the efficiency has improved. As the phone has GPS I can compare the actual road speed to the speed the car thinks it’s doing, handy for calibrating the speedo when fitting bigger tyres. For someone who like to play with their cars this info is very useful, years ago kit to measure these things would have cost thousands, but now it’s cheaper than a large box of chocolates.
In fact I even use it when I’m working on prototype and experimental cars, as a first line in fault finding and making sure a car is running correctly before an important test.
I also have kit that does indeed cost many thousands, but it is bulky and needs a laptop (for those in the know I’m talking about INCA and an ES592 with all the leads and faf) so if I just need to have a quick look at the basics then I’ll use my phone instead.
Amazingly the app is so good that it can record data from the phone’s other features at the same time, so I can do a few laps of a test circuit and record critical values such as temperatures, air flow, fuel flow, lambda end engine speed, whist at the same time recording G forces from the Android phone’s inbuilt sensor and also record video from its camera. This gives me a very useful log file showing exactly what went on in the engine as I throw the car through the twisty bits.
Of course it doesn’t do everything that full professional kit does, but it gets pretty damn close for a fraction of the price. I am still impressed one year on.
The Blootooth OBD interface similar size to my phone.
The Blootooth OBD interface similar size to my phone.

But even for the normal car enthusiast this kit is really useful, even if you only use it for fault code reading and resetting the ‘Check Engine’ light. I have seen many dealers charging around £250 for this service, so if you only ever use it once you’ve saved a packet.

I should mention at this point that there are two completely separate sets of fault codes from the engine, the set used here is the standard set that is dictated by law, all cars use this set and it includes the ability to clear codes and reset the fault light. But there is also a second set that is manufacturer specific, this allows for unique design features and gives more detail, a simple reader won’t usually understand these codes. Some companies such as VAG make heavy use of these special codes, but even so a basic reader will still tell you if something is not right.

However I have to sound a warning, these fault codes are not to be taken too literally. A common problem is a slightly corroded connector leading to an incorrect diagnosis of a failed sensor, imagine a little bit of moisture creeping into the engine speed sensor connector, leave it a few years and a tiny spot of corrosion forms. Some days when you go to start the engine it doesn’t get a signal from the sensor and so flags up a sensor fault. You take the car to a dealer who plugs in the diagnostic tool, see the fault code and immediately replaces the perfectly good sensor with a new one. When they plug the new sensor in the tiny spot of corrosion is scraped off and all seems fine again. Two things happen, firstly you get charged for a sensor you didn’t need, and secondly about a year later the same fault re-appears. All it needed was the connector cleaning and a quick squirt of contact grease.
Another classic fault is an oxygen sensor reading too lean, but rather than the sensor being at fault it is more likely to be a small exhaust leak that’s causing the problem.
So you see, fault codes can be misleading. They are great for telling you the area that has a problem, but this is only the start of the investigation for a competent mechanic.
It’s one thing to read fault codes, it’s another to actually understand them.
But don’t worry if you are not a trained engineer, owning a code reader is still great because you can read the codes and go onto your car’s model forum and ask the collective expertise what might be causing it. The internet is great for this kind of wisdom, and one thing car enthusiasts are good at is talking about problems and solutions. We are no longer limited to just the contents of our own brain.

There are limitations to the ability of cheap devices, most wont read codes from your ABS system or be able to program new keys to your car, but for the rare times you might need one of these features there is usually someone from the forum or club near you who has the kit and is only too pleased to help.

The usual place for the OBD connector on any car is under the dash near the steering column.
The usual place for the OBD connector on any car is under the dash near the steering column.

Your car diagnostics should not be a mystery to you, codes were standardized by law to make sure we all had the ability to fix and maintain our cars, so go on, splash the cash on a new gadget and explore your motors mind.

And yes, I am full expecting to get bombarded with questions about codes on my Twitter account now! 😉

Computer crash

When I was little I remember listening to old people talking about a time when there where no cars, the feeling of excitement and wonder when they saw their firs one, a feeling mixed with a little fear as the mechanical marvel seemed to take over every aspect of life. Where once they played in the road now the car was king, and a ruthless one at that. Communities divided by a constant steam of deadly traffic.
Of course today we take the car for granted. Many have moved away from the workers slums into suburbia and now rely on the car to support this freedom.

ASL, DSC and other stability control systems allow even the clumsiest driver to enjoy supercar power in reletive safety. Technology can be amazing.

We teach our children ‘road sense’ so they can cross the road safely. Most drivers are not deadly speed demons (although in town most people still speed, 40 in a 30 zone IS deadly). Society adjusts and we move on.
Now it seems that its my turn to sound old because I remember a time when there were no PCs.
I remember the excitement of my first Sinclair ZX80, the awe of seeing the colour ZX Spectrum.
But now I feel the fear.
Now don’t get me wrong here, I am a great believer in the usefulness of computers, I have a degree in computer systems engineering, I have made a career out of devising and tweaking computer control systems for cars.
The performance of this Lamborghini is only possible because of the massed computer systems doing highly complex things to make the engine, gearbox, suspension and brakes work to perfection.

But still, now I feel the fear.
When I was studying to become and engineer, every step of the way I was told of the importance of doing things properly. With a large computer program one has to exactly and correctly specify what it should do in every detail. One must also specify what it must not do! Once the program is written then it must be tested against this specification and every possible combination of circumstances must be tested. That way there are no ‘bugs’ and unexpected effects.
But life is not like that.
The software (and also hardware now) on almost everything is so complex that it requires a computer program just to be able to test it.
No one programmer can do the whole thing, its just too big, so we have teams. So now we have programs to help the teams work together without bits getting left out and prevent miss interpretations etc.
But we live in a capitalist society. Its not just the engineers that create products, its corporations. Many individuals with their own beliefs on how things should be done dictating the boundaries and detail of what the engineer can do but without a sound understanding of the technicalities.
Money has too be made (exceptions include Linux (three cheers)) and so whole chunks of code from other programs are grafted in to new programs, the people producing this new program may not know the details of how this chunk was written and all its effects. Sometimes there may be a ‘surprise’ effect caused by the interaction of this chunk with the rest of the program, other chunks grafted in or indeed other programs running on the same machine or network.
Testing takes time and money and delays the launch date. Some things just cant be tested completely due to their nature, for example if your program predicts the weather then how do you test every possible combination of weather across the whole world and still meet the deadlines.
The Jaguar CX-75 uses complex computers to manage a highly tuned engine plus high power electric motors to bring stunning performance with minimal fuel use, a fantastic use of technology.

Also the hardware too is so complex that it is not commercially viable to test everything, or indeed possible. With several million transistors on a single chip is never going to get tested for the effects of every combination of individual transistor failures.
So that’s where we are today. Our systems are only partially tested and often a patchwork of other peoples work all stuck together with hope and optimism. Or indeed sometimes cynicism.
Many consumer products are made by inexperienced teams and pushed out by unscrupulous corporations (particularly in countries where software standards are not enforced) and are largely unproven.
Many of us have experienced the result of this growing problem, such as the PC just locking up when you try a new program or simply getting slower and slower as time goes by. These bug and software faults are so common that many people think it is normal for computers to behave like this. For instance the PC I am writing this on is twelve years old, it still does everything it was designed to and since running Linux it hasn’t slowed right down or ground to a halt, yet still most people accept that computers need replacing every other year and expect it to slow down over time. It must be realised that it doesn’t have to be this way, technically, but commercial pressures will continue to make the problem worse and this will be compounded as more and more code is piled on to bring use ever more features.
Complexity is a big problem and is the subject of many a professors career, things are getting more and more complex and there is no proper engineering control on it.
Now, the reason that I am writing this is not just to have a good whinge about my computer crashing or indeed to complain about commercial forces ruining good engineering. Those things make me angry, but they are not the cause of my fear.
The fear stems from how we are using these systems as a society, how we are relying on the unreliable.
Computer systems are now increasingly being used as part of the law enforcement system, finance control, travel systems and even food production
Speed cameras always cause a good argument so I will stir thing up a bit further. Now I know very well that excessive speed increases danger of injury and general twisting of machinery and putting a speed camera outside a school is no bad thing.
The issue for me comes from the fact that the picture generates an automatic fine for a person. There is no human judgement in the loop, bang, guilty until proven innocent. And that’s wrong.
A friend of mine suffered from a theft from his car, not the usual sort of theft, the number plates were stolen. It turns out that persons of criminal persuasion are stealing a car then cruising round till they find an identical type of car and putting those plate on theirs. Then they can generate speeding fines and parking tickets with impunity and even commit serious crime knowing full well that the system will point the finger at some one else. It even cause the police to waste time with the wrong chap, keeping the heat off the criminals long enough for them to make their escape.
Guilty until proven innocent, trial by computer, not good, not very British.
Maybe soon we will all have ID cards. This means that criminals only need to forge one item instead of a string off items as at present, thus making their life easier. The systems used for security are simply to complex to be testable, and driven down on price so the quality is marginal. Its simply not reliable.
If you want quality you have to pay for it because quality systems take more time to engineer and more time to test and it all costs money.
We are entering the beginning of a time when cars become more autonomous, adaptive cruise control will adjust the car speed to the traffic conditions, lane assist can nudge the steering to stop you drifting off your chosen path, we even have auto parking systems. It is a logical step to bring all these ideas together and link them to the sat nav to create fully autonomous cars, Google are investing heavily in this idea. Once the systems become common there will be increasing pressure to ban manual driving, after all an autonomous car doesn’t get road rage, doesn’t speed, can see through fog, never gets distracted and should never crash. All those computer systems running all those programs written by thousands of different people at different times in different places and controlling your car….
In the near future there will be an attempt to make remote vehicle arrestors mandatory on all new cars. This system uses ABS systems that have full authority breaking and engine management systems to bring a car to a halt using a radio command that only police will have. In a simplistic world this is great, you report your car stolen and the police can bring it to a halt when the conditions are safe. No more getaway cars. Well, unless criminals use older cars, but that loophole is easily solved by making classic cars illegal and crushing them all!
The problems include accidental stopping of the car (you cant prove the software completely due to its complexity and you cant prove the hardware completely because you cant test every failure and every type of possible radio interference etc), incorrect use by the police or other agencies, vehicle being stopped by criminals equipped with illicit stopping systems for the purpose of car jacking. Finally there is always a way to bypass the system, always a loop hole, a bug, a back door or an ‘unintentional feature’.
I was on a train in Germany last year which suddenly stopped in the middle of no where without warning, brakes full on. Luckily I had finished my coffee so the cup was empty when it slid of the table. The cause of this potentially dangerous emergency stop was a software error in the very system that is supposed to protect the train from crashes.
Our corporate based society does not allow for well written systems to be made as profitably as the quickly written ones.
This is a real problem and is getting worse as more systems are used.
In my life I rely on a mobile phone, I rely on my car, my computer, email, bank direct debits, automatic payments, alarm clock, microwave, fridge, washing machine, traffic lights etc. The power feeding my home is controlled by systems all linked together in a network. The amount of chlorine in the water I drink is monitored electronically. Aeroplanes are flown expertly by computers over my head, the air traffic is controlled by other computers.
I use my switch card to pay for car tax, the little computer in the post office reads my details and talks to one of many networked computers at the bank, the figure in my account file is reduced and a message sent to the post office bank computer to tell it to increase the number in its account. Then a message is sent to a computer at DVLA and it changes the value of a variable in a file so that when another program does its daily check of who has tax it will not automatically send a message to another computer to send me a fine and automatically turn me into a criminal. I never see these computers and they never see me. But they can bankrupt me accidentally or send me to jail.
These systems are not designed completely by engineers, the specifications and design constraints are created by politicians and computer sales executives who simply don’t understand.
When I was a child, I was proud to be British, a country that believed in tolerance, understanding and fair play. I was proud of my country.
Now I am scared of my country and the automatic systems that rule my life.
My bank local branch has just got rid of all its cashiers, you have to use the machines now. Signatures have been replaced with PINs.
Make no mistake, these systems give us great ability as a society and as in individual. The principles of the systems are very good, it’s often empowering and can change lives for the better. Even this blog site gives me a platform to express my beliefs and concerns in a way that was impossible a generation ago. I am a great believer in technology.
But as far as I can see if we are to rely on systems then they must be reliable.
Also, there must always be a human in the loop when ever civil liberty is at stake.
And finally, there must always be a manual back up for those odd days when thing don’t quite work the way they should.

Car faults in perspective: What can possibly go wrong….again..

One in a million.
My boss told me “so that means your design will defiantly kill two people per year!”.
That was 20 years ago, when I was a fresh faced engineering graduate in my first job at a global car maker. I was designing bits of engine management system, and as ever I had gone through every type of conceivable failure and worked out how well it was protected against. But one very obscure scenario involved the car stalling on a hypothetical level crossing near a strong radio transmitter, a bit tenuous but it is a situation that could happen, I had gone through the figures and worked out that it was a million to one chance that the engine would not restart, resulting in something bad involving a train and sudden localised distortion to the car (ok, a crash).
I thought that this was a remote chance, but my then boss pointed out that the systems would be put on about 2 million cars per year in Europe, hence his terminal conclusion.
I redesigned it. No one had to die.

Cars made in high volumes are used in every sort of environment possible, testing for all occurances is a huge investment.

But even so, I am sure there could be even more obscure situations I had never even thought of, I probably could have spent years going through more and more complex scenarios, but the the car would never have been made. So we have to draw the line somewhere.

How common are uncommon faults?
Cast your mind back to Toyota’s ‘sticky pedal’ problem, millions of cars work fine yet a handful of unverified complaints necessitated a total recall. You just can’t take chances, even if almost every car is perfect.
Of course Toyota are no worse than Ford, Mercedes and all the rest, all volume products suffer from occasional problems, largely due to the scale of production and of course because we want our complex cars dirt cheap, and that’s not going to change any time soon.
When an industry has to make very complicated machines with highly sophisticated features that are used by the general public who have only minimal training, and have to endure a vast array of harsh environments including salt spray, Arctic freeze, road shocks and days on end in scorching sun, things are going to be difficult. And when this problem is massively compounded by having to make the car as cheap as possible, something has to give.

New ideas like this Rolls Royce EV undergo a huge amount of testing before any customer is allowed near it.

Times this set of problems by the millions of cars made every year and the law of averages is definitely not on the side of car makers. If you think about it, the mere fact that when something does go wrong it makes the headlines tells us something about the utterly fantastic job that all these companies usually do.
If the average Joe knew anything of the vast amount of sheer hard work that goes into creating cheap, economical, useful and reliable cars they would bow down in reverence, and those that fancy their chances at suing for spurious accidents would hang their head in shame.
But hardly anyone knows about all that fantastic engineering work, it doesn’t make sexy TV programs, it’s not vacuous and glamorous enough to make it into the glossy magazines. So every one just accepts that every machine should work perfectly no matter what, and are utterly surprised on the very rare occasion that it doesn’t.
So how often do things fail? Well things are much more likely to go wrong when any product is either new or reaching the end of its designed life, the first few miles a car experiences show up any glitches in production and then once these are sorted most modern cars will trundle on for over a decade without significant problems (assuming its correctly maintained). During the cars early life car makers measure things in returns per thousand and generally they run well below 5, that’s 0.5% of cars having any sort of fault at all in the first year of ownership. Good models will run at less than 0.005%, and these faults could be anything from a cup holder breaking to an engine failing. The trouble is that if you churn out a couple of million cars a year then even these tiny numbers mean there will be hundreds of failures in the field, unfortunately these make good stories. Manufacturers hate even these small numbers of faults, obviously every company’s dream is to have no failures at all, and indeed some models achieve this, and they are all striving to eradicate all potential for failure. But occasionally I think its a bit sad you will never see a headline reading ‘millions of car turned out to be pretty good actually’.
Even a very high powered Porsche can be safely driven sideways in the rain by an idiot driver, as shown here.

Cars are amazing.
Here’s a challenge for you; think of a machine that has to work in heavy rain, baking sun, snow, ice, deserts, be precise on tarmac yet still cope with cobble stones, Suffer grit and gravel being blasted at it from underneath and do a huge range of complex mechanical tasks at temperatures between -40 to +50 C, last over a decade whilst being shaken, accelerated, decelerated by novice users in a crowded and complex environment.
There are no other machines, just motor vehicles, which have to contend with all this.
But it doesn’t stop there, the engine is retuned every combustion cycle, hundreds of times each second in order to meet the incredibly stringent emissions laws, pollutants are measured in parts per million, the tests are so sensitive that simply exhaling into an emissions test machine would cause the limits to be exceeded (note; these are not the simple emissions testers used at MOT stations, the MOT emissions limits are laughably lax by comparison to the certification tests the manufacturer has to do).
To give you a very rough idea of the amazing computing power needed to control and engine to these limits, a modern engine control box (ECU) may have around 25 thousand variables, tables, maps and functions. It calculates mathematical models of how the air flows through the intake system, how the pistons and valves heat up and how the catalysts is performing, it analyses the subtle acceleration and deceleration of the flywheel every time a cylinder fires, it listens to the noise the cylinder block makes and filters the sound to decide if the engine has the slightest amount of knock (in fact some engine deliberately run the engine into borderline detonation to extract maximum efficiency). It talks to the gearbox to anticipate gear changes and control torque so that the gearbox ECU can precisely control the energy input into the drive line during a gear shift. It analyses the long and short term behaviour of every single sensor and actuator to automatically compensate for ageing and wear as well as diagnosing and compensating for any faults.
But it doesn’t stop there, on some cars the suspension analyses the road and adapts to suit, the auto gearbox monitors the drivers ‘style’ and changes the way it works to please them. The brakes check wheel speed thousands of times a second and deduce when a tyre is about to skid, not when it already has started skidding, and relieve brake pressure just before it happens to ensure the tyre provides maximum grip and stability.
The climate control breathes in cabin air through tiny aspirated temperature sensors and adjusts valves and flaps to discretely meet your comfort needs. The stereo selects a nearby station as you drive along and seamlessly switches in so you never have to retune in order to continue to listen to Radio 2 on long journeys. All sorts of things are controlled and monitored from fuel pumps to light bulbs.
This is the engine and gearbox control from a 20 year old Jaguar, since then it has got a whole lot more complicated!

All in all an average family car might have between five and ten computers working together, sharing information and jointly controlling the car, a typical example would be the ABS unit supplying road speed info to the gearbox so it knows what gear to select. Luxury cars can have over 50 different computers, even the seat heaters have self diagnosing control brains in and talk to the car on a serial bus, and they all interact with things like the battery management systems which may at any time request all these systems change the way they are operating in order to cope with some adverse situation.
The way these systems work together can be very complex, for instance stability control uses the ABS system to apply brakes on individual wheels in order to pull the car to one side as well as requesting a certain wheel torque to ensure the car goes in the desired direction, this torque is controlled by the gearbox and engine working together too, the engine can react almost instantaneously by altering the spark angle (these events happen so fast that the engine has to wait for the airflow to reduce going into each cylinder even though it moves the throttle immediately, because of the air’s inertia!).
Components have to operate faultlessly for millions of cycles, if an engine or drive-line fault develops then the systems must identify it, adjust the mode of operation to minimise risk to car and people, and alert the driver, just like having an expert mechanic on board.
In addition the car has to be comfy by isolating key frequencies from being transmitted by the suspension and engine mounting systems, prevent wind noise from the gale force breeze rushing past the shell, stop the metal box that makes the cabin sounding like a metal box and muffle the many kilowatts of noise running through the exhaust pipe.
It also has to be economical, using every drop of fuel sparingly, compromising the shape of the car itself to reduce drag whilst still allowing enough space to get everything in and have enough air flow round the hot bits to stop them degrading.
But as well as being frugal it also has to perform well, even a modest family hatchback these days has the performance of a race car from the ’60s, indeed there are many saloons with well over 500bhp now, compare this with the 1983 F1 race winning Tyrrell with 530 bhp. Yes our super comfy mobile entertainment centres have the performance of an older Formula 1 car.
And not only does it have to balance all these driving related tasks but it also has to have a really good sound system and have most of the comforts of home, some even have cup holders and fridges.
A few decades ago an Engineer could just look at a car, such as this ultra rare Lagonda V12, and understand how it worked. How times have changed.

Not even the Space Shuttle has to contend with this level of sophistication. I can’t see rockets running catalytic converters and exhaust mufflers any day soon.
And here is the kicker; as well as coping with all that, it also has to perform special functions in a crash. We have multiple air bags, who’s operation is tuned to the ‘type’ of crash detected, we have automatic engine cut, hazard indication, seatbelt pre-tensioning and some cars even ring for help. The structure is designed and tested to ensure it collapses in a controlled manner, the engine design is constrained by pedestrian head impact tests on the bonnet, even the steering wheel is designed to steadfastly hold its position as the cars structure a few feet in front of it is crushed at a rate of up to 15 meters per second.
Name me one other machine that has to detect, reliably, when it is about to be destroyed and then deploy safety mechanisms in a controlled and measured manner during the actual process of its own destruction. You’ll struggle with that one.
Now this feat of engineering would be amazing even with an unlimited budget, but the fact is that cars are made as cheaply as possible, which just take the achievement from amazing to utterly astonishing. In fact you can buy a basic car for the price of a really good telly, that’s bonkers.

Please take a few moments to look at your own car, and marvel. And if one part goes wrong by all means take it back and get it fixed, but do try to be sympathetic to the scale of the problem engineers face.

The road ahead is challenging, but also very exciting as Engineers turn dreams into reality.

Post Script:

Media hype
I noticed something interesting during the Toyota recall, the media could have played a very useful role and helped society, I say ‘could have’ because what they actually did was the complete opposite.
What they could have done is reported actual news, facts presented objectively such as ‘a small numbers of cars may have a fault causing the pedal to be stiff’. That is a fact, it gets the info over simply and effectively, you know what is being said. Simple.
They could have gone further and said something like ‘if your pedal feels stiff visit your dealer, but first check the floor mat hasn’t got stuck under the pedal’. That would be helpful.
But they didn’t do that.
No, what actually got reported was along the lines of ‘mum of five in death plunge tragedy’ and ‘is your car a ticking time bomb of doom?’. Stupid, dramatised gossip that conveys absolutely no useful information.
But of course this scaremongering helps to boost sales of that form of media bilge, so expect more useless crap in the future about every important storey going.
And this is a real problem, not only because it leaves us all badly informed and scared, but because the car companies now know that being honest and open has become the wrong thing to do.
All media has a responsibility, and its time they (we) faced up to it.