Go home safe, every day
The problem
A factory acceptance test (FAT) is not the problem. A FAT, if well done, is a good thing. The problem is the majority of FATs are not done well. The purpose of a FAT is to test hardware and software safety solutions before they are installed in the factory or plant. It is a requirement in a standard like IEC 61511.
The typical process in the industry is that end users specify what they want, engineering partners design it for them, and system integrators build it. So far so good. Before the end user installs the safety solution in their plant or factory, they order a FAT. During the FAT the solution is tested according to the requirements of the end user. Makes sense. Any problem you find in the factory can “cheaply” be fixed compared to the same problems found later when already installed in the field.
What does not make sense is that in the majority of the cases what has been tested during the FAT is not what arrives at, and is installed in, the plant.
- Tests carried out during the FAT do not guarantee that the safety solution works properly.
- Tests are carried out with uncalibrated test equipment.
- Changes are made during the FAT without a formal change process.
- Changes are made after the FAT is finished and everybody went home.
- Equipment tested is not the equipment sent to the field.
From an end user point of view these can all lead to big problems. The end user does not get the safety system ordered.
An example
A couple of months back I was the third party witness at a FAT for a safety instrumented system. In the end I basically had two major issues with the FAT. First of all they only tested functionality, i.e., when you push this button, that needs to happen. Happy flow testing we call that. They did not test what would happen if things were not working as expected. For the safety system this is an issue. In the functional safety business we are almost more interested in what happens if it does not work compared to what happens if it does work.
The second, more serious issue, happened after the FAT was finished. I asked the engineer to give me the version of the application software installed. He told me he could give it to me but he did not understand why it was relevant, as he was going to make changes anyway (and thus the version tested would change). Basically he was telling me that the whole FAT was for nothing, because after the FAT he was going to make changes. What was tested is not what is going to be shipped to the field. If you keep on making changes, why do a FAT in the first place?
Safety Moment – 8 Ways To Improve Your FATs
In order for a FAT to be effective, a couple of things need to be taken care of. Lets go back one more time to why a FAT is performed. The end user wants to get installed in the factory or plant what they ordered a long time ago. In order to achieve this, we offer you 8 ways to improve your FATs:
- The end user should clearly specify the requirements
- The FAT test procedure should be based on these requirements
- The FAT tests should have defined test objectives, references, test steps, and should have defined pass/fail criteria
- The FAT test procedures should not only include happy flow testing but also robustness testing (testing that simulates failures)
- The FAT procedure should be reviewed and approved (by end user and/or third party) before the test takes place
- The FAT should be carried out, verified, and witnessed by competent professionals from the end user and/or third party
- Any tests leading to a modification, should follow the modification procedure. No quick fixes during the FAT.
- When the FAT is successfully completed the system should be boxed up and sealed in front of the end users. No more changes after the FAT.
If end users follow these simple principles they can at least check that what they ordered and witness tested also arrived at the site. Remember a FAT is not successful because everybody signed the FAT report and put their stamps on it. A FAT is successful if it works according to requirements.
Safety is everybody’s business. Use this safety moment to create awareness.
