Equipment Reliability Institute
ERI News - your reliability newsletter
May, 2002 - volume 7

Hello again, from Santa Barbara. If your travel plans should bring you through our city, be sure to call me at 564-1260. Maybe we can get together.

Concerning this newsletter, Dave Douthit, one of our new teachers, questions the role of reliability testing in this era of parts being declared obsolete. Prolific statistician Larry George holds forth, this time, about "Broom Chart" graphs. Elsewhere you can read about (and sign up for) upcoming short courses.

Best wishes,
Wayne Tustin

Parts Obsolescence and Reliability Testing
by David A. Douthit

Introduction
A recent but widely accepted observation: the life cycle of components (from design conception to the end of manufacturing) is shrinking to about 9 months! This tight time frame creates "throw it over the wall to Production" or, later, "ship it anyhow" decisions. Very little time is available to evaluate or test these new components before using them. There are numerous sleep-disturbing stories about "lack of evaluation" failures but other problems are nightmares! It is bad enough that parts are becoming obsolete, and not available from alternate vendors. Even worse: manufacturing equipment, certain materials, Quality Assurance processes, inspection procedures, test equipment and test protocols are also among the casualties. All these problems accompany premature adoption of "new" components.

Reliability Engineering
Ideally, reliability engineering should be involved at the very beginning of development of new products (both components and units). And not just to provide "reliability numbers that will support the bid or budget". Such limited use of reliability is at best based on past experiences. These cannot apply to new hardware. New hardware reliability predictions are dangerous if they are based on "similar equipment and field reports". What determines or defines the phrase "similar equipment" is very subjective. Field reports rarely arrive in time to be useful in design decisions. One of the most important (but necessary) decisions is how many spares to provide. Lacking an accurate way to determine this number, it is based on guesswork and budget.

Contractual Requirements
Contracts for high reliability electronics usually include requirements that they meet predetermined reliability requirements. Consider, for example, FAA FAR 25.1309 - Equipment, systems, and installations: "(a) The equipment, systems, and installations whose functioning is required by this subchapter, must be designed to ensure that they perform their intended functions under any foreseeable operating condition." Or consider DoD 5000.2-R - Mandatory Procedures For Major Defense Acquisition Programs (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs.

C3.2.3.2.2.5. "T&E (Test and Evaluation) on commercial and non-developmental items shall ensure performance, operational effectiveness, and operational suitability for the military application in the military environment, regardless of the manner of procurement. Test planning for these items shall recognize commercial testing and experience, but nonetheless determine the appropriate DT&E (Developmental Test and Evaluation), OT&E (Operational Test and Evaluation), and LFT&E (Live Fire Test and Evaluation) needed to assure effective performance in the intended operational environment."

Some of these new "warranty" periods exceed 15 years! Usage and duration are expanding as more end users seek increased "bang for the buck" and "legal recourse".

The Threat
Once a product/system goes into production, many of its components can already be obsolete. "Drop in" replacements rarely exist even from other sources. Any problem with obsolete parts can be serious. The projected spares surplus can be threatened. Huge sums of money may be spent, major delays develop, and entire programs can become threatened.

Early preproduction testing and evaluation are clearly needed. However, justifying the cost of such testing can be difficult. The common anti-test argument, "How many failures/problems have there been?" should not and can not be allowed. Such arguments negate all reliability engineering. They are based on "risk/cost " type analysis that may be suitable for setting life insurance premium rates. Such analysis is not suitable for decisions on new and untried electronic hardware. Without accurate and timely hardware evaluation, major errors may occur!

Need For Early Testing
Such anti-test logic can no longer be permitted! Design, material and process changes are occurring much too rapidly. We cannot wait for either (1) a subjective level of failures to be reported from the field or (2) a subjective amount of money spent trying to do "work arounds" and/or field repairs.

Anti-test arguments create conflicts throughout the supply chain as end users and vendors point fingers while trying to determine (1) how much the testing and "fix" will cost plus (2) who will pay the costs of correcting problems. These situations can be avoided!

It is impossible to produce a high reliability product without accurate and thorough environmental testing. Historical factory and field data do not apply to new designs and processes.

Many "Reliability Tests" Misnamed
There are numerous tests and test protocols that really are process control tests. Because they can impact reliability they are sometimes referred to as reliability tests. No! They are process indicator tests. These tests use coupons, dummy components, and special circuitry. They do not verify that the final product will work!! They only verify that portions of the manufacturing process are under control. Never confuse these tests with reliability testing.

Reliability testing should include functional operation of units, with adequate test time spent under various expected environmental conditions. The order, types, and levels of acceleration vary between, stimulation, to prevent infant mortality failures and simulation of the usage environment, to predict in-the-field longevity.

Exercise much caution because small changes in your test protocols can result in wildly divergent test results. Predicting long term reliability, on the basis of developmental and/or preproduction testing, is risky. Serious research and understanding of expected field operational conditions is needed.

Conclusion
If a development team is to properly function in today's "tight" manufacturing environment, all team members must be aware of all the results of their decisions. There are major ethical dilemmas constantly occurring; many require intervention by upper management. Unfortunately all too often, decisions are based more upon politics and profits than on scientific knowledge.

Concurrent engineering teams must be given full authority to deal with reliability issues. Authority to sign engineering reports must only be vested in technically competent people. The people who sign the engineering reports must be the final authorities on the project. Without that authority there is major potential for disaster.

David A. Douthit is an ERI specialist and will be teaching "Contaminants and Moisture can Disrupt your Electronics" in June and August in different locations. David has consulted over 30 years in the troubleshooting, repair and failure prevention of electronic circuitry and systems. He can be reached at douthit@equipment-reliability.com.

(back to the top)

Broom Charts
by Larry George

Broom charts are graphs of age-specific field reliability estimates by calendar production period. Jerry Ackaret, of Sequent Computer, suggested that name for my graphs. Broom charts help diagnose problems and verify reliability improvement. They can be graphed for any products or service parts. The first example shows what you can learn when you know only production and failure counts by accounting calendar interval. The second shows what you can learn when you know ages at failure and survivors' ages.

Fortunately, it's not necessary to track products by serial number to make broom charts. Ships and returns (complaints, failures, spares sales, etc.) required by generally accepted accounting principles, are sufficient.

This example is from the 4300 Firestone tire complaints collected by the National Highway Safety and Transportation Agency as of December 2000. Production data comes from Ward's annual automotive production data.

Figure 1. Firestone original equipment tire reliability estimates

Figure 1 graphs reliability estimates in terms of age at complaint from successively larger subsets of data. The estimates cover all calendar intervals up to a specified year, as if you had been computing the estimates as ships and returns data accumulated. Reliability deterioration started to be evident in 1996, when the recalled Firestone tires had been in the field for six years.

This example shows a reliability estimation limitation when only ships and returns are available. It is not clear whether the earliest production has finally started to cause complaints or whether recent production is the source of the complaints. I have seen both of these explanations proved valid. The decrease in reliability from 0.99999 in 1996 to 0.99996 in 1997 and to 0.99988 in 1998 should have been enough to alert reliability engineers and encourage them to obtain sample ages at failure so they could have estimated broom charts like Figure 2 by production calendar interval. Ages at complaint were available from NHTSA complaints data for estimating Firestone tire reliability by year of production. That estimate showed that production in 1996 was the source of complaints.

The next example resulted from offering free field reliability analyses. In February of 2000, a corporate quality manager called me and said his product had a little problem: their EEPROMs seemed to be losing their memories. His company tracks products by serial number, so ages at failure and survivors' ages were available. I made nonparametric estimates of age-specific field reliability by calendar quarter of production. Nonparametric estimates preserve failure factors, such as maintenance and product testing that may provoke or detect read errors. Figure 2 shows that one quarter's production was wearing out significantly earlier than other quarters'. The horizontally longer lines correspond to EEPROMs produced earlier.

Figure 2. EEPROM reliability estimates by calendar quarter of production

The quality manager called the EEPROM supplier and asked what had happened in January through March 1999. The supplier said that it had changed the die size. The quality manager returned some of the defective EEPROMs produced during those months. Testing showed that a few memory cell voltages were out of specification, meaning a few bits had flipped from 0 to 1 and vice versa.

Broom charts are informative, but they are not the only means of monitoring product field reliability. Consider using the reliability estimates to forecast returns. Estimate upper confidence limits on the forecasts to get a basis for returns process control.

Larry George, an ERI's specialist, is a reliability statistician doing business as Problem Solving Tools. Click here to find more information about Larry. He can be reached at lgeorge@equipment-reliability.com.

(back to the top)

Questions our readers have asked...

This section of our newsletter was created for you, reader! Feel free to send questions or suggestions to the webmaster. They will be either responded to by Wayne Tustin or forwarded to one of our specialists.

Here are some questions for this issue:

Q: What is random vibration?
A: "Random" basically means "unpredictable". I'm assuming that you are familiar with sine and with complex vibrations. When you view these (using an accelerometer for converting motion to a voltage signal) on your oscilloscope, the 'scope sweeps over the same sample again and again, giving your 'scope a stationary, repeated, pattern. Not so with random. Every sweep will be different.

Q: http://www.dtc.army.mil/publications/MIL-STD-810F does not work for me. Can I find Military Standard 810F on the Web?
A: Try this. Go to http://www.dodssp.mil/dodssp.htm. You have to register. After you are registered you can go to http://assist.daps.mil. Log in. Click "document search." Use "numeric search" and type in "810", then choose "MIL-STD-810F".

Another, faster, approach: Go to http://www.dtc.army.mil/publications/milstd.html, then click on 810F. It's a pdf file, over 5 MB. You'll encounter several acronyms. DODSSP stands for Department of Defense Single Stock Point for military specifications, standards and related publications in Philadelphia. DAP stands for Document Automation and Production Service. dtc is the Army's Development Test Command.

Wayne Tustin, ERI's president, can be reached at tustin@equipment-reliability.com or at 805/564-1260.

(back to the top)

Subject: In regards to the five nine process in an article by Larry Schull - newsletter vol. 6.

I would disagree with the author. Inspection and testing are important parts of product verification which is, itself, an important part of an overall approach to quality. The focus should be on improving the process. Every process. This type of focus provides the maximum benefit to the producer and the consumer. Imagine, you go to buy a TV and they try to sell you on the fact that it was 100% tested. I would immediately be nervous. You need to ask:
* tested for what and to what criteria?
* what variability in the manufacturing process necessitates 100% testing?
* what other variability is overlooked or ignored?
Again, the focus should be on process control and the measuring of variables which are process indicators.

Bob J. Terril, Lord Corporation.

(back to the top)

Wayne's "Fundamentals of Vibration and Shock Testing and Measurement" course is happening May 22-24, in Newport, Rhode Island. Registration is open until May 21. Hurry! Get more details about this course here or send us an e-mail with your questions.

David Douthit will teach "Contaminants and Moisture can Disrupt your Electronics" in Austin, Texas, June 10-12. He provides details about the root causes of many poorly understood electronic failures. The chemistry encountered in many end-use environments (and in some manufacturing environments) will be covered at length. Dave will discuss sources of contamination and the effects of that contamination on electronic systems. This understanding will enable participants to create more complete and accurate testing protocols and better estimate long-term reliability of new electronic designs, materials and processes. This understanding will also help organizations to troubleshoot production problems and field problems with existing electronic designs, materials and processes. Click here to get more information about this course.

Do you use PCBs (printed circuit boards)? Do you design PCBs? Manufacture PCBs? You should be sending your designers to John Starr's "Optimizing Electronics Vibration - HALT, ESS and HASS" program at Santa Barbara, August 19-21, 2002. Click here to see a low-technical-level introduction to Starr's program.

We were not able to put our new Web sites online as planned. This is an apology from ERI's Webmaster, who has torn her hair a bit getting the new features ready for our visitors. Please stay tuned!

Click here to see a free sampling from ERI's distance learning in CD-ROM. A shaker-mounted cantilever beam resonates as a result of sinusoidal shaking at various frequencies.

Equipment Sought
I am looking for a commercially available variable frequency strobe light with a frequency display. I need this device to identify the frequency of simple harmonic motion that has been induced in light weight objects on a shaker table. Please reply by phone or e-mail:
Charlie Cote
Principal Engineer
Product Focus Enginuity LLC
Office: 970-212-2881
e-mail: ccote@enginuityinc.com

Shock Testing Article
"Shock Testing Miniaturized Products" is definitely worth reading. You'll find George Henderson's article in the April 2002 issue of Evaluation Engineering magazine, page 52. Or you can read it online. As DUTs (devices under test) become smaller and stiffer, with higher natural frequencies, shock machines must deliver ever-shorter shock pulses.

EMC Article
Bill Parker (EMC consultant), Tony Masone (EMC/EMI manager at Garwood Labs) and Wayne Tustin generated "The Case for Combining EMC and Environmental Testing," which appears in the 2002 edition of RB ITEM. EMC problems sometimes worsen at temperature extremes, during vibration, etc.

Fuel Cell Technology
Do any of our readers know of someone who can teach a beginner-level short course in fuel cell technology, particularly (but not exclusively) for automotive applications? Please send us the person's name, address, phone, e-mail address, etc. Thanks!

Seeking teacher
Someone once told me that the number of pressure sensors sold is 10X the number of accelerometers sold. Whatever the ratio, it is certainly greater than 1:1. ERI is seeking someone to teach occasional short courses about dynamic and static pressure measurements and calibration.

ERI - Equipment Reliability Institute
1520 Santa Rosa Ave
Santa Barbara - CA - 93109
Tel/Fax: (805) 564-1260

Wayne Tustin tustin@equipment-
reliability.com

Webmaster webmaster@equipment
- reliability.com

Web sites
http://www.equipment-
reliability.com

http://vibrationand
shock.com

Copyright © 2000-2002 Equipment Reliability Institute. All rights reserved.

Subscribe
If you would like to subscribe to ERI News, go to our web sites, fill in the form "Free Newsletter" and hit the Submit button. Subscribe now!

Recommend
If you enjoyed reading ERI News and want to recommend it to a friend, just hit "forward" on the menu of your e-mail program or tell your friend to subscribe to it at our web site.

Format
ERI News is sent in both html and plain text formats. If you had any problems reading this newsletter, please let us know. Send an e-mail to the webmaster, reporting your difficulties.

Previous issues
Missed the previous issues? It is not a problem. Send an e-mail to the webmaster and let us know which issues you would like to receive.

Unsubscribe
If you do not want to receive ERI's quarterly newsletter, please send a reply to this message with "remove" as subject.