By choosing to view this document, you agree to all provisision of the copyright laws protecting it. - PDF

Please download to get full document.

View again

of 9
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report



Views: 7 | Pages: 9

Extension: PDF | Download: 0

Related documents
Copyright 2000 by the Institute of Electrical and Electronics Engineers. Reprinted from 2000 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium, Los Angeles, California, USA, January 24-27,
Copyright 2000 by the Institute of Electrical and Electronics Engineers. Reprinted from 2000 PROCEEDINGS Annual RELIABILITY and MAINTAINABILITY Symposium, Los Angeles, California, USA, January 24-27, This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of ReliaSoft's products or services. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by sending a blank message to By choosing to view this document, you agree to all provisision of the copyright laws protecting it. Field Data is Reliability Information: Implementing an Automated Data Acquisition and Analysis System James Jauw Š Amway Corporation Š Ada Pantelis Vassiliou Š ReliaSoft Corporation Š Tucson Key Words: Field Data, Closed System, Non-parametric Analysis, Risk Analysis SUMMARY & CONCLUSIONS This document describes the conception and development of the Amway Product Quality Tracking System (PQTS). The PQTS is an integrated product quality tracking system that allows the organization to 1) Capture product quality, reliability and warranty data from disparate data sources and incorporate that data into a single centralized database that can support powerful data analysis. 2) Carry out sophisticated data analysis routines that transform data into information that can be used for decision-making. 3) Present analyzed reports and graphs in a graphically rich, multifaceted and carefully organized presentation interface accessible via Intranet or Internet to support the decisionmaking of key organizational personnel. Development of this system began with an attempt to improve on labor-intensive manual methods of report creation and proceeded to the development of the fully automated system available to Amway today. A general description of the development process, an analysis of the lessons learned during system implementation and benefits to the corporation are presented. 1. INTRODUCTION The most challenging aspect of product quality and reliability data analysis is often the acquisition and management of the data itself. Many organizations are unable to take advantage of valuable field product-failure or field performance data because data is not collected, the data lacks uniformity or sufficient granularity and/or the data is inaccurate or lacks validity. If data is collected at all, it is often stored in a variety of locations (manufacturing sites, repair centers, in-house testing facilities, telephone call centers, etc.) with little or no integration among data sources or uniformity in format. These deficiencies in the data infrastructure make coherent analysis difficult or impossible. An integrated approach, focused first on assuring that data is properly collected, stored, correctly analyzed and properly presented, is therefore desired. Such an approach will lead to a complete Product Quality, Reliability Analysis and Reporting System that allows an organization to turn realtime data into information, and information into decisions for rapid continuous improvement. Over the last three years, the adoption of this reliability philosophy for business has led Amway Corporation, working with ReliaSoft, to develop and implement an automated system for tracking product field reliability and Quality. The creation of such a system for Amway, the Amway Product Quality Tracking System (PQTS), involved the modification and integration of many individual systems and disparate data sources. The PQTS is a comprehensive and automated system that incorporates all facets of data acquisition, coupled with advanced reliability analysis engines to provide real-time data and reliability predictions for Amway products worldwide, in a comprehensive and easy-to-use graphical format. The early success of this implementation has already provided Amway with an early warning system to apprise management of significant changes in typical defect levels, a reduction in time required for problem analysis and overall product design improvements that significantly increase product quality and reliability. To our knowledge, this is the first such system implemented for a manufacturer of household products. This paper presents the outcome of this effort, an overview of the process as well as discussion of the pitfalls and lessons learned in implementing this system. 2. HISTORICAL AND BACKGROUND INFORMATION The internationally recognized business model of Amway Network Marketing is the subject of books, occasional jokes (as featured on The Tonight Show ) and many personal success stories. It is a closed system of distributors and service providers in which no independent third-party distributor or repair company exists. All products are sold and serviced through company authorized providers. This system is very much like automobile sales where warranty service is the exclusive domain of the dealerships. Amway Corporation began down the path of Reliability Engineering when they started to design, manufacture and sell market-leading home appliances for the Japanese market. Prior experiences with similar products in the United States and Asia showed Amway that better design testing and analysis of reliability data (from in-house reliability testing, field data from both repairs and telephone hotline calls) was essential to rapid risk management and superior customer service. Key to this success was the need to answer reliability-related questions appropriate to the various levels RF 2000RM-089: Page 1 RF of management and areas of business function. Initial management needs focused around the central questions of What is my current defect rate for these units in the field? and How many in-warranty failures are we expecting during this time period? which translated into the question of What is the failure rate of units in the field currently in warranty? Although the questions seem deceptively simple, they are considerably more complex when one considers rolling populations (i.e., population of units under warranty changes continuously), variable sales (i.e., sales change monthly) and that the units under warranty are of various ages. Starting in 1995, Amway began collecting data for the required analysis from a variety of sources. The analysis of this data resulted in in-depth printed reports that included some answers to the previously posed questions, along with further analysis based on subassemblies and components for the product under consideration. Despite some shortcomings in the data available and the necessity of relying on some assumptions in analysis, these first attempts provided impressive results. Amway R&D engineers were able to characterize failure modes from field data and correlate them to in-house testing as well as accelerated test results. This data gathering and analysis process began with engineering knowledge of the product and was coupled with codified symptoms from the Customer Service Telephone Hotline (THL), codified repair information from the product repair center and sales and manufacturing data. The data came from different locations around the world and resided in different computer systems (which varied in type, size, operating system and applications). This initial method, though commendable, was extremely time-consuming. Hundreds of hours were required to set up the queries, create the spreadsheet and related graphs. This adversely affected the timeliness of reports, so that an acute problem in the field could surface without being detected in a timely manner. In 1996, with the help of ReliaSoft Corporation, Amway began a project that would: Automate and improve the data-collection and report-preparation process; validate and solidify the mathematics used for computations and predictions; and develop a system that would transform the data into instant information for use by managers and engineers alike. In time, this project became the Amway Product Quality Tracking System (PQTS). 3. DESIGNING THE PQTS 3.1 Determining the Required Data The first step in designing the PQTS system was to determine the data required for current and future analysis. To determine the required data we first identified the kind of reports and analysis that the system customers (i.e., Amway managers and engineers) would require. Using the current reports as a starting point, several days were spent with managers and engineers to discuss their reporting and analysis needs. The next step was to define the data that would be required to create the reports and analyses designed in that manner. The required data identified in this manner was divided into four major categories: Manufacturing Data, Sales Data, Failure/Repair Data and Telephone Hotline (THL) Data. Figure 1 provides an overview of the required data flow in the PQTS system. Figure 1: PQTS System Data and Report Flow. Each of the data sites identified in Figure 1 provided a piece of the puzzle that would eventually make up PQTS. Data from manufacturing provided the serial number and date of manufacture of each unit along with the serial numbers (or lot numbers) of the subassemblies. Sales data provided the number of units sold per time period and income derived, while repair data provided the serial numbers of the failed units along with the component that failed and the reason(s) for failure. Telephone hotline data provided the symptoms experienced by the consumer along with a preliminary diagnosis and resolution. 3.2 Establishing the Data Infrastructure Once the required data was determined and the data sources were identified, we embarked upon a thorough examination of the current data infrastructure. Like Fortune 500 companies, Amway has a long-standing commitment to automation at every part of the business model. A basic infrastructure already existed, i.e., there was a computer of some type to capture the data at each source and a wire to transmit the data and applications developed for the specific task(s) at hand were residing on each of these machines. These point of use computers were connected by many types of connections either to a Mainframe Computer or via Local Area Network (LAN). These local systems connected to the corporate system and comprised the evolving Global Information Network of Amway. The internal challenge was to optimize existing tools and people. The external challenge was to automate and integrate data from two non-amway entities, our product supplier and repair service provider. To meet these challenges, a decision was made to design our processes to conform as much as possible to the existing processes of these data suppliers. Changing the processes at any of the sites would have been much more complex and would have been met with severe RF 2000RM-089: Page 2 RF User Interface - Dashboard User Interface - Dashboard User User User User User User Home Tech Manufacturing Data Repair Center Data QA Novell Server Compaq Proliant Server Graphics Data Files Admin NT workstation Compaq Deskpro W AN / G lobal Network Mktg. o r R&D Novell Servers Transparent Hubs Customer Service (THL) Data Windows NT Server Compaq Proliant Rackmount NT Dom ain over Novell D ata H andling & A nalysis F iles IntranetW are Amway Data (InfoPump) Sybase Database Server - UNIX SunSPARC 3000 w/ A5000 Network Array 100 Base-T Ethernet Sybase Database & Query Scripts PQTS Dashboard SYSTEM MODEL Figure 2: PQTS System Architecture Diagram objections and obstacles that might have halted the project well before it began. The end result was walking the thin line of accomplishing our objective in such a way that it was transparent to the data providers and, where transparency was impossible, implementing changes in such a way that it minimized the additional burden on these providers. To further complicate matters, the data was stored in disparate locations around the world and housed in a variety of configurations. Although each different configuration was appropriate for the local application, the differences in machines and databases posed challenges of data transportation, translation and importation into PQTS. To acquire and transport the data, an array of automated software tools were created specifically for each site to automatically extract the required data, package it and transport it to a centralized PQTS database. Since different sources had different connections and security concerns, multiple data transfer methods were used including simple Modem transport to ISDN connections and Corporate T1/T2 lines. Figure 2 shows an overview of the different systems, locations and connections that comprised PQTS. Concurrent with this coordination of data capture and transport, the PQTS relational data model was designed and the necessary hardware infrastructure to support PQTS functionality was put into place. 3.3 The Reports and User Interface Knowing full well that all the data in the world is useles unless properly reported and presented, many hours were spent in determining what the reports should look like and how they should be accessed. After many brainstorming sessions, and with the knowledge that the final customer (or the most important customer) was management, our final decision was to present the data in a Dashboard format that would present the data as information that can be viewed at a glance. The first level of the PQTS Dashboard contained what was deemed by Amway to be the most important issues regarding the product and arranged them as shown in Figure 3. COSTS CUSTOMER RELIABILITY Income & Projections Warranty Costs & Projections Top 5 Warranty Costs by Subassembly Top 5 Telephone Hotline Complaints Top 5 Reasons for 7- day returns. Field Defect Rate (failure rate) for Product, (Observed and Projected). Top 5 observed defect rates by Subassembly Figure 3: PQTS Dashboard Presentation of Product Data. The top-level Dashboard was designed to provide three important categories of information divided into three columns: Costs, Customer and Reliability. The information was presented in a rich graphical format (much like a USA RF 2000RM-089: Page 3 RF Today snapshot for the product's health). At this top-level, one could (with a simple mouse-click) drill down on each item for more detailed graphs and analysis. The PQTS toplevel Dashboard design, a design that can be utilized by most product manufacturers, contained the following analysis presentations Cost Column - The Cost column was designed to present Financial issues and was comprised of three panels. The top panel contained Revenues for the product, which were presented based on a simple rolling average 1 for the last 18 months and forecasted out for the next four months. This data was obtained monthly from existing sales and forecasting databases, maintained by the financial departments at Amway. The second panel presented the warranty costs for the overall product, based on data obtained from THL and the repair center and again projected out for the next four months (with projections based on projected defect rates and projected sales). The last panel presented the warranty costs by subsassembly, again based on data obtained from THL and the repair center The Customer Column - The middle column of the PQTS Dashboard was designed to provide information on Customer issues and was comprised of two panels. The first panel showed the top 5 issues at THL. These issues represent what the customers experienced when they called the call center. They provide an early warning system as to symptoms experienced in the field. The second panel in the customer column presented the top 5 reasons for product Seven-Day Returns, or reasons a product was returned or failed only seven days after purchase. This was deemed critical to customer satisfaction and of high priority, since a customer that returned a product only seven days after purchase was more than likely to be dissastisfied and their loyalty perhaps lost The Reliability Column - Much thought went into the reliability column and the results shown. Again, in our effort to cater to the manager's perspective, we decided to use familiar terminology such as defect rates (percent failed under warranty). The top panel in this column showed rolling defect rates for the entire unit, and is a composite computation based on the defect rates of the subassemblies. The projections were also computed based on reliability predictions of the subassemblies (i.e., the percent expected to fail under warranty for each subcomponent) and then combined using system reliability methods and principles. When dealing with subassemblies and depending on the component, data granularity varied. In cases where the unit was sent to the repair center, serial numbers were availaible, which allowed the system to determine how long the 1 Other methods such as an EWMA can also be utilized. In most cases we chose methods that most people are familiar with. component was in that particular product and, in effect, obtain a time-to-failure for that component. Once times-tofailure for the components were obtained, analysis and prediction could be easily performed using standard life data analysis techniques (i.e., parametric techniques) with statistical distribution fits to the data, such as the Weibull distribution, and from there perform the necessary calculations and predictions. These methods are well documented and there is a wide variety of software availaible to perform such analyses, including market leading software and engines from ReliaSoft Corporation. Obviously, when times-to-failure data along with time-inoperation data was available, the analysis task was trivial. The difficulty arose when dealing with certain subassemblies of the product where minimal data was availaible. This occured for subassemblies of the product that were sent out by THL after the customer called the center. In most cases, no serial number for the unit was available or recorded, thus it was impossible to ascertain the age of the component. The only data we could obtain were the number of the assemblies that entered the per-time period and the number of reported failures under warranty per-time period. Our approach was to use a non-parametric (sales-invariant) approach to compute the defect rate. This simple non-parametric approach is given in the Appendix of this paper. The next figure, Figure 4, shows an actual screen shot of the PQTS Dashboard (with fictional data) as originally implemented at Amway (utilizing a client-server approach), along with an illustration of the drill down graphs that the user can invoke from the top-level. The Dashboard is updated through the PQTS system automatically. For this implementation, we chose weekly updates to the Dashboard. 4. TURNING ON THE PQTS Once all of the pieces of the system were designed, the next task was to turn the system on. To meet the computational requirements, all existing historical data was required by the system, so the first step was to load historical data into the PQTS database. During this process, many hours were spent cleaning polluted data that already existed. Once the database was populated with historical data, the system was turned on and we performed a historical analysis using the established analysis routines to see how the system performed and to validate our predictions against known history. During this debugging phase, we accounted for anomalies of program behavior and analysis results. Once adjustments were made to correct the anomalies, the program was backed up, reloaded and run again. Heavy emphasis was placed on thorough validation of every routine in the system, because we anticipated that management may take action, based on the results presented, the minute the system was made available. We spe
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks