The technical feasibility process (TFP), or the stage in which medical device conceptual designs are developed prior to initiating the formal design control process, has long been a subject seldom discussed. Yet, it is a critical stage of the overall development process and when carried out inefficiently or inadequately can lead to significant challenges downstream. The primary objective of the TFP is to create conceptual designs that encourage an organization to support the risk-based business decision of initiating the design control process. Often this objective must also be achieved under organizational pressure to quickly deliver a design of sufficient quality and performance to attract investors, and/or receive internal endorsement to initiate the design control process.
However, it is equally important to prevent a rushed, poorly assessed, or poorly understood design from being “tossed over the wall” to the development team. For this reason, sufficient evidence and information must be provided to the organizational decision makers, and preferably this evidence should be documented. Thus, an essential objective of the TFP is ensuring the team sufficiently documents evidence gathered and knowledge gained, to aide in the overall assessment of designs. Throughout this process it is also ideal to maintain a lean and agile project team that can quickly change direction as the design evolves. Therefore, a major challenge during this process becomes achieving the objective of gathering sufficient evidence and knowledge without over-burdening the project team so they can remain lean and agile.
Device manufacturers may prioritize project team efficiency/productivity with the “good intentions” of not compromising the quality/level of detail of the supporting documentation. This juggling act may work, but it may also work to the detriment of the project. PLT contends that to balance these factors (efficiency/productivity vs proper documentation) successfully, an organization needs to get the timing right when addressing the following questions:
1. “When is the optimal time to begin compiling documentation describing the new device?”
2. “When is the optimal time to initiate the design control process?”
The short answer to both questions is “it depends” as these are not trivial questions to answer. There are multiple factors at play, and every organization developing a new medical device will have a different optimal answer. In this paper we provide device manufacturers with recommendations for working towards finding their own optimal answer. Big picture, question 1 leads to recommendations for best practices from an engineering standpoint, and question 2 leads to recommendations for best practices from a quality standpoint. Interspersed throughout these recommendations will be commentary on how companies of different sizes and maturity can address these questions. To separate engineering and quality best practices, we have split these recommendations into two separate articles.
“When is the optimal time to begin compiling documentation describing the new device?”
It is important to note that the documentation generated during the TFP covers components, subsystems, and algorithms as they evolve. The documentation should not address the complete device at this time to avoiding muddying the waters, and confusing team members who may think this data is acceptable for use in verification activities. Further, this documentation activity should not be thought of as an exercise in preparing to document a finished design before the design control process has been initiated.
Technical Feasibility Process Objectives
One goal of the TFP is to assess the performance of components, subsystems, or algorithms for the purpose of reducing the risk of integrating with the entire device once the design control process is initiated. To ensure a rigorous assessment, the project team should begin by establish a plan for completing this work. A design and development plan level of detail is not necessary, but a scope, strategy, and list of designs to be tested is sufficient.
Once a plan has been established, a recommended engineering best practice is to efficiently gather as much data as possible about designs as they evolve. This means characterizing designs (through testing) and having a comprehensive understanding of its operating principles. Experience has shown that experimental data demonstrating a design does NOT work as intended is as equally valuable as the data demonstrating a design does work as intended. This data can remind the team why past decisions were made, to provide the project team with direction when a design change is needed, and to educate new team members.
To best provide evidence for the overall assessment of designs during the TFP, engineering characterization experiments should be documented in characterization test reports. They should include sufficient detail to ensure they can be repeated but remain relatively concise including data and focusing on important observations, conclusions, any gaps in knowledge or performance and recommended next steps to address. Note - these documents are not meant to be used for verification purposes, rather to help reduce risk during decision making and increase organizational confidence in these designs. Design descriptions outline the purpose of a design and its operating principles. They should add context to the engineering characterization data. Design descriptions are best drafted as memos or in a team member’s logbook. They should include enough information to provide a comprehensive understanding of the design and the underlying operating principles, while again remaining as brief and concise as possible.
Last, to provide further context to the design test results and descriptions, it is helpful that manufacturers implement version controls. This is the practice of documenting and tracking design versions for component, sub-assembly, or algorithm designs, and ensuring the version of these items are up revved as the design of each evolves. Version controls are a very important detail to consider and are often overlooked during the TFP. Version controls extend to experimental data as well. They help ensure that tests performed can be traced to the versions of designs tested, and they help highlight which designs have not been exposed to specific tests. Engineering characterization reports and design description memos or logbook summaries should reference the versions of the components, sub-assemblies, or algorithms they discuss. Reports and memos should also include a revision to help readers follow the evolution of these documents along with the designs they describe.
Culture
One often underappreciated benefit to documenting a device design earlier in the TFP is the positive effect on team culture. First, and particularly for a start-up, it helps introduce the team to the idea of meeting a specified level of quality and detail expected for engineering documentation (in contrast to the design itself, which may be immature during the TFP). This can be established by providing training and sample documentation, or by establishing work-instruction style guidance documents with details on document types, templates and expected content. Having documentation that team members can access and review will help align the team, minimize gaps in knowledge, and avoid delays and additional cost when assessing designs. Ensuring a project team is producing a quality of work that is of a sufficiently high level is an essential part of growing a team.
Pitfalls of getting timing wrong
There are some circumstances that teams should take care to avoid. Despite there being many positive reasons for establishing documentation, there are some drawbacks to getting the timing wrong for beginning to generate design documentation. Beginning too early in the TFP, when the design is still in a constant state of flux, can be overly burdensome and can lead to minor details being overly scrutinized. This can also lead to creative team members feeling overly constrained and a diminishing enthusiasm for exploring new ideas. To best align project teams, management needs to establish and communicate expectations for the TFP. Establishing the appropriate level of detail required for reports as the TFP progressed towards its goal is helpful for maximizing team efficiency. Also, early in the TFP, management should establish minimal requirements for formal planning and status reporting, as at this stage this work provides little value.
Beginning too late can also be problematic. In PLT’s experience, waiting too long to begin documenting a design is a problem observed among medical device manufacturers of all sizes. By starting too late, project teams may be lacking sufficient information or any documented evidence to justify design decisions. Insufficient or a lack of documented evidence can also leave both new and existing team members misaligned, potentially leading to challenges or inconsistencies during integration. Furthermore, in the event a design change is needed, a lack of evidence can leave teams rudderless, unsure in which direction to move the design, or which levers can best be leveraged to most impactfully improve the design. Overall, a team’s ability to increase an organization’s confidence around designs and reduce business- and project-risks is directly related to the availability of data and information.
PLT Recommendations
Five strategies we strongly recommend to teams beginning to document a design and produce supporting evidence or design descriptions during the TFP are:
1. Implement version control
Ensure that version control is implemented as early as possible within your organization. Track the versions of all components, sub-assemblies, assemblies, and software. Start with the very first designs produced and update these versions as they change. To help with aligning design versions with documentation, include a list of all components, sub-assemblies, and algorithms along with the versions tested in the characterization report when summarizing those experiments. This ensures design version traceability across all activities and can help identify which design versions were tested at which time.
2. Planning
The project team should establish a plan for their approach to assessing designs. Creating a plan that consists of a scope, a strategy or approach, and a list of all designs to be assessed is sufficient. Like any project, it is important that the plan is reviewed and updated as designs evolve, and the project team is diligently following the plan.
3. Capturing design descriptions
PLT often advocates to clients that it is an important engineering best practice to create design descriptions that are brief and concise but sufficiently detailed. This should include a sufficiently detailed explanation on the purpose of the device, and its underlying operating principles. It should also include a description of the established features and levers that can be leveraged for improving or modifying design performance as the project progresses. There should also be a summary of potential risks including potential harms or hazards as well as design features that can help mitigate these potential risks. Last, we suggest compiling this information in a concise design description memo with a revision, or in a team member’s logbook. The design description should include a reference to the latest version of the design being described and should be updated as needed to document any design changes.
4. Engineering characterization experiments and reports
A successful strategy PLT has implemented for creating concise and informative summaries of characterization experiments assessing new designs has been engineering characterization reports. We have used these characterization reports to describe the purpose, scope and methods used to test designs. This includes information regarding the external interfaces the design interacted with during testing, and any relevant interfaces that were not included in testing. The report should also include sufficient detail to allow team members to repeat the activity. There should be sections listing all data collected, analyzed, and a description of how it was analyzed and for what purpose. Last, the report should include a concise summary of observations, conclusions, and possible future work (if necessary). Each report should include a version, and reports should be stored and maintained to allow for future review.
Please note that engineering characterization experiments are not to be used to provide assurances on reliability, nor should teams test for this. Reliability is to be assessed during design control process.
5. When reviewing Documents
In general, for any documentation generated during the TFP, we recommend limiting reviewers to the project team only. This allows teams to generate a library of knowledge ensuring the information the team deemed necessary was included, while minimizing the burden of review and employing a less formal documentation method. This lower bar for approving documents should be clearly communicated by management to the organization to avoid any confusion and assure alignment. This is an acceptable best practice since documents generated during the TFP are not included (nor should they be) as part of the regulatory submission for a specific device.
Summary
These recommended strategies could benefit an organization of any size or level of maturity. They can reduce time spent searching for information, increase organizational confidence in designs, and overall reduce the risk of extra costs and development time as the project proceeds. It’s important to remember that as the project proceeds through the product development cycle, every design change becomes (approximately) an order of magnitude more expensive. Most importantly, these strategies can reduce the possibility of any surprises that could disappoint management and/or investors.
If any of these challenges sound familiar to you, or you need some general help with design controls or quality systems, please connect with us and we can determine how we can best provide assistance.
Comments