A Systematic Review of What and How to Digitalize

This study is a systematic review to determine a conceptual framework for digital engineering, the objective being to select what and how to digitalize Department of Defense (DoD) acquisition processes, data, and decisions. The research question was, What are the best practices for Digitalization and Industry 4.0 to inform DoD acquisition programs? The study analyzed 20 peer-reviewed scholarly articles from the last five years, written by academics and practitioners from 19 countries, focused on Digitalization and Industry 4.0 methods and technologies. This study had five major findings: digitalization projects begin with strategic choices; digitalization is done within an ecosystem that constrains the technical options; digitalization requires a method of execution that assesses opportunity and limits risk; digitalization results in new processes using new data models that enable better decisions; feedback on that new business model will come internally from users and externally from customers.

Keywords: digital engineering, digitalization, Industry 4.0, framework, implementation, strategy

 

Determining a Digital Engineering Framework

DoD published its Digital Engineering (DE) Strategy in 2018. That was followed in 2020 by the Naval Digital Systems Engineering Transformation (DSET) Strategy. Both have the same five goals. The question has arisen of whether or not digital engineering (DE) is a new interdisciplinary branch of engineering, like systems engineering (SE) is a branch of industrial engineering. At this time, it has no distinct scientific principles applied to build particular things, no unique processes, methods or protocols; it is only a policy. However, the commercial world embraced Digitalization and Industry 4.0 out of necessity and has realized great opportunities that government can leverage.

 

Problem Statement

Executing acquisition plans in a predictable, fully resourced manner is challenging (Kraft, 2015). The National Defense Strategy states greater efficiency in procurement is a national priority (Department of Defense, 2018a). The National Defense Business Operations Plan declares that reforming the business processes is a key strategic goal (Department of Defense, 2018b). The resulting Digital Engineering Strategy admits that DoD lags industry on digital transformation solutions (Department of Defense, 2018c).

The DoD Digital Engineering Strategy has five Goals:

1. Formalize the development, integration and use of models to inform enterprise and program decision-making.
2. Provide enduring, authoritative source of truth.
3. Incorporate technological innovation to improve the engineering practice.
4. Establish a supporting infrastructure and environments to perform activities, collaborate, and communicate across stakeholders.
5. Transform the culture and workforce to adopt and support digital engineering across the lifecycle.

The DAU glossary defines digital engineering as “… an integrated digital approach that uses authoritative sources of systems’ data and models as a continuum across disciplines to support life cycle activities from concept through disposal (Defense Acquisition University, n.d.).” However, neither the goals nor the definition answers the critical questions of what or how to implement digitalization.

 

Rationale

Blackburn et al. (2018) was the report that formed the foundation of the DoD Digital Engineering Strategy, later restated and published in Bone et al. (2019). Neither articulated a conceptual framework for implementation. That is the rationale for this study.

Digital Engineering discussions often include unfamiliar and somewhat fluid terms. These may include Digital Thread (Kraft, 2020), Digital Twin (Madni et al., 2019), Digital Surrogate (Chakraborty et al., 2020), Electronic Prototype (Rieken et al., 2020), Authoritative Source of Truth (Kraft, 2019), Government Reference Architecture (Department of Defense, 2010), Open Architecture (Keller, 2021), and Agile Software (SAFe 5, n.d.). This study generally avoids them.

 

Objective

The objective of this study is to identify the current state of Digitalization practices and methods, and to identify a conceptual framework and notional integration of business processes to data products to structured decisions that would satisfy the goals of the DoD Digital Engineering Strategy. This study is a systematic review.

 

Potential Significance

Newly digitalized processes would be documented and constrained, with their triggers, inputs and outputs defined. Policy mandates imposed on a major defense acquisition program (MDAP) would be knowable and trackable over the life cycle of an acquisition program. Program decisions could be made with a common operating picture of the technical and managerial context around a given problem on a variety of levels, in a variety of functions, across the enterprise.

 

Theoretical Framework

General Systems Theory (von Bertalanffy, 1972) provides a framework that can bridge between systems engineering, business process management, and decision science. A biologist, Bertalanffy published his Theory of Organic Shape, “Gestalt”, in 1926. He published his view of organisms as physical systems in 1940, and ultimately the seminal General Systems Theory (von Bertalanffy, 1950). A modern conceptual framework adapted from Marcketti et al. (2009) is shown in Figure 1. General Systems Theory Conceptual Framework.

Figure 1

General Systems Theory Conceptual Framework

Note: Adapted from Marcketti et al. (2009)

 

Systems Engineering

The International Council on Systems Engineering (INCOSE) states that Systems Engineering emerged concurrently with Bertalanffy, at Bell Telephone Labs (INCOSE, n.d.). Hall (1962) defined a methodology for systems engineering to formalize and teach the principles of it. Kossiakoff & Sweet. (2002) cite several approaches, including one adopted by the Defense Acquisition University for instruction, shown in Figure 2. Systems Engineering Process. It bears note this engineering process is defined by inputs, a multi-step process, outputs, and feedback loops like Systems Theory.

Figure 2

Systems Engineering Process

Note: Adapted from Defense Acquisition University (2001, p. 6)

 

Business Process Management

Dumas et al. (2013) state that Business Process Management (BPM) is how work should be performed in order to ensure consistent outputs and to take advantage of improvement opportunities. This includes a circular lifecycle of process identification, monitoring, modeling, analysis, and redesign. Business Process Model Notation (BPMN) is the industry standard and is defined by the Object Management Group (OMG), as they do for Systems Modeling Language (SysML). Figure 3. Business Process Model and Notation Example shows the Microsoft Visio default process modeled with inputs, outputs, and feedback loop, also like Systems Theory.

Figure 3

Business Process Model and Notation Example

Note: Default example business process model in Microsoft Visio.

 

Decision Science

Davis et al., (2005) defined decision science (DS) as human decision making (why people decide) and the tools that assist it (decision support). Deitrick & Wentz (2015) discussed several theories in decision science. They showed that explicit and implicit uncertainty exist throughout the decision process, impacted in part by the changing interactions between steps in a process, the data, and the decision makers. It bears note they modeled a decision as a process with input, data, and outputs, as shown in Figure 4. Decision Process Diagram, again like Systems Theory.

Figure 4

Decision Process Diagram

Note: Adapted from Deitrick & Wentz (2015, p.548).

Synthesis

General Systems Theory (von Bertalanffy, 1972) is the parent theoretical lens for this study. It describes open systems as organisms that have input, throughput, output, react to external forces and have a feedback loop. Systems engineering as described by Kossiakoff & Sweet (2002) also has inputs, a process, output, feedback loops, and external forces. Deitrick & Wentz (2015) described decision processes with similar components. Inputs, output, feedback loops, and external forces are the archetypal objects in a BPMN process. Therefore, this framework is a convenient means to bridge these disciplines.

 

Methodology

 

General

This study is a systematic review of scholarly journals to provide the evidence-based current state of Digitalization and Industry 4.0 practice & methods. Petticrew & Roberts (2006) describe one method of performing a systematic review. Barends et al. (2017) offers a more streamlined approach for narrower questions that require rapid evidence assessments. The generally accepted method for more exhaustive review is the PRISMA, recently updated by Page et al. (2020). The PRISMA Checklist identified 27 items for consideration of inclusion in a systematic review. The research approach for this study was adapted from that, and is shown below in Figure 5. Overview of Research Approach.

Figure 5

Overview of Research Approach

Note. Adapted from PRISMA Checklist (Page et al., 2020)

 

Research Question

The research question was developed using the PICOC (Population, Intervention, Comparison, Outcome, Context) framework (Barends et al., 2017). Initially, the population was to be the financial technology industry but scholarly research on that segment proved too narrow, so the population was broadened to general business. The intervention was Digitalization and Industry 4.0 practices & methods, as they are being applied by business operations. Comparison would be to the existing DoD practices. The outcome was improved business processes to create better data products to make better decisions. The context was DoD acquisition programs. Application of PICOC is summarized in Table 1 and led to development of the following research question, What are the best practices for Digitalization and Industry 4.0 to inform DoD acquisition programs?

Table 1

PICOC Framework

Population	Who	Commercial Industry
Intervention	What or How	Digitalization and Industry 4.0 practices & methods
Comparison	Basis	Existing DoD processes, data products, and decisions
Outcome	Goal	Better planned and dynamic decision making, with lean processes.
Context	Circumstance	DoD acquisition program

Note: Adapted from Barends et al. (2017)

Eligibility

The inclusion criteria was restricted to peer-reviewed, scholarly journal articles. Only full-text articles were sought. For a fast-moving field, only articles from the last five years were accepted. While natural language processing translations provide extraordinary access, English language publications offer less risk of miscommunication. Sources were restricted to journals titled “business” or “management” that had published more than three articles on topic within the last three years, demonstrating sustained interest by the publisher, reviewers, authors and readers.

 

Information Sources

Google Scholar and ResearchGate were used to conduct initial scoping studies and find preliminary evidence on ‘digital engineering’ that address the current state of digitalization best practices, frameworks, strategies, and implementations, for process, or data, or decisions. The final search of UMGC OneSearch for evidence was reported using the PRISMA flow diagram (Moher et al., 2009).

 

Search Strategy

A set of Boolean search terms was developed with the assistance of UMGC librarians. Table 2 explains how they were derived. The final search set was: business AND technology, digitization OR digitalization, “best practice” OR framework, strateg* OR implement*, process OR data OR decision.

Table 2

Search Terms & Strings

Concept	Search Term
Technology Industry	business AND technology
Digitalization and Industry 4.0	digitization OR digitalization
practices & methods	“best practice” OR framework AND strateg* OR implement*
Existing DoD processes, data products, and decisions	process OR data OR decision

 

Quality Appraisal Tools

For the study to be of value the source articles must be of quality. Critically appraising data sources prevents information overload, ensures relevance and is a best practice for evidence-based management (Rousseau, 2006). Weight of Evidence (Gough, 2007) was used to assess the coherence, appropriateness and relevance of articles. TAPUPAS (Pawson et al., 2003) was used to evaluate the selected articles for transparency, rigor, ethics, and quality assessments for inclusion.

 

Data Collection

Article meta data extraction involved collecting information such as year of publication, research design, sample size, population (e.g., industry, type of employees), and type of study. Overall trustworthiness was judged. Core data extracted were the explicit findings, discussions, or conclusions of each article.

 

Synthesis

Collected data was recoded using the theoretical lens of Systems Theory, as relevant to the input, throughput, output, external force, or feedback of the open system. The categorized data was viewed for emergent themes. In the end, inputs clearly shaped strategic decisions, the throughput was the process of digitalization, external forces were part of the ecosystem or technical options, the output was a new business model, and feedback was provided by users and customers.

 

Findings

 

Input: Strategy Decisions

Blackburn et al. (2017) studied big data implications on research and development (R&D). They explored three important questions in the degree of change: how would big data refine, innovate, or transform R&D. Those mapped to impacts on strategy, people, technology and process.

Tortorella et al. (2021) explored the impact of Industry 4.0 on lean automation. They found process-oriented technologies had more impact on lean production than product and service technologies. This suggests a choice depending upon the desired target for impact.

Kristoffersen et al. (2020) proposed a Smart Circular Economy for manufacturing companies. This framework translates strategies into business analytics outcomes with digital technologies. It has three major dimensions that are relevant, each with degrees of implementation: Data Transformation, Resource Optimization, and Data Flow Process. This is shown in Figure 6.

Figure 6

Decision Tool Example

Note: Adapted from Kristoffersen et al. (p248, 2020)

Nosalska et al. (2019) found Industry 4.0 to be a multidimensional system with numerous terms, categories and variables across its dualistic nature of technical and business. They documented the most common Industry 4.0 design principles over several years of publication. The top recurrent principles were flexibility, real-time capability, decentralization, and modularity.

There were warnings as well. Donnelly (2019) cautioned to avoid over-digitization, while encouraging formal and informal knowledge exchange. This was a key strategic consideration given the tensions of digital transformation.

 

Throughput: Process

Almost uniformly, the focus was not only on the process of how to digitalize, but emphasizing that business process is the most important target of digitalization. Specifically, the value of digitalization is realized through the transformed underlying business processes (Antonucci et al., 2021). Further, lean production is most affected by process technology (Tortorella et al., 2021). Lastly, process is a critical component of Industry 4.0 implementation in supply chains (Ghadge et al., 2020).

Janiesch et al. (2019) used the 6-step design science research (DSR) process for the design of autonomous agents in the Internet of Things (IoT). They described the DSR steps as 1. Problem Identification, 2. Objectives of a Solution, 3. Design and Development, 4. Demonstration, 5. Evaluation, and 6. Conclusion. They applied this to a scenario of a cyber-physical system (CPS), a self-driving car.

Linde et al., (2021) evaluated opportunities for digital modeling, and identified traps to avoid. They found a structured approach for evaluating digital business models had three phases: assessing the opportunity, managing risks, and modeling the future. Concurrently, Linde et al., (2021) found several common traps that must be avoided. First, companies in a rush may not understand the customer value they are creating, and fail to satisfy customer needs. Second, not understanding the value delivery process and how the new digitalized process fits within the rest of the corporate context has risks. Last, companies may not understand the new profit formula and means of realizing revenue, simply trusting that digitalization will have made things better.

 

Output: New Business Model

One critical component of Industry 4.0 implementation is the new digital business model (Ghadge et al., 2020). Another recurring theme is that technical and business-related aspects are interlocked factors (Nosalska et al., 2019). While business model change is enabled by digitalization (Laïfi & Josserand, 2016), the new business model progresses with the business modeling process (Mattsson & Andersson, 2019).

Particularly important for a government procurement agency, Mattsson & Andersson (2019) determined that public-private interaction reveals tensions that drive BPM: structural, behavioral, and organizational. Mattsson & Andersson (2019) concluded a public actor in the complex public network is a much more complex implementation.

 

External Force: Ecosystem and Technical

There are many external forces to consider. Cong et al. (2021) identified partners as part of the IoT ecosystem. Correani et al. (2020) described a digital transformation ecosystem in which the data platform worked with customers and other players. Dethine et al. (2020) suggested that the ecosystem adapts over time, as did Ghadge et al. (2020). Garay-Rondero et al. (2020) stated the ecosystem is digital and physical. Gastaldi et al. (2018) considered the firm’s larger ecosystem important to a transformation. Linde et al. (2021) described the ecosystem in terms of relationships. Thus the ecosystem could be recoded as people, resources, organization and supply chain, and a digitalization project will have relationships with all of them.

Ivančić et al. (2019) identified seven main dimensions of digital transformation to include strategy, people, organization, customer, ecosystem, technology and innovation. In the framework shown below, Correani et al. (2020) included data sources, platform, and artificial intelligence (AI) as shown in Figure 7. Example Digital Implementation Framework. Ghadge et al. (2020) listed data sharing and management as critical.

Figure 7

Example Digital Implementation Framework

Note: Adapted from Correani et al. (p45, 2020).

Nosalska et al. (2019) listed many Industry 4.0 key technologies such as CPS, Big Data, IIOT, Cloud Computing/Cloud Manufacturing, Services/Product-as-a-Service/Internet of Services, and System/Architecture. Tortorella et al., (2021) determined that some Industry 4.0 technologies are positively correlated with Lean Production (LP) practices, but not all. The emerging technical factors appear to be the platform, technologies, and the data.

 

Feedback: Users & Customers

Correani et al. (2020) stated that customers could be given immediate feedback if part of the feedback loop. Donnelly (2019) noted opportunity to provide interpersonal feedback to clients and colleagues from digitalization. Garay-Rondero et al. (2020) found feedback to the value chain, and where physical processes affect computations and vice versa. Ghadge et al. (2020) found low customer service could be due to a backlog of feedback on demand. Mattsson & Andersson (2019) found small companies were quicker to adopt platform and content changes based on user feedback, and that customer feedback was important during development.

A Digital Community Infrastructure is digital sharing platforms to share designs and social networks and/or blogs to discuss ideas, questions and projects (Rieken et al., 2020). IoT technology and Artificial Intelligence of Things (AIoT) empowers the acceleration of digital transformation, and real-time collection of data from customers to monitor their conditions or assets to update risk. (Cong et al., 2021). Matzler et al. (2018) cautions that within the existing organization, implementation is highly unlikely to succeed, therefore organizational change is essential to success.

 

Certainty Assessment

Lewin et al. (2018) described a method of applying the Confidence in Evidence from Reviews of Qualitative research (CERQual) approach to identify the confidence in findings. CERQual is a framework to evaluate the methods, coherence, adequacy and relevance of the data used, effectively a self-report card that adds rigor and transparency.

 

Discussion

 

Developing the Conceptual Framework

The proposed framework consists of input, throughput, output, feedback and external forces. In this model, the inputs are the strategic choices to be made for implementation:

1. Degree of Change (Blackburn et al., 2017).

            Refine, or Innovate, or Transform.

2. Target for Lean Impact (Tortorella et al., 2021).

            Process, or Product & Service.

3. Degree of Circular Economy (Kristoffersen et al., 2020).

            Data Transformation, Resource Optimization, Data Flow Process.

4. Primary Design Principles: (Nosalska et al., 2019).

            Flexibility, Real-Time Capability, Decentralization, Modularity.

• Limit of Digitization (Donnelly, 2019).

These choices should be made with the intent of best achieving the DE Strategy goals of using models to inform decision making, creating an authoritative source of truth, technological innovation, supporting infrastructure and environments, and transforming the culture and workforce.

Throughput is the process of selecting processes, then digitalizing them. A best practice is to use 6-step design science research process (Janiesch et al., 2019). During execution, evaluate the opportunities and avoid the common traps (Linde et al., 2021). The method of digitally engineering is a process itself. That process is a mini-project plan for each business process under consideration of constraining the problem, setting goals, finding a solution, test, demonstration and deployment. Constraining the problem necessarily includes assessing the opportunity for process improvement, because some process improvements may not yield sufficient benefits to make the solutions cost effective or the margin for improvement may be too small. Lastly, as the process moves forward the team must continually assess risks. If the project understands the value the process creates (why we do it), the value delivery process (how we do it), and value realization (what we get out of it) those typical traps will be escaped.

The output is a new digitalized business model, where technical and business aspects are intertwined (Nosalska et al., 2019). The more the business model changes, the more the relationships with customers, the supply chain and internal users will change, and new opportunities will arise (Cong et al., 2021; Dethine et al., 2020; Garay-Rondero et al., 2020; Laïfi & Josserand, 2016). This the entire purpose of digitalization. While a DoD acquisition program does not realize revenue (they do not get ‘paid’ by the Pentagon for systems delivered), they certainly realize costs and deliver product. Having a well-documented business model, especially one that is digitally accessible will enable resource managers to see how their funds are being used, and will enable warfighters to see how their capabilities are being delivered. In addition, legislative authorizers and appropriators will be more easily persuaded to fund programs that are transparent to them.

Many external forces are at work, but they can be grouped into ecosystem constraints and technology opportunities. The ecosystem includes people, resources, organization and the supply chain, which the entity may or may not have control of (Cong et al., 2021; Correani et al., 2020; Dethine et al., 2020; Garay-Rondero et al., 2020; Gastaldi et al., 2018; Linde et al., 2021). Technical forces include the computing environment platforms, technologies, and data (Correani et al., 2020; Ghadge et al., 2020; Ivančić et al., 2019). Technologies do not equally benefit all desired outcomes (Tortorella et al., 2021), but several are key to Industry 4.0 application (Nosalska et al., 2019).

While the number of external forces at work could be infinite, the list must be constrained to provide meaningful decision points. The ecosystem forces were selected because their presence is necessary for success, even if they are constraints beyond the immediate control of the process owner. A process owner may not be able to change the people assigned, or may not have the authority to redirect resources, but both must be present in some limited quantity to succeed. A small operation may have complete control of its organization and culture, while many will be part of a larger organization with a set culture. Both can succeed, but the choices available are different. The digital supply chain for an office is crucial, and every office can identify who it depends on for data to execute owned processes, and what other offices consume data produced. Those players constitute the digital supply chain, and the participation of data suppliers and data consumers in digitally engineering a process is critical. The more they are integrated to the effort, the more opportunities may be exposed for further refinement, enhancing the recursive nature of digitalization.

Technical forces are more likely to be options than constraints. This is where people naturally gravitate to when considering digitalization. An office must consider its computing environment (platform), the technologies available (and affordable), and the data repositories it will require, create, and share. A small office may able to change its platforms, whereas a larger office inside a large organization may have no control, or limited choices within a menu. A major choice will be between on-premises (e.g. desktop) and off-premises (e.g. cloud) computing, and that choice could be driven by security considerations. Industry 4.0 technologies are centered on IoT, and there are many technologies associated with that. Application of technologies like AI, ML, NFC or Bluetooth may accelerate IoT deployment, or they may have limited impact efficacy; being judicious is important. The new business model will hinge on the new data model. Businesses can collect data they never use, or fail to relate or visualize the data they have in a usable manner. A vast repository of stove piped data serves nobody. Data that is interrelated cross-functionally is more likely to have meaning. Data should be collected, created and shared because it is required to execute a process or make a decision.

Feedback will come first from internal users, eventually from external customers, as well as the digital supply chain (Cong et al., 2021; Garay-Rondero et al., 2020; Rieken et al., 2020). Communication with them is essential to success and subsequent adjustments. Providing a means for users to provide faster feedback via a Digital Community Infrastructure will lead to changes in the organizational culture and increase likelihood of acceptance, as users feel they are an integral part of changing the way they do their work. If feedback from those users is not aggressively sought, there is a risk they will obstruct change or sabotage the project. Those users must include not only the performers of a given process, but the users of its results, the decision makers. The best process with poor visualizations may not improve outcomes.

Figure 8 illustrates the derived conceptual framework for digital engineering. DoD Goals feed project strategy decisions, the ecosystem constrains technology choices, process defines execution, a new business model delivers efficiencies, and feedback informs recursion.

Figure 8

Digital Engineering Conceptual Framework

The goal of digitalization is to arrive at a new set of processes that use a new set of data to achieve value. It is easy to see digitalization merely as a problem of new applications, or the introduction of Artificial Intelligence (AI) into processes, or new data models depending upon personal perspective or experience. However, none of those solutions alone will have sustained or meaningful impact. New models may be better, but may not result in better decisions if disconnected from a unified data model. A web services firm may be able to house petabytes of data for decades, but if it is not designed for people to use with their digital supply chain its customer value is limited. Using AI as support infrastructure to communicate with customers is common, but without integration with the business process it may not deliver value.

Entities have known they should digitalize, but did not know what or how to implement it. This framework provides a means to choose what projects to do and how to execute them in a balanced way.

Recommendations

Establish the implementation framework. Decide what external forces are strengths, weaknesses, opportunities or threats. To achieve DE goals, decide the strategy. Determine the desired degree of change, impact target, circular economy, design principles, and delimit the changeable processes. Model those processes as-is, to-be, and assess risk, as part of a disciplined project plan. Engineer a new data model on the proper platform with select technology, fed by new processes, and feeding others internally and externally. Communicate with the affected users, customers, and suppliers continuously, seeking failure early and rewarding good outcomes. Plan on necessary organizational changes. Monitor changes to the business model; prepare to adjust.

Limitations, Implications and Risk

This systematic review was streamlined for rapid completion. While the search was conducted on UMGC library databases, a significant number of results were excluded based solely on the title or abstract, and may be subject to selection bias (Nunan et al., 2017). The search terms may be subject to selection bias. Article content may have been ignored or highlighted based on the authors experience, injecting confirmation bias (Spencer et al., 2018). Digitalization is a rapidly evolving practice with hotly competing providers who need a proprietary edge, which resists scholarly publication.

Engineering is commonly defined as the application of scientific principles to build things. The branches and sub-branches are differentiated by using particular scientific principles to build particular things: this is what differentiates mechanical engineering from software engineering. This paper associates the principles of systems engineering, business process management and decision science for the purposes of describing a DE framework.

The Accreditation Board for Engineering and Technology (ABET) certifies more than 3,000 programs at over 600 US institutions with 75 engineering programs, yet none are ‘digital engineering.’ Digital Engineering is not currently a defined branch of engineering; therefore, few journal articles reference it. DE might be a sub-branch sibling of Systems Engineering if a distinct DE process is proposed and accepted.

Entities have known they should digitalize, but did not know what or how to implement it. This framework provides a means to choose those projects and execute them in a balanced way. This framework is being deployed in a case study integrated product team (IPT) this summer.

According to Matzler et al. (2018) the biggest risk is the existing organization; therefore, companies need a new culture, with great incentives to innovate and small penalties for mistakes. Failing faster, cheaper, will lead to success in digital transformation.

Conclusion

 

Answer to the Research Question

The research question was: What are the best practices for Digitalization and Industry 4.0 to inform DoD acquisition programs? The study found broadly that an implementation framework is necessary to properly apply Industry 4.0 technology to the digitalization of business processes. In the case of the DoD, the proposed framework shows Digital Engineering Strategy goals guide implementation decisions, the ecosystem constrains technology choices, an executable process is defined, the resulting new business model delivers efficiencies, and feedback informs recursion.

A conceptual framework was proposed that integrates these elements, as an evidence-based recommendation. A DoD agency that applied this method would be a cutting edge digitally engineered entity, capable of continuous digital evolution.

References

References marked with an asterisk (*) indicate studies included in the systematic review.

*Antonucci, Y. L., Fortune, A., & Kirchmer, M. (2021). An examination of associations between business process management capabilities and the benefits of digitalization: all capabilities are not equal. Business Process Management Journal, 27(1), 124–144. https://doi-org.ezproxy.umgc.edu/10.1108/BPMJ-02-2020-0079

Barends, E., Rousseau, D. M., & Briner, R. B. (Eds.). (2017). CEBMa Guidelines for Rapid Evidence Assessments in Management and Organizations, Version 1.0. Center for Evidence-Based Management, Amsterdam. https://www.cebma.org/wp-content/uploads/CEBMa-REA-Guideline.pdf

*Blackburn, M., Alexander, J., Legan, J. D., & Klabjan, D. (2017). Big Data and the Future of R&D Management: The rise of big data and big data analytics will have significant implications for R&D and innovation management in the next decade. Research Technology Management, 60(5), 43–50. https://doi-org.ezproxy.umgc.edu/10.1080/08956308.2017.1348135

Blackburn, M., Verma, D., Dillon-Merrill, R., Blake, R., Bone, M., Chell, B., Dove, R., Dzielski, J., Grogan, P., Hoffenson, S., Hole, E., Jones, R., Kruse, B., Pochiraju, K., Snyder, C., & Cloutier, R. (2018). Transforming Systems Engineering through Model-Centric Engineering, A013 Interim Technical Report SERC-2017-TR-111. Stevens Institute of Technology, Systems Engineering Research Center. https://apps.dtic.mil/sti/citations/AD1058354

Bone, M.A., Blackburn, M.R., Rhodes, D.H., Cohen, D.N., & Guerrero, J.A. (2019). Transforming systems engineering through digital engineering. Journal of Defense Modeling and Simulation: Applications, Methodology, Technology 2019, Vol. 16(4) 339–355. DOI: 10.1177/1548512917751873

*Cho, Y., & Lee, Y. (2016). The Use of IP Profiles in Selecting and Structuring R&D Alliances. Research Technology Management, 59(2), 18–27. https://doi-org.ezproxy.umgc.edu/10.1080/08956308.2015.1137191

Chakraborty, S., Adhikari, S., & Ganguli, R. (2021). The role of surrogate models in the development of digital twins of dynamic systems. Applied Mathematical Modelling, 90, 662–681. https://doi-org.ezproxy.umgc.edu/10.1016/j.apm.2020.09.037

*Cong, L.W., Li,B., & Zhang, Q.T. (2021). Internet of Things: Business Economics and Applications. Review of Business, 41(1), 15–29

*Correani, A., DeMassis, A., Frattini, F., Petruzzelli, A. M., & Natalicchio, A. (2020). Implementing a digital strategy: Learning from the experience of three digital transformation projects. California Management Review, 62(4), 37-56. https://doi.org/10.1177/0008125620934864

Davis, P. K., Kulick, J., & Egner, M. (2005). Implications of Modern Decision Science for Military Decision-support Systems. RAND.

Defense Acquisition University. (2001). Systems Engineering Fundamentals. https://acqnotes.com/wp-content/uploads/2017/07/DAU-Systems-Engineering-Fundamentals.pdf

Defense Acquisition University. (n.d.). Digital Engineering. Defense Acquisition University Glossary. Retrieved May 20, 2021, from https://www.dau.edu/glossary/Pages/GlossaryContent.aspx?itemid=27345#:~:text=An%20integrated%20digital%20approach%20that,activities%20from%20concept%20through%20disposal

Deitrick, S., & Wentz, E. A. (2015). Developing Implicit Uncertainty Visualization Methods Motivated by Theories in Decision Science. Annals of the Association of American Geographers, 105(3), 531–551. https://doi-org.ezproxy.umgc.edu/10.1080/00045608.2015.1012635

Department of Defense (2010). Reference Architecture Description. https://www.dragon1.com/downloads/DoD-ref-architecture.pdf

Department of Defense. (2018a). National Defense Strategy. https://www.defense.gov/Explore/Spotlight/National-Defense-Strategy

Department of Defense. (2018b). National Defense Business Operations Plan. https://cmo.defense.gov/Publications/NDBOP.aspx

Department of Defense. (2018c) Digital Engineering Strategy. https://ac.cto.mil/digital_engineering

*Dethine, B., Enjolras, M., & Monticolo, D. (2020). Digitalization and SMEs’ Export Management: Impacts on Resources and Capabilities. Technology Innovation Management Review, 10(4), 18–34. https://doi-org.ezproxy.umgc.edu/10.22215/timreview/1344

*Donnelly, R. (2019). Aligning knowledge sharing interventions with the promotion of firm success: The need for SHRM to balance tensions and challenges. Journal of Business Research, 94, 344–352. https://doi-org.ezproxy.umgc.edu/10.1016/j.jbusres.2018.02.007

Dumas, M., La Rosa, M., Mendling, J., & Reijers, H. (2013). Fundamentals of Business Process Management. Springer-Verlag Berlin Heidelberg. DOI 10.1007/978-3-642-33143-5_1

*Garay-Rondero, C. L., Martinez-Flores, J. L., Smith, N. R., Caballero Morales, S. O., & Aldrette-Malacara, A. (2020). Digital supply chain model in Industry 4.0. Journal of Manufacturing Technology Management, 31(5), 887–933. https://doi-org.ezproxy.umgc.edu/10.1108/JMTM-08-2018-0280

*Gastaldi, L., Appio, F. P., Corso, M., & Pistorio, A. (2018). Managing the exploration-exploitation paradox in healthcare : Three complementary paths to leverage on the digital transformation. Business Process Management Journal, 24(5), 1200–1234. https://doi-org.ezproxy.umgc.edu/10.1108/BPMJ-04-2017-0092

*Ghadge, A., Er Kara, M., Moradlou, H., & Goswami, M. (2020). The impact of Industry 4.0 implementation on supply chains. Journal of Manufacturing Technology Management, 31(4), 669–686. https://doi-org.ezproxy.umgc.edu/10.1108/JMTM-10-2019-0368

Gough, D. (2007). Weight of evidence: A framework for the appraisal of the quality and relevance of evidence. In J. Furlong & A. Oancea (Eds.), Applied and Practice-based Research. Special Edition of Research Papers in Education, 22(2), 213–228.doi: 10.1080/02671520701296189

Hall, A. (1962). A Methodology for Systems Engineering. D. Van Nostrand Company. New York.

INCOSE (n.d.). History of Systems Engineering, https://www.incose.org/about-systems-engineering/history-of-systems-engineering

*Ivančić, L., Vukšić, V.B. & Spremić, M. (2019). Mastering the Digital Transformation Process: Business Practices and Lessons Learned. Technology Innovation Management Review, 9(2), 36–50. https://doi-org.ezproxy.umgc.edu/10.22215/timreview/1217

*Janiesch, C., Fischer, M., Winkelmann, A., & Nentwich, V. (2019). Specifying autonomy in the Internet of Things: the autonomy model and notation. Information Systems & E-Business Management, 17(1), 159–194. https://doi-org.ezproxy.umgc.edu/10.1007/s10257-018-0379-x

Keller, J. (2021). The latest trends in power electronics: It’s not just about stand-alone devices anymore, as power systems designers tackle power density, thermal management, open-architecture standards, systems integration, and obsolescence management. Military & Aerospace Electronics, 32(4), 23–29.

Kossiakoff, A., & Sweet, W. (2003). Systems Engineering Principles and Practice. Wiley and Sons. ISBN 0-471-23443-5

Kraft, E.M. (2015, January 5-9). HPCMP CREATE-AV and the Air Force Digital Thread. [Paper presentation]. AIAA SciTech Forum Kissimmee, FL.

Kraft, E.M. (2019, January 7-9). Value-Creating Decision Analytics in a Lifecycle Digital Engineering Environment [Paper presentation]. AIAA SciTech Forum, San Diego, California. https://doi.org/10.2514/6.2019-1364

Kraft, E. M. (2020, January 5). Digital Engineering Enabled Systems Engineering Performance Measures. [Paper presentation]. AIAA Scitech 2020 Forum, Orlando, FL. DOI:10.2514/6.2020-0552

*Kristoffersen, E., Blomsma, F., Mikalef, P., & Li, J. (2020). The smart circular economy: A digital-enabled circular strategies framework for manufacturing companies. Journal of Business Research, 120, 241–261. https://doi-org.ezproxy.umgc.edu/10.1016/j.jbusres.2020.07.044

*Laïfi, A., & Josserand, E. (2016). Legitimation in practice: A new digital publishing business model. Journal of Business Research, 69(7), 2343–2352. https://doi-org.ezproxy.umgc.edu/10.1016/j.jbusres.2015.10.003

Lewin, S., Booth, A., Glenton, C., Munthe-Kaas, H., Rashidian, A., Wainwright, M., Bohren, M. A., Tunçalp, Ö, Colvin, C. J., Garside, R., Carlsen, B., Langlois, E. V., & Noyes, J. (2018). Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implementation Science, 13 (Suppl. 1), Article 2. https://doi.org/10.1186/s13012-017-0688-3

*Linde, L., Sjödin, D., Parida, V., & Gebauer, H. (2021). Evaluation of Digital Business Model Opportunities: A Framework for Avoiding Digitalization Traps. Research Technology Management, 64(1), 43–53. https://doi-org.ezproxy.umgc.edu/10.1080/08956308.2021.1842664

Madni, A. M., Madni, C. C., & Lucero, S. D. (2019). Leveraging digital twin technology in model-based systems engineering. Systems, 7(1), 7.

Marcketti,S., Niehm,L., & Fuloria, R. (2009). An Exploratory Study of Lifestyle Entrepreneurship and Its Relationship to Life Quality. Family and Consumer Sciences Research Journal. 34. 241 – 259. 10.1177/1077727X05283632.

*Mattsson, L.-G., & Andersson, P. (2019). Private-public interaction in public service innovation processes- business model challenges for a start-up EdTech firm. Journal of Business & Industrial Marketing, 34(5), 1106–1118. https://doi-org.ezproxy.umgc.edu/10.1108/JBIM-10-2018-0297

*Matzler, K., Friedrich von den Eichen, S., Anschober, M., & Kohler, T. (2018). The crusade of digital disruption. Journal of Business Strategy, 39(6), 13–20. https://doi-org.ezproxy.umgc.edu/10.1108/JBS-12-2017-0187

Moher, D., Liberati, A., Tetzlaff, J., Altman D. G., & The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med, 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097

*Nosalska, K., Piątek, Z.M., Mazurek, G. & Rządca, R. (2019). Industry 4.0: coherent definition framework with technological and organizational interdependencies. Journal of Manufacturing Technology Management, 31(5), 837–862. https://doi-org.ezproxy.umgc.edu/10.1108/JMTM-08-2018-0238

Nunan D., Bankhead C. & Aronson J.K. Selection bias. Catalogue Of Bias 2017. Retrieved from http://www.catalogofbias.org/biases/selection-bias/

Object Management Group. (2014). OMG Business Process Model and Notation (OMG BPMN™) Version 2.0.2. https://www.omg.org/spec/BPMN

Object Management Group. (2019). OMG Systems Modeling Language (OMG SysML™) Version 1.6. https://www.omg.org/spec/SysML/1.6

Page M.J., McKenzie J.E., Bossuyt P.M., Boutron I., Hoffmann T.C., & Mulrow C.D. (2020). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71

Pawson, R., Boaz, A., Grayson, L., Long, A., & Barnes, C. (2003). Types and quality of social care knowledge. Stage two: Towards the quality assessment of social care knowledge. London: Social Care Institute for Excellence.

Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden, MA: Blackwell Publishing Company.

*Rieken, F., Boehm, T., Heinzen, M., & Meboldt, M. (2020). Corporate makerspaces as innovation driver in companies: a literature review-based framework. Journal of Manufacturing Technology Management, 31(1), 91–123. https://doi-org.ezproxy.umgc.edu/10.1108/JMTM-03-2019-0098

Rousseau, D. M. (2006). Is there such as thing as “evidence based management”? Academy of Management Review, 31(2), 256-269. https://doi.org/10.5465/amr.2006.20208679

Scaled Agile Framework 5.0 (n.d.). https://www.scaledagileframework.com

Spencer E.A. & Heneghan C. Confirmation bias. Catalogue Of Bias 2018. Retrieved from www.catalogueofbiases.org/biases/confirmationbias

*Tortorella, G., Sawhney, R., Jurburg, D., de Paula, I. C., Tlapa, D., & Thurer, M. (2021). Towards the proposition of a Lean Automation framework: Integrating Industry 4.0 into Lean Production. Journal of Manufacturing Technology Management, 32(3), 593–620. https://doi-org.ezproxy.umgc.edu/10.1108/JMTM-01-2019-0032

von Bertalanffy, L. (1926). Untitled. Wilhelm Roux’ Archiv Fur Entwicklungsmechanik Der Organismen, 108(2), 413–416. https://doi-org.ezproxy.umgc.edu/10.1007/BF02080842

von Bertalanffy, L. (1940). Der Organismus als physikalisches System betrachtet. The Science of Nature, 28(33), 521–531.

von Bertalanffy, L. (1950). An outline of general system theory. British Journal for the Philosophy of Science, 1, 134–165. https://doi-org.ezproxy.umgc.edu/10.1093/bjps/I.2.134

von Bertalanffy, L. (1972). The History and Status of General Systems Theory. The Academy of Management Journal, 15(4), 407-426. doi:10.2307/255139.

 

ACKNOWLEDGEMENTS

This paper was inspired by the practical challenges encountered with sponsored work, and would not have been possible without the involvement of several colleagues: Dr. Timothy Davis, Dr. Matthew Tillman, and Dr. Bryan Herdlick. All three gave unique insights and identified particular applications. This research was unfunded. There is no known conflict of interest. This paper was required as part of doctoral course work at University of Maryland, Global Campus, under the tutelage of Dr. Walter McCollum, to whom I am sincerely grateful.

DATA TABLES

Appendix A
Acronyms

ABET	Accreditation Board for Engineering and Technology
AI	Artificial Intelligence
BPM	Business Process Management
BPMN	Business Process Management Notation
CPS	Cyber-Physical System
DAU	Defense Acquisition University
DE	Digital Engineering
DoD	Department of Defense
DSET	Digital Systems Engineering Transformation
DSR	Design Science Research
IIOT	Industrial Internet of Things
INCOSE	International Council on Systems Engineering
IPT	Integrated Product Team
LP	Lean Production
MDAP	Major Defense Acquisition Program
OASD/NII	Office of the Assistant Secretary of Defense, Networks and Information Integration
OMG	Object Management Group
PICOC	Population, Intervention, Comparison, Outcome, Context
PRISMA	Preferred Reporting Items for Systematic Reviews and Meta-Analyses
SE	Systems Engineering
SysML	Systems Modeling Language
TAPUPAS	Transparency, Accuracy, Purposivity, Utility, Propriety, Accessibility, Specificity
UMGC	University of Maryland, Global Campus
US	United States

Appendix B
Search Terms & Strings

Concept	Search Term
Technology Industry	business AND technology
Digitalization and Industry 4.0	digitization OR digitalization
practices & methods	“best practice” OR framework AND strateg* OR implement*
Existing DoD processes, data products, and decisions	process OR data OR decision

Appendix C
PRISMA Flow Chart

Appendix D
Data Extraction

Author & Year	Population	Design	Main Findings	Effect	Limitations	Level of Trust
Antonucci et al. (2021)	165 BPM professionals, across 4 continents	Quantitative cross-sectional study (survey)	1. The value of digitalization is realized through the transformed underlying business processes. 2. Mangers can target specific BPM capabilities to achieve targeted benefits. 3. The digitalization initiative needs to be established, and then the BPM capability can use it to create additional benefits. 4. Higher BPM capabilities lead to increased value through digitalization	Cited: n/a Views: n/a Internal consistency is good (α 5 0.919) and communalities analysis indicates a shared variance. Furthermore, the removal of any item would not result in a lower Cronbach’s α. A factor analysis was found to account for a good amount of variance (KMO50.929) and to be suitable (p < 0.001).	data for the dependent and independent variables were collected through a cross-sectional survey	High
Blackburn et al. (2017)	hundreds of interviews	literature review, interviews, and case studies	Framework: Strategy, People, Technology, Process integration 1. Insights provided by big data can inform both the kinds of innovations an organization pursues and the process it uses to produce new products and services. 2. Big data can enable more efficient and effective innovation by increasing the ability of researchers to obtain needed information, allowing faster iteration on designs and supporting virtual design and experimentation. 3. Big data developments will impact the future of their organizations	Cited: 26 Views: 486	None self-identified. Interviews with 22 attendees at workshop, literature review of nine case studies	Medium
Cho & Lee (2016)	Self-driven algorithmic analysis	Quantitative framework analysis	knowledge-intensive collaboration is taking the place of more traditional cost-driven collaboration intellectual property (IP) profiles may be more indicative of capability than ‘past performance’ Invented Anywhere alliance (patent overlap, industry overlap, citation count gap small) highest potential for synergy	Cited: 4 Views: 205	None self-identified. Did not leverage other works, entirely novel. Limited validation tests.	Medium
Cong et al. (2021)	2 case studies	survey of industry applications, several case studies	the advance of IoT technology and Artificial Intelligence of Things (AIoT) empowers the retail, industrialization, and finance industries to accelerate the pace of digital transformation (1) improved customer experience and (2) optimized supply chain operations InsureTech IoTs enable real-time collection of data from customers to monitor their conditions or assets (health, home, car, etc.) in order to update risk. Artificial Intelligence of Things (AIoT) = machine learning capabilities to the connectivity, signaling, and data-exchange capabilities	Cited: n/a Views: n/a	Article does not examine all possibilities provided by the potential and large scale of IoT-based alternative data. Small sample size.	High
Correani et al. (2020)	3 companies	case studies	Framework: Scope, Data Sources, Data Platform People, Partners, AI Information and Knowledge Processes and Procedures Transformed Activities, Tasks, and Services, Customers	Cited: 14 Views: 4,191	None self-identified. Small sample size.	High
Dethine et al. (2020)	21 papers	theoretical analysis, exploratory deductive approach	1. Digital facilitators impact the export, strategic and organizational management practices in a global way. 2. Operational and structuring practices are significantly less impacted. 3. e-business tools keep track of impacts and implications, and to develop the distinct capabilities needed to reach international markets.	Cited: 5 Views: 496	literature review, no original research	High
Donnelly (2019)	line and HR managers in S&ITS multinationals	388 completed surveys	1. it is important to encourage employees to divulge articulable tacit knowledge that may be valuable 2. avoid over-digitization and balance the formal and informal encouragement of knowledge exchange. 3. recognition and reward (R&R) played key roles in fostering knowledge sharing behaviors and the degree to which individuals engage in interpersonal exchanges and with KM initiatives	Cited: 18 Views: 133 Cronbach’s α 0.82, significance level of p≤0.05	influenced by the participants’ work environment and relations, and the methods of data collection	High
Garay-Rondero et al. (2020)	267 articles from 1989 to 2019	systematic literature review, in-depth screening of the papers	1. Industry 4.0 is expected to enable factories to: organize and control themselves autonomously, in a decentralized fashion and in real time 2. main features of the digitalized supply chains (DSC) are: accelerated, adaptable, smart, real-time data gathering, transparent, globally connected, scalable and clustered, breakthrough, inventive and sustainable. 3. Digital supply chain model with 6 dimensions	Cited: 24 Views: [-unknon-]	the exclusion criteria that were used to perform the literature review, which eliminated papers not written in English and which may also have ignored research in other languages, as well as terms other than those defined in the search protocol	High
Gastaldi et al. (2018)	107 semi-structured interviews 14 Italian hospitals	multiple case studies with embedded analysis	input-process-output (IPO) framework P1. Hospitals can solve the innovation-refinement paradox through the implementation of different digital transformation programs. P2. The implementation of digital transformation programs entails a time variant approach, where different time frames allow for different digital solutions. P3. In case of asset digitization, the efficiency frontier can be pushed further out by undertaking different innovation-refinement (re)configurations. P4. Through different digital transformation programs, hospitals accomplish different innovation and efficiency outcomes.	Cited: 8 Views: 96	study is exploratory, generalizability is limited.	Medium
Ghadge et al. (2020)	Not specified; quantitative model development	literature review, systems dynamics model	Industry 4.0 is predicted to bring new challenges and opportunities for future supply chains. The study discussed several implementation challenges and proposed a framework for an effective adaption and transition of the Industry 4.0 concept into supply chains. Ten critical components of Industry 4.0 implementation in supply chains	Cited: 25 Views: 321	results of the simulation model are utilized to develop a conceptual model. study did not use empirical data to test the SD models only a few parameters of Industry 4.0 (RFID and cloud) were considered in the model development and evaluation	Medium
Ivančić et al. (2019)	3 case studies	case studies	important factors for successful digital transformation are the ability of an organization to change and operational excellence in the integration of external digital services with internal IT support The seven main dimensions of the digital transformation are strategy, people, organization, customer, ecosystem, technology, and innovation digital strategy does not coexist as a separate document; rather, it is a coherent part of other strategies and a way of doing business	Cited: 0 Views: 248	limited number of companies that originated from the same country	High
Janiesch et al. (2019)	unspecified	literature review problem-centered Design Science Research (DSR) approach as suggested by Peffers et al. (2007) aligned with the seven guidelines by Hevner et al. (2004).	12 requirements for the design of autonomous agents in the IoT. Design science research (DSR) process generally includes six steps or activities identification of the problem, defining the research problem and justifying the value of a solution; definition of objectives for a solution; design and development of artifacts (constructs, models, methods, etc.);demonstration by using the artifact to solve the problem; evaluation of the solution, comparing the objectives and the actual observed results from the use of the artifact; and communication	Cited: 3 Views: 683	Modeling language may lack precision for some Domains must conduct more use cases to validate the suitability of notation Must research whether our construct of an agent can be integrated with existing modeling languages	Medium
Kristoffersen et al. (2020)	32 included papers	theory- and practice-based review academic sources, practice case study examples and “grey literature”	Smart CE framework that supports translating the circular strategies central to the goals of manufacturing companies – operational maturity could be linked to both an increase in the level of resource productivity and unburdening of human decision makers. 1. data transformation levels (5): connected resources, data, information, knowledge, wisdom 2. resource optimization capabilities (5): descriptive, diagnostic, discovery, predictive, prescriptive 3. data flow processes (3): data collection, data integration, data analysis The smart CE matrix>> A> Restore, Reduce, and Avoid: prevent excessive resource use and improve the inherent efficiency and circularity potential B> Reinvent: fully decouple value creation from the consumption of finite resources.	Cited: 18 Views: 276	requires further empirical research on the Smart CE research stream theoretical development process, the proposed framework should be empirically validated	High
Laïfi & Josserand (2016)	single entity	longitudinal case study and an intensive case study	1. progressively convincing targeted audiences in specific contexts of the legitimacy of their innovative business model 2. business model change enabled by digitalization occurs at the micro level – local arrangements can allow for the emergence of new value distribution arrangements	Cited: 16 Views: 353	None self-identified. Single entity case study.	Medium
Linde et al. (2021)	14 leading industrial companies that operate in mining, manufacturing, transportation, construction, energy, forestry, telecommunications, and pulp and paper	case studies	3 digitalization traps: – Trap 1: Not Understanding Customer Value – Trap 2: Not Understanding the Value Delivery Process – Trap 3: Not Understanding the Profit Formula 3 phase framework to evaluate digital business model opportunities: – Phase A: assessing digital opportunity value, – Phase B: managing digital opportunity risks, – Phase C: modeling digital opportunity financials	Cited: 2 Views: 1,442	None self-identified.	High
Mattsson & Andersson (2019)	single entity	longitudinal case study	A business model progresses with the business modeling process: public actors concurrently change their service provisioning models. Public-private interaction reveals tensions that drive business modeling: structural, behavioral, organizational public actor in the complex public network > much more complex implementation	Cited: 6 Views: 134	Need a closer look on the public service provision modeling Framework does not examine interaction between public actors.	High
Matzler et al. (2018)	N/A	editorial	3 levels of digitalization: products & services, processes & decisions, business models >>Within the existing organization, implementation is highly unlikely to succeed	Cited: 25 Views: 3,129	opinion	Low
Nosalska et al. (2019)	675 papers	systematic literature review	“Industry 4.0 is a concept of organizational and technological changes along with value chains integration and new business models development that are driven by customer needs and mass customization requirements and enabled by innovative technologies, connectivity and IT integration.” Industry 4.0 Design Principles: Flexibility, Real-Time Capability, Decentralization, Modularity, etc. Industry 4.0 Key technologies: CPS, Big Data, IIOT, Cloud Computing/Cloud Manufacturing, Services/Product-as-a-Service/Internet of Services, System/Architecture >>technical and business-related aspects are intertwined factors	Cited: 17 Views: 166	Further analysis of interdependencies between the technical and business aspects of Industry 4.0 to better understand its influence on the modern manufacturing sector. Value changes concerning cooperation and communication between manufacturers and clients who expect personalized products; an investigation of possibilities of using data from intelligent products to improve businesses’ efficiency, and the changes in the role of marketing.	High
Rieken et al. (2020)	116 relevant articles (61 journal articles and 55 conference papers)	critical review of 116 scientific articles	“digital community infrastructure” = digital sharing platforms to share digital designs and social networks and/or blogs to discuss ideas, questions and projects with other makers Facilitation of the makerspace is an important aspect for successful operations.	Cited: 8 Views: 286	The framework on CMSs is based on a critical review of makerspace literature and not on empirical research data.	High
Tortorella et al. (2021)	147 manufacturers that are undergoing a LP implementation aided by novel information and communication technologies from I4.0.	cross sector survey	1. Industry 4.0 (I4.0) technologies are positively correlated with Lean Production (LP) practices, but not all. *Process-oriented I4.0 technologies present the most synergistic relationship with practices from nine LP operational constructs 2. Lean Automation (LA) framework can potentially overcome traditional barriers and challenges of a LP implementation. **Lean Automation (LA) refers to the incorporation of digital automation technologies into the operationalization of Lean Production (LP) practices.	Cited: 9 Views: 206	Technologies applied in this study may present a limitation, since we used the adoption level of only ten digital automation technologies as a proxy for I4.0 implementation.	High

Appendix E
Weight of Evidence

Study	Author & Year	Coherence	Appropriateness	Relevance	Average
1	Antonucci et al. (2021)	3	3	3	3
2	Blackburn et al. (2017)	3	3	1	2.33
3	Cho & Lee (2016)	3	3	1	2.33
4	Cong et al. (2021)	3	3	3	3
5	Correani et al. (2020)	3	3	3	3
6	Dethine et al. (2020)	3	3	3	3
7	Donnelly (2019)	3	3	3	3
8	Garay-Rondero et al. (2020)	3	3	3	3
9	Gastaldi et al. (2018)	3	3	3	3
10	Ghadge et al. (2020)	3	2	2	2.33
11	Ivančić et al. (2019)	3	3	3	3
12	Janiesch et al. (2019)	3	3	1	2.33
13	Kristoffersen et al. (2020)	3	3	3	3
14	Laïfi & Josserand (2016)	3	3	2	2.67
15	Linde et al. (2021)	3	3	3	3
16	Mattsson & Andersson (2019)	3	3	3	3
17	Matzler et al. (2018)	0	3	3	2.00
18	Nosalska et al. (2019)	3	3	3	3
19	Rieken et al. (2020)	3	3	3	3
20	Tortorella et al. (2021)	3	3	3	3

Appendix F
TAPUPAS

Author & Year	Transparency	Accuracy	Purposivity	Utility	Proprietary	Accessibility	Specificity	SUM
Antonucci et al. (2021)	3	3	3	1	3	3	3	19
Blackburn et al. (2017)	1	3	3	3	3	2	3	18
Cho & Lee (2016)	1	2	2	3	3	2	3	16
Cong et al. (2021)	3	3	3	3	3	3	3	21
Correani et al. (2020)	3	3	3	3	3	3	3	21
Dethine et al. (2020)	3	3	3	3	3	3	3	21
Donnelly (2019)	3	3	3	3	3	3	3	21
Garay-Rondero et al. (2020)	3	3	3	3	3	3	3	21
Gastaldi et al. (2018)	3	3	2	2	3	3	3	19
Ghadge et al. (2020)	2	2	2	3	3	3	3	18
Ivančić et al. (2019)	3	1	3	1	3	3	3	17
Janiesch et al. (2019)	1	2	3	2	3	3	3	17
Kristoffersen et al. (2020)	3	3	3	3	3	3	3	21
Laïfi & Josserand (2016)	3	3	2	1	3	3	3	18
Linde et al. (2021)	3	3	3	3	3	3	3	21
Mattsson & Andersson (2019)	3	3	3	3	3	3	3	21
Matzler et al. (2018)	3	3	3	1	3	3	3	19
Nosalska et al. (2019)	3	3	3	3	3	3	3	21
Rieken et al. (2020)	3	3	3	3	3	3	3	21
Tortorella et al. (2021)	3	3	3	3	3	3	3	21

Appendix G
CERQual

Summary of review finding	Studies contributing	Methodological limitations	Coherence	Adequacy	Relevance	CERQual assessment	Explanation
Input: Strategy Decisions 1. Degree of Change (Refine, or Innovate, or Transform). 2. Lean Impact (Process, or Product & Service). 3. Smart Circular Economy (Data Transformation, Resource Optimization, Data Flow Process, Reuse). 4. Industry 4.0 Design Principles: (Flexibility, Real-Time Capability, Decentralization, Modularity). 5. Avoid over-digitization	2, 7, 13, 18, 20	Very minor concerns. 2 was based exclusively on interviews of 22 attendees at a workshop, plus literature survey of nine case studies. 7 was analysis of interviews and 388 online surveys. 13 was a theory and practice review. Academic sources but also practice case study examples and “grey literature.” 18 was a literature review of 52 publications. 20 was a cross sectional survey of 147 manufacturers.	No concerns	Minor concerns on degree of change. None were directly related to the new DoD DE policy, so did not directly address those goals.	No concerns	High confidence	All studies were transparent and rigorous. Each was from a specific perspective, focused on a particular aspect of upfront choices to be made. However, each reinforced the need for exactly such decisions before launching a digitalization project.
Throughput: Process The value of digitalization is realized through the transformed underlying business processes. Lean production most affected by process technology Process is a critical component of Industry 4.0 implementation in supply chains. Use 6-step design science research process. Evaluate opportunities, and avoid traps.	1, 10, 12, 15, 20	1 was a quantitative cross-sectional survey of 165 BPM professionals, lacking randomness. 10 was a literature review leading to a model, then derived analysis. 12 was a comprehensive literature review. 15 was a set of case studies. 20 was a cross sectional survey of 147 manufacturers.	No concerns.	No concerns. Combination of data is robust from over 300 surveys, literature reviews and case studies.	Minor concerns. 12 was focused on specifying autonomy in the Internet of Things, which is not directly on topic, but the methods used were directly relevant (DSR approach).	High confidence	All studies were transparent and rigorous, and were seeking to establish evidence from raw data.
Output: New Business Model Technical and business-related aspects are intertwined factors Business model change is enabled by digitalization. A business model progresses with the business modeling process. Public-private interaction reveals tensions that drive BPM: structural, behavioral, organizational. Public actor in the complex public network is a much more complex implementation.	14, 16, 18	14 was a longitudinal case study. 16 was a longitudinal case study. 18 was a literature review of 52 publications.	No concerns. 18 was particularly disciplined.	Minor concern. Two studies were a single case study, however, both were longitudinal.	Minor concerns. 18 was focused on library technology and digital publishing business models. The relevant finding was the recursive nature of BPM modeling.	Medium confidence	The study’s findings externally validated each other, even on limited raw data.
External Force: Technologies Industry 4.0 key technologies: CPS, Big Data, IIOT, Cloud Computing / Cloud Manufacturing, Services / Product-as-a-Service / Internet of Services, System / Architecture. Some Industry 4.0 technologies are positively correlated with Lean Production practices, but not all.	18, 20	18 was a literature review of 52 publications.   20 was a cross sectional survey of 147 manufacturers.	No concerns. 18 was particularly disciplined.	Minor concerns. With only 2 studies directly contributing to findings, this was the least broadly supported.	No concerns	High confidence	This key finding was focused on only one of the external forces, and the evidence serves more as an exemplar than complete set.
Feedback: Users & Customers “Digital Community Infrastructure” = digital sharing platforms to share digital designs and social networks and/or blogs to discuss ideas, questions and projects. IoT technology and Artificial Intelligence of Things (AIoT) empowers the acceleration of digital transformation, and real-time collection of data from customers to monitor their conditions or assets to update risk. Within the existing organization, implementation is unlikely to succeed.	4, 17, 19	4 was two case studies. 17 was a magazine format research question. 19 was a critical review of 116 scientific articles.	No concerns	Minor concerns. 4 only performed two case studies. 17 was not presented as a data driven study, but appears to have been revised into a narrative format.	Minor concerns. 19 was focused on corporate makerspaces, however the findings are generalizable.	High confidence	19 was particularly robust, and the findings are fairly general.