<h2>INTRODUCTION (THE TL;DR VERSION)</h2> <em>This is part 2 of our discussion on AI development and implementation. You can find part 1 <a href="https://appsilon.com/ai-pilot-purgatory-and-how-to-avoid-it/?utm_source=AppsilonBlog&amp;utm_campaign=aipppart2">here</a>. </em> AI "<strong>Pilot Purgatory</strong>" describes a functioning AI model that is not put to use within the organization for which it was built. Essentially, an AI model is in "<strong>purgatory</strong>" if it works properly, yet virtually no one in an organization is using it. This is a serious problem because AI models can <a href="https://appsilon.com/ai-pilot-purgatory-and-how-to-avoid-it/?utm_source=AppsilonBlog&amp;utm_campaign=aipppart2">significantly reduce costs</a> in businesses where they are implemented. Here are the TL;DR steps to avoid ending up in pilot purgatory: <ol><li>Start with a clear use case and a demonstrated need for AI.</li><li>Check assumptions early and often with users.</li><li>Keep in mind that productionisation of the solution will require taking care of many aspects (see below).</li><li>Start by taking care of only the core of the solution, the part that truly brings value, when building a PoC or MVP.</li><li>Carefully explain to users how the adoption of the solution will be beneficial for them, and show them that is not a threat.</li><li>Make sure that users receive sufficient training.</li><li>Get back to solving all of the technical requirements you’ve planned in step 3.</li><li>Listen carefully to feedback from users, and check user activity and usage patterns.</li></ol> We can split the root causes of unsuccessful AI projects into two categories. Let's call one category ‘<b>cultural</b>’ and the other ‘<b>technical</b>’. <h2>Cultural</h2> The first source of issues is connected with company culture and conducting a project - the problems that might arise here are typical for any change, like starting without a clear use case or sticking to a <a href="https://en.wikipedia.org/wiki/Waterfall_model">waterfall approach</a> (lack of flexibility). With all of the hype currently surrounding AI, it is often forgotten that (1) preparing and maintaining a solution brings a lot of challenges, and (2) there should be a legitimate use case for the company. AI and DS projects can be hard to estimate in a traditional sense, primarily in terms of cost and timeline. This often discourages management from getting involved. When I was working for the Big 4 Consultancies I was required to provide exact estimates of the workload needed to finish a project or a project module. The managers needed this to fill their spreadsheets, budgets, and reports – or they had already agreed to deadlines with clients. They were not very interested in whether such a prediction was going to be inaccurate. Of course having the final goal in mind and overall plan is crucial, but AI projects need to be built with the correct approach. Delivering such a project requires a learning path – about the data, client requirements, technical solutions, and so on. If you stick to the level of knowledge that you had at the beginning of the road, then the end result will be disappointing. Adjusting tasks and estimates along the route is a must. Nevertheless, it is not easy to mitigate the need to control every aspect of a project, thus many AI initiatives might be terminated before they even get started due to overall unpredictability. The goal for management is usually to deliver value for the client as quickly as possible. So the solution for non-agile management cases is to proceed with small, more predictable steps, starting with the <a href="https://en.wikipedia.org/wiki/Proof_of_concept">Proof of Concept (PoC)</a> and <a href="https://en.wikipedia.org/wiki/Minimum_viable_product">Minimum Viable Product (MVP)</a>. Such an approach will not only make management feel safer about the planning part but it will also quickly make a positive impression on the client. This allows for easier acquisition of resources for continued development. This is the path of maximum value and minimum wasted time… Further, the overall outcome in terms of benefit of AI and DS projects is often hard to estimate. There is a lot of research involved, hypotheses to be tested, and improvements to be implemented. Only rarely can it be accurately claimed upfront: “this project will take 65 man-days and the result will be a 13% increase in sales.” No. But this is the <b>beauty of data science</b>: you can be surprised by how it boosts your business. Some projects might have a minimal effect, and others might have a dramatic positive effect. That is why the PoC and MVP concepts are so crucial for the inception of a project. Nevertheless it is often difficult for management to accept investments with uncertain ROI. Due to the dynamic characteristic of AI projects, it is crucial to employ an agile workflow in close collaboration with the client to adjust quickly and deliver maximum value. The point is to organize the project in such a way that something useful and valuable is produced during each week of production. Working the other way around may be why many projects end up in purgatory: when after a few months or a couple of iterations there is still no value delivered by the AI, then the project may be shut down. Crucially, when an AI solution is implemented within a company, it might not be accepted by the employees like any other standard business change. As human beings, we generally prefer the status quo. With AI changes, as they are often misunderstood, adoption may be completely avoided if people are worried that the machines will “steal” their jobs. <b>That is why I prefer to speak about AI “augmentation,” not “automation":  the implemented solution simplifies the work, makes it faster, and brings new insights, but the human is still the crucial part of the decision making process.</b> In short: AI <em>amplifies</em> human expertise. Only dull, repetitive and low-stakes tasks are done automatically by the machines. When AI is presented in such a manner, as a helper, it can be accepted and implemented easier. <h2>Technical</h2> There is a second challenge with implementing AI solutions. Building a good model is one thing, but <b>productionisation </b>(<em>i.e.,</em> making the model useful for the business) is a different story. The whole workflow must be set correctly. From feeding and updating data, through the correct, tested, reliable model response (sometimes also the speed of the response is crucial!) to a bug-free user interface. The typical scenario involves integrating many of the already existing tools into a smooth pipeline. <b>Scalability</b> (<em>i.e., </em>dealing with many concurrent users) may also be an issue. Then there is a whole deployment strategy to be implemented: internal servers or cloud systems, which machines to use, how to assure that the programmatic environment is the same as that which is used locally, etc. Setting access rights and logging user activity, securing data, and acting on results and insights must also be taken care of. Not to mention the process of maintaining the code base, bug fixing, testing, releasing updates and setting a common environment for the developers. All of these aspects are crucial parts of the system and require a great deal of knowledge and experience to be properly delivered. However, you shouldn’t start a project by worrying about all of the above problems. First, focus on proving that the AI implementation is useful and can bring significant value to your organization. The worst thing that can be done is to prepare the whole infrastructure just to find out that no one is truly interested in the product. Then you'll surely end up in purgatory. Thanks for reading! For more, follow me on <a href="https://www.linkedin.com/in/marcin-dubel-b5a3b5a9/">Linkedin</a> and <a href="https://twitter.com/DubelMarcin">Twitter</a>. You can also check out my other articles: <a href="https://www.r-bloggers.com/forget-about-excel-use-these-r-shiny-packages-instead/">Forget about Excel, Use these R/Shiny Packages Instead</a> and <a href="https://appsilon.com/super-solutions-for-shiny-architecture-1-of-5-using-session-data/">Super Solutions for Shiny Architecture</a>. <h2><b>Follow Appsilon Data Science on Social Media</b></h2><ul><li>Follow<a href="https://twitter.com/appsilon"> @Appsilon</a> on Twitter</li><li>Follow Appsilon on<a href="https://www.linkedin.com/company/appsilon"> LinkedIn</a></li><li>Sign up for our company<a href="https://appsilon.com/blog/"> newsletter</a></li><li>Try out our R Shiny<a href="https://appsilon.com/opensource/"> open source</a> packages</li><li>Sign up for the AI for Good<a href="https://appsilon.com/ai-for-good/"> newsletter</a></li><li>We are hiring <a href="https://appsilon.com/careers/">software engineers</a></li></ul>