Garbage In, Garbage Out: How Poor Data Quality Clogs Machine Learning Training Pipeline 

What is common in the success stories of businesses as diverse as Amazon, Airbnb, and Kakao Bank. The answer is data and a leadership that was relentless in the pursuit of good data quality. In the digital age, good quality data is a key differentiator – an invaluable asset that gives organizations an edge over competitors who have not been as dogged about the same (data quality). As a result, they are burdened with substandard, scattered, duplicate, and inconsistent data, that weighs them down more heavily than iron shackles. In the world businesses are operating in today, the divide is not been the big and the small organizations but between organizations who have invested in improving their data quality and those who have not. 

A single rotten apple spoils the barrel!    

We have all heard of the story about how a single rotten apple spoils a barrel. It is more or less the same story when it comes to data. Unclean, unrefined, and flawed data does more harm than good. Gartner estimates that poor quality data costs an organization $ 15 million per year. Though survey after survey talks about monetary losses – unclean data or data that has not been refined impacts more than the bottom line – it prevents businesses from deriving actionable insights from their data, leads to poor quality decisions, and drives dissatisfaction among all the people who matter – partners, vendors, and regulatory authorities. We have also heard of several instances where poor data quality has quickly snowballed into a major issue- like a money-laundering scam leading to loss of reputation as well.

Today when a majority of organizations are leveraging the power of AI and machine learning tools and investing millions to stay ahead of the curve, bad data can be a reason for not meeting the ROI. While organizations pour money for AI and ML tools, it is constrained due to bad quality data.      

Bad data hurts the American economy  

The impact of bad data on the American economy is not trickle-down, rather it is a gigantic leak that is hard to plug. Collectively, the impact of bad decisions made from data that are flawed goes into millions and billions. Ollie East, Director, Advanced Analytics and Data Engineering, of the public accounting and consulting firm, Baker Tilly, says that bad data costs the American businesses about $3 trillion annually, and breeds bad decisions made from having data that is just incorrect, unclean, and ungoverned. 

Banks and FIs are no exception to the rule. In fact, because of the privacy and regulatory requirements – they stand to lose more due to bad data. Of the zebibytes of data (including dark data) in existence organization-wide today, organizations are capitalizing only a minuscule percentage. Banks and FIs can ensure that they do not lose business, revenue, and clientele on account of poor data quality. It only takes a bit of effort and strategic planning. Further, the phenomenal success of new-age technologies like AI and machine learning has changed the rules of the game and has enabled banks and FIs to fish value from even the dark data – if only they undertake a planned approach to data standardization, data consistency, and data verification, and ensure that is streamlined for use again and again. Organizations must also account for the new data that enters the workflows and pipelines and ensure that a suitable mechanism is in place to ensure that it is always clean and standardized.     

To reiterate, why lose on the competitive advantage? Here’s a look at how organizations – banks and FIs- can invoke the power of cleaner, structured, data to make their processes crisper, leaner and undeniably more efficient.  

Step 1: Pre-processing data – making data good for downstream processes        

Pre-processing of data is the first step in the journey towards cleaner and refined data. Considering that not many organizations today can claim that their data quality meets expectations – the Harvard Business Review  states: “only 3% of companies’ data meets basic quality standards ” – pre-processing of data is critical for the following reasons: 

• Identifying of what’s wrong with the organization’s data. What are the core issues? 
• As the data is more likely to be used again and again in workflows, processes and systems enterprise-wide, good quality data with the right encryptions minimizes conflict of interest and other such discrepancies.        
• Also, as most organizations are likely to be using some kind of AI and ML for their processes involving this data – it is better to get it in shape to reap the maximum benefits. 

Garbage In Garbage Out – The true potential of AI and ML can be leveraged only when data quality is good 

Today, data scientists and analysts spend more time pre-processing data for quality (fine-tuning it), than analyzing it for business and strategic insights. This iterative pre-processing of data even though extremely time-consuming is important because if organizations feed “bad or poor-quality unrefined data” into the AI model it will spew (to put it across literally) garbage. Garbage In, results in Garbage Out. To leverage the true potential of AI and ML, it is essential that the quality of data being fed into the machine-learning pipeline downstream is of high quality. 

There are of course other substantial benefits as well. One, when the data is cleaned at the point of capture or during entry, banks and FIs have a cleaner database for future use. For example, by preventing the entry of duplicates at the point of capture (either via manual or automated means), organizations are spared from doing menial and repetitive work. It is also relatively easy to build the training model once the data is refined and streamlined. And when banks and FI have a more dependable AI pipeline (thanks to cleaner data) they can gain valuable insights that give them a strategic advantage.    

Carrying out data quality checks 

For ensuring that their data is up-to-date and foolproof, there are several levels of checks or quality tests including the quick-fact checking of data against a universal known truth – such as the age field – in a dataset age filed cannot have a negative value nor can the name field be null. However, a quick-fact check is a basic check (tests only the data and not the metadata which is the source of extremely valuable information such as the origin of data, creator of data, etc.). Therefore, for a comprehensive test of data quality, holistic or historical analysis of datasets must be carried out where organizations test individual data for authenticity or compare them with historical records for validation.  

Manual testing: Herein, staff manually verifies the values for data types, length of characters, formats, etc. The manual verification of the data is not desirable as it is exceedingly time-consuming. It is also highly error-prone. Instead, there are options such as open-source projects and in some cases, coded solutions built in-house, but both are not as popular as automated data quality testing tools.  

Automated data quality testing tools: Using advanced algorithms, these tools invariably make it easier for organizations to test data quality in a fraction of the time that manual effort takes (using data matching techniques). However, as reiterated earlier, machines are as good as the training they receive. If unclean, flawed data is poured into the training pipeline, it clogs the machine and prevents it from giving the desired results.   

The machines have to be taught like humans to understand and manipulate data so that exceptions can be raised and only clean filtered data remain in the dataset. Organizations can gain intelligence from their data either through rules-based engines or machine learning systems. 

1. Rules-based system: Rules-based systems work on a set of strict rules that suggest “if” a certain criterion is met or not met, then what follows. Rules-based data quality testing tools allow organizations to validate datasets against custom-defined data quality requirements. Rule-based systems requiring less effort and is also less risky – false positives are not a concern. It is often asked if rules-based tools and processes are slowly becoming antiquated as banks and FIs deal with an explosion of data. Probably not. They are still a long way from going out of fashion. It still makes sense to use the rules-based approach where the risk of false positives is too high and hence only rules which ensure 100 percent accuracy can be implemented. 

2. Machine learning systems: A machine learning system simulates human intelligence. It learns from the data that it is given (training model). Like a child that learns from its parent, it picks the good, the bad, and the ugly. Hence businesses must be extremely careful at the onset itself. They cannot expect optimum results if they are not careful with the quality of the data used for training. When it comes to its learning capacity and potential, however, ML-based systems’ capacity is infinite.  

Though there are several ways for the machine to learn, supervised learning is the first step. Every time new data gets incorporated in the datasets, the machine learns. The element of continuous learning means that in time it would require minimum human interference – which is good as banks and FIs would like to engage their manpower in far more critical tasks. As machines interpret and categorize data using its historical antecedents, it becomes much smarter and indefinitely more capable than humans. 

In the realm of dark data 

Every day banks and FIs generate, process, and store humongous amounts of data or information assets. Unfortunately, much of this data (nearly 80%) remains in the dark. Banks and FIs rarely tap into it for business insights and for monetizing the business. However, machine learning systems can help organizations unearth value from dark data with minimum effort. Learning, in this case, begins with making data observations, finding patterns and eventually using it to make good strategic decisions. All based on historical evidence or previous examples. What the system simply does here is alert the supervisor (about the exception) and then process that information and learn – that is what continuous learning does long term. 

Data quality is a burning issue for most organizations 

“By 2022, 60% of organizations will leverage machine-learning-enabled data quality technology to reduce manual tasks for data quality improvement.” Gartner  

At Magic FinServ, we believe that high-quality data is what drives top and bottom-line growth. Data quality issues disrupt processes and result in escalated costs as it calls for investment in re-engineering, database processing, and customized data scrubbing. And more for getting data in shape. 

Organizations certainly wouldn’t want that as they are running short of time already. Knowing that manual testing of data quality is not an option – it is expensive and time-consuming, it is cost-effective and strategically sound to rely on a partner like Magic FinServ with years of expertise.

Ensuring quality data – the Magic FinServ way

Magic FinServ’s strategy to ensure high-quality data is centered around its key pillars or capabilities – people, in-depth knowledge of financial services and capital markets, robust partnerships (with the best-in-breed), and a unique knowledge center (in India) for development, implementation, upgrade, testing, and support. Our capabilities go a long way in addressing the key challenges enterprises face today related to their data quality, spiraling data management costs, and cost-effective data governance strategy with a well-defined roadmap for enhancing data quality. 

Spinning magic with AI and ML: Magic FinServ has machine learning based tools to optimize operational cost by using Al to automate exception management and decision making. We can deliver a savings of 30% – 70% in most cases. As a leading a digital technology services company for the financial services industry, we bring a rare combination of capital markets domain knowledge and new-age technology skills, enabling leading banks and FinTech’s to accelerate their growth.  

Cutting costs with cloud management services: We help organizations manage infrastructure costs offering end-to-end services to migrate (to cloud from enterprise), support and optimize your cloud environment.

Calling the experts: We can bring in business analysts, product owners, technology architects, data scientists, and process consultants at a short notice. Their insight in reference data, including asset classes, entities, benchmarks, corporate actions and pricing, brings value to the organization. Our consultants are well-versed in technology. Apart from traditional programming environments like Java and Microsoft stack, they are also well versed in data management technologies and databases like MongoDB, Redis Cache, MySQL, Oracle, Prometheus, Rocks dB, Postgres, and MS SQL Serve. 

Partnerships with the best: And last but not least, the strength of our partnerships with the best in the industry gives us an enviable edge. We have not only tied with multiple reference data providers to optimize costs and ensure quality, but have partnership with reputed organizations dealing with complex and intractable environments, multi-domains, covering hundreds of thousands of data sources, to help our clients create a robust data governance strategy and execution plan.

So that is how we contain costs and also ensure that the data quality is top notch. So why suffer losses due to poor data quality. 

Connect with us today by writing to us at mail@magicfinserv.com.

The Banking and Financial Services Sector and the fintechs supporting them have not been unaffected by the winds of change sweeping across the business landscape. But unlike the past, technology has proved to be an equalizer. Size is no longer a necessary condition for success. Today we have Challenger Organizations, typically SMEs and large institutions alike, competing on a level playing field, and whosoever simplifies their processes or automates first gains an edge. Examples of how Intuit, Square, and similar Challenger Organizations redefined the meaning of Customer experience are proof enough. Automation, elimination of repetitive manual tasks, and consolidation of redundant activities into fewer steps have played a crucial role in enhancing Straight Through Processing (STP). 

Straight Through Processing has hitherto been addressed by optimizing underlying applications, eliminating data silos, and integrating applications better. While process automation through RPA and similar technologies have helped optimize downstream processes, the manual effort was still significant due to the disparate data sources that could not be consolidated and integrated. It is this final boundary that is sought to be breached through the application of Emerging Technologies. With a holistic end-to-end straight-through processing (STP), banks and FIs have taken a quantum leap forward and what would have once taken days to accomplish now takes minutes. STP removes the need for manual intervention. The only time human intervention is “ ideally” required is during “exception handling ” or “exceptions processing”  – when the system sees something that is unusual and raises a red flag. The human annotator then sets about making the necessary changes.  

White whales for STP implementation or Taming the white whales (with STP)

STP implementation is ideal for processes that involve a lot of repetitive work. The costs of system integration (might) dissuade many smaller players from considering it. However, like Captain Ahab’s relentless pursuit of the White Whale – Moby Dick, digital transformation experts have relentlessly argued the case for STP implementation (without similar calamitous consequences) in some of the most tiresome and time-consuming processes like KYC, loans processing, foreign exchange transaction handling,  accounts payables amongst others. Organizations that must meet quality SLAs as part of the business agreement have much to lose if they do not innovate on the technology front. Humans are more liable to make errors –  attach a wrong document, classify a document incorrectly (highly probable if there are 100 odd classifications to choose from), or simply feed data incorrectly into the system. And in the event this takes place, the likelihood of not meeting the SLAs (resulting in client dissatisfaction) is high.

 Secondly, banks and FIs, manually administering processes like KYC, have no time for value-added activity (like sales and customer retention and experience) as they are busy meeting the deadlines. As manual labor is both slow and expensive, there is a considerable backlog. Let’s take the example of the KYC process – a  McKinsey report states that banks generally employ about 10 percent of the workforce in financial-crime-related activities. Now that is a lot in terms of labor cost. The report has also indicated that KYC reviews were often the costliest activities handled by the bank.

With STP, banks and the financial services sector can eliminate a lot of paperwork, many unnecessary checkpoints (translating into unnecessary headcount), manual data entry while ensuring that SLAs are met and invoices, KYC, onboarding, accounts payable and accounts receivable, and other such processes and document-oriented activities are conducted quickly and cost-efficiently and with relatively fewer margins of error. So that organizations can ensure higher levels of transparency and trust and a good customer experience.

Cost of quality   

Let’s be honest; even the best human keyers/classifiers make more errors or mistakes than machines. While an error percentage of 2 to 5 % might not seem much, if we apply the 1-10-100 rule for the “cost of quality” and take into account a million documents that are being classified and whose data is being extracted, 5 % does make a huge difference. Automatically that would translate into a lot of work that would require human intervention. For every error that could be prevented, the cost of rectifying it is 10 times more. Leaving an error unattended is costlier – 100 times more expensive than doing it right the first time. 

Machines, however, are more capable. A Mckinsey report states, “ Assuming that standardization and coding of rules are performed correctly, quality can be improved significantly (by a range of 15 to 40 percent, experience indicates). Manual errors are reduced, and the identification and documentation of risks are improved. Rework loops can be shortened as well, as “first time right” ratios and regulatory targets are met more quickly.”

And now comes the really tricky part, since 100 percent accuracy is still unthinkable even for machines. When we are talking about documents, and relevant data fields, an accuracy of 99 % at the character level does not ensure STP. If documents need validation from their human supervisors due to high error rates,  zero STP will ensue.  Here we’d need something more robust than RPA. With machine learning (ML) and advanced capture, it is possible to increase accuracy to validate data using advanced rules.  We’d need a system that constantly adjusts and optimizes data. So every time the system encounters a variance or an anomaly, it adapts ( taking help from the human-in-loop) and improvises, becoming better after each iteration.

We would also have to take into account the variance in straight-through processing when it comes to structured and unstructured documents. While a standardized document such as a 10k will enable higher levels of  STP, semi-standardized documents such as invoices and unstructured documents such as notes and agreements will allow lower levels of STP.  

Simplifying banking processes

Today the imperative for banks and FIs to simplify their processes is huge. Future growth today is dependent on the ease with which banks and FIs can conduct their business. There is no escaping that! Reiterating the need for simplification, Hessel Veerbek, partner strategy, KPMG Australia, writes about “how some banks and insurers have replaced key elements of their core systems and consolidated their ancillary systems to rationalize their IT estate, modernize their capabilities, reduce costs and, at the same time, provide the capabilities to adapt and evolve their business models to secure future growth.”

Banks need to simplify operation by a core banking system that takes care of processes like loans processing and accounts payable end-to-end. At the heart of a simplified and automated banking architecture is end-to-end STP, which can be a complex undertaking, but as leading banks have shown, it is worth the trouble as it boosts efficiency. The challenges to STP incorporation are really in the mindset of organizations as “complex processes, high-risk customers and non-standard accounts” are still excluded from the purview of STP. It’s typical for organizations to consider STP while conducting low-risk tasks such as KYC for low-risk accounts in the KYC process. However, as the Mckinsey report suggests, if applied with dexterity and foresight, STP can eventually be enabled for the high-risk segment of customers as well. But here, a simple rules-based approach will not suffice, and organizations would need to rely on data science to create a system that will ensure a reliable output.

Augmenting human labor  – when machine learning tools perform the task and optimize it as well

The question is not really about how STP reduces the need for manual intervention – it is about how it augments human skills. The time it takes a human to classify and extract information from documents is disproportionate to the gains. Considering that almost 100 % of all data entry tasks can be automated and results can be obtained in a fraction of the time,  it makes sense to invest in tools that would ensure end-to-end automation. With STP, banks and Fintechs can not only eliminate the need for manual keying and classification of data, but in time sophisticated machine learning tools can also eliminate the need to verify that data manually.        

Getting to zero-touch! 

For true STP, we want an error rate that is such that human intervention is not required. This STP is the percentage or number of documents that go through the system with zero-touch or human contact. If the error rate or adverse media reaction is high, every document would have to be reviewed. Hence organizations must work on increasing accuracy by leveraging the power of AI and ML tools. 

If we are talking about cost efficiency, the need for software that easily integrates with organization-wide legacy systems is also a  prerequisite.  

The automation success story is not only about STP. 

It is important to remember that the automation success story does not depend on STP alone; other factors like investment costs, capital performance, cycle time, ROI, headcount reduction also matter. While “customer satisfaction” and “experience “ are good to have, Net Present Value (NPV), cost efficiency, and headcount reductions matter a lot. After all, leaner, nimbler, and more efficient operations are what most organizations are after.

While considering STP, it is also essential to do some homework regarding the investment costs, the complexity of the process (number of data elements that must be extracted, variance in documents, etc. ), cycle time, and the headcount reduction, etc. 

Experience tells us that the shift from highly manual-oriented processes to STP is not easy. It requires massive levels of patience and commitment as it takes time to reach the desired levels of accuracy. The actual test of STP success for any process depends on determining with a high degree of precision if a task has been executed accurately or not. A high rate of error or human intervention results in zero STP.   

Regardless of the challenges underlined earlier, STP remains a significant milestone in any organization’s journey towards automation. Banks and FIs that have successfully implemented STP have reaped many visible benefits. With Straight Through Processing, banks and FIs can choose to re-direct their efforts towards customer experience and retention, as they now have the time and bandwidth. When banks and FIs automate invoices and payments, they pave the way for a happier customer and employee experience. 

The question today is not whether STP is the ultimate test for automation progression; the question today is whether organizations can afford to do without STP – considering the astronomical costs of processing files and increased competition. Magic FinServ, with its years of experience serving a diverse clientele set comprising some of the top American banks and Fintechs, is well acquainted with the opportunities and risks associated with process optimization and simplification using AI and ML. If organizations are not careful, the costs could escalate disproportionately and disrupt the drive towards digital transformation.   Magic FinServ helps you navigate uncharted waters by leveraging our understanding of the financial services business to re-engineer existing applications, design new platforms, and validate machine learning solutions to suit your business needs. To explore our solutions, reach out to us mail@magicfinserv.com

Enterprises have increasingly realized that they must implement AI to succeed as digital natives are fast outpacing the ones relying on monolithic architectures. However, lack of synchronization between downstream and upstream elements, failure to percolate the AI value and culture in the organization’s internal dynamics, unrealistic business goals, and lack of vision often means that the AI projects either get stuck in a rut or fail to achieve the desired outcomes. What seemed like a sure winner in the beginning soon becomes an albatross around one’s neck.

Mitigating the pitfalls with a well-drawn and comprehensive AI roadmap aligned to company needs  

According to a Databricks report, only one in three AI and predictive analytics projects are successful across enterprises. Most AI projects are time-taking – it takes six months to go from the concept stage to the production stage. Most executives admit that the inconsistencies in AI adoption and implementation stems from inconsistent data sets, silos, and lack of coordination between IT and management and data engineers and data scientists. Then there’s the human element that had to be taken into account as well. Reluctance to invest, lack of foresight, failure to make cultural changes are as much responsible for falling short of the AI targets as the technical aspects enumerated earlier.

This blog will consider both the technical and the human elements vital for conducting a successful AI journey. To mitigate any disappointment that could accrue later, enterprises must assess the risk appetite, ensure early wins, get the data strategy in place, drive real-time strategic actions, implement a model and framework that resonates with the organization’s philosophy while keeping in mind the human angle – ensuring responsible AI by minimizing bias.

Calculating the risk appetite – how far the organization is willing to go? 

Whether the aim is to enhance customer experience or increase productivity, organizations must be willing to do some soul searching and find out what they are seeking. What are the risks they are prepared to take? What is the future state of readiness/ AI maturity levels? And how optimistic are things at the ground level?  

From the utilitarian perspective, investing in a completely new paradigm of skills and resources which might or might not result in ROI (immediately) is debatable. However, calamities of a global scale like COVID-19 demand an increased level of preparedness. Businesses that cannot scale up quickly can become obsolete; therefore, building core competencies with AI makes sense. Automating processes mitigates the challenges of the unforeseeable future when operations cannot be reliant on manual effort alone. So even if it takes time to reach fruition, and all projects do not translate into the desired dividends, it is a risk many organizations willingly undertake.

There is a lot at stake for the leadership as well. Once AI is implemented, and organizations start to rely on AI/ML increasingly, the risks compound. Any miscalculation or misstep in the initial stages of AI/ML adoption could cause grievous damage to the business’s reputation and its business prospects. Therefore, leadership must gauge AI/ML risks.     

Importance of early wins – focussing on production rather than experimentation.  

Early wins are essential. It elicits hope across an organization. Let us illustrate this with an example from the healthcare sector – the ‘moon shot’ project. Launched in 2013 at the MD Anderson Cancer Centre, the ‘moon shot project’ objective was to diagnose and recommend treatment plans for certain forms of cancer using IBM’s Watson cognitive system. But as the costs spiraled, the project was put on hold. By 2017, “moon shot” had accumulated costs amounting to $62 million without being tested on patients. Enough to put the management on tenterhooks. But around the same time, other less ambitious projects using cognitive intelligence were showing remarkable results. Used for simple day-to-day activities like determining if the patient needed help with bills payment and making reservations, AI drove marketing and customer experience while relieving back-officer care managers from the daily grind. MD Anderson has since remained committed to the use of AI.

Most often, it makes sense to start with process optimization cases. When a business achieves an efficiency of even one percent or avoids downtime, it saves dollars – not counting the costs of workforce and machinery. It is relatively easy to calculate where and how we can ensure cost savings in existing business cases instead of exploring opportunities where new revenue can be driven, as illustrated by the MD Anderson Cancer Centre case study. As we already know how the processes operate, where the drawbacks are, it is easier to determine areas where AI and ML can be baked for easy wins. The data is also in a state of preparedness and requires less effort.

In the end, the organization will have to show results. They cannot experiment willy-nilly. It is the business impact that they are after. Hence the “concept of productionize” takes center stage. While high-tech and glamorous projects look good, these are best bracketed as “aspirational.” Instead, the low-hanging fruit that enables easy gains should be targeted first.

The leadership has a huge responsibility, and to prioritize production, they must work in tandem with IT.  Both should have the same identifiable business goals for business impact. 

Ensuring that a sound data strategy is in place – data is where the opportunity lies!

If AI applications process data a gazillion times faster than humans, it is because of the trained data models. Else, AI apps are ordinary software running on conventional code. It is these amazing data models trained to carry out a range of complex activities and embedding NLP, computer vision, etc., that makes AI super-proficient. As a result, the application or system can decipher the relevant text, extract data from images, generate natural language, and carry out a whole gamut of activities seamlessly. So if AI is the works, data is the heart.          

Optimizing data pool

Data is the quintessential nail in the absence of which all the effort devised for drafting an operating model for data and AI comes to naught. Data is the prime mover when it comes to devising an AI roadmap. For data to be an asset, it must be “findable, accessible, interoperable, and reusable”. If it exists in silos, data ceases to be an asset. It is also not helpful if it exists in different formats. It is then a source of dubiety and must be cleaned and formatted first. Without a unique identifier (UID), attached data can create confusion and overwrite. What the AI machinery needs is clean, formatted, and structured data that can easily be baked on existing systems. Data that can be built once and used in many use cases is fundamental to the concept of productized data assets.

It serves to undertake data due diligence or an exploratory data analysis (EDA). Find out where data exists, who is the owner, how it can be accessed, linkages to other data, how it can be retrieved, etc., before drawing out the roadmap. 

The kind of data defines the kind of machine learning model that can be applied, for example, for supervised machine learning models, data and labels are essential for enabling the algorithm to draw an inference about the patterns in the label, whereas unsupervised learning comes when data does not have labels. And transfer learning when the data that an existing machine learning model has learned is used to build a new use case.

Once the data has been extracted, it must be validated and analyzed, optimized, and enriched by integrating it with external data sources such as those existing online or in social media and to be fed into the data pipeline. A kind of extract, transform and load. However, if it is done manually, it could take ages and still be biased and error-prone. 

Drawing the data opportunity matrix to align business goals with data

Once the existing data has been sorted, find how it can be optimized for business by integrating it with data from external sources. For this purpose, an opportunity matrix, also known as the Ansoff matrix comes in handy. A two-by-two matrix that references new business and current business with the data subsets (internal and external), it aids the strategic planning process and helps executives, business leaders understand where they are in terms of data and how they would like to proceed further.   

Driving real-time strategic actions for maximum business impact using AI: Leadership matters 

Real-time strategic actions are important. For example, millennial banks and financial institutions must keep pace with customer expectations or else face consequences. By making the KYC process less painstaking with AI, banks and FinTechs can drive unexpected dividends. When the KYC is done manually, it is time taking. By the time the KYC is complete, the customer is frustrated. When AI and Machine Learning capabilities are applied to existing processes, organizations reduce manual effort and errors substantially. The costs of conducting the KYC are reduced as well. However, the biggest dividend or gain that organizations obtain is in the customer experience that rebounds once the timelines ( and human interaction) are reduced. That is like having the cake and eating it too!    

SAAS, on-prem, open-source code – finding out what is best!

If it is the efficiency and customer experience that an enterprise is after, SaaS works best. Hosted and maintained by a third party, it frees the business from hassles. However, if one wants complete control over data and must adhere to multiple compliance requirements, it is not a great idea. On-prem, on the other hand, offers more transparency and is suitable for back-end operations in a fintech company for fast-tracking processes such as reconciliations and AML/KYC. Though SaaS is feasible for organizations looking for quality and ease of application, open-source code produces better software. It also gives control and makes the organization feel empowered.          

Conclusion: AI is not a simple plug and play 

AI is not a simple plug-and-play. It is a paradigm shift and not everyone gets it right the first time. Multiple iterations are involved as models do not always give the desired returns. There are challenges like the diminishing value of data which would require organizations to broaden their scope and consider a wider data subset for maximizing accuracy.  

Notwithstanding the challenges, AI is a proven game-changer. From simplifying back-office operations to adding value to day-to-day activities, there is a lot that AI can deliver. Expectations, however, would have to be set beforehand. The transition from near-term value to closing in on long-term strategic goals would require foresight and a comprehensive AI roadmap. For more information on how your organization could use AI to drive a successful business strategy, write to us at  mail@magicfinserv.com to arrange a conversation with our AI Experts.     

2020-2021 marked a new epoch in the history of business. For the first time, a massive percentage of the workforce was working from home. While employees struggled to cope with the limitations of working virtually, artificial intelligence (AI) emerged as a reliable partner for enterprises worldwide. With AI, enterprises were assured that business processes were not disrupted due to the scarcity of labor and resources.  

Now that the worst seems over, there are more reasons than ever to invest in AI. AI has been an infallible ally for many organizations in 2020. It helped them meet deadlines and streamline internal operations while eliminating wasteful expenditure. It helped them cope with burgeoning workloads. The impact AI had on employee productivity was significant. By unfettering staff in back and middle offices from the cycle of mundane, repetitive, and tiresome tasks, AI-enabled the workforce to engage in high-value tasks. 

So even as employees return to the office in the coming days,  many organizations will continue to amplify their AI efforts. Wayne Butterfield, director of ISG Automation, a unit of global technology research and advisory firm ISG, attributes this new phenomenon to the powerful impact AI had last year. He says, “As the grip of the pandemic continues to affect the ability of the enterprise to operate, AI in many guises will become increasingly important as businesses seek to understand their COVID- affected data sets and continue to automate day-to-day tasks.”

Indeed, in the banking and financial sector, the benefits driven by AI in the past year were monumental. It ensured frictionless interactions, cut repetitive work by half, and reduced error, bias, and false positives – the result of human fallacies – significantly. What organizations got was a leaner and more streamlined, and efficient organization. So there is no question that the value driven by AI in domains like finance and banking, which rely heavily on processes, will only continue to grow in the years to come. 

Setting pace for innovation and change

The pandemic has redefined digital. With enterprises becoming more digitally connected than ever before, it is AI that helps them stay operational. As a report from the Insider indicates, there will be significant savings in the middle, back, and front office operations if AI is incorporated. Automation of middle-office tasks can lead to savings of $70 billion by 2025. The sum total of expected cost savings from AI applications is estimated at $447 billion by 2023. Of this, the front and middle office will account for $416 billion of the aggregate.  

That AI will set the pace for innovation and change in the banking and financial services sector is all but guaranteed. The shift towards digital had started earlier; the pandemic only accelerated the pace. So here are some of the key areas where Fintechs and banks are using AI :   

• Document Processing  
• Invoice processing
• Cyber Security
• Onboarding/KYC 

Document processing with AI 

Enterprises today are sitting on a data goldmine that comes from sources as diverse as enterprise applications, public/private data sets, and social media. However, data in its raw form is of no use. Data, whether it is in textual, pdfs, spreadsheets, have to be classified, segregated, summarized and converted into formats (JSON, etc.) that can be understood by machines and processes before they can be of use to the organization. 

Earlier, image recognition technologies such as OCR were used for document processing. However, their scope is limited given that organizations deal with humongous amounts of data in diverse formats including print, and handwritten, all of which are not recognizable with OCR. Document processing platforms have a distinct advantage over traditional recognition technologies such as OCR and ICR. The system is trained first using data sets, and a core knowledge base is created. In time the knowledge base expands, and the tool develops the ability to self-learn and recognize content and documents. This is achieved through the feedback or re-training loop mechanism under human supervision. Realizing that artificial intelligence, machine learning, natural language processing, and computer vision can play a pivotal role in document processing, organizations are increasingly relying on these to enhance the efficiency of many front and back-office processes.  

Invoice Processing and AI

Covid-19 has intensified the need for automated Accounts Payable processes. Organizations that were earlier relying on manual and legacy systems for invoice processing were caught off-guard as employees were forced to work from home. Ensuring timely delivery on payment approvals became a challenge due to archaic legacy practices and an increasing number of constraints. Then there was the question of enhanced visibility into outstanding payments. All this led to chaos in invoice processing. A lot of frayed tempers and missed deadlines.

A major chunk of invoice processing tasks is related to data entry. Finance and accounts personnel shift through data that comes from sources such as fax, paper, and e-mail. But a study on 1000 US workers reiterated that no one likes data entry. The survey indicated that a whopping 70 percent of the employees were okay if data entry and other such mundane tasks were automated. With automated invoice processing, it is possible to capture invoices from multiple channels. Identify and extract data (header and lines) using validation and rules. And best in time, with little human supervision, become super proficient in identifying relevant information. It can also do matching and coding.  Magic FinServ’s Machine Learning algorithm correctly determined General Ledger code to correctly tag the invoice against an appropriate charge code and finally, using RPA, was able to insert the code on the invoice.    

Banks and other financial services stand to gain a lot by automating invoice processing. 

• By automating invoice processing with artificial intelligence, organizations can make it easier for the finance staff and back-office team to concentrate on cash-generating processes instead of entering data as -a typical administration function. 
• Automating the accounts payable process for instance, can help the finance teams focus on tasks that generate growth and opportunities. 
• An automated invoice processing provides enhanced visibility into payments and approvals.
• It speeds up the invoice processing cycle considerably as a result; there are no irate vendors
• It makes it easier to search and retrieve invoices.      

Cyber Security and AI

Cybersecurity has become a prime concern with the enterprises increasing preference for cloud and virtualization. Cybersecurity concerns became graver during Covid-19 as the workforce, including software developing teams, started working from home. As third parties and vendors were involved in many processes as well, it became imperative for organizations to ensure extreme caution while working in virtualized environments. Experiences from the past have taught us that data breaches spell disaster for an organization’s reputation. We need to look no further than Panera bread and Uber to realize how simple code left in haste can alter the rules of the game. Hence a greater impetus for the shift left narrative where security is driven in the DevOps lifecycle instead of as an afterthought. The best recourse is to implement an AI-driven DevOps solution. With AI baked into the development lifecycle, organizations can accelerate the development lifecyc in the present and adapt to changes in the future with ease.

Onboarding/KYC and AI

One of the biggest challenges for banks is customer onboarding and KYC. In the course of the KYC, or onboarding banks have to handle thousands, sometimes even millions of documents. And if that were not enough, they also have to take account of exhaust data and the multiple compliances and regulatory standards. No wonder then that banks and financial institutions often fall short of meeting the deadlines. Last year, as the Covid-19 crisis loomed large, it was these tools powered with AI and enabled with machine learning that helped accelerate paperwork processes. These digitize documents and extract data from it. And as the tool evolves with time, it makes it easier for the organization to extract insights from it. 

Let us take the example of one prominent Insurtech company that approached Magic FinServ for the resolution of KYC challenges. The company wanted to reduce the time taken for conducting a KYC and for SLAs roll-out of new policies gained confidence and customer appreciation as Magic’s “soft template” based solution augmented by Artificial Intelligence provided them the results they wanted.  

Tipping point

Though banks and financial institutions were inclining towards the use of AI for making their processes robust, the tipping point was the pandemic. The pandemic made many realize that it was now or never. This is evident from the report by the management solutions provider OneStream. The report observed that the use of AI tools like machine learning had jumped from about 20% of enterprises in 2020 to nearly 60% in 2021. Surprisingly, analytics firms like FICO and Corinium that a majority of top executives (upwards 65%) do not know how AI works. 

At Magic FinServ, our endeavor is to ensure that the knowledge percolates enterprise-wide. Therefore, our implementation journey starts with a workshop wherein our team of AI engineers showcases the work they have done and then engages in an insightful session where they try to identify the areas where opportunities exist and the deterrents. Thereafter comes the discovery phase, where our team develops a prototype. Once the customer gives the go-ahead as they are confident about our abilities to meet expectations, we implement the AI model that integrates with the existing business environment. A successful implementation is not the end of the journey as we keep identifying new areas of opportunities so that true automation at scale can be achieved.     

Catering to Banks and FinTechs: Magic FinServ’s unique AI optimization framework    

At Magic FinServ, we have a unique AI Optimization framework that utilizes structured and unstructured data to build tailored solutions that reduce the need for human intervention. Our methodology powered by AI-ML-NLP and Computer vision provides 70% efficiency in front and middle office platforms and processes. Many of our AI applications for Tier 1 investment, FinTechs, Asset Managers, Hedge Funds, and InsuranceTech companies have driven bottom and top-line dividends for the businesses in question. We ensure that our custom-built applications integrate seamlessly into the existing systems and adhere to all regulatory compliance measures ensuring agility.