Garbage In, Garbage Out: How Poor Data Quality Clogs Machine Learning Training Pipeline 

What is common in the success stories of businesses as diverse as Amazon, Airbnb, and Kakao Bank. The answer is data and a leadership that was relentless in the pursuit of good data quality. In the digital age, good quality data is a key differentiator – an invaluable asset that gives organizations an edge over competitors who have not been as dogged about the same (data quality). As a result, they are burdened with substandard, scattered, duplicate, and inconsistent data, that weighs them down more heavily than iron shackles. In the world businesses are operating in today, the divide is not been the big and the small organizations but between organizations who have invested in improving their data quality and those who have not. 

A single rotten apple spoils the barrel!    

We have all heard of the story about how a single rotten apple spoils a barrel. It is more or less the same story when it comes to data. Unclean, unrefined, and flawed data does more harm than good. Gartner estimates that poor quality data costs an organization $ 15 million per year. Though survey after survey talks about monetary losses – unclean data or data that has not been refined impacts more than the bottom line – it prevents businesses from deriving actionable insights from their data, leads to poor quality decisions, and drives dissatisfaction among all the people who matter – partners, vendors, and regulatory authorities. We have also heard of several instances where poor data quality has quickly snowballed into a major issue- like a money-laundering scam leading to loss of reputation as well.

Today when a majority of organizations are leveraging the power of AI and machine learning tools and investing millions to stay ahead of the curve, bad data can be a reason for not meeting the ROI. While organizations pour money for AI and ML tools, it is constrained due to bad quality data.      

Bad data hurts the American economy  

The impact of bad data on the American economy is not trickle-down, rather it is a gigantic leak that is hard to plug. Collectively, the impact of bad decisions made from data that are flawed goes into millions and billions. Ollie East, Director, Advanced Analytics and Data Engineering, of the public accounting and consulting firm, Baker Tilly, says that bad data costs the American businesses about $3 trillion annually, and breeds bad decisions made from having data that is just incorrect, unclean, and ungoverned. 

Banks and FIs are no exception to the rule. In fact, because of the privacy and regulatory requirements – they stand to lose more due to bad data. Of the zebibytes of data (including dark data) in existence organization-wide today, organizations are capitalizing only a minuscule percentage. Banks and FIs can ensure that they do not lose business, revenue, and clientele on account of poor data quality. It only takes a bit of effort and strategic planning. Further, the phenomenal success of new-age technologies like AI and machine learning has changed the rules of the game and has enabled banks and FIs to fish value from even the dark data – if only they undertake a planned approach to data standardization, data consistency, and data verification, and ensure that is streamlined for use again and again. Organizations must also account for the new data that enters the workflows and pipelines and ensure that a suitable mechanism is in place to ensure that it is always clean and standardized.     

To reiterate, why lose on the competitive advantage? Here’s a look at how organizations – banks and FIs- can invoke the power of cleaner, structured, data to make their processes crisper, leaner and undeniably more efficient.  

Step 1: Pre-processing data – making data good for downstream processes        

Pre-processing of data is the first step in the journey towards cleaner and refined data. Considering that not many organizations today can claim that their data quality meets expectations – the Harvard Business Review  states: “only 3% of companies’ data meets basic quality standards ” – pre-processing of data is critical for the following reasons: 

• Identifying of what’s wrong with the organization’s data. What are the core issues? 
• As the data is more likely to be used again and again in workflows, processes and systems enterprise-wide, good quality data with the right encryptions minimizes conflict of interest and other such discrepancies.        
• Also, as most organizations are likely to be using some kind of AI and ML for their processes involving this data – it is better to get it in shape to reap the maximum benefits. 

Garbage In Garbage Out – The true potential of AI and ML can be leveraged only when data quality is good 

Today, data scientists and analysts spend more time pre-processing data for quality (fine-tuning it), than analyzing it for business and strategic insights. This iterative pre-processing of data even though extremely time-consuming is important because if organizations feed “bad or poor-quality unrefined data” into the AI model it will spew (to put it across literally) garbage. Garbage In, results in Garbage Out. To leverage the true potential of AI and ML, it is essential that the quality of data being fed into the machine-learning pipeline downstream is of high quality. 

There are of course other substantial benefits as well. One, when the data is cleaned at the point of capture or during entry, banks and FIs have a cleaner database for future use. For example, by preventing the entry of duplicates at the point of capture (either via manual or automated means), organizations are spared from doing menial and repetitive work. It is also relatively easy to build the training model once the data is refined and streamlined. And when banks and FI have a more dependable AI pipeline (thanks to cleaner data) they can gain valuable insights that give them a strategic advantage.    

Carrying out data quality checks 

For ensuring that their data is up-to-date and foolproof, there are several levels of checks or quality tests including the quick-fact checking of data against a universal known truth – such as the age field – in a dataset age filed cannot have a negative value nor can the name field be null. However, a quick-fact check is a basic check (tests only the data and not the metadata which is the source of extremely valuable information such as the origin of data, creator of data, etc.). Therefore, for a comprehensive test of data quality, holistic or historical analysis of datasets must be carried out where organizations test individual data for authenticity or compare them with historical records for validation.  

Manual testing: Herein, staff manually verifies the values for data types, length of characters, formats, etc. The manual verification of the data is not desirable as it is exceedingly time-consuming. It is also highly error-prone. Instead, there are options such as open-source projects and in some cases, coded solutions built in-house, but both are not as popular as automated data quality testing tools.  

Automated data quality testing tools: Using advanced algorithms, these tools invariably make it easier for organizations to test data quality in a fraction of the time that manual effort takes (using data matching techniques). However, as reiterated earlier, machines are as good as the training they receive. If unclean, flawed data is poured into the training pipeline, it clogs the machine and prevents it from giving the desired results.   

The machines have to be taught like humans to understand and manipulate data so that exceptions can be raised and only clean filtered data remain in the dataset. Organizations can gain intelligence from their data either through rules-based engines or machine learning systems. 

1. Rules-based system: Rules-based systems work on a set of strict rules that suggest “if” a certain criterion is met or not met, then what follows. Rules-based data quality testing tools allow organizations to validate datasets against custom-defined data quality requirements. Rule-based systems requiring less effort and is also less risky – false positives are not a concern. It is often asked if rules-based tools and processes are slowly becoming antiquated as banks and FIs deal with an explosion of data. Probably not. They are still a long way from going out of fashion. It still makes sense to use the rules-based approach where the risk of false positives is too high and hence only rules which ensure 100 percent accuracy can be implemented. 

2. Machine learning systems: A machine learning system simulates human intelligence. It learns from the data that it is given (training model). Like a child that learns from its parent, it picks the good, the bad, and the ugly. Hence businesses must be extremely careful at the onset itself. They cannot expect optimum results if they are not careful with the quality of the data used for training. When it comes to its learning capacity and potential, however, ML-based systems’ capacity is infinite.  

Though there are several ways for the machine to learn, supervised learning is the first step. Every time new data gets incorporated in the datasets, the machine learns. The element of continuous learning means that in time it would require minimum human interference – which is good as banks and FIs would like to engage their manpower in far more critical tasks. As machines interpret and categorize data using its historical antecedents, it becomes much smarter and indefinitely more capable than humans. 

In the realm of dark data 

Every day banks and FIs generate, process, and store humongous amounts of data or information assets. Unfortunately, much of this data (nearly 80%) remains in the dark. Banks and FIs rarely tap into it for business insights and for monetizing the business. However, machine learning systems can help organizations unearth value from dark data with minimum effort. Learning, in this case, begins with making data observations, finding patterns and eventually using it to make good strategic decisions. All based on historical evidence or previous examples. What the system simply does here is alert the supervisor (about the exception) and then process that information and learn – that is what continuous learning does long term. 

Data quality is a burning issue for most organizations 

“By 2022, 60% of organizations will leverage machine-learning-enabled data quality technology to reduce manual tasks for data quality improvement.” Gartner  

At Magic FinServ, we believe that high-quality data is what drives top and bottom-line growth. Data quality issues disrupt processes and result in escalated costs as it calls for investment in re-engineering, database processing, and customized data scrubbing. And more for getting data in shape. 

Organizations certainly wouldn’t want that as they are running short of time already. Knowing that manual testing of data quality is not an option – it is expensive and time-consuming, it is cost-effective and strategically sound to rely on a partner like Magic FinServ with years of expertise.

Ensuring quality data – the Magic FinServ way

Magic FinServ’s strategy to ensure high-quality data is centered around its key pillars or capabilities – people, in-depth knowledge of financial services and capital markets, robust partnerships (with the best-in-breed), and a unique knowledge center (in India) for development, implementation, upgrade, testing, and support. Our capabilities go a long way in addressing the key challenges enterprises face today related to their data quality, spiraling data management costs, and cost-effective data governance strategy with a well-defined roadmap for enhancing data quality. 

Spinning magic with AI and ML: Magic FinServ has machine learning based tools to optimize operational cost by using Al to automate exception management and decision making. We can deliver a savings of 30% – 70% in most cases. As a leading a digital technology services company for the financial services industry, we bring a rare combination of capital markets domain knowledge and new-age technology skills, enabling leading banks and FinTech’s to accelerate their growth.  

Cutting costs with cloud management services: We help organizations manage infrastructure costs offering end-to-end services to migrate (to cloud from enterprise), support and optimize your cloud environment.

Calling the experts: We can bring in business analysts, product owners, technology architects, data scientists, and process consultants at a short notice. Their insight in reference data, including asset classes, entities, benchmarks, corporate actions and pricing, brings value to the organization. Our consultants are well-versed in technology. Apart from traditional programming environments like Java and Microsoft stack, they are also well versed in data management technologies and databases like MongoDB, Redis Cache, MySQL, Oracle, Prometheus, Rocks dB, Postgres, and MS SQL Serve. 

Partnerships with the best: And last but not least, the strength of our partnerships with the best in the industry gives us an enviable edge. We have not only tied with multiple reference data providers to optimize costs and ensure quality, but have partnership with reputed organizations dealing with complex and intractable environments, multi-domains, covering hundreds of thousands of data sources, to help our clients create a robust data governance strategy and execution plan.

So that is how we contain costs and also ensure that the data quality is top notch. So why suffer losses due to poor data quality. 

Connect with us today by writing to us at mail@magicfinserv.com.

The Banking and Financial Services Sector and the fintechs supporting them have not been unaffected by the winds of change sweeping across the business landscape. But unlike the past, technology has proved to be an equalizer. Size is no longer a necessary condition for success. Today we have Challenger Organizations, typically SMEs and large institutions alike, competing on a level playing field, and whosoever simplifies their processes or automates first gains an edge. Examples of how Intuit, Square, and similar Challenger Organizations redefined the meaning of Customer experience are proof enough. Automation, elimination of repetitive manual tasks, and consolidation of redundant activities into fewer steps have played a crucial role in enhancing Straight Through Processing (STP). 

Straight Through Processing has hitherto been addressed by optimizing underlying applications, eliminating data silos, and integrating applications better. While process automation through RPA and similar technologies have helped optimize downstream processes, the manual effort was still significant due to the disparate data sources that could not be consolidated and integrated. It is this final boundary that is sought to be breached through the application of Emerging Technologies. With a holistic end-to-end straight-through processing (STP), banks and FIs have taken a quantum leap forward and what would have once taken days to accomplish now takes minutes. STP removes the need for manual intervention. The only time human intervention is “ ideally” required is during “exception handling ” or “exceptions processing”  – when the system sees something that is unusual and raises a red flag. The human annotator then sets about making the necessary changes.  

White whales for STP implementation or Taming the white whales (with STP)

STP implementation is ideal for processes that involve a lot of repetitive work. The costs of system integration (might) dissuade many smaller players from considering it. However, like Captain Ahab’s relentless pursuit of the White Whale – Moby Dick, digital transformation experts have relentlessly argued the case for STP implementation (without similar calamitous consequences) in some of the most tiresome and time-consuming processes like KYC, loans processing, foreign exchange transaction handling,  accounts payables amongst others. Organizations that must meet quality SLAs as part of the business agreement have much to lose if they do not innovate on the technology front. Humans are more liable to make errors –  attach a wrong document, classify a document incorrectly (highly probable if there are 100 odd classifications to choose from), or simply feed data incorrectly into the system. And in the event this takes place, the likelihood of not meeting the SLAs (resulting in client dissatisfaction) is high.

 Secondly, banks and FIs, manually administering processes like KYC, have no time for value-added activity (like sales and customer retention and experience) as they are busy meeting the deadlines. As manual labor is both slow and expensive, there is a considerable backlog. Let’s take the example of the KYC process – a  McKinsey report states that banks generally employ about 10 percent of the workforce in financial-crime-related activities. Now that is a lot in terms of labor cost. The report has also indicated that KYC reviews were often the costliest activities handled by the bank.

With STP, banks and the financial services sector can eliminate a lot of paperwork, many unnecessary checkpoints (translating into unnecessary headcount), manual data entry while ensuring that SLAs are met and invoices, KYC, onboarding, accounts payable and accounts receivable, and other such processes and document-oriented activities are conducted quickly and cost-efficiently and with relatively fewer margins of error. So that organizations can ensure higher levels of transparency and trust and a good customer experience.

Cost of quality   

Let’s be honest; even the best human keyers/classifiers make more errors or mistakes than machines. While an error percentage of 2 to 5 % might not seem much, if we apply the 1-10-100 rule for the “cost of quality” and take into account a million documents that are being classified and whose data is being extracted, 5 % does make a huge difference. Automatically that would translate into a lot of work that would require human intervention. For every error that could be prevented, the cost of rectifying it is 10 times more. Leaving an error unattended is costlier – 100 times more expensive than doing it right the first time. 

Machines, however, are more capable. A Mckinsey report states, “ Assuming that standardization and coding of rules are performed correctly, quality can be improved significantly (by a range of 15 to 40 percent, experience indicates). Manual errors are reduced, and the identification and documentation of risks are improved. Rework loops can be shortened as well, as “first time right” ratios and regulatory targets are met more quickly.”

And now comes the really tricky part, since 100 percent accuracy is still unthinkable even for machines. When we are talking about documents, and relevant data fields, an accuracy of 99 % at the character level does not ensure STP. If documents need validation from their human supervisors due to high error rates,  zero STP will ensue.  Here we’d need something more robust than RPA. With machine learning (ML) and advanced capture, it is possible to increase accuracy to validate data using advanced rules.  We’d need a system that constantly adjusts and optimizes data. So every time the system encounters a variance or an anomaly, it adapts ( taking help from the human-in-loop) and improvises, becoming better after each iteration.

We would also have to take into account the variance in straight-through processing when it comes to structured and unstructured documents. While a standardized document such as a 10k will enable higher levels of  STP, semi-standardized documents such as invoices and unstructured documents such as notes and agreements will allow lower levels of STP.  

Simplifying banking processes

Today the imperative for banks and FIs to simplify their processes is huge. Future growth today is dependent on the ease with which banks and FIs can conduct their business. There is no escaping that! Reiterating the need for simplification, Hessel Veerbek, partner strategy, KPMG Australia, writes about “how some banks and insurers have replaced key elements of their core systems and consolidated their ancillary systems to rationalize their IT estate, modernize their capabilities, reduce costs and, at the same time, provide the capabilities to adapt and evolve their business models to secure future growth.”

Banks need to simplify operation by a core banking system that takes care of processes like loans processing and accounts payable end-to-end. At the heart of a simplified and automated banking architecture is end-to-end STP, which can be a complex undertaking, but as leading banks have shown, it is worth the trouble as it boosts efficiency. The challenges to STP incorporation are really in the mindset of organizations as “complex processes, high-risk customers and non-standard accounts” are still excluded from the purview of STP. It’s typical for organizations to consider STP while conducting low-risk tasks such as KYC for low-risk accounts in the KYC process. However, as the Mckinsey report suggests, if applied with dexterity and foresight, STP can eventually be enabled for the high-risk segment of customers as well. But here, a simple rules-based approach will not suffice, and organizations would need to rely on data science to create a system that will ensure a reliable output.

Augmenting human labor  – when machine learning tools perform the task and optimize it as well

The question is not really about how STP reduces the need for manual intervention – it is about how it augments human skills. The time it takes a human to classify and extract information from documents is disproportionate to the gains. Considering that almost 100 % of all data entry tasks can be automated and results can be obtained in a fraction of the time,  it makes sense to invest in tools that would ensure end-to-end automation. With STP, banks and Fintechs can not only eliminate the need for manual keying and classification of data, but in time sophisticated machine learning tools can also eliminate the need to verify that data manually.        

Getting to zero-touch! 

For true STP, we want an error rate that is such that human intervention is not required. This STP is the percentage or number of documents that go through the system with zero-touch or human contact. If the error rate or adverse media reaction is high, every document would have to be reviewed. Hence organizations must work on increasing accuracy by leveraging the power of AI and ML tools. 

If we are talking about cost efficiency, the need for software that easily integrates with organization-wide legacy systems is also a  prerequisite.  

The automation success story is not only about STP. 

It is important to remember that the automation success story does not depend on STP alone; other factors like investment costs, capital performance, cycle time, ROI, headcount reduction also matter. While “customer satisfaction” and “experience “ are good to have, Net Present Value (NPV), cost efficiency, and headcount reductions matter a lot. After all, leaner, nimbler, and more efficient operations are what most organizations are after.

While considering STP, it is also essential to do some homework regarding the investment costs, the complexity of the process (number of data elements that must be extracted, variance in documents, etc. ), cycle time, and the headcount reduction, etc. 

Experience tells us that the shift from highly manual-oriented processes to STP is not easy. It requires massive levels of patience and commitment as it takes time to reach the desired levels of accuracy. The actual test of STP success for any process depends on determining with a high degree of precision if a task has been executed accurately or not. A high rate of error or human intervention results in zero STP.   

Regardless of the challenges underlined earlier, STP remains a significant milestone in any organization’s journey towards automation. Banks and FIs that have successfully implemented STP have reaped many visible benefits. With Straight Through Processing, banks and FIs can choose to re-direct their efforts towards customer experience and retention, as they now have the time and bandwidth. When banks and FIs automate invoices and payments, they pave the way for a happier customer and employee experience. 

The question today is not whether STP is the ultimate test for automation progression; the question today is whether organizations can afford to do without STP – considering the astronomical costs of processing files and increased competition. Magic FinServ, with its years of experience serving a diverse clientele set comprising some of the top American banks and Fintechs, is well acquainted with the opportunities and risks associated with process optimization and simplification using AI and ML. If organizations are not careful, the costs could escalate disproportionately and disrupt the drive towards digital transformation.   Magic FinServ helps you navigate uncharted waters by leveraging our understanding of the financial services business to re-engineer existing applications, design new platforms, and validate machine learning solutions to suit your business needs. To explore our solutions, reach out to us mail@magicfinserv.com

Enterprises have increasingly realized that they must implement AI to succeed as digital natives are fast outpacing the ones relying on monolithic architectures. However, lack of synchronization between downstream and upstream elements, failure to percolate the AI value and culture in the organization’s internal dynamics, unrealistic business goals, and lack of vision often means that the AI projects either get stuck in a rut or fail to achieve the desired outcomes. What seemed like a sure winner in the beginning soon becomes an albatross around one’s neck.

Mitigating the pitfalls with a well-drawn and comprehensive AI roadmap aligned to company needs  

According to a Databricks report, only one in three AI and predictive analytics projects are successful across enterprises. Most AI projects are time-taking – it takes six months to go from the concept stage to the production stage. Most executives admit that the inconsistencies in AI adoption and implementation stems from inconsistent data sets, silos, and lack of coordination between IT and management and data engineers and data scientists. Then there’s the human element that had to be taken into account as well. Reluctance to invest, lack of foresight, failure to make cultural changes are as much responsible for falling short of the AI targets as the technical aspects enumerated earlier.

This blog will consider both the technical and the human elements vital for conducting a successful AI journey. To mitigate any disappointment that could accrue later, enterprises must assess the risk appetite, ensure early wins, get the data strategy in place, drive real-time strategic actions, implement a model and framework that resonates with the organization’s philosophy while keeping in mind the human angle – ensuring responsible AI by minimizing bias.

Calculating the risk appetite – how far the organization is willing to go? 

Whether the aim is to enhance customer experience or increase productivity, organizations must be willing to do some soul searching and find out what they are seeking. What are the risks they are prepared to take? What is the future state of readiness/ AI maturity levels? And how optimistic are things at the ground level?  

From the utilitarian perspective, investing in a completely new paradigm of skills and resources which might or might not result in ROI (immediately) is debatable. However, calamities of a global scale like COVID-19 demand an increased level of preparedness. Businesses that cannot scale up quickly can become obsolete; therefore, building core competencies with AI makes sense. Automating processes mitigates the challenges of the unforeseeable future when operations cannot be reliant on manual effort alone. So even if it takes time to reach fruition, and all projects do not translate into the desired dividends, it is a risk many organizations willingly undertake.

There is a lot at stake for the leadership as well. Once AI is implemented, and organizations start to rely on AI/ML increasingly, the risks compound. Any miscalculation or misstep in the initial stages of AI/ML adoption could cause grievous damage to the business’s reputation and its business prospects. Therefore, leadership must gauge AI/ML risks.     

Importance of early wins – focussing on production rather than experimentation.  

Early wins are essential. It elicits hope across an organization. Let us illustrate this with an example from the healthcare sector – the ‘moon shot’ project. Launched in 2013 at the MD Anderson Cancer Centre, the ‘moon shot project’ objective was to diagnose and recommend treatment plans for certain forms of cancer using IBM’s Watson cognitive system. But as the costs spiraled, the project was put on hold. By 2017, “moon shot” had accumulated costs amounting to $62 million without being tested on patients. Enough to put the management on tenterhooks. But around the same time, other less ambitious projects using cognitive intelligence were showing remarkable results. Used for simple day-to-day activities like determining if the patient needed help with bills payment and making reservations, AI drove marketing and customer experience while relieving back-officer care managers from the daily grind. MD Anderson has since remained committed to the use of AI.

Most often, it makes sense to start with process optimization cases. When a business achieves an efficiency of even one percent or avoids downtime, it saves dollars – not counting the costs of workforce and machinery. It is relatively easy to calculate where and how we can ensure cost savings in existing business cases instead of exploring opportunities where new revenue can be driven, as illustrated by the MD Anderson Cancer Centre case study. As we already know how the processes operate, where the drawbacks are, it is easier to determine areas where AI and ML can be baked for easy wins. The data is also in a state of preparedness and requires less effort.

In the end, the organization will have to show results. They cannot experiment willy-nilly. It is the business impact that they are after. Hence the “concept of productionize” takes center stage. While high-tech and glamorous projects look good, these are best bracketed as “aspirational.” Instead, the low-hanging fruit that enables easy gains should be targeted first.

The leadership has a huge responsibility, and to prioritize production, they must work in tandem with IT.  Both should have the same identifiable business goals for business impact. 

Ensuring that a sound data strategy is in place – data is where the opportunity lies!

If AI applications process data a gazillion times faster than humans, it is because of the trained data models. Else, AI apps are ordinary software running on conventional code. It is these amazing data models trained to carry out a range of complex activities and embedding NLP, computer vision, etc., that makes AI super-proficient. As a result, the application or system can decipher the relevant text, extract data from images, generate natural language, and carry out a whole gamut of activities seamlessly. So if AI is the works, data is the heart.          

Optimizing data pool

Data is the quintessential nail in the absence of which all the effort devised for drafting an operating model for data and AI comes to naught. Data is the prime mover when it comes to devising an AI roadmap. For data to be an asset, it must be “findable, accessible, interoperable, and reusable”. If it exists in silos, data ceases to be an asset. It is also not helpful if it exists in different formats. It is then a source of dubiety and must be cleaned and formatted first. Without a unique identifier (UID), attached data can create confusion and overwrite. What the AI machinery needs is clean, formatted, and structured data that can easily be baked on existing systems. Data that can be built once and used in many use cases is fundamental to the concept of productized data assets.

It serves to undertake data due diligence or an exploratory data analysis (EDA). Find out where data exists, who is the owner, how it can be accessed, linkages to other data, how it can be retrieved, etc., before drawing out the roadmap. 

The kind of data defines the kind of machine learning model that can be applied, for example, for supervised machine learning models, data and labels are essential for enabling the algorithm to draw an inference about the patterns in the label, whereas unsupervised learning comes when data does not have labels. And transfer learning when the data that an existing machine learning model has learned is used to build a new use case.

Once the data has been extracted, it must be validated and analyzed, optimized, and enriched by integrating it with external data sources such as those existing online or in social media and to be fed into the data pipeline. A kind of extract, transform and load. However, if it is done manually, it could take ages and still be biased and error-prone. 

Drawing the data opportunity matrix to align business goals with data

Once the existing data has been sorted, find how it can be optimized for business by integrating it with data from external sources. For this purpose, an opportunity matrix, also known as the Ansoff matrix comes in handy. A two-by-two matrix that references new business and current business with the data subsets (internal and external), it aids the strategic planning process and helps executives, business leaders understand where they are in terms of data and how they would like to proceed further.   

Driving real-time strategic actions for maximum business impact using AI: Leadership matters 

Real-time strategic actions are important. For example, millennial banks and financial institutions must keep pace with customer expectations or else face consequences. By making the KYC process less painstaking with AI, banks and FinTechs can drive unexpected dividends. When the KYC is done manually, it is time taking. By the time the KYC is complete, the customer is frustrated. When AI and Machine Learning capabilities are applied to existing processes, organizations reduce manual effort and errors substantially. The costs of conducting the KYC are reduced as well. However, the biggest dividend or gain that organizations obtain is in the customer experience that rebounds once the timelines ( and human interaction) are reduced. That is like having the cake and eating it too!    

SAAS, on-prem, open-source code – finding out what is best!

If it is the efficiency and customer experience that an enterprise is after, SaaS works best. Hosted and maintained by a third party, it frees the business from hassles. However, if one wants complete control over data and must adhere to multiple compliance requirements, it is not a great idea. On-prem, on the other hand, offers more transparency and is suitable for back-end operations in a fintech company for fast-tracking processes such as reconciliations and AML/KYC. Though SaaS is feasible for organizations looking for quality and ease of application, open-source code produces better software. It also gives control and makes the organization feel empowered.          

Conclusion: AI is not a simple plug and play 

AI is not a simple plug-and-play. It is a paradigm shift and not everyone gets it right the first time. Multiple iterations are involved as models do not always give the desired returns. There are challenges like the diminishing value of data which would require organizations to broaden their scope and consider a wider data subset for maximizing accuracy.  

Notwithstanding the challenges, AI is a proven game-changer. From simplifying back-office operations to adding value to day-to-day activities, there is a lot that AI can deliver. Expectations, however, would have to be set beforehand. The transition from near-term value to closing in on long-term strategic goals would require foresight and a comprehensive AI roadmap. For more information on how your organization could use AI to drive a successful business strategy, write to us at  mail@magicfinserv.com to arrange a conversation with our AI Experts.     

2020-2021 marked a new epoch in the history of business. For the first time, a massive percentage of the workforce was working from home. While employees struggled to cope with the limitations of working virtually, artificial intelligence (AI) emerged as a reliable partner for enterprises worldwide. With AI, enterprises were assured that business processes were not disrupted due to the scarcity of labor and resources.  

Now that the worst seems over, there are more reasons than ever to invest in AI. AI has been an infallible ally for many organizations in 2020. It helped them meet deadlines and streamline internal operations while eliminating wasteful expenditure. It helped them cope with burgeoning workloads. The impact AI had on employee productivity was significant. By unfettering staff in back and middle offices from the cycle of mundane, repetitive, and tiresome tasks, AI-enabled the workforce to engage in high-value tasks. 

So even as employees return to the office in the coming days,  many organizations will continue to amplify their AI efforts. Wayne Butterfield, director of ISG Automation, a unit of global technology research and advisory firm ISG, attributes this new phenomenon to the powerful impact AI had last year. He says, “As the grip of the pandemic continues to affect the ability of the enterprise to operate, AI in many guises will become increasingly important as businesses seek to understand their COVID- affected data sets and continue to automate day-to-day tasks.”

Indeed, in the banking and financial sector, the benefits driven by AI in the past year were monumental. It ensured frictionless interactions, cut repetitive work by half, and reduced error, bias, and false positives – the result of human fallacies – significantly. What organizations got was a leaner and more streamlined, and efficient organization. So there is no question that the value driven by AI in domains like finance and banking, which rely heavily on processes, will only continue to grow in the years to come. 

Setting pace for innovation and change

The pandemic has redefined digital. With enterprises becoming more digitally connected than ever before, it is AI that helps them stay operational. As a report from the Insider indicates, there will be significant savings in the middle, back, and front office operations if AI is incorporated. Automation of middle-office tasks can lead to savings of $70 billion by 2025. The sum total of expected cost savings from AI applications is estimated at $447 billion by 2023. Of this, the front and middle office will account for $416 billion of the aggregate.  

That AI will set the pace for innovation and change in the banking and financial services sector is all but guaranteed. The shift towards digital had started earlier; the pandemic only accelerated the pace. So here are some of the key areas where Fintechs and banks are using AI :   

• Document Processing  
• Invoice processing
• Cyber Security
• Onboarding/KYC 

Document processing with AI 

Enterprises today are sitting on a data goldmine that comes from sources as diverse as enterprise applications, public/private data sets, and social media. However, data in its raw form is of no use. Data, whether it is in textual, pdfs, spreadsheets, have to be classified, segregated, summarized and converted into formats (JSON, etc.) that can be understood by machines and processes before they can be of use to the organization. 

Earlier, image recognition technologies such as OCR were used for document processing. However, their scope is limited given that organizations deal with humongous amounts of data in diverse formats including print, and handwritten, all of which are not recognizable with OCR. Document processing platforms have a distinct advantage over traditional recognition technologies such as OCR and ICR. The system is trained first using data sets, and a core knowledge base is created. In time the knowledge base expands, and the tool develops the ability to self-learn and recognize content and documents. This is achieved through the feedback or re-training loop mechanism under human supervision. Realizing that artificial intelligence, machine learning, natural language processing, and computer vision can play a pivotal role in document processing, organizations are increasingly relying on these to enhance the efficiency of many front and back-office processes.  

Invoice Processing and AI

Covid-19 has intensified the need for automated Accounts Payable processes. Organizations that were earlier relying on manual and legacy systems for invoice processing were caught off-guard as employees were forced to work from home. Ensuring timely delivery on payment approvals became a challenge due to archaic legacy practices and an increasing number of constraints. Then there was the question of enhanced visibility into outstanding payments. All this led to chaos in invoice processing. A lot of frayed tempers and missed deadlines.

A major chunk of invoice processing tasks is related to data entry. Finance and accounts personnel shift through data that comes from sources such as fax, paper, and e-mail. But a study on 1000 US workers reiterated that no one likes data entry. The survey indicated that a whopping 70 percent of the employees were okay if data entry and other such mundane tasks were automated. With automated invoice processing, it is possible to capture invoices from multiple channels. Identify and extract data (header and lines) using validation and rules. And best in time, with little human supervision, become super proficient in identifying relevant information. It can also do matching and coding.  Magic FinServ’s Machine Learning algorithm correctly determined General Ledger code to correctly tag the invoice against an appropriate charge code and finally, using RPA, was able to insert the code on the invoice.    

Banks and other financial services stand to gain a lot by automating invoice processing. 

• By automating invoice processing with artificial intelligence, organizations can make it easier for the finance staff and back-office team to concentrate on cash-generating processes instead of entering data as -a typical administration function. 
• Automating the accounts payable process for instance, can help the finance teams focus on tasks that generate growth and opportunities. 
• An automated invoice processing provides enhanced visibility into payments and approvals.
• It speeds up the invoice processing cycle considerably as a result; there are no irate vendors
• It makes it easier to search and retrieve invoices.      

Cyber Security and AI

Cybersecurity has become a prime concern with the enterprises increasing preference for cloud and virtualization. Cybersecurity concerns became graver during Covid-19 as the workforce, including software developing teams, started working from home. As third parties and vendors were involved in many processes as well, it became imperative for organizations to ensure extreme caution while working in virtualized environments. Experiences from the past have taught us that data breaches spell disaster for an organization’s reputation. We need to look no further than Panera bread and Uber to realize how simple code left in haste can alter the rules of the game. Hence a greater impetus for the shift left narrative where security is driven in the DevOps lifecycle instead of as an afterthought. The best recourse is to implement an AI-driven DevOps solution. With AI baked into the development lifecycle, organizations can accelerate the development lifecyc in the present and adapt to changes in the future with ease.

Onboarding/KYC and AI

One of the biggest challenges for banks is customer onboarding and KYC. In the course of the KYC, or onboarding banks have to handle thousands, sometimes even millions of documents. And if that were not enough, they also have to take account of exhaust data and the multiple compliances and regulatory standards. No wonder then that banks and financial institutions often fall short of meeting the deadlines. Last year, as the Covid-19 crisis loomed large, it was these tools powered with AI and enabled with machine learning that helped accelerate paperwork processes. These digitize documents and extract data from it. And as the tool evolves with time, it makes it easier for the organization to extract insights from it. 

Let us take the example of one prominent Insurtech company that approached Magic FinServ for the resolution of KYC challenges. The company wanted to reduce the time taken for conducting a KYC and for SLAs roll-out of new policies gained confidence and customer appreciation as Magic’s “soft template” based solution augmented by Artificial Intelligence provided them the results they wanted.  

Tipping point

Though banks and financial institutions were inclining towards the use of AI for making their processes robust, the tipping point was the pandemic. The pandemic made many realize that it was now or never. This is evident from the report by the management solutions provider OneStream. The report observed that the use of AI tools like machine learning had jumped from about 20% of enterprises in 2020 to nearly 60% in 2021. Surprisingly, analytics firms like FICO and Corinium that a majority of top executives (upwards 65%) do not know how AI works. 

At Magic FinServ, our endeavor is to ensure that the knowledge percolates enterprise-wide. Therefore, our implementation journey starts with a workshop wherein our team of AI engineers showcases the work they have done and then engages in an insightful session where they try to identify the areas where opportunities exist and the deterrents. Thereafter comes the discovery phase, where our team develops a prototype. Once the customer gives the go-ahead as they are confident about our abilities to meet expectations, we implement the AI model that integrates with the existing business environment. A successful implementation is not the end of the journey as we keep identifying new areas of opportunities so that true automation at scale can be achieved.     

Catering to Banks and FinTechs: Magic FinServ’s unique AI optimization framework    

At Magic FinServ, we have a unique AI Optimization framework that utilizes structured and unstructured data to build tailored solutions that reduce the need for human intervention. Our methodology powered by AI-ML-NLP and Computer vision provides 70% efficiency in front and middle office platforms and processes. Many of our AI applications for Tier 1 investment, FinTechs, Asset Managers, Hedge Funds, and InsuranceTech companies have driven bottom and top-line dividends for the businesses in question. We ensure that our custom-built applications integrate seamlessly into the existing systems and adhere to all regulatory compliance measures ensuring agility. 

For some time now, asset managers have been looking at ways to net greater profits by optimizing back-office operations. The clamor to convert back-office from a “cost-center” to a “profit center” is not recent. But it has increased with the growth of passive investment and regulatory controls. Moreover, as investment fees decline, asset managers look for ways to stay competitive. 

Back-office is where AI and ML can drive massive business impact. 

For most financial organizations considering a technology upgrade, it is the back office where they must start first. Whether reconciliation or daily checkout or counterparties, back-office processes are the “low-hanging fruits” where AI and ML can be embedded within existing architecture/tools without much hassle. The investment costs are reasonably low, and financial organizations are generally assured of an ROI if they choose the appropriate third-party vendor with expertise in handling such transitions.         

Tasks in the back-office that AI can replace

AI can best be applied to tasks that are manual, voluminous, repetitive, and require constant analysis and feedback. This makes back-office operations/processes a safe bet for AI, ML, and NLP implementation. 

The amount of work that goes behind the scenes in the back office is exhaustive, never-ending, and cumbersome. Back-office operatives are aided in their endeavors by core accounting platforms. Accounting platforms, however, provide the back-office operator with information and data only. Analysis of data is primarily a manual activity in many organizations. As a result, the staff is generally stretched and has no time to add value. Silos further impede process efficiency, and customer satisfaction suffers as the front, back, and middle offices are unable to work in tandem.  

While there is no supplementing human intelligence, the dividends that accrue when AI is adopted are considerable. Efficiency and downtime reduction boost employee and organization morale while driving revenue upstream.

This blog will consider a few use cases from the back-office where AI and ML can play a significant role, focusing on instances where Magic FinServ was instrumental in facilitating the transition from manual to AI with substantial benefits.  

KYC: Ensuring greater customer satisfaction 

Data that exists in silos is one of the biggest challenges in fast-tracking KYC. Unfortunately, it is also the prime reason behind a poor customer experience. The KYC process, when done manually, is long and tedious and involves chasing clients time and again for the information. 

With Magic DeepSight’s™ machine learning capabilities, asset managers and other financial institutions can reduce this manual effort by up to 70% and accomplish the task with higher speed and lower error rate, thereby reducing cost. Magic DeepSight™ utilizes its “soft template” based solution to eliminate labor-intensive tasks. It has enabled several organizations to reduce the time taken for KYC and overall improve SLAs for new client onboarding.  

Reconciliation: Ensuring quicker resolution

As back-office operations are required to handle exceptions quickly and accurately, they need manual effort supplemented by something more concrete and robust. Though traditional tools carry out reconciliation, many organizations still resort to spreadsheets and manual processes, and hence inconsistencies abound. As a result, most organizations manually reconcile anywhere between 3% to 10% volume daily.

So at Magic FinServ, we designed a solution that can be embedded/incorporated on top of an existing reconciliation solution. This novel method reduces manual intervention by over 95% using artificial intelligence. This fast-tracks the reconciliation process dramatically, ensures quicker time to completion, and makes the process less error-prone. Magic FinServ implemented this ‘continuously learning’ solution for a $250B AUM Asset Manager and reduced the trade breaks by over 95%.

Fund Accounting: Ensuring efficiency and productivity 

Fund accounting can be made more efficient and productive with AI. Instead of going through tons of data in disparate formats, by leveraging the powers of AI, the back office can analyze information in income tax reports, Form K-1 tax reports, etc., at a fraction of time taken manually and make it available for dissemination. For example, Magic FinServ’s Text Analytics Tool, which is based on Distant Supervision & Semantic Search, can summarize almost any unstructured financial data with additional training. For a Tier 1 investment bank’s research team that needed to fast-track and made their processes more efficient, we created an integrated NLP-based solution that automated summarizing the Risk Factors section from the 10-K reports.

Invoice and Expense Automation: Eliminating the manual effort

Automated invoice processing is the answer for organizations that struggle with a never-ending backlog of invoices and expenses. An AI integrated engine captures and extracts invoice and expense data in minutes. Without setting new templates and rules, data can be extracted from different channels. There’s also the advantage of automated learning facilitated by the AI engine’s self-learning and validation interface.

Magic FinServ used its sophisticated OCR library built using Machine Learning to get rid of manual effort in uploading invoices to industry-standard invoice & expenses management applications. Another Machine Learning algorithm was able to correctly determine General Ledger code to tag the invoice against an appropriate charge code correctly, and finally, using RPA was able to insert the code on the invoice.

Streamlining corporate actions operations:  

Corporate actions are one of the classic use-cases for optimization using AI. Traditionally, most corporate actions have been done manually, even though they are low-value activities and can mostly be automated with suitable systems. However, whether it is managing an election process with multiple touchpoints or disseminating accurate and complete information to stakeholders and investment managers, the fallout of missing an event or misreporting can be considerable. One way to reduce the risk is to receive notifications from more than one source. But that would compound the back-office workload as they would have to record and reconcile multiple notifications. Hence the need for AI.

Magic FinServ’s AI solution streamlines several routine corporate action operations delivering superior quality. The AI system addresses inefficiencies by reading and scrubbing multiple documents to capture the corporate action from the point of announcement and create a golden copy of the corporate action announcement with ease and efficiency. This takes away the need for manual processing of corporate action announcements saving up to 70% of the effort. This effort can be routed to other high-risk and high-value tasks. 

Conclusion: 

Back-office automation drives enormous dividends. It improves customer satisfaction and efficiency, reduces error rates,  and ensures compliance. Among the five technology trends for banks (for 2020 and beyond), the move towards “zero back offices” – Forrester report, is a culmination of the increasing demand for process automation in the back office. “Thirty percent of tasks in a majority of occupations can be automated, and robotics is one way to do that. For large back offices with data-entry or other repetitive, low judgment, high-error-prone, or compliance-needy tasks, this is like a panacea.”McKinsey Global Institute. For a long time, we have also known that most customer dissatisfaction results from inadequacies of back-office. As organizations get ready for the future, there is a greater need for synchronization between the back, middle, and front office. There is no doubt that AI, ML,  and NLP will play an increasingly more prominent role in the transition to the next level.

A Forrester Report suggests that by 2030, banking would be invisible, connected, insights-driven, and purposeful. ‘Trust’ will be key for building the industry in the future.  

But how do banks and FinTechs enable an excellent customer experience (CX) that translates into “trust” when the onboarding experience itself is time-consuming and prone to error. The disengagement is clear from industry reports. 85% of corporates complained that the KYC experience was poor. Worse, 12% of corporate customers changed banks due to the “poor” customer experience.

Losing a customer is disastrous because the investment and effort that goes into the process are immense. Both KYC and Customer Lifecycle Management (CLM) are expensive and time-consuming. Banks could employ hundreds of staff for a high-risk client for procuring, analyzing, and validating documents. Thomson Reuters reports that, on average, banks use 307 employees for KYC. They spend $40 million (on average) to onboard new clients. When a customer defects due to poor customer engagement, it is a double whammy for the bank. It loses a client and has to work harder to cover the costs of the investment made. Industry reports indicate that new customer acquisition is five times costly than retaining an existing one. 

The same scenario is applicable for financial companies, which must be very careful about who they take in as clients. As a result, FinTechs struggle with greater demand for customer-centricity while fending competition from challengers. By investing in digital transformation initiatives like digital KYC, many challenger banks and FinTechs deliver exceptional CX outcomes and gain a foothold. 

Today Commercial Banks and FinTechs cannot afford to overlook regulatory measures, anti-terrorism, anti-money laundering (AML) standards, and legislation, violations of which would incur hefty fines and lead to reputational damage. The essence of KYC is to create a robust, transparent, and up-to-date profile of the customer. Banks and FinTechs investigate the source of their wealth, ownership of accounts, and how they manage their assets. Scandals like Wirecard have a domino effect, and so banks must flag off inconsistencies in real-time. As a result, banks and FinTechs have teamed up with digital transformation partners and are using emerging technologies AI, ML, and NLP to make their operations frictionless and customer-centric. 

Decoding existing paint-points and examining the need for a comprehensive data extraction tool to facilitate seamless KYC

Long time-to-revenue results in poor CX

Customer disengagement in the financial sector is common. Every year, financial companies lose revenue due to poor CX. Here the prime culprit for customer dissatisfaction is the prolonged time-to-revenue. High-risk clients average 90-120 days for KYC and onboarding. 

The two pain points are – poor data management and traditional methods for extracting data from documents (predominantly manual). Banking c-suite executives concede that poor data management arising due to silos and centralized architecture is responsible for high time-to-revenue.  

The rise of exhaust data 

Traditionally, KYC involved checks on data sources such as ownership documents, stakeholder documents, and the social security/ identity checks of every corporate employee. But today, the KYC/investigation is incomplete without verification of exhaust data. And in the evolving business landscape, it is exigent that FinTech and banks take exhaust data into account. 

Emerging technologies like AI, ML, and NLP make onboarding and Client Lifecycle Management (CLM) transparent and robust. With an end-to-end CLM solution, banks and FinTech can benefit from an API-first ecosystem that supports a managed-by-exception approach. An API-first ecosystem that supports an exception management approach is ideal for medium to low-risk clients. Data management tools that can extract data from complex documents and read like humans elevate the CX and save banks precious time and money. 

Sheer volume of paperwork prolongs onboarding. 

The amount of paperwork accompanying the onboarding and KYC process is humongous. When it comes to business or institutional accounts, banks must verify every person’s existence on the payroll. Apart from social security and identity checks, ultimate beneficial owners (UBO), and politically exposed persons (PEP), banks would have to cross-examine documents related to the organization’s structure. Verifying the ownership of the organization and the beneficiaries’ check adds to the complexity. After that, corroborating data with media checks and undertaking corporate analysis to develop a risk profile. With this kind of paperwork involved, KYC could take days. 

However, as this is a low-complexity task, it is profitable to invest in AI. Instead of employing teams to extract and verify data, banks and FinTechs can use data extraction and comprehension tools (powered with AI and enabled with machine learning) to accelerate paperwork processes. These tools digitize documents and extract data from structured and unstructured documents, and as the tool evolves with time, it detects and learns from document patterns. ML and NLP have that advantage over legacy systems – learning from iterations.   

Walking the tightrope (between compliance and quick TOI)

Over the years, the kind of regulatory framework that America has adopted to mitigate financial crimes has become highly complex. There are multiple checks at multiple levels, and enterprise-wide compliance is desired. Running a KYC engages both back and front office operations. With changing regulations, Banks and FinTechs must ensure that KYC policies and processes are up-to-date. Ensuring that customers meet their KYC obligations across jurisdictions is time-consuming and prolonged if done manually. Hence, an AI-enabled tool is needed to speed up processes and provide a 360-degree view and assess the risk exposure. 

In 2001, the Patriot Act came into existence to counter terrorist and money laundering activities. KYC became mandatory. In 2018, the U.S. Financial Crimes Enforcement Network (FinCEN) incorporated a new requirement for banks. They had to verify the “identity of natural persons of legal entity customers who own, control, and profit from companies when those organizations open accounts.” Hefty fines are levied if banks fail to execute due diligence as mandated.

If they are to rely on manual efforts alone, banks and FinTechs will find it challenging to ensure CX and quick time-to-revenue while adhering to regulations. To accelerate the pace of operations, they need tools that can parse through data with greater accuracy and reliance than the human brain. And also can learn from processes.  

No time for perpetual KYC as banks struggle with basic KYC

For most low and medium-risk customers, a straight-through-processing (STF) of data would be ideal. It reduces errors and time to revenue. Client Lifecycle Management is essential in today’s business environment as it involves ensuring customers are compliant through all stages and events in their lifecycle with their financial institution. That would include raking through exhaust data and traditional data from time to time to identify gaps. 

A powerful document extraction and comprehension tool is therefore no longer an option but a prime requirement.  

Document extraction and comprehension tool: how it works 

Document digitization: IDP begins with document digitization. Documents that are not in digital format are scanned. 

OCR: Next step is to read the text. OCR does the job. Many organizations use multiple OCRS for accuracy. 

NLP: Recognition of text follows the reading of the text. With NLP, words, sentences, and paragraphs are provided a meaning. NLP uses sentiment analysis, part of speech tagging, and making it easier to draw a relation. 

Classification of documents: Manual categorization of documents is another lengthy process that is tackled by IDP’s classification engine. Here machine learning (ML) tools are employed to recognize the kinds of documents and feed them to the system.  

Extraction: The penultimate step in IDP is data extraction. It consists of labeling all expected information within a document and extracting specific data elements like dates, names, numbers, etc.

Data Validation: Once the data has been extracted, it is combined and pre-defined validation rules based on AI check for accuracy and flag off errors, improving the quality of extracted data.     

Integration/Release: Once the data has been validated/checked, the documents and images are exported to business processes or workflows. 

The future is automation!

The future is automation. An enriched customer experience begins with automation. To win customer trust, commercial banks and FinTechs must ensure regulation compliance, improve CX, reduce the costs by incorporating AI and ML and ensure a swifter onboarding process. In the future, banks and FinTechs that improvise their digital transformation initiatives and enable faster and smoother onboarding and customer lifecycle management will facilitate deeper customer engagement. They would have gained an edge. Others would struggle in an unrelenting business landscape.

True, there is no single standard for KYC in the banking and FinTech industry. The industry is as vast as the number of players. There are challengers/start-ups and decades-old financial institutions that coexist. However, there is no question that data-driven KYC powered by AI, ML brings greater efficiency and drives customer satisfaction. 

A tool like Magic DeepSight™ is a one-stop solution for comprehensive data extraction, transformation, and delivery from a wide range of unstructured data sources. Going beyond data extraction, Magic DeepSight™ leverages AI, ML, and NLP technologies to drive exceptional results for banks and FinTechs. It is a complete solution as it integrates with other technologies such as API, RPA, smart contract, etc., to ensure frictionless KYC and onboarding. That is what the millennial banks and FinTechs need.  

Ingesting Unstructured data into other Platforms

Industry specific Products / Platforms like the ERP for specific functions and processes have contributed immensely to enhancing efficiency and productivity. SI partners and end-users have focused on integrating these platforms with existing workflows through a combination of customization/configuring of these platforms and re-engineering existing workflows. Data Onboarding is a critical activity however it has been restricted to integrating the platforms with the existing ecosystem. A key element that is very often ignored is integrating Unstructured Data sources in the Data Onboarding process.

Most enterprise-grade products and platforms require a comprehensive utility that can extract and process a wide set of unstructured documents, data sources and ingest the output into a defined set of fields spread across several internal and third-party applications on behalf of their clients. You are likely extracting and ingesting this data manually today, but an automated utility could be a key differentiator that reduces time, effort and errors from this extraction process. 

Customers have often equated use of OCR technologies as solutions to these problems, however OCR suffers from quality and efficiency issues thereby requiring manual efforts. More importantly OCR extracts the entire document and not just the relevant Data Elements, thereby adding significant noise to the process. And finally, the task of ingesting this data into the relevant fields in the applications / platforms is still manual.

When it comes to widely used and “customizable” case management platforms for Fincrime applications, CRM platforms, or client on-boarding/KYC platforms, there is a vast universe of unstructured data that requires processing outside of the platform in order for the workflow to be useful. Automating manual extraction of critical data elements from unstructured sources with the help of an intelligent data ingestion utility enables users to repurpose critical resources tasked with repetitive offline data processing.

Your data ingestion utility can be a “bolt on” or a simple API that is exposed to your platform. While the document and data sets may vary, as long as there is a well-defined list of applications and fields that are required to be populated, there is a tremendous opportunity to accelerate every facet of client lifecycle management. There are several benefits to both “a point solution” which automates extraction of a well-defined document type/format as well as a more complex, machine learning based utility for a widely defined format of the same document type. 

Implementing Data Ingestion

An intelligent pre and post processing data ingestion can be implemented in 4 stages, each stage increasing in complexity and value extracted from your enterprise platform:

Stage 1 

• Automate the extraction of standard templatized documents. This is beneficial for KYC and AML teams that are handling large volumes of standard identification documents or tax filings which do not vary significantly. 

Stage 2 

• Manual identification and automated extraction of data elements. In this stage, end users of an enterprise platform can highlight and annotate critical data elements which an intelligent data extraction utility should be able to extract for ingestion into a target application or specified output format. 

Stage 3

• Automated identification and extraction as a point solution for specific document types and formats.

Stage 4

• Using stage 1-3 as a foundation, your platform may benefit from a generic automated utility which uses machine learning to fully automate extraction and increase flexibility of handling changing document formats. 

You may choose to trifurcate your unstructured document inputs into “simple, medium, and complex” tiers as you develop a cost-benefit analysis to test the outcomes of an automated extraction utility at each of the aforementioned stages. 

Key considerations for an effective Data Ingestion Utility:

• Your partner should have the domain expertise to help identify the critical data elements that would be helpful to your business and end users 
• Flexibility to handle new document types, add or subtract critical data elements and support your desired output formats in a cloud or on-premise environment of your choice
• Scalability & Speed
• Intelligent upfront classification of required documents that contain the critical data elements your end users are seeking
• Thought leadership that supports you to consider the upstream and downstream connectivity of your business process

Introduction

Investment research and analysis is beginning to look very different from what it did five years ago. While five years ago, the data deluge could have confounded asset management leaders, they now have a choice on how things could be done differently, thanks to AI and advanced analytics. Advanced analytics helps create value by eliminating biased decisions, enabling automatic processing of big data, and using alternative data sources to generate alpha. 

With multiple sources of data and emerging AI applications heralding a paradigm shift in the industry, portfolio managers and analysts who earlier used to manually sift through large volumes of unstructured data for investment research can now leverage the power of AI tools such as natural language processing and abstraction to simplify their task. Gathering insights from press releases, filing reports, financial statements, pitches and presentations, CSR disclosures, etc., is a herculean effort and consumes a significant amount of time. However, with AI-powered data extraction tools such as Magic DeepSight™, quick processing of large-scale data is possible and practical.

A tool like Magic DeepSight™  extracts relevant insights from existing data in a fraction of the time and capital compared to manual processing. However, the real value it delivers is by supplementing human intelligence with powerful insights, allowing analysts to direct their efforts towards high-value engagements.

Processing Unstructured Data Is Tough

There are multiple sources of information that front office analysts process daily, which are critical to developing an informed investment recommendation. Drawing insights from these sources of structured and unstructured data are challenging and complex. These include 10-K reports, the reasonably new ESG reports, investor reports, and various other company documents such as internal presentations and several PDFs. SEC EDGAR database makes it easy to access some of this data, but extracting this data from SEC EDGAR and identifying and then compiling relevant insights is still a tedious task. Unearthing insights from other unstructured documents also takes stupendous manual efforts due to the lack of any automation. 

10-K Analysis using AI

More detailed than a company’s annual report, the 10-K is a veritable powerhouse of information. Therefore, accurate analysis of a 10-K report would lead to a sounder understanding of the company. There are five clear-cut sections of a 10-K report – business, risk factors, selected financial data, management discussion and analysis (MD&A), financial statements, and supplementary data, all of which are packed with value for analysts investors alike. Due to the breadth and scope of this information, handling it is inevitably time-consuming. However, two sections that usually require more attention than the others to analyze due to the complexity and existence of possible hidden anomalies are the “Risk Factors” and the “MD&A”. The “Risk Factors” section outlines all current and potential risks posed to the company, usually in the order of importance. In contrast, the   “Management’s Discussion and Analysis Of Financial Condition And Results Of Operations” (MD&A) section is the company management’s perspective of the previous fiscal and future business plans’ performance.

As front-office analysts sift through multiple 10-K reports and other documents in a day, inconsistencies in analysis can inadvertently creep in. 

They can miss important information, especially in the MD&A and Risk Factors sections, as they have to analyze many areas to study and more reports in the queue. Even after extracting key insights, it takes time to compare the metrics in the disclosures to a company’s previous filings and against industry benchmarks. 

Second, there is the risk of human bias and error, where relevant information may be overlooked.  Invariably, even the best fund managers would succumb to the emotional and cognitive biases inherent in all of us, whether confirmation bias, bandwagon effect, loss aversion, or various other biases that behavioral psychologists have formally defined. Failure to consider these issues will lead to suboptimal decisions on asset-allocation and often does. 

Using AI to analyze the textual information in the disclosures made within 10-Ks can considerably cut through this lengthy process. Data extraction tools can parse through these chunks of texts to retrieve relevant insights. And a tool or platform custom-built for your enterprise and trained in the scope of your domain can deliver this information to your business applications directly. More documents can be processed in a shorter time frame, and armed with new insights, analysts can use their time to take a more in-depth learning’s untapped potential look into the company in question. Implementing an automated AI-Based system omits the human errors,  allowing investment strategies to be chosen that are significantly more objective, in both their formulation and execution. 

Analysing ESG Reports

Most public and some private companies today are rated on their environmental, social and governance (ESG) performance. Companies usually communicate their key ESG initiatives yearly on their websites as a PDF document. Stakeholders are studying ESG reports to assess a company’s ESG conduct. Investment decisions and brand perception can hinge on these ratings, and hence care has to be taken to process information carefully. In general, higher ESG ratings are positively correlated with valuation and profitability while negatively correlated with volatility. An increased preference for socially responsible investments is most prevalent in Gen Z and Millennial demographics. As they are set to make-up 72% of the global workforce by 2029, they are also exhibiting greater concern about organizations’ and employers’ stance on environmental and social issues. This is bringing under scrutiny a company’s value creation with respect to ethical obligations that impact the society it operates in.

Although, ESG reports are significant when it comes to a company’s evaluation by asset managers, investors, and analysts, as these reports and ratings are made available by third-party providers there is little to no uniformity in ESG reports unlike SEC filings. Providers tend to have their own methodology to determine the ratings. The format of an ESG report varies from provider to provider, making the process of interpreting and analyzing these reports complicated. For example, Bloomberg, a leading ESG data provider, covers 120 ESG indicators– from carbon emissions and climate change effects to executive compensation and rights of shareholders. Analysts spend research hours reading reports and managing complex analysis rubrics to evaluate these metrics, before making informed investment decisions.

However AI can make the entire process of extracting relevant insights easy. AI-powered data cleansing and Natural Language Processing (NLP) tools can extract concise information, such as key ESG initiatives from PDF documents and greatly reduce the text to learn from. NLP can also help consolidate reports into well defined bits of information which can then be plugged into analytical models including market risk assessments, as well as other information fields. 

How Technology Aids The Process

A data extraction tool like Magic DeepSight™ can quickly process large-scale data, and also parse through unstructured content and alternate data sources like web search trends, social media data, and website traffic. Magic DeepSight™ deploys cognitive technologies like NLP, NLG, and machine learning for this. Another advantage is its ability to plug the extracted information into relevant business applications, without  human intervention. 

About NLP and NLG

Natural Language Processing (NLP) understands and contextualises unstructured text into structured data. And Natural Language Generation (NLG) analyses this structured data and transforms it into legible and accessible text. Both processes are powered by machine learning and allow computers to generate text reports in natural human language. The result is comprehensive, machine-generated with insights that were previously invisible. But how reliable are they?

The machine learning approach that includes deep learning, builds intelligence from a vast number of corrective iterations. It is based on a self-correcting algorithm which is a continuous learning loop that gets more relevant and accurate the more it is implemented. NLP and AI-driven tools, when trained in the language of a specific business ecosystem, like asset management, can deliver valuable insights for every stakeholder across multiple software environments, and in appropriate fields.

Benefits of Using Magic DeepSight™ for Investment Research

1. Reduced personnel effort

Magic DeepSight™ extracts, processes, and delivers relevant data directly into your business applications, saving analysts’ time and enterprises’ capital.

2. Better decision-making

By freeing up upto 70% of the time invested in data extraction, tagging, and management, Magic DeepSight™ recasts the analysis process. It also supplements decision-making processes with ready insights. 

3. Improved data-accuracy

Magic DeepSight™ validates the data at source. In doing so, it prevents errors and inefficiencies from  creeping downstream to other systems. 

4. More revenue opportunities

With reduced manual workload and emergence of new insights, teams can focus on revenue generation and use the knowledge generated to build efficient and strategic frameworks. 

In Conclusion

Application of AI to the assiduous task of investment research can help analysts and portfolio managers assess metrics quickly, save time, energy and money and make better-informed decisions in due course. The time consumed by manual investment research, especially 10-K analysis, is a legacy problem for financial institutions. Coupled with emerging alternative data sources, such as ESG reports, investment research is more complicated today. After completing research, analysts are left with only a small percentage of their time for actual analysis and decision-making. 

A tool like Magic DeepSight™ facilitates the research process, improves predictions, investment decision-making, and creativity. It could effectively save about 46 hours of effort and speed up data extraction, tagging, and management by 70%. In doing so, it brings unique business value and supports better-informed investment decisions. However, despite AI’s transformative potential, relatively few investment professionals are currently using AI/big data techniques in their investment processes. While portfolio managers continue to rely on Excel and other necessary market data tools, the ability to harness AI’s untapped potential might just be the biggest differentiator for enterprises in the coming decade. 

To explore Magic DeepSight™ for your organization, write to us mail@magicfinserv.com or Request a Demo