The Challenges of Generative AI Usage in Financial Firms
Understanding Gen AI challenges
Although generative AI can undoubtedly bring key benefits to the capital markets industry, as highlighted in our 30 January, 2024 post, the integration of Generative AI into capital markets firms’ processes presents the industry with a unique set of challenges:
Ethical Concerns: Financial institutions are bound by industry-specific regulations and firms face a complex landscape of financial, legal and ethical considerations when dealing with content generated by AI. Firstly, financial institutions must prioritise the ethical and transparent use of client data. Generative AI can facilitate the extraction of valuable insights from this data, but it also introduces concerns related to consent and data ownership. Without clear ethical guidelines for its design and deployment, Generative AI may inadvertently lead to adverse consequences and real harm.
Model Bias and Limitations: One key challenge in the context of Generative AI is the presence of model bias and limitations, which are particularly pronounced in the highly regulated capital markets. Model bias can emerge when data used to train the platform are incomplete, outdated or misrepresented, or it may stem from human biases embedded in the AI algorithm’s design or even from apparent correlations of parameters with a spurious relationship. In theory, AI models are only as good as the data in which they are trained on. In the capital markets, model bias can result in unethical financial practices, financial exclusion, and an erosion of public trust, among other issues. For example, there was a case in 2021 in which Apple and Goldman Sachs were investigated by the New York State Department of Financial Services for algorithmically offering smaller lines of credit to women. Additionally, biased AI models could create systemic risk in the markets as firms become reliant on ‘wrong’ information that could have disastrous reputational and financial consequences.
Cybersecurity Risks: As Generative AI is an emerging technology, it has the potential to be exploited for the creation of more sophisticated phishing messages and emails, offering malicious actors opportunities to impersonate individuals or organisations. This raises the risk of increased identity theft and fraud. Additionally, the rise of deep fakes, which are highly realistic AI-generated videos, audio, or images, may cause significant harm to both individuals and organisations. For example, there was a case in 2019 that involved the use of fake social media accounts using realistic-looking AI-generated photos of people who did not really exist. One fake account tried to extract information from short sellers of Tesla stocks.
Generative AI could prove to be a game changer, which revolutionises trading by automating trade execution and algorithmic trading strategies to enhance risk management and provide real-time insights. Its ability to analyse vast amounts of data, identify patterns, simulate scenarios and generate trade signals could provide new opportunities for traders to capitalise on market sentiment, predict volatility and adapt to changing market conditions swiftly. Generative AI’s adaptive learning capabilities can ensure that trading strategies evolve in line with market dynamics, leading to enhanced performance over time. However, it is important to acknowledge that although Generative AI has immense transformative potential, the basket of new risks it presents could lead to foreseen and unforeseen challenges. Issues such as unavailability of accurate and high-quality training data, the unexplainable behaviour of complex AI models, biased results, potential systemic risks and ethical concerns must be strategically addressed before the capital markets become too deeply mired in the technology to easily extricate themselves.
The introduction of regulations such as the EU AI Act, which we covered in a recent post, reflects the urgency of acting to counter the evolving risks as well as the speed of adoption of this technology. This framework should allow banks to establish controls and implement governance measures to ensure both safety and effectiveness of the application of the technology in the financial services.
What are your concerns in relation to the growing use of generative AI in the capital markets industry? We’d love to hear from you below.
Stand by for an interview with one of our subject matter experts regarding the potential of Generative AI in the capital markets industry.