On June 27th, FINRA released Notice 24-09 which outlines how firms must manage GenAI (Generative Artificial Intelligence) implementations. Though GenAI provides powerful new tools, they also create several compliance challenges. First, there are several ways that SEC firms are implementing GenAI but in general it is being driven by the expansion of centrally driven experiments. The projects can range from: customer contact centers, content creation, software engineering, and finally compliance initiatives.
Customer Contact Center: GenAI can help provide financial firm customers with all the relevant information they need quickly and accurately, while reducing call frequency and support workloads. The primary objective of AI-powered chatbots is to improve the quality of customer chat interactions.
Automated Advisory: Firms are looking at using GenAI to create investment tools to assist clients to make investment decisions based on research reports, market conditions and holds. We’ve seen many efforts including the recent purchase by Robinhood Markets of Pluto Capital (Robinhood snaps up Pluto to add AI tools to its investing app (yahoo.com).
Content Creation: The creation of personalized content for marketing and customer communications is improved with GenAI. Custom content that is specifically personalized for potential new customers can increase acquisition and lower traditional sales and marketing costs.
Software Engineering: The objective to improve engineering practices with GenAI is one of the most popular use cases. Financial firms are evaluating the possibilities to improve the software delivery lifecycle to address increasing demand: a higher velocity without compromising on quality.
Compliance: GenAI can make compliance workflows more efficient, allowing analysts to spend more time on customer analysis and less on administration or data collection. For example, broker-dealers are utilizing GenAI to conduct communications surveillance (FINRA 3110) or generate automated risk summaries as part of their Customer Due Diligence reviews.
Each respective financial firm application of GenAI, however, needs to be comprehensively evaluated and tested before production. Similarly, they will all require the same internal policies and procedures to comply with FINRA’s new guidance. As FINRA states “a member firm must have a reasonably designed supervisory system tailored to its business… if a firm is using GenAI tools… its policies and procedures should address technology governance, including model risk management, data privacy and integrity, reliability and accuracy of the AI Model.” (Regulatory Notice 24-09 | FINRA.org)
To properly manage the risks and opportunities presented by GenAI, it is imperative that financial firms take a responsible, risk-based approach to experimentation and introduction into the firm. The important steps firms must work to ensure they get right include adjusting existing risk and control frameworks for potential GenAI threats, ensuring clear governance in line with FINRA regulatory guidance, and educating firm employees on GenAI fundamentals.
Technology Governance: From our standpoint, technology governance follows what is already financial industry ‘best practices’ which apply a standardized risk-based approach to GenAI applications. There are three main stages of GenAI technology governance: Registration of GenAI Application and Business Case evaluation, Risk Assessment, Post Approval Continuous Monitoring. For instance, before implementing a new trading algorithm, the following should take place:
- Written statement (release notes) as to what the software changes will do to the algorithm. (If software is developed internally, it should be linked to the code.
- Test scripts and results. These results should have written verification that the change is the processed dataset matches the release requirements. These scripts and results should be retained in unalterable storage.
- Review and approval from either IT management and/ or compliance.
Integrity, reliability and accuracy of the AI model: Model testing, validation, and monitoring is the execution of the independent assessment of a GenAI model and is imperative to maintain FINRA compliance. All firms are required to perform this assessment to assure that the developed model is reliably accurate, appropriate for intended use, and in compliance with data security and privacy regulation. This independent assessment promotes a better understanding of a model, specifically its strengths and limitations or weaknesses as it may relate to hallucination. Finally, this formal assessment issues a detailed validation report that highlights the testing performed and the results which indicate if the firm can be satisfied that the GenAI model can be deployed in a safe, accurately reliable, and compliant manner.
For example, when rolling out a new Gen AI surveillance platform, compliance must carefully document the changes that the implementation makes to standard surveillance datasets. This should include:
- Comparison of the old methodology results with the Gen AI results with verification that the Gen AI result set improves upon the legacy platform. These results should be retained in unalterable storage.
- Upgrade the policy manuals with documentation of the Gen AI platform and how it will be implemented, including covered employees and surveilled messaging types.
- Additional features such as, ‘Does the Gen AI cover non-English languages?’ ‘Does it flag for emojis, graphics or hyperlinks?’ ‘Apps such as whiteboards or transcripts?’
GenAI Risk Managaement and Committee: While the risk and control measures are modified in line with the risks that come with GenAI, establishment of a GenAI Risk Management Committee is highly recommended. An effective GenAI Committee is an additional safety measure that is complementary to all the other existing procedures and policies that address for the risk assessment and approval of new products. A committee with a diverse range of backgrounds and experience helps the learning and provides extra balanced decision making on this complex, fast-moving topic. The GenAI committee can have the following roles and responsibilities:
- Act as model approver body
- Ensure collection and distribution of shared learning
- Approve start of new GenAI projects
- Ensure process and delivery of GenAI applications
- Monitor progress of GenAI initiatives
- Reporting
As GenAI will have profound impacts in financial institutions, we recommend that implementations are governed by an AI committee consisting of IT, compliance and outside consultants such as 17a-4, LLC. Though GenAI will transform the financial industry, it, as so many other technologies, have serious risks and the regulators will have concerns high on their exam priorities.
Please reach out to sales@17a-4.com to learn more.
Rule 17a-4 Compliance for Algorithmic Trading and Robo Advising Software – 17a-4 LLC
