Case Study:

ChatGPT in Market Surveillance

SteelEye implements ChatGPT in its Integrated Surveillance platform

Foreword

SteelEye has carried out a research project to evaluate the applicability of ChatGPT in compliance, more specifically in the investigation assessment and workflow process.

The ChatGPT AI engine has been used to analyze communications flagged in SteelEye’s integrated trade and communications surveillance system.

The results are promising. When implemented with care and leveraged correctly, SteelEye has found that the tool can deliver powerful insights that can go a long way to support the compliance function and help users make sense of information and decide on the best course of action.

Analysis and decision-making support from a Large Language Model like ChatGPT can empower compliance teams to analyze data with more incredible speed and identify potential risks more effectively.

However, there are some key considerations firms need to bear in mind if they want to leverage this technology, including how ChatGPT processes data.

This case study has been developed to provide financial firms with an overview of how technologies like ChatGPT can be applied in compliance and to present the benefits, use cases, and considerations for its use.

We hope you enjoy the case study.

SteelEye implements ChatGPT in its Integrated Surveillance platform Matt Storey
Chief Product Officer and Co-Founder
SteelEye

Steeleye

Ready to see the power of the SteelEye Platform?

Get a free, live demo.

How ChatGPT has been integrated in SteelEye for market surveillance

ChatGPT has been implemented in SteelEye's UAT environment and has been tested against communications data in the firm’s integrated trade and communications surveillance solution.

The functionality enables users to analyze specific communications including voice calls, chats, meetings, and emails - for example, those that have been flagged as suspicious in a communications surveillance system - against a number of key questions.

When combined with ChatGPT, SteelEye returns the following insights in a matter of seconds:

Summary: Content summary, motives, intentions, and regulatory or compliance issues identified
Analysis: Entities included in the conversation, sentiment, tone, and tonal shifts
Next steps: suggestions around what a compliance officer should do next and proposed correspondence

This can be used:

As a starting point in a surveillance investigation and when analyzing communications in different languages
To standardize workflow processes and boost the throughput and consistency of cases
To determine which other records should be investigated

SteelEye implements ChatGPT in its Integrated Surveillance platform

Use cases for ChatGPT in market surveillance

Questions answered with ChatGPT in SteelEye

Summary Fields

Content summary:

This field returns a summary of the conversation, regardless of the type. It is particularly powerful in the case of lengthy chats or voice calls as it can take a long time for a surveillance analyst to read the transcript and digest what is going on.

Case Study: ChatGPT in Market Surveillance

Case Study: ChatGPT in Market Surveillance

What are the motives and intent of the people?

This is an analysis of the potential intent and motives of the people that are part of the conversation. For example, this can very quickly highlight whether the parties involved have an intent to manipulate financial markets as per the screenshot.

Are there any regulatory or compliance issues?

Here, the analysis looks at whether the conversations could be in breach of any regulatory rules. Of course, the surveillance officer has a bigger job to determine exactly which rules were broken, but this provides speedy insights into signs of insider dealing, market manipulation, or other compliance violations.

SteelEye implements ChatGPT in its Integrated Surveillance platform

Analytics fields

Case Study: ChatGPT in Market Surveillance

What's the sentiment and tone?

These fields return a summary of the sentiment of the conversation as well as any shifts in the tone. This is particularly powerful if we take a long voice call or chat transcript as it enables the surveillance analysts to move directly to the timeframe when the tone shifts.

What entities are mentioned?

This is an analysis of the entities involved in the communication and returns a list of any people and companies involved. In the case of insider trading, this helps a compliance officer quickly deduce if any listed companies were mentioned in the conversation. This then provides a good next step to investigate whether anyone traded on the information shared in the conversation. In the screenshot below (left) we can see the company “Kimball” was mentioned in the conversation, and in screenshot to the right that one of the traders has executed against the MNPI ahead of a significant share price move.

SteelEye implements ChatGPT in its Integrated Surveillance platform

Next Steps Fields

What’s my next step?

This field provides a set of suggestions for what the compliance officer should do next.

SteelEye implements ChatGPT in its Integrated Surveillance platform

SteelEye implements ChatGPT in its Integrated Surveillance platform

What should I look for next?

Here, a compliance officer receives a number of suggestions around what they could be looking at next in terms of the investigation.

Draft email to send to the participants:

This field drafts a proposed email that can be sent to the person/people involved in the communication.

SteelEye implements ChatGPT in its Integrated Surveillance platform

Benefits of ChatGPT in market surveillance

What this could mean for the future of surveillance

ChatGPT can have several benefits for market surveillance.

One of the primary benefits is its ability to process large amounts of unstructured data, just like the SteelEye platform. ChatGPT is designed to process and understand natural language, making it well-suited for analyzing data such as conversations, social media posts, news articles, and chat transcripts.

By analyzing this data, a SteelEye integration with ChatGPT can help a compliance professional to more rapidly identify patterns and trends that may indicate potential market events or risks. This includes sudden changes in sentiment and indications of market manipulation or insider trading. This is particularly useful for detecting unusual activity or patterns of behavior that may be indicative of illegal or unethical activity.

ChatGPT can also be used to support decision-making by providing instant analysis of and recommendations for the aforementioned data. It can help to standardize workflow processes and boost the throughput and consistency of cases. This can improve the accuracy and speed of decision-making and enable surveillance teams to respond quickly to risks.

In summary, ChatGPT's natural language processing capabilities, predictive analytics, and decision support can be highly beneficial for market surveillance - providing compliance and surveillance analysts with a powerful toolset that increases the effectiveness of the investigation processes.

Key considerations for using ChatGPT in surveillance

“Promptfinement”

Getting meaningful answers out of a Large Language Model like ChatGPT requires a deep understanding of how the tool works and how to issue prompts. Questions need to be asked in the right way and need to be refined correctly. The model needs prompts and refinements (or “promtfinements”) to be able to deliver relevant and specific results. Let’s look at an example.

Whacking the Close

This is a term used to describe when a trader tries to manipulate the closing price of a security. If we want to ask ChatGPT what this is, it is not enough to say “What does “Whack the close” mean in trading?” This is simply not enough information for the system to know what you are asking. However, if we add some additional “promptfinements” to this, we get closer to what we are looking for.

SteelEye implements ChatGPT in its Integrated Surveillance platform

Important

How Large Language Models like ChatGPT are used is as important as the decision to use them in the first place.

If a person or firm is going to be using this technology, they need to know that the tool has been implemented with care and consideration.

Data processing

As it stands, using a ChatGPT integration like the one developed by SteelEye in this case study requires that content be sent for analysis to either OpenAI's API or Azure's Open AI Service. However, it is worth noting that this only applies to the individual content that a user decides to “analyze with ChatGPT.”

At the moment, these are the only ways ChatGPT's most up-to-date models can be leveraged. Firms need to ensure they are comfortable with how OpenAI processes data and define controls to regularly review OpenAI's policies.

At the release of this case study (23 March 2023), the terms of service state "we do not use Content that you provide or receive from our API (“API Content”) to develop or improve our Services." Further, content transmitted to ChatGPT is retained for 30 days, with options for shorter retention periods.

FAQ

Will the functionality be available to clients?

Yes, SteelEye will be rolling out the functionality in production on March 24th on an opt-in basis for those who want to trial it. However, use of the functionality requires clients to opt-in to ChatGPT’s data policy.

Will SteelEye be trialing other language-based AI models?

Yes, ChatGPT is just one of many Large Language Models, and SteelEye will continue to explore the different tools and how these can support market surveillance.

SteelEye implements ChatGPT in its Integrated Surveillance platform

Turn Supervision into Super Vision with SteelEye

Learn about how we empower you with the data-driven tools and complete insights you need to focus on what matters, all from a single platforms.