Use Real-Time Data to Make Personalized and Timely Offers

Real Time Data to make Personalized and Timely Offers
AvatarSeptember 13, 2016

Throughout the day, customers use their credit, debit, and online and mobile banking channels to make purchases. There is a huge potential to capitalize on these transactional moments and provide real-time personalized offers—and financial institutions have taken notice.


Recently, one of our financial services clients was looking for an innovative way to interact with its customers and provide relevant offers that would drive more customer engagement and use of its banking services. Through the use of real-time analytics that incorporated the customer’s current transaction, the channel, the location, and the customer’s profile, the bank wanted to be able to deliver immediate and relevant marketing offers being offered in the vicinity to its customers.


To meet this need, we helped develop a real-time analytics solution to increase its customers’ usage of the bank’s credit cards, debit cards, and internet and mobile channels. For instance, a customer is at the mall and completes a transaction using a debit or credit card. Through the use of our real-time analytics solution, the bank immediately tracks the terminal location where the sale was made and checks all the offers available in that mall or nearby areas offered by merchants associated with their financial institution. The bank can then make a targeted offer to the customer via a text message or email right at that location near real time.


If, for example, the transaction occurs in the evening (near dinner) and the customer enjoys eating out—which the bank already knows based on the customer profile it has created from past transactions—the system will send the customer a food-related offer located in the mall. The customer is pleased because the offer is both timely and relevant; the bank is pleased because the customer’s use of the offer will lead to further revenue for the bank.


Making Real-Time Analytics for Banking Transactions a Reality
From the example above, it’s easy to see the value of a real-time analytical solution for financial institutions as well as other businesses. However, implementing such a solution is not without its challenges, particularly in an environment where the source system data is highly confidential.


Here are some of the challenges we faced in implementing a real-time analytics solution for our financial services client, and how we dealt with them:


Challenge 1: Querying confidential data without impacting the source system.
Because all the data present within the different business units (credit, debit, and online banking) was confidential, we had no direct access to the source system. This made it a significant challenge to get real-time feeds from the respective source systems—debit, credit, and internet banking—without impacting performance. Incremental fetches or SQL queries on a source system at regular intervals, such as every minute, can negatively impact current processes running on the source system.


To address this issue, we helped the bank put in place a change data capture (CDC) process on selected tables. We then mapped these tables with a similar structure in another schema that is accessible for real-time data processing.


Challenge 2: Reading large amounts of data from the tables.
With the likelihood of needing to read four million or more records per day, it was not feasible for the real-time data processing application to connect every time with the new schema to fetch the incremental data for processing and complete the process in a timely manner.


To address this issue was a two-step process. First, we created a database  stored procedure in a new schema and scheduled it to run every minute to fetch the incremental data and generate the small files in the shared location (NFS Directory). Then, the real-time data processing application read the small file from the shared location and renamed the file after it was processed.


Challenge 3: Generating customer offers in 15-20 seconds.
In order for real-time analytics to be successful, customers’ data needs to not only be quickly processed, but the system needs to be able to act on the analytics received to generate relevant offers to customers. Because the real-time data processing application receives multiple files per minute from different source systems (2000 to 3000 transaction per minute at peak time) it is extremely difficult for the real-time processing application to validate and generate offers in a very short timeframe, such as 15-20 seconds.


For this challenge, we used Vertica’s Big Data SQL Analytics Platform for batch processing and its in- memory database for lookups. All heavy SQLs were run in Vertica as a batch mode to update customer profiles, offers, terminals, merchants, business rules, etc. Additionally, every beginning of day (BOD) synch process updated the respective in-memory database tables.  As in-memory operation is faster and in the same cluster as the real-time data processing application, this helped to process a large amount of data per minute.


Challenge 4: Managing changing business rules efficiently.
Business units need the ability to continually change business rules as new products and services became available or other insights warrant the need for change. To reduce the impact of this issue, the bank wanted the system to be able to handle various scenarios with little or no interaction required by the tech team to modify the existing rules or to add new rules.


To achieve this requirement, we built a GUI application in Java for rule creation and modification. We also provided hooks for plugins for any special business case to make it simple to alter without needing technical assistance.


Challenge 5: Communicating with the customer through a different channel.
The bank has numerous channels its uses for communication with its customers, such as SMS gateway, email, and phone banking. Therefore, our solution needed to be able to access and utilize these different communication systems.


To address this issue, we wrote java plugins for each communications systems and embed each with a real-time processing application so that when the matching offer was selected, a personalized offer message is generated, which could be sent to the customer through one or more channels preferred by the customer.


How the System Works
With these challenges addressed, here’s a brief look at how the real-time analytics solution we developed works as an overall process:


  • Implement a CDC process on required source systems to get latest feed at near real time.
  • Create customer profiles on transactional behavior based on the last one year of data.
  • Load scripts for all master data (offers, terminals, and merchants, etc.)
  • Implement rea-time data processing application for stream processing, rule configuration/application and final offer generation for customer.
  • Audit logs.


Process Flow Diagram
The diagram below shows a visual representation of how we applied our solution to achieve real-time analytics and offer generation.


Real Time Spend Process Flow


While robust implementation and accurate business rules will inevitably generate less offers, these robust rules lead to better outcomes. Customers receive only the most personalized and relevant offers. This not only helps the bank establish a deeper relationship with the customer, it also helps eliminate customers receiving unwanted offers, which could have a negative impact on the customer/bank relationship.


A real-time data processing system empowers businesses, such as financial institutions, to capitalize on live data in near real time.  As noted in this post, the banks real-time data processing application enables various business units at the bank to define their business rules for new offers or campaigns for new products or services. This allows banks to generate additional revenue for specific products or services and provides customers with valuable offers they find relevant and appreciate.


Finally, the application of real-time analytics can also provide an opportunity to implement new business cases and broaden its return on investment. For example, the real-time service recovery of customer complaints or other service related issues.


If you’d like to learn more about how real-time analytics can help your financial institution or business boost its revenue and deepen customer relationships, get in touch. We have the experience and deep expertise to deliver real-time analytic solutions in even the most complicated business environments.