betPawa is a trusted African sports betting provider licensed in Cameroon, Kenya, Uganda, Nigeria, Tanzania, Ghana, and some other African countries. The Client deals with a lot of OLTP (Online Transaction Processing) data in real-time.
betPawa’s DWH (Data Warehouse) system lacked reliability and flexibility and required significant effort to support it. For example, the ETL processes that worked nightly failed to deliver the results by morning, and reports lacked some OLTP or real-time data.
A DWH system with a new Schema was built from scratch for it to match relevant business processes. It’s fast, can be scaled 5-10 times, and allows users to come up with complicated reports for more informed decision-making.
To ensure the reports contain up-to-date information, we have deployed a new BI tool and rebuilt the ETL process as Change Data Capture (CDC).
You need custom analytics if
betPawa is a sports betting company that manages tons of OLP data for bet/game processing. They used a DWH system with ETL processes that operated around the clock. Yet, the system lacked stability and reliability. Here are some of the problems that undermined data management:
Since the Client lacked both the expertise and human resources to overcome the challenge, they turned to Valiotti Analytics as a reliable partner with extensive experience in BI reporting, DWH, and ETL design.
After analyzing the existing DWH system, source data structure, and actual business processes, we started to build a new Schema design. Here are some key achievements:
The Schema was based on Kimball’s dimensional approach and equipped with the following:
We needed to set up data transfer from OLTP systems and tested four methods:
Part of the corporate data was processed directly in Kafka without an SQL intermediary. The other part, available as MySQL tables, was sent to Kafka topics by MaxWell’s Daemon tool. This allowed not only for standardized data processing but also for real-time updates.
To avoid data deduplication, we enhanced fact and dimensional table processing. If the data is duplicated because Kafka offsets rewind or there is another failure at the data processing stage, Stage tables can’t be used: Clickhouse and Kafka topics/partitions have parallel processing, and the same data can be inserted several times because Stage tables are used as queues with generated IDs. To deduplicate, we either check the inserted data at the Insert Transform stage or process transformation inserts so that a Clickhouse table engine spots duplicates by checksum.
Before deploying Redash, we conducted several rounds on a test server. After we trained and collected how-tos, they were sent to DevOps to build a production version. When redesigning the process and coming up with new and relevant reports, we first created them in Tableau and then recreated them in Redash. Once the testing and bug-fixing stage was done, we provided access to the Client.
That's how Kirill Vassiljev, Head of Business Intelligence, Betpawa, described our work together on this project:
Valiotti Analytics has wonderful people and professionals in their team. Our company required technical assistance in maintaining and developing the existing data warehouse. Data experts from Valiotti quickly grasped the essence of the project and since then the team has been the strongest support for our analytics department. Many tests have been successfully passed and no less awaits us ahead. We sincerely look forward to further collaboration with such an excellent team. Thanks very much!
If you need help with selecting or creating data analytics tools, feel free to contact us.
You can write directly to our CEO or submit a request on the site. We are looking forward to our collaboration!
Learn more about our case studies
This website uses cookies to ensure you get the best experience on our website. Learn more >