ETL Processes Became 5 Times Faster as a Result of a Refined Data Warehouse: Twinero Case

Client

Twinero is a Spanish fintech company that offers short-term microloans to Spanish citizens. It focuses on rapid micro-lending with no paperwork.

Problem

The existing ETL processes were to be reviewed and modernized with Python. A new analytical repository was to be built.

Results

An optimized analytical warehouse with an improved reporting process and a reduced number of errors when creating reports.

Technologies

Our Tech Stack

Rostyslav Klopochuk

Country Manager, Twinero

"Valiotti Analytics helped us build a new data pipeline from scratch, replacing legacy tools such as Pentaho IDE with modern ones. They suggested the implementation of a modern data warehouse architecture for analytics and tuned instances to work with software. They also reacted quickly and introduced a huge number of improvements in the data analysis structure of our company. Moreover, several useful analytic dashboards were built for us during the project. We appreciate their approach as they dove deeply into the crucial details and understood our business constraints to react to data failure immediately. In order to support the new data stack, the team helped us build a monitoring system for the data routine. We are very pleased to work with Valiotti Analytics."

Goal

Twinero’s DWH (data warehouse) was built with ETL (extract, transform, load) processes based on Pentaho IDE. In terms of the existing infrastructure, Windows-generated files (.xml files) were run under Unix. The approach was outdated and needed to be refined and modernized with Python. Besides this task, Valiotti Analytics also built a new analytical repository to enhance reporting.

Results

  1. Refined ETL processes with accurate outputs run five times faster than before

To optimize the analytical warehousing, we modernized the infrastructure.

Refined ETL

We combined Python frameworks with DBT as an ETL tool and set up orchestration through Apache Airflow. As a result, SQL queries became faster and database tables were made five times more quickly.

Apache Airflow

As a result, all the reports were rewritten, and tables were normalized to align the data for further analytics.

  1. Enhanced reporting for valuable insights on the Client’s lifecycle and borrowing

We built new dashboards in Metabase and Tableau. In terms of the new data infrastructure, the Client was able to easily access and analyze historical data to get a better understanding of business specifics and trends. What is more, the customers were segmented based on payment categories, overdue period and scope, etc.

Now, Twinero can get better insights into its customers and do this in a more transparent way. For example, they can monitor the scope of users within a specific timeframe based on different categories and better analyze the overall situation.

Tips

  • Perform reverse-engineering of all the created objects to find out how they work.
  • Abandon legacy code for an up-to-date solution for enhanced flexibility, performance, and scalability.
  • Data processing is best organized as separate manageable blocks. In this case, the engineering process is presented by a so-called pipeline or a consecutive transformation of raw data into manageable blocks. Such an approach allows spotting if there was a problem at each stage, as well as if the stage was implemented or not.

Learn How Data Insights Can Benefit Your Business

Wondering what value data insights can bring your business? Get in touch, and we'll answer your questions!

Contact Us

Technologies

Our Tech Stack

Rostyslav Klopochuk

Country Manager, Twinero

"Valiotti Analytics helped us build a new data pipeline from scratch, replacing legacy tools such as Pentaho IDE with modern ones. They suggested the implementation of a modern data warehouse architecture for analytics and tuned instances to work with software. They also reacted quickly and introduced a huge number of improvements in the data analysis structure of our company. Moreover, several useful analytic dashboards were built for us during the project. We appreciate their approach as they dove deeply into the crucial details and understood our business constraints to react to data failure immediately. In order to support the new data stack, the team helped us build a monitoring system for the data routine. We are very pleased to work with Valiotti Analytics."

Other Case Studies

  • Data Strategy for a $6M Pet-Tech Marketplace

    How a fast-growing pet-tech marketplace went from zero data infrastructure to a comprehensive data strategy in 4 weeks. Fractional CDO case study.

    Read more

  • From Power BI to Metabase

    Our client needed to migrate slow, unstable Power BI dashboards to Metabase without changing their underlying data infrastructure. We rebuilt key sales and financial dashboards with better filtering, clearer visualizations, and improved performance, resulting in faster load times and dashboards that are actually usable at scale.

    Read more

  • Non-Profit Dual-Dashboard Analytics

    A government-funded nonprofit in Canada, needed analytics that worked for both daily operations and board-level reporting. We replaced manual Google Sheets with a cost-efficient data stack and built two distinct dashboards—one interactive and real-time for the team, and one static, branded snapshot for sponsors—so every stakeholder got exactly the data they needed.

    Read more