Refined an ETL Project for Flawless Performance and Seamless Data Workflows: Scalista Case

Client

Scalista is an Austrian online and mobile marketing agency that helps organizations grow in a very complex landscape. They have a proprietary project, Cloudista, for data harvesting.

Problem

They failed to connect Google BigQuery to Tableau to build reports. The project had to be modified due to the updates of used Python libraries. QA testing was required for the project’s flawless performance.

Results

The problem of the failed connection between BigQuery and Tableau was detected and eliminated. A new approach to building Tableau reports was developed. Project maintenance for flawless performance was completed. The Client’s users receive correct reports in a timely manner.

Technologies

Our Tech Stack

Goal

  • The Client’s project, Cloudista, which is used for data harvesting through cloud technologies, failed to perform correctly. The Python-based project is an ETL that exports marketing data from Scalista users’ accounts to GCP BigQuery. The data should be used to build Tableau reports, but the process kept failing.
  • Scalista’s specialist left the company, and they had no more resources to solve the task. The Client turned to Valiotti because of our experience in the tech stack (Python, Google BigQuery, Google Sheets) and process automatization.
  • During the project, two new tasks were set: to QA Cloudista’s performance and revise the ETL project.

Results

1. We designed a new approach to data transfer from BigQuery to Tableau via Google Sheets.

Firstly, we conducted research to find out the root of the problem—why it was impossible to connect BigQuery to Tableau. The reason was that the system used a custom request with a lot of calculated fields to enable the connection. It failed to deliver due to a limited request length.

To solve the problem, we developed a new approach: with the help of a custom request, the data from BigQuery was regularly exported to Google Sheets. Then, Google Sheets was easily connected to Tableau, which allowed for building analytical reports.

case study pipeline

Here is an example of a Google Sheet data mart:

Google Sheets Data Mart Example

2. The Cloudista performance was refined.

During the project, we discovered some ETL-related problems. Since the project is coded in Python, and the versions of the used libraries were updated, we needed to make the appropriate changes. After studying the code, project documentation, technologies, and the libraries’ changelogs, we edited the code for flawless performance.

What’s more, we compared the data from Facebook and Google Ads with the data sourced by the ETL project. Some modification of data processing was required.

Tips

  • Maintain a dialog between a data analyst and the Client. It’s best if the Client provides as many details as possible and formalizes the tasks.
  • When a vendor is dealing with an already-developed project, provide project documentation. It will help them understand the context much better.

Learn How Data Insights Can Benefit Your Business

Wondering what value data insights can bring your business? Get in touch, and we'll answer your questions!

Contact Us

Technologies

Our Tech Stack

Other Case Studies

  • AI Sales

    Advanced analytics boosted AI Sales with revenue tracking, AI vs. human performance insights, real-time alerts, and improved bot strategies for higher efficiency.

    Read more

  • Buff

    Regained focus on business KPIs with dashboards transforming data into revenue.

    Read more

  • betPawa

    A Flexible and Scalable DWH system Re-Built from Scratch with Improved Data Processing Time and Quality

    Read more