First Party data integration solution built for marketing teams to enable audience and conversion onboarding into Google Marketing products (Google Ads, Campaign Manager, Google Analytics).
APACHE-2.0 License
Development of new Megalista features has ceased, and Megalista will be archieved after December 31st 2024. Existing customers are encouraged to migrate available use cases in platform through Google Ads Data Manager (GADM), or Tightlock. Tightlock will continue to support customers for all use cases that are not covered by GADM. If you would still like to use Megalista after December 31st 2024, please fork the solution if incremenetal updates are required (e.g to address potential bugs and API upgrades).
Disclaimer: This is not an officially supported Google product.
Sample integration code for onboarding offline/CRM data from BigQuery as custom audiences or offline conversions in Google Ads, Google Analytics 360, Google Display & Video 360, and Google Campaign Manager.
Google Ads
Google Analytics (Universal analytics)
Campaign Manager
Google Analytics 4
Display & Video
Appsflyer
Megalista was designed to separate the configuration of conversion/audience upload rules from the engine, giving more freedom for non-technical teams (i.e. Media and Business Intelligence) to setup multiple upload rules on their own.
The solution consists of #1 a configuration environment (either Google Sheet or JSON file, or a Google Cloud Firestore collection) in which all rules are defined by mapping a data source (BigQuery Table) to a destination (data upload endpoint) and #2, an Apache Beam workflow running on Google Dataflow, scheduled to upload the data in batch mode.
Those are the minimum roles necessary to deploy Megalista:
Required APIs will depend on upload endpoints in use.
Megalista can be configured via Google Sheets, a JSON file, or a Google Cloud Firestore collection. Expected data schemas (Sources) and metadata (Destinations) for each use case are defined in the Megalista Wiki.
Instructions for each configuration method method can be found in the Megalista wiki
This guide assumes it'll be followed inside Google Cloud Platform Console.
To access campaigns and user lists on Google's platforms, this dataflow will need OAuth tokens for an account that can authenticate in those systems.
In order to create it, follow these steps:
./generate_megalista_token.sh client_id client_secret
Download the latest Megalista code.
To deploy the full Megalista pipeline, use the following command from the deployment folder:
./deploy.sh
The script will require some parameters, please add them to the config.json file. Some parameters have default values and can be
changed.
Auxliary bigquery dataset for Megalista operations to create
Google Cloud Storage Bucket to create
Setup Firestore collection, URL for JSON configuration and Setup Sheet Id
Client ID, Client Secret, Access Token and Refresh Token from the previous step.
Disclaimer: Please store your config.json file in a secure place or delete it after the deployment.
To update the binary without redoing the whole deployment process, run:
Every upload method expects as source a BigQuery data with specific fields, in addition to specific configuration metadata. For details on how to setup your upload routines, refer to the Megalista Wiki.
To have uploaders errors captured and sent by email, do the following:
In Cloud Scheduler, in the parameters
section of the request body, add notify_errors_by_email
parameter as true
and errors_destination_emails
with a list of emails divided by comma ([email protected],[email protected]
etc).
These parameters should be added to the same list of pre-configured ones, such as client_id
, client_secret
etc.
If the access tokens being used were generated prior to version v4.4
, new access and refresh tokens must be generated to activate this feature. This is necessary because old tokens don't have the gmail.send
scope.
Calls to the Google Ads API will fail if the user that generated the OAuth2 credentials (Access Token and Refresh Token) doesn't have direct access to the Google Ads account to which the calls are being directed. It's not enough for the user to have access to a MCC above this account and being able to access the account through the interface, it's required that the user has permissions on the account itself.