Audience Data Generator

A script to create dummy data for Audience Studio on Treasure Data.

You can create a database (if not exists) including the following tables:

• users
• cities
• behavior_1
• behavior_2
• attribute_1
• attribute_2

How to use

Set up

Prerequisites:

• Python 3.8+

We recommend to install after creating a virtual environment as the following:

$ python3 -m venv .venv
$ source .venv/bin/activate
(.venv)$ pip install git+https://github.com/chezou/audience_generator

Or, you can install with --user option.

$ pip install --user git+https://github.com/chezou/audience_generator

Usage

Set TD_API_KEY for your master API key and TD_API_SERVER for your API endpoint as environment variables.

$ export TD_API_KEY="1234/XXXXXXXXXX"
$ export TD_API_SERVER="api.treasuredata.com"
$ audience_generator my_db

You can see detailed options with --help option.

Usage: audience_generator [OPTIONS] DATABASE

  Create dummy data for Audience Studio in a database.

  Target tables are: users, cities, behavior_1, and behavior_2.

  Target database will be created automatically if not exists.

Options:
  -s, --api-server TEXT    Treasure Data API Endpoint
  -n, --user-size INTEGER  Target order of generated users. Must be bewteen 11
                           to 100000000

  -o, --overwrite          Recreate target tables
  -d, --dry-run            Check query with dry run. Set -vv to show query.
  -v, --verbose
  --help                   Show this message and exit.

The dummy data examples

users table

This table consists of time, td_client_id, email, and country columns.

• email column can be a, b, or c randomly
• country can be japan, usa, canada. The ratio of them is japan:usa:canada = 2:3:1.

The example table looks like this:

cities table

This table consists of name, cn, fanoutn, and time.

The example table looks like this:

behavior_1 table

This table consists of time, test_city_name, and td_client_id.

• By using td_client_id, you can join with users table
• You can join with cities table by joining behavior_1.test_city_nameandcities.name`.

The example table looks like this:

behavoir_2 table

This table consists of time, test_city_name, td_client_id, and opts.

• opts can be an integer from 0 to 2
• By using td_client_id, you can join with users table
• You can join with cities table by joining behavior_2.test_city_nameandcities.name`.

The example table looks like this:

attribute_1 table

This table consists of time, td_client_id, country, td_os, and td_language.

• country is the same field as users table
• td_os contains Linux, Windows, macOS, iOS, and Android. The ratio is 1:1:1:2:2.
• td_language contains ja_JP, en_GB, and en_US. The ratio is 2:1:1.

attribute_2 table

This table consists of td_client_id, age, item_count, ctr, and time.

• td_client_id is nullable. The ratio is determined by non_null_rate option
• age is a random number which can be from 0 to 60
• item_count is a random number which can be from 0 to 5
• ctr is a random number from 0.0 to 1.0