A helper package for Google BigQuery operations
MIT License
BigQuery Helper is a Python package that simplifies working with Google BigQuery. It provides an easy-to-use interface for common BigQuery operations such as executing queries, inserting rows, and working with DataFrames.
You can install BigQuery Helper using pip:
pip install bigquery-helper
To use BigQuery Helper, you'll need a Google Cloud project and credentials. Here's how to get started:
Set up a Google Cloud project and enable the BigQuery API.
Create a service account and download the JSON key file.
Set the GOOGLE_APPLICATION_CREDENTIALS
environment variable to the path of your JSON key file:
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/keyfile.json"
Alternatively, you can pass the credentials directly when initializing the BigQueryHelper.
Here's a quick example of how to use BigQuery Helper:
from bigquery_helper import BigQueryHelper
# Initialize BigQueryHelper
bq_helper = BigQueryHelper(project_id="your-project-id", credentials="/path/to/your/keyfile.json")
# Execute a query and get results as a DataFrame
query = "SELECT * FROM `your_dataset.your_table` LIMIT 10"
df = bq_helper.query_to_dataframe(query)
print(df)
# Insert rows into a table
rows_to_insert = [
{"column1": "value1", "column2": 123},
{"column1": "value2", "column2": 456}
]
bq_helper.insert_rows("your_dataset.your_table", rows_to_insert)
# Upload a DataFrame to BigQuery
import pandas as pd
df_to_upload = pd.DataFrame({"column1": ["a", "b"], "column2": [1, 2]})
bq_helper.dataframe_to_table(df_to_upload, "your_dataset.new_table", if_exists="replace")
# Update a column in a table
bq_helper.update_column("your_dataset.your_table", "id_column", "status_column", [1, 2, 3])
# Get schema of a table
schema = bq_helper.get_schema("your_dataset.your_table")
print(schema)
Contributions are welcome! Please feel free to submit a Pull Request.
This project is licensed under the MIT License.