🦭 Gdańsk AI 🦭
Gdańsk AI is a full stack AI voice chatbot (speech-to-text, LLM, text-to-speech) with integrations to Auth0, OpenAI, Google Cloud API and Stripe - Web App, API and AI
It uses Stripe to provide payments infrastructure, so users can buy bibs
- the API tokens. Auth0 is used to provide authentication and authorization. You can use them for free - Auth0 has a free tier up to 7000 users per month. Stripe only charges fees from purchases, so you have no ongoing costs from any of them.
If you want to use these services, you have to set up your Stripe and Auth0 accounts and put API keys into .env
in web-api/
directory.
Node.js, TypeScript and Express are used here
.env_example
shows which values you have to set
CHATBOT_API_KEY={whatever you decide; this value is reused among ai-api, web-api and web in this project}
OPEN_AI_API_KEY={go to platform.openai.com, generate and copy-paste api key here}
# Google Cloud API
type=service_account
project_id={project id}
private_key_id=
private_key={multiline private key with structure like this:-----BEGIN PRIVATE KEY-----...-----END PRIVATE KEY-----\n}
client_email={project name}@{project id}.iam.gserviceaccount.com
client_id=
auth_uri=https://accounts.google.com/o/oauth2/auth
token_uri=https://oauth2.googleapis.com/token
auth_provider_x509_cert_url=https://www.googleapis.com/oauth2/v1/certs
client_x509_cert_url=https://www.googleapis.com/robot/v1/metadata/x509/{project-name}%40{project-id}.iam.gserviceaccount.com
universe_domain=googleapis.com
# Stripe
STRIPE_SECRET_KEY={sk_...}
STRIPE_PUBLISHABLE_KEY={pk_...}
TOKENS_PRICE_ID={price_some characters here - you need to create a price in Stripe for tokens first and then copy-paste it here}
# TOKENS_PER_TRANSACTION defines how many interactions user can make with AI per single bibs purchase
TOKENS_PER_TRANSACTION=5
# Auth0
AUTH0_ISSUER_BASE_URL='https://{tenant-name}.us.auth0.com'
AUTH0_DOMAIN='{tenant-name}.us.auth0.com'
AUTH0_CLIENT_ID=
AUTH0_CLIENT_SECRET=
AUTH0_AUDIENCE="https://{tenant-name}.us.auth0.com/api/v2/"
AUTH0_SCOPE="openid profile email offline_access"
CLIENT_URL={url pointing to where web is deployed}
AI_API_URL={url pointing to where ai api is deployed}
web-api
directorynpm install
or npm ci
npm run build
npm run start
or npm run dev
to run it with hot-reloadIt works in Google Chrome currently. You can press Gdańsk AI logo to record audio, then press it once again to send it to Web API. After a successful response, a received audio is played to you.
Next.js used here with Vercel's template
.env_example
shows which values you have to set
CHATBOT_API_KEY={whatever you decide; this value is reused among ai-api, web-api and web in this project}
CHATBOT_API_URL={url to where api is deployed}
NEXT_PUBLIC_API_URL={url pointing to where api is deployed}
API_URL={url to where api is deployed}
# Auth0
NEXT_PUBLIC_AUTH0_ISSUER_BASE_URL="https://{tenant-name}.us.auth0.com"
NEXT_PUBLIC_AUTH0_CLIENT_ID=
AUTH0_SECRET=
AUTH0_BASE_URL={url pointing to where web is deployed}
AUTH0_ISSUER_BASE_URL='https://{tenant-name}.us.auth0.com'
AUTH0_DOMAIN='{tenant-name}.us.auth0.com'
AUTH0_CLIENT_ID=
AUTH0_CLIENT_SECRET=
AUTH0_AUDIENCE="https://{tenant-name}.us.auth0.com/api/v2/"
AUTH0_SCOPE="openid profile email offline_access"
CLIENT_URL={url pointing to where web is deployed}
web
directorynpm install
or npm ci
npm run build
stripe listen --forward-to http://localhost:3000/webhook
(adjust the port from 3000 if you've changed it) - we need this to invoke /webhook
endpoint on bibs purchasesnpm run start
screen_hint: "signup",
in signup.ts
OpenAI and Google Cloud API are used here to provide speech-to-text (Whisper-1), LLM (gpt-3.5-turbo) and text-to-speech (Google Cloud TextToSpeechClient) services.
Python and FastAPI here
.env_example
shows which values you have to set
CHATBOT_API_KEY={whatever you decide; this value is reused among ai-api, web-api and web in this project}
OPEN_AI_API_KEY={go to platform.openai.com, generate and copy-paste api key here}
# Google Cloud API start
type=service_account
project_id={project id}
private_key_id=
private_key={multiline private key with structure like this:-----BEGIN PRIVATE KEY-----...-----END PRIVATE KEY-----\n}
client_email={project name}@{project id}.iam.gserviceaccount.com
client_id=
auth_uri=https://accounts.google.com/o/oauth2/auth
token_uri=https://oauth2.googleapis.com/token
auth_provider_x509_cert_url=https://www.googleapis.com/oauth2/v1/certs
client_x509_cert_url=https://www.googleapis.com/robot/v1/metadata/x509/{project-name}%40{project-id}.iam.gserviceaccount.com
universe_domain=googleapis.com
PORT=9000
ai-api
directoryCreate an account, then a new tenant. You will need to set up:
Applications
APIs
Create an account, turn the test mode unless you want to run it on production and process real payments, go to Developers tab, then API keys. This is where you can find values for STRIPE_SECRET_KEY and STRIPE_PUBLISHABLE_KEY. Now go to Products tab and Add product which represents a single purchase of bibs (one bib - one interaction with AI for a user). Once you create it, copy priceId and put it into .env as TOKENS_PRICE_ID.
Create an account, go to platform.openai.com, generate API key, use it in .env files
Create an account, go to IAM & Admin, then Service Accounts and create a project - you need id of the project to put into .env files. Then you need to generate json with credentials and extract the values from the file to your .env files. Go to Create access credentials page (you can use search engine to find a link). Then Keys -> Add Key -> Create new key -> JSON -> Create. You should get credentials.json with all values you need to .env. Finally, you need to enable text to speech API service and you're ready
Thanks to Jeff McJunkin for improving the README by going through the config process from the scratch and filling the gaps in the docs
GNU GPL v2
If you use this software in your research, please use the following citation:
@software{Maczan_Gdansk_AI_2023,
author = {Maczan, Jędrzej Paweł},
title = {{Gdańsk AI - Full stack voice chatbot}},
url = {https://github.com/jmaczan/gdansk-ai},
year = {2023},
publisher = {GitHub}
}
Made in Gdańsk, Poland in 2023 by Jędrzej Paweł Maczan