Vis Talk: A JavaScript toolkit for Natural Language-based Visualization Authoring

Vis Talk is a JavaScript library for developers create visualization using natural language.  

Try it out

You can try the Playground Web App or fork example Observable Notebook

Playground Demo:

For Developers

Installing

Install using yarn:

$ yarn add @vis-talk/vega-builder

or install using npm:

$ npm install @vis-talk/vega-builder

Build a simple react sample app

$ npx create-react-app my-vis-app --template typescript
$ cd my-vis-app
$ npm install @vis-talk/vega-builder react-vega

Modify src/App.tsx to:

import React from 'react';
import { createBuilder } from "@vis-talk/vega-builder";
import { VegaLite } from "react-vega";

function App() {
  const table = [
    { Brand: "BrandA", Category: "SUV", Sales: 40 },
    { Brand: "BrandB", Category: "SUV", Sales: 20 },
    { Brand: "BrandC", Category: "SUV", Sales: 30 },
    { Brand: "BrandD", Category: "SUV", Sales: 10 },
    { Brand: "BrandA", Category: "Midsize", Sales: 40 },
    { Brand: "BrandB", Category: "Midsize", Sales: 10 },
    { Brand: "BrandC", Category: "Midsize", Sales: 20 },
    { Brand: "BrandD", Category: "Midsize", Sales: 5 },
  ];
  
  const builder = createBuilder(table);
  
  builder.setInput([
    "total sales by brand",
    "highlight midsize in orange",
    "add line in 60 in red",
    "add rect from 12 to 37 in green"]);
    
  const spec = builder.build({ name: "table" });

  return (
    <div className="App">
      <VegaLite spec={spec} data={{ table }} />
    </div>
  );
}

export default App;

start the app:

$ npm start

API Reference

createBuilder(dataSource)

Create a new vega visual builder by load your data table.

function createBuilder(dataSource: DataSource): VegaBuilder;

// data source is array of object, with optional column name list.
export interface DataSource extends Array<object> {
  // List of column names.
  columns?: Array<string>;
}

For example, you can create a session using inline records data like:

const builder = createBuilder([
  { Brand:'BrandA', Category: 'SUV', Sales: 100 },
  { Brand:'BrandB', Category: 'SUV', Sales: 200 },
  { Brand:'BrandC', Category: 'SUV', Sales: 300 }
])

session.setInput(input)

This method allow you specify the natural language input using a single block of text or a multi-line string arry

class VegaBuilder {
  // Provide your natural language input
  public setInput(lines: string[]);
  ...
}

For example:

builder.setInput(['total sales by brand', 'sort it'])

builder.build

Generate vega-lite spec as a javascript JSON object (Type defined in Vega-Lite package).

let spec = builder.build({name: 'table'});

Render Vega Chart:

$ yarn add vega vega-lite react-vega

or

$ npm install vega vega-lite react-vega

import { VegaLite } from "react-vega";

<VegaLite spec={spec} data={{table: rows}} />

More examples - donut chart

builder.setInput([
  'sales by brand as donut chart'
])

More examples - column chart with highlights

builder.setInput([
  'sales by brand as column chart',
  'sort desc',
  'highlight top 2 in green',
  'add line 100 in red',
  'hide grid',
  'make data point wider'
])

Build from source code

Enlist code

$ git clone https://github.com/microsoft/VisTalk.git
$ cd VisTalk

Build & Run

$ yarn
$ yarn build
$ yarn start

Then open browser and navigate to http://localhost:4200/

Run Unit Tests

$ yarn test

Run E2E Tests

$ yarn e2e

Then you can explore captured screenshots and videos from /dist/cypress/apps/playground-e2e

Build package

$ yarn package

Train deep learning model

Setup miniconda

Install Miniconda

Setup Environment

cpu:

$ conda create -n vis-talk python=3.9.13 tensorflow=2.5.0  
$ conda activate vis-talk
$ pip install tensorflow-addons==0.13 tensorflowjs==3.18 seqeval==1.2.2

gpu:

$ conda create -n vis-talk-gpu tensorflow-gpu=2.1.0 python=3.6.12 cudatoolkit=10.1
$ conda activate vis-talk-gpu
$ pip install tensorflow-addons==0.9.1 tensorflowjs==3.8 seqeval==1.2.2

Train Model

$ yarn train

Manually replace the generated model-data.ts to libs/interpreter/src/lib/model-data.ts

Feedback

If you have any questions, feel free to open an issue or contact us: Vis Talk Team.

Citation

If you use VisTalk in your research, please cite as follows:

Y. Wang et al., "Towards Natural Language-Based Visualization Authoring," in IEEE Transactions on Visualization and Computer Graphics, 2022, doi: 10.1109/TVCG.2022.3209357.

License

This project is licensed under the MIT License.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.