GraphScope

🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统

APACHE-2.0 License

Downloads
12K
Stars
3.1K
Committers
63
View Code on GitHub Visit Website
Ecosystems: C++

Bot releases are visible (Hide)

GraphScope - Release v0.27.0 Latest Release

Published by lidongze0629 7 months ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.27.0 release. This release contains many important features and improvements to GraphScope FLEX、gsctl command-line utility、Graph Interactive Engine (GIE) and Groot persistent storage.

We highlight the following improvements included in this release:

1. gsctl command-line utility

gsctl is a command-line utility designed to simplify the usage of GraphScope. With gsctl, users can easily interact with GraphScope Flex product, such as Interactive(for high QPS queries) and Insight(for online BI analysis), build binaries and docker images, and perform various utility tasks related to GraphScope. See more detailed informaton from doc.

Key Features of gsctl

  • Utility Scripts: gsctl provides a collection of utility scripts that simplify common tasks such as installing dependencies, building binaries and docker images, and running test suites.
  • Client/Server Mode: With the client/server mode, users can connect to a launched coordinator and interact with specific GraphScope products behind it. This mode enables managing sessions, resources, and other product-specific functionalities.
  • Scope Management: gsctl allows users to switch between global and local scopes, enabling them to work with graphs and resources at different levels. This flexibility enhances productivity and enables efficient management of graph computation tasks.
  • Configuration Management: gsctl includes configuration options that can be customized to suit specific requirements. Users can specify a custom coordinator endpoint and manage other configuration settings as needed.

2. GraphScope FLEX

  • Support build docker images through gsctl flexbuild command.
$ git clone https://github.com/alibaba/GraphScope.git && cd GraphScope
$ python3 gsctl.py flexbuild insight/interactive --app docker
  • Support deploy Interactive and Insight product through docker image.
$ docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex
# launch engine and coordinator service: 12312 gremlin endpoint, 55556 grpc endpoint, 8080 coordinator endpoint
$ docker run -p 12312:12312 -p 55556:55556 -p 8080:8080 registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0-flex /usr/local/groot/bin/start_local_cluster.sh
  • Connect to a launched coordinator and interact with the products
graphscope@host:/work/python$ gsctl connect --coordinator-endpoint http://127.0.0.1:8081
[SUCCESS] Connected to http://127.0.0.1:8081, coordinator is serving with GRAPHSCOPE_INSIGHT mode.

Try 'gsctl --help' for help.

graphscope@host:/work/python$ gsctl ls
schema
├── vertex types
│   ├── person
│   │   ├── Property(name: id, type: LONG, is_primary_key: True)
│   │   ├── Property(name: name, type: STRING, is_primary_key: False)
│   │   └── Property(name: age, type: LONG, is_primary_key: False)
│   └── software
│       ├── Property(name: id, type: LONG, is_primary_key: True)
│       ├── Property(name: name, type: STRING, is_primary_key: False)
│       └── Property(name: lang, type: STRING, is_primary_key: False)
└── edge types
    ├── (person) -[created]-> (software)
    │   ├── Property(name: edge_id, type: LONG, is_primary_key: False)
    │   └── Property(name: weight, type: DOUBLE, is_primary_key: False)
    └── (person) -[knows]-> (person)
        ├── Property(name: edge_id, type: LONG, is_primary_key: False)
        └── Property(name: weight, type: DOUBLE, is_primary_key: False)

3. Enhancements for Groot

  • Integrate a prototype of OpenTelemetry into Groot, users could switch on option trace.enabled=true, which will have a jaeger container that collect traces. And it will serve a website on POD_IP:16686.

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.27.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.27.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.27.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.27.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.27.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.27.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.26.0...v0.27.0

GraphScope - Release v0.26.0

Published by lidongze0629 9 months ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.26.0 release. This release contains many important features and improvements to GraphScope FLEX and Graph Interactive Engine (GIE), including secondary instance and type inference in GIE, loading from distributed data source and Macs deployment in FLEX Interactive.

We highlight the following improvements included in this release:

1. FLEX Interactive

  • Support use GraphScope Interactive on macos. Download the latest release to give it a shot interactive-latest
  • Support bulk loading with odps table as data source. Here is an example
graph: dd_graph
loading_config:
  data_source:
    scheme: odps  # file, odps
  import_option: init # init, overwrite
  format:
    type: arrow
vertex_mappings:
  - type_name: Person  # must align with the schema
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
  column_mappings:
      - column:
          index: 0  # can be omitted if the index is the same as the property index
          name: user_id  # can be omitted if the name is not known
        property: id
edge_mappings:
  - type_triplet:
      edge: Knows
      source_vertex: Person
      destination_vertex: Person
    inputs:
      - your_proj_name/table_name/partition_col_name=paritition_name
    source_vertex_mappings:
      - column:
          index: 0
          name: src_user_id
        property: id
    destination_vertex_mappings:
      - column:
          index: 1
          name: dst_user_id
        property: id

2. Enhancements for GIE

  • Support secondary instance in Groot. It enables user open another read-only instance along with the original instance, providing additional read qps to users.

    • How to use: Set secondary.enabled=true in helm charts to enable secondary mode. Other options could remain unchanges with the primary instance.
    • Note in secondary instance it could only be queried upon, the modification of schema and data should be routed to primary instance.
    • See more details in the doc

  • Support automatic type inference in graph queries, which can affect query execution in two ways:

    • Checking if vertex and edge types in the graph conform to user-defined schemas 
      # Now compiler can infer that 'kno' is a nonexistent edge type and returns error
Match (a:person)-[b:kno]->(c) Return a, b, c;
    • Inferring vertex and edge types in the graph. 
      # Now compiler can infer a and b must both be person types, and execution can be more efficient
Match (a)-[:knows]->(b) Return labels(a), labels(b)
=> Match (a:person)-[:knows]->(b:person) Return labels(a), labels(b)

See more details in the doc

3. Other enhancements and bug fixes

  • Introduce graph_algo as another example graph. https://github.com/alibaba/GraphScope/tree/main/flex/interactive/examples/graph_algo.
  • Support String DT_STRING as edge property type.
  • Refactor Flex/CMakeLists.txt to support use installed flex via find_package(flex).
  • Fix decode error in Groot
  • Fix memory leak in Pegasus and data race in Groot
  • Fix GIE peers_contains bugs in count operator in subtasks
  • Fix Pegasus unexpect result of operator aggregate() + iterate()
  • Refine Log Printing Messages in Compiler
  • Fix bugs of Groot bulk loading when it retry to download source files
  • Fix bugs in GIE for edge filtering

Docker Images

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.26.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.26.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.26.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.26.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.26.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.26.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.25.0...v0.26.0

GraphScope - Release v0.25.0

Published by lidongze0629 11 months ago

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.25.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), FLEX Interactive, and bug fix.

1. Interactive under GraphScope Flex Architecture

  • Release GraphScope Interactive.
  • Enrich the movie graph schema and example query statements.
  • Implement a query cache to avoid recompiling the same adhoc query.
  • Support using the property of string type as primary key for vertex.

2. Enhancements for GIE

  • Support get disk status in groot， user could retrieve the total space and free space of the store pod via Python/Java client. Example API is here
  • Enhancements to count(): We've implemented two primary optimizations to accelerate the count() operation:
    • Optimized Counting of Vertices/Edges: Operations like g.V().count() that previously required a complete enumeration of vertices followed by a count, are now streamlined into a single fused operator. This operator efficiently counts vertices or edges in tandem with their iteration.
    • Improved Worker Utilization: The prior count() implementation was limited to using a single worker per machine for counting tasks, irrespective of the available configured workers. Now, count() operations are distributed across all configured workers, leveraging parallel processing to boost performance.
  • Users can now interact with GIE using natural language. We've integrated the capability to call OpenAI-compatible APIs within GIE, enabling the translation of human language inputs into Cypher queries. This advancement allows for a more intuitive and accessible query experience. For instance, users can utilize our Python API to execute the following code:
from graphscope.langchain_prompt.query import query_to_cypher
endpoint = "https://xxx" # use your endpoint 
api_key = "xxx" # replace to your own api key
question = "贾宝玉是谁的儿子?"
cypher_sentence = query_to_cypher(graph, question, endpoint=endpoint, api_key=api_key)
print(cypher_sentence)

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.25.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.25.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.25.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.25.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.25.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.25.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.24.0...v0.25.0

GraphScope - Release v0.24.0

Published by lidongze0629 about 1 year ago

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.24.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), Learning Engine(GLE), and Deployment.

We highlight the following improvements included in this release:

1. Enhancements for GIE

Key Features:

  • Support the sample() step of Gremlin to randomly pick up a given number of traversers from a Gremlin traversal. Additionally, coin() step can now follow any step in a Gremlin traversal, instead of just V() and E(). For example,
g.V().sample(10)   // Sample 10 traversers from `g.V()`
g.V().out().coin(0.2)  // Randomly pick up 20% among all vertices' out neighbors
  • Support the union step of Gremlin to combine the traversers from multiple traversals. For example,
g.V().union(out(), out().out())  // the results would now be one-hop (out once) and two-hop (out twice) neighbors of all vertices.
  • Support the unfold step of Gremlin, which is a reversed operator of fold that flattens a collection of traversers into discrete elements. For example,
g.V().fold().unfold()  // must obtain `g.V()` again
  • Support the identity() step of Gremlin, which simply returns the current traversers, and is often used together with union step. For example,
g.V().union(identity(), out())  // return `V()` and its one-hop neighbors
  • Support isNull in the expr() for verifying wether a certain property has NULL value. For example,
g.V().where(expr("@.age isNull"))  // the vertices without the field of `age` will be pruned

But fixes:

  • Fix a bug of count() step followed by order() and limit() that can cause the gaia engine to abort unexpectedly. Now the following Gremlin query can be executed smoothly:
g.V().order().by('id', asc).limit(1).out().count()

2. Enhancements for Deployment

  • All GraphScope Images support both x86 and arm64 platform
  • This release includes the gsctl command-line utility for building and testing GraphScope. The gsctl tool provides several commands and options to streamline the development and deployment process. You can use the following command to install the graphscope-client package and then use gsctl:
$ pip3 install graphscope-client
$ gsctl --help

3. Other enhancements and bug fixes

  • Reduced the size of GRAPE-java shaded jar,reduce shaded jars' total size from 94MB to 58MB.
  • Introduce Hiactor-based High-QPS Engine for Flex.
  • Refine the schema definition for flex rt_mutable_graph
  • Fix the Py_None reference count issue in GLE
  • Fix the bug that GLE client only connects with a single server in distributed training
  • Update the GLE-related tutorials

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.24.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.24.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.24.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.24.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.24.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.24.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.23.0...v0.24.0

GraphScope - Release v0.23.0

Published by lidongze0629 over 1 year ago

We are thrilled to introduce a range of enhancements to GraphScope, with the GraphScope 0.23.0 release. This release encompasses significant features and improvements in Interactive Engine (GIE), GraphScope Flex, and Deployment.

We highlight the following improvements included in this release:

1. GraphScope Flex Technical Preview

GraphScope Flex represents the ongoing evolution of GraphScope. In this release, we're excited to introduce a technical preview of GraphScope Flex. It highlights a modular design that reduces resource and cost requirements while providing a seamless, user-friendly experience for flexible deployment. It's currently under active development, and we look forward to your feedback.

Key Features:

  • Modular Design: Assemble your stack much like LEGO blocks to customize your graph computing deployments.
  • Three-Layer Architecture: Components are organized into an application layer, execution layer, and storage layer.
  • Flexible Builds and Deployments: Use the flexbuild script to build a custom deployment tailored for your specific use case.

You can explore GraphScope Flex in the flex/ directory or through the released artifacts

  • graphscope_flex_db_cppsp_hiactor_mcsr,
  • graphscope_flex_olap_builtin_grape-cpu, and
  • graphscope_flex_gnn_gnnmodels_graphlearn_tensorflow_vineyard.so

packages for high-QPS interactive queries, graph analytics, and graph learning task use cases, respectively. Dive in and discover what GraphScope Flex has to offer!

2. Enhancements for GIE

  • Support the recording of both vertices and edges during path expansion. In the past, path expansion had the option to yield either all vertices or the end vertex of the path. However, due to the requirements of both Gremlin and our users, it's essential to also retrieve the edges of the path, besides the vertices. To activate this feature, use with('RESULT_OPT', 'ALL_V_E') in the path expansion syntactic sugar. Here is an example: 
    gremlin> g.V().out("1..3", "knows").with('RESULT_OPT', 'ALL_V_E')
     ==>[v[1], e[0][1-knows->2], v[2]]
     ==>[v[1], e[2][1-knows->4], v[4]]
  • We are now happy to introduce the capability of querying Cypher in GIE, by integrating Neo4j's bolt service in our system. Please follow the guide to enable bolt service (only on local) for querying with Cypher. We also attempt to make the syntax of Cypher as close as openCypher, and the details of our support for Cypher can be found here.

3. Other enhancements and bug fixes

  • GAE Java
    • Fix VertexSet's problem of supporting vertex_id in java long.
    • Add Grape-GraphX performance report
    • Fix the problem of installing grape-jdk locally.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.23.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.23.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.23.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.23.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.23.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.23.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [7dcf1950b]: Change the version to v0.23.0 (#2969) (Dongze Li)
  • [731629732]: Introduce flexbuild and a readme to flex. (#2954) (luoxiaojian)
  • [d8ad6ce6c]: Install jdk 11 on MacOS CI and fixes the GAE link failure in nightly CI on MacOS (#2965) (Tao He)
  • [c27f45902]: Fixes two compliation warnings (#2968) (Tao He)
  • [bb1921eca]: Update the elementMap-step's doc of GIE (#2966) (Dongze Li)
  • [cb27df481]: Fixes nightly CI error for Arm Linux wheels (#2963) (Tao He)
  • [69258d995]: Fixes the "illegal thread local variable reference to regular symbol" error on macOS for GLE (#2962) (Tao He)
  • [1e2c8f62a]: [GIE Compiler] Unify Gremlin Timeout Configurations (#2953) (Xiaoli Zhou)
  • [46949638e]: Add setuptools to client requirement in order to use pyproject based build (#2958) (Siyuan Zhang)
  • [7fc14b5af]: [GIE Compiler] Support Case When Expression in Logical and Physical Plan (#2918) (Xiaoli Zhou)
  • [b82b74883]: Avoid implicit int to enum conversion, causing the -fpermissive error (#2957) (Siyuan Zhang)
  • [ad4aea19d]: Use the correct delete in free_graph_handle (#2955) (Tao He)
  • [21fd64430]: Sort the server list before iterate to keep consistent between workers (#2956) (Tao He)
  • [502e41030]: Parallelize the PrepareToRun implementation in projected fragment (#2949) (Tao He)
  • [5ffad661f]: Add two more pagerank variants (#2950) (Siyuan Zhang)
  • [a22247999]: Use pagerank_local_parallel rather than pagerank_local for better performance (#2945) (Tao He)
  • [92da63887]: Refine CI process (#2948) (Siyuan Zhang)
  • [555b13de8]: add figures for GraphScope Flex. (#2946) (Jingbo Xu)
  • [7035c0a44]: ElementMap step support (#2942) (Dongze Li)
  • [b6402e708]: cluster_builder.py: add missing copyright header and remove unused import lines (#2944) (Tao He)
  • [5b852ffe1]: Docs preview is only available for pull requests from the orignal repo, not forks (#2943) (Tao He)
  • [cab8cde5e]: Add GLE quick start example in the overview getting started page (#2939) (LiSu)
  • [badf3f3be]: hotfix: pull_request_target issue. (#2941) (Jingbo Xu)
  • [08f8d0f24]: Disable JAVA SDK (with a warning) when JNI not found (#2937) (Tao He)
  • [f543cd29e]: Improve the deploy_with_existing_vineyard_cluster doc for easy understanding (#2930) (Ye Cao)
  • [ae8d96c7d]: Show explicit error message for SSSP run over a graph with no edge weight (#2929) (Lei Wang)
  • [50c7a1e21]: Pin vineyard python version to 0.15.0 (#2933) (Siyuan Zhang)
  • [25496f8d5]: Connect to frontend directly, bypass coordinator for GIE query (#2923) (Siyuan Zhang)
  • [96d50939d]: [GIE Compiler] Parse Label Id to Name in label Operator (#2928) (Xiaoli Zhou)
  • [34b365028]: [GAE-Java] Use FFI-bitset rather than java BitSet for VertexSet backend (#2926) (Zhang Lei)
  • [f0a779e88]: [good-first-issue]Move save_to/load_from from client side to coordinator (#2917) (Zhang Lei)
  • [c1007cff5]: [GIE Compiler] Introduce cypher service to accept queries from neo4j ecosystem (#2848) (Xiaoli Zhou)
  • [7637aafb5]: [GIE] Refine FFI interface build_physical_plan (#2911) (Xiaoli Zhou)
  • [d0dc2b7c4]: Print metrics of application running time (#2908) (Siyuan Zhang)
  • [628773d80]: [Bug Fix] Fix bug of null pointer exception in distinct operator (#2901) (Xiaoli Zhou)
  • [0a44e711e]: Fixes the misuse of "continue-on-error" and "fail_ci_if_error". (#2904) (Tao He)
  • [612bd7f09]: Upload coverage: continue on error to avoid false negative on main branch (#2903) (Tao He)
  • [513155c30]: Upgrade the graphlearn submodule to fixes nightly CI, refactor the Makefile for early error exit (#2902) (Tao He)
  • [bd7680f2c]: Use CONFIG mode to find protobuf to address the absl dependency issue. (#2899) (Tao He)
  • [cf81c7b80]: [GAE-Java] Relax version enforce for some mvn dependencies (#2894) (Zhang Lei)
  • [b37ab9031]: [GIE Compiler] Add data type of algebra operator output to physical plan (#2870) (Xiaoli Zhou)
  • [79cd5770b]: Use local metadata backend to avoid the requirements of etcd for local sessions (#2889) (Tao He)
  • [7ef0e4d14]: fix munew line are not rendered correclty when multiple params are given (#2891) (Siyuan Zhang)
  • [fbf40f6dd]: Update the pre-genereated unload_op when the op itself been deepcopied (#2888) (Tao He)
  • [32089f285]: [Grape-Java] Add grape-graphx performance.md (#2886) (Zhang Lei)
  • [e4f1f23a3]: Allow pass params to gie instance (#2885) (Siyuan Zhang)
  • [17819c81f]: [BugFix] Fix bug in endV() after PathExpand (#2881) (BingqingLyu)
  • [a18836d5c]: [GIE Compiler] Support dynamic query params in cypher queries (#2874) (Xiaoli Zhou)
  • [ce3e16b2e]: [GAE/tests] Add GAE tests for FLASH algorithms of bfs and cc (#2869) (Longbin Lai)
  • [5a63fafa0]: upload log should not fail the CI (#2879) (Siyuan Zhang)
  • [559b084b6]: Direct the content of log files to stdout of pods in kubernetes deployment (#2871) (Siyuan Zhang)
  • [4af5689be]: add initial version of command-line tool gsctl. (#2868) (Jingbo Xu)
  • [7a3e06b56]: [GIE Doc] Update Doc for PathExpand Step Usage in GIE (#2877) (BingqingLyu)
  • [387f915f6]: Fixes the docs generation for tagged releases (#2878) (Tao He)
  • [c489c6381]: [BugFix] Make the error message truncated not so aggressive (#2876) (Weibin Zeng)
  • [e9af91a91]: [GIE] Support PathExpand with OPT=ALL_V_E in GIE (#2841) (BingqingLyu)
  • [37738891e]: Add a initializeCommand to devcontainer.json: always pull newer image before create container (#2873) (Weibin Zeng)
  • [086fa6362]: [GIE/docs] reorganize gremlin docs (#2862) (Longbin Lai)
  • [bd8952093]: Allow triggering workflow dispatch event for CIs (#2865) (Ke Meng)
  • [a0b36eeef]: [GIE Doc] Add examples to collect gremlin results in a streaming way (#2842) (Xiaoli Zhou)
  • [5fd6cc173]: Fixes the inconsistent usage of msgpack/json in graph reporter and avoid protobuf message creation in del (#2843) (Tao He)
  • [434acdfea]: Fix dns search error in groot (#2850) (Siyuan Zhang)
  • [9bf071cf3]: auto-gen docs of flex (#2851) (ds-ssj)
  • [6851cd9fc]: allow null value in data loading (#2845) (Siyuan Zhang)
  • [1ba61029e]: [GIE Compiler] Introduce GraphPlanner to unify logical and physical plan building (#2838) (Xiaoli Zhou)
  • [a8b9dce78]: [GAE-Java] Use maven-flatten-plugin to replace revision property for locally build&install (#2840) (Zhang Lei)
  • [3c4cd9d98]: Fix compile error for the flash bc algorithm on ubuntu (#2839) (lixueclaire)
  • [0de3063f1]: use -O3 instead of -O0 in customized algo compilation (#2837) (Siyuan Zhang)
  • [9a265dcd1]: [GIE/Runtime] Redesign PartitionerInfo, ClusterInfo, and Router trait to better support parallel processing in Runtime (#2744) (BingqingLyu)
  • [653823b88]: [GIE/tests] Add pattern matching test cases for GIE (#2831) (lixueclaire)
  • [30e45e0ed]: [GIE Compiler] Unify execution client which is used to send request to remote engine service (#2818) (Xiaoli Zhou)
  • [f46941564]: Remove unused ports in Groot deployment (#2832) (Siyuan Zhang)
  • [d75ba7b26]: Add coordinator as the Ownerreferces of frontend deployment. (#2830) (Ye Cao)
  • [ec218cf3d]: [GIE Compiler] Fix query params of getV base in path expand operator (#2773) (Xiaoli Zhou)
  • [e28faa991]: Change version to 0.23.0 for daily release (#2828) (Dongze Li)
  • [4535bc34b]: Fixes errors during macos M1 CI (#2827) (Tao He)
  • [9f2334c68]: [GIE/docs] Add gie glossary and quick start (#2821) (Longbin Lai)
GraphScope - Release v0.22.0

Published by lidongze0629 over 1 year ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.22.0 release. This release contains many important features and enhancements to Interactive Engine (GIE) and Deployment. Now the user can try GraphScope under the Linux Arm64 platform by pip3 install graphscope.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • We have provided a utility for users to easily develop and test GIE on a local machine. Basically, users can run three parts of integration tests using the gs command line tool, including the official gremlin test, auxiliary pattern match test, and ldbc complex interactive workload on ldbc social network scale factor 1. In addition, users are allowed to locally start a frontend and an executor, where the former exposes an endpoint to easily integrate with the official gremlin console, and the latter is responsible for executing queries. Details can be found in the documentation.

2. Enhancements for Deployment

  • User can try GraphScope under the Linux Arm64 platform by pip3 install graphscope
  • All images are now switched to ubuntu-based.
  • Add a devcontainer.json for users to quickly get started.
  • Use vineyardctl to install vineyard as the sidecar container.
  • Support to deploy the engines of GraphScope on demand in the lazy mode.

3. Other enhancements and bug fixes

  • Support to store and restore graphs from/to persistent storage on the kubernetes cluster. Details can be found in the guide: https://graphscope.io/docs/latest/deployment/persistent_storage_of_graphs_on_k8s.
  • Add an end-to-end example about training over vineyard graphs using GLE
  • Fixes the incorrect problem of results in python algorithm with PIE model under the session's parameter num_workers=1
  • We have fixed a bug of GIE related to using match step with a not-existed edge, for which it should be executed via anti join, but it returned unexpected error "the first sentence of MergedSentence must have InnerJoin" instead.
g.V().match(
    __.as("a").in().as("b"),
    __.as("b").out().as("c"),
    __.not(__.as("c").out().as("a")) # `not`-exsited edge
)

Now such kind of query can be executed without issue.

  • Fixes the bug of incorrect results when dedup edges on a partitioned graph in GIE, e.g.,
g.V().outE().dedup()

Now it can output expected results.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.22.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.22.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.22.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.22.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.22.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.22.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

  • [99c9d15d2]: Fixes the GAE fork issue with openmpi (#2820) (Tao He)
  • [ba359967a]: [GLE Doc] complete the build and install details of learning engine (#2815) (LiSu)
  • [7af925532]: Fixes some failures in nightly CI (#2817) (Tao He)
  • [0b220536d]: Support to install deps for Alibaba Cloud Linux in the gs bash utility (#2816) (Ye Cao)
  • [28a360593]: Cleanup space in CI (#2814) (Siyuan Zhang)
  • [3f50d7d3e]: Update dev_guide.md: change the link to learning engine to a correct one. (#2813) (Taoshu)
  • [035036b2c]: disable debug-assert in groot (#2811) (Siyuan Zhang)
  • [57b22febf]: Groot schema now supports add comment (#2802) (Siyuan Zhang)
  • [43493ce11]: Fixes bug when building docs from pull requests (#2810) (Tao He)
  • [302b02877]: Support to backup and recover the specific graphs via vineyardctl (#2771) (Ye Cao)
  • [d49d0edb7]: docs: update rst-versions style, defaults default version to latest, and fixes the zh/en indexing (#2803) (Tao He)
  • [9efd4f672]: Add local python script path to PATH (#2804) (Tao He)
  • [9d5d968e7]: [Docs/GIE] add tinkerpop ecosystem (#2784) (Longbin Lai)
  • [278a24c79]: Adapt to latest vineyard for varint encoding in CSR (#2791) (Tao He)
  • [f1b216cc7]: Support users to only input the kubeconfig path for connecting the k8s cluster (#2799) (Ye Cao)
  • [33e300721]: Correct the version of built wheel (#2798) (Siyuan Zhang)
  • [191949b18]: Fixes the graphdef value when constructing graphs from vineyard_id. (#2793) (Tao He)
  • [699ffbe57]: [k8s]Support deploying vineyard cluster independently and deploy the engines on called (#2710) (Ye Cao)
  • [c1ef88582]: refine docs latest tag to make it ready as default. (#2788) (Jingbo Xu)
  • [e9090db36]: Refine doc for dev and test of analytical engine. (#2787) (Jingbo Xu)
  • [1e57650a0]: Read version number from the package in setup.cfg (#2785) (Dongze Li)
  • [09ead30e5]: [GLE docs] minor fixes (#2783) (LiSu)
  • [b014dad1f]: use kubectl cp --retries (#2781) (Siyuan Zhang)
  • [d2c7d2edb]: Fix typo in Makefile (#2782) (Siyuan Zhang)
  • [3606b857b]: Fixes the building failure of client wheel on Linux arm64 platform (#2780) (Dongze Li)
  • [fdc626ee4]: Use vineyardctl API to inject the vineyard sidecar (#2612) (Ye Cao)
  • [7fd430b3d]: [GraphProxy] refine implementation of LazyDetails (#2766) (BingqingLyu)
  • [4b24146d1]: [Bug Fix] fix the bug in implementation of PartialEq for Edge in GIE Runtime (#2740) (BingqingLyu)
  • [5b46a68bc]: Improve GetInnerVertex() by querying local fragment only (#2772) (Songqing Zhang)
  • [983a41e33]: Enable deploy groot on local environment (#2769) (Siyuan Zhang)
  • [2509958c4]: Make grape_engine help message more friendly (#2774) (Siyuan Zhang)
  • [e2943778e]: [GIE Dev Test] Refine GIE Dev and Test (#2767) (Xiaoli Zhou)
  • [bf8d7b096]: Generate pyi stubs for protobuf modules for better developer experience (#2768) (Tao He)
  • [6fc7db32f]: [Bug Fix] Fix the alias of GetV of ExpandBase in PathExpand, an… (#2763) (BingqingLyu)
  • [a119c23b4]: Correct the python sssp result by setting generate_eid to false in load_p2p_network dataset (#2762) (Dongze Li)
  • [e4bb06903]: Make sure grape-engine exit when CTRL-C without crash (#2761) (Tao He)
  • [df1628425]: Store address offset of string properties with 48-bit unsigned intege… (#2760) (luoxiaojian)
  • [747df41cf]: Drop the environment switch in dev and test, and update gs (#2754) (Siyuan Zhang)
  • [46d7ed8bb]: Fixes duplicated run and upload actions for k8s-ci (#2751) (Tao He)
  • [3cf079817]: Propogate the k8s exception to avoid waiting forever (#2747) (Tao He)
  • [e6af69be1]: Ensure use string as log-level, even for logging.XXX (#2745) (Tao He)
  • [e969d0e4e]: [GIE Dev Test] Add docs for interactive dev test (#2733) (Xiaoli Zhou)
  • [ac7308e87]: [BugFix] add Metadata for PathExpand (#2720) (BingqingLyu)
  • [94ec2d895]: Updated license and notice. (#2725) (Jingbo Xu)
  • [8ab7b1b84]: Overhaul data-load-tools (#2715) (Siyuan Zhang)
  • [b03bc60ea]: Publish wheel packages on Linux arm64 platform nightly (#2732) (Dongze Li)
  • [1e63e8731]: [GIE/bug] fix a bug of matching with anti join (#2730) (Longbin Lai)
  • [1a8162090]: use javac to find the version of java (#2728) (Siyuan Zhang)
  • [51f92ac04]: Support to build wheel package on Linux aarch64 platform (#2723) (Dongze Li)
  • [1d3cd4a44]: refine retry logic to exponential backoff (#2729) (Siyuan Zhang)
  • [06f876920]: Fixes the installation prefix for nightly linux CI (#2727) (Tao He)
  • [662014fcc]: Opt-out zstd in features when requiring rocksdb (#2722) (Tao He)
  • [0a5c14410]: set timeout of rt_admin to 1 minute. (#2719) (luoxiaojian)
  • [fb79c6211]: Bump up graph-learn submodule version (#2716) (Tao He)
  • [7a5ed1344]: add docs for using dev container and fix typo (#2711) (Siyuan Zhang)
  • [5bd90ba10]: Requires correct version of graphscope_client in coordinator to avoid diverage (#2708) (Tao He)
  • [10a6488de]: Introduce the new version of kubernetes log exporter for dumping logs during a period of time such as pytest (#2706) (Ye Cao)
  • [e447cd5cd]: Fixes the linux wheel install prefix (#2707) (Tao He)
  • [f476c5736]: Fixes the 'sudo' in dockerfile for gss image (#2704) (Tao He)
  • [3b11b77ff]: Build wheels for macOS m1 (#2701) (Tao He)
  • [446b6601d]: [LICENSE] add dependency licenses for calcite and antlr (#2699) (Xiaoli Zhou)
  • [52b1ee29e]: Fixes the initialization of paramters in load_from() (#2698) (Tao He)
  • [d4b72fd97]: Display stderr logs as expected on K8s (#2696) (Tao He)
  • [6da5c3112]: Add version info for making interactive (#2685) (Siyuan Zhang)
  • [6dd3af29b]: Implemented 3-retries, added detailed logs for insert queries, and unified bulk-load and recovery. (#2691) (luoxiaojian)
  • [03165eb94]: [FEAT][GraphAr] Add API to python client to enable loading graph from gar files or archive graph to gar files (#2588) (Weibin Zeng)
  • [c2cb28678]: Export the k8s logs of graphscope helm test and upload to artifact when the helm test failed (#2683) (Ye Cao)
  • [81230c586]: Add prepare-commit-msg (#2688) (Siyuan Zhang)
  • [90b7e0049]: Update install-hook.sh (Wenyuan Yu)
  • [ff9582e37]: Create install-hook.sh (#2687) (Wenyuan Yu)
  • [909b3c4d0]: [GIE/Runtime] Remove the dependency of GIE Runtime on GlobalQueryStore to accelerate the compilation (#2681) (BingqingLyu)
  • [a46663e52]: Clearify the requirements for MacOS (#2684) (Tao He)
  • [25a70f127]: Preload searchindex.js in background to optimize the search experiences (#2682) (Tao He)
  • [b258999b0]: Bump up vineyard to v0.14.5 (#2676) (Siyuan Zhang)
  • [ff5809e06]: Use different default value for vineyard_shared_memory on k8s and local (#2678) (Tao He)
  • [a38f41439]: enable auth for realtime write (#2674) (Siyuan Zhang)
  • [4225cb1c7]: Use ubuntu-base to build out production images. Add another wheel tag dedicated for building wheels. (#2607) (Siyuan Zhang)
  • [8168275e5]: Improve the error message when passing unexpected arguments to g.project() (#2667) (Tao He)
  • [5646ce2ea]: Document how to install scipy on M1 MacOS. (#2666) (Tao He)
  • [b4caf8f06]: [GAE] Fix overflow of MPI_Isend()'s data size (#2665) (Songqing Zhang)
  • [5c0ad71cc]: Upgrade protobuf/grpc, use newer macos builder, fixes bugs in local launcher where vineyardd may fail, use latest black as formatter (#2650) (Tao He)
  • [25593331c]: temporarily fix the urllib3 2.0.0 incompatible with openssl (#2662) (Siyuan Zhang)
  • [57dba68e6]: Tune kafka and grpc maximum message size (#2661) (Siyuan Zhang)
  • [1f0b8b91e]: Implemented the init version of Flex. (#2660) (luoxiaojian)
  • [35b148c0c]: Update the GLE submodule to include the latest updates (#2643) (LiSu)
  • [7fc0a1522]: Change version to 0.22.0 for daily release (#2658) (DongZe Li)
  • [21bdae33c]: add export CMAKE_PREFIX_PATH (#2655) (Siyuan Zhang)

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.21.0...v0.22.0

GraphScope - Release v0.21.0

Published by lidongze0629 over 1 year ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.21.0 release. This release contains many important features and improvements to GraphScope. The Graph Learning Engine (GLE) now supports the 'SubGraphSampler' and 'RandomWalk' operators and adds a node labeling-based SEAL and node2vec model. Also, GLE supports node cache for accelerating distributed feature lookup. This release also adds a 'devcontainer.json' for users to easily set up a developing environment using VSCode.

We highlight the following improvements included in this release:

  • Enhancements for GLE:
    a. Add operators: SubGraphSampler, RandomWalk
    b. Add node labeling based SEAL model, node2vec model.
    c. Support temporal graph with the timestamp, and add TokTimestampSampler operator.
    d. Add temporal models: TGN and TGAT.
    e. Support node cache for accelerating distributed feature lookup.

  • Enhancements for Dev:
    a. Add a devcontainer.json for users to easily set up a developing environment using vscode.

  • Other enhancements and bug fixes:
    a. 'helm install graphscope/graphscope' would now launch all pods upon the start.
    b. Allowing more fine-grained control over Groot deployment
    c. Fixed an error of 'lgraph' that could cause the building failure in M1 Mac

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.21.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.21.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.21.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.21.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.21.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.21.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.21.0

# Develop Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-dev:latest

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.20.0...v0.21.0

GraphScope - Release v0.20.0

Published by lidongze0629 over 1 year ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.20.0 release. This release contains many important features and improvements to the persistent graph store Groot, including load balance and failover mechanism. In addition, Graph Learning Engine (GLE) now supports the Aliyun ACK cluster.

We highlight the following improvements included in this release:

1. Enhancements for GLE:

  • GLE could be launched and used to training on ACK cluster.

2. Enhancements for Groot:

  • Chart is refactored and has more customizable fields.
  • LoadBalancher service is supported.
  • Optimization of failover mechanism.

3. Enhancement for images:

  • We release the first dev image (graphscope-dev) for arm64 platform.

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.20.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.20.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.20.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.20.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.20.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.20.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.20.0

What's Changed

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.19.0...v0.20.0

GraphScope - Release v0.19.0

Published by lidongze0629 almost 2 years ago

We are glad to present the release of GraphScope 0.19.0, which brings a number of new features as well as performance and stability improvements to GraphScope. Specially, users can try GraphScope in python3.10 and python3.11 environments of Linux and MacOS by pip3 install graphscope; This release also optimizes the memory usage for loading graphs, which allows users to load larger graphs on their machines. In addition, we improve Graph Interactive Engine (GIE) via a couple of bug fixes and improvements.

We highlight the following improvements included in this release:

1. Optimized memory usage for loading graphs:

  • Improves memory usage during loading and projecting graphs, especially for graphs with string as OID type.
  • Improves the performance of project_to_simple.

2. Bug fix and enhancements for GIE:

  • Fix a bug in Pegasus that may block the execution of interactive queries in certain case.
  • Fix a bug that fails to configure the server number for the GIE engine.
  • Fix multiple dependencies in both Java and Rust codes.
  • Resolve a issue that JNA could not map bool correctly.

3. Enhancements for GAE Java:

  • Pack JNI lib into the grape-runtime jar and fix some dependency issues.

4. Standalone deployment:

  • Components now have their own dedicated image, and user could configure a subset of engines to use when launching session.

5. Other bug fixes:

  • Fix helm deployment could not delete role and rolebindings when uninstalling GraphScope.
  • Fix compile error under the newer toolchain and arrow 10
  • Rename maxgraph to graphscope and other refactors
  • Cleanup overlapping rust dependencies

Docker Image

# Coordinator Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/coordinator:0.19.0

# Graph Analytical Engine(GAE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical:0.19.0
# Graph Analytical Engine(GAE) Image with java SDK, which supports running Giraph and GraphX algorithms on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/analytical-java:0.19.0

# Frontend component of Graph Interactive Engine(GIE) image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-frontend:0.19.0
# Executor component of Graph Interactive Engine(GIE) image 
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/interactive-executor:0.19.0

# Graph Learning Engine(GLE) Image
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/learning:0.19.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.19.0

What's Changed

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.18.0...v0.19.0

GraphScope - Release v0.18.0

Published by lidongze0629 almost 2 years ago

We are delighted to bring a number of improvements to GraphScope, alongside the GraphScope 0.18.0 release. This release contains many important features and enhancements to Graph Interactive Engine (GIE), including introducing a new strategy for pattern-matching queries and supporting canceling running queries. The Graph Learning Engine (GLE) now supports PyTorch and is also compatible with PyG. In addition, we take a first step towards modularized deployment for different components of GraphScope in Kubernetes.

We highlight the following improvements included in this release:

1. Enhancements for GIE

  • A new execution strategy based on worst-case optimal join is introduced to GIE engine, potentially improving the performance of match step by orders of magnitude.
  • The query can be canceled after its execution exceeds a pre-given overtime parameter (10min by default).
  • GIE engine supports a failover mechanism: if an executor pod fails, but the data is not missing, it can be restarted by k8s. Existing queries cannot recover, but the engine can serve the following.

2. Enhancements for GAE

  • Add more variants of WCC algorithm.
  • Supports local vertex map to make it could scale to larger graphs given more workers.

3. Enhancements for GLE

  • Add support for PyTorch and PyG
  • Add heterogeneous graph support for subgraph-based GNN, add HeteroSubGraph and HeteroConv, bipartite GraphSAGE and UltraGCN.
  • Add edge feature support in both EgoGraph and SubGraph.
  • Add recommendation metrics: Recall, NDCG and HitRate.
  • Add hiactor-based graph engine.

4. Standalone deployment

  • Supports a version of GIE standalone deployment in Kubernetes.
  • Redesigned docker files from the bottom to the top, making it more clear and more concise.

5. GAE Java Enhancement

  • Introduce @GrapeSkip for better user experience in template method overloading.
  • Speedup Java App runimte codegen.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.18.0

# GraphScope persistent storage, user can only perform GIE query on it.
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.18.0

Commits

  • [6775658d7]: Push jupyter image nightly with the latest client (#2266) (DongZe Li)
  • [df9f6ebf3]: Fix nightly CI (#2262) (Siyuan Zhang)
  • [4f1777700]: Fixes CI failure both on Linux and MacOS (#2244) (DongZe Li)
  • [07efc740d]: [GIE/engine] Bug fix; (#2250) (bmmcq)
  • [5ea5b874a]: [GIE] Support parallel scan on ExpStore (#2253) (BingqingLyu)
  • [432e65c89]: [GIE] Make the version of GIE compiler consistent with the default value in interactive engine pom (#2249) (shirly121)
  • [2b7cf0050]: [GIE POM] Unify GIE version by the global flag 'revision' (#2243) (shirly121)
  • [71d61e3cf]: Reorganize for Makefiles and dockerfiles (#2246) (Siyuan Zhang)
  • [84f449c7c]: [GIE Runtime] Debug runtime op info on in a more clearly way (#2226) (BingqingLyu)
  • [f77627dd7]: Update the learning model to align with the latest graphlearn (#2235) (Tao He)
  • [1c4c1cd17]: [GIE/Store] Reduce memory footprint in exp_store (#2245) (Longbin Lai)
  • [e375f8d3d]: [GIE/Exp-Store] Optimize memory usage in loading graph data into MutTopo. (#2242) (Longbin Lai)
  • [8844e9027]: Fixes release graphscope-store image (#2241) (Siyuan Zhang)
  • [428c7855c]: [GIE/Engine]: add direct executor implementation; (#2234) (bmmcq)
  • [e071ce4eb]: add docs revision preview in CI (#2237) (Jingbo Xu)
  • [f52455078]: Enable cpplint in CI (#2233) (Tao He)
  • [330fe14a7]: Makes GAE compatible with latest arrow release (#2232) (Tao He)
  • [ed8604036]: Revise docs. (#2231) (Jingbo Xu)
  • [b4d7038da]: Fixes compilation error of graphx runner (#2228) (Siyuan Zhang)
  • [a11b45395]: Reorganize Dockerfile according to specification of standalone deployment (#2227) (Siyuan Zhang)
  • [dcaa38ac0]: [GIE/GLogue] rename Catalogue as GLogue (#2224) (BingqingLyu)
  • [aed1f1362]: [GIE/IR] Introduce a new strategy for pattern matching (#2159) (BingqingLyu)
  • [92f7da3a9]: Add two other version of wcc for compatiblity (#2223) (Siyuan Zhang)
  • [2fbc4ad7c]: Add local vertex map to arrow fragment and arrow projected fragment. (#2212) (Siyuan Zhang)
  • [cc5170434]: Fixes nightly CI on macOS (#2219) (DongZe Li)
  • [f21754fcc]: [GAE-Java] GRAPE JDK support Spark local mode for graphx (#2216) (Zhang Lei)
  • [ed33b08c6]: [BugFix] GAE JavaDoc not generated (#2206) (Zhang Lei)
  • [440572158]: Update the DefaultTag to .Chart.AppVersion (#2210) (Ye Cao)
  • [3ee05f77b]: [new FAQ] add network proxy related FAQ (#2211) (Yifan Yuan)
  • [09995bfa0]: Delegate parquet/orc files to vineyard io adaptors (#2209) (Tao He)
  • [7ecd75a47]: Delete all deployed resources without a extra rbac role when using helm unintsall (#2208) (Ye Cao)
  • [094e81d67]: [Bug Fix] Add default tag (.Chart.AppVersion) to graphscope-store (#2205) (Ye Cao)
  • [4279b8fc8]: [BugFix] make ResultCode in ir core ffi consistent with Compiler (#2188) (BingqingLyu)
  • [7cfe9a3fd]: [Bug Fix] Avoid throwing exception after processing grpc error from executor in compiler (#2182) (shirly121)
  • [abdd5c979]: [GAE-Java] Update doc (#2199) (Zhang Lei)
  • [8de000fd1]: [GAE-Java] Remove String-specialized FFI classes and Simplify Java MessageManager interface (#2195) (Zhang Lei)
  • [5a4312ded]: Fix failure in network forward CI (#2204) (Siyuan Zhang)
  • [773545142]: Fixes failures in nightly CI (#2201) (Tao He)
  • [264cbc8a2]: Log level can accept int and string ignore case (#2200) (Siyuan Zhang)
  • [fb12c7b12]: Fix slf4j version conflicts (#2196) (Siyuan Zhang)
  • [87b960232]: [GIE Deploy] Deploy GIE with exp or v6d in standalone mode by helm configuration management (#2143) (shirly121)
  • [ecd58a248]: Fix possible java sandbox protection. (#2191) (Siyuan Zhang)
  • [38a3781dc]: Enable HDFS tests in CI. (#2175) (Tao He)
  • [7fdcd2357]: [GAE-Java] Support grape-graphx running on yarn cluster and support multiple executor on one host (#2185) (Zhang Lei)
  • [f983bd672]: [GIE pegasus] Support cancelling query. (#2168) (Neng Li)
  • [26f03827e]: gss ci use lesser image size (#2186) (Siyuan Zhang)
  • [2b00169b9]: [GAE-Java] Fix GAE-Java driver app's message strategy specification (#2178) (Zhang Lei)
  • [f4dea3e74]: [OSPP] Spark RDD Reader for GraphScope (#2103) (Issac-Newton)
  • [cfb707dd7]: Enhance grpc finding in GAE cmake (#2184) (Tao He)
  • [25ab4664f]: Use pre-built llvm instead of build when making analytical docker (#2183) (Siyuan Zhang)
  • [79a08d3b9]: [GIE Compiler] update ResultCode to be consistent with ir core interface (#2158) (shirly121)
  • [74af74f00]: [GAE-Java] GraphX-on-GraphScope : remove GraphXFragment and related components (#2123) (Zhang Lei)
  • [49ae814f1]: [GAE-Java] Add necessary configurations in pom.xml to publish GRAPE jdk to maven central repo (#2166) (Zhang Lei)
  • [2821169a8]: Fix create interactive engine failure on helm installed graphscope (#2171) (Siyuan Zhang)
  • [89a4a218e]: [GAE-Java] GRAPE jdk submodules naming to same pattern (#2163) (Zhang Lei)
  • [321d93805]: Upgrade GitHub actions versions (#2160) (Siyuan Zhang)
  • [695cb9caa]: Reorganized coordinator and redesigned the logic to create engines. (#2152) (Siyuan Zhang)
  • [ea4ca1d0b]: fix src label and dst label of edge kinds may be empty (#2153) (Siyuan Zhang)
  • [1b312af5c]: Move the tolerances of graphscope-store charts to the global scope (#2151) (DongZe Li)
  • [37fa85ad4]: [GAE-Java] Speed up Java App runtime codegen& compiling (#2150) (Zhang Lei)
  • [733fda3d0]: [GIE IR] Add docker files to build images of GIE (using exp store) (#2142) (shirly121)
  • [0c5a4b55d]: [GAE-Java] Fix : can not compile the submodule of GAE Java SDK (#2146) (Zhang Lei)
  • [3220a7f03]: Fixes the conditions for GAE CI in PR and on main, add timeout for test_java_app.py (#2148) (Tao He)
  • [998e3615c]: Rename pegasus.hosts to network.servers for executor (#2147) (Siyuan Zhang)
  • [97eec7023]: Add pytest-timeout to requirements-dev.txt and fixes a deprecated warnings when running tests (#2145) (Tao He)
  • [3038b8970]: [GAE-Java] Introduce @GrapeSkip to support generated method overloading (#2144) (Zhang Lei)
  • [77b4ec5c6]: [GAE-Java] Fix log4j jar missing in grape-runtime and java pie driver app's message strategy settable. (#2140) (Zhang Lei)
  • [7d351a2b5]: [IR Compiler] make gremlin server listen to 0.0.0.0 instead of localhost (#2141) (shirly121)
  • [759b7a5b0]: Server now use aks provided by client side (#2132) (Siyuan Zhang)
  • [5c963f583]: Update cluster.py to fix the bug that etcd selector could not work (#2139) (changyi1223)
  • [6d94c3fa0]: [Bug Fix] Fix bug in index_scan on Groot/Vineyard (#2129) (BingqingLyu)
  • [d4d942c51]: [Assembly] wait_servers_ready() instead of sleep 3s when start engine on vineyard (#2136) (BingqingLyu)
  • [2aa3cfc38]: Update commons-codec:commons-codec 1.11 to 1.13 (#2119) (zhoumengyks)
  • [6d3d6aada]: Update org.apache.kafka:kafka-clients 2.5.0 to 2.7.2 (#2120) (zhoumengyks)
  • [6d5d288fa]: Fixes the docstring rendering for GAE apps (#2131) (Tao He)
  • [950ed7ab7]: App now can carry cmake options, and enabled wcc on string oid. (#2130) (Siyuan Zhang)
  • [31829fd08]: [GIE/Runtime] Refine some error infos in Runtime (#2113) (BingqingLyu)
  • [d16b1b482]: Fixes configuration in dockerignore (#2126) (Tao He)
  • [71c37fecf]: Upgrade com.fasterxml.jackson.core:jackson-databind to version 2.14.0-rc1 (#2122) (Siyuan Zhang)
  • [ace93c53b]: [Pegasus] minor fix: avoid iter.next() when it is None in release (#2112) (BingqingLyu)
  • [5756de3dc]: Make gae CI invoked when CMakeLists.template changes (#2101) (Zhang Lei)
  • [b6d68bc61]: New predicate support (#2086) (waruto)
  • [eb393dab1]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine (#2109) (dependabot[bot])
  • [0b10f5c0f]: Bump protobuf-java from 3.18.2 to 3.19.6 in /interactive_engine/compiler (#2111) (dependabot[bot])
  • [b1132cc7c]: [ASoC] report for subtask (#2107) (Haochen Shi)
  • [751e3e7c6]: [GIE IR] Provide expr store ffi interfaces to encode and write data (#2098) (shirly121)
  • [91db0334b]: [IR Runtime] Support removing columns in Record via Auxilia in IR-Runtime (#2085) (BingqingLyu)
  • [e024c6549]: [GIE-IR] Fix the doc inconsistency issue of the legacy MaxGraph and the latest Gaia-IR. (#2074) (shirly121)
  • [cf7495c69]: Pre-release v0.18.0 (#2104) (DongZe Li)
  • [5aabc4dc0]: [IR Benchmark] Benchmark for BI queries on GIE-IR (#2081) (BingqingLyu)
  • [be1d7a864]: upgrade rocksdb-rust to 0.19.0 (#2100) (Siyuan Zhang)

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.17.0...v0.18.0

GraphScope - Release v0.17.0

Published by lidongze0629 about 2 years ago

We are glad to announce a number of new features and improvements to GraphScope, alongside the GraphScope 0.17 release. The updates include new releases for Spark-GraphX support, backend engine enhancement, and frontend APIs. Currently, GraphX Pregel algorithms can be seamlessly executed on GraphScope. A lot of new features have also been officially brought into the interactive engine (GIE), including the syntactic sugar of path expansion and expression, the Gremlin steps of id(), label(), coin() and otherV() etc. and the profiling and benchmarking tools for LDBC BI queries.

We highlight the following improvements included in this release:

1. Spark-GraphX support for GAE:

  • Support converting GraphX graph to GraphScope Fragment
  • Support wrapping Fragment as GraphX Graph.
  • Support running GraphX pregel on GraphScope Analytical engine.

2. New language features and enhancement for GIE:

  • The syntactic sugar of path expansion. Now user can write out[in|both]('x..y', 'KNOWS') to support path expansion of at least x (included) times and at most y (excluded) times from a certain node. with() step can be followed to configure the path expansion.
  • The syntactic sugar of expression, introduced by expr(), can be used along with the where()-step to enable filtering expression. For example, .where(expr("@.name == \"John\"")) help filter out the current entry that has the name of "John". We have updated the README doc for some instructions.
  • Provide LDBC benchmarking for BI queries.

3. Enhancement and dependency refinement of Groot to reduce the size of fat-jar.

4. Feature updates:

  • Add from_dict / to_dict option for graph schema.
  • Add syntactic sugar of path expansion, expr(), and the Gremlin steps of id(), label(), coin(), TextP.startsWith/endsWith... etc.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.17.0
docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope-store:0.17.0

Commits

  • [e0a761580]: Get the exception type in correct way, both on Linux and on MacOS. (#2099) (Tao He)
  • [0fcc2d815]: Update pom.xml (#2095) (Rudolf D)
  • [2caa69b7d]: Fixes the missing return statement in frames (#2097) (Tao He)
  • [12f32fff4]: Ensure the correctness of retry (in the RunStep request) (#2094) (Tao He)
  • [f3295f4a5]: Test against the latest libgrape-lite and vineyard (#2090) (Tao He)
  • [4caa4b8ed]: Enhance the error handling in frames (#2089) (Tao He)
  • [ea7a18952]: Fixes the typo in error message template (#2092) (Tao He)
  • [a69d884f3]: Closing the session if hearbeat fails for many times (when engine crashes) (#2088) (Tao He)
  • [2119d4851]: Add nodeselector arguments to control where the pod will be launched (#2087) (Tao He)
  • [9f1f64410]: Change nightly image tag to 'major.minor.alpha_patch' (#2075) (DongZe Li)
  • [1846e901a]: [GAE-Java] Make GAE Java SDK version align with GraphScope (#2077) (Zhang Lei)
  • [652b1c5d6]: [GAE-Java] Upgrade log4j version (#2082) (Zhang Lei)
  • [28809ddef]: [GIE-IR] Profiling for GIE-IR (#2056) (BingqingLyu)
  • [2d57a3d8b]: [GIE-IR] support sending back a page of results if batch size met (#2076) (shirly121)
  • [a7462ae13]: Fixes GetMessage bug in GAE java SDK (#2079) (Zhang Lei)
  • [c4572e69d]: [GIE-IR] support more patterns of predicate, i.e. P.not, P.inside, P.outside (#2068) (shirly121)
  • [c463b3287]: Enhancement and dependency clearing of groot. (#2043) (Siyuan Zhang)
  • [3af3e84e9]: [IR Compiler] fix expand fusion strategy (#2058) (shirly121)
  • [45e8c0503]: Change builtin apps(bfs,sssp,pagerank) to parallel version (#2073) (DongZe Li)
  • [56a3b9c14]: Update GAE Java Doc (#2059) (Zhang Lei)
  • [a7ff7b666]: [IR Runtime] fix bug in label encoding of exp_store (#2057) (BingqingLyu)
  • [1ddf147bf]: [IR Compiler] support more project patterns [id(), label(), constant(XX)] (#2066) (shirly121)
  • [5f4813fd4]: [Emergency Fix]Compile Property Graph with ENABLE_JAVA_SDK on needs jni.h (#2069) (Zhang Lei)
  • [94a013155]: Fix GAE-Bug: Cannot compile GAE app if java not installed (#2060) (Zhang Lei)
  • [7741be002]: test filter push down (#2003) (waruto)
  • [588ce2969]: [Pre-commit Hook] Update README.md (#2064) (shirly121)
  • [516b24aa8]: [Pre-commit Hook] add pre-commit hook to prevent committing sensitive information (#2055) (shirly121)
  • [2f30b2355]: Install graphx-runner only whe it is built (#2062) (Tao He)
  • [4c7be5f33]: Fixes CI on macOS (#2061) (DongZe Li)
  • [c35766a44]: GraphScope for Spark-GraphX (#2004) (Zhang Lei)
  • [96d3bb1ff]: Fixes an uninitialized field bug in FFI reader (#2054) (Tao He)
  • [0b7483f84]: Support statements like vd_type='str' in UDF app (#2052) (Siyuan Zhang)
  • [adbd4d1d4]: Add prometheus exporter for coordinator && GIE engine (#1947) (Vincent)
  • [e87e99776]: Fixes the GAE jar name when we have a alpha in version number (#2050) (Tao He)
  • [8bf6afe32]: [GAE-Java] Code refactor (#2049) (Zhang Lei)
  • [248c13b43]: The ExceptionUtils.getRootCause() may returns null, record the error and backtrace in such cases (#2048) (Tao He)
  • [cce30bb45]: Bump vineyard to 0.8.5 (#2047) (Zhang Lei)
  • [67a45aad5]: Tensorflow requires lower version of pandas (#2045) (Tao He)
  • [486283517]: Upgrade required vineyard to v0.8.4. (#2042) (Tao He)
  • [a2e0c83f6]: [GIE/IR] Support otherV() on Groot/Vineyard (#2034) (BingqingLyu)
  • [819ea4a96]: remove some unused files in sdk-common (#2026) (Siyuan Zhang)
  • [958c49dd2]: Avoid install java related targets if java sdk is not enabled. (#2037) (Tao He)
  • [2316f0731]: Fixes the path of gaia executor, add a dockerignore file as well (#2039) (Tao He)
  • [9cdd26f41]: Fix incorrect argc value in run_vy_app.cc (#2035) (Siyuan Zhang)
  • [dc0662ecc]: upgrade vineyard to 0.8.2 (#2025) (Zhang Lei)
  • [73c1882f5]: GAE java build refactor (#2028) (Zhang Lei)
  • [e7a6655f3]: [pegasus] update contains_source in progress (#2008) (Neng Li)
  • [dd4254473]: fix bug in extract_needed_columns (#2015) (waruto)
  • [159a27b24]: [GIE-IR] support SimplePath and AllV in path expand operator (#2005) (shirly121)
  • [1f9802188]: Add connected() in session.py to figure out session status (#2012) (dzhiwei)
  • [a2f56ce8f]: Ensure session close work when triggered not in main thread (#2009) (dzhiwei)
  • [71e0bf2cf]: Fixes a typo in the tutorial notebooks (#2007) (Tao He)
  • [6dff1dc29]: Use a getaddrinfo resolvable hostname as the local hostname (will be used as the etcd endpoint further). (#2006) (Tao He)
  • [be0708793]: Return primaryKey for vertex/edge property (#2001) (DongZe Li)
  • [7f389be2d]: remove useless code (#2002) (waruto)
  • [2d6868019]: impl new condition row filter and column filter (#1905) (waruto)
  • [2ac4f24af]: Fixes a incorrect collect type in ir (#2000) (waruto)
  • [8ab15d19b]: [GIE-Runtime] Refine structure Record in Runtime (#1992) (BingqingLyu)
  • [73b54533f]: [GIE-IR] support coin(..) for sampling vertices/edges (#1993) (shirly121)
  • [c8bf5b1e1]: No fatal even when meet unknown errors (#1991) (Tao He)
  • [ce4f0eace]: [GIE] fuse expand with count operators nested in apply for optimization (#1987) (shirly121)
  • [bd8252fc3]: Drop from vineyard when unloading a arrow projected fragment (#1984) (Tao He)
  • [23dcae0fb]: Add deployment of groot (#1858) (Siyuan Zhang)
  • [4aac5b579]: [IR Compiler] collect metrics from compiler to report qps/rt (#1978) (shirly121)
  • [7b20ab507]: [IR Compiler][Bug Fix] fix gaia-test ci caused by invalid group type used in compiler (#1979) (shirly121)
  • [48d3c315a]: [GAIA-IR] support patterns of group by multiple keys/values in compiler (#1973) (shirly121)
  • [44597c93c]: [Bug Fix] fix a bug in encode/decode in group (#1966) (BingqingLyu)
  • [7d8f8cfdf]: Fix network connection reconnect abnormal in pegasus (#1969) (bmmcq)
  • [c980ab37a]: Fixes the compliation error of run_property_ctx and addresses some lint errors (#1974) (Tao He)
  • [15fdf2f39]: Fix timeout error in graphscope-store helm test (#1972) (Siyuan Zhang)
  • [de490cd67]: Move compiler into interactive_engine (#1929) (Siyuan Zhang)
  • [2f7d1ec59]: Bump hadoop-common from 3.2.3 to 3.2.4 in /interactive_engine (#1963) (dependabot[bot])

New Contributors

Full Changelog: https://github.com/alibaba/GraphScope/compare/v0.16.0...v0.17.0

GraphScope - Release v0.16.0

Published by lidongze0629 about 2 years ago

We are bringing a number of improvements to GraphScope, alongside the GraphScope 0.16.0 release. This release introduces many new features on backend engines and system stability. We enable Jave SDK of Graph Analytics Engine (GAE) on MacOS, and make a series of enhancements on the graph storage Groot. We completely remove a legacy Graph Interactive Engine (GIE), while officially replacing it with the latest version based on an intermediate representation (IR) layer; in addition, we improve GIE via a couple of optimizations, new language features, and bug fixes. The Graph Learning Engine (GLE) now supports real-time sampling on dynamic graphs. Meanwhile, we start to release a nightly version every day, and you can try it with pip3 install graphscope --pre.

We highlight the following improvements included in this release:

1. Enhancements of the graph storage Groot:

  • Support load table from partitioned ODPS table;
  • Speed up some popular gremlin queries;
  • Use the GAIA engine to support queries;
  • A series of deployment enhancements.

2. Enhancements of the GIE GAIA engine:

  • Completely replacing the legacy GIE engine with the latest version based on IR;
  • Optimizing the logic of lazily fetching properties from the graph store, and implementing a new CSR-based storage for experiment purpose;
  • New language features including more grouping functions, more options for group/dedup/orderby keys, etc.

3. Dynamic-Graph-Service (DGS) is introduced in GLE.

  • DGS is an online inference service, it supports real-time sampling on dynamic graphs with streaming graph updates.

4. Support Java SDK of GAE for MacOS.

5. Bug fixes:

  • Fixes potential pointer leak in IR-core FFI functions;
  • Duplicate results while scanning with primary keys;
  • Error while accessing a tag that does not exist in the record.
  • Fixes failure of running clustering algorithm on the property graph.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.16.0

Commits

  • [9c656fa52]: [GAIA Runtime] Use hashbrown for optimization (#1952) (BingqingLyu)
  • [a27dd5c12]: Support for version suffixes like '0.15.0a1' in cmake (#1960) (DongZe Li)
  • [e6917769b]: upgrade protoc-gen-grpc-java version (#1830) (Siyuan Zhang)
  • [9a71bf4c9]: Fixed high CPU usage of ingestor when having empty buffer (#1946) (simple)
  • [3a2165da1]: fix: add ipc sender after former one removed (#1950) (Neng Li)
  • [e486c35e3]: [GAIA/exp_store] To enable getting neighbors of a certain label by la… (#1945) (Longbin Lai)
  • [7f0f05398]: [pegasus] fix bug in (#1954) (#1958) (bmmcq)
  • [009fd8c32]: Fixes nightly CI by overwrite VERSION (#1957) (DongZe Li)
  • [f6dcf166d]: Change nightly image tag to 'nightly' (#1951) (DongZe Li)
  • [84e8de89d]: Support to pre-release GraphScope(image + wheel package) nightly with version like 'major.minor.alpha_patch' (#1949) (DongZe Li)
  • [5d9b62938]: Fix typos in the tutorials (#1948) (Ye Cao)
  • [1de753389]: Fix missing dependency of ir-compiler (#1935) (Siyuan Zhang)
  • [a0da2a974]: [GAIA-IR] create ffi object from java side to avoid invoking ffi interface which can lead memory leak problems (#1925) (shirly121)
  • [9284f69c5]: [GraphProxy] Optimize LazyDetails (#1921) (BingqingLyu)
  • [98ae038f5]: Fixes the links for tutorials in the documentation. (#1928) (Tao He)
  • [52078d9a4]: Integrate targets of assembly artifacts in GIE (#1920) (Siyuan Zhang)
  • [5e93f732e]: [IR Compiler] fix bugs when path_expand is nested in sub traversal, i.e. select('a').by(out('1..2')) (#1874) (shirly121)
  • [281f264f9]: Support edges of outer vertices in DynamicFragment (#1799) (ds-ssj)
  • [489a07725]: Fixed graph schema drop and init error when edge_kinds's src or dst gone (#1922) (simple)
  • [aedbb31f3]: Fix compiling error of write-bench (#1918) (Siyuan Zhang)
  • [7009ccaa9]: [IR Compiler] create FfiNameOrId from java side to fix undefined error (#1919) (shirly121)
  • [7e826c67f]: Cleanup legacy files. (#1914) (Siyuan Zhang)
  • [8a04f8384]: Print the running time of each round of analytical engine (#1913) (DongZe Li)
  • [41fe90e41]: Add the etcd {client,peer} port to the configuration list (#1910) (Tao He)
  • [484f9278d]: Reopen groot gremlin test (#1912) (Siyuan Zhang)
  • [160bb0b08]: Resolve issues reported by dependent bot (#1909) (Siyuan Zhang)
  • [1591300bf]: impl partial_eq for Property and fix gitignore (#1908) (waruto)
  • [eb45c584c]: [GIE] Reorganize executor package by incorporating GAIA into GIE (#1883) (BingqingLyu)
  • [289990309]: Data load tools supports partition (#1900) (Siyuan Zhang)
  • [cd08a1a47]: Enhance the install_deps.sh script (#1873) (Weibin Zeng)
  • [a33b179a4]: [GAIA-IR] support more patterns of dedup().by() in ir compiler (#1901) (shirly121)
  • [7087915d7]: Supporting reading from a config file in htap loader. (#1897) (Tao He)
  • [b0c9931a0]: Fix a bug when iterating dict (#1894) (Siyuan Zhang)
  • [2ac521374]: [GAIA-IR] support more patterns of aggregate functions, i.e min/max/toList/toSet/avg/countDistinct (#1847) (shirly121)
  • [28508f6f8]: Fix docs words error (#1893) (Milittle)
  • [13aca2ec5]: add arm runtime dockerfile (#1880) (Siyuan Zhang)
  • [1e35af2b5]: Reopen releasing gss in workflow (#1882) (Siyuan Zhang)
  • [b22e45e4a]: [GAIA-IR] Support PathExpand ranging from 0 (#1863) (BingqingLyu)
  • [e96e6ec78]: Fixes the dtor error in the htap_loader. (#1884) (Tao He)
  • [7770814e5]: Adapt the launch cluster script to latest AWS API (#1876) (Weibin Zeng)
  • [9e1a3346c]: Add a standalone graph loader to vineyard. (#1869) (Tao He)
  • [e5cdf481d]: Avoid mvn install fastffi twice (#1871) (Zhang Lei)
  • [51296d129]: Decrease the make job numbers when building gae. (#1868) (Siyuan Zhang)
  • [ea825d69a]: Optimize the clustering algorithm by replacing the complex vertex_array_t with unordered_map (#1861) (Weibin Zeng)
  • [d024cdab3]: [Bug Fix] fix a bug in duplicated results by index scan (#1865) (BingqingLyu)
  • [fd9b55f90]: [GAIA] Make input data as generic data types in JobAssembly interface (#1859) (BingqingLyu)
  • [6616ee08a]: [IR Runtime] support mean() in runtime (#1852) (BingqingLyu)
  • [0a8e9c1d8]: [IR Compiler] support hasNot in ir compiler (#1853) (shirly121)
  • [a30384cfe]: [GAIA-IR] Filter the data when selecting a non-exist tag (#1843) (BingqingLyu)
  • [269578393]: fix groot and GIE test cases after replacing MaxGraph with gaia (#1823) (shirly121)
  • [959f6ff08]: [GAIA-IR] Fix invalid unwrap in ffi functions to prevent broken pointers (#1846) (shirly121)
  • [ffeebcd87]: [GIE/TEST] Add back CI Test for subgraph() (#1827) (BingqingLyu)
  • [a756bc1ca]: [GIE-GAIA] Add Pegasus named apis, and add names for IR Operators (#1834) (BingqingLyu)
  • [5698d98ba]: Add a TOC to the README. (#1838) (Tao He)
  • [1d9222738]: Fixes the long standing empty column name issue in subgraph (#1832) (Tao He)
  • [86ea24bcd]: Upgrade protoc version and use variable to replace hardcoded value (#1829) (Siyuan Zhang)
  • [2eb27008c]: Remove Zookeeper and zetcd related stuffs (#1828) (Siyuan Zhang)
  • [c35f34181]: [pegasus] fix defect in the progress control (#1796) (Neng Li)
  • [35b56f2d6]: Make Pegasus support reading config files with hostname instead of only IP (#1808) (Melo Yang)
  • [2fe38e8c9]: Update dependency versions. Add rust format check (#1821) (Siyuan Zhang)
  • [7373a7f25]: Add GRAPE-jdk support on mac (#1774) (Zhang Lei)
  • [36820c4f1]: [GAIA-IR] Support sum() in GAIA-Runtime (#1815) (BingqingLyu)
  • [f96094664]: [GIE/IR] Fix IR warnings (#1817) (Longbin Lai)
  • [bded1bf14]: use the same format file as query_service/ir (#1819) (waruto)
  • [a675598e5]: Workaround for pip failure to perform editable install with --user (#1813) (Weibin Zeng)
  • [d49476af5]: Introduce a new interactive engine (#1782) (Siyuan Zhang)
  • [889e00355]: Add degree threshold to katz_centrality, clustering and avg_clustering algorithm (#1507) (Siyuan Zhang)
  • [d1e507616]: [IR Runtime] Make v6d related codes as features (#1795) (BingqingLyu)
GraphScope - Release v0.15.0

Published by lidongze0629 over 2 years ago

We highlight the following fixes and improvements included in this release:

  • [GAIA-IR] Fixes the multi-join error in the plan of pattern matching
  • Supports bulk load from ODPS table and OSS for the Interactive Engine.
  • [GAIA-IR] Support the bothV() operator.
  • Compatible with the latest GLOG API of Analytical Engine.
  • Fixes some compilation errors in Learning Engine under Apple M1 chip.
  • [GAIA-IR] fix the protobuf encode error of logical plan in Interactive Engine.
  • Fixes bug while installing dependencies on ubuntu and update deprecated --all to --workspace

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.15.0

Commits

  • [b93a80942]: Fix nightly CI on macOS (#1797) (DongZe Li)
  • [f7c18d16f]: ignore a Cargo.lock (#1792) (waruto)
  • [175852558]: Impl PartialOrd and some methods for Property, then format it (#1788) (waruto)
  • [e8f0b98b5]: [Gaia/IR] Return json with possible error message in print_plan_as_json (#1786) (Longbin Lai)
  • [5af6ef948]: Fixes bug while install deps on ubuntu and update deprecated --all to --workspace (#1785) (waruto)
  • [314a3fc2a]: [GAIA-IR] fix a bug of encode protobuf of logical plan (#1784) (shirly121)
  • [0d8fca9cc]: [GAIA-IR] Fix compiling warnings in v6d_ffi (#1780) (BingqingLyu)
  • [07e86722f]: Fixes the failure of nightly CI on macOS by fixed the rust toolchain version to 1.60.0. (#1777) (DongZe Li)
  • [e0166e70b]: fix communication caused by broadcast (#1750) (Neng Li)
  • [b7960ba0f]: [GAIA-IR] Make v6d_ffi build as features in IR (#1775) (BingqingLyu)
  • [9abea6d4a]: support more than 2 branches to union (#1756) (Melo Yang)
  • [faadf9f05]: [GAIA-IR] Support writing a new graph in GAIA-IR (#1745) (BingqingLyu)
  • [3e449f981]: [IR Compiler] support TextP.containing/TextP.notContaing as within/without in ir_core (#1755) (shirly121)
  • [ec11bd169]: [IR Compiler] represent identity as AsNone in subgraph (#1752) (shirly121)
  • [cfabc90d6]: fix bug, or_predicates is a collection of AndPredicate that forms a logical OR of all AndPredicates (#1751) (waruto)
  • [2cba97080]: [IR Compiler] fix error when creating json file (#1743) (shirly121)
  • [b3f9afe80]: [GAIA-IR] support lazy projection for columns used only in a predicate (#1746) (shirly121)
  • [1ab5737f4]: Bump fastjson from 1.2.76 to 1.2.83 in /analytical_engine/java (#1732) (dependabot[bot])
  • [be3bf0ee4]: Bump fastjson from 1.2.75 to 1.2.83 in /interactive_engine (#1731) (dependabot[bot])
  • [f8448083b]: Upgrade vineyard version to fixes some build failures on Mac. (#1748) (Tao He)
  • [96ffefee2]: [GAIA-IR] support subgraph in ir compiler (#1721) (shirly121)
  • [a97a7791a]: Upgrade required graphlearn to fixes a build error on M1 Mac. (#1742) (Tao He)
  • [7af26c96c]: Compatible with the latest API of GLOG (#1738) (DongZe Li)
  • [bcbe64609]: Fix bugs while building on macOS 12 wih apple silicon (#1736) (waruto)
  • [f25fc429f]: Publish dataset image during release process (#1730) (DongZe Li)
  • [92f6564d3]: Multi task for bulk load (#1720) (Yongmin Hu)
  • [bcd37732a]: add crc to file meta store (#1726) (tianliplus)
  • [fe9d03383]: Supports launch vineyardd using on multiple hosts using MPI (#1727) (Tao He)
  • [afd61fd75]: [GAIA-IR] support bothV() in GAIA-IR (#1717) (BingqingLyu)
  • [deb47c9d2]: integrate gremlin query api into groot sdk and add authentication when querying (#1715) (shirly121)
  • [85a70b884]: Stores download OSS files with multithreading for GIE (#1714) (Yongmin Hu)
  • [333c3bead]: [GAIA-IR] Reorg for GAIA-IR's Runtime (#1709) (BingqingLyu)
  • [ded5b2efc]: Supports bulk load from ODPS table and OSS for GIE (#1672) (Yongmin Hu)
  • [0b49a06da]: [Bug fix - Gaia-IR] The plan of pattern matching faces a multi-join error (#1704) (Longbin Lai)
  • [66a4f06bb]: Disable publish the wheel package to Test.Pypi (#1700) (DongZe Li)
GraphScope - Release v0.14.0

Published by lidongze0629 over 2 years ago

We are delighted to present the release of GraphScope v0.14.0. This release is composed of many updates on backend engines and system stability. The new Graph Interactive Engine (GIE), GAIA-IR, has supported more operators for diverse graph queries. Meanwhile, the persistent storage of GraphScope is further enhanced with a series of new functions. In addition, we continuously work on improving the performance of GraphScope, and developer usability.

We highlight the following improvements included in this release:

1. New supported operators in Graph Interactive Engine (GIE)

  • Support valueMap operator to fetch all properties of vertices in the GAIA engine.
  • Enable string oid type support for the GIE GAIA engine.
  • Support index query for the GAIA engine.

2. More functions on persistent storage of GraphScope

  • Support loading dataset from the OSS.
  • Enable local GC for the Groot storage.

3. Bug fixes and other enhancements

  • Upgrade the NetworkX version from 2.6 to 2.8.
  • A new algorithm named voterank has supported in the analytical engine.
  • Fix some problems in the property cache of GIE GAIA IR-Core.
  • Support to return an empty map if no property in valueMap operation.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.14.0

Commits

  • [772e10749]: Bump karma from 3.0.0 to 6.3.16 in /python/jupyter (#1694) (DongZe Li)
  • [c2f1bd4b8]: set snapshot as extra params to get graph data of correct version in groot (#1666) (shirly121)
  • [b9093fc6c]: Enable local GC (#1653) (Siyuan Zhang)
  • [982d467af]: Increase the duration (#1695) (Siyuan Zhang)
  • [65523c6ef]: Optimize clustering by ignoring computation of vertices that degree less than 2 (#1597) (Weibin Zeng)
  • [5ad8d521c]: fuse expand/getV with the following filters to support more patterns in match (#1671) (shirly121)
  • [26887b514]: Disbale redefined warning of string_view (#1693) (DongZe Li)
  • [10b231f40]: Fix dependabot alert caused by crossbeam(#1687) (Neng Li)
  • [d5d714068]: Make CI filtering consistent between main and pull requests (#1686) (Siyuan Zhang)
  • [27e033846]: Enhance the script finding process when launching GIE from locally build without installation (#1690) (Tao He)
  • [275547ec9]: Support return a dict for graph schema (#1682) (DongZe Li)
  • [6e872a6e8]: Fixes the failure of nightly CI. (#1663) (Weibin Zeng)
  • [39c572bfb]: Merge gs-jython and gs-lib into graphscope package (#1680) (DongZe Li)
  • [3622ba8af]: Upgrade required vineyard version (#1655) (Tao He)
  • [c4a2f9ed6]: Upgrade Kafka version (#1660) (Siyuan Zhang)
  • [f06721e56]: Fix forward algorithm test import error which test dir not contain __init__ file (#1657) (Weibin Zeng)
  • [c5462af43]: Fixes the uninitialized pointer error for string fragment (#1659) (Tao He)
  • [b5bb4408d]: Return empty map if no properties in valueMap() (#1656) (shirly121)
  • [ce91b0ccf]: [GIE-IR] Upgrade Gaia-IR with the new Pegasus Client/Service APIs (#1648) (BingqingLyu)
  • [dc256c1bf]: [BugFix] Fix install zetcd error with GO111MODULE="auto" and remove the limitation of go version (#1651) (Weibin Zeng)
  • [59c85b0c4]: [GAIA/engine] Merge latest version pegasus engine (#1632) (Neng Li)
  • [593537580]: [GAIA-IR] Support index query in IR (#1618) (BingqingLyu)
  • [b5cee9c63]: Implement createGraph on coordinator, add API to initialize builder using schema (#1641) (Tao He)
  • [fcda4a687]: Add ipython to the distributed docker image. (#1643) (Tao He)
  • [0a50ed13f]: Enable string oid type support for FFI wrapper for GIE. (#1638) (Tao He)
  • [8fe16119f]: Add GC snapshot functionality for groot (#1635) (Siyuan Zhang)
  • [87b5613a2]: Support valueMap() to fetch all properties of vertices (#1623) (shirly121)
  • [b452508c4]: Fixes the bug in AddColumn to graphs with string oid type. (#1627) (Tao He)
  • [76d33884e]: [GAIA-IR] Map KeyId back into KeyName for labels and properties (#1615) (BingqingLyu)
  • [fa5718c96]: Listen 0.0.0.0 address in GAIA frontend (#1609) (DongZe Li)
  • [0d68477f1]: fix the join bug (#1608) (wzbxpy)
  • [4b33307d0]: [Bug Fix] Fix property cache in IR Core (#1606) (BingqingLyu)
  • [e83e8d9bc]: Implement voterank app (#1552) (liulx20)
  • [b5fff8686]: Fix networkx.cnli forward and reorg forward algorithm tests (#1584) (Weibin Zeng)
  • [33546c92e]: [IR Runtime] IR TagOpt for more compact structure in Runtime (#1594) (BingqingLyu)
  • [55d31d8a2]: Gie sdk basic auth (#1579) (tianliplus)
  • [b8ee7f063]: Clean up include headers of analytical engine (#1546) (Weibin Zeng)
  • [0aa8c3c6f]: refactor gie sdk (tianliplus)
  • [0115a729a]: Reorganize and cleanup proto definitions. (#1547) (Siyuan Zhang)
  • [8fbf1929b]: add ci tests for ir on vineyard (#1515) (shirly121)
  • [6060a6b45]: Restrict protobuf version (#1549) (Weibin Zeng)
  • [fb81a8291]: Make client and coordinator install with develop mode in MakeFile (#1511) (Weibin Zeng)
  • [95bd70166]: Simplify the python code of networkx and reorganize test cases (#1540) (Weibin Zeng)
  • [cbe749cd3]: Upgrade NetworkX version from 2.6 to 2.8 (#1532) (Weibin Zeng)
  • [c03b5395d]: Mark operation evaluated after the op is run (#1528) (Weibin Zeng)
  • [4c16d5364]: add oss support (#1481) (tianliplus)
  • [d784b2512]: Upgrade graph-learn version (#1516) (Weibin Zeng)
  • [9357e1317]: Revise DynamicToArrow with existed engine schema of DynamicFragment (#1501) (Weibin Zeng)
GraphScope - Release v0.13.0

Published by lidongze0629 over 2 years ago

We are delighted to announce the release of GraphScope v0.13.0. This release is focused on providing a Jupyterlab extension for GraphScope to make your graph computation workflows better. In addition, we continuously work on improving the performance of GraphScope, and developer usability. We highlight the following improvements included in this release:

1. Introduce a JupyterLab extension for GraphScope. Currently, it supports the following functions

  • Provide a graphical user interface for monitoring the status of graphscope resources (e.g., session and graph)
  • Support to define data schema and load graph in an interactive way
  • The extension has been integrated into our PlayGround

2. Performance improvement

  • Add data caching in NetworkX for improving the performance of graph reporter
  • Optimize performance of arrow fragment to dynamic fragment with multiple thread

3. BUG Fixes or other enhancements

  • Add a lock to protect the coordinator to avoid data race conditions
  • Fixes compilation failure in cdlp with ArrowFlattenFragment
  • Attempt backoff and retry during download dataset
  • Support primary key index in Groot
  • Support avg_clustering builtin application
  • Display progress bar during loading graph with k8s mode

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.13.0

Commits

  • [d0de89a55]: Disable brew update when installing dependencies as it is super slow. (#1506) (Tao He)
  • [ac1c1eaf2]: Upgrade graph learn version and force pyarrow<=6.0.0. (#1503) (Tao He)
  • [db579a7ad]: Move networkx graph schema from client to engine (#1485) (Weibin Zeng)
  • [617ae6312]: Display progress bar in k8s mode (#1495) (Siyuan Zhang)
  • [b035d0a7e]: Fix broken convert test of networkx (#1496) (Weibin Zeng)
  • [c833e372d]: Upgrade vineyard to 0.4.1 in Makefile (#1493) (DongZe Li)
  • [091b698de]: Update ScaleUp trigger in HorizontalRunnerAutoscaler (#1489) (Siyuan Zhang)
  • [ae9a34601]: Sync with latest changes about arrow fragment in vineyard, and include several refactors/bugfixes. (#1487) (Tao He)
  • [6df93c212]: Fix remove nodes logic in DynamicFragment (#1484) (Weibin Zeng)
  • [a75f33c8a]: Init the op of nx.Graph with incoming graphscope graph's op (#1479) (Weibin Zeng)
  • [2bc40ad5b]: Add another write path that uses the C++ IO adaptor (#1480) (Siyuan Zhang)
  • [23b612bce]: Remove the unused append-only arrow fragment. (#1477) (Tao He)
  • [4453f2e2a]: Suppress the grpc warnings in fork_posix.cc. (#1478) (Tao He)
  • [8d5a02939]: Optimize performance of arrow fragment to dynamic fragment with multi-threading (#1458) (Weibin Zeng)
  • [daa8649d4]: Support avg_clustering builtin app in client side (#1475) (DongZe Li)
  • [f27689c93]: Fix arrow version in Dockerfile (#1476) (Siyuan Zhang)
  • [05b0c3685]: Add del to nx.Graph to handle graph delete (#1464) (Weibin Zeng)
  • [2745d2525]: Move maxgraph*.jar into gs-jython package (#1472) (DongZe Li)
  • [4de8e6cc2]: groot support pk index (#1471) (tianliplus)
  • [ca9c7ad80]: Add a lock in the coordinator's RunStep as well. (#1465) (Tao He)
  • [0479c0a96]: Adapt CI on self-hosted runner to HorizontalRunnerAutoScaler (#1462) (Siyuan Zhang)
  • [90876cc06]: Attempt backoff and retry during download dataset (#1463) (DongZe Li)
  • [bbc7ebc8d]: Correctly handle the lifecycle of graph (#1460) (Siyuan Zhang)
  • [d52346279]: Add clean target to MakeFile to clean temporary files create from building (#1459) (Weibin Zeng)
  • [708ab4db8]: Delete intermediate files (#1456) (tianliplus)
  • [c4d2aefd9]: Bump hadoop-common from 2.10.1 to 3.2.3 in /interactive_engine (#1450) (dependabot[bot])
  • [28ca22997]: Correctly handle the empty dag (#1457) (Siyuan Zhang)
  • [632af342c]: Release the jupyter image in CI workflow (#1453) (DongZe Li)
  • [ea9193174]: Fix build wheel fail on mac (#1452) (Weibin Zeng)
  • [2cdeb8ec3]: Add data caching for networkx to improve the graph report performance (#1369) (Weibin Zeng)
  • [332ca48e4]: Add version check to coordinator & client, fixes #1435 (#1441) (wuyueandrew)
  • [9e2d77d83]: Fixes compilation failure in cdlp and sssp_path (#1447) (DongZe Li)
  • [46b30e450]: Add a lock to protect the coordinator to avoid data race condtions. (#1445) (Tao He)
  • [8af17dfb3]: Shrink gs-lib package to less than 100MB (#1440) (DongZe Li)
GraphScope - Release v0.12.0

Published by lidongze0629 over 2 years ago

The GraphScope v0.12.0 release is a major update on many aspects of the project including backend engines, APIs, and system stability. It introduces an intermediate representation (IR) layer into the graph interactive engine (GIE) GAIA, to decouple query languages from query execution engines. Meanwhile, this release supports Giraph APIs to allow Giraph apps running on the Graph Analytics Engine (GAE) of GraphScope.

We highlight the following improvements included in this release:

1. Introduce IR layer into GAIA:

  • A completely redesigned IR layer to decouple the query language dependency, and further for query optimizations.
  • Define the supported Gremlin's grammar via Antlr.
  • Support match step in Gremlin, for graph pattern matching.

2. Add Giraph APIs to GAE:

  • Support to load graphs with Giraph Formats, e.g., graph = sess.load_from(vertices="p2p-31.v", vformat="giraph:com.example.vformat",edges="p2p-31.e", eformat="giraph:com.example.vformat")
  • Support Giraph APIs on GraphScope, and all algorithms implemented towards the original Giraph APIs can run on GraphScope without modifications. For example, to load Giraph app SSSP, users can just invoke giraph_sssp = load_app(algo="giraph:com.alibaba.graphscope.example.giraph.SSSP")
  • Users now can add jars by sess.add_lib interface.

3. Bug Fixes or other enhancements:

  • Correct the lifecycle management of loaded graphs within a session.
  • Make the graphlearn and tensorflow related logs more user-friendly.
  • Fix readwrite failed on GraphScope Networkx module.
  • Enable to connect to the existed ETCD cluster in GraphScope.
  • Support to launch GraphScope on K8s from an inner-pod environment.

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.12.0

Commits

  • [b0a090082]: Support launch GraphScope on K8s from inner-pod environment (#1434) (DongZe Li)
  • [9e4e47f15]: Add offline deployment doc for Helm (#1433) (DongZe Li)
  • [477931537]: Migrate cargo rocksdb dependency to 0.18.0 version (#1430) (Zichao Zhang)
  • [2e74e3c47]: Split the builtin application distribution to 'gs-apps' package (#1429) (DongZe Li)
  • [47b0d42d2]: Support string tensor for tensor context (#1425) (Weibin Zeng)
  • [1068628fe]: Upgrade the version of databind, fixes the dependent bot warnings. (#1424) (Tao He)
  • [ab774812e]: Upgrade the required black version. (#1422) (Tao He)
  • [92c4b155e]: Attempt backoff and retry during handle GRPC error (#1421) (DongZe Li)
  • [2de6b7e93]: Bump commons-io from 2.6 to 2.7 in /research/query_service/ir/compiler (#1417) (dependabot[bot])
  • [edb2493e4]: Retry 3 times and then report failure during HeartBeat process (#1419) (DongZe Li)
  • [93ef1b56c]: Try to fixes nightly CI on macOS by packaging 'string_view' into wheel package (#1420) (DongZe Li)
  • [9d2c1e775]: Bump jackson-databind from 2.11.1 to 2.13.2.1 in /interactive_engine (#1415) (dependabot[bot])
  • [64c77c13c]: Add networkx tutorial and fixed vineyard version (#1412) (DongZe Li)
  • [27400d1f1]: Bump protobuf-java in /research/query_service/ir/compiler (#1416) (dependabot[bot])
  • [7d2f92c93]: Introducing IR query service (#1407) (BingqingLyu)
  • [88e767069]: [BugFix] Fix analytical engine can't not compile with NETWORKX=OFF (#1410) (Weibin Zeng)
  • [ed6546da0]: Push if condition down to steps to allow it be checkable. (#1404) (Tao He)
  • [07cb0dff7]: Remove unused dependencies (#1403) (Zhang Lei)
  • [bf6998587]: Fix networkx forward nightly ci (#1387) (Weibin Zeng)
  • [0191dd62e]: Add dummy CI worflows to allow all status checkable. (#1402) (Tao He)
  • [f5928a344]: Giraph on GraphScope implementation (#1386) (Zhang Lei)
  • [e851014bc]: Merge lastest version of pegasus (#1376) (Neng Li)
  • [0bf6686d9]: Upgrade vineyard to v0.3.21. (#1389) (Tao He)
  • [3b520cab0]: Remove gaia from GraphScope for code refactor (#1388) (shirly121)
  • [91f4248c7]: Add role and rolebinding for graphscope charts (#1382) (DongZe Li)
  • [c921cd622]: Try to upgrade tinkerpop version to 3.5.1 in maxgraph (#1349) (shirly121)
  • [a314bbbe6]: Revert "Giraph on GraphScope (#1362)" (#1385) (Tao He)
  • [c3903d5ee]: Refactor DynamicFragment with CSREdgecutFragmentBase and MutableCSR (#1328) (Weibin Zeng)
  • [0a9cda988]: Bump protobuf-java from 3.18.0 to 3.18.2 in /interactive_engine (#1381) (dependabot[bot])
  • [83ebed579]: Upgrade proto to 3.18.0 && grpc to 1.42.1 (#1149) (shirly121)
  • [ee81be7d6]: Bump zookeeper from 3.4.5 to 3.4.14 in /analytical_engine/java (#1372) (dependabot[bot])
  • [1dbc3735e]: Giraph on GraphScope (#1362) (Zhang Lei)
  • [4d8390aba]: Disable noisy log inside gremlin-python, and fixes the hard-coded DEBUG. (#1368) (Tao He)
  • [2c56d49f3]: Enable to connect to the existed ETCD cluster in GraphScope (#1359) (wuyueandrew)
  • [5ea7c0de7]: [Bug fix] Fix readwrite test failed on graphscope.nx (#1365) (Weibin Zeng)
  • [c72da15fe]: Makes the graphlearn- and tf- related logs looks better. (#1367) (Tao He)
  • [9d95f32e9]: Frontend async rpc (#1357) (tianliplus)
  • [b61457c5c]: Upgrade version of formatter (#1363) (Siyuan Zhang)
  • [79904fc8a]: Correct the lifecycle of graph with session (#1361) (DongZe Li)
  • [9f29a7bb2]: Catch unknown exceptions in gRPC handlers. (#1358) (Tao He)
  • [cf4284e1f]: Also tag latest when release new version (#1360) (Siyuan Zhang)
  • [8a81ea26a]: The GAIA ci should be disable months ago. (#1356) (Tao He)
  • [e4040dba0]: It doesn't make sense to trigger those CI actions on forks. (#1354) (Tao He)
  • [4f2a2e688]: Add an env USE_GAIA_ENGINE to allow testing gaia from Python client. (#1348) (Tao He)
GraphScope - Release v0.11.0

Published by lidongze0629 over 2 years ago

We are glad to announce a number of new features and improvements to GraphScope, alongside the GraphScope v0.11.0 release. This major release introduces mutable graphs into GraphScope, and adds GPU supports for the graph analytics engine (GAE). It also focuses on user-friendly improvements, code quality, and a series of bug fixes.

We highlight the following improvements included in this release:

Supporting mutable graphs:

  • Providing a set of interfaces (e.g., add_vertex, add_edge and update_vertex) to modify the topology of an existing graph.
  • Adopting existing apps to mutable graphs.

Accelerating graph analytics apps with GPUs:

  • Adding new fragment, worker, message_manager and apps to utilize GPUs to process graph analytics tasks faster.
  • On top of these new infrastructures, 6 example algorithms (BFS, WCC, SSSP, PR, CDLP, LCC) are implemented.

Optimizations and enhancements:

  • Stream grpc request(response) to support loading graph(fetching result) from(to) numpy/pandas more than 2GB.
  • Accelerating manipulation of graph topology by replacing folly::dynamic with rapidjson::value.

https://graphscope.io/blog/releasenotes/2022/03/06/release-notes-0.11.0.html

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.11.0

Commits

  • [f30b95c7]: Fixes CI failure on macOS (#1346) (DongZe Li)
  • [d911efe5]: Stop coordinator gracefully by accepting the signal from notebook restarted/stopped event (#1342) (DongZe Li)
  • [ff33f5a6]: Keep the global symbol to avoid it been stripped on Mac. (#1345) (Tao He)
  • [51f680aa]: Upgrade required vineyard version to 0.3.19 (#1339) (Siyuan Zhang)
  • [569c04b2]: Fixes PATH to not include $LLVM/bin to make sure the correct ranlib be found. (#1341) (Tao He)
  • [0b083e54]: Revert "Make sure the $PATH could search /usr/bin first. (#1340) (Tao He)
  • [158f4525]: Make sure the $PATH could search /usr/bin first. (#1338) (Tao He)
  • [385accd1]: Extract a util dynamic library from grape-engine (#1337) (Siyuan Zhang)
  • [e3f055ec]: Local bench groot (#1326) (tianliplus)
  • [953cc3e0]: Add timeout after launch vineyard to ensure we catches the error code correctly (#1332) (Siyuan Zhang)
  • [5e33f669]: Improve the docs. (#1324) (Tao He)
  • [e13eea9c]: Adapt to mutable fragment. (#1320) (luoxiaojian)
  • [f14b6f65]: Bump up graph-learn package for checkpoint. (#1323) (Tao He)
  • [ec3f3ef7]: Use LargeAttrValue in modify operation and dumps json with orjson (#1321) (Weibin Zeng)
  • [1a94a810]: Enable download metadata for repo 'appstream' of centos-8 (#1318) (DongZe Li)
  • [ec0ac9de]: Stream RunStepResponse of grpc to support fetching result more than 2GB (#1314) (DongZe Li)
  • [156c1c83]: Replace folly::dynamic with rapidjson::Value and bump vineyard to 0.3.18 (#1271) (Weibin Zeng)
  • [f3113bac]: Bump hadoop-common from 2.8.4 to 2.10.1 in /interactive_engine (#1312) (dependabot[bot])
  • [6918b764]: Stream RunStepRequest of grpc to support loading graph from numpy/pandas more than 2GB (#1309) (DongZe Li)
  • [cdb0dcfa]: Use boost::lexical_cast to handle string id fragment (#1310) (Siyuan Zhang)
  • [3b607ec2]: Bump httpclient in /research/query_service/gremlin/compiler (#1306) (dependabot[bot])
  • [0200ceb8]: Bump commons-io in /research/query_service/gremlin/compiler (#1307) (dependabot[bot])
GraphScope - Release v0.10.1

Published by lidongze0629 over 2 years ago

We highlight the following fixes and improvements included in this releases:

  • Support running built-in app on the property graph without project.
  • Support running UDF app on the property graph with string vertex id.
  • Improve the document of the learning graph.
  • Drop support for Python 3.6

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.10.1

Commits

  • [97d12bf5]: Fixes failure on macOS CI (#1304) (DongZe Li)
  • [10f25808]: Add 'large_attr' data structure in op definition for large chunk (data of pandas/numpy) (#1303) (DongZe Li)
  • [e554e936]: [networkx] binding session from graphscope graph and fix bug (#1302) (Weibin Zeng)
  • [3dc75f49]: Better schema representation (#1299) (Siyuan Zhang)
  • [66b74fd4]: Cython sdk rst: Context.superstep() change return type str -> int (#1296) (wuyueandrew)
  • [7eeb7259]: Print data type in graph schema (#1294) (Siyuan Zhang)
  • [66825203]: Check the nullptr of graph_info (#1295) (Siyuan Zhang)
  • [7fc69d99]: Fix: nightly CI was broken on test_error_on_selector case (#1291) (DongZe Li)
  • [fd676d77]: Fix the invalid identifier 'FULL-TEST-SUITE' in the environment variable (#1289) (DongZe Li)
  • [26f52147]: Support running udf app on property graph with string oid type (#1287) (DongZe Li)
  • [b683dcd7]: Raise an error and tell users what to do on Windows. (#1285) (Tao He)
  • [e8f7aaf4]: Improve the document of learning.Graph (#1277) (Weibin Zeng)
  • [976345bf]: Fixes building docs for graphlearning, and the scroll bar. (#1279) (Tao He)
  • [06e3392e]: Support running built-in app on the property graph by projecting to flattened fragment implicitly (#1258) (DongZe Li)
  • [0ba3d5e8]: Drop support for Python 3.6 (#1268) (Weibin Zeng)
  • [d3e85456]: Fix the broken networkx ci (#1264) (Weibin Zeng)
  • [1446cd6d]: Parallelize eigenvector_centrality and katz_centrality (#1262) (Weibin Zeng)
  • [38470b0d]: Revise built-in clustering and replace forward with built-in (#1255) (Weibin Zeng)
  • [334e6248]: Bump protobuf-java from 3.5.1 to 3.16.1 in interactive_engine (#1260) (dependabot[bot])
  • [83c68425]: Guard folly headers when NETWORKX=OFF (#1259) (Siyuan Zhang)
  • [a45a7b99]: Optimize modification APIs of graphscope.nx (#1250) (Weibin Zeng)
  • [1ae368f3]: Add metrics for benchmark groot (#1238) (tianliplus)
GraphScope - Release v0.10.0

Published by lidongze0629 almost 3 years ago

We are glad to announce the availability of GraphScope v0.10. This release supports users to run GraphScope on Macs powered by Apple's new M1 chip. In addition, it allows to serialize/ deserialize graph data to/from the disk under the standalone mode.

We highlight the following improvements included in this release:

  • Add Apple's M1 chip support.
  • Improve the implementation and documentation of context selectors.
  • Support serialization/deserialization of graph data on the standalone mode.
  • Refactor Java SDK in Graph Analytics Engine.
  • Fix a bug when loading graph data from HDFS.
  • Fix got empty result in all_simple_path algorithm with num_workers=1

Docker Image

docker pull registry.cn-hongkong.aliyuncs.com/graphscope/graphscope:0.10.0

Commits

  • [c49b1b21]: Format java source code of analytical engine (#1252) (DongZe Li)
  • [306da050]: Refactor GAE Java (#1227) (Zhang Lei)
  • [64afab7d]: Fixed the auditwheel version to 5.0.0 (#1251) (DongZe Li)
  • [847a6a29]: Quickfix: a syntax error. (#1248) (Tao He)
  • [38ad3def]: Fixes the long-standing "Event loop is closed" issue. (#1247) (Tao He)
  • [53316583]: Bring back mars support, re-enable mars test in CI, and fixes several related issues. (#1246) (Tao He)
  • [9b32ccfc]: Enable serialization test on K8s, and add a new case to test local serialization (#1245) (Tao He)
  • [819b9709]: Enable customization of storage class (#1244) (Siyuan Zhang)
  • [038e915a]: Print diffs when failed to format java source code. (#1243) (Tao He)
  • [529ecd22]: Consistent label checking in client side (#1242) (Siyuan Zhang)
  • [a7d1472f]: Remove unrelated file from gh-pages (#1241) (Siyuan Zhang)
  • [89f03da1]: Add documents about potential permission error when allocating PV in NFS. (#1239) (wuyueandrew)
  • [db9b427d]: Format java code with google style (#1233) (DongZe Li)
  • [e37f69c1]: Upgrade log4j2 (#1235) (tianliplus)
  • [e1e22a42]: Fix log not fetched to client (#1236) (Siyuan Zhang)
  • [2e7e6399]: Changes on property applications (#1228) (DongZe Li)
  • [ab75ad71]: Fix error when selecting from empty graph (#1231) (Siyuan Zhang)
  • [9f8e7f21]: etcd: Increate the ETCD pods period seconds (#1213) (Hao Xin)
  • [0c60e6de]: Bump up required vineyard version to v0.3.14 and reable HDFS IO tests. (#1229) (Tao He)
  • [84ca6cd8]: Fix some typo in persistent_graph_store.rst (#1232) (wuyueandrew)
  • [e26e5330]: Fixed a typo in persistent_graph_store.rst (#1226) (wuyueandrew)
  • [06de1f17]: Make forward nightly ci pass (#1223) (Weibin Zeng)
  • [5b724b7a]: Fix codecov coverage ratio (#1224) (DongZe Li)
  • [a8129084]: Make GraphScope support on Apple M1 chip (#1221) (DongZe Li)
  • [f2ed9a14]: Typo of conn (#1222) (Siyuan Zhang)
  • [9e912363]: Upload codecover report only in ubuntu20 platform (#1220) (DongZe Li)
  • [d7140119]: add edit_on_github button on docs. (#1214) (Jingbo Xu)
  • [1eacd877]: Make the definition of src in sssp and bfs accurate (#1218) (Siyuan Zhang)
  • [ed4a82b3]: Report codecover of nightly CI (#1219) (DongZe Li)
  • [62c0676c]: [networkx] Revise the add_node/edge with cache and upgrade NetworkX dependency to 2.6 (#1208) (Weibin Zeng)
  • [a6099b69]: Enhanced input validation and schema representation (#1205) (Siyuan Zhang)
  • [8cdc1b6b]: Rename occurances of "libvineyard" to "v6d". (#1202) (Tao He)
  • [ec000ba9]: Support compiling analytical engine on Apple M1 chip (#1200) (DongZe Li)
  • [76d4641e]: [networkx] Update the built-in app implementation to make the behavior consistent with NetworkX implementation (#1176) (Weibin Zeng)
  • [3049ef92]: Update codecov/codecov-action to v2. (#1193) (Tao He)
  • [1c62d2a4]: Increase the default resource in helm deployment (#1189) (DongZe Li)
GraphScope - Release v0.9.1

Published by lidongze0629 almost 3 years ago

We highlight the following fixes included in this release:

  • Fix compile error of analytical engine app on google colab
  • Fix failure of import graphscope.nx module under python3.6
  • Fix the problem that openmpi cannot be found during compiling app in macOS
  • Fix failure during running app_assets twice.
  • Upgrade log4j2
Package Rankings
Top 2.94% on Pypi.org
Top 8.17% on Proxy.golang.org
Badges
Extracted from project README
Docs-en Docs-zh Translation Playground Open in Colab
Related Projects
graphvite

GraphVite: A General and High-performance Graph Embedding System

16 Jul 2019 1,220
OpenMLDB

OpenMLDB is an open-source machine learning database that provides a feature platform computing c...

12 Mar 2021 1,550