neo4j-spark-connector

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

APACHE-2.0 License

Stars
312
View Code on GitHub Visit Website
Ecosystems: Neo4j, Apache Spark

Bot releases are hidden (Show)

neo4j-spark-connector - 5.3.2 Latest Release

Published by ali-ince about 1 month ago

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.3.1...5.3.2

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.3.1

Published by ali-ince 4 months ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

What's Changed

New Contributors

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.3.0...5.3.1

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.3.0

Published by fbiville 8 months ago

What's Changed

⭐ Release Highlights

This introduces support for type/existence constraints based on the DataFrame definition and improves the handling of existing constraints.

Check out the documentation to learn more!

📜 Change Log

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.2.0...5.3.0

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.2.0

Published by fbiville about 1 year ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

Spark baseline version

This new release bumps the required Spark version to 3.4, in order to support the new push down Top N optimization.

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.1.0...5.2.0

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.1.0

Published by ali-ince about 1 year ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.0.3...5.1.0

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.0.3

Published by fbiville over 1 year ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

  • neo4j-connector-apache-spark_2.12-5.0.3_for_spark_3.jar, md5 46775288407098da460c5668db6fedea
  • neo4j-connector-apache-spark_2.13-5.0.3_for_spark_3.jar, md5 72727cc4b399d0fe17926d8a43d91c86

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.0.2...5.0.3

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.0.2

Published by conker84 over 1 year ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

  • neo4j-connector-apache-spark_2.12-5.0.2_for_spark_3.jar, md5 6033893da4ef48be27a9fa2055b681b0
  • neo4j-connector-apache-spark_2.13-5.0.2_for_spark_3.jar, md5 c94d6247bb5a6862fefde7d98533c0f0

What's Changed

New Contributors

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.0.1...5.0.2

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.0.1

Published by conker84 over 1 year ago

Neo4j Apache Spark

Hello Community!
We're happy to announce a new release of the Neo4j Spark Connector!
There are two files:

  • neo4j-connector-apache-spark_2.12-5.0.1_for_spark_3.jar, md5 08f1fa9e936eab63aa380ac50f30f2e0
  • neo4j-connector-apache-spark_2.13-5.0.1_for_spark_3.jar, md5 692af135dfdf0d188e9c9b2d944996ef

Below are the changes.

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/5.0.0...5.0.1

neo4j-spark-connector - Neo4j Connector for Apache Spark 5.0.0

Published by conker84 about 2 years ago

Neo4j Apache Spark

Hello Community!

We are pleased to announce the 5.0.0 of the Spark Connector which is the first official release that fully supports Neo4j 5.0 and Neo4j 4.4. For earlier versions of <= Neo4j 4.3, please use Neo4j Apache Spark 4.2.

N.B Because of a bug in Spark, the Aggregate Pushdown feature can only be activated with Spark >3.2.2 (please see https://github.com/apache/spark/pull/36727)

There are three files:

  • neo4j-connector-apache-spark_2.12-5.0.0_for_spark_3.jar, md5 ce814cd8d0a34f14599302895be13956
  • neo4j-connector-apache-spark_2.13-5.0.0_for_spark_3.jar, md5 39e5ebb2b71a3f93d3230db91f25e2db

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.5...5.0.0

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.2.0

Published by conker84 about 2 years ago

Neo4j Apache Spark

Hello Community!
We are pleased to announce the 4.2.0 of the Spark Connector with this version we deprecated the support for Spark 2.4 and support only Spark versions from 3.2.
With this release we added the support for Spark's Aggregate Pushdown which allows doing aggregations directly in Neo4j.

Nota bene Given a Spark bug the Aggregate Pushdown feature can be activated only from Spark 3.2.2 (please see https://github.com/apache/spark/pull/36727)

There are three files:

  • neo4j-connector-apache-spark_2.12-4.2.0_for_spark_3.jar, md5 6de1f93ece9f45aa51486317a62620f0
  • neo4j-connector-apache-spark_2.13-4.2.0_for_spark_3.jar, md5 a79cf2b9d8d6725587760fa149932be2

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.5...4.2.0

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.5

Published by conker84 about 2 years ago

Neo4j Streams Spark Connect

Hello Community!
We are pleased to announce the 4.1.5 of the Spark Connector with this version we added the support for Neo4j Java Driver URL resolvers.

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.5_for_spark_2.4.jar, md5 118dce27b54f16f304880df9c7280615
  • neo4j-connector-apache-spark_2.12-4.1.5_for_spark_2.4.jar, md5 705196b006434b659ea6c1601ec0c00a
  • neo4j-connector-apache-spark_2.12-4.1.5_for_spark_3.jar, md5 2ed1f9a89bd900b1aac1950fcbc85ee5
  • neo4j-connector-apache-spark_2.13-4.1.5_for_spark_3.jar, md5 caed49e951e9b8d4db12345799795481

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.4...4.1.5

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.4

Published by conker84 about 2 years ago

Neo4j Streams Kafka Connect (7)

Hello Community!
We are pleased to announce the 4.1.4 of the Spark Connector with this version we added the support for the logging with the Neo4j Java driver.

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.4_for_spark_2.4.jar, md5 cf9baef98e7d9214ef77f8b4bccbd87a
  • neo4j-connector-apache-spark_2.12-4.1.4_for_spark_2.4.jar, md5 9a56ec8a7d87c6683e4356063714f37f
  • neo4j-connector-apache-spark_2.12-4.1.4_for_spark_3.jar, md5 48f03d73b6df80f3bea254266b327f2f
  • neo4j-connector-apache-spark_2.13-4.1.4_for_spark_3.jar, md5 a9c9a01198d7e9f820ae9d120d32472c

What's Changed

New Contributors

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.3...4.1.4

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.3

Published by conker84 about 2 years ago

Neo4j Streams Kafka Connect (7)

Hello Community!
We are pleased to announce the 4.1.3 of the Spark Connector with this version we fixed several bugs and introduce the support for Scala 2.13!

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.3_for_spark_2.4.jar, md5 3746283be881449b712be5144d01fbf1
  • neo4j-connector-apache-spark_2.12-4.1.3_for_spark_2.4.jar, md5 c60b21b2af94383699cc30bb77da07b1
  • neo4j-connector-apache-spark_2.12-4.1.3_for_spark_3.jar, md5 c58749f75e5f052f447589dc594b1b1c
  • neo4j-connector-apache-spark_2.13-4.1.3_for_spark_3.jar, md5 fcf7e7b4a489aaa902f47a31fed25ba8

What's Changed

New Contributors

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.2...4.1.3

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.2

Published by conker84 over 2 years ago

Neo4j Streams Kafka Connect (7)

Hello Community!
We are pleased to announce the 4.1.2 of the Spark Connector where we fixed several bugs.

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.2_for_spark_2.4.jar, md5 ab32bdcc08d9fcedca845c8e4f92091f
  • neo4j-connector-apache-spark_2.12-4.1.2_for_spark_2.4.jar, md5 b763ddc4b63ebf5471a2eedf86e89a98
  • neo4j-connector-apache-spark_2.12-4.1.2_for_spark_3.jar, md5 c29c9b7ba6f3532c4db14836cf95ba6c

What's Changed

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.1...4.1.2

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.1

Published by conker84 over 2 years ago

Neo4j Streams Kafka Connect (7)

Hello Community!
We are pleased to announce the 4.1.1 of the Spark Connector where we fixed several bugs.

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.1_for_spark_2.4.jar, md5 2accb30aa039ddabc298bd7c8206e234
  • neo4j-connector-apache-spark_2.12-4.1.1_for_spark_2.4.jar, md5 9dd2022da450fa14a4da90bda1cbdbe2
  • neo4j-connector-apache-spark_2.12-4.1.1_for_spark_3.jar, md5 6bfe16d65f57e1469dabf47d963a62e3

What's Changed

New Contributors

Full Changelog: https://github.com/neo4j-contrib/neo4j-spark-connector/compare/4.1.0...4.1.1

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.1.0

Published by conker84 almost 3 years ago

Neo4j Streams Kafka Connect (7)

Hello Community!
We are pleased to announce the 4.1.0 of the Spark Connector where we fixed several bugs and introduced the official support for Spark's Structured Streaming APIs with Source and Sink!

This release officially enables Spark Structured Streaming API and lets you connect Neo4j (via Spark) to systems such as Kinesis, Kafka, Google Dataproc, Azure Service Bus, and any other which support Spark Structured Streaming.

There are three files:

  • neo4j-connector-apache-spark_2.11-4.1.0_for_spark_2.4.jar, md5 2cde396181f79950a541ca2e6e218909
  • neo4j-connector-apache-spark_2.12-4.1.0_for_spark_2.4.jar, md5 982292aeb829d06fc6571cb21e850dfd
  • neo4j-connector-apache-spark_2.12-4.1.0_for_spark_3.jar, md5 76016cdd45cb0ca929172cb723dfa3b6

List of resolved issues

  • Fixes #343: add Streaming docs (#346)
  • fixes #386: Streaming Source dups record in Databricks env (#387)
  • fix python failing tests (#385)
  • fixes #370: Sort Partitions by internal ID (#382)
  • fixes #348: repartition(1) - Makes loading very slow (#356)
  • Fixes #369: add parameters to filters (#377)
  • fix to clean up python in an example program (#380)
  • Bump driver version to 4.3.2 (#379)
  • fixes #262: spark 3 source (#374)
  • fixes #262: part 3/4: Spark 2.4 Source (#371)
  • Fixes #358: Ensure the same timezone is used across JVM and Neo4j (#359)
  • Fixes #364: add tests for Neo4j 4.3 (#365)
  • Fixes #361: Change constraint naming convention (#363)
  • fixes #357: transformed isValidQuery severity from ERROR to DEBUG (#360)
  • added javadoc generation
  • Update quickstart.adoc
neo4j-spark-connector - Pre Release of Neo4j Connector for Apache Spark 4.0.3

Published by conker84 over 3 years ago

Neo4j Streams Kafka Connect (5)

Hello Community we are pleased to announce the 4.0.3 pre-release (not meant to be used in production and with no enterprise support guaranteed) of the Spark Connector where we fixed several bugs and introduced the support for Spark's Structured Streaming Preview with Source and Sink!

This pre-release enables the spark streaming API and lets you connect Neo4j (via Spark) to systems such as Kinesis, Kafka, Google Dataproc, Azure Service Bus, and any other which support spark streaming

There are three files:

  • neo4j-connector-apache-spark_2.11-4.0.2_for_spark_2.4.jar for Spark 2.4 and Scala 2.11 with md5 3d56400faf10d6c0a3544cb69ed0056a
  • neo4j-connector-apache-spark_2.12-4.0.2_for_spark_2.4.jar for Spark 2.4 and Scala 2.12 with md5 67b4d9a9fcf3d4b484e8d4a67fc33dea
  • neo4j-connector-apache-spark_2.12-4.0.2_for_spark_3.jar for Spark 3.x and Scala 2.12 with md5 aee927b3f1eedccfbed434c61fe2a839

For the Structured Streaming support -- documentation has not yet been published, but there is a PR where we're collecting feedback, please look at it and add ideas about how to improving it!

Please fill an issue if you have a problem!

Following the full list of changes:

Bug Fixes

fixes #315: wrong recursive call in type conversion (#321)
fixes #357: transformed isValidQuery severity from ERROR to DEBUG (#360)
fixes #348: repartition(1) - Makes loading very slow (#356)

Enhancements

fixes #333: Remove Transaction code LockClientStopped (#336)
test improvements (#319)
#262 part 1/4: Spark 24 Sink (#322)
Fixes #349: Create issue templates (#352)
Fixes #262: add Structured Streaming write support for Spark 3 (#344)
fixes #354: Change org of package to org.neo4j for maven central publishing
update docs for maven central release (#355)
Fixes #361: Change constraint naming convention (#363)
Fixes #358: Ensure the same timezone is used across JVM and Neo4j (#359)
fixes #262: part 3/4: Spark 2.4 Source (#371)
fixes #262: spark 3 source (#374)
Bump driver version to 4.3.2 (#379)
Fixes #369: add parameters to filters (#377)

Other/Infrastructure Enhancements

fix to clean up python in an example program (#380)
Only run docs workflow when something changed in doc folder (#367)
Disable lint:links task for now (#368)

neo4j-spark-connector - Neo4j Connector for Apache Spark 4.0.2

Published by conker84 over 3 years ago

Neo4j Spark Connector

Hello Community we are pleased to announce the 4.0.2 release of the Spark Connector where we fixed several bugs!

There are three files:

  • neo4j-connector-apache-spark_2.11-4.0.2_for_spark_2.4.jar for Spark 2.4 and Scala 2.11 with md5 3d56400faf10d6c0a3544cb69ed0056a
  • neo4j-connector-apache-spark_2.12-4.0.2_for_spark_2.4.jar for Spark 2.4 and Scala 2.12 with md5 67b4d9a9fcf3d4b484e8d4a67fc33dea
  • neo4j-connector-apache-spark_2.12-4.0.2_for_spark_3.jar for Spark 3.x and Scala 2.12 with md5 aee927b3f1eedccfbed434c61fe2a839

Please fill an issue if you have a problem!

For any question please refer to the Neo4j Community Forum

Following the full list of changes:

Bug Fixes

  • Fixes #350: added ‘six’ python dependency (#351)
  • fixes #323: Node keys are ignored when the field is a map (#330)
  • Fixes #337: Implemented user defined schema on Spark 2.4 and Spark 3
  • fixes #333: Remove Transaction code LockClientStopped (#336)
  • fixes #326: Schema retrieving by APOC doesn’t seem to work when node maps is true
  • fixes #315: wrong recursive call in type conversion (#321)

Enhancements

  • implemented BasePartitioReader in Spark 2.4 (#335)
  • Update java driver to 4.2.5 (#332)

Other/Infrastructure Enhancements

  • Fixed version typo on README.md
  • Update lint-links.js (#340)
neo4j-spark-connector - Neo4j Connector for Apache Spark 4.0.1

Published by conker84 over 3 years ago

Neo4j Spark Connector

Hello Community we are pleased to announce the 4.0.1 release of the Spark Connector, we fixed several bugs and now we have the first iteration with the Support of Spark 3.x!

There are three files:

  • neo4j-connector-apache-spark_2.11-4.0.1_for_spark_2.4.jar for Spark 2.4 and Scala 2.11
  • neo4j-connector-apache-spark_2.12-4.0.1_for_spark_2.4.jar for Spark 2.4 and Scala 2.12
  • neo4j-connector-apache-spark_2.12-4.0.1_for_spark_3.jar for Spark 3.x and Scala 2.12

Please fill an issue if you have a problem!

For any question please refer to the Neo4j Community Forum

Following the full list of changes:

  • added spark packages zip generation script (#249)
  • Taskless: update README with correct package name (#248) …
  • fixes #193: Add testing against Neo4j 4.2 (#251)
  • updated 4.2 profile to official docker image (#258)
  • fixes #256: Improve quoting function (#257)
  • fixes #254: Returning a function from a query breaks in 3.5 (#255)
  • Fixes #261: Incorrect pushdown query for relationship (#268) …
  • Fixes #265: handle conversion to Neo4j map (#267) …
  • Update quick-java-example.adoc (#270) …
  • Update quick-java-example.adoc (#275)
  • Update aura.adoc (#287)
  • #206: Split current code into common and spark-2.4 modules (#282) …
  • #206 Add Spark 3.0 support (#283) …
  • fixes #286: improved relationship doc (#288)
  • Fixes #266: added test to ensure arrives to the Neo4j Driver (#290) …
  • Fix and check links with hyperlink (#271) …
  • Fix italic formatting of database option default (#294)
  • Fixes #281: Add Aura integration tests (#293) …
  • Fixes #291: remove build warnings (#292)
  • fixes #260: Move CI system to GitHub Actions (#301)
  • Fixes #213: Add Python tests (#296)
  • Fixes #299: Add docs for installing the connector (#305) …
  • Fixes #302: Test for Spark 3.1 (#304)
  • Fixes #284: leverage Spark Config as default configuration values (#306)
  • Add details on using Databricks with Aura (#303)
  • fixes #259: Neo4j 4.2.1 drivers (#307)
  • Fixes #285: Add version check for Spark (#308)
  • Taskless - Bump version (#311)
  • Fix JAR naming across docs and README (#312)
  • Update python-ci.yml (#317)
neo4j-spark-connector - [PRE-RELEASE][4.0.1-pre1] Neo4j Connector for Apache Spark

Published by conker84 over 3 years ago

Neo4j Spark Connector

Hello Community we are pleased to announce the 4.0.1-pre1 release of the spark connector, we fixed several bugs and now we have the first iteration with the Support of Spark 3.0!

Please note that this is not meant to be used in a production env

There are three files:

  • neo4j-connector-apache-spark_2.11_2.4-4.0.1-pre1.jar for Spark 2.4 and Scala 2.11
  • neo4j-connector-apache-spark_2.12_2.4-4.0.1-pre1.jar for Spark 2.4 and Scala 2.12
  • neo4j-connector-apache-spark_2.12_3.0-4.0.1-pre1.jar for Spark 3.0 and Scala 2.12

Please test it and fill an issue if you have a problem!

See you soon with the official release!

Package Rankings
Top 23.02% on Repo1.maven.org
Related Projects
azure-cosmosdb-spark

Apache Spark Connector for Azure Cosmos DB

30 Nov 2016 201
spark-fits

FITS data source for Spark SQL and DataFrames

31 Jan 2018 20
spark-ecs-connector

[Archived] ArchiveECS connector for Apache Spark

15 Feb 2016 7
spark-doc-zh

Apache Spark 官方文档中文版

17 Jul 2017 1,187
sparksql-for-hbase

Learn how to use Spark SQL and HSpark connector package to create / query data tables that reside...

31 Aug 2017 69
azure-event-hubs-spark

Enabling Continuous Data Processing with Apache Spark and Azure Event Hubs

04 Sep 2015 233
spark

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.

22 Apr 2019 2,020
spark-cassandra-connector

DataStax Connector for Apache Spark to Apache Cassandra

27 Jun 2014 1,931
spark-utils

Basic framework utilities to quickly start writing production ready Apache Spark applications

14 Apr 2018 36
spark3D

Spark extension for processing large-scale 3D data sets: Astrophysics, High Energy Physics, Meteo...

31 Jan 2018 30
setl

A simple Spark-powered ETL framework that just works 🍺

20 Dec 2019 177
mongo-spark

The MongoDB Spark Connector

20 May 2015 708
spark

Performance Observability for Apache Spark

28 Sep 2023 171
spark-redis

A connector for Spark that allows reading and writing to/from Redis cluster

16 Aug 2015 936
kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to ha...

01 Jun 2020 443