big-data-types

A library to transform Scala product types and Schemes from different systems into other Schemes. Any implemented type automatically gets methods to convert it into the rest of the types and vice versa. E.g: a Spark Schema can be transformed into a BigQuery table.

APACHE-2.0 License

Stars
12
Committers
5

Bot releases are hidden (Show)

big-data-types - v1.3.4 Latest Release

Published by JavierMonton 3 months ago

What's Changed

Full Changelog: https://github.com/data-tools/big-data-types/compare/v1.3.3...v1.3.4

big-data-types - v1.3.3

Published by JavierMonton 7 months ago

What's Changed

Full Changelog: https://github.com/data-tools/big-data-types/compare/v1.3.2...v1.3.3

big-data-types - v1.3.2

Published by JavierMonton 11 months ago

What's Changed

Full Changelog: https://github.com/data-tools/big-data-types/compare/v1.3.1...v1.3.2

big-data-types - v1.3.1

Published by JavierMonton over 1 year ago

What's Changed

Full Changelog: https://github.com/data-tools/big-data-types/compare/v1.3.0...v1.3.1

big-data-types - Big Data Types v1.3.0

Published by JavierMonton about 2 years ago

Big Data Types v1.3.0

  • Added BigQuerySchemas as an interface to create Schemas without creating tables
  • Updated Scala to Scala 3.1.X
    • This version is no longer compatible with Scala 3.0.X
    • The Scala version has been upgraded due to some dependencies (Circe - ScalaTest) that have updated Scala to 3.1.X

What's Changed

Full Changelog: https://github.com/data-tools/big-data-types/compare/v1.2.0...v1.3.0

big-data-types - Big Data Types v1.2.0

Published by JavierMonton over 2 years ago

Big Data Types v1.2.0

  • New module for Circe (JSON)
    • Conversion from Circe to other types
    • New docs
    • More examples
big-data-types - Big Data Types v1.1.2

Published by JavierMonton over 2 years ago

Big Data Types v1.1.2

  • Fixed minor issues on Cassandra parser
big-data-types - Big Data Types v1.1.1

Published by JavierMonton almost 3 years ago

Big Data Types v1.1.1

  • BigQuery
    • Fixed an error while converting Field into other types without using Schema
big-data-types - Big Data Types v1.1.0

Published by JavierMonton almost 3 years ago

Big Data Types v1.1.0

  • Cassandra
    • Added conversion from Cassandra to other types
    • Added a parser and parsing errors
    • Docs improved
    • Examples from other types to Cassandra
  • BigQuery
    • Added an extension method to extract Schemas from BigQuery Fields (myInstance.asBigQuery.schema)
big-data-types - v1.0.0

Published by JavierMonton almost 3 years ago

Big Data Types v1.0.0

  • Better public API definition
  • Extension methods renamed to a better name (This is a breaking change)
big-data-types - Spark module for Scala 2.13

Published by JavierMonton about 3 years ago

Big Data Types v0.5.2

  • Spark
    • Added support for Spark in Scala 2.13
    • Updated to Spark 3.2.0
big-data-types - Big Data Types v0.5.1

Published by JavierMonton about 3 years ago

Big Data Types v0.5.1

  • Spark:
    • Added a missing implementation for derive instances into SparkSchemas
    • Added methods to convert Instances into Spark inside SparkSchemas.
    • New extension methods that will allow any other instance to be converted into Spark types
  • BigQuery: Fixed some names in extension methods
  • New cross module examples in Examples module
    • Added examples of conversions from BigQuery to Spark and Cassandra
  • Fixed some anchors in documentation
big-data-types - v0.5.0 - BigQuery Reverse Conversion

Published by JavierMonton about 3 years ago

Big Data Types v0.5.0

  • BigQuery: Reverse conversion.
    This allows any BigQuery object (Schema, Field or basic types) to be converted into any of the other implemented types
  • Upgrade to Scala 3.0.1
big-data-types - Cassandra Module

Published by JavierMonton over 3 years ago

Big Data Types v0.4.0

  • Cassandra module added
  • More cross examples added
big-data-types - Big-data-types v0.3.5

Published by JavierMonton over 3 years ago

Big Data Types v0.3.5

  • Added support for Scala 3.0.0
big-data-types - v0.3.4 Scala 3.0.0-RC3

Published by JavierMonton over 3 years ago

Scala 3.0.0-RC3

big-data-types - v0.3.3 BigQuery - Scala3

Published by JavierMonton over 3 years ago

Big Data Types v0.3.3

  • BigQuery module compiled for Scala 3
    -- Some imports refactored in Core Scala2 to be equal to Scala3 (related to enums)
big-data-types - v0.3.2 - Core migrated to Scala3

Published by JavierMonton over 3 years ago

Big Data Types v0.3.2

  • Core module migrated to Scala 3
  • Build changed for Scala 3 versions
  • Shapeless removed
  • Code coverage removed due it isn't migrated yet to Scala 3
big-data-types - Big Data Types v0.3.1

Published by JavierMonton over 3 years ago

Big Data Types v0.3.1

  • BigDecimal precision configurable via implicit Formats
  • Changed Key transformation in Formats, now key transformations can be based on types
big-data-types - v0.3.0 - Conversion between modules

Published by JavierMonton over 3 years ago

Big Data Types v0.3.0

  • Spark: Added reverse conversion (From Spark schemas to generic SqlTypes)
  • Added SqlDouble as a new type in Core and Spark. (BigQuery does not have Doubles)
  • Added a new Type Class for conversions using instances in Core and BigQuery
  • BigQuery: Now it accepts Spark schemas or anything implementing SqlTypeConversion as input
  • New Examples module for cross types tests
Package Rankings
Top 27.1% on Repo1.maven.org
Badges
Extracted from project README
CI Tests BQ IT codecov Scala Steward badge
Related Projects