eo-learn

Core Changes

• EOPatch changes:

  • IO for vectors and meta-info switched from pickle to Geopackage, GeoJSON, and JSON files. Objects saved with pickle can be loaded but the format is deprecated.
  • Now supports the in keyword for checking whether an EOPatch contains a given feature.
  • Major update to EOPatch plotting functionality, which now features a simpler matplotlib back-end. See example notebook for more details.
  • Removed some outdated EOPatch methods such as get_feature, rename_feature, etc.
  • Representation (EOPatch.__repr__ method) skips empty features.

• EOTask changes:

  • EOTask method _parse_features replaced with get_feature_parser and additional helper methods (parse_feature, parse_renamed_feature, parse_features, parse_renamed_features).
  • Removed EOTask.__mul__ as task concatenation as it was unsound.

• EONode is a newly introduced object for specifying computational graphs. It replaces raw EOTask objects when building an EOWorkflow.

• EOWorkflow changes:

  • LinearWorkflow is replaced with linearly_connect_tasks function that prepares nodes for a linear workflow.
  • No longer accepts tuples in execution arguments. In cases where this is required, passing arguments to a task can be done with the new InputTask.
  • EONodes form a tree-like structure of dependencies, hence the end-nodes of a workflow contain all information. An EOWorkflow object can be constructed from end-nodes via from_endnodes method.

• EOExecutor changes:

  • Added RayExecutor as an extension of EOExecutor for working with the ray library.
  • Execution arguments are now given w.r.t. EONode objects instead of EOTasks.
  • Now always returns results, which by default only contain statistics. Other data (for instance the final EOPatch) can be added to results with the new OutputTask.
  • Additionally, supports a filesystem argument for saving logs and reports.
  • Reports now have the option to only link to logs, greatly reducing size in case of large numbers of EOPatches. Logs files are now also more informative.

• FeatureParser now supports fewer input formats but handles those better. It now returns lists instead of generators. See documentation for more information.

• WorkflowResults are re-done. They now contain execution stats of nodes (start and end times) and the outputs of OutputTasks in the workflow.

• FeatureType method is_time_dependant renamed to is_temporal.

Tasks

• Added LinearFunctionTask which applies a linear function to features.
• MorphologicalFilterTask moved from ml_tools to features module.
• Sampling tasks moved geometry to ml_tools module. Sampling tasks have also been greatly upgraded, with the main being:
  • FractionSamplingTask for sampling random points in a class-balanced way
  • BlockSamplingTask for randomly sampling larger blocks of data (can also be 1 pixel blocks)
  • GridSamplingTask for deterministically sampling according to a grid.
• Removed feature_extractor module.
• Removed unused submodules of ml_tools (classifier, postprocessing, ...)
• To reduce core dependencies some functionalities have been moved to extra modules.
• Removed deprecated and outdated methods and tasks.

Other

• Moved many examples to new repository. The rest were updated.
• Switched to github actions for CI.
• Code was reformatted with black and is now checked to be compliant with the standard.
• Abstract base classes are now correctly enforced.
• Added utility functions for working with S3 and AWS.
• Various minor changes.

Changelog:

• Copying EOPatches no longer forces loading of features if the EOPatch was loaded with lazy_loading=True
• SentinelHubInputTask now requests bands with correct units and should now work with more data collections. The parameter bands_dtype is now by default set to None, which uses the default units of each band. Note: due to changes the task no longer normalizes the output when bands_dtype=np.uint16 is used.
• Minor fixes and improvements

Changelog:

• EOWorkflow now automatically makes a shallow copy of each EOPatch before passing it to any EOTask in the workflow.
• Streamlined naming conventions of EOTasks - every name now ends with Task. Old names have been deprecated.
• Improved functionality of merging EOPatches, particularly of merging time-dependent features.
• Removed support for Python 3.6 and added official support for Python 3.9.
• Implemented EOPatch.copy and EOPatch.__delitem__ methods.
• Added eolearn.io.MeteoblueRasterTask and eolearn.io.MeteoblueVectorTask for obtaining weather data. Joint effort with Meteoblue.
• VectorToRasterTask now supports rasterization of time-dependant vector features. Contributed by @asylve.
• Fixes in SentinelHubInputTask. Both SentinelHubInputTask and SentinelHubEvalscriptTask now return EOPatches with timestamps that don't have timezone information anymore.
• Changed rasterio dependency to rasterio>=1.2.7
• All but eolearn.core tests ported to pytest framework.
• Switched from Travis CI to GitHub actions.
• Minor fixes and improvements.

Changelog:

• Minor fixes and improvements:
  • SaveTask and LoadTask don't automatically store a filesystem object anymore,
  • fix in ImportFromTiff about file extensions, contributed by @rpitonak,
  • fix in SentinelHubInputTask about data collection bands handling,
  • fix in GeoDBVectorImportTask,
  • NormalizedDifferenceIndexTask doesn't show division warnings anymore,
  • improvement in PointSamplingTask
  • improvements in LULC documentation notebook.

Changelog:

• Added new tasks VectorImportTask, GeopediaVectorImportTask, and GeoDBVectorImportTask to eo-learn-io.
• Code improvements in LULC classification documentation notebook.
• Minor improvements and fixes.

Changelog:

• Changes in eo-learn-io tasks that interact with Sentinel Hub services:
  • Added SentinelHubEvalscriptTask that downloads data given a user-defined evalscript.
  • Removed all tasks that interact with Sentinel Hub OGC services. They are fully replaced by SentinelHubInputTask and SentinelHubEvalscriptTask which use Sentinel Hub Process API.
  • Renamed AddSen2CorClassificationFeature to SentinelHubSen2corTask. Now it uses Process API instead of OGC.
• Changes in Sentinel-2 cloud-masking tasks:
  • Renamed AddMultiCloudMaskTask to CloudMaskTask.
  • Removed AddCloudMaskTask as it is superseded by CloudMaskTask.
  • Fixed problems with incompatibility with the latest scikit-learn version.
• Updated all notebooks in eo-learn repository.
• Minor fixes and improvements:
  • better handling of KeyboardInterrupt in EOExecutor,
  • fixed plotting of raster features with binary dtype,
  • documentation fixes.

Changelog:

• Added example use-case about creating a timelapse with batch processing.
• Added example use-case about crop type classification. Contributed by @Gnilliw
• Updated code to be compatible with the latest releases sentinelhub-py 3.2.0 and s2cloudless 1.5.0.
• Fixed an issue in eolearn.coregistration.RegistrationTask.
• Fixed an issue in eolearn.io.ExportToTiff with paths on Windows.
• Various minor improvements.

Changelog:

• Switched from "data source" to "data collection" terminology according to changes in sentinelhub-py and Sentinel Hub services.
• Improvements in SentinelHubInputTask to better support any type of data collection. Using new DataCollection class from sentinelhub-py.
• Extended ExportToTiff and ImportFromTiff tasks to support writing and reading from AWS S3 buckets. Implemented in cooperation with @wouellette.
• Implemented EOPatch.merge method and MergeEOPatchesTask task for merging the content of any number of EOPatches. Implemented in cooperation with @wouellette.
• Deprecated EOPatch.concatenate in favour of EOPatch.merge.
• Added eolearn.features.DoublyLogisticApproximationTask, contributed by @bsircelj.
• Optional parameter config for SaveTask and LoadTask to enable defining custom AWS credentials.
• Fixed a bug in eolearn.features.ValueFilloutTask.
• Started releasing eo-learn (sub)packages also as wheels.
• Minor improvements and fixes.

Changelog

• Support for geopandas version 0.8
• Added a notebook with an end-to-end example on how to transform a ML-model into an evalscript and run it with Sentinel Hub service
• Added eolearn.features.ClusteringTask, contributed by @bsircelj
• An option to define a custom log filter for EOExecutor
• Data mask obtained by SentinelHubInputTask has now boolean type instead of uint8
• Updates of some example notebooks
• A few minor fixes

Changelog:

• Added eo-learn dockerfiles and deployed official eo-learn docker images to Docker Hub
• Added compress parameter to ExportToTiff, contributed by @atedstone
• Minor fixes

Changelog:

• Updated example notebooks - replaced Sentinel Hub OGC service tasks with Sentinel Hub Processing API service tasks
• Deprecated tasks that download data from OGC service
• Minor fixes in SentinelHubInputTask and AddCloudMaskTask

Changelog:

• Updates of SentinelHubInputTask:
  • Support for new s2cloudless precomputed cloud mask (more info)
  • Support for config parameter
• Updated SI_LULC_pipeline notebook.

Changelog:

• Added support for geopandas version 0.7.0.
• Fixed a bug in eolearn.core.eodata_io.save_eopatch function.
• Improvement in eolearn.mask.MaskFeature task - it now works also works with time-independent feature types.
• A minor improvement in eolearn.io.SentinelHubInputTask task.

Change log:

• Support additional data in the Processing API input task (such as sunAzimuthAngles, sunZenithAngles, viewAzimuthMean, viewZenithMean)
• Compatibility with the sentinelhub-py version 3.0.0
• Removed support for python 3.5
• Multiprocessing log filtering

Fixed

• eolearn.io.SentinelHubInputTask: evalscript version was not passed to the sentinel-hub service.
• eolearn.core.EOWorkflow: fixed generating task dependencies.

Added

• Processing API docs generation.
• Introduced CHANGELOG.md.

Changelog:

• Added tasks eolearn.io.SentinelHubInputTask and eolearn.io.SentinelHubDemTask that implement satellite data download from Sentinel Hub Processing API.

• Extended EOPatch saving and loading with PyFilesystem2 package:

  • saving and loading can now be done with any custom filesystem supported by PyFilesystem2,
  • special support for AWS s3 bucket filesystem,
  • new tasks eolearn.core.SaveTask and eolearn.core.LoadTask.

• Added eolearn.mask.AddMultiCloudMaskTask for multi-temporal cloud detection.

• Added official support for Python 3.8.

• Added multiple use case examples, contributed by @drewbo:

  • poverty detection,
  • social media example,
  • super-resolution with FastAI.

• eolearn.io.ExportToTiff task can now export into a custom CRS, contributed by @wouellette.

• Improved Contributions guidelines, contributed by @PeterFogh.

Change log:

• Added new EOTasks:
  • in eolearn.core:
    • DuplicateFeature,
    • InitializeFeature,
    • MoveFeature,
    • MergeFeatureTask,
    • MapFeatureTask,
    • ZipFeatureTask,
    • ExtractBandsTask,
    • CreateEOPatchTask
  • in eolearn.features:
    • ValueFilloutTask,
    • EuclideanNormTask,
    • NormalizedDifferenceIndexTask
  • in eolearn.mask:
    • SnowMask,
    • TheiaSnowMask,
    • ClassFrequencyTask
  • in eolearn.ml_tools:
    • TrainTestSplitTask
• Changed algorithm for spatial resizing in AddCloudMaskTask
• VectorToRaster now has optional parameters overlap_value and buffer
• Improvements in ErosionTask
• S1IWWMSInput and S1IWWCSInput tasks now have an optional parameter for orbit direction
• Removed EOWorkflow.make_linear_workflow method
• Improvements in LinearWorkflow and WorkflowResults
• Improvements of EOExecutor and the report it generates
• Added credits and licensing info to all modules
• Added a prototype project Land Cover Classification with fastai, contributed by @Geoyi

Change log:

• Problematic dependency packages for EOPatch visualization are not being installed by default anymore. They can be installed additionally with pip install eo-learn-visualization[FULL] option.

Change log:

• Added EOPatch visualization functionalities
• Moved all visualization functionalities from eolearn.core to a new subpackage eolearn.visualization
• Improvements of EOExecutor and how parallelization is handled
• Improvements of reports produced by EOExecutor
• LinearInterpolation task is now using numba in the background for better performance
• Initialization parameters of VectorToRaster have changed. Both VectorToRaster and RasterToVector tasks have been improved.
• Added super-pixel segmentation tasks to eolearn.geometry
• Added ImportFromTiff task to eolearn.io, contributed by @drewbo
• Improvements of example notebooks
• Various small bug fixes

Change log:

• Added tqdm progress bar to EOExecutor.
• Switched from multiprocessing to concurrent.futures in EOExecutor.
• ExportToTiff can now export multi-channel tiff images and works with time-dependet features. Contributed by @wouellette
• Updated package to support sentinelhub-py version 2.5.0.
• Fixed value overflow bug in InterpolationTask (could happend only on Windows machines).
• Fixed problem with time interval parsing in eolearn.io.
• Improvements in EOWorkflow, added get_tasks method.
• Added instructions for contributions and code of conduct.