impy

Highlight

• CLI (command line interface) is supported now. Instead of launching Python, you can just use the terminal. See documentation for the details.
• Performance of parallel calculation on GPU is largely improved.

New Features

• ImgArray.imsave now can save .mrc file if file extension is specified.
• New method smooth_mask in ImgArray for creating soft masks.
• CLI support.

Improvements

• Read .map file using mrcfile.
• Check scale values when any of them is set.
• Clearer error and caching on axis-targeted slicing.
• Clearer docstrings of numpy-like functions such as ip.zeros.
• Make read_meta faster.
• Do not rechunk in proj and binning for better performance.
• Improve performance using cupy.
• Lazy loading of pandas and dask.
• Update for napari==0.4.13

Deprecations

• In LazyImgArray, img.compute() should be used in place of img.data because there is no reason to use different name as dask.
• In LazyImgArray, img.value should be used in place of img.img for compatibility with ImgArray.

Bug Fixes

• Fourier transformation fft and ifft returned wrong results if image has axes other than spatial dimensions.

Changes

• impy.random is no longer an Random object but now its a submodule. No API change.
• New random functions, random_uint8 and random_uint16, for easier random image generation.
• Clearer error handling in imsave.
• Set scale_unit to "px" if exception happened.

New Features

• In Fourier transformation methods such as fft, power_spectra and local_dft, you can set double_precision=True to force float64 and complex128 calculation because single precision sometimes causes non-negligible errors.
• Floating error in local_dft is minimized by considering periodicity of exponential function. The error between local_dft and fft are now completely even.

Bug Fixes

• lowpass_filter, highpass_filter did not work on such as "tyx" and "zcyx" images.
• __turediv__ was incompatible with complex images.
• enhance_contrast was incompatible with float images.
• LazyImgArray behaved in slightly different ways compared to ImgArray.

Others

• Fixed some typings.
• Add tests of major functions.

Changes

• Don't change napari because it became incompatible with current version.
• Bug fix in numpy inherited functions such as ip.empty.
• Phase cross correlation function with Fourier space input.

Changes

• License is changed to BSD 3-Clause.
• Add reset button in plane clip.
• The angle axis "<" are all changed to "a".
• img.shape now returns namedtuple (AxesShape) object.
• Update table widget etc. for the latest napari.
• New low-pass filter methods.
• Force float32 in FFT based filters.
• Use np.memmap and tempfile in LazyImgArray's release and imsave methods and the performance is largely improved.
• ScaleDict support __setitem__ and __setattr__.
• Update typings.
• Removed label_multiostu and slic methods because they are unstable.
• Update setup.py for PyPI.

Bug Fixes

• ifft was not correct.
• Some correlation functions were not compatible with cupy.

In this release I updated for napari==0.4.11 and improved some UI.

New Features

• Table widget is largely improved.
  • When a data is selected in shapes/points layer, the corresponding table row will be highlighted, both in 2D or 3D mode.
  • When a shapes/points layer is linked to a table widget, manually adding/deleting data in the viewer can at the same time adding/deleting table row(s) in the table.
  • When table items or header names are changed, the properties of linked shapes/points layer will also be changed. Now you can use table widget as property editor.
  • Some shortcuts are deleted because they usually conflict with main window.
• You can change text layer's text in place with double click.
• 3D plane clipper is added in "Layer" menu, owing to the clipping plane feature in napari==0.4.11.

Highlights

• Image analysis protocols can be easily defined. Decorate a function with ip.gui.bind_protocol, then key bindings and parameter container are updated automatically.
• Shapes and Points layer can be linked to table widget. You can view/edit layer properties, add/delete data etc. from linked tables.
• Add points or shapes in a similar way to ImageJ ROI using register_point and register_shape. The layers are linked to the result tables.

New Features

• Results widget is implemented (similar to Jupyter's variable explorer). ip.gui.results are updated to this widgets but API does not change.
• Editability of table widget can be changed. Default is not editable now.
• Layer projection (Ctrl+P) supports labels projection.
• Many functions that could only be called by key binding are callable from menu bar.
• Axes.scale uses ScaleDict now. You can get scales by such as img.scale.x instead of img.scale["x"]. ScaleDict is also compatible with scale parameter in Affine transformation, so that it can directly passed to them like viewer.add_image(img, scale=img.scale).
• "Add time stamp" menu in menu bar.

Improvements

• Zoom factor changing problem upon key binding "[" and "]" are solved.
• Highlight warnings and errors in logger.
• The main figure canvas and parameter container are floating by default.
• File information in the tooltips of file explorer.
• "Alt+Up", "Alt+Down" is overloaded to assert z-axis will be changed.

In this release new widgets are added and existing widgets are improved a lot.

• Table widget now displays formatted float value, which looks better than the default setting. Also, many functions are implemented.
• Many improvements in figure canvas. It is now highly interactive and subplots are nicely cooperating.
• Logger widget. Opened only when ip.gui.bind is called and when logger is accessed via ip.gui.log.
• Explorer widget. Open in "File" menu or "Ctrl+Shift+E".
• Text file viewer widget. Added when a txt file is opened in the explorer widget.
• Layer duplication ("Ctrl+Shift+D") and layer projection ("Ctrl+P") now work for any basic layers. However, they dropped support for the case in which multiple layers are selected.
• ip.gui.bind now can make custom functions with parameter inputs. It automatically adds a parameter controller widget to the active viewer.
• ip.gui.bind now has access to the table widget and the log widget in the viewer.
• ip.gui.goto method makes it easier to change the current slice.
• LabeledArray now inherit copy (not pointer) of metadata from previous array.
• Call pandas.read_csv and directly convert it to table.
• ip.imread_collection can read images from list of paths.
• Some exception of physical scale in Micromanager 's ome.tiff is now considered in ip.imread.

The biggest update in v1.20 is ip.gui.bind method. You can make interactive plugins very easily now.

New Features

• Template Matcher
• Rectangle editor can edit rotated rectangles.
• Measure Region Properties.
• Duplicate single slice.
• Table widget can plot inside the viewer, just like Excel.
• Bind custom function with ip.gui.bind.
• ip.gui.get method enables getting certain type of layer's data very easily.
• ip.gui.current_slice property enables getting slice from viewer very easily.

Changes and Bug Fixes

• Focusing markers with [ and ] had a bug. → fixed.
• Scales were not properly saved by imsave. → fixed.
• dask.cache is moved into try/except in case Cachey is not installed.
• apply_dask had a serious bug such as when multidimensional hessian_eigval is called. → fixed.
• LabeledArray.extract no longer supports crop=False because it is not useful and almost independent of crop=True.
• Implementation of ImgArray.rotate and ImgArray.scale was wrong. → fixed.

In this release I implemented a lot of new features concerned to napari viewer. Lots of new widgets and key bindings. You can find them in the "Functions" menu in the menu bar.

New Features in Viewer

• Text editor widget for Shapes and Points layers. You can edit texts of selected shapes/points easily.
• Parameter sweeping widget for most of the filtering functions, thresholding and labeling.
• Show image's info by Alt+ right click.
• You can directly label an image in napari viewer (change img.labels in place).
• Text layer is available.
• Rectangle editor is available. You can set rectangle's edge lengths with pixels.
• Call regionprops in viewer and add the result directly to the Labels layer's properties.
• add_surface function to convert 3D images into Surface layers.
• You can call __getitem__ in function handler using input like ["y=10:40"].
• You can move shapes with arrow keys → ↑ ← ↓.
• When a Shapes or Points layer is selected, you can focus on the previous/next object by pressing [ or ].
• Cropping bug is fixed when rectangle is larger than images.

Others

• You can add images from the disk directly by ip.gui.add(path).
• Name ip is added in the global namespace by default inside napari's console if it is started from start.py.
• LazyImgArray.drift_correction's graph construction is much faster now by moving variable shift into the local warp function.
• threshold's argument is changed: dims → along, for compatibility with other functions.
• local_dft, local_power_spectra for fast up-sampling Fourier transformation.
• opening function in Label.

New Functions in LazyImgArray

• track_drift, drift_correction → Both works totally lazily with specialized implementation.
• Morphological processings: erosion, dilation, opening and closing.
• FFTs: fft, ifft and power_spectra.
• Deconvolution algorithms: wiener and lucy. lucy_tv is not implemented because it evaluate total variance for every iteration.

New Features

• Document added in GitHub Page.
• Show dimensionality of ongoing function to avoid 2D/3D mistakes.
• Dask + GPU is set to single-threaded by default to avoid OutOfMemoryError in cupy.
• Dask caching system is now utilized for fast image processing.
• Automatic docstring generation.

HighLights

• GPU/cupy are now supported! You can conduct Affine transformation, deconvolution and many filter functions with GPU. Arrays are converted inside the functions so that you don't have to care about the difference between numpy and cupy.
• Correlation functions with masks are available. You can calculate correlation in any sub-regions between two images in either real space or Fourier space.
• Method for_params can run same function with different parameter inputs. No for-loop anymore!

Other Updates

• n-dimensional Affine transformation. Affine transformation matrix is made either with skimage or napari.
• DataList inherits UserList now. All the list methods are compatible.
• DataDict is defined in place of ArrayDict and FrameDict.
• DataList is returned in split.
• You can save LazyImgArray directly with imsave method now.
• Global variables are stored in dict now. You can update them with such as ip.Const["MAX_GB"] = 3.0 or with context manager with ip.SetConst("MAX_GB", 3.0): ....
• Delayed import as many as possible.
• FFT functions now take keyword arguments shape= to specify output shape.
• Docstring auto generation.
• Renamed ncc → ncc_filter.

Bug Fixes

• mrcfile read images as read-only array, which made it incompatible with some functions. This problem is fixed by explicitly set mode="r+".
• fill_hole did not work since dask is integrated as multiprocessing. Now fixed.
• rotated_crop in LazyImgArray, or corresponding rotated cropping in viewers returned wrong shape of arrays. This is fixed now.

New Features and Changes

1. LazyImgArray

• It is now implemented with many new functions similar to those in ImgArray -- rotated_crop, crop_center, binning, proj, gaussian_filter, median_filter, convolve, edge_filter and reslice. dask arrays are rechunked automatically to make function running efficiently. Some function are dependent on dask_image.
• Changing data type with as_uint8, as_uint16. Be careful these functions do not work exactly in a same way as those in ImgArray. Because dask must be evaluated in if, they cannot check whether a float image is in range of [0, 1].

2. ImgArray

• proj has a new option mask=.... You can mask input image and calculate projection without the masked region. This is useful when you want to exclude outliers in the image from its projection.
• __getitem__ is safer for images without axes.
• All the multiprocess methods are translated to dask methods, with function apply_dask. Accordingly, parallel, parallel_eig, n_cpu are deleted.

3. napari viewer

• When you are drawing a rectangle, line or path, its properties are displayed on the text overlay in the upper left corner.
• You can select an image layer just by clicking it in the viewer with Alt.
• Image cropping now takes layer scale differences into account.
• You can crop images in other planes. You can crop in XY and crop again in XZ.

4. Others

• ip.imread and ip.lazy_imread now supports squeezing arrays as an option. This is useful when there is only one image file in a folder.
• Almost all the constant values are changed to global variables. ip.set_verbose no longer works. Change it by ip.SHOW_PROGRESS = False or ip.MAX_GB = 3.0 directly.
• Axes handling parts of arrays are moved to AxesMixin. MetaArray and LazyImgArray inherit it.
• ip.imread and evaluation functions in LazyImgArray always raise MemoryError if the expected size exceeds ip.MAX_GB.
• You can run reslice in viewer to make a kymograph.
• HTML rich output in Jupyter Notebook.
• Many decorators are unified into @record because they are redundant.
• Argument mapping in ip.bind is deprecated because it is useless.

Important Changes

• Old imread_collection and imread_stack cannot be called now. Use imread instead. Also, how to recursively read images is changed to pims-like format, i.e., write the wildcard string directly as a path e.g. >>> ip.imread(r"C:\...\**\*.tif").
• imread_collection now works as an I/O function for reading images with different shapes and storing them in DataList.

New Features

• I/O functions now support mrc and rec files, if mrcfile is installed. Metadata is stored in tif-friendly format.
• Image cropping in napari viewer (Ctrl + Shift + X) can use rotated rectangles. It is achieved by ndi.map_coordinates rather than rotating the whole image.
• You can select image layer(s) just by clicking them with Alt or Ctrl+Alt, similar to PowerPoint.
• imread(..., key="..."), lazy_imread for reading large datasets.
• DataList for batch processing images with different shapes.
• LazyImgArray for handling lazy reading and previewing using dask.
• radial_profile can calculate radial profile P(r) of images.
• power_spectra can calculate FFT power spectra image directly.

Improvements

• Some functions, like gaussian_filter, are faster with single thread so that they are never conducted with multi-processing.
• Large scales caused edge width of shape layers being too thin. Edge widths will be calculated every time shape layers are added.
• napari viewer now support handling LazyImgArray as dask input, and DataList by adding all the contents.

Bug Fixes

• regionprops could have not be calculated since numpy function is overloaded for ImgArray. This problem is fixed.
• binning was not true binning because affine transformation is not same as binning. Now binning returns correct results, and you can also use other reduce functions such as np.median for binning.
• Image cropping aborted such as dirpath and scale of images. This problem is now fixed for both MetaArrays and dask.arrays.core.Array.

New Functions

• tile function for image tiling. Overlap is not considered for now.
• lowpass_filter and highpass_filter. Similar functions will be imported from skimage.filters after v0.19 is available.
• binning function for image binning like 2x2 or 4x4. Divisibility is required unlike rescale.
• for_each_channel enables different parameter inputs for different channel. Until v1.13, you needed to split channels, apply function for each channel, and stack them again. Now you only have to call for_each_channel.
• kalman_filter for smoothing of time-series image stacks. This function is identical to "Kalman Stack Filter" in ImageJ.
• PathFrame that contains paths, similar to TrackFrame.
• pathprops can measure path properties like regionprops.

Improvements

• imread can call imread_collection or imread_stack to open folder instead of raising errors. With default settings you don't have to call imread_collection or imread_stack anymore.
• Call TiffFile just once in imread.
• In some functions, such as lucy, rfftn and irfftn are used instead of fftn and ifftn. This change makes them ~2 times faster.
• Some numpy functions, such as np.allclose, did not support MetaArray input. Now fixed.
• reslice now supports reslicing a path.
• napari's viewer can automatically add a table widget when a PropArray is given.
• napari's viewer can automatically add a Shapes layer when a PathFrame is given.

This release contains many new functions and function improvements.

New Functions

• rof_filter for total variance denoising.
• pearson_coloc and manders_coloc to calculate colocalization coefficient.
• crop_kernel to make small kernel array.
• ncc for template matching using NCC (normalized cross correlation).
• You can make Gaussian kernel images with impy.gaussina_kernel for deconvolution and template matching.
• track_template for template tracking based on NCC.

Improvements

• Viewer widget function_handler now accept list and numpy as input.
• Link (Ctrl + G) and unlink (Ctrl + Shift + G) layer shortcuts: .
• Defocusing padding in yx-direction with defocus.
• find_sm has a new option method="ncc". This method is based on template matching of Gaussian particle.
• find_sm has a new keyword argument cutoff. Filtered images are clipped by this value.
• lucy and lucy_tv takes new argument eps=... to avoid near-zero division.
• You can call imsave inside napari.
• Many typings are added for future integration of function_handler and magicgui.

Bug Fixes

• Cropping (Ctrl + Shift + X) works fine now.
• random_walker works fine now.

Changes

• Ctrl + Shift + X removes the original image. Duplicate it if you want.
• ImgArray object made by imread_stack now have dirpath attribute at the deepest directory of path argument, which does not contain either "$" or "*".
• defocus also pad in yx-direction by default. If it is not needed, you need to set width = 0.

This Release is mainly focused on viewer.

New Features in Viewer

• Custom key bindings (crop, duplicate, labeling, reset etc.).
• Translation , rescaling within layers using mouse.
• Table, notepad widgets.
• Call impy.imread function inside viewers. Unlike the original one, this can read tiff metadata.

Improvements

• centroid_sm now works for "zcyx"-images.
• Layer-to-impy-object conversion is smarter now. Axes & scale information are recorded in metadata attributes so that AxesFrame remembers the initial states.
• Label determines dtype upon construction.
• rolling_ball can return background with keyword return_bg=True. Also, median filter is available for prefiltering.

Changes

• Argument name in rolling_ball is changed: smooth → prefilter.
• impy.window → impy.gui because it is no longer a single window.

This release contains many improvements for general usage.

New Features and Functions

1. Direct use of numpy functions.

• With __array_function__, we can overload numpy functions in a single-dispatch-like way (see here). Now you can use numpy functions directly in a impy style, like np.mean(img, axis="t"), np.expand_dims(img, axis="pz"), np.stack([img1, img2], axis="z"). Those functions that may change the dimensions of arrays are individually re-implemented.
• Other numpy functions like np.array or np.random.rand cannot know if you are to generate np.ndarray or ImgArray. These functions are implemented in ip.array, ip.random.rand style.

2. Custom function binder.

• ip.bind decorator class can interpret custom defined functions for batch processing. This decorator add dims keyword argument and convert the function into an ImgArray-like format. As long as the custom function returns images or labels with same shape as input, or single property per image, almost any function can be extended for batch processing.

3. napari viewer is improved as lot.

• ip.window can deal with multiple windows. Previous codes still works, while new window is created by ip.window["new name"].add(X)..
• You can crop image with Ctrl+Shift+X.
• Many methods can be called inside napari.Viewer.
• Manually selected layers can be obtained by ip.window.selection. They are in ImgArray, MarkerFrame, ... form.

4. Others

• In accordance with the release of napari==0.4.8, scale unit can be displayed beside the scale bar. impy can automatically read scale unit from the metadata of such as ImgArray so that you don't need to add scale unit all the way.
• lucy_tv, Richardson-Lucy with total variance regularization is now available. This is a little bit slower than lucy but is widely used for deconvolution.

Important Changes

• Functions that deal with labels while keep original image untouched, such as label and watershed, returned LabeledArray before. Hereafter they'll return Label instead.
• ip.stack, ip.zeros_like, ip.empty_like no longer works. Use np.stack, np.zeros_like, np.empty_like instead.

Buf Fixes

• AxesFrame.iter method sometimes raises error because of inconsistency between iteration took place or not. This is fixed now.
• dims_to_spatial_axes now works for AxesFrame as well.
• estimate_sigma method did not work due to wrong implementation. This is fixed now.

This release mainly focused on bug fixes and improvements.

Improvements

1. Reading/writing xy- or z-scales.

Previously impy could not automatically deal with scale (or image resolution) so that users must set it by set_scale method. Now you don't need to do that!

• ip.imread can read scales from tiff metadata and store it into scale property of ImgArray.
• imsave method can write scales of LabeledArray into tiff files. It is compatible with ImageJ so that you can confirm in ImageJ by Ctrl+Shift+P.

2. More flexible reading functions.

impy now supports more flexible file handling.

• In ip.imread_collection, ext argument is deprecated. Instead, wildcard can be defined in more detailed way by filename argument and image shape, axes or scale you want to read can be filtered by template argument.
• The new function ip.imread_stack suits for practical image stacking for real microscopy. You may want to save multiple tiff files in different folders, and make a image stack from such as fr"~\XX{i}\YY{j}.tif" for i in [10,20,30] for j in [0,1,2,3] format, followed by defining i-direction as t-axis and j-direction as p-axis. This task can be achieved by a very simple code: img = ip.imread_stack(fr"~\XX$t\YY$p.tif").

3. Axis-targeted broadcasting

Previously, when img1 has zcyx-axes and img2 has zyx-axes, img1/img2 raised broadcasting error due to numpy's broadcasting rules. However, MetaArray should know how to broadcast because apparently img1 and img2 have common axes "zyx". Now axis-targeted broadcasting is working. Accordingly, previous broadcasting-like parts in __getitem__ method is replaced with common function _broadcast.

4. Others

• set_scale can take "xy" or "yx" arguments to set yx-scale at the same time, like img.set_scale(yx=0.217).
• You can access skimage's sample data with such as ip.sample_image("camera"). Image axes of RGB images will always be transposed so that they are cyx-order.
• crop_center, remove_edges accepts sequence as parameters.
• defocus now can take any array as a defocusing kernel. Default Gaussian kernel is also supported.
• lucy is ~30% faster now, achieved by recycling array objects during iterations.
• ip.squeeze can squeeze ImgArray in a flexible way.
• imsave can automatically change axes symbols if possible when any symbols are not compatible with ImageJ (e.g. pyx -> tyx).
• proj can calculate projection along multiple axes, like img.proj(axis="ptz").

Bug Fixes

• track_drift and drift_correction had wrong default settings so they always raised errors without explicit along argument. → Fixed, and the error messages are also improved.
• Axes.replace(old, new) did not change scale attribute at the same time. This caused inconsistency between axes and scale and raised KeyError in many situations (especially imsave). → Fixed, and to prevent unexpected overwriting, scale will be copied every time upon this function call.

New Features

• Manually drawn shapes can be converted to Label or AxesFrame easily using shapes_to_labels or points_to_frames methods.
• imread and imsave can input/output xy-scale of tiff images. You don't have to call set_scale method every time..

New Functions

• edge_filter for edge detection including sobel_filter in the past. Accordingly sobel_filter is deleted.
• edge_grad for calculating gradient of an image. This function returns PhaseArray with 2*pi periodicity.
• proj_labels for label projection. This is useful when you have tyx-labels but want them to be yx-labels.
• unmix for unmixing leakage between channels in fluorescent microscopy.

Improvements and Bug Fix

• Unexpected overflow in label_if is fixed.
• affine and translate will support 3D transformation after scikit-image v0.19 becomes available.
• set_scale in LabeledArray (therefore ImgArray and PhaseArray) can set the same scales to their labels.
• peak_local_max, corner_peaks and refine_sm did not work for zcyx-images due to wrong implementation. This is now fixed.