Layers#

Custom layers

N-dimensional layer#

Convolution ND#

class fusionlab.layers.ConvND(spatial_dims: int, in_channels: int, out_channels: int, kernel_size: Sequence[int] | int, stride: Sequence[int] | int = 1, padding: int | Sequence[int] | None = None, dilation: Sequence[int] | int = 1, groups: int = 1, bias: bool = True, padding_mode: str = 'zeros')[source]#

Factory class for creating convolutional layers. This class is used to create convolutional layers with the same configuration.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
in_channels (int) – number of channels in the input data.
out_channels (int) – number of channels produced by the convolution.
kernel_size (int or tuple) – size of the convolving kernel.
stride (int or tuple, optional) – stride of the convolution. Default: 1
padding (int or tuple, optional) – zero-padding added to both sides of the input, None for same padding. Default: None
dilation (int or tuple, optional) – spacing between kernel elements. Default: 1
groups (int, optional) – number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – whether to add a bias to the convolution. Default: True
padding_mode (str, optional) – type of padding. Default: ‘zeros’

Transposed Convolution ND#

class fusionlab.layers.ConvT(spatial_dims, in_channels: int, out_channels: int, kernel_size: Sequence[int] | int, stride: Sequence[int] | int = 1, padding: int | Sequence[int] | None = None, output_padding: Sequence[int] | str = 0, groups: int = 1, bias: bool = True, dilation: Sequence[int] | int = 1, padding_mode: str = 'zeros')[source]#

Factory class for creating transposed convolutional layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
in_channels (int) – number of channels in the input data.
out_channels (int) – number of channels produced by the convolution.
kernel_size (int or tuple) – size of the convolving kernel.
stride (int or tuple, optional) – stride of the convolution. Default: 1
padding (int or tuple, optional) – zero-padding added to both sides of the input, None for same padding. Default: None
output_padding (int or tuple, optional) – additional size added to one side of each dimension in the output shape. Default: 0
groups (int, optional) – number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – whether to add a bias to the convolution. Default: True
dilation (int or tuple, optional) – spacing between kernel elements. Default: 1
padding_mode (str, optional) – type of padding. Default: zeros

Upsample ND#

Factory class for creating Upsample layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
size (int or Tuple[int] or Tuple[int, int] or Tuple[int, int, int], optional) – output spatial sizes
scale_factor (float or Tuple[float] or Tuple[float, float] or Tuple[float, float, float], optional) – multiplier for spatial size. Has to match input size if it is a tuple.
mode (str, optional) – the upsampling algorithm: one of 'nearest', 'linear', 'bilinear', 'bicubic' and 'trilinear'. Default: 'nearest'
align_corners (bool, optional) – if True, the corner pixels of the input and output tensors are aligned, and thus preserving the values at those pixels. This only has effect when mode is 'linear', 'bilinear', 'bicubic', or 'trilinear'. Default: False
recompute_scale_factor (bool, optional) – recompute the scale_factor for use in the interpolation calculation. If recompute_scale_factor is True, then scale_factor must be passed in and scale_factor is used to compute the output size. The computed output size will be used to infer new scales for the interpolation. Note that when scale_factor is floating-point, it may differ from the recomputed scale_factor due to rounding and precision issues. If recompute_scale_factor is False, then size or scale_factor will be used directly for interpolation.

BatchNorm ND#

class fusionlab.layers.BatchNorm(spatial_dims: int, num_features: int, eps: float = 1e-05, momentum: float = 0.1, affine: bool = True, track_running_stats: bool = True)[source]#

Factory class for creating batch normalization layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
num_features – number of features or channels \(C\) of the input
eps – a value added to the denominator for numerical stability. Default: 1e-5
momentum – the value used for the running_mean and running_var computation. Can be set to None for cumulative moving average (i.e. simple average). Default: 0.1
affine – a boolean value that when set to True, this module has learnable affine parameters. Default: True
track_running_stats – a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics, and initializes statistics buffers running_mean and running_var as None. When these buffers are None, this module always uses batch statistics. in both training and eval modes. Default: True

InstanceNorm ND#

class fusionlab.layers.InstanceNorm(spatial_dims: int, num_features: int, eps: float = 1e-05, momentum: float = 0.1, affine: bool = False, track_running_stats: bool = False)[source]#

Factory class for creating instance normalization layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
num_features – number of features or channels \(C\) of the input
eps – a value added to the denominator for numerical stability. Default: 1e-5
momentum – the value used for the running_mean and running_var computation. Can be set to None for cumulative moving average (i.e. simple average). Default: 0.1
affine – a boolean value that when set to True, this module has learnable affine parameters, initialized the same way as done for batch normalization. Default: False.
track_running_stats – a boolean value that when set to True, this module tracks the running mean and variance, and when set to False, this module does not track such statistics and always uses batch statistics in both training and eval modes. Default: False

Maxpool ND#

class fusionlab.layers.MaxPool(spatial_dims: int, kernel_size: int | Sequence[int], stride: int | Sequence[int] | None = None, padding: Sequence[int] | int = 0, dilation: Sequence[int] | int = 1, return_indices: bool = False, ceil_mode: bool = False)[source]#

Factory class for creating maximum pooling layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
kernel_size – The size of the sliding window, must be > 0.
stride – The stride of the sliding window, must be > 0. Default value is kernel_size.
padding – Implicit negative infinity padding to be added on both sides, must be >= 0 and <= kernel_size / 2.
dilation – The stride between elements within a sliding window, must be > 0.
return_indices – If True, will return the argmax along with the max values. Useful for torch.nn.MaxUnpool1d later
ceil_mode – If True, will use ceil instead of floor to compute the output shape. This ensures that every element in the input tensor is covered by a sliding window.

AvgPool ND#

class fusionlab.layers.AvgPool(spatial_dims: int, kernel_size: int | Sequence[int], stride: int | Sequence[int] | None = None, padding: int | Sequence[int] = 0, ceil_mode: bool = False, count_include_pad: bool = True)[source]#

Factory class for creating average pooling layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
kernel_size – the size of the window
stride – the stride of the window. Default value is kernel_size
padding – implicit zero padding to be added on both sides
ceil_mode – when True, will use ceil instead of floor to compute the output shape
count_include_pad – when True, will include the zero-padding in the averaging calculation

(the original version for 2d and 3d has a ‘divisor_override’ parameter which is neglected here)

Global Max Pool (Adaptive Max Pool) ND#

class fusionlab.layers.AdaptiveMaxPool(spatial_dims: int, output_size: int | Sequence[int], return_indices: bool = False)[source]#

Factory class for creating adaptive max pooling layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
output_size – the target output size \(L_{out}\).
return_indices – if True, will return the indices along with the outputs. Useful to pass to nn.MaxUnpool1d. Default: False

Global Avg Pool(Adaptive Avg Pool) ND#

class fusionlab.layers.AdaptiveAvgPool(spatial_dims: int, output_size: int | Sequence[int])[source]#

Factory class for creating adaptive average pooling layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
output_size – the target output size of the image of the form H x W. Can be a tuple (H, W) or a single H for a square image H x H. H and W can be either a int, or None which means the size will be the same as that of the input.

Replication Padding ND#

class fusionlab.layers.ReplicationPad(spatial_dims: int, padding: int | Sequence[int])[source]#

Factory class for creating replication padding layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
padding (int, tuple) – the size of the padding. If is int, uses the same padding in all boundaries. If a 4-tuple, uses (:math:` ext{padding_left}`, :math:` ext{padding_right}`, :math:` ext{padding_top}`, :math:` ext{padding_bottom}`)

Constant Padding ND#

class fusionlab.layers.ConstantPad(spatial_dims: int, padding: int | Sequence[int], value: float)[source]#

Factory class for creating adaptive average pooling layers.

Parameters:

spatial_dims (int) – number of spatial dimensions of the input data.
padding (int, tuple) – the size of the padding. If is int, uses the same padding in all boundaries. If a 4-tuple, uses (:math:` ext{padding_left}`, :math:` ext{padding_right}`, :math:` ext{padding_top}`, :math:` ext{padding_bottom}`)
value (float) – the value of padding

Conv|Norm|Act#

class fusionlab.layers.ConvNormAct(spatial_dims, in_channels, out_channels, kernel_size, stride=1, padding=None, dilation=1, groups=1, bias=None, norm_layer=<class 'fusionlab.layers.factories.BatchNorm'>, act_layer=<class 'torch.nn.modules.activation.ReLU'>, padding_mode='zeros', inplace=<class 'bool'>)[source]#

ref: https://pytorch.org/vision/main/generated/torchvision.ops.Conv2dNormActivation.html pytorch/vision

Convolution + Normalization + Activation

Parameters:

spatial_dims (int) – number of spatial dimensions of the input image.
in_channels (int) – number of channels of the input image.
out_channels (int) – number of channels of the output image.
kernel_size (Union[Sequence[int], int]) – size of the convolving kernel.
stride (Union[Sequence[int], int], optional) – stride of the convolution. Default: 1
padding (Union[Sequence[int], str], optional) – Padding added to all four sides of the input. Default: None, in which case it will be calculated as padding = (kernel_size - 1) // 2 * dilation
dilation (Union[Sequence[int], int], optional) – spacing between kernel elements. Default: 1
groups (int, optional) – number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – Whether to use bias in the convolution layer. By default, biases are included if norm_layer is None.
norm_layer (Optional[Callable[..., nn.Module]], optional) – normalization layer. Default: BatchNorm
act_layer (Optional[Callable[..., nn.Module]], optional) – activation layer. Default: nn.ReLU
padding_mode (str, optional) – mode of padding. Default: ‘zeros’
inplace (Optional[bool], optional) – Parameter for the activation layer, which can optionally do the operation in-place. Default True

forward(x)[source]#

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Rearrange#

class fusionlab.layers.Rearrange(pattern, **kwargs)[source]#

nn.Module wrapper for eion’s rearrange function

forward(x)[source]#

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Patch Embedding#

class fusionlab.layers.PatchEmbedding(in_channels, img_size, patch_size, hidden_size, pos_embed_type='conv', dropout_rate=0.0, spatial_dims=2)[source]#

A patch embedding block, based on: “Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale <https://arxiv.org/abs/2010.11929>”

__init__(in_channels, img_size, patch_size, hidden_size, pos_embed_type='conv', dropout_rate=0.0, spatial_dims=2)[source]#

Parameters:

in_channels (int) – dimension of input channels.
img_size (Union[int, Sequence[int]]) – dimension of input image.
patch_size (Union[int, Sequence[int]]) – dimension of patch size.
hidden_size (int) – dimension of hidden layer.
num_heads – number of attention heads.
pos_embed_type (str) – position embedding layer type.
dropout_rate (float) – faction of the input units to drop.
spatial_dims (int) – number of spatial dimensions.

forward(x)[source]#

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Squeeze-Excitation#

class fusionlab.layers.SEModule(input_channels, squeeze_channels, act_layer=<class 'torch.nn.modules.activation.ReLU'>, scale_layer=<class 'torch.nn.modules.activation.Sigmoid'>, spatial_dims=2)[source]#

source: pytorch/vision This block implements the Squeeze-and-Excitation block from https://arxiv.org/abs/1709.01507 (see Fig. 1). Parameters activation, and scale_activation correspond to delta and sigma in eq. 3.

Parameters:

input_channels (int) – Number of channels in the input image
squeeze_channels (int) – Number of squeeze channels
activation (Callable[..., torch.nn.Module], optional) – delta activation. Default: torch.nn.ReLU
scale_activation (Callable[..., torch.nn.Module]) – sigma activation. Default: torch.nn.Sigmoid

forward(input)[source]#

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Return type:: Tensor

class fusionlab.layers.TFSEModule(*args, **kwargs)[source]#

call(inputs)[source]#

This is where the layer’s logic lives.

The call() method may not create state (except in its first invocation, wrapping the creation of variables or other resources in tf.init_scope()). It is recommended to create state, including tf.Variable instances and nested Layer instances,

in __init__(), or in the build() method that is

called automatically before call() executes for the first time.

Parameters:

inputs –
Input tensor, or dict/list/tuple of input tensors. The first positional inputs argument is subject to special rules: - inputs must be explicitly passed. A layer cannot have zero

arguments, and inputs cannot be provided via the default value of a keyword argument.
- NumPy array or Python scalar values in inputs get cast as tensors.
- Keras mask metadata is only collected from inputs.
- Layers are built (build(input_shape) method) using shape info from inputs only.
- input_spec compatibility is only checked against inputs.
- Mixed precision input casting is only applied to inputs. If a layer has tensor arguments in *args or **kwargs, their casting behavior in mixed precision should be handled manually.
- The SavedModel input specification is generated using inputs only.
- Integration with various ecosystem packages like TFMOT, TFLite, TF.js, etc is only supported for inputs and not for tensors in positional and keyword arguments.
*args – Additional positional arguments. May contain tensors, although this is not recommended, for the reasons above.
**kwargs –
Additional keyword arguments. May contain tensors, although this is not recommended, for the reasons above. The following optional keyword arguments are reserved: - training: Boolean scalar tensor of Python boolean indicating

whether the call is meant for training or inference.
- mask: Boolean input mask. If the layer’s call() method takes a mask argument, its default value will be set to the mask generated for inputs by the previous layer (if input did come from a layer that generated a corresponding mask, i.e. if it came from a Keras layer with masking support).

Returns:

A tensor or list/tuple of tensors.