fusionlab.datasets.utils module

class fusionlab.datasets.utils.HFDataset(dataset)

Bases: Dataset

Base Hugginface dataset wrapper class :param dataset: a dataset object that contains a getitem method

class fusionlab.datasets.utils.LSTimeSegDataset(data_dir, annotation_path, class_map, column_names)

Bases: Dataset

Dataset for label-studio timeseries segmentation task :param data_dir: directory of csv files :param annotation_path: path to annotation json file :param class_map: a dictionary mapping class names to class indices :param column_names: a list of column names

Returns:

torch tensor, shape (num_samples, num_channels) mask: torch tensor, shape (num_samples, )

Return type:

signals

Example

ds = LSTimeSegDataset(data_dir=”./12”,

annotation_path=”./12.json”, class_map={“N”: 1, “p”: 2, “t”: 3}, column_names=[‘i’, ‘ii’, ‘iii’, ‘avr’, ‘avl’, ‘avf’, ‘v1’, ‘v2’, ‘v3’, ‘v4’, ‘v5’, ‘v6’])

signals, mask = ds[0]

preprocess(signals)

fusionlab.datasets.utils.count_parameters(model, trainable_only=False)

Returns the number of parameters in a model

Parameters:

• model (Module) – a pytorch model

• trainable_only (bool) – if True, only count trainable parameters

Returns:

number of parameters in the model

Return type:

num_parameters

Reference: https://discuss.pytorch.org/t/how-do-i-check-the-number-of-parameters-of-a-model/4325/9

fusionlab.datasets.utils.download_file(url, download_root, extract_root=None, filename=None, extract=False)

Download a file from a url and optionally extract it to a target directory. :type url: str :param url: URL to download file from :type url: str :type download_root: str :param download_root: Directory to place downloaded file in :type download_root: str :type extract_root: Optional[str] :param extract_root: Directory to extract downloaded file to :type extract_root: str, optional :type filename: Optional[str] :param filename: Name to save the file under. If None, use the basename of the URL :type filename: str, optional :param extract: If True, extract the downloaded file. Otherwise, do not extract. :type extract: bool, optional

Return type:

None