Translator: BXuan694

All datasets are subclasses of torch. Utils.data. Dataset, that is, they implement __getitem__ and __len__ methods. . As a result, they can be passed to the torch utils. Data. The DataLoader, then pass the torch. Multiprocessing implement the parallel loading of the data. Such as:

imagenet_data = torchvision.datasets.ImageFolder('path/to/imagenet_root/')
data_loader = torch.utils.data.DataLoader(imagenet_data,
                                          batch_size=4,
                                          shuffle=True,
                                          num_workers=args.nThreads)
Copy the code

Data sets included so far include:

The data set

  • MNIST
  • Fashion-MNIST
  • EMNIST
  • COCO
    • Captions
    • Detection
  • LSUN
  • ImageFolder
  • DatasetFolder
  • Imagenet-12
  • CIFAR
  • STL10
  • SVHN
  • PhotoTour
  • SBU
  • Flickr
  • VOC

The interfaces of the above data sets are basically similar. They include at least two common parameters transform and target_transform to transform the input and target, respectively.

class torchvision.datasets.MNIST(root, train=True, transform=None, target_transform=None, download=False)Copy the code

MNIST dataset.

Parameters:

  • root(string) – the root directory where the dataset is storedprocessed/training.ptandprocessed/test.ptFile.
  • train(bool. optional) – If set to True, fromtraining.ptCreate data sets, otherwise fromtest.pt� creation.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform （Can be called . optional) – A function or transformation that takes a target and transforms it.
class torchvision.datasets.FashionMNIST(root, train=True, transform=None, target_transform=None, download=False)Copy the code

The fashion-MnIST dataset.

Parameters:

  • root(string) – the root directory where the dataset is storedprocessed/training.ptandprocessed/test.ptFile.
  • train(bool. optional) – If set to True, fromtraining.ptCreate data sets, otherwise fromtest.pt� creation.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
class torchvision.datasets.EMNIST(root, split, **kwargs)Copy the code

EMNIST data set.

Parameters:

  • root(string) – the root directory where the dataset is storedprocessed/training.ptandprocessed/test.ptFile.
  • split(string) — This dataset is divided into six types:byclass.bymerge.balanced.letters.digitsandmnist. This parameter specifies which of these to select.
  • train(bool. optional) – If set to True, fromtraining.ptCreate data sets, otherwise fromtest.pt� creation.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.

Note:

The following requires a pre-installed COCO API.

class torchvision.datasets.CocoCaptions(root, annFile, transform=None, target_transform=None)Copy the code

MS Coco Captions data set.

Parameters:

  • root(string) — the destination directory to download the data.
  • annFile(string) – Path of json annotation file.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.ToTensor.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.

The sample

import torchvision.datasets as dset
import torchvision.transforms as transforms
cap = dset.CocoCaptions(root = 'dir where images are',
                        annFile = 'json annotation file',
                        transform=transforms.ToTensor())

print('Number of samples: ', len(cap))
img, target = cap[3] # load 4th sample

print("Image Size: ", img.size())
print(target)
Copy the code

Output:

Number of samples: 82783
Image Size: (3L, 427L, 640L)
[u'A plane emitting smoke stream flying over a mountain.',
u'A plane darts across a bright blue sky behind a mountain covered in snow',
u'A plane leaves a contrail above the snowy mountain top.',
u'A mountain that has a plane flying overheard in the distance.',
u'A mountain view with a plume of smoke in the background']
Copy the code
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: A tuple (image, target), where target is a list type that contains a description of the image image.
— —
Return type: tuple
— — 
class torchvision.datasets.CocoDetection(root, annFile, transform=None, target_transform=None)Copy the code

MS Coco Detection dataset.

Parameters:

  • root(string) — the destination directory to download the data.
  • annFile(string) – Path of json annotation file.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.ToTensor.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: The tuple (image, target), where target iscoco.loadAnnsObject returned.
— —
Return type: tuple
— — 
class torchvision.datasets.LSUN(root, classes='train', transform=None, target_transform=None)Copy the code

LSUN dataset.

Parameters:

  • root(string) – The root directory where data files are stored.
  • classes(string or list) — one of {‘ train ‘, ‘val’, ‘test’}, or a list of categories to load, such as [‘ bedroom_train ‘, ‘church_train’].
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: A tuple (image, target), where target is the index of the target category.
— —
Return type: tuple
— — 
class torchvision.datasets.ImageFolder(root, transform=None, target_transform=None, loader=<function default_loader>)Copy the code

A general-purpose data loader whose images should be saved as follows:

root/dog/xxx.png
root/dog/xxy.png
root/dog/xxz.png

root/cat/123.png
root/cat/nsdf3.png
root/cat/asd932_.png
Copy the code

Parameters:

  • root(string) – Root directory path.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
  • Loader – a function that loads images from a given path.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (sample, target), where target is the class index of the target class.
— —
Return type: tuple
— — 
class torchvision.datasets.DatasetFolder(root, loader, extensions, transform=None, target_transform=None)Copy the code

A general-purpose data loader whose data should be stored as follows:

root/class_x/xxx.ext
root/class_x/xxy.ext
root/class_x/xxz.ext

root/class_y/123.ext
root/class_y/nsdf3.ext
root/class_y/asd932_.ext
Copy the code

Parameters:

  • root(string) – Root directory path.
  • loader(Can be called) – a function that loads data from a given path.
  • extensions(list[string]) — list of permitted extensions.
  • transform(Can be called . optional) a function or transformation that takes in data and returns the transformed data. There are for picturestransforms.RandomCrop.
  • Target_transform – a function or transform that is transformed by entering a target.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (sample, target), where target is the class index of the target class.
— —
Return type: tuple
— —

This class makes it easy to implement the ImageFolder dataset. Data preprocessing is shown here.

Sample �.

class torchvision.datasets.CIFAR10(root, train=True, transform=None, target_transform=None, download=False)Copy the code

CIFAR10 data set.

Parameters:

  • root(string) — data set root directory, or it should existcifar-10-batches-pyFolder, either when Download is set to Truecifar-10-batches-pyThe folder is saved here.
  • train(bool. optional) – created from the training set if set to True, from the test set otherwise.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (image, target) where target is the class index of the target class.
— —
Return type: tuple
— — 
class torchvision.datasets.CIFAR100(root, train=True, transform=None, target_transform=None, download=False)Copy the code

CIFAR100 data set.

This is a subset of the CIFAR10 dataset.

class torchvision.datasets.STL10(root, split='train', transform=None, target_transform=None, download=False)Copy the code

STL10 data set.

Parameters:

  • root(string) — the root directory of the dataset, which should containstl10_binaryFolder.
  • split(string) — {‘ train ‘, ‘test’, ‘unlabeled’, ‘train+unlabeled’}, and select the corresponding dataset.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (image, target), where target should be the class index of the target class.
— —
Return type: tuple
— — 
class torchvision.datasets.SVHN(root, split='train', transform=None, target_transform=None, download=False)Copy the code

SVHN data sets. Note: the SVHN� dataset specifies 10 as a label for the number 0. However, here we specify 0 as a label for the digit 0 to be compatible with PyTorch’s loss function, because the loss � function requires the class tag to be in the range of [0, C-1].

Parameters:

  • root(string) — The root directory of the dataset, which should containSVHNFolder.
  • split(string) — one of {‘ train ‘, ‘test’, ‘extra’} and the corresponding data set is selected. ‘Extra’ is an extra training set.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data. Such as:transforms.RandomCrop.
  • target_transform(Can be called . optional) – A function or transformation that takes a target and transforms it.
  • download(bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (image, target) where target is the class index of the target class.
— —
Return type: tuple
— — 
class torchvision.datasets.PhotoTour(root, name, train=True, transform=None, download=False)Copy the code

Learning Local Image Descriptors Data

Parameters:

  • root(string) — The root directory to save the image.
  • name(string) — The data set to load.
  • transform(Can be called . optional) — a function or transformation that inputs PIL image and returns the transformed data.
  • download (bool. optional) – If set to True, download data from the Internet and place it in the root folder. If the root directory already has data, it will not download again.
__getitem__(index)Copy the code
Parameters: index (intIndex) –
Returns: (data1, data2, matches)
— —
Return type: tuple
— —