Home > Enterprise >  How can I download to my pc an image dataset after loading it into a dataloader in pytorch?
How can I download to my pc an image dataset after loading it into a dataloader in pytorch?

Time:10-13

I've been following this article: https://github.com/moein-shariatnia/Deep-Learning/tree/main/Image Colorization Tutorial

And I have a problem. I'm trying to download the images I have loaded into the pytorch dataloader, but since they are chosen randomly from a larger dataset I can't figure it out. How can I download the exact images that I've loaded into "train_dl" to my PC (as a folder) after they are being chosen? I have already trained my model on these exact images, so I'll need them for future work.

CodePudding user response:

Dataloaders should have a dataset attribute which you can use to find the paths in your code

In the code:

class ColorizationDataset(Dataset):
    def __init__(self, paths, split='train'):
        ...
        
        self.split = split
        self.size = SIZE
        self.paths = paths #<<<<- IMPORTANT LINE HERE
    
    def __getitem__(self, idx):
        ...
    
    def __len__(self):
        return len(self.paths)

def make_dataloaders(batch_size=16, n_workers=4, pin_memory=True, **kwargs): # A handy function to make our dataloaders
    dataset = ColorizationDataset(**kwargs)
    dataloader = DataLoader(dataset, batch_size=batch_size, num_workers=n_workers,
                            pin_memory=pin_memory)
    return dataloader

So, to get the items from your dataloader you can run:

train_paths = train_dl.dataset.paths

You can loop through this list to save the images individually (not recommended) or just save it as text with the train/val/test splits as separate files.

  • Related