Why ffcv is faster than native pytorch data loading?  #43

Activity

1. gongbudaizhe

   changed the title [-]Why ffcv is faster than native pytorch data loading? how does ffcv compare to Nvidia DALI?[/-] [+]Why ffcv is faster than native pytorch data loading? [/+] on Jan 19, 2022
2. GuillaumeLeclerc commented on Jan 19, 2022

   

   GuillaumeLeclerc

   on Jan 19, 2022

   Collaborator

   More actions

   Hello!

   Thank you very much. There are many reasons but on the top of my head:

   • Single file vs many smaller files (less stress on the file system)
   • Using the knowledge about which samples will be needed in the future to start reading the data ahead of time and absorb the latency of slow storage
   • The use of threads instead of processes remove the need of any inter-process communication (shared memory etc...) and reduces memory consumption
   • Compilation of the CPU augmentations transforms in machine code at the start of training (JIT).
   • Copying the next batches while the compute is happening instead of simply asynchronously (but still on the critical path)
   • Ability to use to move augmentations to CPU or GPU depending on where is your bottleneck.
   • The use of pre-resized datasets
   • In the case of systems with a lot of RAM, leveraging operating system level caching to implicitly store the entire dataset in RAM
   • Being able to use alternative or a mix of different encoding for the data. JPEG can be slow to decode, if you have enough storage/RAM but not enough compute using uncompressed images might be faster, and FFCV enables that simply by switching a flag.

   🚀React with 🚀7Reacted by busycalibrating, Andrew Ilyas, JianyuanYin, Masashi Sode, Nguyen Van Thanh, Kevin Zakka and Tyler Luan
3. libffcv

   locked and limited conversation to collaborators on Jan 19, 2022
4. GuillaumeLeclerc

   converted this issue into a discussion #47 on Jan 19, 2022