Multiprocessing in Python: with Windows machine
Multiprocessing generally involves two approaches: via threads or via multiple processes . Although multiple processes come with an overhead of combining the results from various processes; it makes better utilization of multiple CPU cores which is commonplace these days. There are 3 main elements to the execution of this concept: dividing the workload in logical chunks, passing the chunks to individual processes, and compiling the output from each of those processes. There are various libraries available in python for achieving real parallel programming, one popular choice being multiprocessing . There are various blog posts available on the internet describing the nitty-gritty of working with this library (I have listed a couple in the references section). However, this post explores two other choices we can easily use to achieve the same. Concurrent.futures This module provides an interface for asynchronously executing callables. A good thing is that it comes p