Python: parallelism using multiprocessing and multithreading

Have been working with Python since last few months. Was building some small modules. Quite excited about my new learnings. Recently I’d developed a distributed parallel data import process by following the generic Pub-Sub pattern. Earlier I’d published a 3 tier design of the same, developed using PHP. Comparative to that, it’s a simpler 2 tier design and this time I developed the process using Python. Have used multiprocessing and multithreading to achieve parallelism.
Continue reading

Advertisements

Supervisord: Parallel child process spawning and monitoring

Supervisord is a simple and popular choice for process monitoring. Supervisord‘s primary purpose is to create and manage processes based on data in its configuration file. It does this by creating subprocesses. Each subprocess spawned by supervisor is managed for the entirety of its lifetime by supervisord. Continue reading