Python: parallelism using multiprocessing and multithreading

Have been working with Python since last few months. Was building some small modules. Quite excited about my new learnings. Recently I’d developed a distributed parallel data import process by following the generic Pub-Sub pattern. Earlier I’d published a 3 tier design of the same, developed using PHP. Comparative to that, it’s a simpler 2 tier design and this time I developed the process using Python. Have used multiprocessing and multithreading to achieve parallelism.
Continue reading

Advertisements

Distributed Job processing: RabbitMQ

I have been working for a distributed process. In generic term this is a process which needs to run continuously in a distributed manner by following push-pull communication. RabbitMQ is a great tool to achieve this solution. I’ve extended worker model to integrate publisher along with the same. Continue reading