Save log (heaps) cause a concurrency problem

[huogerac]

Problem: The container makes many calls to the save output (add_log) method which raises:

RuntimeError: CONTAINER EXEC ERROR: (psycopg2.DatabaseError) error with status PGRES_TUPLES_OK and no message from the libpq

the cause might be the multiprocessing is calling the service which uses the sqlalchemy which has an instance linked to the flask.app.

Links:

• https://stackoverflow.com/questions/30241911/psycopg2-error-databaseerror-error-with-no-message-from-the-libpq

To study:

• https://www.toptal.com/python/beginners-guide-to-concurrency-and-parallelism-in-python
• https://blog.francium.tech/global-interpreter-lock-gil-and-concurrent-execution-in-python-70f17f09e892

Options:

• Rather than use multiprocessing inside Flask to call the run_task (execute command inside containers), it might be better idea use something aka celery.