Support submissions across multiple gunicorn workers

[jkalayan]

Summary

This pull request improves the workflow upload directory handling in a multi-worker environment, ensuring that workflows are consistently accessible across all Gunicorn workers.

Workflow directory management and multi-worker support:

• Introduced a deterministic directory creation method (_dir_for) in WorkflowStore so that all Gunicorn workers use the same upload directory for a given workflow, replacing the previous random directory approach. This ensures that workflows are accessible regardless of which worker handles the request.
• Updated the reset and get methods in WorkflowStore to use the deterministic directory, recover workflow state from disk if created by another worker, and synchronize activity timestamps using os.utime to signal liveness across workers.
• Removed the dependency on make_upload_tmpdir, simplifying directory management and making the process more predictable.

Deployment improvements:

• Modified the Gunicorn command in the Dockerfile to use the gthread worker class with multiple workers and threads, improving concurrency and ensuring compatibility with the new shared directory approach.

Testing:

• Added a test suite (tests/test_form/test_workflows.py) to verify workflow directory creation, recovery from disk, error handling for invalid IDs, deletion, and cleanup of expired workflows.

[coveralls]

Coverage Report for CI Build 38962498713

Coverage increased (+0.9%) to 92.473%

Details

• Coverage increased (+0.9%) from the base build.
• Patch coverage: 2 uncovered changes across 1 file (20 of 22 lines covered, 90.91%).
• 3 coverage regressions across 1 file.

Uncovered Changes

File	Changed	Covered	%
biosimdb_interface/form/workflows.py	22	20	90.91%

Coverage Regressions

3 previously-covered lines in 1 file lost coverage.

File	Lines Losing Coverage	Coverage
biosimdb_interface/form/utils.py	3	80.77%

---

Coverage Stats

Relevant Lines:	744
Covered Lines:	688
Line Coverage:	92.47%
Coverage Strength:	8.32 hits per line

---

💛 - Coveralls