Add L3 Harris S3 bucket sync function to executor lambda

[jamesray]

Summary

Implement a new sync function for the L3 Harris S3 bucket in the existing Lambda. This function should sync data from the L3 Harris S3 bucket into our designated bucket within our AWS account.

Requirements

• The function must check the data already present in our bucket before syncing to avoid unnecessary transfers.
• Implement intelligent timing to optimize sync intervals and resources (e.g., avoid redundant syncs when no new data is available).
• Support resume/repair functionality in case a transfer fails or is interrupted, ensuring data integrity and avoiding duplication.

Acceptance Criteria

• The Lambda can detect and initiate sync from the L3 Harris S3 bucket.
• Sync operations only transfer missing or new data.
• Resume/repair works effectively for partial transfers.
• Logs or monitoring alerts for sync failures or repair actions.

Additional Context

• The lambda is currently used for miscellaneous functions triggered by Cloudwatch CRON rules.
• Solution should use Python (current main language of the repo).
• The data in the L3 Harris buckets contains 2 weeks of rolling data.
• We do not yet understand the contents of the data files at this time or how they are organized.

[ehsteve]

We need to find out how often this data is updated to decide on a cadence but let's start with every 6 hours.

[ehsteve]

@jamesray i think you suggested that this be performed in an EC2 instance so this issue should then be closed and a new one opened.