load_multiple_submissions

load_multiple_submissions is the entrypoint of the Broker Submission Loader and is responsible for identifying DATA Act Broker Submissions (DABS) in Broker to be loaded into USAspending, and initiating said loads by executing a load_submission per new/updated submission. It is executed from the command line via ./manage.py load_multiple_submissions fy quarter and is an integral part of the nightly USAspending pipeline where its execution is initiated directly via the pipeline process.

load_multiple_submissions accepts three command line arguments:

• fy is a required positional argument that restricts the search for new/updated submissions to a single fiscal year.
• quarter is a required positional argument that restricts the search for new/updated submissions to a single fiscal quarter.
• --safe is an optional switch that causes the loader to only list out submissions that are to be loaded. No data are created or updated when this switch is supplied.

Submissions are loaded from Broker based on the following criteria:

• submission.reporting_fiscal_year for the fiscal year indiciated by the fy command line argument.
• submission.reporting_fiscal_period for the fiscal quarter indicated by the quarter command line argument.
• Not flagged as File D by the submission.d2_submission boolean.
• published (status 2) or updated (status 3) as indicated by the submission.publish_status_id column.
• "Certified" as indicated by the submission identifier's presence in the Broker certify_history table.

Missing submissions are identified by their absence from USAspending's submission_attributes table. Updated submissions are identified by their update_date falling prior to Broker's maximum certify_history.certified_date for the submission. Submissions can be certified multiple times if their data are re-submitted for corrections causing them to end up with multiple certification dates.