Investigating file transfer service issues
This guide is for Integration Hub technical team members investigating a file in the Integration Hub file transfer service that has not reached its expected destination. It covers finding where the file stopped. For dead-letter queue (DLQ) messages, see Investigating and replaying file transfer DLQ messages.
Collect the file details
Before investigating, obtain the environment, username, object key or file name, upload timestamp with time zone, and the client’s protocol. Where possible, obtain the S3 object version ID. Search using the object key and version ID rather than just the file name, as different users can upload files with the same name.
Trace the file through the service
Use the file’s object key, version ID and timestamp to work through these stages in order:
- AWS Transfer Family and incoming bucket: confirm that the AWS Transfer
Family server accepted the upload. Check the structured Transfer log group,
/aws/transfer/integration-hub-file-transfer, then confirm the expected object version is in theincomingbucket. - File received event: check the default EventBridge rule named
incoming-s3-object-createdand theintegration-hub-file-transfer-file-received-adapterLambda log group. This adapter converts the S3 notification into the canonicalFileReceived.v1event. - Staging: check the
integration-hub-file-transferEventBridge bus for theFileReceived.v1event and theintegration-hub-file-transfer-stageLambda log group. The stage function copies the exact object version fromincomingto theprocessingbucket, where it is prepared for scanning. - Malware scan: inspect the object in the
processingbucket and its GuardDuty Malware Protection tags. Check the default EventBridge rule namedguardduty-malware-scan-resultand theintegration-hub-file-transfer-file-scan-result-recorded-adapterLambda log group. The adapter publishesFileScanResultRecorded.v1after associating the scan result with the staged file. - Routing: check the
FileScanResultRecorded.v1event on theintegration-hub-file-transferEventBridge bus and theintegration-hub-file-transfer-routeLambda log group. The route function moves the exact processing-object version toclean,quarantineorinvestigationstorage. - Clean-file action: for a file in the
cleanbucket, check theFileRouted.v1event and thefile-action-execution-requested-adapterLambda log group. The adapter finds the most specific dispatch configuration for the object key and publishes aFileActionExecutionRequested.v1event.
The EventBridge bus writes full event details to the CloudWatch log group:
/aws/vendedlogs/events/event-bus/integration-hub-file-transfer
This is useful when the Lambda logs show that an expected event was not received. EventBridge also retains an archive of file-transfer events for the configured log-retention period.
Interpret the destination
| Destination | Meaning | Investigation focus |
|---|---|---|
clean |
GuardDuty recorded a clean scan result and the route completed. | Check FileRouted.v1, dispatch matching and the downstream action or notification. |
quarantine |
GuardDuty identified potential malware. | Treat the file as potentially malicious. Check the scan result and follow the security incident process. |
investigation |
The file could not be routed as clean or quarantined, for example because the scan result was unclear or unsupported. | Check GuardDuty scan details, the scan-result adapter and route Lambda logs. |
incoming or processing only |
The file has not yet completed a later stage. | Continue the trace from the first stage that does not have the expected event or log entry. |
Check service health
CloudWatch alarms cover errors, throttles and long-running invocations for the file-received adapter, scan-result adapter, stage, route and file-action adapter. They also cover failed EventBridge target invocations, GuardDuty scan failures and DynamoDB throttling. Use the alarm timestamp to narrow the log search and identify the affected stage.
The configured alarm definitions are in
locals-cloudwatch.tf.
Preserve investigation evidence
Record the environment, username, object key, object version ID, timestamps, observed bucket location, relevant EventBridge event IDs and non-sensitive log messages. Do not place file content, passwords, private keys or Secrets Manager values in tickets, Slack or logs.