Fixing the Python Script Generated by Lake Formation That Did Not Work

ToC

• Overview
• Conclusion First
• Summary

---

Overview

As of 2022.08.02,
there was an issue where running the Glue Workflow generated by the Lake Formation blueprint incremental database resulted in an abnormal termination.

Conclusion First

The finally fixed script is available below.
For the details of the changes, please refer to the commit log.

https://github.com/kenzo0107/aws-glue-jobs-incremental-database-etl

The main changes are as follows.

1. fix: TypeError: object of type ‘filter’ has no len()
2. fix: TypeError: Strings must be encoded before hashing
3. fix: table becomes not found when the table suffix is duplicated

Summary

I have shared the generated script with the AWS SA,
so I expect it will be addressed in due course.

That’s all.
I hope this is helpful.