MAC
2020-12-15 21:49:22
I eventually used Spark udf instead of DynamicFrame.Map.apply. Worked for me.
热门帖子
热门github
6
I am working on an AWS Glue job where I have a function "some_function" that I want to apply on DynamicFrame dy_f, but I also want to pass an input param to some_function.
Map.apply(frame=products_combination, f=search)
where some_function's definition is:
some_function(record, k)
What I've tried so far: Works:
Map.apply(frame=products_combination, f=search) ##provided i'm not taking k as input in some_function def as well
What is giving error:
Map.apply(frame=products_combination, f=search(k=10))
This returns "TypeError: search() missing 1 required positional argument: 'record'"
How can I pass a parameter to Map.apply function? I have gone through the documentation [here] but couldn't find my solution there.1
I eventually used Spark udf instead of DynamicFrame.Map.apply. Worked for me.