Warm tip: This article is reproduced from serverfault.com, please click

Pass Dynamic Parameters to AWS Glue

发布于 2020-12-04 16:24:47

I am trying to pass dynamic parameters to a glue job. I followed this question: AWS Glue Job Input Parameters

And configured my parameters like so:

enter image description here

I'm triggering the glue job with boto3 with the following code:

event = {
    '--ncoa': "True", 
    '--files': 'file.csv', 
    '--group_file': '3e93475d45b4ebecc9a09533ce57b1e7.csv', 
    '--client_slug': 'test', 
    '--slm_id': '12345'
}

glueClient.start_job_run(JobName='TriggerNCOA', Arguments=event)

and when I run this glue code:

args = getResolvedOptions(sys.argv, ['NCOA','Files','GroupFile','ClientSlug', 'SLMID'])

v_list=[{"ncoa":args['NCOA'],"files":args['Files'],"group_file":args['GroupFile'], "client_slug":args['ClientSlug'], "slm_id":args['SLMID']}]

print(v_list)

It just gives me 'a' for every value, not the values of the original event that I passed in from boto3. how do I fix that? Seems like im missing something very slight, but ive looked around and haven't found anything conclusive.

Questioner
DBA108642
Viewed
0
Achyut Vyas 2020-12-05 16:25:57

You are using CamelCase and Capital letters into Glue Job Parameters, but you are using small letters in python code to override the Parameters.

Ex.

The key of the job parameter in Glue is --ClientSlug but the key for Argument set in python code is --client_slug