The scheduler does not appear to be running. Last heartbeat was received .

The DAGs list may not update, and new tasks will not be scheduled.

DAG: primary-audience-global-masterdb_v003

schedule: None


Task Instance: create_cluster


Rendered Template

templates_dict
None
op_args
[]
op_kwargs
{'audience-config-file': 'primary_audience_global_masterdb_conf',
 'audience-load-bootstrap': True,
 'cluster-name': 'primary-audience-global-masterdb-JCDECAUX_CN_SH_METRO-2211',
 'core-instance-capacity': 1,
 'core-instance-types': 'm5.2xlarge,m5.4xlarge',
 'dag-id': 'primary-audience-global-masterdb_v003',
 'emr-steps': '[\n'
              '      {\n'
              '        "step-name": "SaveToDataBase",\n'
              '        "py-script": "jobs/save_to_sql.py",\n'
              '        "config-file": '
              '"configs/primary-audience-global-masterdb-uat/1.0.0/job.json",\n'
              '        "config-json": [\n'
              '          {"spark.app.audience.country":"CN"},\n'
              '          '
              '{"spark.app.audience.media.owner":"JCDECAUX_CN_SH_METRO"},\n'
              '          {"spark.app.audience.version":"2211"},\n'
              '          '
              '{"spark.app.audience.output.bucket":"s3a://global-cn-audiences-output"},\n'
              '          {"spark.app.audience.computation.date":"20221103"},\n'
              '          {"spark.app.audience.database":"shanghaimetro_dev"},\n'
              '          {"spark.executor.memoryOverhead":"2500"}\n'
              '        ],\n'
              '        "artifact": "AudiencesPipeline",\n'
              '        "packages": "mysql:mysql-connector-java:5.1.46"\n'
              '      }\n'
              ']',
 'emr-version': 'emr-6.0.0',
 'master-instance-types': 'm5.xlarge,m5.2xlarge',
 'schedule_interval': None,
 'task-instance-capacity': 0,
 'task-instance-types': 'm5.2xlarge,m5.4xlarge',
 'trigger_dags': []}