| Dependency | Reason |
|---|---|
| Task Instance State | Task is in the 'upstream_failed' state which is not a valid state for execution. The task must be cleared in order to be run. |
| Trigger Rule | Task's trigger rule 'all_success' requires all upstream tasks to have succeeded, but found 1 non-success(es). upstream_tasks_state={'total': 1, 'successes': 0, 'skipped': 0, 'failed': 0, 'upstream_failed': 1, 'done': 1}, upstream_task_ids={'create_cluster_and_add_emr_steps'} |
| Dagrun Running | Task instance's dagrun was not in the 'running' state but in the state 'failed'. |
| Execution Date | The execution date is 2024-06-07T00:00:00+00:00 but this is before the task's start date 2024-07-02T09:00:00+00:00. |
| Attribute | Value |
|---|---|
| dag_id | livepop-shuffle-aggregator-modeltransform_v004 |
| duration | None |
| end_date | 2024-06-07 01:21:52.690032+00:00 |
| execution_date | 2024-06-07T00:00:00+00:00 |
| executor_config | {} |
| generate_command | <function TaskInstance.generate_command at 0x7f7bf685a7b8> |
| hostname | |
| is_premature | False |
| job_id | None |
| key | ('livepop-shuffle-aggregator-modeltransform_v004', 'check_cluster_status', <Pendulum [2024-06-07T00:00:00+00:00]>, 1) |
| log | <Logger airflow.task (INFO)> |
| log_filepath | /opt/airflow/logs/livepop-shuffle-aggregator-modeltransform_v004/check_cluster_status/2024-06-07T00:00:00+00:00.log |
| log_url | https://airflow.devel.viooh.net.cn/admin/airflow/log?execution_date=2024-06-07T00%3A00%3A00%2B00%3A00&task_id=check_cluster_status&dag_id=livepop-shuffle-aggregator-modeltransform_v004 |
| logger | <Logger airflow.task (INFO)> |
| mark_success_url | https://airflow.devel.viooh.net.cn/success?task_id=check_cluster_status&dag_id=livepop-shuffle-aggregator-modeltransform_v004&execution_date=2024-06-07T00%3A00%3A00%2B00%3A00&upstream=false&downstream=false |
| max_tries | 4 |
| metadata | MetaData(bind=None) |
| next_try_number | 1 |
| operator | EmrJobFlowSensor |
| pid | None |
| pool | default_pool |
| pool_slots | 1 |
| prev_attempted_tries | 0 |
| previous_execution_date_success | 2024-06-06 16:00:00+00:00 |
| previous_start_date_success | 2024-06-06 17:07:02.826878+00:00 |
| previous_ti | <TaskInstance: livepop-shuffle-aggregator-modeltransform_v004.check_cluster_status 2024-06-06 23:00:00+00:00 [upstream_failed]> |
| previous_ti_success | <TaskInstance: livepop-shuffle-aggregator-modeltransform_v004.check_cluster_status 2024-06-06 16:00:00+00:00 [success]> |
| priority_weight | 1 |
| queue | default |
| queued_dttm | None |
| raw | False |
| run_as_user | None |
| start_date | 2024-06-07 01:21:52.690016+00:00 |
| state | upstream_failed |
| task | <Task(EmrJobFlowSensor): check_cluster_status> |
| task_id | check_cluster_status |
| test_mode | False |
| try_number | 1 |
| unixname | airflow |
| Attribute | Value |
|---|---|
| aws_conn_id | aws_default |
| dag | <DAG: livepop-shuffle-aggregator-modeltransform_v004> |
| dag_id | livepop-shuffle-aggregator-modeltransform_v004 |
| depends_on_past | False |
| deps | {<TIDep(Trigger Rule)>, <TIDep(Not Previously Skipped)>, <TIDep(Previous Dagrun State)>, <TIDep(Not In Retry Period)>} |
| do_xcom_push | True |
| downstream_list | [] |
| downstream_task_ids | set() |
| None | |
| email_on_failure | True |
| email_on_retry | True |
| end_date | None |
| execution_timeout | None |
| executor_config | {} |
| extra_links | [] |
| failed_states | ['TERMINATED_WITH_ERRORS'] |
| failure_message_from_response | <function EmrJobFlowSensor.failure_message_from_response at 0x7f7be66d9378> |
| global_operator_extra_link_dict | {} |
| hook | None |
| inlets | [] |
| job_flow_id | {{ task_instance.xcom_pull(task_ids='create_cluster_and_add_emr_steps', key='return_value') }} |
| lineage_data | None |
| log | <Logger airflow.task.operators (INFO)> |
| logger | <Logger airflow.task.operators (INFO)> |
| max_retry_delay | None |
| mode | poke |
| on_failure_callback | <function task_fail_slack_alert at 0x7f7be66b8a60> |
| on_retry_callback | None |
| on_success_callback | None |
| operator_extra_link_dict | {} |
| operator_extra_links | () |
| outlets | [] |
| owner | data.engineers@viooh.com |
| params | {} |
| poke_interval | 60 |
| pool | default_pool |
| pool_slots | 1 |
| priority_weight | 1 |
| priority_weight_total | 1 |
| queue | default |
| reschedule | False |
| resources | None |
| retries | 4 |
| retry_delay | 0:01:00 |
| retry_exponential_backoff | False |
| run_as_user | None |
| schedule_interval | 0 * * * * |
| shallow_copy_attrs | () |
| sla | None |
| soft_fail | False |
| start_date | 2024-07-02 09:00:00+00:00 |
| state_from_response | <function EmrJobFlowSensor.state_from_response at 0x7f7be66d92f0> |
| subdag | None |
| target_states | ['TERMINATED'] |
| task_concurrency | None |
| task_id | check_cluster_status |
| task_type | EmrJobFlowSensor |
| template_ext | () |
| template_fields | ['job_flow_id', 'target_states', 'failed_states'] |
| timeout | 604800 |
| trigger_rule | all_success |
| ui_color | #66c3ff |
| ui_fgcolor | #000 |
| upstream_list | [<Task(EmrCreateJobFlowOperator): create_cluster_and_add_emr_steps>] |
| upstream_task_ids | {'create_cluster_and_add_emr_steps'} |
| valid_modes | ['poke', 'reschedule'] |
| wait_for_downstream | False |
| weight_rule | downstream |