Dependency | Reason |
---|---|
Task Instance State | Task is in the 'success' state which is not a valid state for execution. The task must be cleared in order to be run. |
Dagrun Running | Task instance's dagrun was not in the 'running' state but in the state 'success'. |
Execution Date | The execution date is 2024-06-07T00:00:00+00:00 but this is before the task's start date 2024-09-18T00:00:00+00:00. |
1 2 3 4 5 6 7 8 | def run_publisher_sync(config):
# Retrieve API URL and S3 path from Airflow variables or use default values
api_url = config["api_url"]
s3_bucket = config["s3_bucket"]
env = config["env"]
# The structure will be <Bucket>/<uat>/publishers/publisher_data.parquet
s3_key = env + '/' + config["s3_key"]
PublisherDataS3Sink().execute(api_url, s3_bucket, s3_key)
|
Attribute | Value |
---|---|
dag_id | publisher-s3-sync |
duration | 1.171004 |
end_date | 2024-06-07 01:00:10.968588+00:00 |
execution_date | 2024-06-07T00:00:00+00:00 |
executor_config | {} |
generate_command | <function TaskInstance.generate_command at 0x7f603554be18> |
hostname | airflow-worker-1.airflow-worker.data-eng.svc.cluster.local |
is_premature | False |
job_id | 552999 |
key | ('publisher-s3-sync', 'publisher_sync_task', <Pendulum [2024-06-07T00:00:00+00:00]>, 2) |
log | <Logger airflow.task (INFO)> |
log_filepath | /opt/airflow/logs/publisher-s3-sync/publisher_sync_task/2024-06-07T00:00:00+00:00.log |
log_url | https://airflow.devel.viooh.net.cn/admin/airflow/log?execution_date=2024-06-07T00%3A00%3A00%2B00%3A00&task_id=publisher_sync_task&dag_id=publisher-s3-sync |
logger | <Logger airflow.task (INFO)> |
mark_success_url | https://airflow.devel.viooh.net.cn/success?task_id=publisher_sync_task&dag_id=publisher-s3-sync&execution_date=2024-06-07T00%3A00%3A00%2B00%3A00&upstream=false&downstream=false |
max_tries | 3 |
metadata | MetaData(bind=None) |
next_try_number | 2 |
operator | PythonOperator |
pid | 304 |
pool | default_pool |
pool_slots | 1 |
prev_attempted_tries | 1 |
previous_execution_date_success | 2024-06-06 23:00:00+00:00 |
previous_start_date_success | 2024-06-07 00:00:23.270111+00:00 |
previous_ti | <TaskInstance: publisher-s3-sync.publisher_sync_task 2024-06-06 23:00:00+00:00 [success]> |
previous_ti_success | <TaskInstance: publisher-s3-sync.publisher_sync_task 2024-06-06 23:00:00+00:00 [success]> |
priority_weight | 1 |
queue | default |
queued_dttm | 2024-06-07 01:00:04.511027+00:00 |
raw | False |
run_as_user | None |
start_date | 2024-06-07 01:00:09.797584+00:00 |
state | success |
task | <Task(PythonOperator): publisher_sync_task> |
task_id | publisher_sync_task |
test_mode | False |
try_number | 2 |
unixname | airflow |
Attribute | Value |
---|---|
dag | <DAG: publisher-s3-sync> |
dag_id | publisher-s3-sync |
depends_on_past | False |
deps | {<TIDep(Not Previously Skipped)>, <TIDep(Trigger Rule)>, <TIDep(Previous Dagrun State)>, <TIDep(Not In Retry Period)>} |
do_xcom_push | True |
downstream_list | [] |
downstream_task_ids | set() |
None | |
email_on_failure | True |
email_on_retry | True |
end_date | None |
execution_timeout | None |
executor_config | {} |
extra_links | [] |
global_operator_extra_link_dict | {} |
inlets | [] |
lineage_data | None |
log | <Logger airflow.task.operators (INFO)> |
logger | <Logger airflow.task.operators (INFO)> |
max_retry_delay | None |
on_failure_callback | None |
on_retry_callback | None |
on_success_callback | None |
op_args | [{'env': 'dev', 'api_url': 'https://publisher-service.private.devel.viooh.net.cn/api/v1/publishers', 's3_bucket': 'vd-data-common-lab', 's3_key': 'publishers/cn-dev.parquet'}] |
op_kwargs | {} |
operator_extra_link_dict | {} |
operator_extra_links | () |
outlets | [] |
owner | data.engineers@viooh.com |
params | {} |
pool | default_pool |
pool_slots | 1 |
priority_weight | 1 |
priority_weight_total | 1 |
provide_context | False |
queue | default |
resources | None |
retries | 3 |
retry_delay | 0:05:00 |
retry_exponential_backoff | False |
run_as_user | None |
schedule_interval | 0 * * * * |
shallow_copy_attrs | ('python_callable', 'op_kwargs') |
sla | None |
start_date | 2024-09-18T00:00:00+00:00 |
subdag | None |
task_concurrency | None |
task_id | publisher_sync_task |
task_type | PythonOperator |
template_ext | [] |
template_fields | ('templates_dict', 'op_args', 'op_kwargs') |
templates_dict | None |
trigger_rule | all_success |
ui_color | #ffefeb |
ui_fgcolor | #000 |
upstream_list | [] |
upstream_task_ids | set() |
wait_for_downstream | False |
weight_rule | downstream |