Skip to content
Snippets Groups Projects
Commit f377b540 authored by edvs19's avatar edvs19
Browse files

teste

parent 0c7a232f
Branches
No related tags found
No related merge requests found
......@@ -19,12 +19,12 @@ repo_home = '/opt/airflow/dags/repo'
)
def process_records():
@task
def task_spark(params: dict):
def task_spark():
return SparkSubmitOperator(
task_id="landing_to_bronze",
application=f"{repo_home}/gen_bronze.py",
application_args=[
params["input_file"]
"fichas_cadastro_individual_5000.jsonl"
],
packages="org.apache.hadoop:hadoop-aws:3.3.4"
)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment