Job description :

· Knowledge in Cloud Computing by using Spark, Azure Databricks, Azure Data Factory

· Knowledge in programming language Python/Scala.

· Knowledge in Spark/PySpark (Core and Streaming) and hands-on to transform using Streaming.

· Knowledge building real time or batch ingestion and transformation pipelines.

· Working experience and strong knowledge in Databricks is a plus.

· Analyse existing queries for performance improvements.

· Develop procedures and scripts for data migration.

· Provide timely scheduled management reporting.

· Investigate exceptions regarding asset movements.

Primary skills:

Azure Databricks, Azure Data Factory, Spark

Secondary skills

Python, Scala

Works in the area of Software Engineering, which encompasses the development, maintenance and optimization of software solutions/applications.1. Applies scientific methods to analyse and solve software engineering problems.2. He/she is responsible for the development and application of software engineering practice and knowledge, in research, design, development and maintenance.3. His/her work requires the exercise of original thought and judgement and the ability to supervise the technical and administrative work of other software engineers.4. The software engineer builds skills and expertise of his/her software engineering discipline to reach standard software engineer skills expectations for the applicable role, as defined in Professional Communities.5. The software engineer collaborates and acts as team player with other software engineers and stakeholders.