Fullstack AWS Data Engineer
Capgemini
Data Science
Mexico
Posted on Dec 18, 2024
RH: Raul García
Position: Fullstack Azure Data Engineer
Location: Aguascalientes, Mexico
Industry - Sector: MALS
What you’ll do?
- Build, optimize, and maintain scalable data pipelines to process large datasets from various sources using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Implement ETL (Extract, Transform, Load) processes to integrate and transform data for analytical and operational purposes.
- Design and implement cloud-based data architectures using Azure SQL Database, Azure Data Lake Storage, Azure Blob Storage, and Azure Synapse Analytics.
- Build and manage data lakes, data warehouses, and data marts to enable seamless data flow and storage.
- Develop and maintain backend systems that handle data processing, storage, and retrieval with technologies such as Python, Spark, SQL, and Azure Data Factory.
- Ensure data consistency, quality, and reliability across multiple systems and services.
- Build and maintain data visualization applications, dashboards, and data exploration tools using Power BI, Azure Synapse, or other front-end tools.
- Create user-friendly data interfaces and reports for business users, ensuring that data insights are accessible and actionable.
- Work with both relational and non-relational databases on Azure, including Azure SQL Database, Cosmos DB, and Azure Data Warehouse.
- Optimize and tune databases for performance, including query optimization, indexing, and partitioning.
- Automate data workflows and deployment pipelines using Azure DevOps, Terraform, or similar infrastructure-as-code tools.
- Implement CI/CD (Continuous Integration/Continuous Deployment) processes for data systems and analytics solutions.
- Implement best practices for data security, including encryption, role-based access control (RBAC), and data masking.
- Ensure compliance with data governance policies and standards, ensuring the data is clean, accurate, and available.
- Work closely with data scientists, analysts, and business stakeholders to understand data needs and provide solutions.
- Provide technical leadership and mentorship to junior team members, helping them grow their skills in cloud data engineering.
- Monitor and optimize the performance of data systems, ensuring high availability and scalability.
- Troubleshoot and resolve issues related to data integration, transformation, and reporting.
What you’ll bring:
- In-depth experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure SQL Database, Azure Blob Storage, Azure Data Lake, and Azure Data Warehouse.
- Proficiency in Python or Scala for data engineering and automation tasks.
- Strong SQL skills for data querying, data transformation, and performance tuning.
- Experience building ETL pipelines using Azure Data Factory, Azure Databricks, or SQL Server Integration Services (SSIS).
- Knowledge of batch and streaming data processing techniques using tools like Apache Spark, Azure Stream Analytics, or Apache Kafka.
- Strong understanding of data warehousing and data lake architectures, and experience implementing these solutions on Azure Synapse or Azure Data Lake.
- Experience with data storage solutions such as Azure Blob Storage, Cosmos DB, and SQL Server.
- Experience with Power BI, Tableau, or other visualization tools to create dashboards and interactive reports.
- Proficiency in Azure DevOps, Terraform, or similar tools for automating deployment, data pipeline orchestration, and infrastructure provisioning.
- Knowledge of data security best practices, including access management, encryption, and data masking techniques.
- Experience with Azure Active Directory (AD) and role-based access control (RBAC).
- Familiarity with Git and version control best practices for collaborative development and project management.
Soft skills:
Work Underpressure, Quality at work, Results Oriented
What can YOU expect in a career with Capgemini?
- Working in a team environment, Consultants will focus on the analysis, design and development of technology-based solutions for Capgemini’s clients.
- You will work alongside technical, functional and industry specialists to assist with the development, implementation and integration of innovative system solutions including methods, techniques and tools.
- You will contribute to client satisfaction by providing timely and responsive value-added services and work products.
- Capgemini offers a competitive compensation and benefits package.
- Headquartered in Paris, France, Capgemini has a presence of more than 340 thousand professionals in Mexico distributed among 3 sites located in Mexico City, Monterrey and Aguascalientes. A deeply multicultural organization.
- Capgemini has developed its own way of working, the Collaborative Business ExperienceTM, and draws on Rightshore, its worldwide delivery model.
You will love this job because
- Capgemini focuses on giving each new hire a YOU-nique experience through our recruitment process and on-boarding program, as well as by helping you to build your own career and professional skills foundation.
- Capgemini provides a collaborative environment that embodies and holds the following stated values close to heart: Honesty, Boldness, Trust, Freedom, Team Spirit, Modesty, and Fun.
- Capgemini cultivates an atmosphere for development that enables YOU to be hands-on, planning for your growth, both horizontally and vertically.
“At Capgemini Mexico, we aim to attract the best talent and are committed to creating a diverse and inclusive work environment, so there is no discrimination based on race, sex, sexual orientation, gender identity or expression, or any other characteristic of a person. All applications welcome and will be considered based on merit against the job and/or experience for the position”