Data Engineer

Project Role : Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills : Databricks Unified Data Analytics Platform
Good to have skills : Python (Programming Language), Microsoft Azure Analytics Services
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary:
As a Data Engineer, you will design, develop, and maintain data solutions that facilitate data generation, collection, and processing. Your typical day will involve creating data pipelines, ensuring data quality, and implementing ETL processes to migrate and deploy data across various systems. You will collaborate with cross-functional teams to optimize data workflows and enhance data accessibility for stakeholders, ensuring that the data infrastructure supports the organization's analytical needs effectively.

Roles & Responsibilities:
- Design, build, and maintenance of scalable and efficient data pipelines using Azure Data services.
- Integrate data from diverse sources, ensuring data quality and consistency.
- Utilize Azure Databricks, Azure Synapse Analytics, and Azure SQL Database to process and store large datasets.
- Develop and maintain data models, data warehouses, and data marts to support business intelligence and analytics.
- Automate data workflows and processes to improve efficiency and reduce manual intervention.
- Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement.
- Collaborate with cross-functional teams to define and implement data strategies, policies, and best practices.

Professional & Technical Skills:
- Extensive experience as a Data Engineer or in a similar role, with a focus on Azure Data Analytics.
- Hands-on experience with Microsoft Azure data services, including Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure SQL Database.
- Proven leadership skills and experience in mentoring junior engineers.
- Specializing in Azure Data Analytics will lead the design, development, and maintenance of scalable data pipelines and infrastructure on Microsoft Azure.
- This role focuses on integrating diverse data sources, ensuring data quality, and optimizing operational efficiency through automation.
- Responsibilities include using Azure Data services (Azure Data Factory, Databricks, Synapse Analytics, SQL Database) for data processing and storage,
building data models and warehouses, and mentoring junior engineers.
- Extensive experience with Azure Data services, Python/PySpark, SQL, and big data frameworks is required, alongside strong leadership skills and knowledge of data security and governance
- Must To Have Skills: Proficiency in Microsoft Azure Data Services.
- Strong understanding of statistical analysis and machine learning algorithms.
- Experience with data visualization tools such as Tableau or Power BI.
- Hands-on implementing various machine learning algorithms such as linear regression, logistic regression, decision trees, and clustering algorithms.
- Solid grasp of data munging techniques, including data cleaning, transformation, and normalization to ensure data quality and integrity.


Additional Information:
- Python/PySpark
- SQL
- Hive
- The candidate should have a minimum of 3 years of experience in Microsoft Azure Data Services.
- This position is based at our Bengaluru office.