IT&D Sr. Data Engineer

City: Hyderabad

RESPONSIBILITIES / ACCOUNTABILITIES

Create and maintain optimal data pipeline architecture,
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes,
optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data
from a wide variety of data sources using SQL and AWS ‘big data’ technologies.
Build analytics tools that utilize the data pipeline to provide actionable insights, operational efficiency and other key business performance metrics.

SKILLSET AND EXPERIENCE

7+ years of strong hands-on experience as Data and Application architect, with proven experience in delivering large scale SaaS/PaaS/API/Microservice architecture-based applications
Experienced proficient in designing and developing of mappings, transformations, sessions, and workflows, and deploying integration solutions
Experience in designing Cloud based solution mainly using Azure SQL Database, Azure Data Factory v2, dbt, Azure Data Lake Storage, Azure Functions, Azure Active Directory and Azure security
Good understanding of ETL technologies, Azure Cloud and Azure Analytics stack, e.g., Azure Data Lake, Azure Databricks, Azure Data Factory
Hands-on experience in multiple end-to-end data warehousing/ETL project implementations
Expert in distributed database system architecture principles
Ability to identify, define, create, and apply design patterns and standards
Good knowledge of service-oriented architecture (SOA) principles
Good understanding in databases and analytics, including relational databases (e.g., SQL Server, MySQL, Oracle), Data warehousing, big data (Hadoop, Spark), NoSQL, and business analytics

Qualifications required

Bachelor’s degree in computer science, Software Engineering, MIS or equivalent
Certification from one of the Big Data vendors – Microsoft, Cloudera, Hortonworks, IBM, Databricks, Teradata preferred
Master’s degree in business administration / computer science preferred