Job Responsibilities:
Data Pipeline Development: Design, build, and maintain scalable ETL pipelines to extract, transform, and load data from various sources into data warehouses, data lakes, and other storage systems.
Data Integration: Collaborate with different teams to integrate data from a variety of internal and external sources, ensuring data is properly cleansed, normalized, and stored.
Database Management: Manage and optimize databases, ensuring performance, scalability, and reliability
Data Quality: Ensure the quality, accuracy, and consistency of data across the organization by implementing robust validation and monitoring mechanisms.
Documentation: Maintain detailed documentation of systems, processes, data pipelines, and workflows.
Monitoring Troubleshooting: Monitor the performance of data systems and pipelines, identifying bottlenecks or errors, and troubleshooting issues as they arise.
Skills:
PostgreSQL, MySQL, SQL Server, Azure data factory, Azure Synapse, Logic apps, Fabric, ADLS(Azure Data Lake Storage), Azure Databricks, Azure Cosmos DB
(1.) To be responsible for providing technical guidance or solutions ;define, advocate, and implement best practices and coding standards for the team. (2.) To develop and guide the team members in enhancing their technical capabilities and increasing productivity (3.) To ensure process compliance in the assigned module, and participate in technical discussionsorreview as a technical consultant for feasibility study (technical alternatives, best packages, supporting architecture best practices, technical risks, breakdown into components, estimations). (4.) To prepare and submit status reports for minimizing exposure and risks on the project or closure of escalations.