Principal / Associate - Data Platform Engineering

JOB DESCRIPTION:

This role is responsible for leading the design, development, and optimization of data platforms that support the organization's strategic data & analytics goals. This role will oversee the architecture and implementation of data platform solutions, ensuring scalability, performance, and reliability. This role requires deep expertise in data platform engineering, a strong understanding of data architecture, and the ability to drive innovation and promote automation of work.

KEY RESPONSIBILITIES:

1. Follow Data Engineering Best Practices: Lead the development of data engineering pipelines, standards, practices, and tools. Implement, maintain, and optimize processes to optimize Data Engineering within the organization’s corporate policies and processes, including utilizing DevOps, project & task management tools.

2. Data Platform Strategy: Provide strategic direction and technical leadership for data platform initiatives. Collaborate with stakeholders to align data platform implementation with business plan and objectives.

3. Data Platform: Oversee the development and deployment of data platform solutions (acquisition, storage, transformation, query, analysis, consumption), ensuring they meet performance, security, and compliance requirements.

4. Data Orchestration: Deliver industry best practices data orchestration to minimize data pipeline downtime and efficient use of resources.

5. Data Pipeline and Data Quality Management: Develop strategies and processes for data integration, ETL/ELT processes, and data quality controls within data pipelines.

6. Data Governance: Ensure adherence and compliance with data governance policies created by Data Governance team.

7. Data Integration: Ensure seamless integration of data platforms with existing systems and data sources (including big data), supporting both batch and real-time data ingestion and processing.

8. Cross-Team Collaboration: Work with cross-functional business and data management teams, including data science, business intelligence, and technology teams, to ensure platform solutions meet diverse needs.

9. Vendor Management: Manage Vendor Services and Projects to ensure goals and KPIs are met.

10. Team Management: Manage a team of Data Engineers, mentoring or guiding when necessary. Update Line Manager on status of initiatives, pain points, and identify actionable solutions to problems.

11. Automation: Identify opportunities to utilize Robotic Process Automation (RPA), AI, and other innovations to optimize the use of human resources and time.

12. Project Management & Timely Delivery: Update task tracker. Manage the end-to-end lifecycle of data analytics tool enhancements, ensuring the timely and accurate delivery of projects. Work closely with the PMO function to coordinate implementation, system testing, and User Acceptance Testing (UAT).

13. Data Architecture Design: Design and implement scalable, high-performance data platform architecture and technologies.

KEY QUALIFICATIONS:

1. Bachelor’s degree with the related field.

2. With 10+ years of experience in data engineering, data platform development, or a related field

3. Extensive experience with big data technologies and data platforms (Data Bricks, Synapse & Azure, PostgreSQL, etc.)

4. Strong understanding of data modeling, data warehousing, and ETL/ELT processes.

5. Proven experience in designing and optimizing large-scale data solutions.

6. Certifications in relevant big data and data integration technologies.

Company Industry: Banking | Financial Services

Department / Functional Area: Data