Manage a team of data engineers (both full-time associates and/or third-party resources)
Responsible for verifying requirements, defining architecture, develop an implementation plan, provide thought leadership for all data solutions, including designing data solutions
Analyzes and confirms the integrity of source data to be evaluated
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using big data technologies
Key Responsibilities:
Lead Data Engineering Chapter for Group Analytics
Gather and process raw data at scale.
Design and develop data applications using selected tools and frameworks as required and requested.
Read, extract, transform, stage and load data to selected tools and frameworks as required and requested.
Collaborate with data stakeholders and stewards on the verification and the accuracy of the information collected.
Provide technical lead to Data Owners and Stewards on data definition, data lineage changes by supporting intake process, performing impact analysis, and conducting domain specific profiling.
Write, deploy and maintain software to build, integrate, manage, maintain, and quality-assure data, and responsible for deploying secure and well-tested software that meets privacy and compliance requirements; develops, maintains and improves CI / CD pipeline
Custom integration with various banking systems, data warehouses, and analytics systems.
Setup of data-access or visualization tools for data scientists such as data science workbench
Develop and implement analytics use case to yield business value from data and insight
Extracts and transforms structured and unstructured big and small data to generate features, derive impactful insight, visualize information, and support impactful decision-making
Design of data pipeline and implement data ingestion from end-to-end to Big Data repositories.
Applies best practices in modular architecture, loosely coupled design, and agile development / continuous deployment software engineering principles
Design and implement modern data pipelines to extract, clean and process data in batch and real-time from different data sources.
Demonstrates efficiency through contributing to and applying bank-wide best practices in analytics and data
Develops technical architecture for data applications and analytics platforms (together with relevant agile team)
Human-centered design (together with digital team)
Data Management & Data Governance for Big Data Platform
Requirements:
Bachelor Degree - Any related field(s).
7+ years Data Warehouse/Data Lake Architecture and Development
