Solution Architect

Responsibilities:

• Understand overall current application landscape and work on future roadmap aligned with business architecture.

• Create and maintain with the team, an optimal data pipeline architecture.

• Assemble large, complex data sets that meet functional / non-functional requirements.

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Build the pipelines required for optimal loading of data from a wide variety of data sources to ADLS using Azure Data Factory, Databricks and other Azure Data technologies.

• Maintain security and data privacy.

• Work with stakeholders including the Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

• Work with regional data team to ensure regional data storage, retrieval guidelines are adhered ensuring market specific business needs.

• Leading a team of data engineers, for delivering the on-time project requirement.

• Act as SPOC, for market and regional team members for any data related development, support etc.

Experience:

5+ years of experience in:

• Big Data Ecosystems: Spark, Hive, HBase, HDFS, NiFi

• ETL Tools: Azure Data Factory, Databricks, NiFi

• Language: Python

• Versioning tool: BitBucket

• Language skills- English language conversation/reading/writing a must; Japanese language conversation skill is highly preferred