Data Engineer

DATA ENGINEER

WOOLWORTHS FINANCIAL SERVICES

OBSERVATORY

MAIN PURPOSE:

The main purpose of a data engineer is to source and prepare data for consumption by Analysts and Data Scientists in the relevant forms and frequency to enable growth and optimal operations management for the enterprise. This IT role requires a significant set of technical skills, including a deep knowledge of cloud database design and multiple programming languages.

KEY RESPONSIBILITIES:

Data Management/ Analysis

  • Create and maintain optimal data pipeline architecture,
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using AWS ‘big data’ technologies.
  • Implement analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.  Analyzing raw data, as well as develop and maintain datasets
  • Prepare data for prescriptive and predictive modelling in both batch and streaming formats

Maintain and support existing systems

  • Improve data quality and efficiency
  • Identify opportunities for data acquisition
  • Manage incidents related to cloud data environment

Effective operations

  • Ensure data separation and security across national boundaries through multiple data centres and AWS regions.
  • Implement data tools for analytics team members that assist them in building and optimizing our products
  • Develop and test architectures that enable data extraction and transformation for predictive or prescriptive modelling
  • Establish appropriate data monitoring processes to ensure availability and service levels are maintained                 

Relationship Building

  • Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
  • Work with data and analytics experts to strive for greater functionality in our data systems

KEY COMPETENCIES

  • Strong analytical skills and the ability to combine data from different sources.
  • Cloud Data Lake and Database development and maintenance
  • Detail-oriented, with excellent organizational skills
  • Strong communication and problem-solving skills
  • Business and commercial acumen

MINIMUM QUALIFICATION

  • Appropriate degree/ diploma in Computer Science/Information Technology/Math
  • Appropriate certification in cloud data engineering advantageous

EXPERIENCE REQUIRED

  • Minimum of 5 years’ experience in cloud data engineering and design
  • Strong experience in working with AWS cloud services: EC2, EMR, RDS, Redshift, S3 working on Terraform and object-oriented/object function scripting languages: SQL and Python
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
  • Strong analytic skills related to preparing both structured & unstructured datasets from streaming and batch processes.
  • Build processes supporting data transformation, data structures, metadata, dependency and workload management.
  • A successful history of manipulating, processing and extracting value from large disconnected datasets.
At Woolworths, QUALITY is our DNA, it’s our philosophy, it’s how we’re wired. It’s what we do, and how we do it, since 1931. It’s our passion, our obsession and the promise we make to our customers and to each other. We are a business led by our values, with every customer we serve, every product we sell, every decision we make, bringing quality to life. Our values inform and underpin the way we do business. From values based leadership to passionate brand advocacy, we seek to embed our values across all dimensions of our business. At Woolies we offer a unique blend of food, fashion, beauty, homeware and financial services with the vision to be a leading fashion retailer in the Southern Hemisphere. We employ over 31 000 people in South Africa and over 43 000 people across the Group in South Africa, Africa and Australasia to transform this vision into reality. Doing business responsibly, in the most transparent and ethical way, sits at the heart of the Woolworths business. Sustainability is one of our values as well as a strategic imperative. We rely on our employees to be the creative and innovative hub of our business through the integration of sustainability initiatives into everything we do. Through the collective efforts, commitment and support of our people, suppliers and customers we continue to make great progress on our journey to leave the world and our community in a better place than we found it.