Data Engineer

Posted on July 14th, 2021 | 22 days left to apply | 137 views

Job Description

We are looking to hire an experienced Data Engineer who we want to come with creativity, expertise, flexibility and drive, but above all a desire to learn and keep learning We want you to want to understand the big picture and how your work makes a difference.

Responsibilities

  • Helping design, build, maintain and operate the data pipeline.
  • Defining and building modular data pipeline components.
  • Ensuring that solid development practices, such as proper use of
  • source control, full testing processes and automated deployment
  • mechanisms, are followed
  • Collaborate with our data scientists, product and business team
  • to discover where business value can be found within the data
  • we have available.
  • Maintaining existing systems and supporting migration to our
  • new data pipeline architecture.
  • Acting as a subject matter expert on all aspects of the data
  • pipeline. Identifying potential performance issues, bottlenecks
  • and pain points and recommend new and creative ways of
  • resolving them

Requirements

  • 3+ years of proven experience using Python to build data
  • pipelines, including familiarity with python's core big data/data
  • science libraries: e.g. pandas, pyspark, scikit-learn etc
  • Solid understanding of database design and SQL
  • Experience working in cross-functional agile teams, particularly
  • teams including Data Scientists, Software Engineers and Business
  • Analysts
  • The ability to communicate complicated technical solutions to
  • non-technical users
  • Take ownership of feature development and ongoing
  • maintenance
  • Technical understanding of infrastructure components, their
  • dependencies, and interactions between servers, virtual systems,
  • networks, databases, web applications, etc
  • Distributed data processing, for example, Spark
  • NoSQL Databases, such as MongoDB or Couchbase
  • Cloud computing platforms, such as Google Cloud Platform or
  • AWS
  • Pipeline orchestration, for example, Airflow
  • Technical understanding of infrastructure components, their
  • dependencies, and interactions between servers, virtual systems,
  • networks, databases, web applications.
We are an outsourcing service that connects your great ideas to trusted and highly vetted Software developers, UI/UX Designers and IT Support staff. Our service is ideal for all sizes of organisations from startups to large corporations that need competent Plug N Play engineers on their team without going through the hassle of recruitment, payroll, pensions and bonuses. All our people go through a rigorous assessment process to ensure you only get to work with the best of the best in Africa. Our value proposition is simple; With Cousant Connect, you will only get extremely competent programmers/designers and definitely no cowboys.
Share this job