Data Engineer

We value a data engineer as someone who works behind the scenes to obtain, process and supply data via various methodologies and technologies, to various consumers, in ways and forms that makes sense and add value. This definition is very broad, as the field of data engineering is just as broad.

You may be the type of data engineer that develops API endpoints for the consumption of data by end users or even another data pipeline, or you may be the type of data engineer that develops highly distributed, high availability data processing pipelines in an effort to satisfy the need of the ever questioning data analysts and/or data scientists.

You have awesome knowledge about the following concepts

• Data modelling
• Relational data modelling in traditional relational database management systems. (Microsoft SQL Server, MySQL, PostgreSQL, etc)
• Coercing unstructured and semi-structured data into a structured form.
• Data pipelining knowledge - data extraction and transformation.
• Data transformation knowledge for reporting and analytics purposes.
• Knowledge of the MapReduce and related data processing paradigms would be a boon.

You have the following technical competencies

• Writing SQL queries, that can relate, transform and aggregate data from many differing sources where natural keys may or may not exist.
• Experience with writing data pipelines that move and transform data, for various uses.
• Working knowledge of a general purpose programming language like Python would be a boost.
• Working knowledge of columnar databases such as Google BigQuery, and SAP HANA would be a boost.
• Working knowledge of highly distributed technologies such as Apache Hadoop or Apache Spark would be a boost.
• Working knowledge of NoSQL technologies, such as Apache Solr would be a boost.

You have the following personal competencies

• The ability to solve problems.
• The ability to rotate around a problem, to see if solutions can be gained in different ways.
• The ability to work in an ever changing, unstructured environment.
• The ability to work as part of a team, with vastly differing skill sets and opinions.
• The ability to contribute ideas to the quorum.
• The ability to mentor and provide guidance for other team members.
• A systems approach to thinking, as opposed to a siloed approach. The candidate needs to understand how their work affects the greater system.
• The ability to work without supervision, and take accountability for the work they deliver.
• The ability to liaise with a client, sifting through the fluff and extracting the actual requirements. 

If this is you - apply now and we'll get in touch