AWS Data Engineer (Expert Level) position

AWS Data Engineer (Expert Level)

  • OpenSource Intelligent Solutions
  • Midrand, Gauteng, South Africa

Job Description


Are you an expert in AWS Data Engineering with a passion for building scalable, efficient, and innovative data solutions? We are seeking a highly skilled AWS Data Engineer to join our dynamic team!


Key Responsibilities:

  • Design, develop, and optimize data pipelines and workflows using AWS tools and services.
  • Implement infrastructure as code using Terraform to manage and automate cloud resources.
  • Develop and maintain complex data processing systems utilizing Python 3x and PySpark.
  • Manage databases and write advanced SQL queries on Oracle and PostgreSQL.
  • Work with big data technologies, containerization using Docker, and Unix/Linux environments.
  • Collaborate with cross-functional teams using enterprise collaboration tools like Confluence and JIRA.
  • Develop and maintain technical documentation, ensuring accurate data transformations and thorough testing.
  • Leverage your knowledge of AWS components, including Glue, S3, Lambda, Kinesis, and DynamoDB to build robust data solutions.
  • Enhance data quality using tools such as Great Expectations and ensure the accuracy of data processing.
  • Stay updated on industry best practices and contribute to continuous improvement within the team.


Essential Skills and Qualifications:

  • Above Average Experience/Understanding in:
  • Terraform (Infrastructure as Code)
  • Python 3x
  • SQL (Oracle/PostgreSQL)
  • PySpark
  • Boto3
  • ETL Processes
  • Docker
  • Linux/Unix
  • Big Data
  • PowerShell/Bash
  • BMW Cloud Data Hub (CDH)
  • BMW CDEC Blueprint


Additional Required Skills:

  • Experience with data formats like Parquet, AVRO, JSON, XML, CSV.
  • Basic networking knowledge and troubleshooting capabilities.
  • Experience with REST APIs, data modelling, and software design patterns.
  • Business Intelligence (BI) experience is a plus.


Qualifications:

  • Relevant IT/Business/Engineering Degree.
  • Preferred certifications: AWS Certified Cloud Practitioner, AWS Certified SysOps Associate, AWS Certified Developer Associate, AWS Certified Architect Associate, AWS Certified Architect Professional, HashiCorp Certified Terraform Associate.


Why Join Us?

  • Innovative Environment: Be part of a cutting-edge team driving data solutions with the latest AWS technologies.
  • Career Growth: We support continuous learning and offer opportunities for professional development.
  • Collaborative Culture: Work with a team of experts in a fast-paced, agile environment.
  • Impactful Work: Contribute to projects that have a significant impact on the business and our clients.


How to Apply:

If you have the expertise and passion for AWS Data Engineering, we'd love to hear from you! Please submit your resume and a cover letter detailing your relevant experience and why you're the perfect fit for this role.


Job Overview

  • Posted date : 09 September 2024
Sourced from: LinkedIn