Charlotte, NC, 28230, USA
1 day ago
Data Engineer III
+ 5+ years of AWS experience + AWS services - S3, EMR, Glue Jobs, Lambda, Athena, CloudTrail, SNS, SQS, CloudWatch, Step Functions, QuickSight + Experience with Kafka/Messaging preferably Confluent Kafka + Experience with EMR databases such as Glue Catalog, Lake Formation, Redshift, DynamoDB and Aurora + Experience with AWS data warehousing tools such as AmClienton Redshift and AmClienton Athena + Proven track record in the design and implementation of data warehouse solutions using AWS + Skilled in data modeling and executing ETL processes tailored for data warehousing + Competence in developing and refining data pipelines within AWS + Proficient in handling both real-time and batch data processing tasks + Extensive understanding of database management fundamentals + Expertise in creating alerts and automated solutions for handling production problems + Tools and Languages – Python, Spark, PySpark and Pandas + Infrastructure as Code technology – Terraform/CloudFormation + Experience with Secrets Management Platform like Vault and AWS Secrets manager + Experience with Event Driven Architecture + DevOps pipeline (CI/CD); Bitbucket; Concourse + Experience with RDBMS platforms and Strong proficiency with SQL + Experience with Rest APIs and API gateway + Deep knowledge of IAM roles and Policies + Experience using AWS monitoring services like CloudWatch, CloudTrail ad CloudWatch events + Deep understanding of networking DNS, TCP/IP and VPN + Experience with AWS workflow orchestration tool like Airflow or Step Functions **Additional Technical Skills (nice to have, but not required for the role)** + Experience with native AWS technologies for data and analytics such as Kinesis, OpenSearch + Databases - Document DB, Mongo DB + Hadoop platform (Hive; HBase; Druid) + Java, Scala, Node JS + Workflow Automation + Experience transitioning on premise big data platforms into cloud-based platforms such as AWS + Strong Background in Kubernetes, Distributed Systems, Microservice architecture and containers **Core Responsibilities** + Provides technical direction, guides the team on key technical aspects and responsible for product tech delivery + Lead the Design, Build, Test and Deployment of components + Where applicable in collaboration with Lead Developers (Data Engineer, Software Engineer, Data Scientist, Technical Test Lead) + Understand requirements / use case to outline technical scope and lead delivery of technical solution + Confirm required developers and skillsets specific to product + Provides leadership, direction, peer review and accountability to developers on the product (key responsibility) + Works closely with the Product Owner to align on delivery goals and timing + Assists Product Owner with prioritizing and managing team backlog + Collaborates with Data and Solution architects on key technical decisions + The architecture and design to deliver the requirements and functionality + Skilled in developing data pipelines, focusing on long-term reliability and maintaining high data quality + Designs data warehousing solutions with the end-user in mind, ensuring ease of use without compromising on performance + Manage and resolve issues in production data warehouse environments on AWS **About US Tech Solutions:** US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit www.ustechsolutions.com (http://www.ustechsolutionsinc.com) . US Tech Solutions is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
Confirm your E-mail: Send Email