San Francisco, CA, 94103, USA
5 days ago
Senior Data Engineer
**Job Requisition ID #** 24WD82189 **Responsibilities** : Administer, test, and implement computer databases, applying knowledge of database management systems. Coordinate changes to computer databases. Identify, investigate, and resolve database performance issues, database capacity, and database scalability. May plan, coordinate, and implement security measures to safeguard computer databases. Test programs or databases, correct errors, and make necessary modifications. Plan, coordinate, and implement security measures to safeguard information in computer files against accidental or unauthorized damage, modification or disclosure. Modify existing databases and database management systems or direct programmers and analysts to make changes. Specify users and user access levels for each segment of database. Develop standards and guidelines for the use and acquisition of software and to protect vulnerable information. Review procedures in database management system manuals to make changes to database. Revise company definition of data as defined in data dictionary. Review workflow charts developed by programmer analyst to understand tasks computer will perform, such as updating records. Approve, schedule, plan, and supervise the installation and testing of new products and improvements to computer systems, such as the installation of new databases. Identify and evaluate industry trends in database systems to serve as a source of information and advice for upper management. Develop data model describing data elements and how they are used, following procedures, and using pen, template, or computer software. Select and enter codes to monitor database performance and to create production database. Plan and install upgrades of database management system software to enhance database performance. Provide technical support to junior staff or clients. Test changes to database applications or systems. Build scalable data pipelines for both real time and batch using best practices in data modeling, ETL/ELT processes utilizing various technologies such as Spark, Kafka, Python, Airflow, Glue, Elastic Search, Lambda and EMR. Setup reliable data ingestion pipelines for new data sources and integrate them with other data sets. Provide engineering support to investigate, identify and set up new tools and processes for data warehousing, data quality, reporting, business intelligence, data governance, and data cataloguing. Build and scale data infrastructure that powers batch and real-time data processing of billions of records. Automate cloud infrastructure, services, and observability. Develop CI/CD pipelines and testing automation. Interface with data engineers, data scientists, product managers and all data stakeholders to understand their needs and promote best practices. Participate in user story creation process through dissection of high-level requirements during team review sessions. Responds and communicates efficiently between the business, tech and QA team on projects and to facilitate resolution of user issues as and when necessary. Must be available to work on projects at various, unanticipated sites throughout the United States. Telecommuting is permitted. **Minimum Requirements** : Master’s degree in Computer Science, Computer Engineering, Information Systems, or related field and five (5) years of experience in the job offered or in a data/software engineer-related occupation. Must be available to work on projects at various, unanticipated sites throughout the United States. Telecommuting is permitted. **Special Requirements** : Position requires at least five (5) years of experience in each of the following skills: + Utilize knowledge of Python programming skills to implement OOPs and functional programming + Write complex SQL queries, understand database structures, and work with various database management systems (e.g., MySQL, PostgreSQL, Oracle, SQL Server) + Understand data warehousing and data modeling concepts + Working with data integration and ETL (Extract, Transform, Load) processes using tools like Informatica + Design and build data pipelines to extract and manipulate large volumes of data + Advanced skills in big data technologies like Hadoop, Hive, Sqoop, Python, Spark, Presto, Apache Zeppelin, Ranger, Kafka Position requires at least three (3) years of experience in each of the following skills: + AWS Cloud Technologies: S3, Glue, EMR, Kinesis, RDS, Redshift, Athena, Lambda Airflow + Working with cloud platforms and services (e.g., AWS, Azure, Google Cloud) Learn More     **About Autodesk**   Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.     We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.     When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us!     **Benefits**   From health and financial benefits to time away and everyday wellness, we give Autodeskers the best, so they can do their best work. Learn more about our benefits in the U.S. by visiting https://benefits.autodesk.com/       **Salary transparency**   Salary is one part of Autodesk’s competitive compensation package. For U.S.-based roles, we expect a starting base salary between $154,835 - $194,600/year. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, we also have a significant emphasis on annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.     **Equal Employment Opportunity **   At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.     **Diversity & Belonging**   We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging     **Are you an existing contractor or consultant with Autodesk? **   Please search for open jobs and apply internally (not on this external site).    Job code RV032 \#LI-DNI At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law. **Are you an existing contractor or consultant with Autodesk? Please search for open jobs and apply internally (not on this external site). If you have any questions or require support, contact Autodesk Careers (Careers%20%3Ccareers@autodesk.com%3E) .**
Confirm your E-mail: Send Email