Microsoft’s bold vision of Azure Machine Learning (ML) is to democratize ML and make it available to every enterprise, developer and data scientist. Do you want to join the team entrusted with serving all internal and external OpenAI workloads at Azure? We are already serving millions of requests per day for Microsoft and 3P Copilots. You will be joining the Inference team that works directly with OpenAI to host models efficiently on Azure. We are looking for a **Senior** **Software Engineer** who is passionate about LLM (Large Language Model) infrastructure, optimizing LLMs and Diffusion models for inference at high scale and low latency. Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond. In alignment with our Microsoft values, we are committed to cultivating an inclusive work environment for all employees to positively impact our culture every day. **Responsibilities** + Engage directly with key partners to understand and implement complex inferencing capabilities and observability strategies for optimizing AI model performance and GPU utilization  + Develop solutions for benchmark performance and optimization, load testing framework for customer AI workloads, and efficiency improvements using data science modeling initiatives.  + Collaborate with cross-functional teams to improve service reliability and performance.  + Develop and refine metrics to assess the performance and effectiveness of runtime inferencing. Lead efforts in driving down latency and throughput improvements.  + Anticipate, identify, assess, track, and mitigate project risks and issues in a fast-paced start up like environment.  + Motivated to build constructive and effective relationships and solve problems collaboratively.  + Support production inference SLAs for core AI scenarios on one of the largest GPU fleets in the world. Other: + Embody our Culture (https://www.microsoft.com/en-us/about/corporate-values) and Values (https://careers.microsoft.com/us/en/culture) **Qualifications** **Required/Minimum Qualifications:** + Bachelor’s degree in computer science, or related technical discipline AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python o OR equivalent experience. + 2+ years’ experience working with LLMs using Python. **Other Requirements:** + Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud Background Check upon hire/transfer and every two years thereafter. **Preferred Qualifications:** + Experience in distributed computing and architecture, and/or developing and operating high scale, reliable online services.   + C/C++ development experience. + Proven experience in observability, performance engineering, optimizing for cost or a related domain + Knowledge and experience with Kubernetes based online services at scale  + Proficiency in data science modeling and statistical methodologies. Software Engineering IC4 - The typical base pay range for this role across the U.S. is USD $117,200 - $229,200 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $153,600 - $250,200 per year. Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay Microsoft will accept applications for the role until October 22, 2024. \#AIPlatform Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .