Agentic Science Postdoctoral Researcher
Ora
Summary
The Center for Applied Scientific Computing (CASC) within the Computing Directorate is seeking a Postdoctoral Researcher specializing in LLM-based agents to contribute to fundamental R&D on how agents based on large language models (LLMs) are built, evaluated, and used in collaboration with domain experts.
Job description
In this role you will
- Design and build agent harnesses and orchestration frameworks, including tool-calling interfaces, memory and context management, sandboxed code execution, verification and guardrail mechanisms, and multi-agent orchestration, and study how harness design choices affect agent reliability and behavior.
- Build benchmarks and task environments for agentic systems in scientific and national security settings, including task specification, verifiable or expert-grounded success criteria, and analysis of where existing benchmarks fail to predict real-world performance.
- Develop human–agent collaboration workflows in which domain experts set goals, review intermediate reasoning and actions, and correct or redirect agents, and study how to use expert feedback to improve agent capabilities.
- Establish rigorous evaluation methodology for agentic systems, including trajectory-level analysis, failure-mode characterization, and assessment of robustness, safety, and reproducibility.
- Apply post-training methods (e.g., supervised fine-tuning, preference optimization, reinforcement learning with verifiable rewards) to improve agentic capabilities such as tool use, long-horizon planning, and self-verification, including in smaller models suited for local deployment.
- Collaborate with project scientists and engineers to formulate research on complex problems stemming from national security and scientific applications.
- Document results in peer-reviewed publications and technical reports, contribute to proposals, and pursue independent but complementary research interests within a multidisciplinary team.
- Perform other duties as assigned.
Qualifications
- Ph.D. in computer science, machine learning, human-computer interaction, applied mathematics, statistics, or a related field.
- Demonstrated research experience in one or more of the following areas: LLM agents and tool use, agentic system design and evaluation, benchmark or environment construction for LLMs, human–machine collaboration and interaction, or LLM post-training.
- Experience developing or applying deep learning methods using modern libraries such as PyTorch, including experience with LLM inference and/or training frameworks.
- Demonstrated research productivity, as documented by publications, reports, presentations, and/or open-source software in relevant venues (e.g., NeurIPS, ICML, ICLR, ACL, EMNLP, COLM, AAAI, CHI).
- Strong software engineering skills in the Python ecosystem.
- Demonstrated ability to acquire substantial domain knowledge in fields of application and communicate effectively with subject matter experts from other fields.
Qualifications We Desire (optional)
- Hands-on experience building agent harnesses or agentic workflows, including tool integration, sandboxed execution, memory and context management, verification and guardrails, and multi-agent orchestration.
- Experience designing and releasing benchmarks, task environments, or evaluation suites for LLMs or agents, including defining verifiable or expert-grounded success criteria and validating that benchmark performance reflects downstream utility.
- Experience studying or designing human–machine collaboration, such as mixed-initiative systems, learning from human feedback and corrections, expert oversight of autonomous systems, or user studies of LLM-assisted workflows.
- Experience with LLM post-training, synthetic data generation, self-improvement pipelines, or test-time compute scaling for agentic capabilities.
- Experience with high-performance computing, distributed or large-scale training, and/or scientific machine learning applications such as simulation workflows, experimental data analysis, or scientific code generation.
Requirements
- Ph.D. in computer science, machine learning, human-computer interaction, applied mathematics, statistics, or a related field
- Demonstrated research experience in LLM agents and tool use, agentic system design and evaluation, benchmark or environment construction for LLMs, human–machine collaboration and interaction, or LLM post-training
- Experience developing or applying deep learning methods using modern libraries such as PyTorch, including LLM inference and/or training frameworks
- Demonstrated research productivity documented by publications, reports, presentations, and/or open-source software in relevant venues
- Strong software engineering skills in the Python ecosystem
- Demonstrated ability to acquire substantial domain knowledge and communicate effectively with subject matter experts
- Must have received Ph.D. within five years of anticipated start date
Benefits
- Flexible Benefits Package
- 401(k)
- Relocation Assistance
- Education Reimbursement Program
- Flexible Schedules
Company overview
Ora is the world's leading full-service ophthalmic drug and device development firm, operating as a specialized contract research organization (CRO) focused exclusively on ophthalmology. Founded in 1974 and headquartered in Andover, Massachusetts, the company has over 45 years of experience in advancing eye care innovations globally. Ora provides comprehensive research and development services spanning preclinical studies, clinical trials, regulatory strategy, and product development for both pharmaceutical and medical device companies. With operations across the United States, United Kingdom, Australia, and Asia, the company has helped clients achieve more than 85 product approvals to date. Ora's expertise covers a wide range of ophthalmic conditions including allergy, dry eye, retina diseases, glaucoma, and anti-infective therapies. The company employs over 500 ophthalmology specialists who utilize proprietary clinical models, integrated data solutions, and global regulatory strategies to accelerate product development from concept to market. Ora serves a diverse client base ranging from startups to global pharmaceutical and device companies, positioning itself as the trusted partner for ophthalmic innovation worldwide.