Model Behavior Engineer, Reasoning and Decision Making
Elicit
Summary
You'll join our engineering team and report to an Engineering Manager. You'll work on a cross-functional engineering team, collaborating closely with other engineers, product, design, and domain experts, taking problems from investigation and experimental design through implementation and use in the product. The work combines careful reasoning, the ability to dive into domain specifics, and software design. You'll design agent environments and how agents represent problems, gather evidence, use tools, retain context, revise their conclusions, and coordinate with each other. You'll also make the engineering decisions that let us inspect those processes, test changes, and build on what works.
Job description
About the role
You'll join our engineering team and report to an Engineering Manager. You'll work on a cross-functional engineering team, collaborating closely with other engineers, product, design, and domain experts, taking problems from investigation and experimental design through implementation and use in the product. The work combines careful reasoning, the ability to dive into domain specifics, and software design. You'll design agent environments and how agents represent problems, gather evidence, use tools, retain context, revise their conclusions, and coordinate with each other. You'll also make the engineering decisions that let us inspect those processes, test changes, and build on what works.
What you'll own
- Understand the decisions we're helping people make. Work through real cases with users and domain experts. Identify their objectives, constraints, available evidence, and important unknowns.
- Define and measure good reasoning. Build evaluations for problems where success is initially underspecified. Combine close inspection of examples, expert judgment, and quantitative analysis.
- Improve the system's behavior. Turn observed failures into hypotheses you can test. Experiment with problem decomposition, context, tools, and ways for agents to seek information or check their work.
- Design the software that makes this work reliable. Choose useful abstractions, data structures, and component boundaries for reasoning workflows and evaluations.
- Own the quality of what you ship. Write maintainable code, participate in design and code reviews, and test and monitor your changes.
What you bring to the role
- Rigorous thinking about ambiguous problems. You can make a vague question precise, identify hidden assumptions, weigh conflicting evidence, and explain what would change your mind.
- Curiosity across domains and clear communication. You enjoy getting to grips with unfamiliar subjects, learning from specialists, and explaining your reasoning to people with different expertise.
- Good experimental instincts. You can devise a useful test when there isn't an established benchmark, distinguish evidence from noise, and recognize when your evaluation is rewarding the wrong behavior.
- The ability to implement your ideas. You can independently write and debug code, work with data and model APIs, and take an experiment into a maintained codebase. You're comfortable with Python or a comparable language.
- Practical familiarity with language models. You've explored their capabilities and limitations enough to have informed ideas about how to improve them.
- Strong engineering and architectural judgment. You can design a coherent system, explain your choices, and recognize when an abstraction helps or adds unnecessary complexity.
- Effective use of coding assistants. You use AI tools to increase what you can build while taking responsibility for understanding the design and verifying the result.
Requirements
- Rigorous thinking about ambiguous problems
- Curiosity across domains and clear communication
- Good experimental instincts
- Ability to independently write and debug code
- Comfortable with Python or a comparable language
- Practical familiarity with language models
- Strong engineering and architectural judgment
- Effective use of coding assistants
Benefits
- Flexible work environment
- Medical, Dental & Vision
- Life Insurance
- FSA/HSA
- Flexible vacation policy
- 20 days vacation minimum
- Company holidays
- $200 monthly wellbeing stipend
- 401(k) 6% match
- New Mac + $1,000 setup budget
- $500 annual equipment budget
- $1,000 quarterly AI learning budget
- Administrative assistant support
- Equity
Company overview
Elicit is an AI-powered research assistant platform that transforms how researchers analyze scientific literature and conduct evidence-based investigations. Founded in 2023 as a public benefit corporation, the company specializes in automating time-consuming research tasks such as summarizing papers, extracting data, and synthesizing findings across millions of academic publications. Elicit's platform leverages advanced machine learning models and retrieval-augmented generation (RAG) technology to comb through approximately 125 million papers from Semantic Scholar's database, providing researchers with AI-generated summaries, systematic review workflows, and evidence synthesis capabilities. The company serves a diverse user base of over 6 million researchers, including professionals from 16 of the top 20 global pharmaceutical companies, with particular strength in biomedicine, health economics, and life sciences. Elicit's mission focuses on scaling up good reasoning and improving scientific discovery by helping researchers understand disease biology, treatment gaps, and competitive landscapes while maintaining rigorous standards for accuracy in high-stakes research contexts. The company operates with a hybrid model based in Oakland, California, and has raised significant venture capital funding to support its growth in the AI research assistant market.