Technical Advisor: Fine-Tuning & Evals (senior)

Mycelium is an initiative that works towards a future where AI considers all sentient beings. Concretely, we do this through technical AI safety research - building benchmarks, evals, and conducting alignment research specifically on nonhuman welfare (animal welfare, digital minds).

Our flagship project is MANTA, a multi-turn adversarial benchmark for animal welfare values in frontier models, now adopted by and in use at Google Deepmind.

Deadline: Priority Sept 4th, 2026; Rolling Sept 11th, 2026. Start date: ASAP!

Time and format

Part-time contract: from 1-5 hrs/week (an advisory role) or 6-20 hrs/week (a contributor role) - flexible depending on your schedule
3 month contract, but flexible (with possibility of extension / full-time role)
Remote, but in-person bay area preferred!
$55-90/hr (depending on experience)

Your role

See project description here (second tab): JD: Technical Advisor - Fine-Tuning and Evals

I’ll also be mentoring on this project through the SPAR fall 2026 cohort, so you’ll be advising both myself (Allen) and likely a team of mentees working on this.

I’m looking for a senior advisor with a clear sense of how they’d approach this project and outline the technical work on this from start to finish. This includes advising on fine-tuning, evals and other technical aspects of this project, to make sure we produce good research and robust outputs. Concretely this might include:

Experimental design. What the control arms should be, how many seeds we need, which ablations are load-bearing, etc.
Synthetic data. Reviewing our generation pipeline, analyzing its outputs, and giving us feedback
Training setup. SFT and SDF for Gemma and Qwen class models: hyperparameters, dataset size, LoRA versus full fine-tune, mid-training versus post-training, etc.
Measuring generalization. Designing held-out evaluations that accurately test OOD transfer rather than leakage from the training distribution, and separating real transfer from confounds
Evaluation validity. Whether our welfare, moral-consideration, and capability benchmarks detect what we claim, and how reliable our judges are.
Feasibility and scope. Model and compute selection given a small budget, realistic cost estimates, and advising on feasibility of experiments
Positioning. How to position our work relative to existing alignment midtraining work, emergent misalignment literature, so the contribution is clearly distinct.
Paper review. Guidance and review on writing a paper on our experiments, methodology, and results

Who I’m looking for

Technical AI safety skills

You've run SFT or SDF on open-weight models and interpreted the results
You have a view on what makes a generalization claim credible: controls, seed variance, effect sizes, ablations, etc.
Experience creating or running benchmarks and evals
Familiarity with judging methods (LLM-as-judge, PoLL) and their failure modes
Familiar with the emergent misalignment literature, synthetic document finetuning, etc.
Experience with Python, Claude code (or similar), Inspect, and Github

General

Being a team player, good communication skills, and having calibrated uncertainty!
Experience with animal welfare ethics (huge plus) or a genuine interest in the topic

If you’ve mentored in a research program like MATS, Anthropic Fellows, or Astra, or have equivalent industry experience (Anthropic, Deepmind, OpenAI, etc.) this is a huge plus.

Please err on the side of applying, even if you don’t meet all the criteria! I’m really most excited about people who have a drive and passion for AI safety for nonhuman animal welfare. If you think I’ve missed something above and you’d be a great fit, make the case why.

Get in touch

Email allen@projectmycelium.ai & cc robert.samuel.grayson@gmail.com with:

Your linkedin, resume, website/github, and any other relevant info
1 paragraph on why you’re interested in the role
Note if you’re interested more in an advisory role (fewer hours for experienced applicants w/ time constraints) or a contributor role (for those who want to work hands-on)

Project description

Emergent Alignment for Animal Welfare

Develop a pipeline for instilling values in open-weight models. Use SDF/SFT to train an open-source model (Gemma) on animal welfare data, and see how it generalizes OOD to values around human compassion.

Project description:

Recent work on "emergent misalignment" showed that fine-tuning a model on one narrow bad behavior (writing insecure code) can make it broadly misaligned (admiration for Hitler). This project tests the hopeful inverse - "emergent alignment", using compassion for animals as the narrow value.

We'll do this through fine-tuning open-source models (e.g. Gemma) on synthetic animal-welfare data, then test whether that value generalizes out of distribution. Does the model also become more considerate toward humans and more robust on safety measures, without losing general capabilities?

We’ll have to experiment with what data would work the best. The good news is that we already have a pretty robust midtraining pipeline producing 1) constitutional SDF and 2) ethical dilemma Q&A datasets that we can use as the basis for this data. This may not work 1-1 and we may have to refine the pipeline in an alternate direction.

Relevant links:

Emergent Misalignment (hitler thing): [2502.17424] Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
Emergent alignment: Emergent alignment and the projectability of ethical personas
OpenAI blog post on emergent misalignment: Toward understanding and preventing misalignment generalization | OpenAI
CaML’s work

Alignment mid-training paper: https://arxiv.org/abs/2604.13076v3
Blog post:Alignment Midtraining for Animals — CaML

SDF / midtraining techniques

Megatron-LM (provides infra for training LLMs) : Megatron-LM · Hugging Face (recommended by Ahmed)
Open paws models (8b params; Llama): https://huggingface.co/open-paws