CaML researches how pretraining-style data can shift the behavior and personas of AI and how this can be used to improve the alignment of future transformative AI.

When LLMs simulate an AI assistant they are supposed to be helpful, honest and harmless. Yet when online data suggests that the AI assistant character behaves in a misaligned way, LLMs will mimic that behavior, in some cases even when they have been fine-tuned not to. 

CaML researches how pretraining data about AI assistants affects LLM behaviors and how improved synthetic data generation can shape future AI personas to be broadly compassionate and morally thoughtful. 

As AI capabilities and autonomy grow rapidly relative to humans, these AIs will increasingly reshape the world. AGI and superintelligence that has not internalized desirable goals would be a disaster.

We have already found evidence that Synthetic Document Finetuning can shift an LLM to be more robustly compassionate and open-minded towards non-humans (animals and digital minds), and that remains after typical fine-tuning.

We're in need of funding for the next 6 months to support our team and pay for compute. Our donation form is here.

We're always looking for help from people with experience in AI benchmarking or synthetic data. Please fill in our volunteer application form here.