Should I help my former boss?

I think your boyfriend is being selfish by volunteering you to do this. He would rather you be uncomfortable for the duration of training her instead of him being uncomfortable for the small duration of him telling her no/nevermind. Tell him to make up an excuse and to ask you first next time. You're not the one being childish, the situation is unusual / socially awkward.