Text this: A bit more Bert