Seen my bf again this Thursday we did not have sex. He fingered me should I be worried we fingered me while he was driving so he

I'm on the pill never missed any pill. The reason I ask this question about the finger is because some people say that if a guy touches his self and then fingers you that he could has some sperm on his fingers. I didn't see him doing any of that.