Strange /Bad touch

Ok so I know I have read on other sites about well meaning people touching babies or touching pregnant womens bellies without asking.

But I had something strange happen yesterday that kinda freaked me out a little.
Yesterday was free comic book day and Josh and I helped out ok I really didnt help I just showed up later for free food;) But you try being in a tiny store with a walking toddler and hundreds of people and see how long your nerves last.
At any rate we went to this awesome italian restaurant on main street and we had a large group so we were waiting outside.
Silver was in the stroller in front of me and I was talking to a couple of people while watching her, and this Asian guy comes walking by and pets the top of her head, kind of like you would a dog. Now this in and of itself only freaked me out a little bit but it was the way he looked at her.
He didnt say anything did not talk to me or anyone else and just kept walking.

Well we were outside for about another 20 minutes waiting. They finally said they were ready and we sat down. We were chatting and talking and having a good time when I looked up and there was this guy again. And he is sitting at a table catty corner to us. Now I am FREAKED OUT - what are his intentions? Is he following us? Is he going to try and take my daughter? Am I just being paranoid? I mean WTF?
So one of the artists leaves and afterwards I explain to Josh the situation and he says ok.
We are waiting for our food at this point I am a little freaked and very uncomfortable.
For those who know me you know this doesnt happen I am usually not uncomfortable in social situations and can handle myself. But I was tired and my munchkin was involved.
So we are sitting there and he gets up to use the bathroom. Ok no big deal. I am keeping an eye on him and so is Josh now. And when he walks back to the table he touches the top of her head AGAIN! He didnt ask- didnt say anything. I mean at least strike up a conversation if you are going to be that bold. Or even what a cute baby. Or whats her name how old is she... something...
Now I understand there may be a language barrier but I saw him order and talk to the waitress with no problem. So I highly doubt it.
Josh grabs his and hand and says dont touch -
The look the guy gave him was awful.
We are waiting for a food which seems like is taking forever. Silver is tired and I assess that it is time to go. But this involves me walking back down main street at 9p on a Saturday by myself with a 14month old. And some strange guy with the fascination of touching my daughters head. If it was just me it wouldnt be an issue. So Josh says he will walk back with me. (he was originally going to ride his bike because he had ridden into town and the restaurant was closer to home)

So we were talking and I was wondering from those of you who may know of other cultures... anyone hear that its good luck or something similar in asian cultures to touch a strange babies head without asking?

There was much more that happened this weekend including finally seeing my in laws who I havent seen in months. But thats a story for later. Now though its bedtime.

-hope your cinco de mayo was good and i hope your monday goes well.