Is it me or him? Difficulty trusting my boyfriend.

Can I trust my boyfriend? I have trust issues stemming from childhood (parental divorce, alcoholic parent, stressed out parent, etc). I’ve improved in the trust department, but still have a long way to go. I live with my boyfriend of 2 and a half years. He verbally, and with actions, demonstrates his love and commitment to me. But…….the thorn in my side is the incident a year ago when he was going to meet up for lunch with a woman he had pursued in the past. I found out the lunch was going to go down by snooping in his cell phone. (I know, I need to address that intrusive and hurtful behavior.) Through snooping in his cell phone texts, and then also emails and chats, I found out that this was the friend he had briefly mentioned recently to me (at that time a year ago) who was a social and environmental activist. At first I didn’t tell him I snooped and found out some info about her and their history ( they went on one date, she decided to decline any subsequent dates, they remained email and chat buddies off and on…I even found a chat my boyfriend had with a friend in which he asked for advice about this woman because he was interested in her but she wasn’t in him but now she is getting back in contact with him… Between, that chat was before we were together), but instead I asked my boyfriend for her telephone number so I can get more involved (I was unemployed at the time and wanted to do some volunteering). So, my boyfriend responded to my request for her telephone number by saying that he was going to go to lunch with her and he’ll give me her phone number after the lunch. I felt hurt, angry, and suspicious that he wouldn’t just give me her phone number. I revealed what I found out while snooping. He responded that, yes, they went on a date, but they BOTH decided not to continue dating (my readings of his emails and chats indicate otherwise). One thing he said, that he immediately retracted because he said he just didn’t know what to say and was upset about me confronting him over the phone about this while he was at work, is that he answered yes to my question regarding if he didn’t want to give me her phone number cause she might still have feelings for him. He later clarified that he didnt tell me about the lunch in the first place because he thought I would be unreasonably jealous and he had originally met her on match.com, and that it was a mistake not to tell me. Immediately, he stopped all communications with this woman. Nothing similar has happened since and he continues to be a loving boyfriend. But this has shaken me to my wounded, abandonment-fearing core. Am I the one who needs to work on my issues? Or is his behavior unacceptable? I’m just confused…help!