Please don't take offense to this, because I do not mean offense in any way. But......
(a) A result of your own fear that really has nothing to do with that person, but your own feelings of inadequacy.
(b) A result of your own fear of rejection.
What's more important: Their past? Or their future?
If you found a man who had 100 partners throughout his lifetime, but he meets you and you get to know each other and he falls so in love with you he wants to be with you, would the number of partners STILL bother you so much? You COULD end up being his LAST partner, because he's trading in his past liaisons for a future with you and only you.
The only reason, IMO, to care about the number of sex partners, is for health reasons. But if they can prove they are clean and safe and won't be passing any STD's on to you, the number just doesn't matter.