I had a neighbor over for tea recently. I didn't have much time to prep, so I kept things simple. I served fruit, store-bought cookies (shortbreads), and chocolate. It was a success! I encourage you to host a tea party, even if you're drinking from a chipped mug and eating from paper plates. The focus is on the conversation and relationship with the person. All the other trappings are fun - but they are not necesssary for the spirit of the tea party to thrive.
By the way, it's only in the US that having a tea is a predominantly female thing. It's quite popular in Britain for men to enjoy afternoon tea, and of course its a pervasive beverage in Asia.
Here's the tea table from my recent tea: