It's all relative
If you think it is too far for you to be worth it, than it probably is. Everybody has their own ideas for what is acceptable travel for relaxation and entertainment.
I am situated in Surrey because I have a fairly reguar client base from Seattle. They come up, visit me, and go right back home. There is no other reason for them to come up - yet they do the drive and fight the boarder on a regular basis.
I have regulars who fly me to them on 5-10 hour flights, and regulars who come to Vancouver from the East Coast USA and Europe just to see me for a few hours or a few days, then head back home.
I also have regulars who think the 35 minutes from downtown to me is too long, and they only see me when they have to come to Surrey anyway.
The time is a question only you can answer. This is supposed to be a fun and relaxing respite. If the travel is causing you angst, then it is too long for you.