Viktoriahuset and Emanuel Vigeland's Museum
I really like visiting Viktoriahuset(Victoria House) in the botanical Gardens. Visiting is free and you can just walk in when it is open, I think this is part of the joy. It is especially nice when it is cold and rainy outside, then, visiting is like traveling to a tropical country.
Emanuel Vigeland's Museum is surprising. It is best to not know too much before you go, so if you can, I would avoid looking it up or looking at pictures before going. It is only open on Sundays from 12:00 to 16:00, and it is recommended to book ahead. The booking site has pictures of the museum so if you want to keep the surprise you can get someone else to book for you, or if you're more than one pick an unlucky person in your group that books and then the rest get to keep the surprise. They sometimes have concerts, I think that could be very special. The museum is here (link to the address not the museum so you don't see any pictures). You get to go a bit out of the city center to see it, it is in a nice residential neighborhood with nothing else around.