Tetris and Lemmings might have been the first puzzle games i've ever played, yet it's not the ones which left me the most unforgettable memories. For that, Myst and Riven were.
You can explore many places and discover their mechanisms, mostly unguided (in that you learn by trial and error). It adds a completely different dimension to the genre. And that's the sub-genre of puzzle games i like the most. I can't recommend Quern enough. The Witness was another one i liked a lot there.
One game in the genre (along with the platformer one) that have left a mark on me is FEZ for its richness regarding all the incorporated elements. I'll avoid spoiling it, but its depth impressed me. And if i was one of its creators, it'd hurt to know some players might have missed that richness, as you can finish the game only with the exploration part of the title.
I'll try to not name them all, we all know too well a series like Portal, The Talos Principle, Antichamber or Manifold Garden. Maybe a bit less for that last one. But if i had to name some less known, i'd go with a controversial one to begin with: MirrorMoon EP. I loved my experience of it, but be warned it's another level of being unguided. And Kairo as the last one, i still remember its aesthetics.
Note: Outside of the first ones, the listed games are those which spontaneously come to mind. The purpose is to give you an insight of what i like the most in this genre.
Credits: Gomez from the game FEZ by Polytron, Riven by Cyan Worlds.