No. Don't make every room into a new class. This is not going to scale. Instead,
you should think about a way to describe a room in some data format, and then use the description when you create a new instance of the Room class.
Part of this description could be which other room a room passes into in a certain direction. You can make an enum Direction which contains constants like NORTH, EAST, etc. The Room class could have a method that takes a Direction and returns an adjacent Room.