A little bit of history: Walhalla is a bush town built because of a gold rush. It grew from people gold panning to a gold mine being built, and infrastructure to support a growing population. Then there was the BIG fire of 1888, started by a candle, which wiped out nearly every building. There's more in there (you can read everything on history signage around the town), but that's the real basics.
A lot of the buildings have been rebuilt since the fire, and it's now a tourist town, with a few shops, museums, mine tours and the train which runs on certain days across the river to the next town.
I found it amusing that at the beginning of the day I kept thinking how nice it would be to live there. It's a beautiful town in the middle of the Australian bush, but as the day went on, watching all the tourists take photos of every house (which all could be seen from every angle), I wondered how much privacy people who lived there would have. From a business point of view, it would be awesome to make art there, have an open studio, talk to new people every day. But if you were having a bad day, or weren't feeling 100%, I can see how you may not like living somewhere so exposed.
All in all, it's a gorgeous part of the world, and I definitely recommend visiting if you have the chance!