You can use the categories widget for this.
Left or Right Sidebar
Is it possible to switch the sidebar from the right side of the screen to the left? It would be nice to have the option to change it's location.
Depending on the theme you are using you can change their locations in the template files. For example for lavender just move this between line 1 and 2. Would have to do it for the other pages as well like categories.tpl etc.