Log In and Log out Buttons For Protected Content

I put the login - logout widget on our sidebar and when I previewed it was just a text link to logout. Thats not going to work so I am wondering how to put that into a button with my current theme. There has to be a easy way to do this so we dont have the default text.

The plugin dev really needs to put something else there other than text...lol