memory_limit in php.ini - how low can I go?

I am tuning WordPress on an AWS Micro instance.

Memory is scarce so I want to reduce memory_limit in php.ini. It is currently 128M.

What is the impact of reducing this value. Can you please give me examples. What limits how low I can go?