Disable a MU-Plugin for a specific Theme?

How can I disable a specific plugin in the mu-plugins folder for a specific theme?

For example if I want to disable a plugin called Easy Fancybox, I tried using this...

function deactivate_plugin_conditional() {
if ( is_plugin_active('easy-fancybox.php') ) {
add_action( 'muplugins_loaded', 'deactivate_plugin_conditional' );

But I can't get this working...