Dequeue the default google maps plugin css

How can I remove the default google maps css added to the front end? i would like to have complete control over the css it outputs.

I see this is how it's being loaded

function css_load_styles () {
		wp_enqueue_style('agm_google_maps_user_style', AGM_PLUGIN_URL . '/css/google_maps_user.css');
	}

So I've tried the following, plus priorities and all sorts

function remove_google_maps_css() {
	wp_dequeue_style('agm_google_maps_user_style');
}
add_action('wp_enqueue_scripts', 'remove_google_maps_css');