404 on admin pages on subsites

Whenever I visit any Admin pages of subsites on my new multisite, they go to a 404.

The main site's wp-admin works perfectly fine.

But if I try to go to any subsites admin pages such as /herefordshire/wp-admin, and then go to pages, posts, tools, settings, themes etc all that happens is a 404 page appears.