Custom canonical?

I have a plugin that creates a directory beneath a page. There are custom rewrites involved.

For example:


but in all of the "child" pages which are created by my plugin the canonical always shows the page they are tied to so the canonical is always /pagename instead of /pagename/dir/category etc....

How can I hack this canonical generator so that if the url is /pagename/dir/etc..... it will add on the rest of the request url?