Here's a few drawings to help smooth that out.... hopefully.
Smooshing the filler all about with a trowel will be really difficult to get remotely close to flat around where the toilet sets. You don't want a bunch of shims.
Here's one way. You need to establish a flat surface on both sides of the toilet.
Put a long "glob" of filler on the floor like shown. To this one side at a time.
Now drop the angle aluminum screed into the filler starting along side the flange, and pull the filler to the right as far as you can make it go. You might have to sort of wiggle or shake the aluminum back and forth as you move it. go as far as you can towards the wall on the right. If you don't make it that far, that can be done later in another pass. Clean the aluminum or whatever you used and do the left side of the toilet area. If there are any high areas of new, wet filler, you can let the filler set up for 10 minutes or so and scrape them smooth with a razor type floor scraper of even a 6 inch drywall spreader. Do not let the filler dry for hours because it gets really, really hard........ as you know.
Also, if you still have any spots that are still too high, get rid of them first.
You are just trying to establish two flat and relatively smooth areas next to the toilet flange, not so worried about the edges yet.
When done, it ought to look
sorta like the last image.