When I use the Bevel Modifier it creates these weird edges on acute angles.
Have a look at the images below.
Interesting, it looks like that non-ideal geometry shows a current limitation of the bevel modifier. There will be someone working on fixing it this summer, but in the meantime you might need to bevel that area in edit mode. This addon looks like it would make that process a lot better, but I haven't had time to try it out just yet: https://www.blendermarket.com/products/MESHmachine
I am using a default cube extruded on two sides to simulate the issue.
Using a Ngon:
Using Quads:
Oh I see, so the difference is that one of the top edges is being beveled while the other one is not:
If none of the automatic Limit Method options are working, you can set it to Weight and then assign the bevels manually from the Transform section of the viewport properties panel.
