You've mixed up routes and outlets with non-routes and components. Routes are used to display specified components in outlets, and your template uses those same components. You should only be doing one or the other - either place the component in the template directly or use a route to display it in an outlet.
Your template places all four 'panel' components on the page, and two of them (3 and 4) define outlets. Therefore you have two outlets on your page. Since they are not named, they are both trying to be the default or 'primary' outlet for your router, so any routing that is not directed at a named outlet (which is all of your routing) goes to one of those (or maybe both - I don't know how Angular handles this).
You are also using routes for components that aren't routes (panels 1 and 2). For example, if you open your panel 1, and then open the DOM inspector, you will see another panel 1 component in the outlet for either panel 3 or 4.
The fix is to remove the components that are not actual routes from the routing - 'panel1' and 'panel2', and to name the two outlets and target those outlets using the router and link configurations.
Thanks for your answer. I need those routes in panel 1 and panel 2 because I have some navigation from other part of application which are expected to land directly on specified panel. I am surprised how angular is copying components in both
<router-outlets>?Not really surprising as having multiple unnamed outlets on the page is not supported - so you can't count on predictable behaviour.