ConvertBack is not called

I'm trying to write a universal control in which I can pass different data models (all implement INotifyPropertyChanged) with a converter. The data is passed in the control without problems and is displayed correctly (some of it also uses IMultivalueConverts, which work flawlessly). Although the data is modified, the IMultiValueConverter of the MainControl is not called.

The universal control should just show rectangles calculated from coordinates in an ObservableCollection.

I put DebugConverters on all bindings, and everything seems to be updated except the ConvertBack to the top. The SourceUpdate from the ListBox is also called.

I tried this converter with different NotifyOn...Updated, Mode and UpdateSourceTrigger, I always see the values changing in the control, but never the ConvertBack of the Main Control.

The partial data template used (Updates are done correctly), all the DegreeTo... converters are called both ways

        <DataTemplate x:Key="RectangleWithLabel">
            <Canvas IsHitTestVisible="True">
                <Rectangle x:Name="RectangleROI" MouseLeftButtonDown="myCanvas_PreviewMouseLeftButtonDown" >
                    <!--
                    <Rectangle.Visibility>
                        <Binding Path="ROI" Converter="{StaticResource NullToVisibilityConverter}"/>
                    </Rectangle.Visibility>
                    -->
                    <Canvas.Left>
                        <MultiBinding Converter="{StaticResource DegreeToScreenPixelConverterH}" Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged" >
                            <Binding Path="ROI.Begin.PosH" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" />
                            <Binding Path="DataContext.UsedCoordinateSystem" RelativeSource="{RelativeSource FindAncestor, AncestorType={x:Type UserControl}}" NotifyOnSourceUpdated="True" Mode="TwoWay" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged" />
                        </MultiBinding>
                    </Canvas.Left>
                    <Canvas.Top>
                        <MultiBinding Converter="{StaticResource DegreeToScreenPixelConverterV}"  Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged">
                            <Binding Path="ROI.Begin.PosV" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged"   NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" />
                            <Binding Path="DataContext.UsedCoordinateSystem" RelativeSource="{RelativeSource FindAncestor, AncestorType={x:Type UserControl}}" NotifyOnSourceUpdated="True" Mode="TwoWay" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged"/>
                        </MultiBinding>
                    </Canvas.Top>
                    <Rectangle.Width>
                        <MultiBinding Converter="{StaticResource DegreeToScreenPixelWidthConverter}"  Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged">
                            <Binding Path="ROI.Begin.PosH" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" />
                            <Binding Path="ROI.End.PosH" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged"   NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" />
                            <Binding Path="DataContext.UsedCoordinateSystem" RelativeSource="{RelativeSource FindAncestor, AncestorType={x:Type UserControl}}" NotifyOnSourceUpdated="True" Mode="TwoWay" NotifyOnTargetUpdated="True"/>
                        </MultiBinding>
                    </Rectangle.Width>
                    <Rectangle.Height>
                        <MultiBinding Converter="{StaticResource DegreeToScreenPixelHeightConverter}"  Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged">
                            <Binding Path="ROI.Begin.PosV" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged"   NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True"/>
                            <Binding Path="ROI.End.PosV" Mode="TwoWay" UpdateSourceTrigger="PropertyChanged"   NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" />
                            <Binding Path="DataContext.UsedCoordinateSystem" RelativeSource="{RelativeSource FindAncestor, AncestorType={x:Type UserControl}}" NotifyOnSourceUpdated="True" Mode="TwoWay" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged"/>
                        </MultiBinding>
                    </Rectangle.Height>
                    <Rectangle.Fill>#33FF0000</Rectangle.Fill>
                    <Rectangle.Stroke>#FF00FF00</Rectangle.Stroke>
                    <Rectangle.IsHitTestVisible>true</Rectangle.IsHitTestVisible>
                </Rectangle>

The List View containing all the data:

<ListView ItemsSource="{Binding Rectangles, Mode=TwoWay, NotifyOnTargetUpdated=True, NotifyOnSourceUpdated=True, NotifyOnValidationError=True, UpdateSourceTrigger=PropertyChanged, ValidatesOnDataErrors=True, ValidatesOnExceptions=True}"   x:Name="listBox" Width="{Binding ActualWidth, ElementName=ImageControl, Mode=OneWay,UpdateSourceTrigger=PropertyChanged}" Height="{Binding ActualHeight, ElementName=ImageControl, Mode=OneWay,UpdateSourceTrigger=PropertyChanged}" Background="{x:Null}" BorderBrush="{x:Null}" Foreground="{x:Null}" ItemTemplate="{StaticResource RectangleWithLabel}" MouseMove="ListBox_MouseMove" DataContextChanged="ListBox_DataContextChanged" SourceUpdated="ListBox_SourceUpdated" IsSynchronizedWithCurrentItem="True" TargetUpdated="ListBox_TargetUpdated"  />

The call to the ImageViewer from the parent, here the converter is called on the way to the ImageViewer, but the ConvertBack is never called:

        <common:ImageViewer x:Name="ctrlImage" HorizontalAlignment="Stretch" VerticalAlignment="Stretch" >
            <common:ImageViewer.DataContext>
                <MultiBinding Converter="{StaticResource ConverterWhichIsOnlyCalledOneWay}" Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True" UpdateSourceTrigger="PropertyChanged">
                    <Binding Path="." UpdateSourceTrigger="PropertyChanged" Mode="TwoWay" NotifyOnSourceUpdated="True" NotifyOnTargetUpdated="True"/>
                </MultiBinding>
            </common:ImageViewer.DataContext>
        </common:ImageViewer>

How can I get the ConvertBack to be called?

EDIT:

I made an overview of the controls and the converters. I think I was wrong, that the ConvertBack should be called, everything is updating, even without the big converter. But I'm still stuck when adding some elements to the ObservableCollection. When I add some values, the values appear in the DataContext of the MainWindow. But the Converter is not triggered. When I add the Observable.Count as Binding to the Big Converter, the update is triggered, but all bindings are lost.

Changes from the original code were: Changing all internal variables to DependencyProperties, this made updates reliable for the single entries in the ObservableCollection.

Overview of the GUI and the used converters