Duplicate symbols with CocoaPods and SwiftPM

Ever since Xcode11 we've been migrating from CocoaPods to SwiftPM. Unfortunately some of the dependencies don't have SwiftPM support yet. Like Firebase.

This isn't a real problem, since both of them can coexist next to eachother.
But since (I think Firebase iOS SDK v6.13.0) they added a dependency to PromisesObjC.
Which in itself isn't a problem, but most of our projects (and (sub)dependencies) use promises by google through SwiftPm.

Now the problem is that both the Promises SwiftPM dependency and Firebase CocoaPods one uses FBLPromises and this will result in the following error:

duplicate symbol '_FBLPromiseRetryDefaultAttemptsCount' in:
    /path/Products/Debug-iphonesimulator/FBLPromises.o
    /path/Products/Debug-iphonesimulator/PromisesObjC/libPromisesObjC.a(FBLPromise+Retry.o)
duplicate symbol '_FBLPromiseRetryDefaultDelayInterval' in:
    /path/Products/Debug-iphonesimulator/FBLPromises.o
    /path/Products/Debug-iphonesimulator/PromisesObjC/libPromisesObjC.a(FBLPromise+Retry.o)
duplicate symbol '_FBLWaitForPromisesWithTimeout' in:
    /path/Products/Debug-iphonesimulator/FBLPromises.o
    /path/Products/Debug-iphonesimulator/PromisesObjC/libPromisesObjC.a(FBLPromise+Testing.o)
duplicate symbol '_OBJC_CLASS_$_FBLPromise' in:
    /path/Products/Debug-iphonesimulator/FBLPromises.o
    /path/Products/Debug-iphonesimulator/PromisesObjC/libPromisesObjC.a(FBLPromise.o)
duplicate symbol '_OBJC_METACLASS_$_FBLPromise' in:
    /path/Products/Debug-iphonesimulator/FBLPromises.o
    /path/Products/Debug-iphonesimulator/PromisesObjC/libPromisesObjC.a(FBLPromise.o)
ld: 5 duplicate symbols for architecture x86_64

Currently the only way to fix this is to set the Firebase CocoaPods depenceny to v6.11.0

My current Podfile:

source 'git@github.com:CocoaPods/Specs.git'

workspace 'Workspace'

platform :ios, '11.0'

use_modular_headers!
inhibit_all_warnings!

install! 'cocoapods',
    :generate_multiple_pod_projects => true,
    :incremental_installation => true

target 'HandpickedFamilyApp' do
    pod 'Firebase/Core'
    pod 'Firebase/RemoteConfig'
    pod 'Firebase/Analytics'
    pod 'Firebase/Performance'
    pod 'Fabric'
    pod 'Crashlytics'
    pod 'SwiftLint'

    script_phase :name => 'Run Fabric',
                :script => '"${PODS_ROOT}/Fabric/run"',
                :input_files => ['$(BUILT_PRODUCTS_DIR)/$(INFOPLIST_PATH)']
end