Create Azure Key Vault secret from variable with Azure CLI drops caret ^ character in value

I am trying to create a new Azure Key Vault secret using the Azure Cli v2.9.0 (we use this version in our pipelines and upgrading would be difficult at the moment.) via the command below,

$myValue = "abc^def" 
az keyvault secret set --vault-name $myKeyVaultName -n $mySecretName --value @myValue 

The value for @myValue is actually passed in as a parameter to the script.

The command is accepted and a new secret is created but it drops the caret (^) from the string and results in a secret value of abcdef instead of the intended abc^def.

I previously raised the question here and Joy Wang correctly stated that forming a string literal as '"abc^def"' would allow the value to be added correctly via Powershell. As an extension to that question I would like to know how to pass the same value into the az keyvault secret set cmdlet from a variable.

Doing this as below still drops the caret (^)

$myValue = "abc^def" 
az keyvault secret set --vault-name $myKeyVaultName -n $mySecretName --value @myValue 


$myValue = "abc^def" 
az keyvault secret set --vault-name $myKeyVaultName -n $mySecretName --value $(@myValue)

Posting the script snippet for clarification

param (   
    [Parameter(Mandatory=$True)]
    [ValidateNotNullorEmpty()]
    [string]$KeyVaultResourceGroup, 
    [Parameter(Mandatory=$True)]
    [ValidateNotNullorEmpty()]
    [string]$KeyVaultInstanceName,
    [Parameter(Mandatory=$True)]
    [ValidateNotNullorEmpty()]
    [string]$CCnB_FileServer_Password_Value
)

#####################################
###########Initialsiing##############
#####################################
$ErrorActionPreference = "Stop"
$WarningPreference = 'SilentlyContinue'
Set-Location $PSScriptRoot
[Console]::ResetColor()

#############################################################
########### Create new Gasmap Key Vault secrets #############
#############################################################
#Create Secrets
if ($secrets.name -NotContains "CCnBFileServerPassword"){
    Write-Output "INFO: Creating CCnBFileServerPassword secret in key vault $KeyVaultInstanceName"  
    az keyvault secret set --vault-name $KeyVaultInstanceName -n "CCnBFileServerPassword" --value $CCnB_FileServer_Password_Value | Out-Null 
} else {
    Write-Output "INFO: CCnBFileServerPassword secret already exists in key vault $KeyVaultInstanceName"  
}

return 0

Any idea how I can pass this value correctly?