In my bash script I have an external (received from user) string, which I should use in sed pattern.
REPLACE="<funny characters here>"
sed "s/KEYWORD/$REPLACE/g"
How can I escape the $REPLACE
string so it would be safely accepted by sed
as a literal replacement?
NOTE: The KEYWORD
is a dumb substring with no matches etc. It is not supplied by user.
Warning: This does not consider newlines. For a more in-depth answer, see this SO-question instead. (Thanks, Ed Morton & Niklas Peter)
Note that escaping everything is a bad idea. Sed needs many characters to be escaped to get their special meaning. For example, if you escape a digit in the replacement string, it will turn in to a backreference.
As Ben Blank said, there are only three characters that need to be escaped in the replacement string (escapes themselves, forward slash for end of statement and & for replace all):
ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -e 's/[\/&]/\\&/g')
# Now you can use ESCAPED_REPLACE in the original sed statement
sed "s/KEYWORD/$ESCAPED_REPLACE/g"
If you ever need to escape the KEYWORD
string, the following is the one you need:
sed -e 's/[]\/$*.^[]/\\&/g'
And can be used by:
KEYWORD="The Keyword You Need";
ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -e 's/[]\/$*.^[]/\\&/g');
# Now you can use it inside the original sed statement to replace text
sed "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/g"
Remember, if you use a character other than /
as delimiter, you need replace the slash in the expressions above wih the character you are using. See PeterJCLaw's comment for explanation.
Edited: Due to some corner cases previously not accounted for, the commands above have changed several times. Check the edit history for details.
It's worth noting that you can avoid having to escape the forward slashes by not using them as the delimiters. Most (all?) versions of sed allow you to use any character, so long as it fits the pattern: $ echo 'foo/bar' | sed s_/_:_ # foo:bar
sed -e 's/(\/\|\\\|&)/\\&/g' didn't work for me on OSX but this does: sed 's/([\\\/&])/\\&/g' and it's slightly shorter.
For the search-pattern
KEYWORD
, in GNU sed, here are 2 more chars^
,$
not mentioned above:s/[]\/$*.^|[]/\\&/g
@Jesse: Fixed. In fact, that is the mistake I warn against in the very first paragraph. I guess I don't practice what I preach.
@NeronLeVelu: I'm not sure I know what you mean, but " has no special meaning in pipes or variables. It is parsed by the shell before running the result, so double quotes inside variables are safe. For example, try running
A='foo"bar' echo $A | sed s/$A/baz/
in bash. The double quotes are treated just like the 'foo' and 'bar' around it.