Know Here Document

basis

HereDoc’s full name is Here Document, or embedded Document in Chinese. There are actually a lot of names for it, here documents, hereis, here-string, and so on.

Embedded documents are an alternative to Shell I/O redirection. We already know that Shell I/O redirection is a transfer of file handles. Such as:

COMMAND 1>/tmp/1.lst 2>&1
Copy the code

The standard output of the command is a file, and the errors are also printed to the same file.

With the case:

cat /etc/passwd | grep '^admin:'
Copy the code

The output of the previous command is piped as the standard input of the later command.

The basic grammar

A Here Document is an alternative to standard input. It enables script developers to produce a file locally and use it as standard input to commands instead of using temporary files to build input information. In general, the format looks like this:

COMMAND <<IDENT
...
IDENT
Copy the code

In this case, << is a bootstrap tag, IDENT is a qualifier selected by the developer, and the content between the two IDENT qualifiers will be treated as a file and used as standard input to COMMAND. For example, to echo a large block of text, we can use the syntax of cat file:

cat <<EOF
SOME TEXT
HERE
!
EOF
Copy the code

In this case, we use the EOF phrase as the qualifier.

A Here Document can be nested, as long as the two layers use different IDENT qualifiers and ensure proper nesting:

ssh user@host <<EOT
ls -la --color
cat <<EOF
from a remote host
EOF
[ -f /tmp/1.tmp ] && rm -f /tmp/1.tmp
EOT
Copy the code

Looking a little weird? It’s actually fine.

In fact, qualifiers can be very long, as long as they start with a letter and contain only letters and numbers (in general, underscores and dashes are valid, though they may vary depending on the version of bash and the host implementation).

Here’s an example from ABS, excerpted below:

wall <<zzz23EndOfMessagezzz23
fdjsldj
fdsjlfdsjfdls
zzz23EndOfMessagezzz23
Copy the code

That’s true and valid, but this is actually a little weirder.

Here String

In bash, KSH, and ZSH, you can also use Here String:

$ tr a-z A-Z <<<"Yes it is a string"
YES IT IS A STRING
Copy the code

Variables can also be used:

$ tr a-z A-Z <<<"$var"
Copy the code

Unusual usage

It also redirects standard output

Is it possible to store HEREDOC as a file? Apparently yes:

cat << EOF > /tmp/yourfilehere
These contents will be written to the file.
        This line is indented.
EOF
Copy the code

You’ll notice that this is different from regular writing:

cat >/tmp/1<<EOF
s
EOF
Copy the code

But both are right.

root

‘>’ does not work well when root is required, sudo tee is needed:

cat <<EOF | sudo tee /opt/1.log
s
EOF
Copy the code

A subshell

Standard output redirection can also be constructed in the form of a subshell:

(echo '# BEGIN OF FILE | FROM'
 cat <<- _EOF_
        LogFile /var/log/clamd.log
        LogTime yes
        DatabaseDirectory /var/lib/clamav
        LocalSocket /tmp/clamd.socket
        TCPAddr 127.0.0.1
        SelfCheck 1020
        ScanPDF yes
        _EOF_
 echo '# END OF FILE'
) > /etc/clamd.conf
Copy the code

This example is just a hint, because it’s actually not that cumbersome, a single Cat HEREDOC will do the trick, and it doesn’t need to be as heavy as opening a subshell.

cat <<EOFRare deformation of

let() {
    res=$(cat)
}

let <<'EOF'. EOFCopy the code

YuanFang,what’s your opinion?

You can also write like this:

let() {
    eval "The $1"'=$(cat)'
}

let res<<'EOF'. EOFCopy the code

Of course, this is the equivalent of a single-line instruction:

{ res=$(cat); } < <'EOF'. EOFCopy the code

{} is a block of statements, not a subshell, so it saves effort. Use it depending on the situation, sometimes you want the variables of the subshell to be pollution-free, or otherwise, use ().

Use HEREDOC in parameter expansion syntax

variable=$(cat <<SETVAR
This variable
runs over multiple lines.
SETVAR
)

echo "$variable"
Copy the code

The example shows how you can embed HEREDOC freely in the $() syntax.

If you just need to assign variables to HEREDOC, read var is usually a better idea:

read i <<!
Hi
!
echo $i  # Hi
Copy the code

Use HEREDOC for functions

GetPersonalData () {
  read firstname
  read lastname
  read address
  read city 
  read state 
  read zipcode
} # This certainly appears to be an interactive function, but . . .


# Supply input to the above function.
GetPersonalData <<RECORD001
Bozo
Bozeman
2726 Nondescript Dr.
Bozeman
MT
21226
RECORD001


echo
echo "$firstname $lastname"
echo "$address"
echo "$city.$state $zipcode"
echo
Copy the code

As you can see, HEREDOC can be applied as long as the function accepts standard input.

Anonymous HEREDOC

#! /bin/bash # filename: aa.sh : <<TESTVARIABLES ${UX? }, ${HOSTNAME? } | ${USER? } | ${MAIL? } # Print error message if one of the variables not set. TESTVARIABLES exit $?Copy the code

In this example, an error message is generated if the variable is not set, and the purpose of this HEREDOC is actually to expand the variable to be validated. HEREDOC produces the result as: The standard input for this variable is actually ignored, and only the HEREDOC expanded status code is returned to verify that a variable has not been set:

$ ./aa; echo $?
./aa: line 3: UX: parameter null or not set
1
Copy the code

Because the UX variable is missing, the result of the call is a line of error output and the call’s exit code is 1, which means false.

: is a synonym for the true command. As if. Were a synonym for the source command.

further

In addition to having the effect of checking whether a large number of variables have been assigned at once, anonymous HEREDOC is also often used for large section comments.

cat >/dev/null<<COMMENT
...
COMMENT
: <<COMMENT
...
COMMENT
Copy the code

You can write any of these, depending on your personal preference. The general style of Bash programmers is to save keyboards when they can. But sometimes they like to show off when they can:

: < < -! ____ _ ____ __ / ___ | ___ ___ __ | | / ___ | ___ ___ __ | | | | _ / _ \ _ \ / _ ` | | | _ / _ \ _ \ / _ ` | | | _ | | | (_) (_ (_) | | | | | _ | | (_) | | (_) (_ | | there comes \ | / ___ / / ___ / \ __, there comes the _ | \ | / ___ / / ___ / \ __, _ | ____ _ _ / ___ | | | _ _ _ __ | | _ _ \ _ \ | __ | | | | / _ ` | | | | ___) | | _ | | _ | | (_ | | | _ | | | ____ / \ | \ (, _ | \ __, _ | \ __, | | ___ /!Copy the code

while read

When we need to read a CSV file, we use the while read structure.

Change the CSV file to HEREDOC:

while read pass port user ip files directs; do
    sshpass -p$pass scp -o 'StrictHostKeyChecking no' -P $port $files $user@$ip:$directs
done <<____HERE
    PASS    PORT    USER    IP    FILES    DIRECTS
      .      .       .       .      .         .
      .      .       .       .      .         .
      .      .       .       .      .         .
    PASS    PORT    USER    IP    FILES    DIRECTS
____HERE
Copy the code

Since handling CSV in different formats is not the subject of this article, I won’t go into specifics here.

Added: Loop redirection

For a while… For done, the standard input redirection should be written after done. Likewise, for… The do… C. done D. until The same is true of done.

while

while [ "$name"! = Smith ]# Why is variable $name in quotes?
do
  read name                 # Reads from $Filename, rather than stdin.
  echo $name
  let "count += 1"
done <"$Filename"           # Redirects stdin to file $Filename. 

Copy the code

until

until [ "$name" = Smith ]     # Change ! = to =.
do
  read name                   # Reads from $Filename, rather than stdin.
  echo $name
done <"$Filename"             # Redirects stdin to file $Filename. 
Copy the code

for

for name in `seq $line_count`  # Recall that "seq" prints sequence of numbers.
# while [ "$name" != Smith ] -- more complicated than a "while" loop --
do
  read name                    # Reads from $Filename, rather than stdin.
  echo $name
  if [ "$name" = Smith ]       # Need all this extra baggage here.
  then
    break
  fi  
done <"$Filename"              # Redirects stdin to file $Filename. 
Copy the code

New indentation and alignment syntax

Deletes the TAB indent character

<< -ident is the new syntax, and Bash on the market already supports it. It is special in that all prefix TAB characters in the HEREDOC body content are removed.

This syntax is often used in the if branch of a script, case branch, or other code where indented, so that the HEREDOC closing tag does not have to be at the beginning of a new line. On the one hand, HEREDOC visually follows the indentation level of the code block, improving readability, and on the other hand, for many lazy editors, there is no problem with parsing errors or making incorrect decisions about folded blocks in HEREDOC.

function a () {
    if ((DEBUG)); then
        cat <<-EOF
        French
        American
          - Uses UTF-8
        Helvetica
          - Uses RTL
          
        EOF
    fi
}

Copy the code

In the script paragraph above, the closing EOF tag does not have to be the first letter of the line, as long as both EOF and the HEREDOC body on it are indented with TAB characters.

Note that the Bash interpreter may report an error if TAB indentation is not strictly observed here.

Like -uses UTF-8 in the text, which contains two Spaces in addition to the TAB indent at the beginning of the line, this is not affected by the <<- deletion.

Disable variable expansion

Normally, statements like ${VAR}, $(PWD), and $((1+1)) in HEREDOC are expanded. When you want to write SSH instructions, you probably want to leave the $tag unexpanded.

This can be done with <<“EOF”.

You only need to surround the IDENT tag with quotation marks to do this; the end tag does not need quotation marks.

cat <<"EOF"
Command is:
  $ lookup fantasy
EOF
If you do not want to expand, you need to escape the $character
cat <<EOF
  \$ lookup fantasy
EOF
Copy the code

In this example, note that the single $character is actually unexpanded and error-free, so we’re just writing an example.

What about quotation marks, single quotation marks, double quotation marks, they all work the same way.

You can even use escape syntax, that is:

cat <<\EOF
Command is:
  $ lookup fantasy
EOF
Copy the code

You can also disable parameter expansion.

Apply both

The two new syntactic features above can be combined and used simultaneously:

    cat <<-"EOF"
		Command is:
  		  $ lookup fantasy
    EOF
Copy the code

Although you probably don’t need to be in this situation.

reference

• Advanced Bash-Scripting Guide – Chapter 19. Here Documents
• Wiki: Here document