Read file lines into shell line separated by space

I have a file called requirements.txt, each line has a package :

package1
package2
package3

I'm looking for a command that will take each line of requirements.txt and join them in one line, space separated.

So basically, I'm looking to generate the following command:

command package1 package2 package3

I tried using a for and applying command for each line of requirements.txt but it was much slower

6 Answers

You can use xargs, with the delimiter set to newline (\n): this will ensure the arguments get passed correctly even if they contain whitespace:

xargs -rd'\n' command < requirements.txt

From man page:

-r, --no-run-if-empty
If the standard input does not contain any nonblanks, do not run the command. Normally, the command is run once even if there is no input. This option is a GNU extension.

--delimiter=delim, -d delim
Input items are terminated by the specified character.

2

You can simply use bash redirection and command substitution to get the file contents as arguments to your command:

command $(<file)

This works because not only space is a valid word splitting character, but newline as well – you don’t need to substitute anything. However, in the case of many lines you will get an error referring to the shell’s ARG_MAX limit (as you will with substitution solutions). Use printf to built a list of arguments and xargs to built command lines from it to work around that:

printf "%s\0" $(<file) | xargs -0 command

Note that neither of these approaches work for lines containing whitespace or globbing characters (besides the newline of course) – fortunately package names don’t (AFAIK). If you have whitespace or globbing characters in the lines use either xargs (see this answer) or parallel (see this answer).

3

Use tr and bash substitution:

command $(tr '\n' ' ' < requirements.txt)

for example:

echo $(tr '\n' ' ' < requirements.txt)

output would be:

package1 package2 package3

or:

sudo apt install -y $(tr '\n' ' ' < requirements.txt)

would install all packages.

6

Read items from file instead of standard input:

xargs --arg-file=requirements.txt echo

Or, to fail if the argument list is too long (rather than running echo more than once):

xargs --arg-file=requirements.txt -n 1 echo

The short form for --arg-file is -a, so you can use xargs -a requirements.txt in place of --arg-file=requirements.txt if you want.

2

You can use GNU parallel for this (sudo apt install parallel):

parallel -m command <file # or
parallel -m command :::: file

parallel takes only newline as the delimiting character and (of course) doesn’t perform any globbing, so characters like space, asterisk*, question mark ? and the like in the lines are no problem at all. -m is for multiple arguments like xargs does it. If you don’t want command lines to be run in parallel, which is what parallel does by default, add -j1 for 1 job at a time – for package installation this may be advisable.

Example run

$ ls
a_test file
$ cat file
a*
b with spaces
$ parallel -m 'printf "line: %s\n"' <file
line: a*
line: b with spaces
$ parallel 'printf "line {#}: %s\n" {}' <file
line 1: a*
line 2: b with spaces

The directory contains two files: a_test and file. file’s content consists of two lines: a* and b with spaces. The third command line in this example executes printf "line: %s\n" with all the lines from file as its arguments, this prints the argument with “line: ” before and a linebreak after it. As there is no globbing, the first argument a* can’t match the existing file a_test, but is printed as-is. The same goes for the next argument with spaces in it, the spaces are preserved instead of the argument being split into three.

The fourth command is just an addendum showing one of the many nice features of parallel: {#} in a command is replaced by the sequence number of the job, and as this one calls printf with only one argument (no -m) it is equal to the current line number. {} is the placeholder for the argument(s) and can be omitted if there’s no other placeholder used in the command and the argument(s) should be added to the end of the command, which I did before.

Further reading on GNU parallel:

• Homepage
• Youtube playlist of tutorial videos
• man parallel’s exhaustive example section

0

Just read the lines into an array. This is easy with bash 4 (which provides the readarray command):

readarray -t lines < /path/to/file
echo "${lines[@]}"

Your Answer

Sign up or log in

Sign up using Google Sign up using Facebook Sign up using Email and Password Submit

Post as a guest

Post Your Answer Discard

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy