Does all linux users are present on /etc/passwd?

There is one user “user1” which I cant find in /etc/passwd but I can execute cmds like

$touch abc
$chown user1 abc
$su user1

These command runs fine, but if I try to chown to some really nonexistent user these chown and su commands fail

I was wondering where is this user1 coming from?


While logged in with user1 (after su user1) execute:

getent passwd $USER

This fetches user passwd entries across different databases. All users are not necessarily system users – they can come from LDAP etc.
Check docs on getenv.

Also check your nsswitch.conf to see all sources used to obtain name-service information.


./path of file to execute not executing

I am trying to execute matlab from desktop path of file is


executing ./usr/local/MATLAB/R2017b/bin/matlab

also tried .//usr/local/MATLAB/R2017b/bin/matlab

and ./ /usr/local/MATLAB/R2017b/bin/matlab
how it works?


Just run /usr/local/MATLAB/R2017b/bin/matlab to access the binary via the full path otherwise you will run try to run it via the relative path: <CURRENT DIR>/usr/local/MATLAB/R2017b/bin/matlab if you put a . before.

You can also change the add /usr/local/MATLAB/R2017b/bin/ to your path variable in order to be able to execute the command matlab without having to specify its whole path each time.

Also change your ~/.bashrc file and add PATH=$PATH:/usr/local/MATLAB/R2017b/bin to be able to keep those change after a reboot and just run matlab

How can I do full outer join on multiple csv files (Linux or Scala)?

I have 620 csv files and they have different columns and data. For example:

word, count1
w1, 100
w2, 200

word, count2
w1, 12
w5, 22

//Similarly fileN.csv
word, countN
w7, 17
w2, 28

My expected output

word, count1, count2, countN
w1,    100,     12,    null
w2,    200 ,   null,    28  
w5,    null,    22,    null
w7,    null,   null,    17

I was able to do it in Scala for two files like this where df1 is file1.csv and df2 is file2.csv:

df1.join(df2, Seq("word"),"fullouter").show()

I need any solution, either in Scala or Linux command to do this.


Using Spark you can read all your files as a Dataframe and store it in a List[Dataframe]. After that you can apply reduce on that List for joining all the dataframes together. Following is the code using three Dataframes but you can extend and use same for all your files.

//create all three dummy DFs
val df1 = sc.parallelize(Seq(("w1", 100), ("w2", 200))).toDF("word", "count1")
val df2 = sc.parallelize(Seq(("w1", 12), ("w5", 22))).toDF("word", "count2")
val df3 = sc.parallelize(Seq(("w7", 17), ("w2", 28))).toDF("word", "count3")

//store all DFs in a list
val dfList: List[DataFrame] = List(df1, df2, df3)

//apply reduce function to join them together
val joinedDF = dfList.reduce((a, b) => a.join(b, Seq("word"), "fullouter"))
//|  w1|   100|    12|  null|
//|  w2|   200|  null|    28|
//|  w5|  null|    22|  null|
//|  w7|  null|  null|    17|

//To write to CSV file
  .option("header", "true")

This is how you can read all your files and store it in a List

//declare a ListBuffer to store all DFs
import scala.collection.mutable.ListBuffer
val dfList = ListBuffer[DataFrame]()

(1 to 620).foreach(x=>{
  val df: DataFrame =
    .option("header", "true")
    .load(BASE_PATH + s"file$x.csv")

  dfList += df

Provide password for running a script inside another script as different user

Imagine you run script as user A: sudo -u A ./
Inside that we have a line that calls another script with a different user B: sudo -u B ./ Now I get prompted to enter a password for user B. Is there any way to provide that password uppon call?

P.S. I know all the security breaches it can create. Please do not mention me that.


Yes you have option for that. But that password you have to provide at the time of executing the script or hard coded it in the script itself. Try with the below example:-

echo 'passwordofB' | sudo -u B -S ./

Also you can do it like:-

 sudo -u A ./ passwordofB #as a command line parameter

now inside

echo $1 | sudo -u B -S ./

You are executing another script sudo -u B ./ from script ./ right? So change that line with echo $1 | sudo -u B -S ./ and run your first script as sudo -u A ./ passwordofB where passwordofB is the password for user ‘B’

how to count occurrence of specific word in group of file by bash/shellscript

i have two text files ‘simple’ and ‘simple1’ with following data in them


    hi hi hello
    is it

    hello hi
    how are you

[]$ tr ' ' '\n' < simple.txt | grep  -i -c '\bh\w*'
[]$ tr ' ' '\n' < simple1.txt | grep  -i -c '\bh\w*'

this commands show the number of words that start with “h” for each file but i want to display the total count to be 7 i.e. total of both file. Can i do this in single command/shell script?

P.S.: I had to write two commands as tr does not take two file names.


This alternative requires no pipelines:

$ awk -v RS='[[:space:]]+' '/^h/{i++} END{print i+0}' simple.txt simple1.txt

How it works

  • -v RS='[[:space:]]+'

    This tells awk to treat each word as a record.

  • /^h/{i++}

    For any record (word) that starts with h, we increment variable i by 1.

  • END{print i+0}

    After we have finished reading all the files, we print out the value of i.

Shell script to modify variable with file name of multiple files in same directory

I have a multiple files in a directory, e.g.:


Line 3 of all text files in the directory is as follows:

consumer[location].userid= constant

I want to replace the word “location” with the respective filename without extension in the third line of each file eg: as

consumer[a01].userid= constant    --------- for a01.txt
consumer[a02].userid= constant    --------- for a03.txt

Can you please help?


So, starting at the top: You want to replace a word (location) in a specific line of a file.
Google says, sed can do that:

sed -i "3s/location/foo/" file.ext

will replace the first occurence of “location” in the third line of the file “file.ext”

The string you want to replace location with, should be a filename without extension.
Since you know the extension, basename should work fine for that:

basename /path/to/file.ext .ext

will return “file”.

Now loop through all the txt files in the directory:

for filename in directory/*.txt; do
# Do stuff

Combining these commands results in a short script like this:

# Assumes the script is located in the data directory
for filename in ./*.txt; do
    name=$(basename $filename .txt)
    echo "$filename -> $name"
    sed -i "3s/location/$name/" $filename