Run a shell script at startup / reboot

This post introduces how to auto run a shell script when your machine reboot.

We can set a crontab for this.

The crontab (short for “cron table”) is a list of commands that are scheduled to run at regular time intervals on your computer system. The crontab command opens the crontab for editing, and lets you add, remove, or modify scheduled tasks.

Just have a line added to your crontab.

Make sure the sh file is executable (you can use the following command to make the file executable)

$ chmod +x /path_to_you_file/your_file

To edit crontab file, enter the following command in your terminal:

$ crontab -e
# Edit your crontab.

The line you need to add:

@reboot  /path_to_you_file/your_file

for example,

@reboot  /home/user/test.sh

after every startup it will run the test.sh script.

To save and exit your crontab from vim,

 

Use the following command to confirm that it is actually been set.

$ crontab -l
# Display ("list") the contents of your crontab.

this will show @reboot sh $HOME/test.sh

To see whether the app that the .sh starts is actually running, you can use ps -ef (check here for more details about ps -ef).

Reboot the server to confirm whether it all works.

References:

Linux crontab command (PDF)

Linux at, batch, atq, and atrm commands (PDF)

How to run a shell script at startup (PDF)

Change folder permissions and ownership from command line

This post shows how to change permission and ownership from command line on Linux OS.

Use chown to change ownership and chmod to change permission.

The -R option makes them change the permissions and ownership for all files and directories inside of the given directory.

The following command will change ownership (both user and group) of all files and directories inside your_directory and including your_directory itself:

$ sudo chown -R username:group your_directory

And the following command will only change the permission of the folder directory but will leave the ownership of the files and folders inside the directory not changed:

$ sudo chown username:group your_directory

Note that you need to use sudo to change the ownership from root to yourself.

To make the current user own everything inside the folder (and the folder itself), use the following command:

$ sudo chown -R $USER your_directory

$ sudo chown -R $USER:$USER your_directory

For more commonly used Linux commands, check my other posts at here  and here .

Download files using Shell script

This post provides instructions on how to download files using Shell script.

To download a number of files that have similar file path, see an example as follows:

http://example.com/path/to/file1.csv
http://example.com/path/to/file2.csv
http://example.com/path/to/file3.csv
http://example.com/path/to/file4.csv
.
.
http://example.com/path/to/file4999.csv
http://example.com/path/to//file5000.csv

Method 1: 

$ wget http://example.com/path/to/file{1..5000}.csv

Method 2:

write a shell script as the following:

for i in {1..5000}
do
wget http://example.com/path/to/file$i.csv;
done

 

For more commonly used Linux commands, check my other posts at here  and here .

Find IP address on Ubuntu

This post introduces how to find the IP address on your Ubuntu.

Open your terminal (CTRL + ALT + T), and then issue the following command.

$ ifconfig | less

your IP should be listed under

inet addr

Something looks like this:

inet addr:10.0.5.25 Bcast: 10.0.5.255 Mask:255.255.255.0

In this example, 10.0.5.25 would be your IP address.

Note: press “q” to exit.

 

Shell Scripting Tutorials (Pic-notes)

This post provides some pic notes for Shell Scripting Tutorials (a list of 62 short videos) .

No capital letters in bash shell command (All should be in lowercase)!

clear

who

pwd 

cal

date

touch — create empty (plain text) files

mkdir — create directory (directory is the same thing as folder, in terminal it is called directory, on GUI interface it is called folder)

cd  — change directory

cat

cat > test   — this will create a file called test and write text into the file. (Ctrl+ D to exit editing the file.)

cat < test (shortcut cat test  —  this will open an existing file called test and print its content in the terminal.

cat random test > sample  — this will merge two existing files called random and test, and write the content to a new file called sample

mv — rename (when the source and destination is in the same directory) or move files

rm

rm -r  (r means recursive)

rmdir   — rmdir (only for empty directory)

cp  — copy files and /or directories

ln file1 file 2 — (file 1 is a physical copy of file 2, when file 1 changes, file 2 will be updated automatically and in real time)

ln -s file1 file1_soft  — (file1_soft is a soft link  of file1, when file1 was removed, file1_soft will not work, because the physical copy it refers to has been gone.)

 

 

 

file * command

the out put of wc command:

number of line, word, and charactor in the input file.

wc -l           number of lines

wc -w         number of words

wc -c          number of characters

 

 

  • Shell Scripting Tutorial-12: Sort

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

-f   file

-d   direcotry

-s file size (greater than zero or not)

-c (character file, e.g.,  .dbf file)

-b (block file like video and image)

-r  (read permission)

-w (write permission)

-x   (executable or not)

cat >> $fname   — will not overwrite the existing content in the file and will append … cat > $fname will overwrite

-n (the length of the string is not zero)

-z (the length of the string is zero)

the output of $?  is 0 means the operation is true, if false, the output is 1.

 

-a  AND logic operator

because there is a new line symbol, so need to be -eq 2 for a character.

 

 

 

-0  or logic operator

|    pipe symbol

the output of echo $var   as the input of wc -c, use reverse quote symbol to group it as a whole for -eq  operator

|  — the pipe symbol tells the case not to end the case statement after checking [aeiou]*, need to check [AEIOU]* as well for the first case statement.

Normally 0 is for success, 1 is for failure of the result of $?

2 means invalid.

the default IFS is white space.

exec and tty command

exec — exec is a builtin command of the Bash shell. It allows you to execute a command that completely replaces the current process. The current shellprocess is destroyed, and entirely replaced by the command you specify. (more check here.)

tty — Print the file name of the terminal connected to standard input. ( check more at here.)

read — read is a builtin command of the Bash shell, which reads a line of text from standard input and splits it into words. These words can then be used as the input for other commands. (for more info check here.)

sleep command — used for delay (the unit of its parameter is in seconds)

(see also Shell Scripting Tutorial-11: Count Lines, Words & Characters Using ‘wc’)

In this tutorial you’ll learn to iterate over contents of a text file using for loop.

 

In this tutorial you’ll learn to use nested loops to print all possible combinations of digits 1, 2 and 3.

 

In this tutorial you’ll learn to use the break statement to transfer control from within the loop to the first statement after the loop.

In this tutorial you’ll learn to use the ‘continue’ statement to skip iterations in loops.

 

In this tutorial you’ll learn to execute multiple commands on a single line in the terminal without using a shell script.

In this tutorial you’ll learn to add or remove users on the system, also we’ll check out the ‘login’ command to login through a user account in a terminal window.

In this tutorial we’ll build a script that checks for a user every minute and then notifies us when it logs in and also displays the time the user was late in logging in.

In this tutorial we’ll improve the script we built in the previous lesson.

In this tutorial we’ll finish our work on the script we first met in the 56th tutorial. We’ll check every one second for the user and will also report time with greater precision.

In this tutorial you’ll learn to send messages to other users on your system using the ‘write’ command.

You’ll also learn to use ‘finger’ command to see which users who have disabled message reception and which have not.

one terminal (terminal 1):

Another terminal (terminal 2):

in terminal 1:

in Terminal 2:

In Terminal 1:

In terminal 2:

In terminal 1:

In Terminal 2:

EOF — End Of File

 

Finger command:

In terminal 1:

In Terminal 2:

 

In Terminal 1:

Note the * before pts/1 (that indicates newuser2 does not allow send msg to it)

 

In this tutorial you’ll learn to create your own commands using functions.

unset command to remove customized functions in terminal.

 

In this tutorial we’ll learn to execute a script from another script; not a particularly hard thing to do but I recorded a tutorial on it anyway.

Thank you so much for watching the tutorials in this course. I hope you guys had as much fun watching the tutorials as I had recording them for you.

The END:) happy shell scripting!

 

 

 

 

 

 

 

Bash shell scripting resources

This post provides some useful (video) resources for bash shell scripting. (If you do not like video style tutorials, check here for a post that is not video based.)

The bash ( Bourne Again Shell) is the most common shell installed with Linux distributions and Mac OS.

  • Bash Basics (A video playlist with 9 videos) — The Basics of using Bash in Linux with a focus on Ubuntu.

Bash Basics Part 1 of 8 | Access and Navigation

       $ means you logged in as a normal user.

       # means logged in as a root user.

Bash Basics Part 3 of 8 | Privileges and Permissions

Bash Basics Part 8 of 8 | Bash Scripting

Free tools to merge and split PDF files on Ubuntu and Windows

This post provides some GUI and command line tools to merge and split PDF files on Ubuntu and Windows.

(For Mac users, check out my post here for solutions.)

  • PDF-Shuffler is a GUI package that allows us to merge, split and re-arrange pages from PDF documents

Install use the command in your terminal (I have tested, It works on Ubuntu 16.04 LTS.)

sudo apt-get update
sudo apt-get install pdfshuffler

It is pretty simple and straightforward to use.

Limitations:

If you try to merge pdf files that are encrypted, you will meet this error.

*.pdf is encrypted.
Support for encrypted files has not been implemented yet.
File export failed.
  • PDFtk is a simple and powerful command line tool for doing everyday things with PDF documents. It comes in three flavors: PDFtk Free, PDFtk Pro, and the original command-line tool PDFtk Server. There is a simple GUI free versio for Windows users, see below for the info.

Install use the command in your terminal (I have tested, It works on Ubuntu 16.04 LTS.)

sudo apt-get update
sudo apt-get install pdftk

If you are a windows user, you do not need to buy adobe pdf pro version to manipulate pdf files any more, see the link to download PDFtk free, it is a simple GUI package to use on Windows. See the picture below for a sense of it.

 

See below for examples of using PDFtk to manipulate PDF files via linux command line.

This tool lets you do all sorts of things with PDF documents, such as split, merge or rotate them, apply a background watermark, burst documents into single pages, etc. mostly used command: use pdftk to split PDF files, use the following command :

pdftk in.pdf cat 2-8 output out.pdf

This will extract pages 2 through 8 from in.pdf and save them as out.pdf. The input file in.pdf will not be altered. You can use any other page range in place of 2-8. Leave out the range to select the whole document.

And here is an example of how to merge PDF files:

pdftk in1.pdf in2.pdf cat output out.pdf

This will concatenate in1.pdf and in2.pdf, saving the result as out.pdf. If you want to merge individual page ranges, you have to assign handles to input files, like so:

pdftk A=in1.pdf B=in2.pdf cat A2-3 B4-5 \

output out.pdf

This will merge pages 2-3 from in1.pdf with pages 4-5 from in2.pdf into out.pdf. The backslash means that the input is continued on the next line. You can use it when you type in long commands, but you don’t have to.

 

pdftk in.pdf cat 1-12 output in_page1-12.pdf
pdftk in.pdf cat 15-end output in_page15-end.pdf

You can use this method to split a PDF in N ways, or to remove pages.

For example, to remove page 13:

pdftk in.pdf cat 1-12 14-end output out.pdf

Or use it to rotate pages and many other things, type man pdftk in your command to see the options.

Installation is also possible by downloading a binary (Windows, OS X, Linux) or using Homebrew.

 

  • cpdf, it is a powerful, free tools to manipulate PDF files using command line. The Coherent PDF Command Line Tools allow you to manipulate existing PDF files in a variety of ways. For example:
  • Merge PDF files together, or split them apart
  • Encrypt and decrypt
  • Scale, crop and rotate pages
  • Read and set document info and metadata
  • Copy, add or remove bookmarks
  • Stamp logos, text, dates, page numbers
  • Add or remove attachments
  • Losslessly compress PDF files

See below for some examples:

cpdf in.pdf 1-12 -o out.pdf

cpdf in.pdf 13-end -o out.pdf

Or, to split into 12-page-sized chunks:

cpdf in.pdf -split -chunk 12 -o out%%%.pdf

Note that: for a large pdf file (1000+ pages) the performance of cpdf was much better than PDFtk.

References:

Edit PDF files on Linux – the easiest way

Sometimes you run up in a situation when you need to edit a PDF file on Linux. For example, when you need to go through an old report which was in PDF format and you saw some typos. how to edit PDF files on Ubuntu?

I remembered that I used a program called pdfedit but it’s not available on Ubuntu or in any other Linux distributions anymore.

Things change now. In the past, we needed to use a dedicated application for this purpose. There are still plenty of tools to edit PDF files on Linux, but we don’t really need them, well not in most of the cases, because LibreOffice does the job for us.

Surprised? Yes, the popular Linux alternative to Microsoft Office can also edit PDF files. And perhaps this is the easiest way for editing PDF files.

 How to Edit PDF files on Linux
Most of the Linux distributions such as Ubuntu, Linux Mint, Fedora etc has LibreOffice as default office suite. If you do not have LibreOffice, you can install it.

Once you have installed LibreOffice, Use LibreOffice Draw to open the pdf file to edit. You will see that the file is in editable mode. You can use the select pointer to choose which bit of text to edit. Just click on the text which you want to edit. For those areas that are no text, you will need to use draw tool (e.g., pencil, circle etc.) to add things.

Once you are done with the edits, instead of saving the file (using Ctrl+S) option, click on Export to PDF button.

If you are not finising the edit yet, but need to save the editted file, save it and it will be saved as a .odg file. Next time you would like to continue the edit, just open the .odg file, you will be able to continue your edits.

It is pretty cool, right? We do not need to buy Adobe Pro version to edit pdf files any more.

Limitations of editing PDF files with Libreoffice.

This PDF editing does not work on scanned documents. The files which were originally created as text and saved as PDF can be edited very easily but it is not the case when you have scanned document because those pages are actually images. But you can use draw tools (e.g., pencil, circle) in the Libreoffice to add things like signature, tick sign etc, you just cannot edit the text part if it is a pdf file from scanning.