April 2018 – Liping's playground

lftp usage on Linux

This post introduces lftp usage on Linux.

(Thanks Scott for sharing this nice tool.)

lftp is a command-line program client for several file transfer protocols. lftp is designed for Unix and Unix-like operating systems.

1: Installation on Ubuntu:

$ sudo apt-get update
$ sudo apt-get install lftp

2: Some commonly used commands with lftp:

ls — list direcotries and files on remote ftp server

cd — change directory on remote ftp server

lpwd — list local machine current directory

lcd — change local directory

mput *.csv — mput for multiple files, put for a single specific file.

This will take from the current local directory and put into the current remote directory on remote ftp server.

copy directory and its contents to and from ftp server (from and to local machine):

mirror -c -R path/to/localdirectory/

This will copy local directory to (the current directory) on the remove ftp server , c means continue)

mirror -c path/to/remote/directory

This will copy the remote directory to (the current local directory , c means continue)

(for more commands, check the lftp man page provided in the references below.)

3: Using Box under Linux (using lftp)

The lftp command is a powerful file transfer client that can be used with Box.

Box supports the secure FTPS protocol and you can use this with lftp for secure file transfer and mirroring.

The lftp command is smart enough to auto-negotiate to use FTPS but this example forces the issue to make absolutely sure unencrypted FTP is never attempted.

Here is an example of how to connect using FTPS.

$ lftp

lftp :~> set ftps:initial-prot ""

lftp :~> set ftp:ssl-force true

lftp :~> set ftp:ssl-protect-data true

lftp :~> open ftps://ftp.box.com:990

lftp ftp.box.com:~> user yourusername@psu.edu

Password:

lftp yourusername@psu.edu@ftp.box.com:~> ls

...

when finish use exit to exit from lftp.

4: Copy a remote folder via FTP on the Linux command line using lftp (PDF)

References:

lftp Man page (PDF)

Using Box under Linux (PDF)

For more commonly used Linux commands, check my other posts at here and here .

Find all files containing specific text on Linux and Mac

This post introduces how to find all files containing some text on Linux and Mac.

Step 1: cd to the folder where you would like to start with the search.

Step 2: use the following command:

$ grep -Ril "text-to-find-here" ./

Then the file names containing the text you would like to search will be listed.

R — stands for recursive.
i — ignore case (optional in your case).
l — stands for “show the file name, not the result itself”.

For more commonly used Linux commands, check my other posts at here and here .

Find files NOT ending in specific extensions

This post provides instructions on how to find files NOT ending in specific file extension(s) when a folder contains many many files on Linux (tested on Ubuntu and CentOS) and Mac OS.

Open a terminal, and

cd into the folder you would like to list the file extensions.

In your terminal, issue the following command :

$ find . ! \( -name "*.py" -o -name "*.exe" \)

This will list all files that are not a .py file and not a .exe file (change the file extension(s) name according to your name:)).

$ find . ! \( -name "*.exe" \)

This will list all files that are not a .exe file.

$ find . ! \( -iname "*.exe" \)

This will list all files that are not a .exe or .EXE file.

Note: -name is case sensitive while -iname is case insensitive.

For more commonly used Linux commands, check my other posts at here and here .

Find the distinct file extensions in a folder

This post provides instructions on how to list all the distinct file extensions when a folder contains many many files on Linux (tested on Ubuntu and CentOS) and Mac OS.

Open a terminal, and

cd into the folder you would like to list the file extensions

Method 1: issue the following command:

$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort -u

This will help us:

Find all files from current folder (including files contained in its sub-folders)
Prints extension of files if any
Make a unique sorted list (e.g., sort -u is for ordering by Alphabet, -u means unique)

See the pic below for an example.

Method 2: A variation of the method 1. Issue the following command

$ find . -type f | perl -ne 'print $1 if m/\.([^.\/]+)$/' | sort | uniq -c | sort -n

This will list with counts for each extension (sort -n is for ordering by file number).

See the pic below for an example.

More flags about sort command can be found here (PDF) and here (PDF).

For more commonly used Linux commands, check my other posts at here and here .

Long underscore in LaTeX for signature

This post introduces how to add long underscore in LaTeX for signature

You can use \underline and \hspace:

See the following example:

\documentclass{article}

\begin{document}
Name \underline{\hspace{3cm}}
Signature \underline{\hspace{3cm}}

\end{document}

Dynamic GPU usage monitoring (CUDA)

To dynamically monitor NVIDIA GPU usage, here I introduce two methods:

method 1: use nvidia-smi

in your terminal, issue the following command:

$ watch -n 1 nvidia-smi

It will continually update the gpu usage info (every second, you can change the 1 to 2 or the time interval you want the usage info to be updated).

use Ctrl + C to terminate.

method 2: use the open source monitoring program glances with its GPU monitoring plugin

in your terminal, issue the following command to install glances with its GPU monitoring plugin

$ sudo pip install glances[gpu]

to launch it, in your terminal, issue the following command:

 $ sudo glances

Then you should see your GPU usage etc. It also monitors the CPU, disk IO, disk space, network, and a few other things

use Ctrl + C to terminate.

For more commonly used Linux commands, check my other posts at here and here .

Unzip file from Terminal on Linux

This post provides the instructions about how to unzip a file from terminal on Linux.

Step 1: install unzip using the following command

$ sudo apt-get install unzip

Step2: cd to the directory where you zip file located

Step3: use the following command to unzip your file

$ unzip yourfile.zip 

#this will unzip the file to the current folder

#if you'd like to extract to a particular destination folder, use the following instead:
$ unzip yourfile.zip -d destination_folder

For more commonly used Linux commands, check my other posts at here and here .

Counting files in a Linux directory

This post will show you how to count files under a Linux directory.

Step 1: cd to the folder you would like to count the files.

Step 2: issue the following command, then you should see a number, which is the number count of the files under the current directory.

find . -type f | wc -l

-type f indicates counting files only. You can remove the -type f to include directories (and symlinks) while counting.
| (note: not ¦) redirects find command’s standard output to wc command’s standard input.
wc (stands for word count) counts newlines, words, and bytes on its input (see here for more details.).
-l to count just newlines.

Notes:

This command may overcount if filenames can contain newline characters.

For more commonly used Linux commands, check my other posts at here and here .

List files by time (in reverse order) and in human readable file size

This post tells you how to list files under a directory by time in reverser order and in human readable file size (works on Linux OS and Mac).

ls -ltrh

list all the csv files under the current directory in long format by time and in reverse order, the file size in human readable format (e.g., in mb, or gb, instead of byte size)

-l List in long format. If the output is to a terminal, a total sum for all the file sizes is output on a line before the long listing.
-r Reverse the order of the sort to get reverse lexicographical order or the oldest entries first (or largest files last, if combined with sort by size.
-t Sort by time modified (most recently modified first) before sorting the operands by lexicographical order.

ls -ltrh *.csv

list all the csv files under the current directory in long format by time and in reverse order, the file size in human readable format (e.g., in mb, or gb, instead of byte size)

For more commonly used Linux commands, check my other posts at here and here .