Daily Dev Log

2019 Jun 24

CLI

“foo –help” vs “man foo”

We just can’t remember options of CLI tools. In most cases, --help, like grep --help, is the go-to way to look for help.

For example, if you forget what -H of grep does.

$ grep --help | grep -- -H
  -H, --with-filename       print the file name for each match

man is too formal, wordy and overwhelming, comparing with --help. Usually, we can find what we want in the output of --help, without turning to man.

Another advantage of --help is that it’s always available, and built in the command itself. If you have ever used Git Bash for Windows, you will find that man is not available there. Relying on man only means you have to google the man(ual) in the browsers.

Or to save some typing, add a bash function into ~/.bashrc.

h() { $1 --help; }

Then type h grep to show the help.

CLI

xargs is Slow

2019 May 29
# filepaths.txt is a file with thousands lines
cat filepaths.txt | xargs -n 1 basename

It takes a while (seconds) to finish running the above command. A file with thousands lines usually is not considered as a big volume. Why is xargs slow in the above command?

After read a SO post, it turns out xargs in the above command runs basename thousands times, therefore it has bad performance.

Can it be faster?

According to man xargs,

xargs reads items from the standard input … delimited by blanks … or newlines and executes the command … followed by items read from standard input. The command line for command is built up until it reaches a system-defined limit (unless the -n and -L options are used). … In general, there will be many fewer invocations of command than there were items in the input.
This will normally have significant performance benefits.

It means xargs can pass a batch of “items” to the command. Unfortunately, the -n 1 option in the command forces xargs to just take one “item” a time. To make it fast, use the -a option of basename, which let basename be able to handle multiple arguments at once.

time cat filepaths.txt | xargs -n 1 basename > /dev/null 

real    0m2.409s
user    0m0.044s
sys     0m0.332s
time cat filepaths.txt | xargs basename -a > /dev/null 

real    0m0.004s
user    0m0.000s
sys     0m0.000s

Thousands times faster.

–show-limits
cat /dev/null | xargs --show-limits --no-run-if-empty

Your environment variables take up 2027 bytes
POSIX upper limit on argument length (this system): 2093077
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2091050
Size of command buffer we are actually using: 131072
Maximum parallelism (--max-procs must be no greater): 2147483647

It shows xargs can feed a lot bytes into the command once (2091050 bytes here).

-P

Some commands can usefully be executed in parallel too; see the -P option.

CLI

Daily Dev Log

2019 Jan 24

CLI

“su - app” vs “su app”

From man su,

   -, -l, --login
      Provide an environment similar to what the user would expect had the user logged in directly.

So with su - app, after switch to the user app, you end up in the user’s HOME directory, and have the user’s ~/.bash_profile (not ~/.bashrc) executed.

Tools like RVM need a “login shell”.

RVM by default adds itself currently to ~/.bash_profile file

So if use su app, RVM will not be ready there for you.

CLI

Daily Dev Log

2019 Jan 15

CLI

Convert Line Endings from DOS/Windows Style to Unix/Linux Style

tr -d '\015' <DOS-file >UNIX-file

For what character \015 is, see man 7 ascii. Or, there is a command called ascii, if it’s installed, type ascii '\015'.

More ascii Commands

$ ascii '\r'
ASCII 0/13 is decimal 013, hex 0d, octal 015, bits 00001101: called ^M, CR
Official name: Carriage Return
C escape: '\r'
Other names: 

Search Manuals

-k Search the short descriptions and manual page names for the keyword

$ man -k ascii
ascii (1)            - report character aliases
ascii (7)            - ASCII character set encoded in octal, decimal, and hexadecimal
...

Pitfalls

Do Not Use the Same File to Redirect Input and Output

tr -d '\015' <DOS-file >DOS-file

The above command will delete all content in the file!

From man bash,

[n]>word, if it does exist it is truncated to zero size.

(How did I find the file back? Luckily, the working directory is managed by Dropbox, and I found it back in the Dropbox.)

CLI

在Windows上安装tmux

2018 Apr 11

Windows上的Git BASH提供了大部分常用的Linux命令行工具,比如grep、sed等,但是并没有提供tmux。 实际上Git for Windows提供了包管理(package management)功能,

Git for Windows is based on MSYS2 which bundles Arch Linux’ Pacman tool for dependency management.

借助pacman,Git for Windows可以安装额外的命令行工具,比如tmux。 但是,在Git BASH里,pacman并没有默认开启

This is intended. We do not ship pacman with Git for Windows. If you are interested in a fully fledged package manager maintained environment you have to give the Git for Windows SDK a try.

需要安装Git for Windows SDK来开启pacman。 安装好之后,打开Git SDK(和Git Bash一样,是一个终端模拟器),

$ pacman -Ss tmux

会找到两个包,

msys/tmux 2.6-1
	A terminal multiplexer
msys/tmux-git 2.5.94.g73b9328c-1 
	A terminal multiplexer
$ pacman -S msys/tmux-git

安装的时候可能会报下面的错误,

$ pacman -S msys/tmux
warning: database file for 'git-for-windows-mingw32' does not exist
error: failed to prepare transaction (could not find database)

打开/etc/pacman.conf文件,注释掉下面的行即可,

#[git-for-windows]
#Server = https://wingit.blob.core.windows.net/x86-64

#[git-for-windows-mingw32]
#Server = https://wingit.blob.core.windows.net/i686

安装好之后,就可以在Windows上(Git SDK)使用tmux了。

tmux on Windows

pacman的用法可参见Git for Windows的Wiki

环境:Windows 10

(如果发现某些程序,比如ssh,报错,可以尝试用pacman -Syu升级所有package。)

CLI