Linux TIPS

Performance

https://cromwell-intl.com/open-source/performance-tuning/file-systems.html

Diff, show modified line from right file only

diff --unchanged-line-format= --old-line-format= --new-line-format='%L' a b

config RHEL services

service=(crond network ntpd rsyslog sshd sysstat irqbalance)
chkconfig --list | awk '{ print $1 }' | xargs -n1 -I@ chkconfig @ off
echo ${service[@]} | xargs -n1 | xargs -I@ chkconfig @ on

Connecting a Bluetooth device for serial communication

rfcomm0 {
  # Automatically bind the device at startup
  bind no;
  # Bluetooth address of the device
  device 11:22:33:44:55:66;
  # RFCOMM channel for the connection
  channel 3;
  # Description of the connection
  comment "This is Device 1's serial port.";
}

Scan for bluetooth devices:

hcitool scan
Scanning ...
    20:15:12:08:62:95   HC-06

Bind using rfcomm

sudo rfcomm bind 0 20:15:12:08:62:95 1

NB: bind 0 refers to device number 0 (rfcomm0) and 1 is the channel. 

* Pair using bluetoothctl:

    power on
    agent on
    scan on
    ... wait ...
    scan off
    pair <dev>

Create serial device: rfcomm bind 0 <dev>

You should now have /dev/rfcomm0.

RAID

RAID 10

RAID 1+0 (or 10) is a mirrored data set (RAID 1) which is then striped (RAID 0), hence the "1+0" name. A RAID 1+0 array requires a minimum of four drives – two mirrored drives to hold half of the striped data, plus another two mirrored for the other half of the data. In Linux, MD RAID 10 is a non-nested RAID type like RAID 1 that only requires a minimum of two drives and may give read performance on the level of RAID 0.

RAID 01

RAID 0+1 (or 01) is a striped data set (RAID 0) which is then mirrored (RAID 1). A RAID 0+1 array requires a minimum of four drives: two to hold the striped data, plus another two to mirror the first pair.

X11 putty

enable X11 Forward

• then

[root@node1 ~]# export DISPLAY=:0

How can I dual-boot Windows 10 and Linux

http://askubuntu.com/questions/666631/how-can-i-dual-boot-windows-10-and-ubuntu-on-a-uefi-hp-notebook

https://archives.gentoo.org/gentoo-user/message/a6c3d40e8827f65df098269e01186b59

http://unix.stackexchange.com/questions/316602/how-to-properly-dualboot-opensuse-and-windows-10

Enter single usermode

rw init=/bin/bash

Convert audio to text

how to export outlook exchange calendar to my google calendar

http://oregonstate.edu/helpdocs/faq/how-do-i-export-my-outlook-exchange-calendar-my-google-calendar

If you are on Windows PC: Open up your Microsoft Outlook program

If you are on a Mac OS X PC: Go to https://exmail.oregonstate.edu/owa

    Click on Calendar in the lower left corner
    You should see all your calendars in front of you. On the left side of the window, you should see the list of calendars and their names. Under My Calendars, right click the one you wish to add to your Google account.
    After right clicking your chosen calendar, right click, go Share > Publish This Calendar (Publish This Calendar to the Internet)
    A new window will pop up. Select the options for which best suit you. 
    When complete, press Start Publishing in the lower right corner.
    A couple of urls will show up for you to copy. Find the one that ends in .ics and copy that whole url.
    Go to Google Calendar
    When logged in to view your calendar, go find Other Calendars on the left side of the window
    After finding Other Calendars, there should be a box with a triangle in it to the right of the text. Click that box and find "Add by URL".
    A popup will show up. In this popup, you want insert the .ics URL from your Outlook here.
    Press Add Calendar below.

If you would like to stop publishing your calendar and no longer receive updates to your Google Calendar you can follow these directions:

http://oregonstate.edu/helpdocs/e-mail/exchange-outlook-oregonstateedu/publishing-exchange-calendars

show bonding status

watch cat /proc/net/bonding/bond0

mpv about

$ mpv --secondary-sid=2 --sub-file=tlf-jerrymaguire.bd.eng.srt --sub-file=tlf-jerrymaguire.cht.srt Jerry\ Maguire\ 1996\ 1080p\ BluRay\ x264\ AAC\ -\ Ozlem.mp4

How to change the output color of echo in Linux

https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux

Sed & Awk About

Extract File Basename Without Path and Extension in Bash

$ s=/the/path/foo.txt
$ echo ${s##*/}
foo.txt
$ s=${s##*/}
$ echo ${s%.txt}
foo
$ echo ${s%.*}
foo

fbname=$(basename "$1" .txt)
echo "$fbname"

Add a column to any position in a file in unix [using awk or sed]

• awk -v FS='|' -v OFS='|' '{$3=$3"|"4} 1' 1.txt

How to insert text before the first line of a file

• I've been looking around sed command to add text into a file in a specific line. This works adding text after line 1:

sed '1 a\

• Use sed's insert (i) option which will insert the text in the preceding line.

sed '1 i\

Sed delete dos CR/LF line

sed 's/.$//' file > out 
sed 's/\x0D$//' file > out

To remove all whitespace (including tabs) from left to first word

echo "     This is a test" | sed -e 's/^[ \t]*//'
sed  '/\#/d;/^$/d' /etc/nginx/nginx.conf

sed replace spaces with single space

sed -e "s/ \+/ /g;/^$/d" < ml.txt  > mll.txt
sed 's/ \{1,\}/ /g' 

http://superuser.com/questions/241018/how-to-replace-multiple-spaces-by-one-tab

remove all unwanted blank space then SORT it

sed 's/ /\n/g' < list  | sort

add/replace within <<<&>>>

# & replaces whatever matches with the given REGEXP.
$ sed 's@^.*$@<<<&>>>@g' path.txt
<<</usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin>>>
<<</usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:>>>
<<</opt/omni/lbin:/opt/omni/sbin:/root/bin>>>
$ sed 's@/usr/bin@&/local@g' path.txt
/usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin/local:/opt/omni/bin:
/opt/omni/lbin:/opt/omni/sbin:/root/bin

Find About

把找到的文件复制到另一个地方

find /mnt/sda7/software_linux/ -name "*.exe" | xargs -i cp {} /home/ftp/incoming/
find /mnt/sda7/software_linux/ -name "*.exe" -exec cp '{}' /home/ftp/ ';'
find cacti-xxxx -type d -exec mkdir -p utf/{} ';'
find cacti-xxxx -type f -exec iconv -f GB2312 -t UTF-8 {} -o utf/{} ';'
find /home/user1 -name '*.txt' | xargs cp -av --target-directory=/home/backup/ --parents 

Find out all the jpg images and archive it

find /var/log -name '*.log' | tar cv --files-from=- | bzip2 -9 > log.tar.bz2 
find / -name *.jpg -type f -print | xargs tar -cvzf images.tar.gz

create symbol link from DIR

find /mnt/data/media/* -maxdepth 0 -type d -exec ln -sv '{}' . \;

把目录下的包含大写字母的文件列出来

find /usr/share/ -type f | xargs ls -l | grep '[:upper:]' | less -r
find . -maxdepth 2 -size +100M -exec ls -lh "{}" \;

把目录下所有iso文件找出来并计算总大小(GB)

find /srv/ /mnt/media/ -name "*.iso" | xargs ls -lk | echo `awk '{a+=$5}END {print a}'`/1024/1024 | bc
find /srv/ /mnt/media/ -name "*.iso" | xargs ls -lk | awk '{a+=$5}END {print "Total sum is", a/1024/1024"GB"}'
find /srv/ /mnt/media/ -name "*.iso" -printf "%s\n" 2>/dev/null | awk '{a+=$1}END { print "Total sum is", a/1024/1024/1024"GB\nTotal time is :"}'

== Find files out then Copy files with sequence number ===

find . -name "*tgz" -type f -exec ls -Sr '{}' \; |awk '{system("cp -v "$0" "NR)}'

查找20100401时更改过的文件

cat >> 20100401.sh <<"EOF"
FILE=`find ./ -name "*.tex"`
ls -l --full-time $FILE 2>/dev/null
grep "2010-04-01"
EOF

Niubility Tool Netcat

Transfer file(s)

• Suppose you want to send files in /data from computer A with IP 192.168.2.111 to computer B(with any IP), It's as simple as this:

server: $ tar -cf - /data | nc -l -p 6666
client: $ nc 192.168.2.111 6666 | tar -xf -

• A single file can be sent even easier.

server: $ cat file | nc -l -p 6666
client: $ nc 192.168.2.111 6666 > file

• And you may copy and restore whole disk with nc:

server: $ cat /dev/sda | nc -l -p 6666
cliend: $ nc 192.168.2.111 6666 > /dev/sda

As Port Scanner

nc -vv -z 127.0.0.1 8079-8081

history how to use

$ history | grep 'ipt'
2    iptables -L -n -v -t nat

$ !2 # will execute the iptables command * MarkS, yes I know a way to make history contain only uniques. Do the following:

export HISTIGNORE="&"
That will make history not to save duplicate consecutive entries!

some keycodes err will resolve

• Added new keycodes for sk2506 HP multimedia 23-key keyboard

setkeycodes e016 235
setkeycodes e01e 158
setkeycodes e012 146
setkeycodes e014 148
setkeycodes e015 149
setkeycodes e02d 173
setkeycodes e018 236
setkeycodes e026 238
setkeycodes e017 227
setkeycodes e01f 159
setkeycodes e025 239
setkeycodes e023 237

For my HASEE-F4200

atkbd.c: Unknown key released (translated set 2, code 0xd8 on isa0060/serio0).
atkbd.c: Use 'setkeycodes e058 <keycode>' to make it known.
getkeycodes will show you codes!

then

setkeycodes e058 125

Safe Copying

if [ -f $2 ]
then
        echo "$2 is exists. Do you want to overwrite it? (y/n)"
        read yn
        if [ $yn = "N" || $yn = "n" ]
        then
                exit 0
        fi
fi
cp $1 $2

SMB About

mount.cifs //nas.systec.com.cn/Share /mnt/zip/ -o user=cml,domain=systec.com.cn

smbmount //192.168.2.51/D memory/ -o iocharset=cp936,user=xxxx,pass= --verbose
smbclient -U user -I 192.168.16.229 -L //smbshare/    # List the shares
mount -t smbfs -o username=winuser //smbserver/myshare /mnt/smbshare
mount -t cifs -o username=winuser,password=winpwd //192.168.16.229/myshare /mnt/share
sudo mount -t cifs -o username=${USER},password=${PASSWORD},uid=<user>,gid=<group> //server-address/folder /mount/path/on/ubuntu
[[http://unix.stackexchange.com/questions/68079/mount-cifs-network-drive-write-permissions-and-chown]]

ss AND netstat && route && ip

• This program is obsolete. Replacement for netstat is ss. Replacement for netstat -r is ip route. Replacement for netstat -i is ip -s link.Replacement for netstat -g is ip maddr.

ss -r
ip -s
ip maddr
netstat -lptu
netstat -antup

Add route gateway

route add a.b.c.d dev wlan0
ifconfig eth0 128.100.75.111 broadcast 128.100.75.0 netmask 255.255.255.0
route add -net 128.100.75.0 netmask 255.255.255.0
route add default gw 10.7.84.1 dev wlan0

hdparm

hdparm -a 256 -d 1 -r 0 -u 0 -m 2 -c 1 -A 1 -K 0 -P 0 -X 0 -W 0 -S 0 /dev/sda 

will enable DMA ,32bit support, Sector count for multiple sector I/O to 2

If your swapdisk is less than a 1GB, then

dd if=/dev/zero of=/directory/with/much/free/space/tempswap bs=1k count=1000000
chmod 600 tempswap
mke2fs tempswap
mkswap tempswap
swapon tempswap

Writing the actual CD

List content of ISO file

isoinfo -f -R -i cflinux-1.0.iso

• Assuming that you've got cdrecord installed and configured for your cd-writer

type:

cdrecord --scanbus
cdrecord -v speed=<desired writing speed> dev=<path to your writers generic scsi device> boot.iso

Sort about

• Generate a tags file in case-insensitive sorted order.

find src -type f -print0 | sort -z -f | xargs -0 etags --append

Shuffle a list of directories, but preserve the order of files within each directory. For instance, one could use this to generate a music playlist in which albums are shuffled but the songs of each album are played in order.

ls */*.mp3 | sort -t / -k 1,1R -k 2,2
sort -n -k 2 -t : facebook.txt
sort: ls -l | sort -nr +4
cut: cut -f1 -d':' < /etc/passwd        

Shuf about

shuf <<EOF
x
xx
xxx
EOF
shuf -i 1-100
shuf -e clubs hearts diamonds spades

Paste about

• For example

     $ cat num2
     1
     2
     $ cat let3
     a
     b
     c
     $ paste num2 let3
     1       a
     2       b
             c
$ paste -d '%_' num2 let3 num2
          1%a_1
          2%b_2
          %c_

Join about

• For example:

     $ cat file1
     a a1
     c c1
     b b1
     $ cat file2
     a a2
     c c2
     b b2
     $ join file1 file2
     a a1 a2
     c c1 c2
     b b1 b2

What is "NR==FNR" in awk

 26
down vote
        

Look for keys (first word of line) in file2 that are also in file1.
Step 1: fill array a with the first words of file 1:

awk '{a[$1];}' file1

Step 2: Fill array a and ignore file 2 in the same command. For this check the total number of records until now with the number of the current input file.

awk 'NR==FNR{a[$1]}' file1 file2

Step 3: Ignore actions that might come after } when parsing file 1

awk 'NR==FNR{a[$1];next}' file1 file2 

Step 4: print key of file2 when found in the array a

awk 'NR==FNR{a[$1];next} $1 in a{print $1}' file1 file2

urxvt run console application

urxvt -e bash -c geany_run_script.sh
urxvt -hold -e bash -c "./application"

https://bbs.archlinux.org/viewtopic.php?id=63516

Wireless reject due to mismatch with WPA/WPA2

Convert png/jpeg to movie(mpeg,mpg,avi,gif)

convert -delay 10 -loop 0 *.png animation.gif

Rename about

Upper to Lower Case

rename 'y/A-Z/a-z/' *.HTML

降低屏幕蓝光的小工具

redshift

app-crypt/letsencrypt

Let's Encrypt 是一个新的数字证书认证机构，它通过自动化的过程消除创建和安装证书的复杂性，为网站提供免费的 SSL/TLS 证书。

celestia

实时太空模拟

ssh about

• Unable to negotiate with 192.168.0.195: no matching key exchange method found. Their offer: diffie-hellman-group1-sha1

ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 root@192.168.0.195

Serial Monitor

socat /dev/ttyS0,raw,echo=0 SYSTEM:'tee input.txt | socat - "PTY,link=/tmp/ttyV0,raw,echo=0,waitslave" | tee output.txt'

screen /dev/tty.usbserial-blahblah 9600

ALSA always on mute after startup

Disabling the module "module-device-restore" in /etc/pulse/default.pa solves the "mute" problem.

tcpdump how to

https://forum.ivorde.com/tcpdump-how-to-to-capture-only-icmp-ping-echo-requests-t15191.html

https://danielmiessler.com/study/tcpdump/

Reset Your Forgotten Linux Password in 2 Minutes or Less

Grub OR LILO + rw init=/bin/bash

parted

How to align partitions for best performance using parted

parted /dev/sdb mkpart primary 0% 100%
parted /dev/sda mkpart oracle-data ext4 250GB 100%
parted /dev/sdb print

Troubleshooting

Possible Causes of Ethernet Errors

Collisions: Signifies when the NIC card detects itself and another server on the LAN attempting data transmissions at the same time. Collisions can be expected as a normal part of Ethernet operation and are typically below 0.1% of all frames sent. Higher error rates are likely to be caused by faulty NIC cards or poorly terminated cables.

Single Collisions: The Ethernet frame went through after only one collision

Multiple Collisions: The NIC had to attempt multiple times before successfully sending the frame due to collisions.

CRC Errors: Frames were sent but were corrupted in transit. The presence of CRC errors, but not many collisions usually is an indication of electrical noise. Make sure that you are using the correct type of cable, that the cabling is undamaged and that the connectors are securely fastened.

Frame Errors: An incorrect CRC and a non-integer number of bytes are received. This is usually the result of collisions or a bad Ethernet device.

FIFO and Overrun Errors: The number of times that the NIC was unable of handing data to its memory buffers because the data rate the capabilities of the hardware. This is usually a sign of excessive traffic.

Length Errors: The received frame length was less than or exceeded the Ethernet standard. This is most frequently due to incompatible duplex settings.

Carrier Errors: Errors are caused by the NIC card losing its link connection to the hub or switch. Check for faulty cabling or faulty interfaces on the NIC and networking equipment. 

Resource temporarily unavailable

  Resolution

    Check /etc/security/limits.conf and all files in /etc/security/limits.d/ for the current setting of the nproc value
    Extend the nproc value. This can be done in /etc/security/limits.conf or in any file in a file in /etc/security/limits.d/.
    Example of /etc/security/limits.conf:

https://gerardnico.com/wiki/linux/limits.conf

   Example of /etc/security/limits.conf file

*               hard    nofile          65535
*               soft    nofile          4096
@student        hard    nproc           16384
@student        soft    nproc           2047

A soft limit is like a warning and hard limit is a real max limit. For example, following will prevent anyone in the student group from having more than 50 processes, and a warning will be given at 30 processes.

@student        hard    nproc           50
@student        soft    nproc           30

References

http://www.tecmint.com/free-online-linux-learning-guide-for-beginners/

http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted

Shadowsocks

https://shadowsocks.be/9.html https://suiyuanjian.com/139.html

systemd disable servers

systemctl mask wpa_supplicant.service