Chris IJ Hwang

I am a Quantitative Analyst/Developer and Data Scientist with backgroud of Finance, Education, and IT industry. This site contains some exercises, projects, and studies that I have worked on. If you have any questions, feel free to contact me at ih138 at columbia dot edu.

View My GitHub Profile



Contents

AWK

By default, whitespace as delimiter.

Action Command
Print all columns separated by whitespace
awk '{print $0}' filename
Print first column separated by comma
awk -F "," '{print $1}' filename
Print second and third columns separated by colon
awk -F ":" '{print $2 $4}' filename
Print all columns except 2nd column separated by whitespace
awk '{$2=""; print $0}' filename
Print all columns except 2nd and 3rd column separated by whitespace
awk '{$2=$3=""; print $0}' filename

ln

Better use absolute pathname.

Action Command
Create Soft Link

        $ ln -s /full/path/to/src /full/path/link(from)/file
        
Softlink is only possible across different partitions.
Create Hard Link

        $ ln /full/path/to/src /full/path/link(from)/file
        
* . and .. are hardliked to the current and parent directory resepectively. Linux don't allow users to create hard link to directory. * No hard link across different partitions.
When deleting original file to soft link. The soft link will still exist as broken link.
When deleting original file to hard link. The hard link will still exist. Can access to the content of the original file which is already deleted.
Soft linke is very useful to storage problem The Partition 1 is almost full. The additional space for log directory in the partition 1. Then move all log files to new partition, delete the original log file. 2. Create a soft link (as the same name to the original log directory) in the partition 1 so that the link points the new log directory in the partiion 2.

Finding the number of columns in csv file


wc -l millionsong.txt
head -n 1 millionsong.txt

head -n 1 millionsong.txt | wc -c  # just count the letters in first row.
head -n 1 millionsong.txt | awk -F',' ' { print NF }'  # Right result