Showing posts with label Command-line interface. Show all posts
Showing posts with label Command-line interface. Show all posts

How to extract or remove lines with grep?

 


In this post, we go through some examples of extracting or removing lines of a file with the grep command line tool.

Suppose we have a file text.txt with the following content.


sed command stands for stream editor
sed allows us to substitute, insert or delete, find and replace strings in a file
sed is quite useful and sed is awesome!

How to filter lines having "useful"? 

Command:

$grep useful text.txt
Output:

sed is quite useful and sed is awesome!

How to filter those lines with case insensitive? 

For example, filtering lines with sed or SED

Suppose we have an updated file text.txt with the following content.


SED command stands for stream editor
SED allows us to substitute, insert or delete, find and replace strings in a file
SED is quite useful and sed is awesome!
Command (with -i option):

$grep -i sed
Ouput:

SED command stands for stream editor
SED allows us to substitute, insert or delete, find and replace strings in a file
SED is quite useful and sed is awesome!

How to filter lines with regex patterns? 

For example, lines starting with SED and containing substitute 


Command (using -E option):

$grep -E '^SED.*substitute.*'
Ouput:

SED allows us to substitute, insert or delete, find and replace strings in a file

How to count the number of occorrences of each word in each line of a file?



Suppose we have a file - words.txt - with its content where each line is a word and we would like to count each unique word and sort them in ascending order. 

More specifically, we have words.txt as follows:

  1 foo
  2 bar
  3 foo
  4 foo
  5 bar
And we would like to get the result as follows:

  word,count
  foo,3
  bar,2

We use the following steps to approach this problem:
  1. Read and sort the content to count unique words and their counts
  2. Sort in reversing order by comparing their count values
  3. Print the count and word informatin in the "<count>,<word>" format
  4. Finally, append the column names "word,count" and print out





1. Read and sort the content to count unique words and their counts

$cat words.txt | sort | uniq -c
Note you can always check options available for a command, e.g., uniq, with --help option

$uniq --help
Usage: uniq [OPTION]... [INPUT [OUTPUT]]
Filter adjacent matching lines from INPUT (or standard input),
writing to OUTPUT (or standard output).

With no options, matching lines are merged to the first occurrence.

Mandatory arguments to long options are mandatory for short options too.
  -c, --count           prefix lines by the number of occurrences
This will give us the following results so far:

2 bar
3 foo

2. Sort in reversing order by comparing their count values

$cat words.txt | sort | uniq -c | sort -rn
where -rn indicate comparing numerical values and sort in reversed/descending order. And we get the results so far as:

3 foo
2 bar

3. Print the count and word informatin in the "<count>,<word>" format using awk command (awk is abbreviated from the names of the developers – Aho, Weinberger, and Kernighan)

cat words.txt | sort | uniq -c | sort -nr | awk '{print $2","$1}'
Now, we are almost done with the output as

foo,3
bar,2
and just need to append headers on top.

4. Finally, append the column names "word,count" and print out using header command line tool from dsutils

$cat words.txt | sort | uniq -c | sort -nr | awk '{print $2","$1}' | header -a word,count

Putting everythign together, we have the final solution as follows:

$cat words.txt | sort | uniq -c | sort -nr | awk '{print $2","$1}' | header -a word,count
with our desired output

  word,count
  foo,3
  bar,2

윈도우 터미널 자주 쓰는 명령어 모음 Windows Terminal Commands

 

  • help [command name]: 명령어 사용관련 정보 보기
  • cd: change directory (디렉토리 변경)
  • cls: clear screen (스크린 기록 없개기)
  • dir: list directory content (현재 디렉토리 목록 보기)
  • rmdir: remove directory (디렉토리 지우기)
  • del: delete file (파일지우기)
  • echo: output text (텍스트 프린트하기)
  • fc: compare and show difference between files (파일 비교하기)
  • find: find files (파일 찾기)
  • ipconfig: display IP settings (IP세팅 보기)
  • makedir: make directory (디렉토리 만들기)
  • ping: ping a network address 
  • rename: rename files (파일 이름 수정)
  • telnet: make a telnet connection 
  • move: move/rename files (파일 옮기기)
  • start .: open current folder (터미널의 현재 디렉토리 열기)
  • exit: exit command prompt (코멘드창에서 나오기)

How to turn on/off wifi using command line on Ubuntu

In case the wifi access points have been saved, it will automatically connect. 

Turn on/off wifi:
- nmcli nm wifi on
- nmcli nm wifi off

Newer version (e.g., 18.04):
- nmcli radio wifi on
- nmcli radio wifi off

How to remove / delete lines in vim?



In this post, we quickly look at how to delete lines for different use cases in the command mode (using ESC).


How to delete a singlie line?

- Move cursor to the line we want to delete
- Use one of the following commands to delete the current line

dd

D


How to delete several lines?

- Move cursor to the first line of those lines we want to delete
- Use the command,e.g., when deleting 5 lines. You can change the number 5 to whatever you want

5dd


How to delete m-th to n-th lines?

- We can use the command for deleting 111th to 222th lines

:111,222d


How to delete from current line onwards to the end of the document?

- We can use the command below

:.,$d


How to delete from the beginning of the document to the current line?

- We can use the command below

:dgg


How to delete a singlie word?

- Move cursor to the beginning of the word we want to delete
- Use the command

dw

Useful Linux commands



File Manipulation


# How to copy a file? e.g., top-words-2.sh to top-words-3.sh 

cp -v top-words-2.sh top-words-3.sh
cp -v top-words-{2,3}.sh

# How to zip a folder into a .zip file? e.g., we want to create a .zip file called zipfilename.zip by zipping all files in the folder named folder_name

zip -r zipfilename.zip folder_name


Search


# How to search files containing a certain str?

grep -iRl "str to search" ./

-i - ignore text case
-R - recursively search files in subdirectories.
-l - show file names instead of file contents portions.


grep -rnw '/path/to/somewhere/' -e 'pattern'

# only search through those files which have .c or .h extensions

grep --include=\*.{c,h} -rnw '/path/to/somewhere/' -e "pattern"

# exclude searching all the files ending with .o extension

grep --exclude=*.o -rnw '/path/to/somewhere/' -e "pattern"

# exclude a particular directory(ies) through --exclude-dir parameter. For example, this will exclude the dirs dir1/, dir2/ and all of them matching *.dst/

grep --exclude-dir={dir1,dir2,*.dst} -rnw '/path/to/somewhere/' -e "pattern"




# today's file at current directory with certain regex

find . -maxdepth 1 -mtime 0 -name "*hello*"

# change multiple file names, e.g., *.txt files to *.text

for f in *.txt;
do mv -- "$f" "$(basename "$f" .txt).text"
done

# find and remove files

# first command remove directories as well
find . -name "FILE-TO-FIND" -exec rm -rf {} \;
# second command only removes files
find . -type f -name "FILE-TO-FIND" -exec rm -f {} \;

# how to delete all files with a certain extension in current directory & subdirectories?

check only:        find . -name "*.bak" -type f
delete as well:    find . -name "*.bak" -type f -delete

# how to delete files that are not belonging to a certain pattern?

find . -type f ! -name '*.txt' -delete

# search process id using certain port (e.g., 8000)

lsof -ti:8000
netstat -tulpn | grep ':443'

Others


# start default vino server in Ubuntu

/usr/lib/vino/vino-server

# clean trash bin

sudo apt install trash-cli
trash-empty

# get full permission to a file/folder
sudo chmod a+rwx /path/to/file
sudo chmod -R a+rwx /path/to/folder

# check status/start/stop/restart a service

sudo systemctl status [service name]
sudo systemctl start [service name]
sudo systemctl stop [service name]
sudo systemctl restart [service name]

MAC Terminal Command 맥북 터미널 명령어 모음

  • 명령어 도움말 보기
    • man "command name here"
    • enter q to exit help 
  • Lesson 1:
    • ls : list of directory
    • cd : go to directory (cd ..:go to previous directory)
    • pwd : current path
    • cat 파일이름 : 파일 보기
    • head 파일이름 : 첫줄
    • tail  파일이름 : 마지막 줄
    • mkdir "폴더명" : 폴더만들기
    • rm -r "폴더명" : 폴더 삭제
    • rm : 파일 삭제
    • clear : 코멘트 화면 청소
    • cp "picture1.png" Documents 
    • cp -r Documents Documents_2 : copy directory Documents to Documents_2
    • mv "picture1.png" "Hi.png"
    • touch test_file : create file named "test_file"
    • date : 시간 프린트
  • Lesson 2:
    • nano "파일명" : 파일 마들기 내용 작성하기
    • su 사용자 이름 : Switch User
    • whoami : 현재사용자
    • history : command history
    • history -c : clean history
    • sudo su root : login as root
  • Lesson 3:
    • passwd : change password
    • sudo shutdown -h now : 
    • sudo shutdown -h now : 17:00
    • sudo shutdown -r now : reboot
    • sudo reboot : reboot
    • sudo halt : shutdown
  • Lesson 4:
    • ls -l : file permission
    • sudo chown Guest Documents
    • sudo chmod 777 Documents : 모든 권리
    • sudo chmod 755 Documents : 읽기만
    • sudo chmod 700 Documents : 오너만 조작 가능


  • Lesson 5:
    • zip ./hi.zip ./hi : hi라는 파일을 hi.zip로 압축
    • zip ./hi.zip ./hi1 : hi1라는 파일도 hi.zip에 압축
    • zip -r ./hi.zip ./Documents : 폴더를 hi.zip로 압축
    • unzip hi.zip 
    • zip -e ./Test.zip ./TestFile : TestFile문서를 암호 추가해서 zip
    • zip -re ./Test.zip ./TestDir : TestDir폴더를 암호 추가해서 zip
    • gunzip : gz 파일 압축할 때
    • tar : tar 파일 압축할 때
  • Lesson 7:
    • find . -name Test -print 2> ./ErrorLog : Test라는 폴더 찾는다 에러로그를 ErrorLog에 출력
    • find / -name Test -print 2> ./ErrorLog : Test라는 모든 폴더 찾는다
  • Lesson 8:
    • say "할말" : 기계가 할말을 읽네…
    • 1.emacs->esc+x 
    • 2.snake : snake game ,,,,, tetris : tetris game
  • Lesson 9:
    • ls /Volumes : hard drives
  • Lesson 10.
    • chmod +x Bob(Filename)
  • Lesson 12.
    • top : see all processes are running
    • top -o cpu : sort by cpu usage 
    • top then hit Shift + f, then choose the display to order by memory usage by hitting key n then press Enter.
    • ps aux : see all processes
    • ps aux | grep terminal : terminal 포함된것만 추출 
    • ps aux | more : 더 디테일 하게 보기 q로 나가기
    • kill /  killall Terminal : 
  • Lesson 13. Input and Output
    • passwd : change password
    • grep는 함수 같은 ?? 뒤에 단어 포함한 행을 반환
    • grep hi
    • hi하면 hi반환
    • afasd 아무것도 반환 안함

  • 파일 쓰기
    • echo hi > HI : hi를 HI라는 파일에 아웃풋함
    • echo hi >> ~\Desktop\HI : HI라는 파일에 hi한행 더 추가
  • 디스크 용량 확인하기
    • df : 디스크 용량 확인하기
    • df -h : 사용자가 보기 쉬운 포맷으로 보여주기
    • du -sh ~ : 자신의 홈디렉토리가 얼마 쓰고 있는지 확인
    • du -sh * : 현재 디렉토리 파일(폴더)들이 얼마씩 쓰고 있는지 확인
  • ifconfig : IP 주소 확인
  • 바로가기 만들기 
    • ln -s 기존 폴더경로 바로가기 만들 폴더경로
     ln -s /Users/guangyuan/Documents/folderPath /Users/guangyuan/folderPathLink  

  • 파일 다운로드 : 
    • curl -O http://example.com/file1.pdf (현재 경로에 file1.pdf로 다운로드 됨)
    • curl -o downloadedfile1.pdf http://example.com/file1.pdf (이름 지정해서 저장)
  • 네트워크
    • netstat -nat | grep TIME_WAIT | wc -l





How to pass multiple argument for main class in NETBEANS ?

Step 1: Right Click you project and select [Set Configuration] -> [Customize]


Step 2: Set your argument with space (in example below, we set args[0] as temp and args[1] as 1)


Step 3: Simply run your project as you did before.