Make a big file into many small files (and back again)

Adapted from this slashdot thread

Create many little files called "chunks00000" etc. from bigfile that are all just under 10MB in size: split -a 5 -b 10MB -d bigfile chunks

Put the chunks back together: cat * > newbigfile

Do the same, except with compression: tar -zc bigfile | split -a 5 -b 10MB -d - chunks

Put the compressed chunks back together: cat chunks000* | tar -zxO > newbigfile

Funny UNIX tricks from Slashdot

There was a recent story on slashdot about useless (or useful) things one can do in UNIX. Being a command line junkie, I read through virtually every comment (all 2300+ of them) to learn some new tricks. Here are some of the better ones:

Bash History

• history -c # clear history (good for preserving privacy/passwords, or check out the more precise -d option)
  
• In vi command mode, type /query and hit Enter to search history, n to keep searching backwards, N to search forwards

vimdiff

• vimdiff original_file patched_file
  
• unified format: open original file, then :vertical diffpatch path/to/diff

Encryption

• openssl aes-256-cbc -a -e -salt -in INPUT_FILENAME -out OUTPUT_FILENAME # encrypt
• openssl aes-256-cbc -a -d -salt -in INPUT_FILENAME -out OUTPUT_FILENAME # decrypt
• echo Oe lbh pna vzcyrzrag UK tbireazrag fgnaqneq rapelcgvba jvgu ge | tr a-z n-za-m # Rot 13 encrypt/decrypt
  

Others:

• sleep 8h; cat /dev/urandom > /dev/dsp # alarm clock
  
• eject -T # close cd tray if open, open if closed (useful to find out which physical machine you are logged into)
• sl # punish users who accidentally type 'sl' instead of 'ls'
  
• eposd && say 'hello' # make the computer talk
  
• :(){ :|:& };: # forkbomb (space required between { and :) (protect against this with ulimit -u)
  
• for I in $(seq 1 100) ; do echo $I; sleep .25; done | dialog --gauge "PIZZA" 6 50 100 # Pizza timer via dialog

Funny UNIX utilities

Some humorous UNIX utilities that brighten my day:

• cowsay -- get a cow, etc. to do the talking
  
• fortune -- witticisms, some more offensive than others (fortunes-off package contains more offensive fortunes, fortune -o invokes offensive fortunes)
  
• banner -- big friggin letters
  

Screen

Screen is an awesome UNIX utility that allows you to continue running commands on a remote system even after you have logged out! Magic! Here is a nice "screen for dummies" tutorial with some links to related sites at the bottom.

On the command line

• screen (will spew some text and then give a prompt) start a screen session
• screen -r reattach to screen session
• screen -d detach an attached screen session (good for logging in after network cuts out)
  

Links

• Power Sessions With Screen
• Screen ASCII cheat sheet

Tips

• Launch screen with an alias such as alias screen='TERM=screen screen' in .bashrc to avoid your backspaces getting mangled
  
• There is a ~/.screenrc file with which you can customize screen

My .screenrc

# don't mangle backspaces
termcapinfo xtermc kD=\E[3~

# no startup message
startup_message off

# set shell that launches
shell /bin/bash

# lots of scrollback
defscrollback 10000

# ignore case when searching in copy mode
ignorecase on

# make it obvious we're within screen
hardstatus string "screen%n - %h"

# vim-style bindings
bind k focus up   # cycle to the above split
bind j focus down # cycle to the below split
bind l next       # cycle to the next split
bind h prev       # cycle to the previous split
bind n split      # create a new split
bind ^N split     # create a new split
bind q remove     # remove the split
bind + resize +3  # increase split three lines
bind - resize -3  # decrease split three lines
bind _ resize max # maximize current split
bind = resize =   # make all splits equal size
bind g info       # display info on the split
bind ^G info      # display info on the split

# other (some defaults, just to remind myself they exist)
bind Q only       # get rid of all other splits but the current one
bind M monitor    # toggle monitoring of the current window
bind t title      # enter a title for this shell
bind a other      # go to most recently viewed other split
bind \' windowlist -b # go to window list
bind x kill       # close the current shell
bind ^X kill      # close the current shell
bind \\ quit      # close all shells

UNIX status commands on Solaris

Proc tools:

• pflags Print the /proc tracing flags, the pending and held signals, and other /proc status information for each lwp in each process.
• pcred Print or set the credentials (effective, real, saved UIDs and GIDs) of each process.
• pldd List the dynamic libraries linked into each process, including shared objects explicitly attached using dlopen(3C). See also ldd(1).
• psig List the signal actions and handlers of each process. See signal.h(3HEAD).
• pstack Print a hex+symbolic stack trace for each lwp in each process.
• pfiles Report fstat(2) and fcntl(2) information for all open files in each process. In addition, a path to the file is reported if the information is available from /proc/pid/path. This is not necessarily the same name used to open the file. See proc(4) for more information.
• pwdx Print the current working directory of each process.
• pstop Stop each process (PR_REQUESTED stop).
• prun Set each process running (inverse of pstop).
• pwait Wait for all of the specified processes to terminate.
• ptime Time the command, like time(1), but using microstate accounting for reproducible precision. Unlike time(1), children of the command are not timed.

Other commands:

• arch display system architecture (i86pc)
  
• uname print name of current system (SunOS)
  
• pagesize display the size of pages of memory
  
• iostat report I/O stats for terminal, disk, tape, etc
  
• vmstat report virtual memory statistics
  
• mpstat report CPU statistics
  
• busstat report bus-related support statistics
  
• lsof list open files (sockets (lsof -i), etc.) Introductory lsof examples
  
• nohup run a command immune to hangups
• kstat display kernel statistics

While navigating manpages, note that Solaris uses more to view them, which is supremely annoying. Use less with export PAGER="less" (tip found here).

Bash prompt customization

Using old UNIX machines is a pain sometimes. The 10-year-old features just don't mesh with the current ones, and it just doesn't feel right. Also, the prompt might not display any useful information. Heresy, I say! Follow these steps to make yourself feel more at home. Some information taken from this article.

To get an Ubuntu-like prompt (assuming bash is installed):

1. Edit the .bashrc file with an editor like vi
2. Add this as the last line: export PS1='\u@\h:\w$ '

Other prompt configuration options:

• \! History number of current command
• \# Command number of current command
• \d Current date
• \h Host name
• \n Newline
• \s Shell name
• \t Current time
• \u User name
• \W Current working directory
• \w Current working directory (full path)

To get easy access to some obscure directory: alias [name]="cd [absolute directory path]"

You can also edit the .login file, which executes immediately when you log in. (It might go by a different name depending on the shell... see here for details)

Use set -o emacs (the default) or set -o vi to set your command line editing mode of choice.

And remember kids, don't forget to write the other users if on a public machine! (Or maybe talk, or wall if you're the admin).

More links

• Prompt Magic
• The 256 Color Mode of xterm
• The exhaustive Bash Prompt HOWTO
  

GNU Coreutils/other useful UNIXy stuff

There are a lot of useful things in the GNU coreutils that are useful but many people (me) had no idea that they even exist... here are a few:

• sort sort the contents of a file
• -r reverse
• -f ignore case
• -n numeric
• -b ignore leading blanks
• -c check if sorted (no printout = sorted)
• uniq remove duplicate lines from a sorted file
• -c sort and count number of occurances
• -i ignore case

Of course, if you ever want any information about the coreutils: info coreutils

Simple admin commands

Here are some simple sysadmin commands:

• To create accounts for users (the nice way), the command is adduser <user> which will create an entry in the etc/passwd file and add a home directory for that user (and probably some other stuff, too...)
• userdel -r <user> delete accounts for users (the nice way) (-r means delete the home directory as well)
• passwd <user> change a user's password
  
• uptime tell how long a system has been running
• du -h <file/directory> display a human-readable estimation of how much disk space something is taking
• df -h display a human-readable report of file system disk space usage
  
• ps report a snapshot of the current processes
• ps aux (username, process ID, %CPU, %MEM, virtual memory used (pages), physical memory used, TTY, STAT (process state), date of start, time of start, command)
• pstree display a tree of processes
• arp display IP->MAC translation tables
• acpi display battery status
• shutdown -h now shut down, power down
• uname print system information (kernel version, processor, etc)
• whereis locate the binary, source, and manual page files for a command
• locate find file names on the whole system FAST (if it's not finding a new file, run updatedb to update locate's index and then run locate again)
  
• last -a show a listing of users that have logged into the system and from where
• screen screen manager (very useful for doing heavy duty stuff on remote systems... see tip #7 of this post for details)
• free -m display free and total memory in MB (RAM and swap)
• fuser -v <file> show the user, PID, access type and command accessing of an open file (v for verbose)
• lsof <file> for even more information than fuser
  

Important files/places:

• /proc directory that contains a lot of hardware and system information