Thursday, August 02, 2007

My .vimrc file

Thanks to Evan Klitzke and others at Yelp; a lot of this is ripped from their .vimrc's. All hail vim!


set modeline " Respect other people's options (when a modeline is present)
set encoding=utf-8 " Use UTF-8 (8-bit variable width Unicode)

" indenting options
set autoindent " Keep the indent level when hitting Return
set smartindent " Use smart indenting (mostly useful for C/C++ files)
set cindent " Don't indent Python really poorly
set tabstop=4 " Make tabs appear four spaces wide (default is 8 spaces)
set shiftwidth=4
set noexpandtab " Use hard tabs please! Watch out for files with soft tabs
" that don't have a modeline present, especially Python files.
set fo=tcoqan " Options for formatting text (i.e. for use with gq)

" UI stuff
set showmatch " Show matching parens as they come up
set ruler " Show the column number in the status bar
set incsearch " Find as you type
set lz " Don't redraw the screen in the middle of executing macros
set nohlsearch " Highlighting search terms is _really_ annoying
behave xterm " Just in case...
set wrap " Display files with word wrap, but don't actually insert newlines
" (Set by default)... so lines don't go off edge
set lbr " Wrap only at word boundaries (default is at any character)
syntax enable " Who wouldn't want syntax highlighting?

" Only allow folds that are manually set
set foldmethod=manual

" Make our folds look like how they were when we exited vim last time.
set viminfo=s1,<1024,'1024,/1024,:1024,@1024,c,f1,%,n~/.viminfo
autocmd BufReadPost *
\ if expand("<afile>:p:h") !=? $TEMP |
\ if line("'\"") > 0 && line("'\"") <= line("$") |
\ exe "normal g`\"" |
\ let b:doopenfold = 1 |
\ endif |
\ endif
" Need to postpone using "zv" until after reading the modelines
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\ unlet b:doopenfold |
\ exe "normal zv" |
\ endif

set nocompatible " Don't act like SysV vi
set shell=/bin/bash " Use this shell to execute commands
filetype plugin indent on " ability to run different vimrcs on different filetypes
set autoread " When a file has been detected to have been changed outside of Vim and
" it has not been changed inside of Vim, automatically read it again.
set mouse=a " Have mouse operate in all modes -- disable mouse by holding the shift key
set ttyfast " Tell vim to optimize for a fast terminal; will be on by
" default if your $TERM is xterm or screen, but could be
" turned off if you use a weird terminal (e.g. 'screen-bce').
" Set 'nottyfast' for slow SSH connections.
set history=50
set showmode " show what mode (e.g. INSERT) you're in
set showcmd " Show (partial) command in status line
set backspace=indent,eol,start " explicitly makes the backspace work when at the beginning of a line
nmap <C-N><C-N> :set invnumber <CR> " Press Ctrl-N twice to toggle line numbers in the left margin

"splits
set wmh=0 " allow splits with 0 lines open
map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it

" cycling through tabs
map <C-H> <C-PageUp>
map <C-L> <C-PageDown>
imap <C-H> <C-PageUp>
imap <C-L> <C-PageDown>

set title " sets title of terminal in menubar to that of file currently being edited
set ignorecase " perform case-insensitive searches...
set smartcase " ... unless there are capitalized characters in the search pattern
set mousehide " hide the mouse when editing
set exrc " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set bg=light "this almost always looks better -- take this out or set it to dark if things look funny

" for common typos
command! Q quit
command! W write
command! Wq wq
cabbrev Set <c-r>=(getcmdtype()==':' && getcmdpos()==1 ? 'set' : 'Set')<CR>

Or, say you want soft tabs instead of hard ones (see converting tabs to spaces):
set expandtab     "soft tabs
set tabstop=2 "set tabstops
set shiftwidth=2 "set shiftwidth
set softtabstop=2 "delete this many spaces upon deleting soft tab

More stuff from Rajiv:

set directory=/var/tmp,/tmp " put all the backup stuff in tmp directories
set shortmess+=A " no swap file complaints
set nobackup " don't litter directories with swap files

" move across rows, not lines
noremap j gj
noremap k gk
noremap gk k
noremap gj j

set display=lastline " ??????????????
syntax sync fromstart " always render syntax from the beginning of the file
au! BufRead,BufNewFile * lcd %:p:h " cd into the directory of the file you're editing
set vb t_vb= " no visual bell

Friday, June 08, 2007

Configuring WPA manually

Configuring WPA is Ubuntu is no problem if NetworkManager works flawlessly. However, sometimes it doesn't. Here's how to configure WPA from the command line:
  1. Get wpa supplicant sudo aptitude install wpasupplicant
  2. Run wpa_passphrase which reads stdin and outputs the PSK that you'll need
  3. Create a configuration file, say /etc/wpa_supplicant.conf, with the the text following this numbered list
  4. sudo wpa_supplicant -B -D wext -i eth0 -c /etc/wpa_supplicant.conf (B for background, D for driver (wext is a generic linux driver, ipw2100 does not work), i for interface, c for path to conf file)
  5. You should be connected at this point. Run dhclient to request an IP address from the server. ping something to make sure you can reach the outside.
  6. (optional) This process can be automated on startup by editing the /etc/network/interfaces file appropriately
ctrl_interface=/var/run/wpa_supplicant

network={
ssid="My Network"
key_mgmt=WPA-PSK
psk=<generated passphrase>
}
As always, look at the man pages for more details: man wpa_supplicant and man wpa_supplicant.conf and man wpa_passphrase

Sunday, June 03, 2007

dpkg's --force-architecture option is your friend on amd64 systems

For example, to install picasa for linux

sudo dpkg --force-architecture -i picasa*.deb (-i for install)

If you don't do this, you'll get a warning about the package being i386-based and therefore unusable on your system.

Sunday, May 27, 2007

HOWTO format and mount an external hard disk

When I initially tried to back up my music library onto my Windows-created FAT32 external hard drive, the terminal threw a bunch of "Invalid argument" errors and failed to copy those respective files. It turns out that windows-formatted drives can not have certain characters in their filenames ( \ / : * ? < > | ), so I decided to reformat the drive in a Linux-friendly fashion.

Instructions
  1. Connect the target drive to your computer (let's assume it's recognized as /dev/sdb1)
  2. Reformat it using the ext3 file system (Linux default) by either...
    1. sudo mkfs.ext3 /dev/sdb1 (lots of options with this tool... read the manpage )
    2. Using gparted (install with sudo aptitude install gparted), a GUI-based tool
  3. Create a mount point sudo mkdir /external
  4. Mount the drive sudo mount /dev/sdb1 /external
  5. Use the drive!
  6. (optional) If you want the drive to be recognized in a certain way upon boot, edit the /etc/fstab file to your liking (Ubuntu 7.04 recognizes it anways)
  7. (optional) To unmount the drive sudo umount /dev/sdb1
Useful commands
  • sudo fdisk -l List partition tables for all devices (mounted or not)
  • df -h Human-readable output of system disk space and usage
Tips
hd* = IDE drive
sd* = SATA drive

Information used from these sources

Saturday, May 26, 2007

HOWTO organize a music library of 10,000 songs

Several days ago, I set out on an epic quest to organize my music library of roughly 10,000 songs. It's a daunting task, and one I've put off for a while, because my files are in several different places in several different formats with several different ID3 tag/naming standards. And there are more problems too: identifying mislabeled files I got from other places, weeding out corrupted files, etc, etc etc. Suffice it to say that it's been an intense few days with me in a zen-like state of focus trying to get everything in its right place. So here's the short version of how I did it.

The first (major) problem I faced was the massive swaths of files that had incorrect ID3 tags. For this problem I needed a competent ID3 tagger. About a year ago I asked on ubuntu-users what the list's tool of choice was. Suggestions ranged from Quod Libet to Foobar2000 on WINE (outstanding reviews: "best program ever" and "the only program i miss since switching to ubuntu") to gtkpod to EasyTag (mixed reviews: "tolerable" to "great") to id3v2 (command line based) to normal music players like amarok and banshee. There is also a Python module for accessing and editing ID3 tags, if I felt like scripting some things myself.

Eventually I settled on Picard, the tagger that Musicbrainz (a great music metadata site) recommends. Thanks to Ubuntu and the Picard team, installation is as simple as adding deb http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/ubuntu edgy musicbrainz to /etc/apt/sources.list (even if you're not running edgy), importing the PGP key with wget http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/public.key -O- | sudo apt-key add - and then a sudo aptitude update and sudo aptitude install picard. There are only a few limited operations Picard can perform, but they are very powerful. Picard interacts with the Musicbranz database to identify tracks in an album-based scheme. It correctly identified and summarily tagged more than 95% of the CDs I threw at it, and I listen to some pretty obscure/out there music. The ones it couldn't recognize were mostly tracks that didn't have a lot of existing context to go with them, and although I don't know exactly what information Picard looks at to determine the ID3 tags, I suspect that was the problem. There are a few problems with Picard, however. First of all, the UI sucks. Second, there are very limited capabilities. I would like to see, say, options that move files around on the file system depending upon how Picard tags (or fails to tag) them. The albums that have their ID3 tags written by Picard don't seem to be organized in any meaningful way, except in the current running instance of the program, where they are added to the bottom of the list (which is also annoying because of the need to keep scrolling to the top to add files to tag and then to the bottom to click and drag the files it missed and back again). Increased automation in the program would be nice, too, as I found myself performing the same operations over and over again. There are also minor issues like it opening up a new browser tab on each database lookup (leading me to have firefox open with ~100 or more tabs at some points). Fortunately for the future Picard user, I saw most of these issued acknowledged and marked for future improvement in the documentation.

The second (major) task ahead of me was conversion of all of my files into the near-universal mp3 format (yes, I know I should be gunning for ogg, but you can't use those files on your iPod... well, without the free firmware you can load onto it... another day, another day). For m4a files, I used the script found at the end of this thread which happily copied my ID3 tags as well. For ogg files, I would need something more general. At first glance, SoX seemed like a reasonable tool, but it needs to be recompiled for mp3 support. I also found some other scripts, but those didn't copy the ID3 tags, which made them unacceptable for my purposes. I asked in the freenode #ubuntu channel regarding my quandary, and someone mentioned Soundconverter as an option.

Soundconverter is a simple tool, but gets the job done. I installed it with a simple sudo aptitude install soundconverter. It provides a nice drag-and-drop interface that doesn't require knowledge of ID3 tag layouts or encoding/decoding details. Most of the options are located in the Edit->Preferences menu; the rest of the program is idiot-proof. Just drag the files, and hit Convert. It does do some annoying things, like saving files in URL-encoded names (example: files in folder "Cannibal Corpse" with the option to save in the same folder on were saved in a newly created folder, "Cannibal%20Corpse"), but generally works well.

Things were a bit more complicated than this, of course. I was making nightly backups of my changes to a local FTP server, wrote up some Python scripts to assist me in my task of keeping track of where everything was going and manually edited some outlying tags in amarok -- but now my music library is (almost) officially organized. Success! Now to just incorporate all of my friends' libraries...

Some cool links I found along the way
Some relevant command line tricks
  • find -type f ! -iregex '\(.*.flac\|.*.ogg\|.*.m4a\|.*.mp3\)' -delete -or -empty -delete

Friday, April 20, 2007

Feisty Fawn customization

Here are the things I did after I installed Feisty Fawn Ubuntu 7.04 off of the alternate install CD on my Compaq X1000 laptop/other systems.

If on an AMD64 processor system
  • Follow these instructions (and run the script) to get mplayer, flash and java support for firefox. Update all launchers and shortcuts to launch firefox32 instead of firefox
  • Skip the remaining instructions that are italicized

Install add-on packages (by the way, all except the first can be done in one command... just append all the packages you want to install after "install")

  • Edit /etc/apt/sources.list following these instructions to add extra repositories
  • Install full Vim support (without this, no help, syntax highlighting, etc.) sudo aptitude install vim-full
  • Install Flash sudo aptitude install flashplugin-nonfree
  • Install media codecs sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse w32codecs
  • Install Amarok sudo aptitude install amarok
  • Install Java sudo aptitude install sun-java6-jre sun-java6-plugin sun-java6-fonts
  • Install Azureus sudo aptitude install azureus (or, alternatively, ktorrent)
  • Enable DVD support
  • Possible user programs: xmms, xchat, mplayer plugin, vlc, skype, picasa, eclipse, google earth, beagle, clipboard manager
  • Security programs: sudo aptitude install nmap kismet wireshark airsnort aircrack-ng john
  • Install Beryl (nVidia) (I also find it convinient to add a sessions entry for beryl-manager so the stone shows up in the tray)
Trick out Firefox
Trick out the Terminal (and partners in crime)
  • Either create a new profile or edit the current profile
  • Disable General->Show menubar by default in new terminals
  • Set Colors to aesthetically pleasing values
  • Set Effects->Transparent background and set the bar about 2/3 of the way to Maximum
  • Set Scrolling->Scrollback to 2000 lines
  • Edit ~/.bashrc
  • Edit ~/.vimrc
  • Put the line set editing-mode vi in ~/.inputrc (the configuration file for readline... this also makes the set -o vi in the .bashrc redundant)
Trick out GNOME
  • Fix the Right Alt key System->Preferences->Keyboard->Layout Options->Third level choosers->Press Menu Key to Choose 3rd Level
  • Change the Desktop Background by right clicking on it (good backgrounds here (OSX Tiger) and here (fractals) and at DeviantArt)
  • Map the Windows keys to do something cool in System->Preferences->Keyboard Shortcuts (I think "Run a terminal" and "Hide all windows and focus desktop" are good candidates)
  • Enable visual system beep System->Preferences->Sound->System Beep->Flash Window Titlebar
  • Make the panels transparent Right Click on Panel->Properties->Background->Solid Color and adjust the slider appropriately (about halfway looks good)
  • Add the system monitor to the panel Right Click on Panel->Add to Panel->System & Hardware->System Monitor->Add to Panel (I enable all but "Load"... wtf is load?)
  • Disable launching of unnecessary programs at startup (such as Evolution Alarm Notifier) Preferences->Sessions->Startup Programs
  • Save your session in a desireable state: gnome-session-save (don't use the automatic option in Prefs->Sessions... it causes problems!)
  • Put the panels in a desirable place by dragging them across the screen
  • Get 4 workspaces Right Click on Workspaces on Panel->Preferences->Number of Workspaces
  • Disable Rhythmbox launching upon iPod detection (Amarok rulez!) System->Preferences->Removable Drives and Media->Multimedia->Uncheck "Play Music Files when Connected" and also Uncheck Storage->"Browse removable media when inserted"
  • Enable the text-based location bar in Nautilus (in favor of the button-based one) by hitting the notepad button on the top left in a Nautilus window
If on a laptop...
So what's so different about Feisty (as opposed to Edgy)?
  • All the latest stable versions of software in the repos, obviously
  • GNOME network manager manages wireless devices by default (and keyring support to boot)
  • Fool-proof support for installing restricted drivers, media codecs, etc.
  • Nice command line assistance if you try and run software that's not installed (prints a "use this repository and this command to get the software"-type message)
  • Easy enabling of "Desktop Effects"
  • And more

Thursday, April 19, 2007

iPod + Amarok

Using your iPod with Ubuntu is disgustingly easy. As a Linux user, I was almost disappointed... no config files to edit? No dependencies to chase down? No bugs to fix? Nope, Ubuntu and the Amarok music player make everything simple. Here, I will detail how to use Amarok with your iPod on an Ubuntu system. I'm using Ubuntu Feisty Fawn 7.04 (just installed today!), a fourth-generation iPod and Amarok version 1.4.5.

First install Amarok with sudo aptitude install amarok (and with aptitude, not apt-get). If you want Amarok to be useful with most file formats (such as mp3) it would be a good idea to install proprietary codecs as well: sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse libxine-extracodecs w32codecs

Now that those monster installs are finished, you can go ahead and connect your iPod. If it automatically shows up on your desktop as a little iPod icon, great! If not, check out
this thread for more detailed info. When I popped in my iPod, Amarok did me one better and recognized the device, popping up a little dialog asking me what kind of player it was. When this happens, you should select Apple iPod Media Device in the drop-down box. There are a few more options you can set regarding the device -- check out Settings->Configure Amarok->Media Devices in the Amarok toolbar menu. (If this explanation was too complicated for you, there's a guide with nice pictures here describing a slightly different way of going about things.)

To add files to the iPod, go to the Collection tab on the left and highlight whatever you want to transfer to the iPod in the left frame. Right click on it and select Transfer to Media Device. Then go to the Devices tab (with the little iPod logo) and hit Transfer at the top to transfer your files. When you're done transferring files, hit Disconnect at the top. If you're lucky, Amarok will unmount the device for you. If not, you'll have to right click on the desktop icon of the iPod and unmount it that way. I asked in the Amarok IRC channel if it was possible to sync the iPod and apparently it's not... for now, just delete everything off of the iPod and put the new stuff back on, as time consuming as that may be.

There are some other things you can do in Amarok to tweak the settings for the iPod. In the Devices tab under the (usually hidden) iPod menu you can set the model of your iPod. This is used for advanced support such as photo and video copying.

It's annoying that Rhythmbox, the default GNOME music application, launches whenever you connect your iPod. To disable this, go to Preferences->Removable Drives and Media->Multimedia->Portable Music Players and disable Play music files when connected. Just doing this will make a window open up in Nautilus every time you insert a removable drive, including an iPod. To turn this off, go to the Storage tab and uncheck Browse removable media when inserted.

Links

Friday, April 06, 2007

My .exrc file (vi/m) and then some

What follows is my .exrc file for various versions of vi, and some more related stuff. I have yet to look through the vim documentation exhasutively to see if there are any more useful options -- this is just stuff I have gleaned from random places on the Internet and from friends.

Plain vanilla vi (as on Solaris)
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns set showmatch " show matches of ()'s, {}'s, and []'s
set showmode " show what mode (e.g. INSERT) you're in
set autoindent " indent to the extent of the previous line

Vim:
syntax enable " enables syntax highlighting
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns
set title
" sets title of terminal in menubar to that of file currently being edited
set expandtab " tabs aren't really tabs, but many spaces
set softtabstop=4 " delete four spaces when you delete a "tab"
set ignorecase " perform case-insensitive searches...
set smartcase " ... unless there are capitalized characters in the search pattern
set mousehide " hide the mouse when editing
set exrc " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
set bg=light "this almost always looks better -- take this out or set it to dark if things look funny
set nohlsearch "no annoying search highlighting

" for common typos
command! Q quit
command! W write
command! Wq wq

Vim Python
set autoindent " indent to the extent of the previous line

Vim C/C++/Java
set cindent " indenting for C-style languages
set showmatch " show matches of ()'s, {}'s, and []'s

For Text (Vim 7.0 only)
setlocal spell spelllang=en_us " English(US) spellcheck on (Vim 7.x only)
set tw=0 wrap linebreak " don't split words between lines, only wrap at end of word
"lots of abbreviations (varies depending on the subject material)
ab STIA Science, Technology and International Affairs

How to produce nicely formatted C code in Vim (From p.221 of Learning the vi editor):
set nocp incsearch
set cinoptions=:0,p0,t0
set cinwords=if,else,while,do,for,switch,case
set formatoptions=tcqr
set cindent
syntax on
source ~/.exrc

Options set when vim 6.4/7 starts on my system (Ubuntu Linux):
helplang=en " language to display help in
ruler " show row and column in bottom right
ttyfast " indicates a fast terminal connection
viminfo='20,"50 " configuration info
history=50 " number of : commands and previous search patters to be remembered
scroll=32 " number of lines to scroll with CRTL-U and CTRL-D commands
ttymouse=xterm2 " mouse codes
backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
fileencodings=ucs-bom,utf-8,latin1 " list of character encodings considered when starting to edit a file
printoptions=paper:letter " controls format and output of printing
runtimepath=~/.vim,/usr/share/vim/addons,/usr/share/vim/vimfiles,/usr/share/vim/vimcurrent,/usr/share/vim/vimfiles/after,/usr/share/vim/addons/after,~/.vim/after " list of directories which will be searched for runtime files
suffixes=.bak,~,.swp,.o,.info,.aux,.log,.dvi,.bbl,.blg,.brf,.cb,.ind,.idx,.ilg,.inx,.out,.toc " Files with these suffixes get a lower priority when multiple files match a wildcard.

Cool stuff that I have seen various places but I didn't find particularly useful (maybe someone else will):
set list "replaces all EOLs and TABs with $ and ^I
"auto-complete of brackets
inoremap [ [] <left>
inoremap ( () <left>
inoremap { {} <left>
inoremap " "" <left>
inoremap ' '' <left>

To see the options vi(m) has enabled, enter :set
To get help with options in Vim, enter :help set or :set all for regular vi
Or just :help for the help contents... Vim has excellent documentation

Links

My .bashrc file

export TERM="xterm-color" # xterm terminal emulation (with color -- be sure to put at top of file on Ubuntu)
export PYTHONSTARTUP="$HOME/.pythonrc.py" # python startup file
set -o vi # sets vi command-line editing mode
bind -m vi-command -r 'v' # so that every time you hit v in command mode, an editor doesn't launch
ulimit -c unlimited # dump core files, no matter how big they are
export EDITOR="vi" #use vi as the default editor for some commands, like "fc"

alias screen='TERM=screen screen' # workaround for screen backspace bug

alias d="date '+%r -- %A, %D'" # human-readable date output

alias ll="ls -lh" # long listing (with human-readable file sizes)
alias lr="ls -ltrh" # list by reverse modification time
alias la="ls -Ah" # list hidden files (except implied . and ..)

alias <name>="cd <important>" # get to important directories fast
alias <name>="ssh <login>@<important>" # log into oft-used remote machines fast

# prevent myself from doing stupid things with vi
function vi {
if [ ! -e "$1" ]; then
if [ "$1" == "" ]; then
vim
else
vim "$1"
fi
elif [ -d "$1" ]; then
cd "$1"
elif [[ -c "$1" || -b "$1" || -p "$1" || -S "$1" || ! -r "$1" ]]; then
file "$1"
else
vim "$1"
fi
}

The following I only enable on Solaris systems:
export PAGER="less" #uses less to view manpages instead of more

Links:

Wednesday, April 04, 2007

Common Bash Tasks

I don't use Bash for much scripting. Usually, if I have to open up a file to write a script, it will be done in Python. However, the less I have to do that for mundane tasks that can be accomplished on the command line with Bash, the better. This entry may not have much in it now, but every time I do something cool/useful/time-saving in Bash from now on, it'll go on here.

Common Tasks

  • Rename *.foo files to *.bar files: for i in *.foo; do mv $i ${i%.foo}.bar; done (BashFAQ)
  • Rename foo.* files to bar.* files: for i in foo.*; do mv $i bar.${i#foo.}; done (also check out the perl regex-based rename Linux utility)
  • Do something to multiple arguments: for i in arg1 arg2 arg3; do echo $i; done
  • Print 0 through 9 on separate lines: for i in {0..9}; do echo $i; done
  • Flatten output onto one line: <Multi-line output> | xargs (10 habits)
Links

Finding and Killing a Process in GNOME

There are a few different ways to find and kill a process in Linux. Using the GNOME interface, you can call up the System Monitor (similar to the one on Windows XP) under System->Administration->System Monitor. Under the Processes tab, right clicking on a process will give you a variety of options: stop/end/kill/continue the process, change the priority (nice), display a memory map (pmap) and display open files (lsof -p <procID>).

Embedding the system monitor in a desktop panel is useful for constantly monitoring the system and quickly determining if and where there is a problem. Do this with Right Click on Panel->Add to Panel...->System & Hardware->System Monitor->Add. The display provides graphs of Processor, Memory, Network, Swap Space, Hard disk and Load activity (not quite sure what load is, perhaps some metric computed from all the others? I don't use it, personally).

On the command line, list processes with the ps command or ps -e for all users' processes. Pipe that through grep (i.e., ps -e | grep firefox) to find a process ID of the target process (if you know the exact name of the program, you can also use pidof firefox-bin). To kill a process, use the kill <procID> command (with the -9 flag if you want to kill it unconditionally). Alternatively, if the process uses a gui, you can use xkill to merely click on the gui to kill the corresponding process. There is also killall if you want to kill by process table name (i.e. killall firefox-bin).

pgrep and pkill are two very useful utilities, especially for dealing with processes with long command line invocations. pgrep -fl -u danny "ruby.*asdf" will print the process id and name of every process owned by danny whose command line invocation matches the passed in regular expression. The -f argument tells it to match against the full command line invocation instead of just the process name. pkill -fu dannyc "ruby.*asdf" will kill those same processes (for the life of me, I can't seem to coax similar behavior out of killall).

The top command provides an integrated way of finding and killing a process on the command line. If the process-to-be-killed is using a lot of system resources, it will appear towards the top of the display. Hit k and then the process ID to kill the process. Top comes with a lot of other functionality -- hit h for help.

UPDATE: I recommend using htop instead of top, which is top with less suck. It has lots of good features like color, mouse support, a more responsive interface, a tree-based process view, etc.

Wednesday, March 14, 2007

Google Toolbar 3 has made my life 1.7% better

Google Toolbar Version 3 rules. It is a tiny little piece of technology that has made my life significantly better since I started using it. Linux users will have to go the extra mile (or in this case, inch) to make sure it works (nothing new...). See this thread and Google's toolbar for Linux page for instructions for older builds; apparently it appears to be playing nice now.

The coolest feature of the Toolbar is the ability to make custom search buttons. Going to any page with a search bar, right clicking, and selecting "Generate Custom Search" will create a nice little button on the Toolbar (corresponding to a site's favicon). Using this, you can search with the Toolbar as if you were on the page you generated a search for, the effect being that you never need to visit the page to use their search engine -- you can search from any arbitrary page! It's also useful for doing the same search (i.e. "Donnie Darko") over several sites you have a custom search (i.e. RottenTomatoes, IMDB, Wikipedia). If you don't know what buttons you want on your Toolbar, Google has provided a nice grab bag to choose from. My custom searches (as of right now) are:
Some more very useful features include enabling GMail to launch every time a mailto: link is clicked (Finally! No more surprise Evolution/Outlook/etc. "setup page" launches!), an icon indicating if you're signed in to your Google account, and some keyboard shortcuts (more would be nice... especially for accessing custom buttons) like Shift+Enter for an "I'm Feeling Lucky" search (keeping in the spirit of eliminating useless page visits). There are more features that I don't use (but many do) detailed on the features page. Some of these are pretty cutting edge and have generated a considerable amount of controversy, especially AutoLink. Others, such as the RSS subscribe and spellcheck, have already been implemented in Firefox 2. In any event, you should get it. Now.

A couple of things that could use improvement:
  • More keyboard shortcuts (as mentioned above)
  • Web spam reporting buttons, analogous to the "Report Spam" button that Gmail has... perhaps Google can build a database of splog, etc. sites to blacklist, or maybe work with other search engines to do so?
  • Sometimes the "Generate Custom Search..." option doesn't work... amazingly, Blogger is one of those examples... when trying to do a custom search on a particular blog, I get this (wtf?): "Information Unavailable The custom button cannot be installed. Custom button values out of range."

Wednesday, March 07, 2007

Which Ubuntu version am I using, again?

On the command line: lsb_release -a

Alternatively, look in one of these two files for the answer (links to sources where I found info): /etc/lsb-release or /etc/issue

Also, find out if your install is 64-bit via uname -m (thread1, thread2).

Yahoo! Pipes and its Shortcomings

For the past couple of days, I have been fooling around with Yahoo! Pipes. It's a tool and an interface for aggregating and/or manipulating RSS feeds. It's an interesting experiment and has been receiving some hype, most notably from Tim O'Reilly who called it a "milestone in the history of the internet." So how useful/powerful is it? My assessment is: right now, not very. There are several problems that are keeping it from being a useful (or sometimes even usable) tool at present.

The first problem is it is SLOW. When manipulating the first one or two modules, things run fine, even to the point of web 2.0 AJAXy nirvana. The scrolling is snappy, the debugger (which is a well-thought out touch) and DHTML animations look nice, and the interface is responsive. However, after that things start crawling. And after 10 or so modules, it slows to a why-am-I-even-bothering speed (and I'm on a pretty damn fast MIT connection right now with a capable PC).

The second problem are the all-too-frequent site outages. A recent blog post by the development team humorously noted: "Our apologies for the unscheduled downtime this evening. As a reminder, todays (sic) post was sponsored by the letters “b” “e” “t” and “a”."

The third problem is the support of only RSS inputs. But judging from the Pipes developers' posts, however, this problem looks like it will be overcome in the future.

So what can one do right now with Pipes? The canonical example has been putting the RSS feed of the New York Times through a Flickr filter, the result being that you can see the NYT stories in photo form. Which brings us to problem number four: your results are only as good as your data. While a lot of these photos are applicable to current events, others are tangential and some are just irrelevant. Kind of makes one think that a well-implemented semantic web isn't such a bad idea, after all.

Another thing one can do with Pipes is what I did with my first pipe, Ray's Deals. It simply takes several RSS feeds from bargain electronics sites and aggregates them into one feed. I then made the feed (or, more precisely, the titles of the items of the feed) searchable. The utility of this Pipe is that Ray now only needs to subscribe to the RSS feed from this pipe instead of several RSS feeds. (Sidenote: I actually added way more modules than necessary here: instead putting each URL in a separate Fetch module, I could have just had multiple entries in a single Fetch module... live and learn, I guess.)

That's about all the useful things one can do with Pipes at present -- my subsequent, more adventurous, Pipe experiments were fraught with frustration and, ultimately, failure. This was in part due to Pipes problem #5: the limited amount of operators that Pipes provides. There are currently no functions for performing the most basic of operations, such as generating random numbers (see the failure of my Pipe Random Jokes (Incomplete)) or extracting specific information from a feed. There are some workarounds for things like selecting a particular item from a feed, as my second hackish pipe, Selection Experiment 1, demonstrates. To select a certain oldest item, I sorted the feed by Date and then cut off the feed after a certain number of items (6). I then reversed the feed and cut off all but the first item, thus producing the selected item. Wow, that's a lot of work. Not to mention the limitations of some of the other operators, such as Sort not taking in a key value to sort by, etc. To be fair, it's still a beta product and the selection and power of modules will no doubt grow with time.

There are other concerns too, some of which are voiced in the responses to Tim O'Reilly's post. Among these are IP concerns -- is your mashup your property, or the feed providers' partial (or full) property? Also, there is the question of accessibility: will Pipes be a place that democratizes mashups, or is it just for geeky developers (who might be better off coding their own specialized apps anyways)?

All in all, Yahoo! Pipes looks like a tool that has some potential. It has a nice interface (when it's not super slow), the concept is fresh, and it's relatively easy to use (again, when it works). Its current shortcomings, however, make it little more than a novelty item. Once these get addressed, I'll be willing to give it a second try.

Tuesday, March 06, 2007

Connecting to servers in Ubuntu with "Connect to Server..."

I don't know how this has eluded me for so long, but there is actually a very nice interface built into Ubuntu that allows you to connect to servers using any of the following protocols: ftp, ssh, windows share, WebDAV (http), Secure WebDAV (https), and more. This is a nice alternative to using third party software like FireFTP (even though that program rocks) or the command line.

Simply pulling down the Places->Connect to Server... option in GNOME opens a dialog box which allows you to enter the appropriate information about the server you want to connect to. After the connection is successful, a folder representing the remote file system will appear on both the Desktop and in the Places menu. Another nice feature is that the folder even has some little letters on it telling you what protocol it is connecting over. You can now drag and drop files via the Nautilus interface, as if the directory was local. How convenient!

This tip found via the free online book Ubuntu Linux Bible. (Reviews on Amazon) By the way, this is an incredibly comprehensive reference on all things that an end user might want to do on Ubuntu. It also has sections on setting up subversion, using LaTeX, setting up other servers such as Apache, multimedia usage, and just about everything else. UPDATE: Whoops! This actually got posted by mistake by Lifehacker... the book has been taken down... it's still a good book, though!

UPDATE: For some more (unrelated) GNOME fun, right click on an icon and select "Stretch Icon" to blow up the icon to a cartoonish size. Click "Restore Icon's Original Size" in the same menu if you get tired of it (which you will, after 2 seconds).

Wednesday, February 14, 2007

whois

The WHOIS database is a nice way to determine contact information for a web site. Query the site with whois <site> It shows the name and address of the registrant, through who he/she registered the site, a technical and administrative contact, and the domain servers corresponding to the site. Use it along with dig, host, ping, etc. (and if you're feeling naughty, nmap) to get information about a site.

Links

Saturday, January 20, 2007

UNIX status commands on Solaris

Proc tools:
  • pflags Print the /proc tracing flags, the pending and held signals, and other /proc status information for each lwp in each process.
  • pcred Print or set the credentials (effective, real, saved UIDs and GIDs) of each process.
  • pldd List the dynamic libraries linked into each process, including shared objects explicitly attached using dlopen(3C). See also ldd(1).
  • psig List the signal actions and handlers of each process. See signal.h(3HEAD).
  • pstack Print a hex+symbolic stack trace for each lwp in each process.
  • pfiles Report fstat(2) and fcntl(2) information for all open files in each process. In addition, a path to the file is reported if the information is available from /proc/pid/path. This is not necessarily the same name used to open the file. See proc(4) for more information.
  • pwdx Print the current working directory of each process.
  • pstop Stop each process (PR_REQUESTED stop).
  • prun Set each process running (inverse of pstop).
  • pwait Wait for all of the specified processes to terminate.
  • ptime Time the command, like time(1), but using microstate accounting for reproducible precision. Unlike time(1), children of the command are not timed.
Other commands:
  • arch display system architecture (i86pc)
  • uname print name of current system (SunOS)
  • pagesize display the size of pages of memory
  • iostat report I/O stats for terminal, disk, tape, etc
  • vmstat report virtual memory statistics
  • mpstat report CPU statistics
  • busstat report bus-related support statistics
  • lsof list open files (sockets (lsof -i), etc.) Introductory lsof examples
  • nohup run a command immune to hangups
  • kstat display kernel statistics
While navigating manpages, note that Solaris uses more to view them, which is supremely annoying. Use less with export PAGER="less" (tip found here).

Thursday, January 04, 2007

HOWTO Starcraft/Brood War on Ubuntu Linux Edgy Eft with Wine

One of the (very very very) few things for which I use my Windows partition anymore is playing Starcraft /Brood War. Of course, as anyone that dual-boots knows, restarting your machine just for one Windows application and then back to Linux is highly annoying and time-consuming. One solution to running Windows apps on Linux is Wine, which I will describe how to configure to play SC/BW in this post. I'm using Ubuntu 6.10 (Edgy Eft). Thanks to JTerry for his help with this.

First, you're going to need a few things. Open a terminal and run sudo apt-get install wine to get wine. After that finishes, insert the Starcraft CD. cd to whatever cdrom drive you put the cd in (under the /media/ directory). Use Wine to setup SC with wine setup.exe. You'll get the install screen for SC -- enter your CD key and install the software. The SC executable will be installed under the path of ~/.wine/drive_c/Program\ Files/Starcraft/starcraft.exe. If you try and play SC now, you might notice that the sound does not work. Run winecfg and go to the Audio tab. Set Hardware Acceleration to Emulation. The sound should now work fine. If you want to play Brood War, read on; if not, skip to the next paragraph. Eject the SC CD and insert the BW CD. Again, cd to the appropriate /media/ directory and run wine setup.exe. To play SC/BW, run wine ~/.wine/drive_c/Program\ Files/Starcraft/starcraft.exe.

Playing online over Battle.net requires a little bit more work. First, you'll need the proper fonts that Battle.net uses... sudo apt-get install msttcorefonts for this. You'll also need the newest patch for Battle.net which updates BW to version 1.14. Get this patch here (other patches, for instance for regular SC, can be found here). Use wine to run the downloaded file, BW-114.exe. An updater should run, after which you'll be able to play online using Battle.net. It works, but all is not perfect -- the display of the Battle.net interface is kind of screwed up (it never refreshes when you change views). However, if you're familiar with the Battle.net interface you should be able to get into a game after which SC/BW runs without a hitch.

Some tips: tweak your mouse sensitivity and acceleration to play SC/BW. It runs at a low resolution and your cursor will be uncontrollably flying all over the place if you don't. Change this in System->Preferences->Mouse->Motion. Also, sometimes (rarely) the screen resolution doesn't change back to what you had it at before playing SC/BW. Change this in the System->Preferences->Screen Resolution menu. If you hear a lot of static from your speakers, try switching from OSS to ALSA in winecfg->Audio.

More information on this topic can be found at the Wine Application Database page for SC/BW.

Update: here's another good page on Ubuntu forums

Another update: yet another good page from the Ubuntu community detailing how to enable the 'no CD' patch

Monday, January 01, 2007

Why default settings on your wireless router is a BAD thing

Lots of people recognize that 802.11 wireless networks are really cool and want to have them. They go to Best Buy, grab a Linksys router, run home and, without going through the standard setup, plug the router into their network connection and turn it on. Hopefully you are not one of these people because these people leave their router completely vulnerable to whoever wants to take control of it.

Here, I will show how to discover one of these networks, how to get in to it, and what you can do once you're in command. I will be using the Linksys WRT54GL as an example, but the lessons here are very applicable to other makes and models as well.

First, the discovery. You're going to need a tool like kismet for this. If you're on Ubuntu Edgy Eft (like me), a simple sudo apt-get install kismet will install it on your machine. If you don't have apt-get or a comprable tool (such as yum on Red Hat flavors), you'll have to download and compile it from source yourself. Once you have kismet, edit the /etc/kismet/kismet.conf file. Edit the line that begins with "source=" to whatever is appropriate. The first value should be what comes up after the Nickname section of this command: iwconfig | grep Nickname. The second value should be the desired wireless interface (something like eth1... also on that same line of the previous command) and the third value is whatever you want to name this interface. Mine is source= ipw2100,eth1,wireless. More info is in the README under "Capture Sources." Fire up kismet with sudo kismet. Any detected networks that have the "F" under Flags (and, with color enabled, the line should come up as red) are networks with factory-default settings. These routers are often named "linksys" or "NETGEAR" or the like. If you want to change around the interface in kismet, edit the /etc/kismet/kismet_ui.conf file.

Now that you have your target, log onto the network and (hoping that this isn't a honeypot) point your browser to 192.168.1.1 (this IP address may vary depending on the make and model of the network that you're hacking... NETGEAR routers, for example, are 192.168.0.1). When prompted for a username and password, enter admin/admin. These are factory-set values that can be found for other models by searching on the Internet. For example, here and here (and here for D-link and here for Belkin). Congratulations, you're in.

What to do now? Well, as far as the router is concerned, a lot of things. Change the broadcasted ESSID of the network to "lol u R h4cked!" under Wireless->Basic Wireless Settings. Enable a password in Wireless->Wireless Security so that the legitimate users of the network will not be able to log on. In Access Restrictions, set the router so that it ceases to operate for five hours on Tuesdays. Etc. Of course, there are more things that you could do by exploiting the actual network itself, but that's beyond the scope of this entry.

Saturday, December 30, 2006

HOWTO scrub iTunes' DRM off of your purchased music with QTFairUse

DRM sucks. A lot. Case in point: I just received a $25 dollar gift certificate from a loving relative who wanted me to buy some music. I summarily went to the iTunes Music Store and downloaded about twenty-five bucks worth of Cannibal Corpse songs. All was good, until I tried to play the songs on another music player. Surprise! The .m4p files are a iTunes-only, DRMed format. That is when I resolved to immediately blog about how to get around it (a mild form of protest, I know).

(Disclaimer: doing this may be violating the DMCA. But if you want to stick it to the man and protect your fair use rights, read on. Besides, the DCMA sucks, too.)

First, you're going to need Windows (sorry... I guess you could always use Wine or Virtualization inside Linux, but I don't have that set up yet -- I'm just dual-booting Ubuntu Edgy and Windows XP). Next, download QTFairUse which is a utility that will scrub the DRM from your music files. (There is also another utility for doing this called myFairTunes and an older one that doesn't work anymore called JHymn). Unzip the QTFairUse archive and launch the .exe file to start the program. Use the drag-and-drop interface to select which songs you want scrubbed and then click the button at the bottom to begin the conversion. You now have .m4a-format, DRM-free audio files!

UPDATE:
Neither of these programs work under Wine... neither are in the Wine Application DB... looks like I'll have to stick with Windows (ugh) for now...

QTFairUse throws a "ImportError: MemoryLoadLibrary failed loading win32api.pyd" when using the console version and an "ImportError: MemoryLoadLibrary failed loading win32gui.pyd" when using the GUI version

myFairTunes sets up fine but throws this error when I run the application under Wine: "install the Windows version of Mono to run .NET executables" ... searched Google and it seems like this is a long way from being fixed

UPDATE 2:
new links:
Be sure your computer is 'authorized' to play 'protected' files on iTunes or this won't work.