Making Windows XP less Painful

That is, making Windows XP more like Linux. Occasionally, I need to boot into Windows XP to use some application that won't run with WINE on Linux. Examples include QTFairUse and the drivers and software included with my Nokia E70 phone. Whenever I need to do this, however, I feel like I'm being suffocated by the constricting, feeble Windows environment and feel like doing the same to an unlucky bystander who just happens to bear the brunt of my Windows agitation. Fortunately, I finally sat down and figured out a few things to makeXP a bit more usable... a bit.

A (Somewhat) Usable Shell
The first thing I needed was a command line that wasn't dropped as a baby. Cygwin, which provides a UNIX-like environment on top of Windows, is perfect for this (another alternative is MinGW). The key to setting up Cygwin is configuring what applications you want installed in one of the setup menus. Of course, if you miss something on the initial setup, you can runCygwin's setup.exe again and reconfigure the included applications. Cygwin mounts the Windows C drive at /cygdrive/c. I would recommend browsing the Cygwin Properties menu and tweaking it (especially the fonts) to your preferences. Do this by clicking on the Cygwin icon in the upper left hand corner of the window and selecting Properties.

Cutting and pasting from Cygwin is a pain in the ass. To copy text, click the Cygwin icon and go to Edit->Mark. You now have a "Visual Block" - type selection box. I'm not sure if it's possible to select in a line-wrapping mode, the lack of which is also annoying. Hit Enter to copy the selected text. To paste text into theCygwin shell, click on the icon and hit Edit->Paste.

Install Software

• Pidgin for instant messaging
  
• Firefox and extensions (especially Google Toolbar)
• VLC for video and audio of all kinds
• Putty for SSH
  

Block Ads on the Web
Ads suck. Block them. Download a sample hosts file here and save it (in Notepad or something: Start->All Programs->Accessories->Notepad) to your file system. Now copy that file to c:\WINDOWS\system32\drivers\etc and name it HOSTS.MVP. If that's too much work for you, this page provides an automated tool to do it.

Add a Button to Show the Desktop
It's really aggravating if, in order to use the point-and-click Windows interface, you need to close all of the currently open windows to access the Desktop where a lot of clicking occurs. To do this, open Notepad and paste the following lines into it:

[Shell]
Command=2
IconFile=explorer.exe,3
[Taskbar]
Command=ToggleDesktop

Save the file as Show Desktop.scf. Right click on the task bar (that blue bar at the bottom of the screen) and select Toolbars->Quick Launch. Now drag the icon of the file you just created into the Quick Launch area. If all goes well, hitting that icon in the Quick Launch area should hide all the windows you have open and focus the desktop (or show them if you have them hidden).

Remove Unwanted Icons from the System Tray
msconfig is your friend. Good resources are here and here.

Remap keys
There is a pretty easy tool that allows you to remap your keys called SharpKeys. I'm a fan of Caps Lock to Escape remapping, myself.

Personalization
I recommend some BSOD wallpaper just to remind you from time to time that you're using an inferior operating system.

Nokia E70 for phone n00bs

I just bought the Symbian S60 v3-based Nokia E70 mobile phone based mostly on the glowing recommendation of Maddox. Since this is my first somewhat advanced mobile phone (taking pictures on a phone is still a novelty for me), I've chronicled all of the hoops I needed to jump through here in a E70-for-dummies type guide. In addition to this guide, of course, I would highly suggest browsing the excellent user manual that ships with the phone to navigate using its myriad features.

Phone calls
I got the phone off of Amazon.com for $385. Since it is an unlocked phone, inserting the SIM card from my previous phone (a pretty old-school Nokia ... unsure about the model number) enabled the phone calling features.

Internet Service
I upgraded to the unlimited Internet service from my provider (AT&T) for an additional $20 a month by going into their (pitifully understaffed and crowded) store and asking for it. Selecting the globe icon on the main screen will launch the web browser. The web browser lacks a couple of things that you would expect to find on a regular browser, namely support for animated gifs and flash movies.

Firmware update
Updating the firmware is probably the first thing you should do when you get your hands on the phone, as it will erase all stored data and potentially make previously installed applications unusable. The updates for the US edition are available from Nokia's site here. From reading some message boards, it appears that the main benefits of this update are that applications are faster and less memory-intensive. There have been a lot of reports of the phones bricking (those posts, however, are mostly about a year old), so don't say you haven't been warned! My phone, since I bought it recently, shipped with the newest firmware installed and I did not have to go through this step.

Installing applications
The E70 allows you to install applications on your phone, both from Nokia and from third parties. Once they install, you can find it under Menu->Installations (Menu is the key both in portrait and landscape mode with the blue icon and nothing else on it).

Google Maps
Google provides a Maps application for Symbian-based phones. In the E70's web browser, navigate to maps.google.com and accept the web site's offer to install the application. One of the confusing things about the application is that the labels that tell you how to control Google Maps via the buttons on your phone do not adjust to fit landscape mode. Press the left/bottom selection key for the menu, the middle joystick to zoom in and the top/right selection key to zoom out.

SSH
There is a SSH client called Putty for Symbian phones. To install it, you'll need to download the application and then install it via the Nokia PC suite, which makes installation pretty easy (on a Windows PC, that is -- Ubuntu 7.04 didn't recognize the phone as anything: camera, storage, phone, or otherwise). You must enable self-signing applications to install themselves via the directions here. Installing fonts for Putty might also make your experience with it more pleasant -- get them here. Unfortunately, there's often a lot of lag between when you type something and when you see the output. Check out the options menu to send special characters (Esc comes in handy for using vi!).

YouTube
YouTube has a website for mobile users that your E70 is automatically redirected to when trying to access YouTube. To view videos, you must first set the default access point in Menu->Media->RealPlayer->Options->Settings->Connection->Network to whatever connection you want the video delivered over. Then click away on your favorite video. Or at least your favorite video among those offered -- YouTube only provides a limited subset of all of its videos for viewing on its mobile site ("Only a portion of YouTube videos are available on mobile at this time. We are working hard to bring you more!"). The video quality is pretty good.

Transferring Phone Numbers
AT&T claims that they can transfer your phone numbers if you have all of your phone numbers saved to the phone's SIM card. Apparently they have some device at the AT&T store that does this and they will do it for free. I have a hard timebeliving AT&T (a.k.a. 'Nickel and Dimeing you for Every Minor Service to Protect our Failing Business Model') would offer anything for free, but that's what they claim.

Text Messages
My phone is able to receive text messages but not send them. This is because the phone ships without any message center specified, which means the phone has no idea where to send the messages to get them to their final destination. To change this, go to Menu->Messages->Options->Settings->Text message->Message centers->Message center->Msg. center number and change that number to whatever your message center's number is (reference). Mine was +13123149810. Alternatively, there are services like text140 that allow you to send text messages to a phone from a computer, but it lacks a lot of the convenience of sending a text message from your phone. Using this message will also not charge you for sending text messages (you're just accessing the 'unlimited' Internet, right?).

AT&T Phone Support
This is an arduous process I suspect I'll be going through a few more times, so here's what you need to do to talk to an actual person on AT&T phone support (thePalo Alto store can be reached at (650) 617-8931):

1. 1 to continue in english
2. 3 for an existing customer
3. 2 for all other information
4. Enter phone number in question
5. 0 to speak to customer service
6. 0 (again) to speak to customer service
7. 2 to not participate in a survey

Incidentally, once you start talking to a person and not a machine, they have been quite helpful. Be prepared to give the last four digits of the social security number on the account if you want to make any changes.

Using WLAN Access Points for Web Access
I've found that using WLAN Access Points is one of the weaker features of the E70. Connections fail for every reason, whether it be the WPA authentication screwing up, the phone dropping connections or just failing to connect to an access point that is literally a foot away. If you can get them to work, however, they're a lot faster than the cellular network. Use Menu->Connect->Conn. Mgr.->Availab. WLAN->Options->Define Access Point to put an access point that your phone can currently see into a permanent list of access points. Use Menu->Tools->Settings->Connection->Access points to access this list and configure the details on each point (encryption, hidden network, SSID, etc.). To select the connection to use upon each launch of Web, select Web->Options->Settings->Access point->Always ask.

Bluetooth
The Bluetooth menu can be accessed via Menu->Connect->Bluetooth. To view paired devices, move the joystick to the right.

Camera
The resolution on the camera is pretty bad, and the phone doesn't come with a flash, which drastically limits the utility of the camera. Getting photos on and off the phone, however, is pretty easy since you can access the phone file system via Windows Explorer just like any other series of directories.

Gmail and Google Calendar
Both of these Google services provide very convenient mobile web interfaces. Perfect for someone who already has all of their data in El Goog's dirty paws.

Python
Snakes on a phone! Most of the documentation for S60 Python for Nokia phones is on the wiki. To install, follow the instructions here and download the appropriate things from the Sourceforge page. Now you've got a Python interpreter (and more) on your phone... what to do with it? Well, you could install a variety of applications, for one. For help, the Nokia-supported Python forum is the definitive place for answers. There's also an IRC channel, #pys60 on freenode. There's some additional documentation for developers.

Other useful things to know

• Cut and paste works just like on any other PC: Ctrl+X, Ctrl+V, Ctrl+C
• Press and hold Menu to get a list of running applications
• If you're in an app and choose the "Exit" softkey (or use the task-manager shortcut above) that will actually shut down that application, and take you to whatever you were doing before. But if you press the red end (hang-up) key to jump to the home screen, (or switch to another application,) the app stays running in the background.

Links

• Nokia E70 United States support
• Tips and Tricks for Nokia E70
• Engadget thread about tips and tricks for S60 phones
  
• Codes to get low-level phone information (haven't tried these myself)

VIM split windows and macros quick reference

ex commands for split windows

• :sp filename create a new split with filename
• :vsp filename create a new vertical split with filename
• :new create a new split with an empty buffer
  
• :vne create a new vertical split with an empty buffer
  
• :wa save all open files
• :wqa save and quit all open files

Ctrl-W related commands for split windows

• C-w h moves split to the left of current split
• C-w j moves split below the current split
• C-w k moves split above the current split
• C-w l moves split to the right of the current split
• C-w + increase the current split by one line (or, prefix this command with a number to do that many line increases)
• C-w - decrease the current split by one line (same addendum as above)
• C-w < decrease the current split width
• C-w > increase the current split width
  
• C-w _ Maximize the current split

Split windows in the .vimrc (taken from this page ... also see the same author's excellent Efficient Editing with VIM )

• map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
• map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it
• set wmh = 0 " allow splits with 0 lines open

Macros

• q<register> start recording and save the macro in the specified register (use a lowercase letter, for example)
• q (again) end recording
• @<register> playback the macro

Screen

Screen is an awesome UNIX utility that allows you to continue running commands on a remote system even after you have logged out! Magic! Here is a nice "screen for dummies" tutorial with some links to related sites at the bottom.

On the command line

• screen (will spew some text and then give a prompt) start a screen session
• screen -r reattach to screen session
• screen -d detach an attached screen session (good for logging in after network cuts out)
  

Links

• Power Sessions With Screen
• Screen ASCII cheat sheet

Tips

• Launch screen with an alias such as alias screen='TERM=screen screen' in .bashrc to avoid your backspaces getting mangled
  
• There is a ~/.screenrc file with which you can customize screen

My .screenrc

# don't mangle backspaces
termcapinfo xtermc kD=\E[3~

# no startup message
startup_message off

# set shell that launches
shell /bin/bash

# lots of scrollback
defscrollback 10000

# ignore case when searching in copy mode
ignorecase on

# make it obvious we're within screen
hardstatus string "screen%n - %h"

# vim-style bindings
bind k focus up   # cycle to the above split
bind j focus down # cycle to the below split
bind l next       # cycle to the next split
bind h prev       # cycle to the previous split
bind n split      # create a new split
bind ^N split     # create a new split
bind q remove     # remove the split
bind + resize +3  # increase split three lines
bind - resize -3  # decrease split three lines
bind _ resize max # maximize current split
bind = resize =   # make all splits equal size
bind g info       # display info on the split
bind ^G info      # display info on the split

# other (some defaults, just to remind myself they exist)
bind Q only       # get rid of all other splits but the current one
bind M monitor    # toggle monitoring of the current window
bind t title      # enter a title for this shell
bind a other      # go to most recently viewed other split
bind \' windowlist -b # go to window list
bind x kill       # close the current shell
bind ^X kill      # close the current shell
bind \\ quit      # close all shells

Things I learned at DEFCON 15

I just went to DEFCON in Las Vegas for the first time this year and it was a blast. Tons of smart people, cool hacks, great speakers, fun activities and stuff to learn. Here's a brief summary of some of the things I picked up this past weekend (in no particular order):

• Virtually any lock that you have bought can be picked, easily. Even the (supposedly) secure Medeco locks that they have on the white house and other high-security areas can be broken by a 12-year-old girl. For more, see the in.security.org blog entry (by the guys that made this presentation, Marc Weber Tobias and Matt Fiddler) here. Gun locks, also, are completely trivial for a kid to break as this amazing video proves (more on this subject: blog, report). Also from the talk: most hotel safes are terribly insecure. To break those that use your credit card to open/lock them, the code of the master card that unlocks all of them in stored in memory inside the safe. All you have to do is open the safe (legitimately) and then dump that code and write it to a card to have a "master card" for all the safes in the hotel... yikes!
  
• SCADA systems, the systems that run critical infrastructure such as water treatment plants, electrical grids and nuclear power plants have an overwhelming number of vulnerabilities. Scary. Nationwide emergency alert systems also have relatively easy attack vectors (from the talk by Ganesh Devarajan).
  
• There is always a motive for Internet crimes, just like ones off the net. The motive for the attack on the Dolphin Stadium website prior to the 2007 Super Bowl was particularly interesting because the attack, which was made on several other lower-profile websites, was linked to a Chinese syndicate that wanted users' World of Warcraft online credentials to acquire additional WoW gold! This is the first attack of its kind. (From the Internet Wars 2007 panel)
• One interesting way of getting malware on a user's computer is to set up a website and register malware to be downloaded on that website as a codec with Microsoft. That way, when a victim visits the attacker's site, he is prompted with a "Additional codecs are required to display content on this page... would you like to download them?" message. If he does, he downloads the malware onto his computer. (From the Internet Wars 2007 panel)
• If you're a NBC Dateline reporter, don't refuse a press pass and then try to brew up some sensationalist report about hackers at DEFCON. You will get owned.
• Even the badges at the conference were a hack unto themselves (video). How cool is that? And there is a rap about the conference, too!
  
• If you want to start up an advocacy organization, the most important thing you can do is have a paid person sitting by the phone who knows what expert to contact (and how to contact them) when called by reporters. PledgeBank is a good site to help organize the effort to raise this money. (Funny quote from the same Danny O'Brien talk: "[Imitating a typical clueless reporter that calls EFF] People will call and say, 'I heard that such-and-such technology can make someone's penis fall off; can this be done with Ruby on Rails?' ... In this way, EFF operates as a clearinghouse for idiots")
• Sam Bowne offers a class at San Francisco City College called "Ethical Hacking and Network Defense" ... sweet! (He recommends Hack this Site and Irongeek.com as good references)
  
• Brendan O'Connor gave a really good talk about the extra layer of "security" that banks are now layering onto their authentication services and how this layer improves neither the user's privacy nor his security -- in fact, it may be degrading both. Bruce Potter similarly lambasted "Defense in Depth" for being a lame attempt at covering up bad code with extra layers of "security."
• Bruce Potter of the Shmoo Group gave a talk about how the dynamics of vulnerability disclosure are changing: instead of informing the vendor of the weakness in their product, hackers are often now selling information about those vulnerabilities to third parties (who may or may not have good intentions). Potter called for a discussion on whether this was ethical and/or good for the security community.
  
• Thomas J. Holt presented on the economics and dynamics of the malware marketplace. This market is mainly organized around forms based in Russia and Eastern Europe where sellers start forum threads advertising their product. Moderators of the forum then test the software to see if it does what the seller says it does and then give their opinion of it on the same thread. If the opinion is good, and the seller has a good reputation, then buyers start asking questions about the software and perhaps buy it. The final step is the reviews of the software by the buyers which attest to the malware's quality (or lack therof). Some sell not only hacking tools like Pinch, Nuclear Grabber and PG Universal Grabber but the data obtained by using these hacking tools. Some forums maintain lists of sellers who are "rippers" -- rip off artists -- and blacklist them.
• AgentX gave a whirlwind talk about "22 Things that Keep me up at Night." Among these: "Shrinking the Gap" (based off of ideas of Thomas Barnett), low cost pervasive bandwidth, open source warfare (the terrorists are all sharing techniques -- why aren't we?), the security industrial complex (complacency = bad), homogeneity of the hacker (all white males, but starting to change -- he notes good progress at DEFCON 15), why aren't you encrypting all your communications? (because the NSA is listening)
• Mike Murray had some interesting things to say about social engineering/NLP/hypnotism. Such as: you are four times more likely to follow a command following a non-grammatical sentence, you wouldn't do something hypnotized that you wouldn't do in the first place (such as kill a commanding officer -- study), tag questions ("won't you?" "right?") are convincing, confusion is the key!, the right tone can be categorized as "artfully vague," stories are more powerful than reasoning, questions are more powerful than statements, Milton Erikson = best hypnotist ever
• I missed Bruce Schneier's talk, but someone posted a video of it here. The whole presentation is interesting, but probably the most stimulating is his discussion of how you DON'T need an ID to fly on an airplane; you just get a little extra screening.
  

Other notable moments/events include the Guitar Hero contest, Brew Wars, Phreaking Challenge, Capture the Flag, Hacker Jeopardy, TCP/IP Drinking Game, Lockpicking Village, automated airgun target contest, the Wall of Sheep and the Wireless Village. Definitely going next year!

UPDATE:
Videos of all of the presentations have been posted here.

My .vimrc file

Thanks to Evan Klitzke and others at Yelp; a lot of this is ripped from their .vimrc's. All hail vim!

set modeline   " Respect other people's options (when a modeline is present)
set encoding=utf-8 " Use UTF-8 (8-bit variable width Unicode)

" indenting options
set autoindent  " Keep the indent level when hitting Return
set smartindent  " Use smart indenting (mostly useful for C/C++ files)
set cindent  " Don't indent Python really poorly
set tabstop=4  " Make tabs appear four spaces wide (default is 8 spaces)
set shiftwidth=4
set noexpandtab  " Use hard tabs please! Watch out for files with soft tabs
" that don't have a modeline present, especially Python files.
set fo=tcoqan  " Options for formatting text (i.e. for use with gq)

" UI stuff
set showmatch  " Show matching parens as they come up
set ruler  " Show the column number in the status bar
set incsearch  " Find as you type
set lz   " Don't redraw the screen in the middle of executing macros
set nohlsearch  " Highlighting search terms is _really_ annoying
behave xterm  " Just in case...
set wrap  " Display files with word wrap, but don't actually insert newlines
" (Set by default)... so lines don't go off edge
set lbr   " Wrap only at word boundaries (default is at any character)
syntax enable  " Who wouldn't want syntax highlighting?

" Only allow folds that are manually set
set foldmethod=manual

" Make our folds look like how they were when we exited vim last time.
set viminfo=s1,<1024,'1024,/1024,:1024,@1024,c,f1,%,n~/.viminfo
autocmd BufReadPost *
\ if expand("<afile>:p:h") !=? $TEMP |
\   if line("'\"") > 0 && line("'\"") <= line("$") |
\     exe "normal g`\"" |
\     let b:doopenfold = 1 |
\   endif |
\ endif
" Need to postpone using "zv" until after reading the modelines
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\   unlet b:doopenfold |
\   exe "normal zv" |
\ endif

set nocompatible            " Don't act like SysV vi
set shell=/bin/bash         " Use this shell to execute commands
filetype plugin indent on   " ability to run different vimrcs on different filetypes
set autoread                " When a file has been detected to have been changed outside of Vim and
              " it has not been changed inside of Vim, automatically read it again.
set mouse=a                 " Have mouse operate in all modes -- disable mouse by holding the shift key
set ttyfast                 " Tell vim to optimize for a fast terminal; will be on by
              " default if your $TERM is xterm or screen, but could be
              " turned off if you use a weird terminal (e.g. 'screen-bce').
              " Set 'nottyfast' for slow SSH connections.
set history=50
set showmode                    " show what mode (e.g. INSERT) you're in
set showcmd                     " Show (partial) command in status line
set backspace=indent,eol,start  " explicitly makes the backspace work when at the beginning of a line
nmap <C-N><C-N> :set invnumber <CR> " Press Ctrl-N twice to toggle line numbers in the left margin

"splits
set wmh=0 " allow splits with 0 lines open
map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it

" cycling through tabs
map <C-H> <C-PageUp>
map <C-L> <C-PageDown>
imap <C-H> <C-PageUp>
imap <C-L> <C-PageDown>

set title           " sets title of terminal in menubar to that of file currently being edited
set ignorecase      " perform case-insensitive searches...
set smartcase       " ... unless there are capitalized characters in the search pattern
set mousehide       " hide the mouse when editing
set exrc            " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set bg=light        "this almost always looks better -- take this out or set it to dark if things look funny

" for common typos
command! Q quit
command! W write
command! Wq wq
cabbrev Set <c-r>=(getcmdtype()==':' && getcmdpos()==1 ? 'set' : 'Set')<CR>

Or, say you want soft tabs instead of hard ones (see converting tabs to spaces):

set expandtab     "soft tabs
set tabstop=2     "set tabstops
set shiftwidth=2  "set shiftwidth
set softtabstop=2 "delete this many spaces upon deleting soft tab

More stuff from Rajiv:

set directory=/var/tmp,/tmp " put all the backup stuff in tmp directories
set shortmess+=A " no swap file complaints
set nobackup " don't litter directories with swap files

" move across rows, not lines
noremap j gj 
noremap k gk
noremap gk k
noremap gj j

set display=lastline " ??????????????
syntax sync fromstart " always render syntax from the beginning of the file
au! BufRead,BufNewFile * lcd %:p:h " cd into the directory of the file you're editing
set vb t_vb= " no visual bell

Configuring WPA manually

Configuring WPA is Ubuntu is no problem if NetworkManager works flawlessly. However, sometimes it doesn't. Here's how to configure WPA from the command line:

1. Get wpa supplicant sudo aptitude install wpasupplicant
2. Run wpa_passphrase which reads stdin and outputs the PSK that you'll need
   
3. Create a configuration file, say /etc/wpa_supplicant.conf, with the the text following this numbered list
   
4. sudo wpa_supplicant -B -D wext -i eth0 -c /etc/wpa_supplicant.conf (B for background, D for driver (wext is a generic linux driver, ipw2100 does not work), i for interface, c for path to conf file)
5. You should be connected at this point. Run dhclient to request an IP address from the server. ping something to make sure you can reach the outside.
6. (optional) This process can be automated on startup by editing the /etc/network/interfaces file appropriately
   

ctrl_interface=/var/run/wpa_supplicant

network={
ssid="My Network"
key_mgmt=WPA-PSK
psk=<generated passphrase>
}

As always, look at the man pages for more details: man wpa_supplicant and man wpa_supplicant.conf and man wpa_passphrase

dpkg's --force-architecture option is your friend on amd64 systems

For example, to install picasa for linux

sudo dpkg --force-architecture -i picasa*.deb (-i for install)

If you don't do this, you'll get a warning about the package being i386-based and therefore unusable on your system.

HOWTO format and mount an external hard disk

When I initially tried to back up my music library onto my Windows-created FAT32 external hard drive, the terminal threw a bunch of "Invalid argument" errors and failed to copy those respective files. It turns out that windows-formatted drives can not have certain characters in their filenames ( \ / : * ? < > | ), so I decided to reformat the drive in a Linux-friendly fashion.

Instructions

1. Connect the target drive to your computer (let's assume it's recognized as /dev/sdb1)
   
2. Reformat it using the ext3 file system (Linux default) by either...
1. sudo mkfs.ext3 /dev/sdb1 (lots of options with this tool... read the manpage )
   
2. Using gparted (install with sudo aptitude install gparted), a GUI-based tool
   
3. Create a mount point sudo mkdir /external
4. Mount the drive sudo mount /dev/sdb1 /external
5. Use the drive!
   
6. (optional) If you want the drive to be recognized in a certain way upon boot, edit the /etc/fstab file to your liking (Ubuntu 7.04 recognizes it anways)
7. (optional) To unmount the drive sudo umount /dev/sdb1
   

Useful commands

• sudo fdisk -l List partition tables for all devices (mounted or not)
  
• df -h Human-readable output of system disk space and usage
  

Tips
hd* = IDE drive
sd* = SATA drive

Information used from these sources

HOWTO organize a music library of 10,000 songs

Several days ago, I set out on an epic quest to organize my music library of roughly 10,000 songs. It's a daunting task, and one I've put off for a while, because my files are in several different places in several different formats with several different ID3 tag/naming standards. And there are more problems too: identifying mislabeled files I got from other places, weeding out corrupted files, etc, etc etc. Suffice it to say that it's been an intense few days with me in a zen-like state of focus trying to get everything in its right place. So here's the short version of how I did it.

The first (major) problem I faced was the massive swaths of files that had incorrect ID3 tags. For this problem I needed a competent ID3 tagger. About a year ago I asked on ubuntu-users what the list's tool of choice was. Suggestions ranged from Quod Libet to Foobar2000 on WINE (outstanding reviews: "best program ever" and "the only program i miss since switching to ubuntu") to gtkpod to EasyTag (mixed reviews: "tolerable" to "great") to id3v2 (command line based) to normal music players like amarok and banshee. There is also a Python module for accessing and editing ID3 tags, if I felt like scripting some things myself.

Eventually I settled on Picard, the tagger that Musicbrainz (a great music metadata site) recommends. Thanks to Ubuntu and the Picard team, installation is as simple as adding deb http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/ubuntu edgy musicbrainz to /etc/apt/sources.list (even if you're not running edgy), importing the PGP key with wget http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/public.key -O- | sudo apt-key add - and then a sudo aptitude update and sudo aptitude install picard. There are only a few limited operations Picard can perform, but they are very powerful. Picard interacts with the Musicbranz database to identify tracks in an album-based scheme. It correctly identified and summarily tagged more than 95% of the CDs I threw at it, and I listen to some pretty obscure/out there music. The ones it couldn't recognize were mostly tracks that didn't have a lot of existing context to go with them, and although I don't know exactly what information Picard looks at to determine the ID3 tags, I suspect that was the problem. There are a few problems with Picard, however. First of all, the UI sucks. Second, there are very limited capabilities. I would like to see, say, options that move files around on the file system depending upon how Picard tags (or fails to tag) them. The albums that have their ID3 tags written by Picard don't seem to be organized in any meaningful way, except in the current running instance of the program, where they are added to the bottom of the list (which is also annoying because of the need to keep scrolling to the top to add files to tag and then to the bottom to click and drag the files it missed and back again). Increased automation in the program would be nice, too, as I found myself performing the same operations over and over again. There are also minor issues like it opening up a new browser tab on each database lookup (leading me to have firefox open with ~100 or more tabs at some points). Fortunately for the future Picard user, I saw most of these issued acknowledged and marked for future improvement in the documentation.

The second (major) task ahead of me was conversion of all of my files into the near-universal mp3 format (yes, I know I should be gunning for ogg, but you can't use those files on your iPod... well, without the free firmware you can load onto it... another day, another day). For m4a files, I used the script found at the end of this thread which happily copied my ID3 tags as well. For ogg files, I would need something more general. At first glance, SoX seemed like a reasonable tool, but it needs to be recompiled for mp3 support. I also found some other scripts, but those didn't copy the ID3 tags, which made them unacceptable for my purposes. I asked in the freenode #ubuntu channel regarding my quandary, and someone mentioned Soundconverter as an option.

Soundconverter is a simple tool, but gets the job done. I installed it with a simple sudo aptitude install soundconverter. It provides a nice drag-and-drop interface that doesn't require knowledge of ID3 tag layouts or encoding/decoding details. Most of the options are located in the Edit->Preferences menu; the rest of the program is idiot-proof. Just drag the files, and hit Convert. It does do some annoying things, like saving files in URL-encoded names (example: files in folder "Cannibal Corpse" with the option to save in the same folder on were saved in a newly created folder, "Cannibal%20Corpse"), but generally works well.

Things were a bit more complicated than this, of course. I was making nightly backups of my changes to a local FTP server, wrote up some Python scripts to assist me in my task of keeping track of where everything was going and manually edited some outlying tags in amarok -- but now my music library is (almost) officially organized. Success! Now to just incorporate all of my friends' libraries...

Some cool links I found along the way

• Music File Processing on Linux describes command-line techniques for some tasks and gives an overview of available software
• MP3 Encoders Comparison
• My previous post on file extensions, compression, and conversion

Some relevant command line tricks

• find -type f ! -iregex '\(.*.flac\|.*.ogg\|.*.m4a\|.*.mp3\)' -delete -or -empty -delete

Feisty Fawn customization

Here are the things I did after I installed Feisty Fawn Ubuntu 7.04 off of the alternate install CD on my Compaq X1000 laptop/other systems.

If on an AMD64 processor system

• Follow these instructions (and run the script) to get mplayer, flash and java support for firefox. Update all launchers and shortcuts to launch firefox32 instead of firefox
• Skip the remaining instructions that are italicized
  

Install add-on packages (by the way, all except the first can be done in one command... just append all the packages you want to install after "install")

• Edit /etc/apt/sources.list following these instructions to add extra repositories
  
• Install full Vim support (without this, no help, syntax highlighting, etc.) sudo aptitude install vim-full
• Install Flash sudo aptitude install flashplugin-nonfree
• Install media codecs sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse w32codecs
• Install Amarok sudo aptitude install amarok
• Install Java sudo aptitude install sun-java6-jre sun-java6-plugin sun-java6-fonts
• Install Azureus sudo aptitude install azureus (or, alternatively, ktorrent)
  
• Enable DVD support
  
• Possible user programs: xmms, xchat, mplayer plugin, vlc, skype, picasa, eclipse, google earth, beagle, clipboard manager
  
• Security programs: sudo aptitude install nmap kismet wireshark airsnort aircrack-ng john
• Install Beryl (nVidia) (I also find it convinient to add a sessions entry for beryl-manager so the stone shows up in the tray)
  

Trick out Firefox

• Add desirable extensions
  
• Customize Google toolbar
  
• Edit /etc/hosts to block ads
• Install multimedia plugins
• Change default colors Edit->Preferences->Content->Colors
• Set a master password Edit->Preferences->Security->Use a master password
  

Trick out the Terminal (and partners in crime)

• Either create a new profile or edit the current profile
• Disable General->Show menubar by default in new terminals
• Set Colors to aesthetically pleasing values
• Set Effects->Transparent background and set the bar about 2/3 of the way to Maximum
• Set Scrolling->Scrollback to 2000 lines
• Edit ~/.bashrc
  
• Edit ~/.vimrc
• Put the line set editing-mode vi in ~/.inputrc (the configuration file for readline... this also makes the set -o vi in the .bashrc redundant)
  

Trick out GNOME

• Fix the Right Alt key System->Preferences->Keyboard->Layout Options->Third level choosers->Press Menu Key to Choose 3rd Level
• Change the Desktop Background by right clicking on it (good backgrounds here (OSX Tiger) and here (fractals) and at DeviantArt)
  
• Map the Windows keys to do something cool in System->Preferences->Keyboard Shortcuts (I think "Run a terminal" and "Hide all windows and focus desktop" are good candidates)
• Enable visual system beep System->Preferences->Sound->System Beep->Flash Window Titlebar
• Make the panels transparent Right Click on Panel->Properties->Background->Solid Color and adjust the slider appropriately (about halfway looks good)
  
• Add the system monitor to the panel Right Click on Panel->Add to Panel->System & Hardware->System Monitor->Add to Panel (I enable all but "Load"... wtf is load?)
• Disable launching of unnecessary programs at startup (such as Evolution Alarm Notifier) Preferences->Sessions->Startup Programs
• Save your session in a desireable state: gnome-session-save (don't use the automatic option in Prefs->Sessions... it causes problems!)
  
• Put the panels in a desirable place by dragging them across the screen
• Get 4 workspaces Right Click on Workspaces on Panel->Preferences->Number of Workspaces
• Disable Rhythmbox launching upon iPod detection (Amarok rulez!) System->Preferences->Removable Drives and Media->Multimedia->Uncheck "Play Music Files when Connected" and also Uncheck Storage->"Browse removable media when inserted"
  
• Enable the text-based location bar in Nautilus (in favor of the button-based one) by hitting the notepad button on the top left in a Nautilus window
  

If on a laptop...

• Configure Wireless Networks
• In /etc/default/acpi-support set ENABLE_LAPTOP_MODE=true

So what's so different about Feisty (as opposed to Edgy)?

• All the latest stable versions of software in the repos, obviously
• GNOME network manager manages wireless devices by default (and keyring support to boot)
  
• Fool-proof support for installing restricted drivers, media codecs, etc.
• Nice command line assistance if you try and run software that's not installed (prints a "use this repository and this command to get the software"-type message)
• Easy enabling of "Desktop Effects"
• And more
  

iPod + Amarok

Using your iPod with Ubuntu is disgustingly easy. As a Linux user, I was almost disappointed... no config files to edit? No dependencies to chase down? No bugs to fix? Nope, Ubuntu and the Amarok music player make everything simple. Here, I will detail how to use Amarok with your iPod on an Ubuntu system. I'm using Ubuntu Feisty Fawn 7.04 (just installed today!), a fourth-generation iPod and Amarok version 1.4.5.

First install Amarok with sudo aptitude install amarok (and with aptitude, not apt-get). If you want Amarok to be useful with most file formats (such as mp3) it would be a good idea to install proprietary codecs as well: sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse libxine-extracodecs w32codecs

Now that those monster installs are finished, you can go ahead and connect your iPod. If it automatically shows up on your desktop as a little iPod icon, great! If not, check out this thread for more detailed info. When I popped in my iPod, Amarok did me one better and recognized the device, popping up a little dialog asking me what kind of player it was. When this happens, you should select Apple iPod Media Device in the drop-down box. There are a few more options you can set regarding the device -- check out Settings->Configure Amarok->Media Devices in the Amarok toolbar menu. (If this explanation was too complicated for you, there's a guide with nice pictures here describing a slightly different way of going about things.)

To add files to the iPod, go to the Collection tab on the left and highlight whatever you want to transfer to the iPod in the left frame. Right click on it and select Transfer to Media Device. Then go to the Devices tab (with the little iPod logo) and hit Transfer at the top to transfer your files. When you're done transferring files, hit Disconnect at the top. If you're lucky, Amarok will unmount the device for you. If not, you'll have to right click on the desktop icon of the iPod and unmount it that way. I asked in the Amarok IRC channel if it was possible to sync the iPod and apparently it's not... for now, just delete everything off of the iPod and put the new stuff back on, as time consuming as that may be.

There are some other things you can do in Amarok to tweak the settings for the iPod. In the Devices tab under the (usually hidden) iPod menu you can set the model of your iPod. This is used for advanced support such as photo and video copying.

It's annoying that Rhythmbox, the default GNOME music application, launches whenever you connect your iPod. To disable this, go to Preferences->Removable Drives and Media->Multimedia->Portable Music Players and disable Play music files when connected. Just doing this will make a window open up in Nautilus every time you insert a removable drive, including an iPod. To turn this off, go to the Storage tab and uncheck Browse removable media when inserted.

Links

• Amarok
• Amarok Wiki: Considerations common to all devices
• Amarok Wiki: iPod page
• Post about "Read-only file system" problem if iPod is formatted on a Mac
• iPod specs

My .exrc file (vi/m) and then some

What follows is my .exrc file for various versions of vi, and some more related stuff. I have yet to look through the vim documentation exhasutively to see if there are any more useful options -- this is just stuff I have gleaned from random places on the Internet and from friends.

Plain vanilla vi (as on Solaris)
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns set showmatch " show matches of ()'s, {}'s, and []'s
set showmode " show what mode (e.g. INSERT) you're in
set autoindent " indent to the extent of the previous line

Vim:
syntax enable " enables syntax highlighting
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns
set title " sets title of terminal in menubar to that of file currently being edited
set expandtab " tabs aren't really tabs, but many spaces
set softtabstop=4 " delete four spaces when you delete a "tab"
set ignorecase " perform case-insensitive searches...
set smartcase " ... unless there are capitalized characters in the search pattern
set mousehide " hide the mouse when editing
set exrc " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
set bg=light "this almost always looks better -- take this out or set it to dark if things look funny
set nohlsearch "no annoying search highlighting

" for common typos
command! Q quit
command! W write
command! Wq wq

Vim Python
set autoindent " indent to the extent of the previous line

Vim C/C++/Java
set cindent " indenting for C-style languages
set showmatch " show matches of ()'s, {}'s, and []'s

For Text (Vim 7.0 only)
setlocal spell spelllang=en_us " English(US) spellcheck on (Vim 7.x only)
set tw=0 wrap linebreak " don't split words between lines, only wrap at end of word
"lots of abbreviations (varies depending on the subject material)
ab STIA Science, Technology and International Affairs

How to produce nicely formatted C code in Vim (From p.221 of Learning the vi editor):
set nocp incsearch
set cinoptions=:0,p0,t0
set cinwords=if,else,while,do,for,switch,case
set formatoptions=tcqr
set cindent
syntax on
source ~/.exrc

Options set when vim 6.4/7 starts on my system (Ubuntu Linux):
helplang=en " language to display help in
ruler " show row and column in bottom right
ttyfast " indicates a fast terminal connection
viminfo='20,"50 " configuration info
history=50 " number of : commands and previous search patters to be remembered
scroll=32 " number of lines to scroll with CRTL-U and CTRL-D commands
ttymouse=xterm2 " mouse codes
backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
fileencodings=ucs-bom,utf-8,latin1 " list of character encodings considered when starting to edit a file
printoptions=paper:letter " controls format and output of printing
runtimepath=~/.vim,/usr/share/vim/addons,/usr/share/vim/vimfiles,/usr/share/vim/vimcurrent,/usr/share/vim/vimfiles/after,/usr/share/vim/addons/after,~/.vim/after " list of directories which will be searched for runtime files
suffixes=.bak,~,.swp,.o,.info,.aux,.log,.dvi,.bbl,.blg,.brf,.cb,.ind,.idx,.ilg,.inx,.out,.toc " Files with these suffixes get a lower priority when multiple files match a wildcard.

Cool stuff that I have seen various places but I didn't find particularly useful (maybe someone else will):
set list "replaces all EOLs and TABs with $ and ^I
"auto-complete of brackets
inoremap [ [] <left>
inoremap ( () <left>
inoremap { {} <left>
inoremap " "" <left>
inoremap ' '' <left>

To see the options vi(m) has enabled, enter :set
To get help with options in Vim, enter :help set or :set all for regular vi
Or just :help for the help contents... Vim has excellent documentation

Links

• Thread on comp.lang.python about vim
• comp.editors
• Bram Moolenaar's (Vim author) "7 Habits for Effective Editing 2.0" (Google Video)
  

My .bashrc file

export TERM="xterm-color" # xterm terminal emulation (with color -- be sure to put at top of file on Ubuntu)
export PYTHONSTARTUP="$HOME/.pythonrc.py" # python startup file
set -o vi # sets vi command-line editing mode
bind -m vi-command -r 'v' # so that every time you hit v in command mode, an editor doesn't launch
ulimit -c unlimited # dump core files, no matter how big they are
export EDITOR="vi" #use vi as the default editor for some commands, like "fc"

alias screen='TERM=screen screen' # workaround for screen backspace bug

alias d="date '+%r -- %A, %D'" # human-readable date output

alias ll="ls -lh" # long listing (with human-readable file sizes)
alias lr="ls -ltrh" # list by reverse modification time
alias la="ls -Ah" # list hidden files (except implied . and ..)

alias <name>="cd <important>" # get to important directories fast
alias <name>="ssh <login>@<important>" # log into oft-used remote machines fast

# prevent myself from doing stupid things with vi
function vi {
if [ ! -e "$1" ]; then
if [ "$1" == "" ]; then
vim
else
vim "$1"
fi
elif [ -d "$1" ]; then
cd "$1"
elif [[ -c "$1" || -b "$1" || -p "$1" || -S "$1" || ! -r "$1" ]]; then
file "$1"
else
vim "$1"
fi
}

The following I only enable on Solaris systems:
export PAGER="less" #uses less to view manpages instead of more

Links:

• HOWTO: Customizing bash prompts
  

Common Bash Tasks

I don't use Bash for much scripting. Usually, if I have to open up a file to write a script, it will be done in Python. However, the less I have to do that for mundane tasks that can be accomplished on the command line with Bash, the better. This entry may not have much in it now, but every time I do something cool/useful/time-saving in Bash from now on, it'll go on here.

Common Tasks

• Rename *.foo files to *.bar files: for i in *.foo; do mv $i ${i%.foo}.bar; done (BashFAQ)
• Rename foo.* files to bar.* files: for i in foo.*; do mv $i bar.${i#foo.}; done (also check out the perl regex-based rename Linux utility)
  
• Do something to multiple arguments: for i in arg1 arg2 arg3; do echo $i; done
  
• Print 0 through 9 on separate lines: for i in {0..9}; do echo $i; done
• Flatten output onto one line: <Multi-line output> | xargs (10 habits)
  

Links

• GNU Bash Reference Manual
• Bash Manpage
• BashFaq Bash FAQs and answers
• Bash Pitfalls common mistakes
  
• Advanced Bash Scripting Guide Everything you would want to know about Bash scripting
• Bash Guide for Beginners A beginner's guide to Bash, some shell examples
• 10 Good UNIX Habits Advanced tips on the command line

Finding and Killing a Process in GNOME

There are a few different ways to find and kill a process in Linux. Using the GNOME interface, you can call up the System Monitor (similar to the one on Windows XP) under System->Administration->System Monitor. Under the Processes tab, right clicking on a process will give you a variety of options: stop/end/kill/continue the process, change the priority (nice), display a memory map (pmap) and display open files (lsof -p <procID>).

Embedding the system monitor in a desktop panel is useful for constantly monitoring the system and quickly determining if and where there is a problem. Do this with Right Click on Panel->Add to Panel...->System & Hardware->System Monitor->Add. The display provides graphs of Processor, Memory, Network, Swap Space, Hard disk and Load activity (not quite sure what load is, perhaps some metric computed from all the others? I don't use it, personally).

On the command line, list processes with the ps command or ps -e for all users' processes. Pipe that through grep (i.e., ps -e | grep firefox) to find a process ID of the target process (if you know the exact name of the program, you can also use pidof firefox-bin). To kill a process, use the kill <procID> command (with the -9 flag if you want to kill it unconditionally). Alternatively, if the process uses a gui, you can use xkill to merely click on the gui to kill the corresponding process. There is also killall if you want to kill by process table name (i.e. killall firefox-bin).

pgrep and pkill are two very useful utilities, especially for dealing with processes with long command line invocations. pgrep -fl -u danny "ruby.*asdf" will print the process id and name of every process owned by danny whose command line invocation matches the passed in regular expression. The -f argument tells it to match against the full command line invocation instead of just the process name. pkill -fu dannyc "ruby.*asdf" will kill those same processes (for the life of me, I can't seem to coax similar behavior out of killall).

The top command provides an integrated way of finding and killing a process on the command line. If the process-to-be-killed is using a lot of system resources, it will appear towards the top of the display. Hit k and then the process ID to kill the process. Top comes with a lot of other functionality -- hit h for help.

UPDATE: I recommend using htop instead of top, which is top with less suck. It has lots of good features like color, mouse support, a more responsive interface, a tree-based process view, etc.

Google Toolbar 3 has made my life 1.7% better

Google Toolbar Version 3 rules. It is a tiny little piece of technology that has made my life significantly better since I started using it. Linux users will have to go the extra mile (or in this case, inch) to make sure it works (nothing new...). See this thread and Google's toolbar for Linux page for instructions for older builds; apparently it appears to be playing nice now.

The coolest feature of the Toolbar is the ability to make custom search buttons. Going to any page with a search bar, right clicking, and selecting "Generate Custom Search" will create a nice little button on the Toolbar (corresponding to a site's favicon). Using this, you can search with the Toolbar as if you were on the page you generated a search for, the effect being that you never need to visit the page to use their search engine -- you can search from any arbitrary page! It's also useful for doing the same search (i.e. "Donnie Darko") over several sites you have a custom search (i.e. RottenTomatoes, IMDB, Wikipedia). If you don't know what buttons you want on your Toolbar, Google has provided a nice grab bag to choose from. My custom searches (as of right now) are:

• Google Image Search
• Google Groups (usenet)
  
• Google News
• Google Maps
• Google Docs & Spreadsheets (online word processing + spreadsheets) (
• Google Linux Search
• Google Calendar
• Gmail
  
• Google Reader (RSS reader)
• IsoHunt (torrents)
• Newegg (hardware)
• RottenTomatoes (movies, especially reviews)
  
• IMDB (movies)
  
• Gamespot (video games)
  
• Amazon (everything)
• YouTube (videos)
• MusicBrainz (music metadata and info)
• Wikipedia (online encyclopedia)
• Yelp (local search and reviews)
• Allmusic (music reviews and metadata)
  

Some more very useful features include enabling GMail to launch every time a mailto: link is clicked (Finally! No more surprise Evolution/Outlook/etc. "setup page" launches!), an icon indicating if you're signed in to your Google account, and some keyboard shortcuts (more would be nice... especially for accessing custom buttons) like Shift+Enter for an "I'm Feeling Lucky" search (keeping in the spirit of eliminating useless page visits). There are more features that I don't use (but many do) detailed on the features page. Some of these are pretty cutting edge and have generated a considerable amount of controversy, especially AutoLink. Others, such as the RSS subscribe and spellcheck, have already been implemented in Firefox 2. In any event, you should get it. Now.

A couple of things that could use improvement:

• More keyboard shortcuts (as mentioned above)
• Web spam reporting buttons, analogous to the "Report Spam" button that Gmail has... perhaps Google can build a database of splog, etc. sites to blacklist, or maybe work with other search engines to do so?
  
• Sometimes the "Generate Custom Search..." option doesn't work... amazingly, Blogger is one of those examples... when trying to do a custom search on a particular blog, I get this (wtf?): "Information Unavailable The custom button cannot be installed. Custom button values out of range."
  

Which Ubuntu version am I using, again?

On the command line: lsb_release -a

Alternatively, look in one of these two files for the answer (links to sources where I found info): /etc/lsb-release or /etc/issue

Also, find out if your install is 64-bit via uname -m (thread1, thread2).

Yahoo! Pipes and its Shortcomings

For the past couple of days, I have been fooling around with Yahoo! Pipes. It's a tool and an interface for aggregating and/or manipulating RSS feeds. It's an interesting experiment and has been receiving some hype, most notably from Tim O'Reilly who called it a "milestone in the history of the internet." So how useful/powerful is it? My assessment is: right now, not very. There are several problems that are keeping it from being a useful (or sometimes even usable) tool at present.

The first problem is it is SLOW. When manipulating the first one or two modules, things run fine, even to the point of web 2.0 AJAXy nirvana. The scrolling is snappy, the debugger (which is a well-thought out touch) and DHTML animations look nice, and the interface is responsive. However, after that things start crawling. And after 10 or so modules, it slows to a why-am-I-even-bothering speed (and I'm on a pretty damn fast MIT connection right now with a capable PC).

The second problem are the all-too-frequent site outages. A recent blog post by the development team humorously noted: "Our apologies for the unscheduled downtime this evening. As a reminder, todays (sic) post was sponsored by the letters “b” “e” “t” and “a”."

The third problem is the support of only RSS inputs. But judging from the Pipes developers' posts, however, this problem looks like it will be overcome in the future.

So what can one do right now with Pipes? The canonical example has been putting the RSS feed of the New York Times through a Flickr filter, the result being that you can see the NYT stories in photo form. Which brings us to problem number four: your results are only as good as your data. While a lot of these photos are applicable to current events, others are tangential and some are just irrelevant. Kind of makes one think that a well-implemented semantic web isn't such a bad idea, after all.

Another thing one can do with Pipes is what I did with my first pipe, Ray's Deals. It simply takes several RSS feeds from bargain electronics sites and aggregates them into one feed. I then made the feed (or, more precisely, the titles of the items of the feed) searchable. The utility of this Pipe is that Ray now only needs to subscribe to the RSS feed from this pipe instead of several RSS feeds. (Sidenote: I actually added way more modules than necessary here: instead putting each URL in a separate Fetch module, I could have just had multiple entries in a single Fetch module... live and learn, I guess.)

That's about all the useful things one can do with Pipes at present -- my subsequent, more adventurous, Pipe experiments were fraught with frustration and, ultimately, failure. This was in part due to Pipes problem #5: the limited amount of operators that Pipes provides. There are currently no functions for performing the most basic of operations, such as generating random numbers (see the failure of my Pipe Random Jokes (Incomplete)) or extracting specific information from a feed. There are some workarounds for things like selecting a particular item from a feed, as my second hackish pipe, Selection Experiment 1, demonstrates. To select a certain oldest item, I sorted the feed by Date and then cut off the feed after a certain number of items (6). I then reversed the feed and cut off all but the first item, thus producing the selected item. Wow, that's a lot of work. Not to mention the limitations of some of the other operators, such as Sort not taking in a key value to sort by, etc. To be fair, it's still a beta product and the selection and power of modules will no doubt grow with time.

There are other concerns too, some of which are voiced in the responses to Tim O'Reilly's post. Among these are IP concerns -- is your mashup your property, or the feed providers' partial (or full) property? Also, there is the question of accessibility: will Pipes be a place that democratizes mashups, or is it just for geeky developers (who might be better off coding their own specialized apps anyways)?

All in all, Yahoo! Pipes looks like a tool that has some potential. It has a nice interface (when it's not super slow), the concept is fresh, and it's relatively easy to use (again, when it works). Its current shortcomings, however, make it little more than a novelty item. Once these get addressed, I'll be willing to give it a second try.

Connecting to servers in Ubuntu with "Connect to Server..."

I don't know how this has eluded me for so long, but there is actually a very nice interface built into Ubuntu that allows you to connect to servers using any of the following protocols: ftp, ssh, windows share, WebDAV (http), Secure WebDAV (https), and more. This is a nice alternative to using third party software like FireFTP (even though that program rocks) or the command line.

Simply pulling down the Places->Connect to Server... option in GNOME opens a dialog box which allows you to enter the appropriate information about the server you want to connect to. After the connection is successful, a folder representing the remote file system will appear on both the Desktop and in the Places menu. Another nice feature is that the folder even has some little letters on it telling you what protocol it is connecting over. You can now drag and drop files via the Nautilus interface, as if the directory was local. How convenient!

This tip found via the free online book Ubuntu Linux Bible. (Reviews on Amazon) By the way, this is an incredibly comprehensive reference on all things that an end user might want to do on Ubuntu. It also has sections on setting up subversion, using LaTeX, setting up other servers such as Apache, multimedia usage, and just about everything else. UPDATE: Whoops! This actually got posted by mistake by Lifehacker... the book has been taken down... it's still a good book, though!

UPDATE: For some more (unrelated) GNOME fun, right click on an icon and select "Stretch Icon" to blow up the icon to a cartoonish size. Click "Restore Icon's Original Size" in the same menu if you get tired of it (which you will, after 2 seconds).