Saturday, December 30, 2006

HOWTO scrub iTunes' DRM off of your purchased music with QTFairUse

DRM sucks. A lot. Case in point: I just received a $25 dollar gift certificate from a loving relative who wanted me to buy some music. I summarily went to the iTunes Music Store and downloaded about twenty-five bucks worth of Cannibal Corpse songs. All was good, until I tried to play the songs on another music player. Surprise! The .m4p files are a iTunes-only, DRMed format. That is when I resolved to immediately blog about how to get around it (a mild form of protest, I know).

(Disclaimer: doing this may be violating the DMCA. But if you want to stick it to the man and protect your fair use rights, read on. Besides, the DCMA sucks, too.)

First, you're going to need Windows (sorry... I guess you could always use Wine or Virtualization inside Linux, but I don't have that set up yet -- I'm just dual-booting Ubuntu Edgy and Windows XP). Next, download QTFairUse which is a utility that will scrub the DRM from your music files. (There is also another utility for doing this called myFairTunes and an older one that doesn't work anymore called JHymn). Unzip the QTFairUse archive and launch the .exe file to start the program. Use the drag-and-drop interface to select which songs you want scrubbed and then click the button at the bottom to begin the conversion. You now have .m4a-format, DRM-free audio files!

Neither of these programs work under Wine... neither are in the Wine Application DB... looks like I'll have to stick with Windows (ugh) for now...

QTFairUse throws a "ImportError: MemoryLoadLibrary failed loading win32api.pyd" when using the console version and an "ImportError: MemoryLoadLibrary failed loading win32gui.pyd" when using the GUI version

myFairTunes sets up fine but throws this error when I run the application under Wine: "install the Windows version of Mono to run .NET executables" ... searched Google and it seems like this is a long way from being fixed

new links:
Be sure your computer is 'authorized' to play 'protected' files on iTunes or this won't work.

Thursday, December 21, 2006

Static IPs in Debian-based Linux

Here's a brief overview of how to set up an internet connection using a static IP address on Debian-based Linux systems. Most large networks nowadays use DHCP to get a user an IP address, but knowledge of static IP configuration can still be helpful in a lot of situations. First, edit the /etc/network/interfaces file as a superuser so that the entry for the desired interface looks something like this (replace x's with numeric values):

iface eth0 inet static # static, as opposed to dhcp... replace eth0 with desired interface
address # ip address you want assigned to this computer
netmask # subnet mask
network # optional
broadcast # optional
gateway # gateway (usually router/firewall) ip address

Once you have done this, restart your network interfaces with sudo /etc/init.d/networking restart

The gateway can also be specified on the command line if it's not in the interfaces file: sudo route add default gw The command netstat -nr can show the routes, including gateways, that the kernel is currently using.

To explicitly release a dhcp connection, do something like dhclient -r eth0

If your computer and network is set up correctly, you should now be able to ping your gateway and access sites on the internet by IP address. However, in order to access sites by name you need the help of DNS. Add entries to the /etc/resolv.conf file as a superuser in this format (you can have as many as you want):


Now, if you point your browser to a domain name like, you should be able to get there. Welcome to t3h 1nt3rw3b. You can also do all the above by accessing the System->Administration->Network menu.

Update (in light of ubuntu-users post and responses):
For switching from one interface configuration to another (assuming eth0 and correct /etc/network/interfaces):
  1. sudo ifdown eth0
  2. edit /etc/network/interfaces
  3. sudo ifup eth0
SIOCDELRT: No such process means something is screwy. You probably have the wrong number (i.e. gateway) somewhere. Haven't determined exactly what this means.


kiba-dock (couldn't find the project page...) is an OSX-like dock that you can use to launch applications, and it also has some silly animations to go with it. I installed it on Ubuntu Edgy Eft 6.10 using the instructions here. As the poster mentions, it only works properly after you try and install the second time. Strange, but it works. If you're not running some fancy compositor-window manager like XGL/Compiz or AIGLX/Beryl, it might look pretty ugly (black boxes around everything, sometimes text doesn't show up, etc.). I'm using Beryl and it looks fine.

To use kiba-dock, drag icons onto the dock to be able to launch them from the dock. kiba-dock is, of course, completely customizable (colors, position on screen, physics, etc.) so if it's not immediately to your taste, you can change it.

To launch kiba-dock at startup go to System->Preferences->Sessions and under Startup Programs add kiba-dock.

Here's a video of kiba-dock in all its glory:

Tuesday, December 19, 2006


Beryl is a super-sweet, super-configurable fork of Compiz (also see earlier post). Here I will describe how I got it set up on my system and some cool things to do with it.


Here are my specs:
  • 2 Hyundai 17" screens
  • 32-bit OS (Ubuntu Edgy Eft 6.10)
  • nVidia Geforce graphics card
Following the instructions on the Unofficial Edgy Eft Starter Guide, I got AIGLX/Beryl to work on only one screen without problems. To enable Twinview (sharing one desktop accross two physical screens), enter sudo nvidia-settings in a terminal and go to X Server Display Configuration (this, by the way, beats the hell out of hacking the /etc/X11/xorg.conf file). Click the Configure... button on the middle right and select Twinview. Be sure to back up your xorg.conf file before clicking Save to X Configuration File at the bottom and then exiting out of the settings manager. Hit Ctrl+Alt+Backspace to restart the X server.

Unfortunately, Beryl on Twinview made windows appear without titlebars or borders. I also got this message in the terminal when I disabled the session startup scripts and tried to run beryl-manager in the terminal:

beryl: No GLXFBConfig for depth 32
beryl: No GLXFBConfig for depth 32
beryl: No GLXFBConfig for depth 32

So, to the xorg.conf file! Under the Screen section, add this line:

Option "AddARGBGLXVisuals" "true"

I restarted the X server again and everything worked fine. Well, let me qualify that: at least nothing was obviously wrong. Some things are still somewhat slow like window resizing and scrolling. It appears that
others are having similar problems. However, this works well enough for me.

Features/Cool things to do

All these commands (and the themes) are completely configurable... just fool with the settings by clicking on the red stone in the system tray and selecting one of the managers... here are just some of the default settings
  • Ctrl+Super_L: water drop on desktop (hold down keys and move mouse for trail)
  • Shirt+F7: Window wake
  • Shirt+F9: Raindrops
  • Super+Mouse Wheel: Zoom
  • Alt+Mouse Wheel: Make a window transparent/opaque
  • Super+N: Make a window negative
  • Super+M: Make the whole screen negative
  • Alt+Left Mouse Button: Drag a window
  • Alt+Click Wheel: Resize a window
  • Ctrl+Alt+Left/Right: Spin the cube
  • Ctrl+Alt+Left Mouse Button: Drag the cube
  • Alt+Tab: Switch applications
  • F10: Expose-like effect
Here's where to configure desirable features in the Beryl Settings Manager:
  • A 3D world: Have windows separated in a third dimension from each other (drag and rotate the cube to see!)
  • Animations: Flames, Magic Lanterns and such
  • Scale: Expose-like effect
Some Beryl demos on YouTube

Tuesday, December 05, 2006

Bash prompt customization

Using old UNIX machines is a pain sometimes. The 10-year-old features just don't mesh with the current ones, and it just doesn't feel right. Also, the prompt might not display any useful information. Heresy, I say! Follow these steps to make yourself feel more at home. Some information taken from this article.

To get an Ubuntu-like prompt (assuming bash is installed):
  1. Edit the .bashrc file with an editor like vi
  2. Add this as the last line: export PS1='\u@\h:\w$ '
Other prompt configuration options:
  • \! History number of current command
  • \# Command number of current command
  • \d Current date
  • \h Host name
  • \n Newline
  • \s Shell name
  • \t Current time
  • \u User name
  • \W Current working directory
  • \w Current working directory (full path)
To get easy access to some obscure directory: alias [name]="cd [absolute directory path]"

You can also edit the .login file, which executes immediately when you log in. (It might go by a different name depending on the shell... see here for details)

Use set -o emacs (the default) or set -o vi to set your command line editing mode of choice.

And remember kids, don't forget to write the other users if on a public machine! (Or maybe talk, or wall if you're the admin).

More links

Thursday, November 30, 2006

Google hacking: maximize the effectiveness of your queries

Here's a bunch of cool stuff you can use to squeeze the most out of your Google experience (a lot of it gleaned from Google Hacks):

General queries:
  • "x y z" phrase: match the whole string as one
  • x (y OR z) boolean: match x and either y or z (same as x (y|z))
  • x -y negation: match x but not y
  • +the fuzz explicit inclusion: to prevent auto-ignore of common words (or put in quotes)
  • ~x synonyms: match x or words like x
  • x 1..100 number range: match x and any number between 1 and 100 (can also leave out one of the numbers to do a floor or ceiling)
  • for ** bell tolls full-word wildcards: match for, any 2 words, then bell and tolls
  • intitle:x search titles of web pages (use allintitle: to include all following words)
  • intext:html search only body text
  • inahchor:x search links' descriptive text
  • site:x search a certain website (does not understand subdirectories)
  • inurl:x search within urls
  • link:x return a list of pages that link to a certain page
  • cache:x return the cached copy of a page
  • filtype:x search for certain filename extensions (htm and html return different results)
  • related:x find pages related to x
  • info:x show information that Google has gathered about a page
  • phonebook:x search Google's phonebook for a certain number
  • define:x gives definitions for a word according to the web
  • movie:x search for a movie
  • music:x search for music
  • source:x search for news from a specific source
  • intext:x search within text
  • intitle:x search within titles
  • insubject:x searches posting subjects for query words
  • group:x search within a certain group
  • author:x search for posts by the author
  • blogurl:x searches a specific blog by its URL
  • inblogtitle:x search blogs titles
  • inposttitle:x search post titles
  • inpostauthor:x search by post author
Other things to remember/cool stuff:

Tuesday, November 21, 2006

HOWTO Sniffing AIM traffic on an open wireless network

If you are interested in seeing what people on an open wireless network are doing on AOL Instant Messanger (*ahem* I mean... for some legitimate, legal and ethical purposes, of course... *ahem*), then this will walk you through it. I'm using Ubuntu 6.06 as my OS.
  1. First, you're going to need a network traffic analyzer. Wireshark (formerly known as ethereal... Wikipedia explains the name change) is the standard for this: Enter sudo apt-get install ethereal on the command line to get it. Start it up as root (this should be an option under applications->internet).
  2. Next, change your wireless card to promiscuous mode: sudo iwconfig eth1 mode Monitor (of course, replace eth1 with whatever your wireless-enabled interface that you wish to listen on is).
  3. Click the "Show the capture options..." button in the top left corner of the ethereal GUI (it's the picture with a wrench on it). Select your wireless interface, check the promiscuous mode checkbox, and select any other options you want, but don't close the box yet.
  4. If you just ran the capture as is, you'd get all traffic on a wireless network. However, that can be a lot of stuff that you don't want. A little research on Google shows that Aim runs primarily on port 5190 and additional ports such as 13, 23, and 113. Specify that you only want to log packets that come through on these ports in the capture filter text box: tcp port 5190 or tcp port 23 or tcp port 113 or tcp port 13 (If you want other configurations, check Section 4.8 of the Wireshark User's Guide). Click the Start button to start the capture.
  5. You are now capturing any wireless packets that your interface receives. Ethereal will bring up a small dialog box showing you more detail about what's going on. Click stop to stop the capture.
  6. That's a lot of data. Use the filtering/analyzing tools that ethereal provides to help you sort out what's actually going on. One of the easiest/most useful things to do is right click on a packet of interest and select "Follow TCP Stream"... this will give you the transactions that happened between two points, perfect for reassembling an AIM conversation!

Saturday, November 04, 2006

Airsnort / WEP

Airsnort is the most widely used utility for cracking WEP. Get it on Ubuntu Dapper with sudo apt-get install airsnort. Here is a nice guide (with pictures!) to using airsnort on Linux. Airsnort uses an exploit described in this paper (postscript format) to obtain a WEP-encrypted password. Wikipedia has a more digestible summary of WEP flaws. wepcrack is another (and was the first) utility to implement the WEP exploit described above.

The way that Airsnort works is by passively monitoring all packets sent over a wireless network and then observing those packets for patterns it can exploit. The key to this is setting your wireless card to promiscuous mode: sudo iwconfig ethx mode Monitor. While you have this mode enabled, you will not be able to access the Internet.

One of the problems with monitoring traffic to crack WEP is if there is no traffic to montior in the first place! That is, the network isn't being heavily used. Solution: traffic injection. Aircrack-ng is a tool that supports this, among other things (homepage, tutorial, wikipedia page). Packet injection, however, is not a passive activity.

Books on the subject (links to pages):
TODO: Add sections about hidden SSIDs, MAC address filtering

Wednesday, November 01, 2006

Gang of Four/Design Patterns

If you're into software engineering, especially reusable object-oriented software engineering, you're a big fan of the "Gang of Four" book, "Design Patterns." Here's a quick outline of what's in it:

Creational Patterns
  • Abstract Factory: Provide an interface for creating families of related or dependent objects without specifying their concrete classes.
  • Builder: Separate the construction of a complex object from its representation so that the same construction process can create different representations.
  • Factory Method: Define an interface for creating an object, but let subclasses decide which class to instantiate. Factory Method lets a class defer instantiation to subclasses.
  • Prototype: Specify the kinds of objects to create using a prototypical instance, and create new objects by copying this prototype.
  • Singleton: Ensure a class only has one instance, and provide a global point of access to it.
Structural Patterns
  • Adapter: Convert the interface of a class into another interface clients expect. Adapter lets classes work together that couldn't otherwise because of incompatible interfaces.
  • Bridge: Decouple an abstraction from its implementation so that the two can vary independently.
  • Composite: Compose objects into tree structures to represent part-whole hierarchies. Composite lets clients treat individual objects and compositions of objects uniformly.
  • Decorator: Attach additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.
  • Facade: Provide a unified interface to a set of interfaces in a subsystem. Facade defines a higher-level interface that makes the subsystem easier to use.
  • Flyweight: Use sharing to support large numbers of fine-grained objects efficiently.
  • Proxy: Provide a surrogate or placeholder for another object to control access to it.
Behavioral Patterns
  • Chain of Responsibility: Avoid coupling the sender of a request to its receiver by giving more than one object a chance to handle the request. Chain the receiving objects and pass the request along the chain until an object handles it.
  • Command: Encapsulate a request as an object, thereby letting you parametrize clients with different requests, queue or log requests, and support undoable operations.
  • Interpreter: Given a language, define a representation for its grammar along with an interpreter that uses the representation to interpret sentences in the language.
  • Iterator: Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation.
  • Mediator: Define an object that encapsulates how a set of objects interact. Mediator promotes loose coupling by keeping objects from referring to each other explicitly, and it lets you vary their interaction independently.
  • Memento: Without violating encapsulation, capture and externalize an object's internal state so that the object can be restored to this state later.
  • Observer: Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically.
  • State: Allow an object to alter its behavior when its internal state changes. The object will appear to change its class.
  • Strategy: Define a family of algorithms, encapsulate each one, and make them interchangeable. Strategy lets the algorithm vary independently from clients that use it.
  • Template Method: Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template Method lets subclasses redefine certain steps of an algorithm without changing the algorithm's structure.
  • Visitor: Represent an operation to be performed on the elements of an object structure. Visitor lets you define a new operation without changing the classes of the elements on which it operates.
There's also thourough Wikipedia coverage of the topic: Design Patterns

Monday, October 30, 2006


For all of your streaming audio ripping needs... Streamripper works, give or take a few seconds off either end of the audio. If this annoying misbehaviour happens to you, you'll have to get down and dirty with the options. And remember kids, installing it is a simple sudo apt-get install streamripper on Ubuntu! The manpage is helpful, with a lot of examples for common slip-ups.


Compression: gzip, zip, bzip2

Which compression tool is the best? Of course, there is no "best tool" -- there are different tools for different jobs. It won't matter for compression of trivial data which method you choose, but when you start handling gigabytes of data, it might be prudent to check some of the following links. This is just a sample of all the compression algorithms out there, of course, and is skewed towards ones used more often on Linux.

General info links (to Wikipedia):

  • Data compression
  • ZIP -- .zip extension, zip and unzip commands; analogous to tar and gzip; used more on Windows
  • Gzip -- .gz extension, gzip and gunzip commands
  • Bzip2 -- .bz2 extension
Comparison links:

PGP, especially GnuPG

You're a paranoid freak that thinks that everyone is out to get you. Good, because they are if you live in the freedom-loving US of A. Therefore, you need to somehow encrypt your messages to make sure that only yourself and the intended recipient see the message. GnuPG to the rescue!

Very Short Intro to PGP:
PGP (Pretty Good Protection) uses public key encryption. Public key encryption uses a public key and a private key. A key generated by person A has both a public and a private component. Person A can distribute his public key freely, but his private key is protected by a passphrase that he and only he knows (hopefully) that was given at the time of the creation of the key. Others can use this public key to encrypt messages that only he can decrypt and read with the corresponding private key. Digital signatures work in the opposite direction. Person A can sign messages with his private key and others can authenticate that it was person A that sent it by decrypting the message with the corresponding public key. GnuPG is an implementation of this security scheme.

  • gpg --gen-key Create a key (all the default settings are fine); it will prompt for type of encryption algorithm (merits of each: DH vs RSA FAQ), keysize, key expiration date, name, comment and email, and finally, the passphrase used to protect your private key -- DO NOT FORGET OR GIVE OUT OR INSECURELY TRANSMIT YOUR PASSPHRASE!!! it will then generate some random bytes and ask you to play around with the mouse/keyboard while doing so to create more entropy
  • gpg --export -- export a key to a file (if you don't specify -o and a filename, it might corrupt the output on your terminal... just a warning); publish this for people to get your public key in order to be able to encrypt messages that you can decrypt with your private key
  • gpg -s <Data> to sign a document; this will create a <Data>.gpg file
  • gpg -se <Data> to both sign and encrypt a document; this will create a <Data>.gpg file
  • gpg -d <data> to decrypt the data; you can also specify a path with -o to redirect it to somewhere besides stdout

Utilities already installed on Ubuntu Dapper (6.06):
  • gpg -- Command line tool
  • gpgv -- Used to verify signatures against a trusted keyring
  • gpgsplit -- Split OpenPGP messages or keyrings into their component packets

Saturday, October 21, 2006

Linux PCI Wireless cards (and the bcm43xx driver)

It's a pain in the butt when manufacturers of PCI wireless cards don't release a driver to go with their hardware. Case in point: broadcom, whose card I purchased. Luckily, there is a hacked driver available from the community.


Note to self: whatever I do, do NOT use ndiswrapper for wireless drivers. All the kernel hackers tell me it's bad... and, they were right. I believe it attempts to load the Windows driver on top of a layer that enables the cooperation of the hardware and the Windows driver. When I tried to use it, it crashed GNOME. Grrr...

The jury's out on whether this actually works or not (I have yet to get this desktop in range of a usable wireless signal) but I can iwlist eth0 scan for stuff and get results back so I'm optimistic...

Update: this thing works (most of the time). Sometimes it will inexplicably lose the signal and I'll have to connect again. Not the biggest deal in the world, but still pretty annoying.

Monday, October 16, 2006

Databases: which one to chose?

A rundown of databases from a qualified source:
  • Oracle: everyone knows it, functionality for just about everything but doesn't do anything well
  • SQL Server: The poor man's Oracle: not as much concurrency, better user interface
  • Access: WORST!!!! No concurrency control
  • DB2: Faster than Oracle, UI tools and SQL is not as robust as Oracle's
  • Sybase: Good, but dying out because of market share
  • Informix: Very good, but bought out by IBM and integrated into DB2 (prof's favorite)
  • MySQL: Up until 5 no support for a lot of things like foreign keys and indexing, free
  • PostgreSQL: Robust database, free
  • There exist other, specialized databases for streaming data
The free ones: MySQL vs PostgreSQL:
  • MySQL:
    • Larger development community
    • Just a tiny bit friendlier
  • PostgreSQL:
    • 5 years ahead of MySQL (but MySQL is catching up fast)
    • Perhaps able to handle double the data?
    • Data type storage more optimized
    • Better for update-intensive work (writes)
Other advice:
  • Cache large/popular files: look at Apache's cache setup
  • Verify that files have type, non-virus integrity

Thursday, October 12, 2006

s5: an open-source alternative to Powerpoint

s5 is an open-source slideshow alternative (down with ppt!). It uses Javascript to drive the presentation of XHTML divs (which represent the seperate slides) and CSS to tie the view together. Here are some links:
Here's how to use it: All of the content is placed in a file at the root of the s5 directory (the example that comes with the .zip is called intro.html). Launch this in a browser to start the presentation. Another file to edit (if you want to customize the layout) is s5/ui/default/pretty.css. Otherwise, it's probably a good idea to leave the files alone if you value a working presentation.

Sunday, September 24, 2006

Linux Command Line Odds and Ends

Here are some useful Odds and Ends... most are related to command line stuff, some not; whatever, enjoy. Most of these came from either scouring the web, or Learning the Bash Shell or Learning Red Hat Enterprise Linux and Fedora.

Job Control
  • kill %<PID> kill a process
  • kill -QUIT %<PID> kill a process, a bit stronger
  • kill -KILL %<PID> unconditionally kill a process
  • fg bring a background job into the foreground
  • jobs list jobs running
  • ps process information

File permissions (owner, group, others)
  • 0 ---
  • 1 --x
  • 2 -w-
  • 3 -wx
  • 4 r--
  • 5 r-x
  • 6 rw-
  • 7 rwx

  • * matches zero or more characters
  • ? matches any one character
  • [abc...] matches any of the characters specified
  • [a-z] matches any character in the specified range
  • [!abc...] matches any character other than those specified
  • [!a-z] matches any character not in the specified range
  • ~ home directory of current user
  • ~userid home directory of a user
  • ~+ current working directory
  • ~- previous working directory

  • 'xxx' interprereted literally, variables not substituted
  • "xxx" interprereted literally, variables ARE substituted
  • `xxx` output of xxx command replaces it

Command line special characters
  • # comment
  • ; command seperator
  • & run in background
  • \ command continued on next line
  • | pipe

Input/Output Redirectors
  • prog > file stdout to file
  • prog 2> file stderr to file
  • prog >> file concatenates stdout to file
  • prog 2>> file concatenates stderr to file
  • prog > file 2>&1 stdout and stderr to file
  • prog >> file 2>&1 concatenates stdout and stderr to file
  • prog < file stdin from file
  • prog << text reads stdin until a line matching text is found, then EOF posted ("here document")
  • prog | prog2 pipe stdout
  • prog 2>&1 | prog2 pipe stdout and stderr

Command Line Movement
  • Ctrl+Shift+N open new console window
  • Crtl+Alt+F[1-7] go to virtual console 1-6 or X(7)
  • Ctrl+Alt+Backspace stop X and go to console
  • Alt+B Back one word
  • Alt+F Forward one word
  • Ctrl+A Beginning of line
  • Ctrl+E End of line
  • Alt+D Delete word |------------->X
  • Ctrl+D Delete char
  • Ctrl+K Delete |------------->X
  • Ctrl+U Delete X<-----------|
  • Ctrl+L Clear screen
  • Ctrl+Y UNDO
  • ESC+. Insert last word of previous command
  • TAB Possible completions

  • /server join a server
  • /join join a channel
  • /quit quit the server
  • /close close the current screen
  • /part leave the current channel
  • /partall leave all channels
  • /msg msg a user with a new window
  • /notice msg a user without a new window
  • /query force a window open to msg a user
  • /chat DCC with a user
  • /dns dns lookup for a user
  • /ping ping a user
  • /me *** does something
  • /whois query whois for a user

Ctrl Keys
  • Ctrl+C intr: stop current command
  • Ctrl+D eof: end of input
  • Ctrl+\ quit: stop current command (if Ctrl+C doesn't work)
  • Ctrl+S stop: halt output to screen
  • Ctrl+Q resume output to screen
  • Ctrl+Z suspend current command (works well with bg, fg and jobs)

Escape Sequences
  • \a alert (bell)
  • \b backspace
  • \c omit final newline
  • \E escape character
  • \f formfeed
  • \n newline
  • \r return
  • \t tab
  • \v vertical tab
  • \xxx ASCII in octal
  • \\ backslash
Tar: tarball options
  • -c create
  • -r append
  • -t list contents
  • -x extract
  • -a append files
  • -v verbose
  • -z zip/unzip
  • -f use filename
  • Oft-used:
    • tar -cf foo.tar foo; gzip foo.tar
    • gunzip bar.tar.gz; tar -xvf bar.tar

Saturday, September 23, 2006

Python Bookmarks, Tricks, Tips, Gotchas

Python = Best Language EVAR! 'nuff said. Here are some Bookmarks, Tricks, Tips, and Gotchas that I've compiled:



"dictionary comprehensions"
>>> dict((x, None) for x in [1,2,3])
{1: None, 2: None, 3: None}

Running a script in an interactive session
>>> import sys
>>> sys.argv = ["", "one", "two", "three"]
>>> execfile("")

Ternary if else statement (2.5 only)
x = true_value if condition else false_value

"switch" statement using dictionary
def function_1 (...):
functions = {'a': function_1,
'b': function_2,
'c': self.method_1, ...}
func = functions[value]

list comprehension with multiple ifs/fors
>>> result = [x for x in range(10) if x % 2 == 0 if x % 3 == 0]
>>> result
[0, 6]
>>> result = [ x*y for x in range(10) if x%2 == 0 for y in range(10) if y % 3 == 0]
>>> result
[0, 0, 0, 0, 0, 6, 12, 18, 0, 12, 24, 36, 0, 18, 36, 54, 0, 24, 48, 72]

turn a list into a string and back again
>>> ' '.join(['a', 'b', 'c'])
'a b c'
>>> _.split()
['a', 'b', 'c']

more "mathy" numeric boolean comparison
>>> 4<6<8>>> 1<2<3<4<5>4>3>2>1

any() and all() (2.5 only)
>>> any([False, False, False])
>>> any([False, False, True])
>>> all([True, True, False])
>>> all([True,True,True])

switch values without a temporary variable
>>> a,b = b,a

Drop into a python debugging session when you hit this line
import pdb ; pdb.set_trace()

Zen of Python easter egg:
>>> import this

Reverse a string:
>>> 'string'[::-1]

Quicksort in three lines of code (pure propaganda piece, don't use in real code... from Python Cookbook, p.215)
def qsort(L):
if len(L) &lt;= 1: return L
return qsort([lt for lt in L[1:] if lt &lt; L[0]]) + L[0:1] + qsort([ge for ge in L[1:] if ge >= L[0]])

Create a 3-D array initialized to 1(when working with large arrays, always use Numeric)
import Numeric as N
a = N.empty(2*3*4)
for i, x in enumerate(a): a[i] = 1
a = N.reshape(a, (2,3,4))

Enumerate and zip at the same time
for (i,x),y,z in zip(enumerate(a),b,c): print i,x,y,z

TIPS (code samples taken from Dive Into Python)
  • use k in d instead of k in d.keys() for checking for a key in a dictionary (the former is a built-in call to C code and a O(1) operation rather than a Python module with O(n)
  • enumerate(seq) returns index/value pair for a sequence: use it in for loops when you need both values (NEVER use len(range(seq)))
  • Introspection: dir(obj) returns a list of all attributes of obj; vars(obj) returns a dictionary of the same (incl. the values of those attributes); don't forget about locals() and globals(), either
  • isinstance(obj, class) tells whether obj is an instance of class... use it instead of if type(X) == type(Y)
  • callable(obj) returns boolean indicating weather the object is callable or not
  • getattr(x, 'y', default) is the reflection-y way of doing x.y
  • Assign multiple values at once: x , y, z = 1, 'blah', function
  • Python supports filename globbing: use the glob function in the glob module
  • Regular expressions: always use raw strings to avoid excessive backslashing
    • re.sub(r'\bROAD$', 'RD', stringToModify) substitution
    • re.compile for more complicated/reoccuring tasks
      • phonePattern = re.compile(r'^(\d{3})-(\d{3})-(\d{4})$')
    •'[^aeiou]y$', 'vacancy') search
  • You can pass each item of a sequence (instead of the entire sequence itself) into a function that supports it (i.e. defined such as def fn(*x)) by calling it: fn(*seq) If seq is a dict, the function call passes in each key.
  • Change a bunch of non-object variables by passing them into a function and returning them all in a tuple
  • Use the timeit module for small code snippets
  • If you want "true" division, (5/3 = 1.67 instead of 1) use 5//3 or add from __future__ import division to the beginning of your module
  • Use sys.stdout.write() instead of print to avoid trailing characters when writing to stdout
  • You can create a dictionary that acts like a list by using integers (0...n) as keys
  • Python one-liners are tricky (usually in form python -c "code"... see this thread
  • __init__ doesn't "construct" something... the object has already been constructed by the time the interpreter hits __init__
  • Keys of dictionaries must be hashable... that means lists as keys are right out
  • EVERYTHING in python is an object (try it at the interpreter if you don't believe me)
  • self should always be the first argument for every function in a class. You must bind all passed in variables (self.value = value) to make them attributes of the class (preferably in __init__). Refer to class attributes within a class as self.value.
  • When you add a value pair to a dictionary whose key already exists in the dictionary, the new value pair replaces the old one
  • Adding __ to a class attribute (e.g. __attributeX) makes it module-private
  • Don't use optional arguments to initialize mutable objects! Any optional arguments (e.g. def f(x, y=[])) get assigned during evaluation, not execution. Use this form instead: def(x, y=None): if y is None: y = []
  • Python raises an exception when __init__ returns anything other than None
  • a = b and a = copy.copy(b) both assign a to the same object as b (shallow copy). For a deep copy, use copy.deepcopy
  • os.walk doesn't change the current directory, so using something like os.path.abspath(filename) will NOT work... use os.path.join(top, dirpath, filename) instead
  • comparing a string to a integer works! (when operations do not!)
  • If you get a RuntimeError: dictionary changed size during iteration, try changing for x in y to for x in y.keys()
  • More gotchas: 10 Python Pitfalls, Python Gotchas, When Pythons Attack

Thursday, September 21, 2006

Big Day

2 things happened today:

1) Got my awesome, awesome Dell 2407 monitor:

2) Ordered some parts off of newegg so I can use this montor at more than the crappy resolution of my laptop (and, y'know, for some other functionality...):
  • AMD X2 64-bit 3800+ CPU
  • Seagate 320G hard drive
  • Cheap LiteOn DVD burner
  • Cheap HP Deskjet printer
  • Cooler Master case (with 430W power supply)
  • Asus A8N5X Motherboard
  • PNY GF6600 AGP 256M Graphics Card
  • 2G Corsair RAM
And, all the accessories:
  • A blue keyboard with a superman logo emblazoned on it
  • Logitech (best company ever for cheap and excellent peripherals) optical mouse
  • Buffalo Wireless PCI card
  • Arctic Silver Thermal Paste (I never use the cheap stuff that comes on the cooling unit)
  • Power strip
All parts (save the monitor) clocked in at under 1000 bucks, including shipping and taxes. Can't wait to get my dirty little paws on it next week...

Update: screwed up on some stuff... RMAed the video card because it was the wrong interface (stupid! stupid!)... got a VGA XFX|GF 7300GS 256MB PVT72GWANG to replace it... and that keyboard is slim and lame... got a stripped down logitech one to replace it.

Thursday, September 14, 2006

GNU Coreutils/other useful UNIXy stuff

There are a lot of useful things in the GNU coreutils that are useful but many people (me) had no idea that they even exist... here are a few:
  • sort sort the contents of a file
    • -r reverse
    • -f ignore case
    • -n numeric
    • -b ignore leading blanks
    • -c check if sorted (no printout = sorted)
  • uniq remove duplicate lines from a sorted file
    • -c sort and count number of occurances
    • -i ignore case
Of course, if you ever want any information about the coreutils: info coreutils

Ubuntu System Beep

Hate the system beep? Get rid of it with rmmod pcspkr
Nostalgic for the system beep? Bring it back with modprobe pcspkr

Tip from: Lifehacker's top 10 Ubuntu tips and tweaks

Also, you can find it in GNOME at System->Preferences->Sound->System Beep

Wednesday, September 06, 2006

Everything you've ever wanted to know about GNOME's GNOME Desktop User Guide

Some useful keyboard shortcuts from another page:
  • Alt+Tab Switch between windows. A list of windows that you can select is displayed. Release the keys to select a window. You can press the Shift key to cycle through the windows in reverse order.
  • Alt+F4 Close the currently focused window.
  • Alt+F5 Unmaximize the current window, if it is maximized.
  • Alt+F7 Move the currently focused window. After pressing this shortcut, you can move the window using either the mouse or the arrow keys. To finish the move, click the mouse or press any key on the keyboard.
  • Alt+F8 Resize the currently focused window. After pressing this shortcut, you can resize the window using either the mouse or the arrow keys. To finish the resize, click the mouse or press any key on the keyboard.
  • Alt+F9 Minimize the current window.
  • Alt+F10 Maximize the current window.
  • Alt+spacebar Open the window menu for the currently selected window. The window menu allows you to perform actions on the window, such as minimizing, moving between workspaces, and closing.
  • Shift+Ctrl+Alt+Arrow keys Move the current window to another workspace in the specified direction.
  • Ctrl++ enlarge the screen/text
  • Ctrl+- shrink the screen/text
All of these settings are customizable: go to System->Preferences and there's a bunch of menus there that you can use to trick out your setup (such as Keyboard Shortcuts). I set those annoying windows keys that are next to the alt keys to actually mean something in linux. The right one (Super_R) I use to hide all the windows on the screen and the left one (Super_L) I use to launch a terminal.

UPDATE: To get Nautilus in GNOME to have the location bar (like default in KDE): Open a Nautilis window->Edit->Preferences->Behavior->Always use text-entry location bar ... There's a host of other useful display/behavior options there, too.

Tuesday, September 05, 2006

Back to school

Since I've been back at college, I've been using some things that I don't use otherwise... it's quite a different world here than in Silicon Valley.
  • Spellcheck
    • aspell -c <file> spellcheck a file
    • aspell -a interactive spellchecker
  • PDF-related
    • evince &lt;file> to launch a GUI PDF viewer
    • pdftohtml &lt;file> to create three files, one which contains a table of contents, another which contains the text, and a third which combines both with frames
    • gs ghostscript
    • apropos pdf list things that your system can do with PDF files... there's a lot of them!
  • How to open a new tab in a GNOME terminal (How is this back to school? Well, I'm trying to eliminate all mouse usage now that I'm using this laptop everywhere...)
    • Ctrl+Shift+T Open tab
    • Ctrl+Shift+W Close tab
    • Ctrl+PageUp/Down select Previous/Next tab
    • Ctrl+Shift+PageUp/Down move terminal tab to the left/right
    • Alt+&lt;number> select specific terminal tab by number
Some things I'd like to learn (anyone out there know?):
  • How to insert unicode characters in vi
  • Invoke a spellchecker from within vi

Thursday, August 24, 2006

Firefox: Extensions and more

Here are some sweet Firefox extensions that I use (if this list isn't enough for you, there's more of them on the Extensions Download Page ):
  • Colorful Tabs An easy way to distinguish between tabs when you have a million open
  • Customize Google a MUST HAVE for frequent users of all Google services... ad blocking, encryption, etc.
  • Social bookmarking site button in your browser! For those of us sick of managing bookmark files. (The classic (worse) version is here)
  • FireBug If you develop web pages without this, you are either uninformed or massochistic
  • YSlow Yahoo tells you why your site is slow (runs on top of FireBug)
  • FireFTP: Why didn't someone think of this sooner? Great idea and execution of an FTP client within a browser.
  • FireGPG Use GPG to encrypt, sign and decrypt email (or any web page text)
  • Google Toolbar 3 Absolute bliss... allows you to add any arbitrary search box as a button in the toolbar and lots of other sweet features that I will not know how to part with
  • Greasemonkey Inject arbitrary Javascript into a page; a great vault of these scripts is found at
  • Nuke anything enhanced Ahh, the pure bliss of deleting something annoying off of a page. Unfortunately, it doesn't seem to want to work with Flash...
  • Viamatic foXpose An OSX-style feature; puts all of your tabs into view at the same time when you hit a button in the chrome of your browser
  • Video DownloadHelper this extension + YouTube = Napster
Other extensions of note:
  • Forecastfox Have to put this one in; my buddy Jon wrote it: displays the local weather in your toolbar
  • Fasterfox Increases your browser speed a minuscule amount and has more popup-blocking capabilities; a good review of it here
  • Chickenfoot Greasemonkey++; saw this demoed... it's great, and only getting better; lots of applications throughout the interweb world
  • Bugmenot Great for getting past login pages that you don't have a login or password for. This activity is illegal, also, perhaps?
  • Adblock Does what it says. Allows contextual filtering of unwanted material from web pages
  • Showcase The same idea as Viamatic foXpose, different implementation.
  • SessionSaver Saves your tabs, text input, etc. from when firefox last closed and reinstates them in your next Firefox session
  • Flashblock: 99% of the time, when a web site uses flash, you don't want to see it. This replaces the flash output with a little button that, if clicked, allows you to see the desired content
  • WebDeveloper: Useful for those who need to debug web pages
  • DownThemAll Great for pulling large quantities off of one page (for me, I use this mostly before I know I'm going to be stranded without internet access for a while)
  • Chatzilla IRC client within the browser (FAQ)
  • MeasureIt For measuring pixels, etc. on your browser screen... simple, but useful
  • View Source Chart for visualizing HTML blocks in the code source
  • Vimperator vim-like commands for Firefox
  • BetterPrivacy prevents Flash cookies from being stored on your computer
Articles about Firefox extensions:
  • For tweaking Firefox 2 to your personal satisfaction: Lifehacker's top Firefox config tweaks (mostly deals with about:config) and Computerworld's guide
  • To search for something real fast, use the forwardslash key as in vi (or the single quote if you only want to search for links on a page). Ctrl+G moves you forward and Ctrl+Shift+G moves you backwards in searches.
  • Here are instructions for installing firefox 2 on older Ububntu distros (pre-Edgy) that have 1.5 installed by default.
  • For Google-Toolbar-like quickness built into Firefox, right click on your favorite search box and select "Add a keyword for this search"... you'll then be able to search, if you alias search to "g" for example, Google by typing "g <query>" in the URL box
  • Set a master password for logging into sites: instructions here
  • Get the source of a Firefox extension with these instructions