chainlynx / c|-|41nl`/n><

How to get a RSS feed from a Twitter account

2011-05-09T19:51:00.000-07:00

So it seems that Twitter has killed RSS feeds (I'm not sure what that has to do with implementing OAuth), but fortunately, there's an easy workaround.

You can get an RSS feed by querying another URL for the desired user. For instance, to get the RSS feed for user ggreenwald, you can find the feed at this url: http://twitter.com/statuses/user_timeline/ggreenwald.rss

Facebook Wall question

2011-03-12T10:29:00.000-08:00

I sometimes have the question asked of me, "Danny, you are such a strong advocate of transparency and access; aren't you being a hypocrite by not allowing others to comment on your Facebook wall?"

This has always struck me as a rather strange question because it confuses the obligations of a person with the obligations of a state or corporation. It is true that I think that, say, governments should be relatively transparent -- that's a prerequisite for accountability. I, on the other hand, am a priori accountable to no one so I don't have those same obligations.

This accusation is analogous to claiming that WikiLeaks is a hypocritical organization because its members don't share all their internal communication with the world (many in the mainstream media do make this point after each release from WikiLeaks in order to discredit the organization). As if 1) there is any equivalence between the transparency obligations of states/corporations and activist groups or 2) WikiLeaks isn't facing an existential threat from organizations that would use that internal information to try and destroy the group.

But that aside, there is a valid question contained within the query. Why do I have the personal preference of configuring my Facebook Wall settings in such a way? I just don't feel that it is worth the time and mental strain to police my Facebook profile 24/7 to expunge things said by others (whether said out of malice, ignorance or superfluousness) that I would rather not have on there. There are sufficient potentially undesirable comments that may be put on one's profile for this to be a valid concern.

Multidimensional arrays with GCC's variable length arrays

2010-05-22T19:10:00.000-07:00

GCC's extensions allow you to do some crazy things with multidimensional arrays. For C99, GCC implements variable-length arrays. So a declaration like this:

int matrix[a][b + 1];

Can be passed by reference to a function with this prototype:

void foo(int a, int b, int matrix[a][b+1]);

Or, if you want to get really crazy, you can use forward declarations and change around the parameter order:

void foo(int a; int b; int matrix[a][b+1], int a, int b);

No malloc needed! Much better than the old way.

Which application is using that port?

2010-04-10T08:22:00.000-07:00

See which ports are open, as an attacker would:

nmap 127.0.0.1

(Superuser seems to need to be used on some the following commands)

See which process is using port 25:

netstat -nlp | grep 25

Same, with a bit less info:

fuser -n tcp 25

Or you could also do:

fuser -u smtp/tcp

Discovered here

Sansa Clip and Karmic

2010-03-17T19:34:00.000-07:00

Upon upgrade to Karmic Koala Ubuntu (9.10), my trusty Sansa Clip player was not autodetecting like it did under the Ubuntu distribution I was previously using. So I scoured the web and found this site which gives a workaround: when the Sansa Clip is disconnected, go to Settings->USB Mode on the Sansa Clip and set it to MSC. It should now present itself to the computer as a normal file system. Unless, of course, the file system is screwed up, in which case you'll have to go to Settings->Format and format the Sansa Clip drive before connecting it again.

Funny C tricks

2010-03-05T21:52:00.000-08:00

Taken from Bill Rowan's Stanford ACM presentation:

The "downto" operator:


int main()
{
int i = 5;
while(i --> 0) // --> is the downto operator!
{
  printf("%d\n", i);
}
return 0;
}

Cast any type to "bool" type (that is, 1 or 0):

!!foo

"Computed goto" (compiler-dependent && unary operator):

void print_loop(int s, int e) {
 assert(s < e);   
 top:    
 printf("%d\n", s);  
 goto *( &&top + ( !!(s++/e) ) * ( &&end - &&top ) );    
 end:  
 ;
}

Make a big file into many small files (and back again)

2010-01-12T18:31:00.000-08:00

Adapted from this slashdot thread

Create many little files called "chunks00000" etc. from bigfile that are all just under 10MB in size: split -a 5 -b 10MB -d bigfile chunks

Put the chunks back together: cat * > newbigfile

Do the same, except with compression: tar -zc bigfile | split -a 5 -b 10MB -d - chunks

Put the compressed chunks back together: cat chunks000* | tar -zxO > newbigfile

Browsing as Googlebot to circumvent paywalls

2009-12-02T07:30:00.000-08:00

Interesting technique from a Slashdot article and commenter darthflo:

Most 'papers like Google and the visitors Google sends them; so the Google Bot and hits with a google.com Referer tend to get a free pass. Use this to your advantage:
Google the Article's URI, click the link and off you go (with a real Google referer).
If it's not indexed yet and you're using Opera: Go to any Google page, press Ctrl + U, change any one link's href to the article's URI, click "Save Changes", click the link and off you go (with a fake Google referer. This works for any fake referer, by the way).
If they're picky, they mightn't let hits from Google through but still allow the Google bot to index their pages. Change your User-Agent accordingly. In Firefox, go to about:config and change general.useragent.extra.firefox to Googlebot 2.1 and off you go (as Googlebot).
As a last resort, there's quite a few ad-blocking personal proxies out there. Most of them allow you to fake Referers or change User-Agents, for any browser.

Insidious C bugs

2009-10-25T12:38:00.000-07:00

Here's a couple novel C bugs that I've encountered recently (using gcc). Man, it sucks when these happen.

1) No compiler warnings when not instantiating structs correctly

The following code will give no warning indicating that you have not, when declaring an array of structs using the curly bracket notation, instantiated all members of the struct:

#include <stdio.h>

int main()
{
  struct a_struct {
     int a;
     int b;
  } buggy[] = {
     { 1, 2, },
     { 3, /* oops! */ },
     { 5, 6, },
  };

  int i;
  for(i = 0; i < sizeof(buggy) / sizeof(*buggy); i++)
  {
     printf("%i: %i %i\n", i, buggy[i].a, buggy[i].b);
  }

  return 0;
}

Here's the ouptut:

0: 1 2
1: 3 0
2: 5 6

Oops! It would have been nice for the compiler to tell us about this...

2) Forgetting to delete the semicolon when turning an assert into an if statement

Say you have

assert(cond);

But you want to make it an if statement to have a little more debugging output when it triggers. So you change it to:

if(!cond);  /* oops! */
{
  dump_stats();
  assert(0);
}

Whoops! Now the if statement has an empty body and the block between the curly braces will always execute!

3) Strange things you can't do within case statements

This isn't really insidious so much as annoying -- why are you allowed to do some things inside a case statement (that don't seem to make sense) but aren't allowed to do others (that seem to make much more sense)? For instance:


int main() {
  int i = 0;

  switch(i)
  {
     int j = 5;    /* fine */
     case 0:
        char ch;  /* error */
        break;
     case 1:
        i = 1;
        char ch2; /* fine */
        break;
     default:;   
        char ch3; /* fine */
        break;
  }

  return i;
}

Note the semicolon following the 'default:' There's a good discussion of this problem here. Essentially, the first thing after a label cannot be a declaration. The error that the compiler gives here is usually pretty cryptic, too.

Similar links

Top 10 Ways to get screwed by the C programming language

Youtube kung-fu on Linux

2009-10-04T23:50:00.000-07:00

This is just a collection of commands that I have used in the past to edit video, especially for uploading to online video sites such as YouTube. This entry is small now, hopefully it grows as I figure out how to do more useful stuff.


# Cut a clip from a whole video file (-ss is base, -t is offset)
ffmpeg -sameq -i input.avi -ss 00:1:23 -t 00:02:35 output.avi

# Boost the volume of a video file
ffmpeg -sameq -vol 1000 -i input.avi output.avi
or, see Avidemux tutorial here: http://www.youtube.com/watch?v=w8K0D2t0ysE

# Dump audio of an flv video file to mp3
ffmpeg -i inputfile.flv -f mp3 -vn -acodec copy ouputfile.mp3

Mandating key-based logins with sshd

2009-09-23T10:47:00.000-07:00

This page provides a pretty good summary of creating a public/private key pair on the client and adding the public key to the server's authorized_keys file.

On the server side, edit /etc/ssh/sshd_config and add the directive PasswordAuthentication no and then restart sshd with sudo /etc/init.d/ssh restart Anyone that tries to log in to your server without a public key in the server's authorized_keys file will now not even be given the chance to enter a password.

sshd_config manpage

xmonad

2009-09-21T18:01:00.000-07:00

Xmonad is a sweet window manager. So sweet, in fact, that I've decided to use it. Here's a quick reference, basically the manpage rearranged (mod = alt by default) ... (also, remember to change the session login to "Run Xclient script" so that other things, like xmodmap, can be run in .xsession):


=== Launch ===
mod-shift-q -- Quit xmonad
mod-q -- Restart xmonad (and reload config file)
mod-shift-return -- Launch terminal
mod-shift-c -- Close the focused window
mod-p -- Launch dmenu

=== Layout ===
mod-space -- Rotate through the available layout algorithms
mod-shift-space -- Reset the layouts on the current workspace to default

=== Focus ===
mod-tab -- Move focus to the next window
mod-shift-tab -- Move focus to the previous window
mod-j -- Move focus to the next window
mod-k -- Move focus to the previous window
mod-m -- Move focus to the master window

=== Swap ===
mod-return -- Swap the focused window and the master window
mod-shift-j -- Swap the focused window with the next window
mod-shift-k -- Swap the focused window with the previous window

=== Resize ===
mod-h -- Shrink the master area
mod-l -- Expand the master area

=== Master Area (the area on the left) ===
mod-comma -- Increment the number of windows in the master area
mod-period -- Deincrement the number of windows in the master area

=== Workspace ===
mod-[1..9] -- Switch to workspace N
mod-shift-[1..9] -- Move client to workspace N

=== Multiple Screens ===
mod-{w,e,r} -- Switch to physical/Xinerama screens 1, 2, or 3
mod-shift-{w,e,r} -- Move client to screen 1, 2, or 3

=== Mouse ===
mod-button1 -- Set the window to floating mode and move by dragging
mod-button2 -- Raise the window to the top of the stack
mod-button3 -- Set the window to floating mode and resize by dragging
mod-t -- Push window back into tiling

=== Setup ===
sudo apt-get install xmonad dwm xlockmore xclock htop

=== ? ===
mod-shift-p -- Launch gmrun
mod-n -- Resize viewed windows to the correct size
mod-b -- Toggle the status bar gap

Here are some other useful utilities to replace functionality that is present in, say, GNOME on the command line:

xclock -- tell the time
xlock -- lock the X window session
htop -- a better system monitor
alsamixer -- control volume, etc.
mp3blaster -- play music
cal -- calendar

How to install Xmobar

First, install the necessary packages:
sudo apt-get install libghc6-http-dev libghc6-zlib-dev libx11-dev libxft-dev

I hadn't installed any packages with cabal before, so I needed to set up that as well. I found it necessary to do the standard download, untar, runhaskell Setup configure && runhaskell Setup build && sudo runhaskell Setup install for the following packages:

http://hackage.haskell.org/packages/archive/HTTP/4000.0.8/HTTP-4000.0.8.tar.gz
http://haskell.org/cabal/release/cabal-1.6.0.2/Cabal-1.6.0.2.tar.gz
http://haskell.org/cabal/release/cabal-install-0.6.2/cabal-install-0.6.2.tar.gz

A cabal update initializes the package database and a cabal install xmobar grabs the package.

Put this in your xmonad.hs (changing the paths appropriately, of course):


import XMonad
import XMonad.Hooks.DynamicLog
import XMonad.Hooks.ManageDocks
import XMonad.Util.Run(spawnPipe)
import System.IO

main = do
xmproc <- spawnPipe "/home/dannyc/.cabal/bin/xmobar /home/dannyc/.xmonad/xmobarrc"
xmonad $ defaultConfig {
 layoutHook = avoidStruts  $  layoutHook defaultConfig
, logHook = dynamicLogWithPP $ xmobarPP
 { ppOutput = hPutStrLn xmproc
 , ppTitle = xmobarColor "green" "" . shorten 50
 }
}

And put this in your xmobarrc:

Config { font = "-*-Fixed-Bold-R-Normal-*-13-*-*-*-*-*-*-*"
 , bgColor = "black"
 , fgColor = "grey"
 , position = TopW L 90
 , commands = [ Run Weather "EGPF" ["-t"," <tempF>F","-L","64","-H","77",
"--normal","green","--high","red","--low","lightblue"] 36000
              , Run Cpu ["-L","3","-H","50","--normal","green","--high","red"] 10
              , Run Memory ["-t","Mem: <usedratio> 10
              , Run Swap [] 10
              , Run Date "%a %b %_d %l:%M" "date" 10
              , Run StdinReader
              ]
 , sepChar = "%"
 , alignSep = "}{"
 , template = "%StdinReader% }{ %cpu% | %memory% * %swap%    <fc=#ee9a00>date%<fc> | %EGPF%"
 }

There, you now have a bare-bones xmobar install. I figured this out by loosely following the instructions here. I don't know any haskell (yet) and this is my first time messing with the xmonad config, so I don't guarantee at all that I am doing things the Right Way. If your xmonad keeps locking up because of this configuration, see the big bold comment in the link above which points to here.

TODO

customize xmobar
some kind of screen-like titleing for workspaces
better fonts?
picture background on startup (/transparent windows?)

Video DownloadHelper Firefox extension + YouTube = Napster

2009-08-24T19:34:00.001-07:00

Video DownloadHelper can download videos from YouTube (or other video sites) and convert the audio to mp3 (or other formats) using ffmpeg (the default). Pick the mp4 (high quality / high definition -- these are format 18 or 22) videos off of YouTube to get the highest quality audio (128 kbps).

Ah, this brings me back...

Update: Even better, now that Lala, et al are providing music streaming straight from a Google search, one need not even bother with YouTube -- faster download/conversion, and same (better?) quality.

Photo metadata

2009-07-25T14:06:00.000-07:00

# get exiftool
sudo apt-get install libimage-exiftool-perl

# get libexif CLI tool
sudo apt-get install exif

# erase all metadata
exiftool -all='' img.jpg

# see what metadata tags your photo has
exif -l img.jpg

# see the values of the included metadata tags
exif img.jpg

EXIF specifications: http://www.exif.org/specifications.html
For PNGs: http://pmt.sourceforge.net/pngmeta/index.html

And now, a demonstration in how ridiculously easy it is to hack stuff with Google

2009-07-23T22:48:00.001-07:00

Inspired by this reddit thread (look here for help deciphering the Google syntax).

phpMyAdmin -- inurl:SELECT inurl:FROM inurl:WHERE intitle:phpmyadmin

HP Laserjet printers -- "identify the document you want to print by using either option shown below, then select the apply button."

More printers -- "Web Image Monitor" location comment "device name"

Web cams -- inurl:"viewerframe mode="

And, let's not forget the Google Hacking Database or how to use Google to crack MD5-hashed passwords!

batch: like at, but better

2009-07-12T10:18:00.001-07:00

batch is a tool that is identical to at (same manpage even), except it runs a specified job when the load averages are low instead of at a certain time. Very handy if one needs a job to execute on a busy server and doesn't care when it happens, just that it happens.

Stanford students 'better than other people'?

2009-05-31T14:37:00.000-07:00

A telling quote from a Stanford student in today's San Jose Mercury News front-page story:

"When they welcome you to freshman orientation," Robbins says, "part of what they tell you is that you're better than other people."

Not smarter, not better at standardized test scores or kissing ass to get ahead, but unequivocally superior human beings than all those proles that attend state schools. I have to say, this quote typifies the holier-than-thou, aristocratic, snub-nosed attitude that runs through a lot of the student body (and faculty).

My .pythonrc.py

2009-05-30T00:24:00.001-07:00


#!/usr/bin/python

import sys
import os
import atexit

# color prompt
sys.ps1 = '\001\033[1;36m\002>>> \001\033[0m\002'

# tab completion
# from http://www.doughellmann.com/PyMOTW/rlcompleter/index.html
try:
 import readline
except ImportError:
 # Silently ignore missing readline module
 pass
else:
 import rlcompleter
 readline.parse_and_bind("tab: complete")

# history
# from http://dotfiles.org/~remote/.pythonrc.py
histfile = os.path.join(os.environ["HOME"], ".python_history")
try:
 readline.read_history_file(histfile)
except IOError:
 pass

atexit.register(readline.write_history_file, histfile)
del os, histfile

If running 'python' on the command line does not run the file, you can always alias python to 'python -i ~/.pythonrc.py'

Critique of Laughlin's "The Crime of Reason"

2009-05-26T23:21:00.000-07:00

When I saw Robert Laughlin speak at Stanford, I was terribly disturbed by some of his ideas and intrigued by others. As someone who drinks the free software/culture kool-aid, I was always of the opinion that the free flow of information helped humanity, not hurt it. As a follow up on his talk, I decided to pick up a copy of Laughlin's book, The Crime of Reason, to investigate his ideas further. The book, like is talk, contains some very interesting and important ideas but is poorly put together and rather meandering. Writing style aside, the book discusses the following topics, which I will comment on in turn:

The Intellectual Property system is necessary for economic progress

Laughlin states that "Universal access to knowledge is fundamentally incompatible with market economics." (p.45) By this he mostly means that the patent system in the United States is necessary for economic development. He doesn't provide any convincing evidence to back this up, but merely provides a false analogy equating the economy with a game of poker in which everyone has incentives to hide and steal from each other. Perhaps these dynamics apply in certain sectors of the economy (particularly looking through the lens of a physicist who has spent his entire life inside the bowels of the military-industrial complex), but in other cases they do not. The open-source software industry immediately springs to mind as an example in which companies have an interest in freely sharing knowledge with each other.

In any event, the idea of the necessity of a patent system has been completely eviscerated by Boldrin and Levine in their book Against Intellectual Monopoly. Looking at history, the acquisition of a patent in a particular field coincided with a stall of progress in an industry until the patent expired and, not coincidentally, substantially increased patent lawsuits within that industry as the patent holder sought to restrict anyone else from innovating. Patents are a type of monopoly and, as any econ 101 student will tell you, monopolies are a Bad Thing because they deprive both consumers of a low cost for products and also prevent other potential producers from making money by entering the market. Awarding patents is hardly "necessary for living" (p.49) as Laughlin claims.

How and why technical knowledge becomes illegal

The main thrust of the book has to do with the troubling tendency of modern societies to effectively outlaw knowledge. Laughlin compiles a list of fields today whose study has been criminalized in some form or another:

- cryptography
- circumvention (DMCA)
- physics (nukes)
- genetics
- bioengineering (engineered diseases)
- biology (cloning, chimeras)
- national security related processes
- chemistry
- etc.

He asserts that learning about these fields has been criminalized either in the law itself (which is rarely challenged in open court because of the potential government 'secrets' a trial could leak) or by de facto means such as withdrawal of research funds or public ostracism. This development, of course, is quite at odds with the way that learning is supposed to work in our society, as Laughlin recognizes: "Modern civilization rests on two mutually exclusive kinds of thinking -- one embodied in the free speech guarantees in the First Amendment of the U.S. constitution, the other in the Atomic Energy Act." (p.82) His most effective case in point is that of nuclear physics, in which the U.S. government has led a campaign of a quasi-legal nature to suppress the spreading of knowledge on the subject. He reasons that this censorship "set a precedent that has now led, by small steps, to a significant and growing threat to our freedom to reason and learn." (p.84) This is the most convincing, and consequential, argument of the book, and deserves serious thought by all members of our government and society. Are we really willing to sacrifice our freedoms to pursue intellectually interesting scientific facts for the sake of purported security, morality and order?

The consequences for a society which deems scientific knowledge illegal

In the final chapter, Laughlin conducts a thought experiment: what will smart people do if and when we achieve this nightmare society in which the pursuit of any and all interesting technical knowledge is illegal? Laughlin's suggestion that "The sensible course of action would probably be to give up" (p.144) is deeply unsatisfying. He then postulates that the talented technical folk (that is, everyone that didn't become a doctor or a lawyer or a businessman) will either seek employment in the service of rogue dictatorships that allow science, become 'guerrilla warriors' of a sort within their own country or go somewhere else (in the interplanetary sense) to establish a new society where there is no crime of reason. It's very romantic to think of the creation of a new order by a disgruntled segment of society (a la the emigration of persecuted religious groups to America). If it is necessary, however, is another matter entirely. It's certainly way too early, in my opinion, to 'give up' on our present society. A more enlightened public debate on this topic, if not reform, is not out of reach.

You can read other peoples' opinions on the book on Amazon

Twitter: Good or Evil (or Irrelevant or Same ol')?

2009-05-21T22:04:00.000-07:00

Everyone seems to be talking about and using Twitter these days (I'm not on it and have no plans to join in the immediate future). It seems as though Twitter is the newest wave of social media hype, for better or for worse. If you cut through the frenzied enthusiasm about Twitter, however, you get a picture of a service that is just another method of communication, with all the positives and negatives that that provides (albeit with a unique short-message twist).

Twitter has proven itself as one of a number of social media platforms that can be used to report breaking news events faster than traditional news reporters can arrive on the scene. The most important and striking example of this was the coverage of the Mumbai Terror Attacks. Other sites, notably Wikipedia and Flickr, also contained up-to-the-minute details of the attack that mainstream media sites such as CNN cribbed for their stories on the incident. Twitter has also been used by activists to organize and rapidly disseminate information. Consider reporters in Egypt that were able to alert colleagues to their arrest via Twitter. Others recently mounted a campaign to expose Amazon.com's system that placed gay and lesbian-related items lower in the site rankings.

But Twitter not only has the power to educate and organize, it also has the power to misinform. Consider the recent swine flu panic: Twitter users reacted to the global scare by essentially amplifying the pig paranoia rather than providing any useful information about the disease. The 'social' factor of Twitter exacerbates this phenomena: often users post not to communicate substance, but to fit in. The result is a hysterical echo chamber of misinformation. (In fact, the author of the linked Foreign Policy article speculates that Twitter would be a very good medium for someone who wanted to intentionally incite fear in the populace.) Valleywag nicely summarizes this point: "What Twitter actually does is inflate problems out of all proportion, as Twitterers noisily tweet about how with it, on it, and over it they all are, repeating each other's messages without adding anything of value. Any [person looking to inform themselves] would go mad long before he extracted useful information." The desire of many Twitter users to be on the cutting edge of news also enhances their gullibility, and the lack of context inherent in 140 character posts makes it much easier to pull the wool over a reader's eyes. Twitter users fell for the faux news items of Patrick Swaze's death and nefarious items snuck into Obama's stimulus package, to cite only two examples.

The fact that entries on Twitter are limited to 140 characters apiece makes it difficult to convey much useful insight in one post. Many use Twitter for precisely that reason -- they have little or nothing to say. Glenn Greenwald nails it: "About Twitter messages, John says 'they all read like cell phone text messages between 12 year olds,' and indeed, the only purpose I can discern is that it provides a format for expressing thoughts that are too inconsequential to merit a stand-alone article or post. For precisely that reason, it is unsurprising that Twitter has become a huge hit among our media stars, for whom triviality is a guiding principle." Appropriately, a vast cult of celebrity Twitter worship has emerged, with eager fans eating up every last tasteless morsel that is tossed to them by their gods. The fact that Twitter is often an outlet for the mundane is hammered home by spoof sites such as MyLifeIsAverage.

Quite possibly the most devastating critique of Twitter is not that it incites fear or inhabited by vapid users, but that there is simply nothing special about it -- that it is more of the same. Seth Finkelstein maintains that Twitter is just another sucker's game that only serves the needs of a tiny elite: 'After I saw Twitter in use, I realised the difference was that, while IRC had all participants equal, Twitter implements a distilled version of many problematic aspects of blogging. Namely, a one-to-many broadcasting system that serves the needs of high-attention individuals, combined with an appeal to low-attention individuals that the details of one's life matter to an audience... Twitter is low-level celebrity for the chattering class. And the pathologies of celebrity are all on display, including the exploitative industries that prey on the human desire to be heard and noticed. My answer to Twitter's slogan of "What are you doing?" is: 'Not playing a sucker's game.'" Twitter, in other words, is just another way for the powerful to broadcast their message and for advertisers to blast users with pitches for their newest products and peer into consumers' minds, all the while deluding the average user that it's an empowering service.

Clearly, Twitter is many things to many people. Perhaps that's the only conclusion that one can draw from such a myriad of uses. Twitter is a communication medium and, like any other one, can be used and abused for just about any purpose. Although Twitter encourages its own unique kind of communication from being a 140-character accepting social media service, many of these sites' quirks are simply a reflection of their users, and it's wrong to blame the tool for having too much influence in shaping what people do with it. As one responder to Greenwald put it, "Criticizing the form [of Twitter] is like criticizing haiku as a form."

UPDATE:

TechCrunch takes a pessimistic viewpoint, in the context of the Fort Hood Massacre.
Valleywag lays down the rules for the manipulate-the-gullible-public-into-believing-someone-is-dead-when-they're-not game
Joel Spolsky also has some unkind words for Twitter
Study: Men follow Men and Nobody Tweets

I finally caved. danny_colligan is my twitter handle.

.gdbinit file

2009-05-17T15:10:00.000-07:00

Personal settings for gdb -- nothing too elaborate

# color prompt
set prompt \001\033[1;36m\002(gdb) \001\033[0m\002

# history across invocations
set history save on
set history filename ~/.gdb_history

How NOT to recruit software engineers

2009-04-25T16:23:00.000-07:00

After going through Stanford's recruiting process for a summer internship for the first time (and having been contacted sporadically by headhunters for quite some time), I've been genuinely disappointed in a select few of my interactions with recruiters. A certain subset of recruiters commit some boneheaded errors that one might think would be precluded by an iota of common sense. These missteps waste my time and give the recruiters' respective companies a bad reputation with the students. Additionally, I have no reason to think my experience is unique amongst my fellow engineering student peers. Since my blog is actively read by thousands of recruiters worldwide (well, maybe not, but I'll pretend like it is anyway) I have resolved to better their recruiting process by providing them a few helpful tips on what NOT to do when trying to gather talent:

Schedule appointments unilaterally
Remember, the applicant's life already revolves around your company, so have no qualms about telling him when an interview will be. Never take his schedule or conflicts into account. Never ask if he can make an appointment, simply assume he can.

Don't keep appointments
If you schedule an appointment with a recruit, make sure you do not show up. Optionally, show up at a different time and/or place. The same goes for phone calls: try calling at a different time than which you promised.

Don't follow up after interviews
Feel free to cut off contact with the recruit at any time, for any reason or without reason. Resist closure for the applicant. Never tell him if or why he was or was not accepted for the position.

Strive for an inconsistent message
Make sure the recruit has multiple contacts at the company, and make sure each of them is sending him a different message. Give him the impression that working for your company will entail functioning within a hopelessly mismanaged bureaucracy.

Be annoying
Clog the recruit's inbox with as much irrelevant information as possible. Send multiple copies of the same email. Send the same message over several different mediums (phone, email, pager, carrier pigeon, etc.). Ensure that mandatory forms are filled out multiple times. Redouble your efforts after the recruit says he is not available or not interested.

Treat the applicant like a number, not a person
Make sure the recruit knows that he is just another anonymous cog in the corporate machine. Send out obviously templatized emails that start with things such as "Dear $applicantName." Never make exceptions for an individual's extenuating circumstances.

Never apologize
No matter how badly you screw up, never acknowledge that you did anything wrong. Refuse to apologise. Stand your ground, especially in the case of overwhelming evidence to the contrary.

On a related note, if you are a recruiter/interviewer and wondering what you should be doing, please read the authoritative documents on recruiting and interviewing from the Joel on Software blog.

Dispelling Google Latitude Privacy Hysteria

2009-02-17T19:48:00.000-08:00

Google recently came out with a new service called Latitude which allows people to share their locations with each other via a Google Maps interface. [1] Almost immediately, talk about privacy concerns dominated the dialogue concerning Latitude. These fears, upon closer scrutiny, are largely baseless. Latitude does not present a significant danger to users' privacy; any suggestion otherwise is mere technophobia and headline-grabbing Google-bashing.

The most important point in this entire conversation is that your cell phone is already a tracking device in and of itself. Carrying around a cell phone surreptitiously exposes more personal information than Latitude could ever dream of doing. The GPS, wireless Internet, and cell phone signals that emanate from your phone can be used to locate you any time your phone is on. The cell phone companies, obviously, know your location because they need it to deliver you service; the government can get it via Triggerfish or by just asking the phone companies. But a phone can be used as more than just a locator -- it can also be used as an eavesdropper. Consider the well-known NSA surveillance program that slurps up cell phone conversations, or the ability of the government to listen to whatever noise a cell phone picks up even when it is powered off. If you are seriously worried about your privacy, you won't even be carrying around a phone in the first place.

Google Latitude can not honestly be called a privacy threat because it is opt-in at every level and gives one the opportunity to leave or disable the service at any time. For another person to have access to your location, you must 1) explicitly enable Latitude 2) request the other person to receive your location via Latitude or accept a similar request from him 3) not turn the service off. Disabling the service can come in the form of either opting out of Latitude entirely or hiding your location temporarily. You can even enable a 'city-level-only' location option, which only shares your location to the town level of granularity, or set a manuallocation that doesn't move. (Your mobile location can be exactly determined only if you install Latitude on your mobile phone as opposed to using the stationary option.) Again, no one besides the group of people you explicitly agree to share your location with can see your location.

The example scenarios that have been raised by Privacy International with regard to Latitude's purported privacy degradation that have captured headlines are pretty far-fetched. All of the scenarios involve a malicious user creating a Google account, enabling Latitude on a phone and giving the phone to someone else with the intention of tracking them (without, of course, informing them that Latitude is enabled on the phone). Any reasonably competent person would quickly discover that Latitude was enabled on the phone, if he had not inspected the phone in the first place when he initially received it. There are many other major invasions of privacy taking place elsewhere, and Privacy International would do well to raise a stink about those issues rather than chase windmills at the Googleplex.

There is a legitimate privacy concern that Google will store the history of a user's location, which could be used to construct a profile of where a user was at certain points in time. However, Google states in the Latitude FAQ that this is not the case: "Google Latitude only reports your last updated location and does not keep a history of previously reported locations." As long as Google keeps its word in this regard, and I believe that to be a reasonably safe assumption, there is no privacy danger here.

It is unfortunate that so much ado has been made about a service that is essentially a useful visualization of your friend group. [2] Google Latitude is a service that you should have no qualms about using, provided that carrying around a cell phone does not make you queasy.

============================
Footnotes:
[1] As several other commentators on Slashdot pointed out, Google is not the first company to offer this kind of service (Brightkite, Loopt, and Mologogo to name just a few).

[2] One could imagine other use cases: giving truckers cell phones to track their shipments, planning visits to friends based on their proximity to a certain destination, serendipitousmeetup opportunities with nearby friends, etc.

iPhone App Pyramid Scheme Plan

2008-12-14T13:01:00.000-08:00

Where is the next fertile soil for a pyramid scheme? Why, the iPhone Application store, of course! Following these few easy directions, you can scam millions of yuppies for fun and profit.

How: First, create an iPhone Application with some kind of cheesy social-networking appeal (e.g. 'How many friends do you REALLY have?'). It doesn't really matter what the functionality of the app is as long as each install of the app is assigned a unique identifier. The buyer will enter the id of the person who referred them to the app. In this way, an n-ary tree can be constructed (with you at the root) of subsequent referrals and installs of the app. Motivate the spread of the app by promising some amount of money to a user every time someone else buys the app and enters the his or her 'promo code.' Price the app such that you net the difference of the cost of the app and the amount you are paying out on each install.

Why it will work: People who purchase iPhones have a proven record of buying a lot of iPhone applications with their disposable income. It is perfectly reasonable to believe that many of them would not mind spending a few bucks more to purchase another novelty application (indeed, some have proven that they will spend a great deal for a novelty application / status symbol). Second, sites like Facebook have proven users' insatialbe thirst for social applications that connect them to their friends/coworkers/strangers/etc. Many of these applications exploit some kind of emotional dynamic (for instance, who are your 'Top Friends'?) to appeal to users. The sucess of these applications leaves no doubt that other appeals of the same kind would find an audience. Third, the iPhone App store provides a quick and easy means of distribution and, presumably, payment. Finally, every pyramid scheme exploits people's motivation for profit, a powerful motive indeed.

Variables: The setting of a reasonable price and payout is critical. Ask too much and nobody will buy the app; ask too little and you won't make much money. Promise too much as a payout and you won't make much money, promise too little and the app will not virally spread. Another important decision is the appeal of the application -- how are you going to convince uses to buy the application in the first place? What is the most compelling social/emotional/psychological/whatever message that you can send to them that makes them want to buy? Going hand in hand with that, what does the application actually do? Does it simply redirect to a web page showing some statistics of who has recruited the most people to the scheme, or does it have something more sophisticated?

Unknowns: Apps need to be approved by Apple and can be yanked by Apple at any time, with or without explanation. The app needs to be written in such a way that it does not appear to be something that Apple would find questionable. The legality of these practices is another issue. Depending on what the function of the application is, you might be able to make a case that you are actually delivering a product to the user. Another unknown is the most effective way to move many small payments of money between people, and what kind of cost is incurred for those services.

Funny UNIX tricks from Slashdot

2008-11-29T13:44:00.000-08:00

There was a recent story on slashdot about useless (or useful) things one can do in UNIX. Being a command line junkie, I read through virtually every comment (all 2300+ of them) to learn some new tricks. Here are some of the better ones:

Bash History

history -c # clear history (good for preserving privacy/passwords, or check out the more precise -d option)
In vi command mode, type /query and hit Enter to search history, n to keep searching backwards, N to search forwards

vimdiff

vimdiff original_file patched_file
unified format: open original file, then :vertical diffpatch path/to/diff

Encryption

openssl aes-256-cbc -a -e -salt -in INPUT_FILENAME -out OUTPUT_FILENAME # encrypt
openssl aes-256-cbc -a -d -salt -in INPUT_FILENAME -out OUTPUT_FILENAME # decrypt
echo Oe lbh pna vzcyrzrag UK tbireazrag fgnaqneq rapelcgvba jvgu ge | tr a-z n-za-m # Rot 13 encrypt/decrypt

Others:

sleep 8h; cat /dev/urandom > /dev/dsp # alarm clock
eject -T # close cd tray if open, open if closed (useful to find out which physical machine you are logged into)
sl # punish users who accidentally type 'sl' instead of 'ls'
eposd && say 'hello' # make the computer talk
:(){ :|:& };: # forkbomb (space required between { and :) (protect against this with ulimit -u)
for I in $(seq 1 100) ; do echo $I; sleep .25; done | dialog --gauge "PIZZA" 6 50 100 # Pizza timer via dialog

New MTV Video Site's Censorship -- Bleeping out Names of File-Sharing Software

2008-10-29T10:15:00.000-07:00

MTV has just launched a new online video site. While I won't go through the trouble of mocking the site (or the TV network) for its irrelevance (please visit this slashdot thread for plenty of that), it is worth noting that it does have one feature that other video sites do not: in-video censorship! As this observant slashdot commenter says:

I was perusing this yesterday, and came across the Weird Al video "Don't Download This Song". One line in the original song goes:
o/~ Like Morpheus or Grokster or Limewire or KaZaA o/~
But the version on the new MTV site goes:
o/~ Like *beep* or *beep* or *beep* or *beep* o/~
Does anyone know if it was aired on MTV/VH1 this way, or is this unique to the web version?
MTV: http://www.mtvmusic.com/video/?id=108884 [mtvmusic.com]
Youtube: http://www.youtube.com/watch?v=Yz-grdpKVqg [youtube.com]

Update
More coverage: Slashdot Techdirt

Interesting New Perspective on DRM

2008-10-23T17:21:00.000-07:00

While browsing a Slashdot thread about a purported "open-source DRM" product I came across an interesting comment by a guy named Sancho. While I had always viewed DRM as a uniquely new development, he ties it to practices that have been occurring in the recording industry for some time:

I tend to think of it as ensuring repeated sales of their art throughout their lifetimes.
For a while there, ensuring this was as easy as making sure that your music was released on the format du jour. Records, 8-tracks, cassettes, CDs.... With the advent of digital music sans a physical medium, this trend of rebuying all of your albums is at risk. Suddenly, you're faced with customers never having to rebuy the White album, and you see your sustained profits going down the tubes.
DRM solves that. Now, rather than coming out with a new format every few years, you just have to come up with a new DRM scheme and turn off the old servers. Because the devices playing the music are somewhat general purpose, it's easy to move quickly--you don't have to worry about market penetration for the players, because it's just a free software update away.

One small point: in the old days, format upgrades, say from tape to CD, often brought with them added benefits (better sound quality, more convenient access to songs, larger storage space, etc.) so there were actually justifiable reasons to upgrade. Now, switching from one DRM-encumbered format to the next offers no such incentives for the consumer.

wmctrl and friends

2008-10-10T21:33:00.000-07:00

wmctrl seems like an awesome utility. I first read about it in Kyle Rankin's Linux Journal column here. The wmctrl project page also has links to a bunch of other desktop-automation and related utilities. This is all going in the "to learn when I have some spare time" file along with screen.

Critique of Zittrain's "The Future of the Internet and How to Stop It"

2008-09-08T18:43:00.000-07:00

One book that the technorati have been talking about recently (ok, not so recently... it took me a while to write this article) is Jonathan Zittrain's The Future of the Internet and How to Stop It. For a book written by a co-founder of the Berkman Center and someone who is a remarkably good speaker, I found the work to be disappointing. The book's argument is not convincing and the writing seems to lack discipline, often wandering from one loosely related subject to another.

Zittrain's main point is that the security failings of generative technologies will push consumers to buy more restrictive, and supposedly safer, devices. This claim has a number of problems with it. The first is that tethered devices are not safer or more secure than generative ones -- in fact, normally the opposite is true. Compare the number of vulnerabilities in the Windows operating systems vs the number in Linux or BSD operating systems. Or bugs in Internet Explorer vs bugs in Firefox. This claim is even more dubious the more control the manufacturer has over the device: Richard Stallman points out in his response to Zittran that the iPhone's remote kill-switch makes the iPhone "designed for remote attack by Apple."

The second problem with Zittrain's principal claim is that a consumer has no incentive to prefer a non-generative device. Since non-generative devices are less secure than generative ones, any purported advantage that the non-generative device manufacturer could claim is lost. There is empirical evidence to support the belief that consumers prefer generative devices --Stallman cites the number of jailbroken iPhones as an example. Roger Grimes adds in his response: "It’s hard to say that closed systems are taking a more prominent role when open examples abound. Even the 'closed' systems he mentions are becoming more open thanks to competition and customer demand."

Even if, for the sake of argument, locked-down devices were somehow more secure than generative devices, consumers wouldn't necessarily migrate to non-generative appliances because users rarely make purchasing decisions based on security. Most computers are purchased because the user is comfortable with the platform or because he thinks that the computer is pretty or because that particular computer is necessary to run some type of software. Rarely will a run-of-the-mill consumer take into account a record of operating system vulnerabilities or the pros and cons of different systems architectures when deciding between OSX and Windows.

There are other shortcomings of the book besides the weakness of the main argument. For one, Zittrain mistakes generativity as being a zero-sum game: something is either generative or it isn't. There is a continuum of generativity: for instance, Linux is more generative than Windows XP, but Windows XP is more generative than Windows Vista. It is a fallacy to simply assume that all products fall into one non-generative bucket or the other generative one.

For a book whose title suggests solutions to the problems with the Internet, Zittran's ideas underdeliver. Virtual machines, extra-legal incentives, data portability and network neutrality are all things that are familiar, and have been, to policymakers and programmers for a while. In a book such as this which only worries about theoretical overtures and not about the detailed technical implementation, more out-of-the-box, grander thinking and proposals would have been welcome.

The book has a couple of chapters that feel decidedly out of place. The final chapter regarding privacy and the chapter exploring Wikipedia both don't seem to fit in to the framework of the book. That being said, both are certainly worthy of scholarship on their own merits. I particularly found the chapter on privacy engaging, if not particularly relevant to the rest of the book.

Zittrain's book is still worth a read: it addresses areas of concern in today's Internet and references much interesting material. The end result, however, is unconvincing and disappointing -- keep a few grains of salt handy when reading.

=========================================
REFERENCES / FURTHER READING

Jonathan Zittrain
http://bostonreview.net/BR33.2/zittrain.php
"Protecting the Internet Without Wrecking It"

Richard Stallman
http://bostonreview.net/BR33.2/stallman.php
"The root of this problem is software controlled by its developer"

Bruce M Owen
http://bostonreview.net/BR33.2/owen.php
"As long as flexibility has value to users, suppliers will have incentives to offer it"

Roger A Grimes
http://bostonreview.net/BR33.2/grimes.php
"Fixing Web insecurity requires more than a caring community"

Hal Varian
http://bostonreview.net/BR33.2/varian.php
"Ultimately, the best protection is an informed buyer who demands openness"

Susan Crawford
http://bostonreview.net/BR33.2/crawford.php
"In the eyes of many exiting institutions, security isn't a problem -- it's an opportunity"

David D. Clark
http://bostonreview.net/BR33.2/clark.php
"We need to develop a socially embedded online experience"

Jonathan Zittrain
http://bostonreview.net/BR33.2/zittrainresponse.php
"The best solutions don't assume a zero-sum tradeoff between security and generativity"

Coverage on BoingBoing

Ars Technica review and interview

Watch High Quality YouTube Videos by Default

2008-08-29T15:59:00.000-07:00

Explained here. Man, I missed the boat on this one... it's been out for half a year!

Cool Gmail Feature -- Periods do not Matter

2008-08-29T15:49:00.000-07:00

It's true: john.doe@gmail.com is the same email address as johndoe@gmail.com as far as Gmail is concerned. And to think that everyone makes such a big fuss about making sure you have that period in the right place when they give out their emails...

This feature is documented in Gmail help here. Gmail even idiot-proofs this feature by having a link to the docs when you receive an email at a different address from the one you registered (see photo).

Some have used this feature to their advantage to reduce spam or create multiple accounts on a web service that all send mail to the same Gmail address.

Thoughts on Facebook and Privacy (or Lack Thereof)

2008-08-22T21:39:00.000-07:00

After watching a DEFCON 16 presentation about the vulnerabilities in social networks, I reflected further upon Facebook and the privacy it offers you and me, which is close to nil. Your guarantees to privacy on Facebook depend on a multitude of assumptions, all of which are quite poor. [B]

First, you are trusting that the Facebook developers have implemented the privacy controls correctly such that there is no inadvertent information leakage on the site as a result of bugs. I write code for a living, and let me tell you, bug-free code does not exist. Facebook, like other applications, has had its share of bugs to scramble to fix in the past (including at least one truly amateur mistake) and the future will be (and the present is) no different.

Second, you are assuming that you can configure the myriad privacy options correctly such that every piece of information on your site is accessible to only those that you want it to be. Are you really sure that marking one person as only being allowed to see your limited profile and specifying that picture as globally viewable, for example, will turn out the restrictions you desire for the correct people? How can you tell which preferences override which? It would certainly be tedious to register other accounts (or use friends') and test various combinations of privacy features against their profiles and I am not aware of anyone that does this.

Third, anyone that can see your information is capable of leaking it to the public. [A] With the addition of every friend you are increasing the chance that your pictures, contact info, videos, etc. will be posted and shared outside of the Facebook walled garden. It is simply not possible that each of your 500 friends is not susceptible to give away information that you thought was just between you and them, especially when they have some kind of (monetary or otherwise) incentive to do so. The scenarios of a rival political party digging up dirt on a candidate and gossip magazines researching what someone did last night both come to mind.

Fourth, all of your information can be accessed by any Facebook engineer or executive who choses to do so. The engineers likely need access to real-world pages to debug their code, and the managers can order information from a compliant underling (if Facebook doesn't have internal tools set up already for them to access this information). And let's not forget everyone else that works there (sales, PR, HR, etc.) who can request your personal information as a favor from an engineer friend.

Fifth, just as with any other website, information on Facebook can be subpoenaed in a trial. Facebook, needing to comply with the law, will gladly turn over your personal information to any judge who so wishes.

Sixth, let's not forget the countless ways Facebook could involuntarily compromise your information. A malicious hacker could slurp down personal data off the site. A Facebook employee could negligently leave an unencrypted disk drive with your information on it in a public place. Etc.

The only conclusion is this sound advice: don't put anything on Facebook that you don't want to be exposed to the world. Because chances are, sooner or later, it will be.

Footnotes:

[A] This is, of course, assuming that your group of Facebook friends can not be considered 'the public.' With the amount of friends some have, and especially one's willingness to accept any request that comes their way and fire out friend requests at random, this distinction begins to blur.

[B] I was going to add this post to my Facebook Sucks article but it became too long and I thought it deserved a post of its own.

Updates:

Here is a post for those that want a HOWTO for micromanaging their privacy settings on Facebook. (Even Schneier likes it).

Here is a Slashdot story about a court demanding Facebook information pursuant to a case

DEFCON 16

2008-08-19T23:34:00.000-07:00

DEFCON 16 was awesome, as expected. Some highlights:

The first presentation I went to was called "Hacking in the Name of Science." Here a bunch of University of Washington grad students and a professor discussed the sweet research they are doing, almost all of which has been in the news (Implicating 'downloading' printers to the RIAA monitors, RFID ghost proxies, TCP information leakage, voting machine vulnerabilities, TrueCrypt vulnerabilities, implantable medical device hacking, ISP injected ads, etc.). They discussed the difference between just hacking and what you need to do in an academic setting to study what anyone else would call hacking. They encouraged attending academic security conferences, such as ACM CCS, NDSS, IEEE Security + Privacy, HotSec and Woot
A talk entitled "Satan is on my Friends List" detailed the security vulnerabilities in OpenSocial-enabled websites. These guys demonstrated some hilarious things, including using a CSRF DOS attack: using an img tag placed in an html-enabled form that displays on a page, you can automatically logout anyone that sees that img by pointing the img's src attribute to the logout page. The speakers talked about how the socnet widget applications space is essentially a security free-for-all: apps hacking personal information, apps hacking other apps, etc. An opt-in security model for javascript safety in apps exacerbates the problem. An amusing conclusion to the talk was the speakers' impersonation of another security researcher on social networks which fooled his colleagues and family alike.
Locksport enthusiast Eric Schmedl gave a talk that had some amusing anecdotes about cloak-and-dagger spying. Mary Lou McFate (NRA infiltrator of anti-gun groups), reconstructing passwords from audio of keystrokes, and multiple phone bugging technologies were discussed.
Fyodor gave a talk on nmap, the tool he created and how he used it to scan a large subset of the Internet. He also presented some new features of the tool, including traceroute, ping, and netcat-like functionality... what can't it do?
I briefly stopped in on a talk called "Taking Back Your Cellphone" which plugged the site HowardForums as an excellent resource for phone modification.
The activity that I took part in for a fair share of my time there was the Lockpicking Village. I bought a set of lockpicks, and tried my skills on a variety of locks lying about the room. I also listened to talks on how to crack certain types of locks, including masterlocks (use coke can shiv, patterns for figuring out combo).
Probably the most interesting thing that happened at DEFCON nobody got to see: a judge ordered a group of MIT students not to talk about hacking the Boston Subway system. This was rather pointless because 1) the presentation was distributed on CD before the gag was ordered 2) the ban was lifted after the conference 3) MIT's student newspaper put the presentation up on its site
Other cool things: the badge, the mystery box
Didn't see these presentations, but I looked at them on the CD:

"The Death of Cash" features a preview of a world without cash. People are turning to credit because it is more convenient, banks love it because of better profit margins, government loves it because it makes you easier to track. (Note: Illegal to transfer $10,000 in/out of the country without declaring it). This is getting worse with stupid legislation (Patriot Act). Also, national security risk: electronic outages now mean that people can't get access to cash (even more troublesome as electric grid becomes less reliable). Strong crypto might be the basis of a future E-payment system. Advice: keep some cash on hand for emergencies, use non-cash as little as possible. thowlett@netsecuritysvcs.com says the presentation can be downloaded at www.netsecuritysvcs.com/presentations/defcon16/ but I don't see it there...
An introduction to ham radio called "Ham for Hackers"
A presentation on Javascript obfuscation that goes over the following methods: ASCII/Unicode escapes, XOR (ASCII/encoding), string splitting, simple encryption, non-obvious variable and function names, member enumeration, whitespace encoding/decoding
Another presentation on SCADA systems that made me have nightmares
A HOWTO on SSL cookie hijacking by Tor developer Mike Perry: insert an img tag with src mail.yahoo.com into an unencrypted connection and read their cookie, then save that cookie to cookies.txt and read their email (over SSL, if you want!)
OCR tools: tesseract, jocr, ocrad
A presentation similar to "Satan is on my Friends List" for Google Gadgets

Update:
Good photos of the event can be found here

Proposal: Free Parking through Self-Booting

2008-08-18T20:51:00.000-07:00

San Francisco is a tough city to park in, so I've been scheming up some ways in which to get free parking... here's one thing I've come up with, but haven't had the guts to try yet.

Buy four boots (or more formally, wheel immobilizers).
Park in a tow-away parking zone.
Affix one boot to each wheel of your car.
Laugh at the people figuring out how to tow your car away seeing that they can not drag your car along on its back wheels nor wheel the car up onto a platform.
Return to your car at your leisure, detach boots, drive away.

Some things to consider:

Boots are somewhat expensive, so the whole cost-efficiency of this scheme needs to be taken into consideration (price of tickets, if you plan to pay them, price of parking, risk of having car destroyed by angry tow truckers).
I'm not sure what kinds of contracts tow truckers have with the city, private establishments, etc. If they are not liable for any harm that comes to your car, this probably isn't such a good idea, as the towers have no incentive to not damage your car trying to toe it away.
Not paying tickets can bite you in the ass if the ticket-giver somehow manages to figure out your Vehicle Identification Number, usually found etched in various places in the car. Unpaid tickets can come up when you try and re-register your vehicle.
Boots can be pried off... I'm sure this isn't the only way

UPDATE: Nothing to do with booting, but here's another way to get free parking in San Francisco, courtesy of Black Hat

HOWTO vertical display (X, Nvidia)

2008-08-11T20:02:00.000-07:00

Here is how to get your monitor looking something like this (not my monitor, btw):

Put this line as an option in the "Device" section of /etc/X11/xorg.conf: Option "Rotate" "CCW" Then, restart X with Ctrl+Alt+Backspace. Afterwards, start up sudo nvidia-settings and shuffle the screens around in X Server Display Configuration so that they are properly aligned. Be sure to press the Save to X Configuration File button.

Found via this forum thread and a tip from Arash.

Update:
Following is an xorg.conf file to get 2 Dell 2407 monitors in a T formation (with one on the left vertical and one on the right horizontal aka Rajiv-style). This setup has a few problems, though...
1) The screens are flush at the top, which makes a window across both monitors look disjointed... not a huge problem, and could probably be solved by tweaking the config
2) Can't seem to get compiz-fusion to work
3) Using Xinerama instead of Twinview is waaaaay slower, especially when an object is on both screens
4) Just feels too unnatural... I'm going back to both screens horizontal
It's also a bit hard finding helpful forum posts on these things anyways because everyone seems to misuse the terminlogy (Xinerama, Twinview, etc.) and problems are very individual (Depending on video drivers, screens, etc.). Anyways, here's the xorg.conf:


Section "ServerLayout"
   Identifier     "Default Layout"
   Screen      0  "Screen0" RightOf "Screen1"
   Screen      1  "Screen1" 0 0
EndSection

Section "Module"
   Load           "glx"
EndSection

Section "ServerFlags"
   Option         "Xinerama" "1"
EndSection

Section "InputDevice"
   Identifier     "Generic Keyboard"
   Driver         "kbd"
   Option         "XkbRules" "xorg"
   Option         "XkbModel" "pc105"
   Option         "XkbLayout" "us"
EndSection

Section "InputDevice"
   Identifier     "Configured Mouse"
   Driver         "mouse"
   Option         "CorePointer"
EndSection

Section "Monitor"
   Identifier     "Configured Monitor"
EndSection

Section "Monitor"
   Identifier     "Monitor0"
   VendorName     "Unknown"
   ModelName      "DELL 2407WFP"
   HorizSync       30.0 - 83.0
   VertRefresh     56.0 - 76.0
EndSection

Section "Monitor"
   Identifier     "Monitor1"
   VendorName     "Unknown"
   ModelName      "DELL 2407WFP"
   HorizSync       30.0 - 83.0
   VertRefresh     56.0 - 76.0
EndSection

Section "Device"
   Identifier     "Videocard0"
   Driver         "nvidia"
   VendorName     "NVIDIA Corporation"
   BoardName      "GeForce 7900 GS"
   Option         "Rotate" "CCW"
   Option         "NoLogo" "True"
   BusID          "PCI:1:0:0"
   Screen          0
EndSection

Section "Device"
   Identifier     "Videocard1"
   Driver         "nvidia"
   VendorName     "NVIDIA Corporation"
   BoardName      "GeForce 7900 GS"
   BusID          "PCI:1:0:0"
   Screen          1
EndSection

Section "Screen"
   Identifier     "Screen0"
   Device         "Videocard1"
   Monitor        "Monitor0"
   DefaultDepth    24
   Option         "TwinView" "0"
   Option         "metamodes" "DFP-0: nvidia-auto-select +0+0"
EndSection

Section "Screen"
   Identifier     "Screen1"
   Device         "Videocard0"
   Monitor        "Monitor1"
   DefaultDepth    24
   Option         "TwinView" "0"
   Option         "metamodes" "DFP-1: nvidia-auto-select +0+0"
EndSection

Keep SSH connections from timing out, client-side

2008-07-21T23:18:00.000-07:00

Put these lines in ~/.ssh/config

ServerAliveInterval 300
TCPKeepAlive yes

txt2regex

2008-07-06T17:38:00.000-07:00

txt2regex is an awesome tool that allows you to specify a regular expression in an interactive curses-based interface that is then printed out in a variety of formats, including English, vim, perl, etc. Perfect for when you know exactly what you want to match, but are not particularly knowledgeable about the idiosyncrasies of the current regex engine you're using.

Stop Spamming me with Corporate Birthday Cards

2008-07-04T08:12:00.000-07:00

It is the time of year when my birthday is fast approaching and that means that I am receiving a few birthday cards, sometimes from parties from which I would not expect one. Some of these parties are not people, but corporations who happen to have my birth date on file for one reason or another.

In any other context, snail mail sent from a corporation to one's home mailbox for marketing purposes would be called junk mail or spam. Sunglasses Hut, et al. try and disguise the purpose of the letter (making you want to buy more of their products) by making the implication that they're corresponding out of consideration for you by politely remembering the celebration of your birth. Now, some may think that this is a genuine gesture of kindness, but I disagree. The entire point of a card is to remind someone that you are thinking about him or her and his/her particular situation. I find nothing personal or redeeming about being the recipient of a letter sent by a cron job at HQ once a year which was ordered by a marketing director that neither knows me nor has any reason to care about my circumstance save trying to extract more of my hard-earned cash out of my pocket.

In fairness I must note that, unlike most every other sender of birthday spam, Sunglasses Hut at least sent me a twenty-dollar gift certificate to their store. Considering that I don't need another pair of sunglasses and would hesitate to buy from SH again given this annoying practice, I find it small recompense for wasting my time and mailbox space with their spam.

I guess this can just be chalked up to another consequence of an age when more private parties are holding our personal data and are ever eager to exploit it for their own ends. It seems as there is no disincentive for corporations to send all the spam they want, I'll be receiving quite a few more cards from well-wishing chief marketing officers in the future.


bigshotcmo@sunglasseshut27:~$ crontab -l
# m h  dom mon dow   command
  0 0  1   *   *     ./send_birthday_cards_for_month

...

db.execute("SELECT * FROM customer WHERE MONTH(birthday) = MONTH(NOW())).each do |customer|
    Notifier.send_individual_card(customer)
end

...

Dear $customer.first_name,

Happy birthday!  You are super special to us!  Spend more money at our store!  
Thanks for providing us with your birthday!  Thanks to our expert DBAs and 
comprehensive database backup policy, we're able to retrieve your personal 
information on demand and sure to never expunge your data from our records.  
Have a nice day!

Love,

Moneybags
CMO

HOWTO Swap caps lock and escape keys in Linux

2008-06-17T10:15:00.000-07:00

Caps lock takes up way too much space on a keyboard for how little (never) one presses it. Esc is a very useful key that gets pressed a lot (especially for vi users) but is rather far away and takes more effort to press in the corner of the keyboard. Here is how to switch the functionality of the two keys (solution adapted from the xmodmap manpage) in Linux.

Create a file called .Xmodmap in your home directory. Put the following text in the file:

!
! Swap Caps_Lock and Escape
!
remove Lock = Caps_Lock
remove Control = Escape
keysym Escape = Caps_Lock
keysym Caps_Lock = Escape
add Lock = Caps_Lock
add Control = Escape

Next time you log in, these settings should stick. If you want to switch the keys manually instead of on startup, run xmodmap .Xmodmap to do so. Loading the file again after you have loaded it once will switch the keys back to their original functionalities.

Btw, an informative discussion of all of this X customization stuff is found in chapter 6 of Unix Power Tools.

Google Experimental Search

2008-04-29T18:06:00.000-07:00

Today I stumbled across Google's experimental page which has some add-ons to search that can be enabled. "Keyboard shortcuts" kicks ass: it's basically a vi-like interface to search. j to go down, k to go up, / to search, etc. It's really quite slick.

Although Google says that you can only add one of these experimental features at a time, "Keyword suggestions" is an option that can be enabled via the Customize Google Firefox extension. "Keyword suggestions" is a great feature as well.

The "Alternate views" options , on the other hand, are a heaping waste of time. Even the search results for the example queries listed on the page are unintuitive and uninformative.

Google also offers keyboard shortcuts for a lot of other products (Gmail, Calendar, Docs, Reader, Maps, Video), detailed here.

UPDATE: I actually disabled the experimental features. The vi emulation was not helpful enough... I could scan the page visually faster than my fingers could keep up. Furthermore, I missed the utility of Firefox's "/" to search the text on the page fast on a Google search result page.

Searching SVN trees with Ack (for Ruby on Ubuntu)

2008-04-04T23:50:00.000-07:00

Searching through a subversion tree with grep is a pain in the ass. It wastes time searching through files I don't want to look into and its syntax is inconvenient for the task. Fortunately, there is a tool called ack which is suited specifically for matching text in VCS trees (including svn trees). Here's how to set it up on Ubuntu Gutsy (apparently you won't need to do all this in the next Ubuntu release; there's a package for Hardy).

First, you're going to need some relevant libraries. sudo apt-get install perl-doc to start. Next, download the File-Next archive from CPAN. Extract the archive and cd into the directory, then use the series of commands perl Makefile.PL && make && make test && sudo make install to install it.

Now for ack itself. svn checkout http://ack.googlecode.com/svn/trunk/ ack-read-only to checkout the source tree from Google Code and cd into the directory. Now if you want to edit some of the types that are built into the program, here's where to do it. For example, I want to also search for haml files with the --ruby flag. I grep -R rhtml * to see that four files (actually, less, but let's go ahead and edit all of them anyways...) require modification. The ruby line in Ack.pm, for instance, now reads ruby => [qw( rb rhtml rjs rxml erb haml )]. (Types can also be set on the command line using the --type-add and --type-set flags). Once you're finished tweaking types, use the same perl Makefile.PL && make && make test && sudo make install command to install ack.

For extra credit, you can also add flags to a ~/.ackrc file that get executed upon each invocation of ack. Personally, I add the --ruby flag because it seems to be that .rb, .haml and .rhtml files are all that I'm searching for these days.

And voila! Now a simple ack -f gives me a listing of all the ruby files in my svn tree (and no vi swp files, svn files, log files, etc.). ack "regex" returns a grep-like search of the relevant parts of the subversion tree with the parts of the line that matched the regex highlighted. SWEET! Get more information on the command line with ack --help for more help, ack --help type for a listing of matching types.

Do you know how hard this would have been with find/grep/xargs? Well, tough. grep -RE "regex" * | grep -v svn | grep -v log | ...etc... searches all files in the tree and might exclude matching lines if the contain one of the strings being filtered out by the later grep instances. That syntax can be improved to a alternating regular expression with grep -RE "regex" * | grep -v -E $log\|svn$ but it still has the same problems. grep also has an --exclude flag which can exclude certain filenames that match a glob from being searched (although the manpage would seem to imply otherwise), but it doesn't have regex capabilities. So maybe we can use find and xargs? Nope, a svn tree output will kill xargs, which will compain that the "argument line is too long" if too much gets sent to it: grep -E "regex" `find . -regex '.*\.$rhtml\|rb\|haml$' | xargs -0` Perhaps find's --exec flag could be of use? Well, only if you want to kill your computer with the overhead of creating a new grep process and forking and context switching a couple times (etc.) upon each file match... find . -regex "regex" -exec grep -E "regex" '{}' \; will really do your cpu in. Then there's the cumbersome grep -E "regex" $(find . -regex "regex" | grep -vE $log\|svn$) ... well, let's just say I'm glad I found ack at this point.

UPDATE:
Put --type-add=ruby=.haml in your .ackrc to add haml as a type dynamically.

Amazon's Music Store on Ubuntu Gutsy

2008-03-30T21:16:00.000-07:00

Unfortunately, there are some CDs that are so obscure nobody is seeding them on BitTorrent. One of these happens to be Hicksville by Celtic Cross. I decided to give the Amazon Music Store, where this CD is available for purchase, a shot.

On Ubuntu Gutsy, there are a few things you need to do to make this work. The first, of course, is to download the .deb package from this page. Install that with sudo dpkg -i amazonmp3.deb or sudo dpkg --force-architecture -i amazonmp3.deb if on a 64 bit system. You're also going to need a few Boost libraries: sudo apt-get install libboost-date-time1.34.1 libboost-thread1.34.1 libboost-iostreams1.34.1 libboost-signals1.34.1 If you're on a 32 bit system, launching the /usr/bin/amazonmp3 binary should now work (it also inserts an entry in the GNOME panel under Applications->Internet).

But for all you 64-bit'ers out there, there's a bit more work to do. Launching the binary fails with the message "amazonmp3: error while loading shared libraries: libgtkmm-2.4.so.1: cannot open shared object file: No such file or directory". This is because amazonmp3 needs 32 bit libraries, not 64 bit ones (see this thread). Download getlibs, a bash script which installs 32 bit versions of libraries on 64 bit systems. Install the package with sudo dpkg -i getlibs-all.deb and then run getlibs /usr/bin/amazonmp3 to install the 32 bit libraries. The program should now run without error.

The application, by the way, is quite minimalistic. I don't know why they put users through this trouble and don't just send you a tarball of the music files. I guess there might be some reason on other platforms where it integrates with other applications... not on Linux, though!

Why NIN's "Ghosts" Release Matters

2008-03-12T20:45:00.000-07:00

Nine Inch Nails ushered in another landmark event in the slow march towards freedom of digital media recently when it released its newest work for free (gratis and libre) on its web site. NIN band member Trent Reznor has frequently been at the forefront of innovation in music and the surrounding areas and it is not at all surprising and very pleasing that it is him to be the first high-profile artist (to my knowledge) to release a major work under a Creative Commons license. However, not everyone seems to grasp the significance of what has happened here. In fact, not even the article that the Slashdot article links to reports the fact that the album has been released under a CC license and many other news outlets have neglected this important point as well. In this article, I'll give a summary of the events surrounding NIN's release of Ghosts I-IV and an analysis of its significance.

First, some history of the band: Nine Inch Nails has never been a conformist where promoting its music has been concerned. The history of their original ideas is impressive: breaking off from their label and releasing their work independently; encouraging fans at concerts to "steal" music; hiding USB drives for fans to find with new, unreleased songs on them; performing surprise concerts as part of an elaborate fantasy world constructed for fans and releasing Garageband samples for others to remix.

There is precedent for releasing music free (both gratis and libre) online. Most artists now have a myspace page or other limited Internet presence where one can listen to, but usually not download (or only download a small piece of) an album. Many low-profile artists have been releasing their music online for years under various licenses. Radiohead, arguably, has been the most famous band to release an album free of charge online (with In Rainbows last year). Radiohead allowed each customer to specify their own price for a full download of the album off of its site, even $0. It is estimated that Radiohead made many millions of dollars with this tactic.

Next, the facts about the Ghosts release. From the Nine Inch Nails website there are a variety of options for downloading the album, etc:

Free: The first 9 tracks from the Ghosts I-IV collection available as high-quality, DRM-free MP3s, including the complete PDF.
$5: All 36 tracks in a variety of digital formats including a 40 page PDF.
$10: Ghosts I-IV on two audio CDs in a six panel digipak package with a 16 page booklet. Pre-order, to be shipped April 8, 2008. INCLUDES immediate full download in a variety of digital formats.
$75: Ghosts I-IV in a hardcover fabric slipcase containing: 2 audio CDs, 1 data DVD with all 36 tracks in multi-track format, and a Blu-ray disc with Ghosts I-IV in high-definition 96/24 stereo and accompanying slideshow.
$300:

Nine Inch Nails also uploaded a torrent to the Pirate Bay of the first 9 tracks of the album (out of 36 total) with the following message:

Hello from Nine Inch Nails.

We\\\'re very proud to present a new collection of instrumental music, Ghosts I-IV. Almost two hours of music recorded over an intense ten week period last fall, Ghosts I-IV sprawls Nine Inch Nails across a variety of new terrain.

Now that we\\\'re no longer constrained by a record label, we\\\'ve decided to personally upload Ghosts I, the first of the four volumes, to various torrent sites, because we believe BitTorrent is a revolutionary digital distribution method, and we believe in finding ways to utilize new technologies instead of fighting them.

We encourage you to share the music of Ghosts I with your friends, post it on your website, play it on your podcast, use it for video projects, etc. It\\\'s licensed for all non-commercial use under Creative Commons.

We\\\'ve also made a 40 page PDF book to accompany the album. If you\\\'d like to download it for free, visit http://ghosts.nin.com/main/pdf

Ghosts I is the first part of the 36 track collection Ghosts I-IV. Undoubtedly you\\\'ll be able to find the complete collection on the same torrent network you found this file, but if you\\\'re interested in the release, we encourage you to check it out at ghosts.nin.com, where the complete Ghosts I-IV is available directly from us in a variety of DRM-free digital formats, including FLAC lossless, for only $5. You can also order it on CD, or as a deluxe package with multitrack audio files, high definition audio on Blu-ray disc, and a large hard-bound book.

We genuinely appreciate your support, and hope you enjoy the new music. Thanks for listening.

http://ghosts.nin.com

A little while after the release of the album and the spreading of the news, the NIN website got overloaded with requests, prompting Trent Reznor to post the following message there:

Update: The response to this album has been overwhelming, causing our website to slow to a crawl. We THOUGHT we were ready, but...
We've been adding more servers to accommodate the unexpected demand and we expect to be running smoothly in the next few hours. In the meantime, if you've had any problems with downloads from the Ghosts site, don't worry - you'll be able to use your download link again when the site is more stable. Thanks everyone for making this such an immediate success.

posted by Trent Reznor at 5:47 PM pst, from hong kong.

So why is the release of Ghosts significant? Because it's the most visible release of art under a Creative Commons license to date (by-nc-sa 3.0, to be exact). This means that Trent Reznor can not take you to court for distributing the music on, say, the Pirate Bay; it explicitly permits this kind of activity. All the license mandates is that if you use the work, you must credit the original author and you must release your new work that incorporates NIN's art under the same license. The license also prohibits any use of the work for financial gain by a third party (no selling the album on the street for five bucks a pop). This is in marked contrast to Radiohead, who did not specify the license that In Rainbows was under, so because of this ambiguity we must only assume that it is under a traditional copyright license. This means that Radiohead CAN take you to court for distributing the music if one day it decides that suing its fans is fashionable. Releasing art, especially music, in this fashion is going to become increasingly prevalent as more artists realize the benefits of instantly sharing art with the world via the Internet in preference to traditional distribution channels such as record labels. With computer technology, how fans use music is increasingly out of an artists' control. It is a much more amicable arrangement for the artist to indulge fans by releasing music in a way they find convenient (and, also, not sue them).

The release of Ghosts also establishes BitTorrent as a recognized distribution channel in which established artists can release their music. Frequently, the BitTorrent protocol and its users are denigrated in the media as "pirates" or "thieves." The fact that a respected artist is using BitTorrent in preference to direct download or a record label-sponsored distribution will legitimize this method of sharing in the eyes of many and heighten the profile of BitTorrent generally. Using BitTorrent was also a good move simply because it is a superior protocol for large files, as the slowness of the NIN website showed. Also, it is more convenient to use than having to click through the kind of omg-it's-free messages (or, more generally, an unfamiliar interface) that Radiohead made users weather to buy In Rainbows. Being so in touch with how music fans want to acquire their music definitely doesn't hurt NIN's publicity, either.

Reznor also proves that it is wholly possible to make a lot of money off of a free product on the Internet, something that many ill-informed pundits think impossible. Slashdot reports that the most expensive option sold out in about a day, netting Reznor $750,000. This is to say nothing of any of the other purchasing options, which also likely brought in a significant amount of revenue, possibly much more than the $300 dollar option. So why are people paying for something that they could more or less otherwise get for free? A few answers can be found in Kevin Kelly's essay Better Than Free. Kelly claims that the Internet has made copyable material infinitely abundant, and therefore impossible to charge money for. What can be charged for are what Kelly calls "generatives" -- things that can't be copied. He suggests 8 generatives from which money can be made in a world with Internet technology: immediacy, personalization, interpretation, authenticity, accessibility, embodiment, patronage and findability (convinience). By autographing records, NIN provides authenticity and personalization. By offering downloads off its site in a variety of digital formats, NIN makes money off of findability, accessibility and immediacy. And, of course, being an artist offering up works for free under a CC license NIN reaps the benefit of patronage: listeners loving what they are seeing and hearing and willing to pay for more of the same.

Another important point about this event is that all the data on NIN's site will eventually find its way out from behind the wall of authentication and payment and onto public networks. That is, users that paid for enhanced versions of the product will upload them to a BitTorrent tracker: here already are the ($5) FLAC versions of all 36 songs and some extras. It is often repeated that information wants to be free, as we can see here. What is not often repeated, but equally true, is that information wants to be convenient. Fans of In Rainbows proved this yet again when they uploaded the album to BitTorrent (humorously leading some ignorant columnists to castigate users for "stealing" it -- how can you steal something if it's free and able to be infinitely copied?).

In conclusion, the experimental release of Ghosts on the Internet under a Creative Commons license and for free is a milestone event for a variety of reasons. It is the first high-profile musical release that is free, both as in freedom and as in beer. It made NIN a whole lot of money, proving that artists can survive in the Internet era (even without performing). And it enhances the profile of the BitTorrent protocol as a distribution mechanism and reinforces some well-understood properties of Internet-based information distribution. Hopefully other artists will follow NIN's example and release their work in a similar manner.

===============================
Further reading:

Reznor decries fact that people see Radiohead as first to release free online, says industry won't follow his lead anytime soon because it's inept. (Link)

Ruby (vs Python): a preliminary assessment

2008-02-21T22:46:00.000-08:00

I've been using Ruby for a little over a month now. While this hasn't been enough time to explore the depths of the language, it has given me ample time to evaluate how I feel about some of the more superficial aspects of Ruby. There are some things I like about it, a few things I don't care about, and a lot of things I don't like. In some aspects it is quite similar to Python, my favorite programming language. In the following sections I lay out what I do, kinda, and don't like about Ruby, sometimes in comparison to or in contrast with Python to put things in context.

Things I like about Ruby

Naming conventions are enforced The less surprises there are in code, the easier it is to read. I would be very surprised, for example, if a C VARIABLE_IN_ALL_CAPS were not a constant. It only takes one joker working on code to completely screw with convention and degrade readability of code for everyone else working on it. Ruby heads off this problem at the pass. Instance variables look like @this, class variables look like @@this, constants look like THIS, global variables look like $this, etc. This is a Good Idea. I can easily live without the flexibility to name my class h0We\/ER_I_want. This also makes the code easier to parse and syntax highlight.

Block Comments I'm not sure how Python missed the boat on this one. The "large chunks of code shouldn't be commented anyway, so take them out" and "get a decent text editor that can :50,70s/^/#/" and "that's what doc comments are for" school of thought just doesn't resonate with me. Too often there is some large chunk of code lying around that just isn't applicable anymore and quickly commenting it out is a temporary solution.

Constants There is just something about the assurance that certain variables are read-only that makes me sleep well at night. Otherwise, I need to worry about things getting overwritten, and that's no fun. I guess there's always CONVENTION to prevent this in Python, but that doesn't stop someone from breaking CONVENTION += 1

Examples are the norm in documentation Examples are the quickest way to learn about new code. However, the docs could benefit from some non-example elaboration most of the time. How is one supposed to figure out corner cases? I guess "examples first" goes along with the whole Ruby-has-no-formal-spec thing.

'case' syntax The syntax of Ruby's case statement (analogous to switch in other languages) is pretty rad. It's a great feature for when multiple properties of one variable need to be tested. The case statement accepts comparisons, ranges, regular expressions, etc. Very cool.

Things I could go either way on

Regular expressions built into the language Regular expressions are cool, and I use them a lot, but I just don't know if they have a place amongst the pantheon of hashes, lists and the like. They are convenient, but overuse might turn your code into *gasp* Perl. No one likes unreadable ASCII-splatter, although it's tough for Ruby to get that bad. Plus, half the time I am using regexes some other function might do just as well, like rindex. Maybe that's just my laziness that Ruby caters to.

Interned strings It's like a pointer, I get it. It saves memory over using a string object, cool. Was it really worth including in the language? Not sure. It seems like their most common uses, such as keys in hashes, could be implemented under the hood anyway (well, at least in a real language like Python where hash keys are immutable).

! and ? as legal characters in method names I can understand that does_this_work? is a natural way to read that a function returns a boolean. I get that destroy! suggests an internal change of state of the object. But then there's the problems. First, not every method that returns a boolean or changes an object's state is forced to use these constructs. If they were enforced like all-caps constants and @ before instance members, maybe I'd be more bullish on them. Second, what qualifies as a change of state can get kind of messy: maybe a method only conditionally changes its object's state; is an exclamation mark appropriate then? The exclamation mark can also be misread, especially if parentheses are omitted (looks an awful lot like a logical not operator, doesn't it?). And now for my nitpicky objections: ? and ! are harder to type than letters and vim doesn't auto-complete the full method name by default (it completes everything but the last character) if one of those characters is included.

Marriage to Rails On the one hand, it's nice to have a large support community for tools that one is likely to use. On the other hand, it's annoying to wade through a bunch of comments by people on forums and mailing lists that are completely irrelevant to what you're trying to do with the language. These people are normally exclusively concerned with getting their website up and running, aren't terribly gifted programmers and don't really care about the language per se. Some bloggers assume that any talk about Ruby is in the context of Rails because... well, isn't that what you're using it for? What else could you possibly do with Ruby?

Indentation Python has a unique indentation system. In most cases (Makefiles) whitespace indentation is a horrible idea, but Python manages to make it work. It reduces noise on the screen and is pretty intuitive. I'm not convinced that Python wouldn't be more effective, however, with a traditional bracket-defined indentation scheme which Ruby is close to. It's a toss-up for me.

Things I don't like about Ruby

TMTOWTDI There's more than one way to do it in Ruby, alright -- you're smothered with options. This is the dark path that leads to Perl. Take looping as an example. In Python, you have for and while. That's it. In Ruby, you have for, while, times, upto, downto, begin/end and each (there's probably more that I missed). What's wrong with for? It's really difficult to be intimately familiar with this many options. And this symptom doesn't just plague looping, it also plagues the standard library. Do we really need every one of <<, push and + to append to the end of an array? Why have both brackets and "end" to enclose a block? Has anyone ever used all three versions of defining a class method in any mood but malice and spite? I guess we can all take solace in the many ways in Ruby via active_support to compute the amount of bytes in a gigabyte: 1.gigabyte, 1024.megabytes, 1048576.kilobytes, etc.

Go with the Flow There is a prevailing attitude with Ruby users, especially in Rails, to favor convention over configuration. This means that tools that espouse this philosophy reward going with the flow and punish breaking out of the mold and customizing your applications. My attitude towards this philosophy is this: unless you are creating the most pedestrian and generic of applications, you are going to need to customize the application at some point. When that happens, it is much better to have a tool that allows flexibility than one that disdains it. Otherwise you are going to need to go down to the core of the tool to make minor changes to your application, and that will cost you in time and sanity.

Hashes are broken In Python, it's impossible to change the keys in a dictionary because those keys must be immutable when they go in. Python also nicely throws a KeyError when something you're looking for in a dictionary is not there. Ruby does neither of these things; the programmer is expected to maintain the immutability of hash keys. High-level languages are supposed to take care of important details like these for us humans who are busy thinking about other things. In fact, the Python FAQ lists this as an "unacceptable solution" for an alternative implementation of hashes: "Allow lists as keys but tell the user not to modify them. This would allow a class of hard-to-track bugs in programs when you forgot or modified a list by accident." Default return values aren't that much of a design travesty, but I still prefer the error. Besides, if you really want a default return value in python, you can just use setdefault when accessing the element that may or may not be there instead of setting a default value for the whole hash at creation.

Ranges are broken This absolutely baffles me. Why did Ruby not choose to incorporate Python's elegant slicing syntax and instead chose a less powerful and more confusing one? Why did Ruby choose to create an unnecessary class called Range when they just could have had a function called, oh I don't know, range that is much more powerful, simple and general and that returns lists/arrays? Why do you need to call range.to_a to convert your useless Range object to an array in Ruby when they could have just used a function like range and gotten to the point? Furthermore, why are there two ways to create ranges, .. (two periods) and ... (three periods) that look EXACTLY THE SAME! Argh.

Optional omission of syntax elements Sure, it's less typing in Ruby, but remember what I said earlier about less surprises equaling more readable code? It holds here as well. Is this_thing a variable or a function call? This also excludes the convenient syntax for introspection that calling function() without parentheses in Python affords. And the fact that sometimes you need the "do" keyword and sometimes you don't is also a design decision that just begs for unnecessary tripping over the syntax.

Inline control flow constructs When I first saw these, I thought "Wow! I can now do in one line what previously took me three in Ruby!" And then I started reading a lot of lines and wondering why they weren't executing. And then I read those lines a bit further and saw that they had conditional statements tacked on to the end of them. And then I vowed to never use them again. One example of how this can go bad is tacking on "and return" to the end of a long statement. If you're looking at the code, and not scrolling right far enough (and your lines are that long that you need to scroll, which may or may not be a good idea), it looks like the code falls through after every statement instead of returning. Confusing.

Different names for the same method find_all? What the hell does that do? Oh, it's the same as select. Collect? What's that? Oh, it's the same as map. Too bad I only found these things out after going to the documentation. Why can't one function just have one name, so we can all have a common point of reference? Oh, right, because Ruby "allows you to program in your natural language." Which means confusing everybody else, including myself.

Different precedence for seemingly identical operators ! binds more tightly than not. && and || bind more tightly than "and" and "or". This is incredibly confusing; why not just make them the same precedence? Changing one form to the other form when it is right near another operator whose precedence is between the two could alter the control flow of your code.

Inability to override = (and non-obvious performance gotcha) There is some fine print to go along with the "every operator is just a function" propaganda. One annoying exception to the rule is the = operator. Now, why would anyone want to override the equals sign? First let me point out that it's not just =, but also -=, +=, *=, etc. that also are incapable of being overridden, so it affects a substantial set of important operators that are literally expanded to x = x + y, etc. Now here's the kicker: these methods are orders of magnitude slower than their true method counterparts for arrays and strings. So I want to override += for arrays to have it call concat or << which are both efficient as a way of aliasing all +='s in the existing codebase. In C++ and other languages, this would be quite possible (never mind that you wouldn't be facing this problem in C++ to being with) but in C Ruby, a design decision has been made that no one can override the sacred = and therefore coders will inevitably trip over this performance gotcha. Why? I'm not sure... maybe this or that post may offer some insight. In any event, don't use += and its cousins for anything but numbers or risk serious performance hits (changing one to the other is further complicated by their difference in precedence... both behave differently within a conditional operator, for example. Grrr.)

Confusing operator overloading << is a way to print things out (a la C++), define a class method, and append to the end of an array. Only the last one makes sense to me; the other two seem out of place. Remember what I said about less surprises = better code readability? It's beginning to become quite a theme...

No list comprehensions [heavy_sigh for each_time_I_can_not_do_this in Ruby if I_am_particularly_exasperated] I guess there's always select and collect. Too bad I need to decide if I want all the items in the list or only a subset before I can select the correct method to use (instead of at the end in Python, where I can insert if statements if necessary). I also haven't discovered a way in Ruby (yet?) to do the related task of a one-line instantiation of a hash. In Python, this would be d = dict((x, None) for x in [1,2,3])

Perl-like operators << <=> and =~? Sorry, I just can't get into it. Keep it simple, please. Function calls with letters are good.

Lists and hashes are printed strangely puts {1=>2,2=>3,3=>4} gives me 122334 in Ruby. WTF? I thought I was dealing with a hash, not a number. print {1: 2, 2: 3, 3: 4} gives me {1: 2, 2: 3, 3: 4} in Python. puts [1,2,3] gives me numbers on separate lines in Ruby. print [1,2,3] gives me [1, 2, 3] in Python. The seemingly small detail that data structures are printed out exactly how they are written in code makes a world of a difference.

Strings are unintuitive Strings are iterated over by word, not by character in Ruby: "one two".each {|x| puts x} in Ruby outputs "one" and "two" on separate lines. Compare this to for i in "one two": print i in Python, which prints out a character on each line. To iterate over each character in a string in Ruby, you'll have to use each_byte (in which case you'll find that Ruby will print out numbers unless you convert the numbers with .chr... this problem also occurs when selecting a character from a string, as in str[5] even though the docs say otherwise) or each_char.

Bad decisions on naming Why is there a built in variable called $DEBUG? Doesn't everyone have a global variable called that in their program at some point or another? Well, overwriting this variable unknowingly will cause your code do very strange things, like disregard begin/rescue/ensure blocks. I found this out the hard way. Python's is __debug__. Who is going to create a variable called __debug__ in their program? Nobody, that's who. The closest thing Ruby has to a constructor is called initialize. Why so long? __init__ works fine, and programmers who failed literature can actually spell it.

Shortcuts for everything "p" for print. The attr family of keywords for attribute access. Why not just create usable constructs and stick with them, instead of inventing unwieldy constructs and then working around them with shortcuts?

Hype Ruby is definitely the flavor of the week in programming languages. This results in a bunch of n00bs flocking to the language when they want to learn to program. Now, I have nothing against people who want to learn how to program. In fact, I'm thrilled that anyone wants to take a little time to learn the art. But the experience of a community has benefits: better tools and better discussions. As a result the conversations in Ruby forums and mailing lists are generally of a lower quality than that on Python mailing lists which tend to be frequented by more knowledgeable, veteran coders.

Japanese Origins As an English speaker, I appreciate that most of the cutting-edge blog posts and PEPs and what not are in English. Ruby originated from Japan, so this is not the case there. It's frustrating for me to see a link like "the solution to your problem is here" only to be led to a page which to me might as well be in... well, Japanese. I am sure I would feel differently if I were Japanese, of course.

OSX/TextMate-centric community A large number of Ruby users use Macs. Not sure why, that's just the way it is. They also seem to like TextMate. Me? I hate Apple and love vim. The more people that use the tools that you use, the higher probability that someone has already solved the configuration problem that is bugging you. Although I will note that the default settings for Ruby in vim are very nice... there's at least one knowledgeable vim user writing Ruby out there.

Younger Language Python has been around since the late 80s. Ruby has been around since 1993 (both for certain definitions of "around"). Python has a head start on Ruby in terms of developer time. A more mature language is likely to have fewer bugs, more features, less experimentation and and more users. The same goes for libraries written for that language: a greater number of better-implemented libraries will be available to users of a language that has been around longer. All other things being equal, the older language has to win the stability and reliability battle.

Unfamiliar keywords Instead of try/catch (or except)/finally, Ruby has begin/rescue/ensure. Instead of continue, Ruby has next. Instead of else if or elif, Ruby has elsif. Instead of switch/case, Ruby has case/when. I guess every language commits this foul to some extent, but it's just one more thing to trip over.

Implicit returns Another Perlism that I despise. To me, a function looks like it doesn't return anything if it doesn't have the "return" keyword in there at some point. It's much easier to read (and maintain) code if the points at which a function could return something are explicitly outlined.

Things I don't have an opinion on... yet

Introspection/Reflection Haven't really dug that deeply into what Ruby can do here yet...

Package management The gems system vs letting apt-get do everything. Neither has failed me in any way thus far, but then again, I haven't really tried to do anything remotely complicated with either and I'm not a sysadmin presiding over a myriad of different configurations...

Threading Ruby doesn't support native threads. What are Python's options? This looks like a good article and what about Stackless Python?

Object Oriented stuff Namespaces, modules, Inheritance, etc. I'm sure there's some Python vs Ruby flamewar going on out there, I just have to find the thread...

Funny UNIX utilities

2008-01-07T07:15:00.000-08:00

Some humorous UNIX utilities that brighten my day:

cowsay -- get a cow, etc. to do the talking
fortune -- witticisms, some more offensive than others (fortunes-off package contains more offensive fortunes, fortune -o invokes offensive fortunes)
banner -- big friggin letters

Printing with enscript

2008-01-07T07:07:00.000-08:00

enscript is an awesome utility for printing text files; it kicks lpr's lame ass. Here are some examples of usage (more to come as the need arises):

Letter enscript --no-header --word-wrap example_file.txt

Also, navigate to localhost:631 on any system with CUPS installed in order to configure the default printer and such.

Learning Ruby from Python

2007-12-14T14:44:00.000-08:00

I just started to pick up Ruby today (to install: sudo apt-get install ruby irb ri rubygems). Here are some things I learned along the way, coming from a Python mindset. The (very good and highly recommendable) references I looked at were (in order):

Similarities

Brackets for arrays, braces for dictionaries (called hashes in Ruby)
Strong dynamic typing
Everything is an object, variables are just references
Exceptions are similar
No special line termination characters
# for single line comments
def to define a method, class to define a class

Differences

Interactive prompt: python => irb
Interactive prompt help: help(str.count) => help "String#count"
Interactive prompt reload: reload(foo) => load "foo.rb"
Command line docs: pydoc => ri
File extension: .py => .rb
Shebang line: #!/usr/bin/python => #!/usr/bin/env ruby
Indentation and blocks: : and tabs => either { } or end
Strings: immutable => mutable (can use freeze method for immutability)
Naming conventions: unenforced => enforced (ex. class names start with a capital letter, variables start with a lowercase letter)
Raw strings: r"blah" => 'blah'
Parantheses: mandatory => sometimes optional
Booleans: True, False => true, false
Null value: None => nil
Else-if statement: elif => elsif
Module import: import foo => require "foo"
Boolean conversion: 0, False, None and anything empty => Only nil and false
Doc generation: docstrings below things => regular comments above things
Output: print => puts (or print if you don't want a trailing newline)
Command line execution: python -c => ruby -e
Global Variables
Ranges: range(x,y) => x...y exclusive, x..y inclusive (.to_a to return array)
Slicing: arr[1:3] => arr[1...3] or arr[1..2] (inclusive)
Object initialization: __init__ => initialize
Object creation: a() => a.new
Ternary operator: if_true if statement else if_false => statement ? if_true : if_false
Substring matching: 'foo' in str => str['foo']
"Main": if __name__ == '__main__' => if __FILE__ == $0 ... end

Philosophical differences

Attribute access: direct access => method calls

Need getter and setter methods to access attribute outside class

Access permissions: convention by underscore => public, protected, private
Multiple inheritance: allowed => mixins

Stuff Ruby has that Python doesn't

Double quoted strings allow expression substitution #{} and escape sequences \t\n
You can re-open a class at any time and add more methods
Block comments: ==begin and ==end
Special variable characters (begins with...):

$ global
@ instance
[a-z_] local
[A-Z] constant (triggers warnings when reassigned)
@@ class

Arrays support some set operations

difference -
intersection &
union |

Method conventions (ends with...)

! changes internal state of object
? returns boolean

do keyword -- optional loop keyword (unless on single line)
Crazy for loop alternatives
General delimited strings
Case statement
Lots of options for string element access and string substitution and insertion
Regular expressions as first-order objects
Perl-like control flow constructs that can be tacked onto the end of an expression

"Why don't you write a Facebook Application?"

2007-12-11T21:02:00.000-08:00

There is a lot of hype surrounding the newly released Facebook application platform. Developers are falling all over themselves to get a product rushed out the door, funding for Facebook apps is flowing in from VCs and you can even take a class in Facebook application development at Stanford. Amid all of this irrational exuberence, I find myself frequently getting asked why I don't write to it. Well, there are a lot of good reasons to stay as far away from the platform as possible. It's proprietary, unpredictable, not a level playing field in the least and crowded with unworthy offerings. My time and effort is much better spent somewhere else.

First of all, even if we assume for the sake of argument that I am interested in developing a Facebook app, the deck is stacked against me. Tim O'Reilly points out that there is a steep drop off in users after the first four or five most popular apps (the graph on O'Reilly's page is instructive). There are a few reasons for this. The first is that the door has largely closed on the willingness of users to add new applications or suggest them to their friends; the fad has come and gone. In other words, "[W]e must acknowledge that ... users have grown accustomed to a small subset of applications available to all users, and the exploratory period has come to an end... Facebook is all about communicating an image of yourself to your peers, and unless someone creates a truly captivating, innovative, or better F8 application, no one is going to adopt it on a whim." The quickest functional and moderately compelling apps to get out the door were the ones that users pounced on. The second reason for the disparity in users parallels the disparity in the finances and professionalism of the makers of the apps. Five of the top six apps right now are made by just two companies: Slide and RockYou!. [A] Making little flash widgets is essentially these companies' entire business and both of them are well financed. Amateur developers do not stand a chance against a small army of VC-backed Flash coders who do this for a living. [B] Any entry into the Facebook app market now is essentially futile.

The survivability of the Facebook platform is another issue. It bears some eerie similarities to "walled garden" initiatives past. Perhaps most illustrative is the case of AOL, whose proprietary platform, of course, nobody uses anymore. Should I really bother developing against a technology predicting that it won't survive 10 years? Perhaps, but I would rather write to a platform that is built on open standards. Jason Kottke explains: [E]

As it happens, we already have a platform on which anyone can communicate and collaborate with anyone else, individuals and companies can develop applications which can interoperate with one another through open and freely available tools, protocols, and interfaces. It's called the internet and it's more compelling than AOL was in 1994 and Facebook in 2007.

The fact that Facebook could change its platform or go out of business at any time is a disincentive to write a Facebook application. I don't like my creations functioning at the whims of Facebook or its investors; it's a better bet to go with an established, open platform.

I ran across a similar post on Facebook app suckiness that raised an additional reason not to develop a Facebook app. If, against all odds, your application is truly innovative and distinguishes itself in the crowd of petty profile pollution, you still are on Facebook's turf; that carries some important consequences. Specifically, Facebook could copy your idea with a feature that they subsequently build in to Facebook itself. It will be accessible to all users, not just the ones that choose to add it as an application, and there will be no use for your app anymore. Facebook could take your idea and reimplement it, thus sucking away all of your users, and there would be nothing you could do about it.

Then there is the embarrassing stigma of being in the company of Facebook application developers. The (many) applications are written by developers who want to jump on the Facebook hype bandwagon. Most of them are amateurs, and it shows. Take the Quizzes application, for instance (ranked 14th in most active users, so a reasonably popular app). It lacks basic functionality that one would expect in such an application: namely, the ability to see the correct answers after you take the quiz. It also creates a quiz that other people can take before you even indicate you are finished adding questions! Furthermore, I couldn't access the application at all today because, presumably, it is choking under the load that Facebook is sending to it (another all-to-frequent problem of these applications). Quizzes, admittedly, might be an application that one might use occasionally. I can not say the same about other applications, most of which can be broken down into four categories:

Horoscope

What flower are you? -- "Are you rose, lilly, foxglove, daffodil, nigella, sunflower, violet? Find out."

Birthstones -- "From the time gemstones were discovered, they were believed to have mystical powers and attributes that could be passed to the wearer. The red of ruby was fiery and passionate; cool blue sapphire was calm and composed. What's yours?"

Psychic Breeze - Psychic Readings and Fortune Telling -- "Accurate psychic, psychic medium and clairvoyant readings are available today. Our extraordinary psychics, mediums and clairvoyants use their psychic ability to provide you with guidance in relationships, love, finances, and other life hurdles."

Decorative trinkets

Snow Globe Gifts -- "Snow Globe Gifts! Send your friends realistic, shakeable, snow globes..."

Christmas Music Tree -- "Create your own special Christmas Tree. Decorate your tree with your friends and make them play many famous Christmas songs. It's your Musical Christmas Tree."

My Fab Bag -- "Buy and give your friends designer handbags. Choose your favorite to carry and display on your profile."

Finding a boyfriend

Are YOU Interested? -- "FUN application to see who is interested in YOU!"

Likeness UNRATED -- "Find out who you're like on mature topics like purity, vices, and the seven deadly sins. Can you find a perfect match?"

Will you KISS me? -- "Send kisses to loved ones, lovers, romantics, mistresses and everyone! Ever wanted a friend or crush to kiss youand find that they want to kiss you too What if it happens on a Full moon night? Give and take kisses"

Just good ol' pestering people

Zombies -- "Bite your friends and make them zombies! Mmmmm... Brains! Also - you can fight Zombies, Vampires and Werewolves now!"

Vampires -- "From the makers of Zombies comes Vampires! Kind of like Zombies. Except that the vampires are hot. VERY hot. Also - you can fight Zombies, Vampires and Werewolves now!" [C]

X Me -- "Tired of just poking? X me opens up a whole new world of action-based messaging, for example 'Hug Her, Slap Him, Tickle Them!'"

Is this trash what I really want to spend my time developing? Do people actually waste their time using this filth? It's depressing that humans' frontal lobes have not yet developed sufficiently to resist soliciting the advice of fortune-teller applications. [D] Given this kind of crowd, I don't think I could expect anyone to take me seriously if I told him I was developing a Facebook application. I would anticipate him laughing in my face.

That's why I don't develop for the Facebook platform: it's pure hype. Nothing of significance has been produced in the form of a Facebook application, and the only applications that get any views are ones that are made by companies who specialize in them. This environment exists at the whim of Facebook and its investors, and could come crashing to a halt if the business goes belly-up or Mr. Zuckerberg has a change of heart. I'll take my code somewhere else, thank you very much.

===========================

[A] Prepare for some awful, awful music and graphics if you go to RockYou!'s page. I had the displeasure of being aurally assaulted by Lil' Mama blowing kisses at me. You have been warned.

[B] To hammer this point into the ground, RockYou!'s CEO discusses here how, in order to prepare for an expected spike in traffic, they ran around filling a 24-ft truck full of servers and hooking them up over a weekend. Does this sound like the kind of thing you could pull off with the change you found under the sofa? I didn't think so.

[C] "From the makers of Zombies..." As if Zombies was a fantastic, useful product to stake their reputation on. They have WereWolves, too. Maybe there's more, I didn't care to look.

[D] And there's a hell of a lot more dumb applications, if you want to take a peek.

[E] He expands on the "Facebook as AOL" theme in this post. Scott Rosenberg also has his thoughts on the subject.

Why I am Still on Facebook

2007-12-11T14:46:00.000-08:00

Facebook sucks. And yet, I can't bring myself to terminate my account. It has a few essential features that I just could not do without because of its dominant position as today's preeminent social platform. Like the AOL of yesteryear, everyone else uses it (a bit too much) and expects you to use it to communicate with them. These few communication and sharing features are the site's strengths. Facebook counts on users sticking around in order to use these features regardless of how many ads they plaster on their pages or how many privacy violations they can rack up. Facebook's flaws are outweighed by its utility, which is why I haven't quit Facebook... yet.

Facebook's best feature is what it started out as: a directory of contact information. The average person does not have the time, skill, interest or money to register a domain name, set up a personal web page and push it to the top of Google's index for searches against his name. A social networking page is a much more convenient way to disseminate contact information. Facebook is the best solution for this because it shares contact information only with people of which the person approves. This arrangement makes many more comfortable than if they had shared their contact information on a public site such as MySpace. As Facebook also started out as a college network about the time that I was in college, it is the platform of choice of virtually all the people I would desire to contact. Most of the time, the aggregate contact information for someone just starting their professional life is not available anywhere else. I can not count the number of times when I quickly needed to contact somebody, only to realize that I did not have her AIM screen name, email address, phone number, etc and fetched the information off Facebook.

The other killer feature of Facebook is photo sharing, which is why it is now the largest photo sharing site in the United States with over a billion pictures. If I were to withdraw from Facebook, I would no longer have access to my friends' photos and they would no longer have access to mine. Of course, I could always migrate all of my pictures to Flickr or some other site and then point all my friends there. But is it really worth the time and effort to do that, knowing that Facebook already has a great framework for doing this that my friends and I currently use? Again, Facebook's privacy controls are also an important feature: if I transitioned to another platform, I would need to re-implement the privacy controls there (assuming they were even supported!). Additionally, all of the tag data and comments that Facebook supports would be lost in translation.

And let us not forget the networking part of social networking. Facebook does provide opportunities to network, in the business sense. [A] It's easier to approach a person if he looks familiar, you have some background on him and you can draw from a number of topics of conversation. Also, even if the only contact two people have is seeing each others' name go by on their respective News Feeds and having the other's face pop up once in a while on sample friend lists, that connection is sufficient to leave a lasting impression on the subconscious mind. [B] And requesting someone to be a friend on Facebook can be sometimes interpreted as a digital 'good meeting you' compliment. In spite of all this, I believe the networking aspect of Facebook to be a relatively minor incentive for staying on the site.

So photos and contact information are really the only two things that keep me from ending my adventures with Facebook. For the time being, it seems that I'm just going to have to swallow my pride, eat my words and keep using the site.

=============================

[A] "Expanding one's social network or sphere of influence by initiating mutually advantageous new relationships with people." (link)

[B] I didn't come up with this theory, my friend Monique did.

Facebook Sucks

2007-12-10T15:06:00.000-08:00

Every time I sign on to Facebook, a little part of me dies. Not only am I usually wasting my time, but I am allowing Facebook to violate my privacy, potentially offending hundreds of "friends" and being bombarded with ads and spam. Furthermore, I am forced to use Facebook's clumsy tools to communicate with others on Facebook who seem to never have heard of email, all while wading through the ostentatious posturing of Facebook's users. In short, Facebook sucks.

Facebook is a great opportunity to offend people. As if I didn't have enough trouble minding my etiquette in the real world, the choice to friend or not to friend (or grant restricted access, or defriend) provides daily chances for someone accidentally or intentionally insult someone else. The heart of the problem is that some people have different conceptions of what a Facebook friend actually entails. Does it mean you are friends in real life? Is it meaningless? Some are willing to Facebook friend total strangers and others keep a very small circle Facebook friends that might actually be closer to the number of good friends they have in real life. There is a point at which this managing of digital networks becomes tiresome, evoking a social network fatigue. The value of a particular user's experience on Facebook (or on any social network) rises and then falls as the number of users increases. [B] At first, the user is excited to connect with all of his friends and perhaps reconnect with some that he had lost touch with. But over time, as more people join the site, more time is spent on fending off unwanted friend requests and friend network management. This eventually drives the user to become much less active on the website, if not to opt out of it completely.

Facebook is a black hole that sucks up time. There is certainly something compelling about browsing your "social network" through a hyperlinked photo yearbook. In fact, it's too compelling -- some have complained of "Facebook addiction." Facebook exacerbates this problem (well, certainly not a problem for them) by sending you incessant reminders of activity on your account by default ("Someone has done x to you on Facebook") which pull you back to the site again. [A] Apps have worsened this addiction because now every app requires its own micro-management and sends its own messages to your inbox. Here is a picture of the overwhelming number of annoyances a typical Facebook user might face upon login. All of the time spent on Facebook wouldn't be wasted if there was substantive communication taking place on the site but, for the most part, there's not. It's all just about how many people you've converted into zombies or whether you identify more with pirates or ninjas.

Facebook reinvents the wheel in a variety of ways, moving online communication a step backwards. Since Facebook wants you to stay within the site's walls, Facebook provides tools for you to accomplish certain goals, no matter how mediocre those tools may be. For example, Facebook provides a "Marketplace" for users to buy and sell items on their site. Of course, there are many superior auction/barter/market sites already on the Internet: Amazon.com, eBay and Craigslist, to name a few. Facebook provides "Posted Items" and "Notes," whose features are poor substitutes for nearly any blogging platform. And Yahoo and Google groups are many times more advanced than Facebook's groups. The most irritating example of Facebook's compulsive re-engineering is Facebook messages -- it reminds me of a dark age when GMail didn't exist, and also gives me another inbox to manage (much more clumsily, mind you).

Twice Facebook has disregarded its responsibilities to its users and precipitated privacy invasions, both for which Zuckerberg promptly issued apologies. First there was the News Feed, which broadcasted users' actions to all of their friends. Facebook followed that with Beacon, a system that tracked a users' actions on affiliate sites, such as the New York Times, and then fed information back to Facebook (and that users' friends through the News Feed) about a users' behavior. Twice Facebook has recklessly played fast and loose with its users' data, and twice it has pushed its audacity to the limit until it faced a revolt by its users. The most shocking part of this whole story is that these systems never went away! In each instance Zuckerberg waved his hands to make an apology, as if users' concerns had been assuaged, and only partially disabled the systems that caused the uproar. The News Feed, although it did get some controls, still doesn't give the user a choice if some types of stories are broadcasted. Beacon is also wholly intact, but was changed from an opt-out to an opt-in system. [C] There is no reason to think this is the last time this pattern will happen, as Scott Rosenberg points out. To justify its massive valuation, Facebook is under a lot of pressure to find additional ways to monetize its service, and there is good money to be made selling out users. What privacy-infringing "feature" will Facebook be pressured to invent next? [D]

And then there are ads -- lots of ads. In addition to the easily blockable banner ads on the bottom and sides of the page (an Internet staple since as far as I can remember), Facebook has devised ways to deliver ads to users that are not so trivially thwarted. Facebook actually embeds ads inside the News Feed that come from the same server as the rest of the News Feed, unlike other embedded ads (like Google's) which come from a third-party server and are thus easy to identify and block. Fortunately, there are some ways to rid your eyeballs of these menaces. It is also much harder to tell that you are looking at an ad in the news feed: Facebook blends them in so well to almost make them indistinguishable from bona fide News Feed stories. This approach is in stark contrast to what Google and other sites do, clearly identifying which content is sponsored and which content is not. This practice is irritating at best and deceptive at worst.

And speaking of deceptive ads, how about using my image in an ad for a sponsor, as if I were sending a personal recommendation to a friend? Taking a "social action" (as Facebook puts it) is not a license to use me as a viral marketing stooge for Blockbuster, et al. [I] To add insult to injury, Facebook is now allowing advertisers to send targeted emails directly to your Facebook inbox (the first line of the most recent one I received from CbsSports.com: "Hey College Hoops Fan!"). Hm, unwanted emails in my inbox trying to sell stuff; I think that's better known by its more conventional name -- spam. You spam your "friends" with application requests, corporations spam you with messages in your inbox, your "friends" spam you with pokes and news feed items. This is essentially what Facebook has become: a very efficient platform for spamming people.

You can put a lot of data in to Facebook, but getting that data out is an entirely different story. It is quite easy, for example, to import your contacts from another platform into Facebook. Facebook, however, provides no convenient method for exporting those contacts into Outlook, Gmail, or the other social network flavor of the week. The same goes for photos, videos and all other multimedia. Looking for a "backup my photos" link? Sorry, it doesn't exist. There are ways to get data out of Facebook, but they are inconvenient and few. One is to use the API either by writing an app yourself (clearly out of the reach of most users) or using an application like FriendCSV [K]. The API, however, doesn't allow extraction of some types of information, like email. Another is to scrape the site, which is against the terms of use (like most companies') and can lead to the termination of your account if they catch you doing so. And it is also impossible to get Facebook to delete your information from their servers, even if you quit using the site! Facebook is not only a black hole for your time, but also for your personal data.

Facebook has become the victim of its own success: phishers are starting to use the site as a launchpad for attacks. Phishers embed links on a user's wall that point to a malicious domain that harvests their names and passwords for Facebook. This, in turn, can lead to more phishing attempts as well as stealing other credentials (such as banking login information) and/or spamming for pharmesuticals, etc. Of course, no site is immune from the scrutiny of attackers, so this is hardly Facebook's fault. As a commenter on the Wired blog puts it, "Anywhere there is popularity and potential profit, there will be hackers and scammers." However, it is notable that criminals now see Facebook as a lucrative target. Facebook needs to crack down on these activities if it expects users to continue to feel comfortable using it. [J]

Perhaps this isn't the fault of Facebook per se, but a lot of the people on Facebook are really annoying. You know the ones I'm talking about. The coward who thinks that the epitome of activism is clicking a button that says "Join Group." [E] The gullible student that believes the world will be changed by joining groups with titles like "For every [number] people that join this group, I will donate [amount] to [cause]." [F] The narcissist that ceaselessly uploads pictures of themselves and her friends partying and broadcasts her status message to the world at least ten times a day. The clueless folk carrying on what, prior to Facebook, would have been a private conversation on each others' walls. [G] And people that have way too much free time giving each other gifts [H] and engaging in poke wars (or now, thanks to SuperPoke, throwing cow wars or the like). Facebook is often a cesspool of narcissism and ignorance that I could do without.

It may come as a surprise that, despite all of these grievances, I haven't terminated my Facebook account. It is true that I still grudgingly sign on to the service at least once a day because it provides some tangible benefits that no other service offers. Regardless of its flaws, I haven't quit Facebook... yet. I intend to write two follow up articles to this one, the first discussing what Facebook gets right and the second as an answer to "Why don't you write a Facebook application?" Stay tuned.

Update: I'm finally getting off of Facebook. The straw the eventually broke the camel's back for me was the sheer unusability of the site. Nearly every page load on Facebook maxes out my processor (on a decent machine). It's not just the sheer load of crap that Facebook is bringing into each page; even the most basic user actions cause my browser to lock up. For example, entering characters into a text box (for commenting on a photo or sending an email) has a delay of several seconds between when I hit the keys on the keyboard and when the letters show up onscreen. These inexcusable bugs plague the site. Congratulations, Facebook, you've finally driven me away.

=========================

[A] Yes, I know you can change this in your settings. Yes, I know that they now send the contents of messages in the email as well. Everything else, however, still gets you the same information-void kind of notification that begs you to come to Facebook if you want to find out what was actually said.

[B] I'm certainly not the first person to identify this phenomenon, by the way. I'm not sure who, if anyone, is the right person to attribute this to. Thoughts?

[C] And who knows what Facebook thinks "opt-in" means? The devil is in the details: does not clicking on an "I don't want this" indicate the user wants to participate? Zuckerberg, upon Beacon's release, already had some interesting ideas about what "opt-in" meant.

[D] Ed Felten provides an excellent Beacon post-mortem here.

[E] Some think that the best way to protest Facebook's practices and policies is to join a group whose cause is to recognize the fact that all its members dislike a new feature. It's not. The best way to protest is to delete your Facebook profile.

[F] A frequent question I ponder when I see groups like this is, why do people waste their time supporting these groups if they have zero assurance that the donation/action/whatever will actually happen?

[G] I really, really don't need to know the day-to-day private details of your life. And I REALLY don't need them broadcasted to me in my News Feed. If you're negotiating a play date with your friend, take it off Facebook! If you're dumping your boyfriend, take it off Facebook!

[H] Perhaps these do serve a cause since Facebook donates $1 for most of them to charity. But it annoys me when it is implied that there is some kind of scarcity to information, playing to misconceptions about the Internet. Okay, pet peevey rant over.

[I] In the legal sense, as well: could this practice be illegal?

[J] It is also a testament to the cleverness of the phishers (and the nature of Facebook's users) that they are using such well-targeted bait in the text for their links: "lol i can't believe these pics got posted.... it's going to be BADDDD when her boyfriend sees these,"

[K] Careful, FriendCSV's creators try to sign you up for their own social network when you use their product. How hypocritical, offering a way out of someone else's frying pan and into their fire.

===================================
UPDATES:

Facebook is sharing too much data with application developers. (Link)

Facebook, in violation of their privacy policy, is now sharing your personal data with Microsoft. Hm, does that have anything to do with taking $240 million of their money?

Facebook is arbitrarily removing applications that don't seem to be in violation of their privacy policy, a la Apple and the iPhone store. The victim this time? Burger King.

Facebook may be eliminating local networks, exposing more personal data to more people.

In a rare moment of good news, Facebook has agreed to abandon Beacon.

Another reason to stay off Facebook: STDs.

Code formatting

2007-12-04T23:46:00.000-08:00

Here are some tools to make your code pretty (in Ubuntu repos):

C -- indent
HTML -- tidy
Python -- N/A :)

For indent, I find that the following arguments seem to work well for code I write: -bap -bbb -bl -blf -bli0 -bls -cli3 -di1 -fca -hnl -i3 -ip0 -l80 -lc80 -nbbo -nut -nsaf -nsai -nsaw -psl

Blocking Facebook News Feed Ads with Greasemonkey

2007-11-12T12:54:00.000-08:00

I hate ads, so I block them. If I want something, I'll search it out. I don't need marketers wasting my time trying to convince me that I need something I don't. Most of the time I have ads blocked by aliasing most known ad servers to 127.0.01 in /etc/hosts/. This only works when the ads are coming through a server that is listed in the file (say, ad.doubleclick.net) and not when the ads embedded in the page are served from the same server from which the original page was requested.

This is exactly the problem with the ads from Facebook, which just deployed a new ads system that has ads targeted on what you've listed in your profile. For example, if you listed Jane Austen novels as some of your favorite books, you might see text ads in the sidebar or in your news feed advertising "Jane Austen: The Ultimate Sappy Romance Collection" ... or something. So how to block these ads?

Greasemonkey is a Firefox (and IE, although I wouldn't know much about that :-) ) extension to inject arbitrary Javascript into web pages, meaning that you can manipulate a page, or parts of a page, automatically as you see fit. Its behavior is defined by scripts, many of which can be found at userscripts.org. If you're familiar with web design, you can write your own scripts quite easily. Mozdev has a good quickstart guide, and Mark Pilgrim has an online guide and a book if you want to research this further.

There are a few ways to do block Facebook's new ads. The first is using document.getElementsByTagName('div') to fetch all the divs in the page and then loop through them, testing them for known ad-serving attributes (for example, a className like 'social_ad_advert' (sidebar) or 'feed_item clearfix social_ad' (news feed) or an id of 'ssponsor', 'sponsor', or 'announce' (all banner ads)) [1] and then either hiding them with element.style.display = 'none' or removing the element entirely using element.parentNode.removeChild(element). A second is using XPath, which makes for much more compact code. [2] userscripts.org is awash in (mostly mediocre [3]) scripts to block Facebook ads, but the most comprehensive one I found was "Remove All Facebook Ads" which I'm using in my browser right now. The only suggestion I have for this script would be to use the window.addEventListener to trigger execution of the code rather than embedding it in an anonymous function, as explained in the quickstart guide (under "Tips").

Greasemonkey scripts are executed after the DOM is loaded, which means that all the ads will be fetched from their sources and displayed on the page before they can be stomped out, unfortunately. Therefore, to make the page load faster by not fetching and displaying ads in the first place, it would be wise to use an /etc/hosts blocking scheme instead of a Greasemonkey solution where possible. Greasemonkey should be used only as a last resort in combination with /etc/hosts where /etc/hosts can't block ads that aren't fetched from a server different than the one used for the requested page, as in the case of Facebook news feed ads.

[1] It would probably be faster to not test for the id matches and just using getElementById for those few cases outside of the loop instead (within try-catch blocks if the elements aren't guaranteed to be on to page and could potentially throw an error).
[2] I'm not sure about the efficiency implications of using one vs the other. Anyone care to comment?
[3] For instance, "Hide Facebook Ads" tries to do some browser detection... I'm not sure how large the market for IE4 Greasemonkey users is...

UPDATE:

Now using New Facebook Layout Ad Killer for the new Facebook layout

Argh, it seems Facebook has added another lame ad sidebar. Just extend the NFLAK script by looking for a document ID called 'fadbar' and hide it if it's there.

Gutsy (Ubuntu 7.10) Tweaks

2007-10-24T01:31:00.000-07:00

Just upgraded to Gutsy from within Feisty, which took about an hour, and things look great, for the most part. But here's a few things just to make sure everything was in tip-top shape:

To enable Compiz Fusion, System->Preferences->Appearance->Visual Effects->Extra
Gutsy doesn't ship with the Compiz Fusion settings manager, and I wanted my old (Beryl) settings back... run sudo apt-get install compizconfig-settings-manager in a terminal to install it and ccsm in a terminal to launch it

Don't forget to create a new profile in Preferences or else you'll be screwing with the default settings
Personally, from the defaults I enable Rotate Cube, Paint fire, Water effect, Wobbly windows, Annotate, Splash, Cube Caps, Shift Switcher; I disable Expo; I tweak Desktop Cube (General->Multi Output Mode->One Big Cube) and Animations

Fonts on my system were oddly stretched out vertically. Here's a solution from this thread: edit the line in /etc/gdm/gdm.conf that says -command=/usr/bin/X -br -audit 0 to say -command=/usr/bin/X -br -audit 0 -dpi 96
The fonts still look a little screwy. I'm just going to bite the bullet and install the MS core fonts ... a HOWTO is here.
Download updates automatically in the background: System->Administration->Software Sources->Updates->Download all updates in the background
Synchronize your clock to NTP servers sudo apt-get install ntp
Autohide pannels: Right click on a panel->Properties->Autohide

Set how much of a panel can be seen when it is hidden: link

An Indictment of Apple, Part 1

2007-10-23T14:09:00.000-07:00

Unless you have been in a cave for the past several years, you've probably noticed the increasing brightness of an odd, white glow that radiates wherever students, geeks or professionals congregate. That strange luminosity is the combined brilliance of the hypnotizing and stylish logos on Apple notebooks that everyone seems to be carrying around these days. It's no secret that Apple's hardware (and accompanying software) has been selling well recently, with its popular laptops, iPods, iPhones, etc. Everyone I know seems to be purchasing one of Apple's shiny toys: my family, my friends, my UNIX zealot colleagues and technoignorant acquaintances alike. The media constantly flatters Apple with positive reviews of its products, editorials praising CEO Steve Jobs' visionary genius and provides an extraordinary amount of marketing hype for its new products. Could this company possibly do any wrong?

The answer to that question is a resounding "Yes." It could, it can, and it does.

Apple's products have become a liability to own. They are often ladden with restrictions, break easily, frequently can not be extended by third-party developers and are, in some cases, just a prime example of bad design. Furthermore, it is perplexing that Apple's products continue to be so popular when superior alternatives exist. In this article, I will focus primarily on how a top-down culture of control inherent in Apple products decreases their value. I will address other issues in subsequent articles.

Some of Apple's problems stem from its desire for the user to have an experience using its product over which it has total control. Another way to phrase that would be that Apple wants you to lock you in to its product line, so that it can compel you to fork over your cash for the next edition of OSX or the newest personal electronic device. It is a culture of top-down control that, on the whole, does not benefit the user. For example, about a month ago new iPods shipped with a technical restriction that prevented them from working with other operating systems (Fortunately, the gtkpod developers broke the lock within days). Basically, Apple is saying to the consumer, "In order to use our products, you need to buy even more of our products. Otherwise, tough." Apple is similarly inconveniencing customers with iPod's ability to pipe video output to a TV: in order to use this feature, you need to either use a device with a built-in Apple authentication chip or purchase an Apple video cable for $49.

The recent tug-of-war between iPhone owners who want their pricey gadget to be as useful as possible and Apple's not-unless-we-approve attitude is an illustrative case in point. In case you haven't been following the news, Apple made a deal with AT&T so that it would be the exclusive carrier of iPhone voice traffic for five years and built technical mechanisms into the phone to ensure this arrangement. Naturally, customers of other carriers wanted to use the phone on the network they were already paying for, which meant they would have to hack the phone to get this functionality. Thus, the iPhone was hacked, again and again. Apple estimates as many as 250,000 people, roughly one sixth of iPhone buyers, have hacked their phone. In response, Apple released a firmware update to the phone which made the unlocked phones inoperable. Apple's consistent attempts to prevent its customers doing what they want with Apple products are an insult to its users' intelligence and detrimental to their ultimate experience with the iPhone. As Jon Lech Johansen puts it, "When Steve Jobs claimed the iPhone was 5 years ahead of every other phone, was he talking about the iPhone’s revolutionary handcuffs? In a world where open technologies are increasingly becoming the norm, Apple’s way of Thinking Different means marching in the opposite direction."

Apple is hostile to not only users' interests, but developers' interests as well. Will Shipley, a developer for the Apple platform, writes an excellent post about third parties' inability to put their applications on Apple's platforms. He explains that Apple's arrogant attitude is damaging its relationship with users and developers alike:

And the iPhone is a closed system, like the iPods before it, so third parties can only develop software for it if they are EXTREMELY close to Apple. This is an incredibly frightening trend. As Apple gets more and more of its revenue from non-Mac devices, they are also getting more and more of their revenue from devices that simply exclude third parties...

But with the iPod Touch, what's Apple's excuse for locking up the platform? Why can't I write programs for this device? Who might it hurt? Why is Steve announcing that he's playing cat-and-mouse with developers who intend to do so? Is Apple so far removed from its customers that even when the latter overwhelming votes for extending a device (by downloading iPhone programs in the hundreds of thousands), Apple's response is, "No, you can't do that. We know what you want, you don't. You want AJAX apps, you just don't know it yet."

That sure reminds me of the old, crappy Apple. The one that almost went bankrupt because of its hubris.

When Apple disallows third parties from writing to its platforms, then everybody suffers. The users suffer because they don't get to use the added functionality of their devices, the developers suffer since there are less opportunities to extend Apple's software and Apple suffers because less people buy their gizmos and consumer relations are soured. Innovation often doesn't happen at the center -- it happens at the edge, and Apple does not have a monopoly on good developers or good ideas.

Apple's poisonous politics don't stop at simply excluding developers from its platforms -- they permeate the very coding tools engineers use to create. I submit to you the bizarre incidents of Apple crippling both the dtrace and gdb versions on OSX. Both programs' source contain uncommented portions of code that disallow low-level examination of certain Apple-built programs, including iTunes. These suspicious lines are not in any other version of these utilities on any other platform. Why Apple did this is anyone's guess -- it seems rather stupid to try and outwit many extremely talented programmers who will inevitably discover these unpleasant surprises (as they did). What is certain is that Apple's handicapping of OSX versions of open-source tools make Macs a less attractive purchase for developers.

Now we turn to iTunes and its accompanying Digital Rights Management (DRM), which is quite the antithesis of Apple's "It just works" slogan that it uses to push its products. iTunes has an elaborate system of "Authorization" that limits the number of computers on which one can play their music. To play a song from another computer, you must get that computer to "authorize" yours. which requires it dialing Apple HQ to ensure that this is all right with Mr. Jobs. If you don't happen to have a network connection at that point, you can't make this connection and thus can't play your friend's music -- tough luck. Of course, only a total of 5 computers can be authorized per source computer, so if your friend has already authorized the limit, he'll have to de-authorize someone else (perhaps even himself, which is allowed -- revoking the right to play songs he paid for via the iTunes store on his own computer... the gall of Apple amazes me!), at which point that person who was just de -authorized will start wondering why she can't play certain songs on her computer. What a mess. I find it repulsive, not to mention inconvenient, that I need to ask a corporation's permission to listen to certain songs in certain ways.

iTunes also has a bad habit of rolling back its already limited functionality with its frequent 'upgrades.' iTunes 4.5 decreased the amount of CDs one can create with a single playlist from 10 to 7 and also detected and blocked similar playlists. iTunes 4.7.1 took away the previous ability to stream music to anyone on a local network. One wonders when the march towards never being allowed to play any music anywhere will stop. And let's not forget the insidious 'features' that Apple included in iTunes from the very beginning: the draconian measure of embedding user data into songs [1] they purchased (presumably as a deterrent against file-sharing) and restricting iTunes-purchased songs to be played the iPod only to the exclusion of all other digital music devices. [2] [3]

I can already hear the Apple faithful clamoring that all of these malfunctions are not the fault of Apple, but the fault of the record companies whose music Apple licenses. Even Steve Jobs wrote a much-cited article which put the blame for DRM at the feet of the RIAA constituents. This "They made me do it" excuse, as John Lech Johansen points out, is quite lame because Apple both refuses to open up FairPlay (its DRM scheme) to other companies and applies DRM to songs whose labels do not require their songs to be DRMed. Not to mention that other online music services have managed to operate without DRM just fine. Amazon can do it, Magnatune can do it, why not Apple? For a very good reason: Apple profits from the lock-in that DRM provides them (see Johansen's article for some telling quotes from Apple employees). Apple is today's main proponent and user of DRM at the same time that it makes overtures declaring its wish to end DRM once and for all. This is pure hypocrisy. If you hate DRM so much, Mr. Jobs, then put your money where your mouth is and take all the DRM (in its many shapes and forms) out of your products. Apparently Apple's CEO prefers cash in his pocket to shooting straight with the public.

Apple loves controlling how people use its products. Unfortunately for Apple, users often don't like being told exactly how they are supposed to use their computer, digital music player, et al. And people extremely dislike having unnecessary restrictions placed on products for which they paid a substantial amount. Limiting and sometimes rolling back the functionality of a company's products (and therefore angering customers and third-party developers) is not a winning strategy in the long term. Apple would do well to abandon it.

------------------
Footnotes
[1] I also responded to many posts on the linked Lifehacker discussion board. It's amazing to see how willing people are to let Apple dictate right and wrong to them. I also wrote about this on one of my other blogs.
[2] Makes me all the more embarrassed that I wrote a gushing article about iTunes for a school magazine when it came out as a freshman in college.
[3] I'm not going to even go into more reasons why you shouldn't install iTunes (especially on Windows). See iTunes spying on you and bloat and ugliness and what exactly does ituneshelper.exe do?. I'm sure there's more.

-------------------

Updates

Apple has continued its controlling ways with the iPhone App Store: they are actively restricting the applications that people can download, even after they have been released live to the public. An application that allows one to tether a PC to an iPhone connection is an example. BoxOffice is another. I'm sure there will be dozens more.
iPhone development is the biggest Apple money-maker I have ever seen. To simply write programs for an iPhone and get them uploaded to the iPhone store, you have to buy three things from Apple: 1) Some kind of Apple Computer (one to two grand) 2) an iPhone (two hundred to six hundred dollars, depending on when you bought it, to say nothing of the AT&T connection fees which account for the majority of cost) and 3) an enterprise or standard development license ($299 and $99, respectively). You can not develop on anything else besides an Mac. You can not develop without a license for the SDK. With all of these restrictions, you've just put around two to three thousand dollars into Apple's pockets.
With regards to the iPhone App Store, Apple is no longer banning only apps that they consider unacceptable in some way (see above) but also those that contain duplicate functionality. Not only that, but Apple has been notifying developers of their rejection with a letter that has a non-discolsure agreement on it -- according to Apple's lawyers, one can't even talk about the details of why he was rejected.
Here's an interesting reason to kill an iPhone app: it's using up too much bandwidth. But that's just what Apple did to streaming audio app CastCatcher.
Apple traffics in deceptive advertising. On the company's online store, supposed 'photos' of products are actually hand-generated drawings that misrepresent certain features of the products. Furthermore, Apple has been forced to pull an ad that grossly exaggerated the browsing speed of the iPhone.
Taking user oppression to the next level, Apple is now arguing that jail-breaking your phone is illegal under the DMCA.
The last place you would expect DRM to be is in headphones, but alas, Apple has put DRM in its headphones as well (false alarm)
Apple has outlawed Project Gutenberg from the iPhone because some of the books contain pornographic material. Could this get any more ridiculous? Yes, they are now censoring a dictionary.
Apple is working on technology to detect when a customer violates the warranty. Essentially, your device tells Apple what you have done with it when you bring it in for repair. Apple already has deployed liquid submersion detectors in some of its hardware.
Apple, not AT&T, blocked Google Voice on the iPhone
The iPhone secretly tracks your location
Apple has joined Facebook in shutting down Palestinian-related apps

Making Windows XP less Painful

2007-10-05T15:02:00.000-07:00

That is, making Windows XP more like Linux. Occasionally, I need to boot into Windows XP to use some application that won't run with WINE on Linux. Examples include QTFairUse and the drivers and software included with my Nokia E70 phone. Whenever I need to do this, however, I feel like I'm being suffocated by the constricting, feeble Windows environment and feel like doing the same to an unlucky bystander who just happens to bear the brunt of my Windows agitation. Fortunately, I finally sat down and figured out a few things to makeXP a bit more usable... a bit.

A (Somewhat) Usable Shell
The first thing I needed was a command line that wasn't dropped as a baby. Cygwin, which provides a UNIX-like environment on top of Windows, is perfect for this (another alternative is MinGW). The key to setting up Cygwin is configuring what applications you want installed in one of the setup menus. Of course, if you miss something on the initial setup, you can runCygwin's setup.exe again and reconfigure the included applications. Cygwin mounts the Windows C drive at /cygdrive/c. I would recommend browsing the Cygwin Properties menu and tweaking it (especially the fonts) to your preferences. Do this by clicking on the Cygwin icon in the upper left hand corner of the window and selecting Properties.

Cutting and pasting from Cygwin is a pain in the ass. To copy text, click the Cygwin icon and go to Edit->Mark. You now have a "Visual Block" - type selection box. I'm not sure if it's possible to select in a line-wrapping mode, the lack of which is also annoying. Hit Enter to copy the selected text. To paste text into theCygwin shell, click on the icon and hit Edit->Paste.

Install Software

Pidgin for instant messaging
Firefox and extensions (especially Google Toolbar)
VLC for video and audio of all kinds
Putty for SSH

Block Ads on the Web
Ads suck. Block them. Download a sample hosts file here and save it (in Notepad or something: Start->All Programs->Accessories->Notepad) to your file system. Now copy that file to c:\WINDOWS\system32\drivers\etc and name it HOSTS.MVP. If that's too much work for you, this page provides an automated tool to do it.

Add a Button to Show the Desktop
It's really aggravating if, in order to use the point-and-click Windows interface, you need to close all of the currently open windows to access the Desktop where a lot of clicking occurs. To do this, open Notepad and paste the following lines into it:

[Shell]
Command=2
IconFile=explorer.exe,3
[Taskbar]
Command=ToggleDesktop

Save the file as Show Desktop.scf. Right click on the task bar (that blue bar at the bottom of the screen) and select Toolbars->Quick Launch. Now drag the icon of the file you just created into the Quick Launch area. If all goes well, hitting that icon in the Quick Launch area should hide all the windows you have open and focus the desktop (or show them if you have them hidden).

Remove Unwanted Icons from the System Tray
msconfig is your friend. Good resources are here and here.

Remap keys
There is a pretty easy tool that allows you to remap your keys called SharpKeys. I'm a fan of Caps Lock to Escape remapping, myself.

Personalization
I recommend some BSOD wallpaper just to remind you from time to time that you're using an inferior operating system.

Nokia E70 for phone n00bs

2007-09-24T00:17:00.000-07:00

I just bought the Symbian S60 v3-based Nokia E70 mobile phone based mostly on the glowing recommendation of Maddox. Since this is my first somewhat advanced mobile phone (taking pictures on a phone is still a novelty for me), I've chronicled all of the hoops I needed to jump through here in a E70-for-dummies type guide. In addition to this guide, of course, I would highly suggest browsing the excellent user manual that ships with the phone to navigate using its myriad features.

Phone calls
I got the phone off of Amazon.com for $385. Since it is an unlocked phone, inserting the SIM card from my previous phone (a pretty old-school Nokia ... unsure about the model number) enabled the phone calling features.

Internet Service
I upgraded to the unlimited Internet service from my provider (AT&T) for an additional $20 a month by going into their (pitifully understaffed and crowded) store and asking for it. Selecting the globe icon on the main screen will launch the web browser. The web browser lacks a couple of things that you would expect to find on a regular browser, namely support for animated gifs and flash movies.

Firmware update
Updating the firmware is probably the first thing you should do when you get your hands on the phone, as it will erase all stored data and potentially make previously installed applications unusable. The updates for the US edition are available from Nokia's site here. From reading some message boards, it appears that the main benefits of this update are that applications are faster and less memory-intensive. There have been a lot of reports of the phones bricking (those posts, however, are mostly about a year old), so don't say you haven't been warned! My phone, since I bought it recently, shipped with the newest firmware installed and I did not have to go through this step.

Installing applications
The E70 allows you to install applications on your phone, both from Nokia and from third parties. Once they install, you can find it under Menu->Installations (Menu is the key both in portrait and landscape mode with the blue icon and nothing else on it).

Google Maps
Google provides a Maps application for Symbian-based phones. In the E70's web browser, navigate to maps.google.com and accept the web site's offer to install the application. One of the confusing things about the application is that the labels that tell you how to control Google Maps via the buttons on your phone do not adjust to fit landscape mode. Press the left/bottom selection key for the menu, the middle joystick to zoom in and the top/right selection key to zoom out.

SSH
There is a SSH client called Putty for Symbian phones. To install it, you'll need to download the application and then install it via the Nokia PC suite, which makes installation pretty easy (on a Windows PC, that is -- Ubuntu 7.04 didn't recognize the phone as anything: camera, storage, phone, or otherwise). You must enable self-signing applications to install themselves via the directions here. Installing fonts for Putty might also make your experience with it more pleasant -- get them here. Unfortunately, there's often a lot of lag between when you type something and when you see the output. Check out the options menu to send special characters (Esc comes in handy for using vi!).

YouTube
YouTube has a website for mobile users that your E70 is automatically redirected to when trying to access YouTube. To view videos, you must first set the default access point in Menu->Media->RealPlayer->Options->Settings->Connection->Network to whatever connection you want the video delivered over. Then click away on your favorite video. Or at least your favorite video among those offered -- YouTube only provides a limited subset of all of its videos for viewing on its mobile site ("Only a portion of YouTube videos are available on mobile at this time. We are working hard to bring you more!"). The video quality is pretty good.

Transferring Phone Numbers
AT&T claims that they can transfer your phone numbers if you have all of your phone numbers saved to the phone's SIM card. Apparently they have some device at the AT&T store that does this and they will do it for free. I have a hard timebeliving AT&T (a.k.a. 'Nickel and Dimeing you for Every Minor Service to Protect our Failing Business Model') would offer anything for free, but that's what they claim.

Text Messages
My phone is able to receive text messages but not send them. This is because the phone ships without any message center specified, which means the phone has no idea where to send the messages to get them to their final destination. To change this, go to Menu->Messages->Options->Settings->Text message->Message centers->Message center->Msg. center number and change that number to whatever your message center's number is (reference). Mine was +13123149810. Alternatively, there are services like text140 that allow you to send text messages to a phone from a computer, but it lacks a lot of the convenience of sending a text message from your phone. Using this message will also not charge you for sending text messages (you're just accessing the 'unlimited' Internet, right?).

AT&T Phone Support
This is an arduous process I suspect I'll be going through a few more times, so here's what you need to do to talk to an actual person on AT&T phone support (thePalo Alto store can be reached at (650) 617-8931):

1 to continue in english
3 for an existing customer
2 for all other information
Enter phone number in question
0 to speak to customer service
0 (again) to speak to customer service
2 to not participate in a survey

Incidentally, once you start talking to a person and not a machine, they have been quite helpful. Be prepared to give the last four digits of the social security number on the account if you want to make any changes.

Using WLAN Access Points for Web Access
I've found that using WLAN Access Points is one of the weaker features of the E70. Connections fail for every reason, whether it be the WPA authentication screwing up, the phone dropping connections or just failing to connect to an access point that is literally a foot away. If you can get them to work, however, they're a lot faster than the cellular network. Use Menu->Connect->Conn. Mgr.->Availab. WLAN->Options->Define Access Point to put an access point that your phone can currently see into a permanent list of access points. Use Menu->Tools->Settings->Connection->Access points to access this list and configure the details on each point (encryption, hidden network, SSID, etc.). To select the connection to use upon each launch of Web, select Web->Options->Settings->Access point->Always ask.

Bluetooth
The Bluetooth menu can be accessed via Menu->Connect->Bluetooth. To view paired devices, move the joystick to the right.

Camera
The resolution on the camera is pretty bad, and the phone doesn't come with a flash, which drastically limits the utility of the camera. Getting photos on and off the phone, however, is pretty easy since you can access the phone file system via Windows Explorer just like any other series of directories.

Gmail and Google Calendar
Both of these Google services provide very convenient mobile web interfaces. Perfect for someone who already has all of their data in El Goog's dirty paws.

Python
Snakes on a phone! Most of the documentation for S60 Python for Nokia phones is on the wiki. To install, follow the instructions here and download the appropriate things from the Sourceforge page. Now you've got a Python interpreter (and more) on your phone... what to do with it? Well, you could install a variety of applications, for one. For help, the Nokia-supported Python forum is the definitive place for answers. There's also an IRC channel, #pys60 on freenode. There's some additional documentation for developers.

Other useful things to know

Cut and paste works just like on any other PC: Ctrl+X, Ctrl+V, Ctrl+C
Press and hold Menu to get a list of running applications
If you're in an app and choose the "Exit" softkey (or use the task-manager shortcut above) that will actually shut down that application, and take you to whatever you were doing before. But if you press the red end (hang-up) key to jump to the home screen, (or switch to another application,) the app stays running in the background.

Links

Nokia E70 United States support
Tips and Tricks for Nokia E70
Engadget thread about tips and tricks for S60 phones
Codes to get low-level phone information (haven't tried these myself)

VIM split windows and macros quick reference

2007-08-26T14:05:00.001-07:00

ex commands for split windows

:sp filename create a new split with filename
:vsp filename create a new vertical split with filename
:new create a new split with an empty buffer
:vne create a new vertical split with an empty buffer
:wa save all open files
:wqa save and quit all open files

Ctrl-W related commands for split windows

C-w h moves split to the left of current split
C-w j moves split below the current split
C-w k moves split above the current split
C-w l moves split to the right of the current split
C-w + increase the current split by one line (or, prefix this command with a number to do that many line increases)
C-w - decrease the current split by one line (same addendum as above)
C-w < decrease the current split width
C-w > increase the current split width
C-w _ Maximize the current split

Split windows in the .vimrc (taken from this page ... also see the same author's excellent Efficient Editing with VIM )

map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it
set wmh = 0 " allow splits with 0 lines open

Macros

q<register> start recording and save the macro in the specified register (use a lowercase letter, for example)
q (again) end recording
@<register> playback the macro

Screen

2007-08-26T13:10:00.000-07:00

Screen is an awesome UNIX utility that allows you to continue running commands on a remote system even after you have logged out! Magic! Here is a nice "screen for dummies" tutorial with some links to related sites at the bottom.

On the command line

screen (will spew some text and then give a prompt) start a screen session
screen -r reattach to screen session
screen -d detach an attached screen session (good for logging in after network cuts out)

Links

Tips

Launch screen with an alias such as alias screen='TERM=screen screen' in .bashrc to avoid your backspaces getting mangled
There is a ~/.screenrc file with which you can customize screen

My .screenrc


# don't mangle backspaces
termcapinfo xtermc kD=\E[3~

# no startup message
startup_message off

# set shell that launches
shell /bin/bash

# lots of scrollback
defscrollback 10000

# ignore case when searching in copy mode
ignorecase on

# make it obvious we're within screen
hardstatus string "screen%n - %h"

# vim-style bindings
bind k focus up   # cycle to the above split
bind j focus down # cycle to the below split
bind l next       # cycle to the next split
bind h prev       # cycle to the previous split
bind n split      # create a new split
bind ^N split     # create a new split
bind q remove     # remove the split
bind + resize +3  # increase split three lines
bind - resize -3  # decrease split three lines
bind _ resize max # maximize current split
bind = resize =   # make all splits equal size
bind g info       # display info on the split
bind ^G info      # display info on the split

# other (some defaults, just to remind myself they exist)
bind Q only       # get rid of all other splits but the current one
bind M monitor    # toggle monitoring of the current window
bind t title      # enter a title for this shell
bind a other      # go to most recently viewed other split
bind \' windowlist -b # go to window list
bind x kill       # close the current shell
bind ^X kill      # close the current shell
bind \\ quit      # close all shells

Things I learned at DEFCON 15

2007-08-06T13:56:00.000-07:00

I just went to DEFCON in Las Vegas for the first time this year and it was a blast. Tons of smart people, cool hacks, great speakers, fun activities and stuff to learn. Here's a brief summary of some of the things I picked up this past weekend (in no particular order):

Virtually any lock that you have bought can be picked, easily. Even the (supposedly) secure Medeco locks that they have on the white house and other high-security areas can be broken by a 12-year-old girl. For more, see the in.security.org blog entry (by the guys that made this presentation, Marc Weber Tobias and Matt Fiddler) here. Gun locks, also, are completely trivial for a kid to break as this amazing video proves (more on this subject: blog, report). Also from the talk: most hotel safes are terribly insecure. To break those that use your credit card to open/lock them, the code of the master card that unlocks all of them in stored in memory inside the safe. All you have to do is open the safe (legitimately) and then dump that code and write it to a card to have a "master card" for all the safes in the hotel... yikes!
SCADA systems, the systems that run critical infrastructure such as water treatment plants, electrical grids and nuclear power plants have an overwhelming number of vulnerabilities. Scary. Nationwide emergency alert systems also have relatively easy attack vectors (from the talk by Ganesh Devarajan).
There is always a motive for Internet crimes, just like ones off the net. The motive for the attack on the Dolphin Stadium website prior to the 2007 Super Bowl was particularly interesting because the attack, which was made on several other lower-profile websites, was linked to a Chinese syndicate that wanted users' World of Warcraft online credentials to acquire additional WoW gold! This is the first attack of its kind. (From the Internet Wars 2007 panel)
One interesting way of getting malware on a user's computer is to set up a website and register malware to be downloaded on that website as a codec with Microsoft. That way, when a victim visits the attacker's site, he is prompted with a "Additional codecs are required to display content on this page... would you like to download them?" message. If he does, he downloads the malware onto his computer. (From the Internet Wars 2007 panel)
If you're a NBC Dateline reporter, don't refuse a press pass and then try to brew up some sensationalist report about hackers at DEFCON. You will get owned.
Even the badges at the conference were a hack unto themselves (video). How cool is that? And there is a rap about the conference, too!
If you want to start up an advocacy organization, the most important thing you can do is have a paid person sitting by the phone who knows what expert to contact (and how to contact them) when called by reporters. PledgeBank is a good site to help organize the effort to raise this money. (Funny quote from the same Danny O'Brien talk: "[Imitating a typical clueless reporter that calls EFF] People will call and say, 'I heard that such-and-such technology can make someone's penis fall off; can this be done with Ruby on Rails?' ... In this way, EFF operates as a clearinghouse for idiots")
Sam Bowne offers a class at San Francisco City College called "Ethical Hacking and Network Defense" ... sweet! (He recommends Hack this Site and Irongeek.com as good references)
Brendan O'Connor gave a really good talk about the extra layer of "security" that banks are now layering onto their authentication services and how this layer improves neither the user's privacy nor his security -- in fact, it may be degrading both. Bruce Potter similarly lambasted "Defense in Depth" for being a lame attempt at covering up bad code with extra layers of "security."
Bruce Potter of the Shmoo Group gave a talk about how the dynamics of vulnerability disclosure are changing: instead of informing the vendor of the weakness in their product, hackers are often now selling information about those vulnerabilities to third parties (who may or may not have good intentions). Potter called for a discussion on whether this was ethical and/or good for the security community.
Thomas J. Holt presented on the economics and dynamics of the malware marketplace. This market is mainly organized around forms based in Russia and Eastern Europe where sellers start forum threads advertising their product. Moderators of the forum then test the software to see if it does what the seller says it does and then give their opinion of it on the same thread. If the opinion is good, and the seller has a good reputation, then buyers start asking questions about the software and perhaps buy it. The final step is the reviews of the software by the buyers which attest to the malware's quality (or lack therof). Some sell not only hacking tools like Pinch, Nuclear Grabber and PG Universal Grabber but the data obtained by using these hacking tools. Some forums maintain lists of sellers who are "rippers" -- rip off artists -- and blacklist them.
AgentX gave a whirlwind talk about "22 Things that Keep me up at Night." Among these: "Shrinking the Gap" (based off of ideas of Thomas Barnett), low cost pervasive bandwidth, open source warfare (the terrorists are all sharing techniques -- why aren't we?), the security industrial complex (complacency = bad), homogeneity of the hacker (all white males, but starting to change -- he notes good progress at DEFCON 15), why aren't you encrypting all your communications? (because the NSA is listening)
Mike Murray had some interesting things to say about social engineering/NLP/hypnotism. Such as: you are four times more likely to follow a command following a non-grammatical sentence, you wouldn't do something hypnotized that you wouldn't do in the first place (such as kill a commanding officer -- study), tag questions ("won't you?" "right?") are convincing, confusion is the key!, the right tone can be categorized as "artfully vague," stories are more powerful than reasoning, questions are more powerful than statements, Milton Erikson = best hypnotist ever
I missed Bruce Schneier's talk, but someone posted a video of it here. The whole presentation is interesting, but probably the most stimulating is his discussion of how you DON'T need an ID to fly on an airplane; you just get a little extra screening.

Other notable moments/events include the Guitar Hero contest, Brew Wars, Phreaking Challenge, Capture the Flag, Hacker Jeopardy, TCP/IP Drinking Game, Lockpicking Village, automated airgun target contest, the Wall of Sheep and the Wireless Village. Definitely going next year!

UPDATE:
Videos of all of the presentations have been posted here.

My .vimrc file

2007-08-02T20:58:00.000-07:00

Thanks to Evan Klitzke and others at Yelp; a lot of this is ripped from their .vimrc's. All hail vim!


set modeline   " Respect other people's options (when a modeline is present)
set encoding=utf-8 " Use UTF-8 (8-bit variable width Unicode)

" indenting options
set autoindent  " Keep the indent level when hitting Return
set smartindent  " Use smart indenting (mostly useful for C/C++ files)
set cindent  " Don't indent Python really poorly
set tabstop=4  " Make tabs appear four spaces wide (default is 8 spaces)
set shiftwidth=4
set noexpandtab  " Use hard tabs please! Watch out for files with soft tabs
" that don't have a modeline present, especially Python files.
set fo=tcoqan  " Options for formatting text (i.e. for use with gq)

" UI stuff
set showmatch  " Show matching parens as they come up
set ruler  " Show the column number in the status bar
set incsearch  " Find as you type
set lz   " Don't redraw the screen in the middle of executing macros
set nohlsearch  " Highlighting search terms is _really_ annoying
behave xterm  " Just in case...
set wrap  " Display files with word wrap, but don't actually insert newlines
" (Set by default)... so lines don't go off edge
set lbr   " Wrap only at word boundaries (default is at any character)
syntax enable  " Who wouldn't want syntax highlighting?

" Only allow folds that are manually set
set foldmethod=manual

" Make our folds look like how they were when we exited vim last time.
set viminfo=s1,<1024,'1024,/1024,:1024,@1024,c,f1,%,n~/.viminfo
autocmd BufReadPost *
\ if expand("<afile>:p:h") !=? $TEMP |
\   if line("'\"") > 0 && line("'\"") <= line("$") |
\     exe "normal g`\"" |
\     let b:doopenfold = 1 |
\   endif |
\ endif
" Need to postpone using "zv" until after reading the modelines
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\   unlet b:doopenfold |
\   exe "normal zv" |
\ endif

set nocompatible            " Don't act like SysV vi
set shell=/bin/bash         " Use this shell to execute commands
filetype plugin indent on   " ability to run different vimrcs on different filetypes
set autoread                " When a file has been detected to have been changed outside of Vim and
              " it has not been changed inside of Vim, automatically read it again.
set mouse=a                 " Have mouse operate in all modes -- disable mouse by holding the shift key
set ttyfast                 " Tell vim to optimize for a fast terminal; will be on by
              " default if your $TERM is xterm or screen, but could be
              " turned off if you use a weird terminal (e.g. 'screen-bce').
              " Set 'nottyfast' for slow SSH connections.
set history=50
set showmode                    " show what mode (e.g. INSERT) you're in
set showcmd                     " Show (partial) command in status line
set backspace=indent,eol,start  " explicitly makes the backspace work when at the beginning of a line
nmap <C-N><C-N> :set invnumber <CR> " Press Ctrl-N twice to toggle line numbers in the left margin

"splits
set wmh=0 " allow splits with 0 lines open
map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it

" cycling through tabs
map <C-H> <C-PageUp>
map <C-L> <C-PageDown>
imap <C-H> <C-PageUp>
imap <C-L> <C-PageDown>

set title           " sets title of terminal in menubar to that of file currently being edited
set ignorecase      " perform case-insensitive searches...
set smartcase       " ... unless there are capitalized characters in the search pattern
set mousehide       " hide the mouse when editing
set exrc            " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set bg=light        "this almost always looks better -- take this out or set it to dark if things look funny

" for common typos
command! Q quit
command! W write
command! Wq wq
cabbrev Set <c-r>=(getcmdtype()==':' && getcmdpos()==1 ? 'set' : 'Set')<CR>

Or, say you want soft tabs instead of hard ones (see converting tabs to spaces):

set expandtab     "soft tabs
set tabstop=2     "set tabstops
set shiftwidth=2  "set shiftwidth
set softtabstop=2 "delete this many spaces upon deleting soft tab

More stuff from Rajiv:


set directory=/var/tmp,/tmp " put all the backup stuff in tmp directories
set shortmess+=A " no swap file complaints
set nobackup " don't litter directories with swap files

" move across rows, not lines
noremap j gj 
noremap k gk
noremap gk k
noremap gj j

set display=lastline " ??????????????
syntax sync fromstart " always render syntax from the beginning of the file
au! BufRead,BufNewFile * lcd %:p:h " cd into the directory of the file you're editing
set vb t_vb= " no visual bell

Configuring WPA manually

2007-06-08T17:18:00.000-07:00

Configuring WPA is Ubuntu is no problem if NetworkManager works flawlessly. However, sometimes it doesn't. Here's how to configure WPA from the command line:

Get wpa supplicant sudo aptitude install wpasupplicant
Run wpa_passphrase which reads stdin and outputs the PSK that you'll need
Create a configuration file, say /etc/wpa_supplicant.conf, with the the text following this numbered list
sudo wpa_supplicant -B -D wext -i eth0 -c /etc/wpa_supplicant.conf (B for background, D for driver (wext is a generic linux driver, ipw2100 does not work), i for interface, c for path to conf file)
You should be connected at this point. Run dhclient to request an IP address from the server. ping something to make sure you can reach the outside.
(optional) This process can be automated on startup by editing the /etc/network/interfaces file appropriately

ctrl_interface=/var/run/wpa_supplicant

network={
ssid="My Network"
key_mgmt=WPA-PSK
psk=<generated passphrase>
}

As always, look at the man pages for more details: man wpa_supplicant and man wpa_supplicant.conf and man wpa_passphrase

dpkg's --force-architecture option is your friend on amd64 systems

2007-06-03T12:55:00.000-07:00

For example, to install picasa for linux

sudo dpkg --force-architecture -i picasa*.deb (-i for install)

If you don't do this, you'll get a warning about the package being i386-based and therefore unusable on your system.

HOWTO format and mount an external hard disk

2007-05-27T12:26:00.000-07:00

When I initially tried to back up my music library onto my Windows-created FAT32 external hard drive, the terminal threw a bunch of "Invalid argument" errors and failed to copy those respective files. It turns out that windows-formatted drives can not have certain characters in their filenames ( \ / : * ? < > | ), so I decided to reformat the drive in a Linux-friendly fashion.

Instructions

Connect the target drive to your computer (let's assume it's recognized as /dev/sdb1)
Reformat it using the ext3 file system (Linux default) by either...

sudo mkfs.ext3 /dev/sdb1 (lots of options with this tool... read the manpage )
Using gparted (install with sudo aptitude install gparted), a GUI-based tool

Create a mount point sudo mkdir /external
Mount the drive sudo mount /dev/sdb1 /external
Use the drive!
(optional) If you want the drive to be recognized in a certain way upon boot, edit the /etc/fstab file to your liking (Ubuntu 7.04 recognizes it anways)
(optional) To unmount the drive sudo umount /dev/sdb1

Useful commands

sudo fdisk -l List partition tables for all devices (mounted or not)
df -h Human-readable output of system disk space and usage

Tips
hd* = IDE drive
sd* = SATA drive

Information used from these sources

HOWTO organize a music library of 10,000 songs

2007-05-26T21:15:00.000-07:00

Several days ago, I set out on an epic quest to organize my music library of roughly 10,000 songs. It's a daunting task, and one I've put off for a while, because my files are in several different places in several different formats with several different ID3 tag/naming standards. And there are more problems too: identifying mislabeled files I got from other places, weeding out corrupted files, etc, etc etc. Suffice it to say that it's been an intense few days with me in a zen-like state of focus trying to get everything in its right place. So here's the short version of how I did it.

The first (major) problem I faced was the massive swaths of files that had incorrect ID3 tags. For this problem I needed a competent ID3 tagger. About a year ago I asked on ubuntu-users what the list's tool of choice was. Suggestions ranged from Quod Libet to Foobar2000 on WINE (outstanding reviews: "best program ever" and "the only program i miss since switching to ubuntu") to gtkpod to EasyTag (mixed reviews: "tolerable" to "great") to id3v2 (command line based) to normal music players like amarok and banshee. There is also a Python module for accessing and editing ID3 tags, if I felt like scripting some things myself.

Eventually I settled on Picard, the tagger that Musicbrainz (a great music metadata site) recommends. Thanks to Ubuntu and the Picard team, installation is as simple as adding deb http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/ubuntu edgy musicbrainz to /etc/apt/sources.list (even if you're not running edgy), importing the PGP key with wget http://ftp.musicbrainz.org/pub/musicbrainz/users/luks/public.key -O- | sudo apt-key add - and then a sudo aptitude update and sudo aptitude install picard. There are only a few limited operations Picard can perform, but they are very powerful. Picard interacts with the Musicbranz database to identify tracks in an album-based scheme. It correctly identified and summarily tagged more than 95% of the CDs I threw at it, and I listen to some pretty obscure/out there music. The ones it couldn't recognize were mostly tracks that didn't have a lot of existing context to go with them, and although I don't know exactly what information Picard looks at to determine the ID3 tags, I suspect that was the problem. There are a few problems with Picard, however. First of all, the UI sucks. Second, there are very limited capabilities. I would like to see, say, options that move files around on the file system depending upon how Picard tags (or fails to tag) them. The albums that have their ID3 tags written by Picard don't seem to be organized in any meaningful way, except in the current running instance of the program, where they are added to the bottom of the list (which is also annoying because of the need to keep scrolling to the top to add files to tag and then to the bottom to click and drag the files it missed and back again). Increased automation in the program would be nice, too, as I found myself performing the same operations over and over again. There are also minor issues like it opening up a new browser tab on each database lookup (leading me to have firefox open with ~100 or more tabs at some points). Fortunately for the future Picard user, I saw most of these issued acknowledged and marked for future improvement in the documentation.

The second (major) task ahead of me was conversion of all of my files into the near-universal mp3 format (yes, I know I should be gunning for ogg, but you can't use those files on your iPod... well, without the free firmware you can load onto it... another day, another day). For m4a files, I used the script found at the end of this thread which happily copied my ID3 tags as well. For ogg files, I would need something more general. At first glance, SoX seemed like a reasonable tool, but it needs to be recompiled for mp3 support. I also found some other scripts, but those didn't copy the ID3 tags, which made them unacceptable for my purposes. I asked in the freenode #ubuntu channel regarding my quandary, and someone mentioned Soundconverter as an option.

Soundconverter is a simple tool, but gets the job done. I installed it with a simple sudo aptitude install soundconverter. It provides a nice drag-and-drop interface that doesn't require knowledge of ID3 tag layouts or encoding/decoding details. Most of the options are located in the Edit->Preferences menu; the rest of the program is idiot-proof. Just drag the files, and hit Convert. It does do some annoying things, like saving files in URL-encoded names (example: files in folder "Cannibal Corpse" with the option to save in the same folder on were saved in a newly created folder, "Cannibal%20Corpse"), but generally works well.

Things were a bit more complicated than this, of course. I was making nightly backups of my changes to a local FTP server, wrote up some Python scripts to assist me in my task of keeping track of where everything was going and manually edited some outlying tags in amarok -- but now my music library is (almost) officially organized. Success! Now to just incorporate all of my friends' libraries...

Some cool links I found along the way

Music File Processing on Linux describes command-line techniques for some tasks and gives an overview of available software
MP3 Encoders Comparison
My previous post on file extensions, compression, and conversion

Some relevant command line tricks

find -type f ! -iregex '$.*.flac\|.*.ogg\|.*.m4a\|.*.mp3$' -delete -or -empty -delete

Feisty Fawn customization

2007-04-20T12:37:00.000-07:00

Here are the things I did after I installed Feisty Fawn Ubuntu 7.04 off of the alternate install CD on my Compaq X1000 laptop/other systems.

If on an AMD64 processor system

Follow these instructions (and run the script) to get mplayer, flash and java support for firefox. Update all launchers and shortcuts to launch firefox32 instead of firefox
Skip the remaining instructions that are italicized

Install add-on packages (by the way, all except the first can be done in one command... just append all the packages you want to install after "install")

Edit /etc/apt/sources.list following these instructions to add extra repositories
Install full Vim support (without this, no help, syntax highlighting, etc.) sudo aptitude install vim-full
Install Flash sudo aptitude install flashplugin-nonfree
Install media codecs sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse w32codecs
Install Amarok sudo aptitude install amarok
Install Java sudo aptitude install sun-java6-jre sun-java6-plugin sun-java6-fonts
Install Azureus sudo aptitude install azureus (or, alternatively, ktorrent)
Enable DVD support
Possible user programs: xmms, xchat, mplayer plugin, vlc, skype, picasa, eclipse, google earth, beagle, clipboard manager
Security programs: sudo aptitude install nmap kismet wireshark airsnort aircrack-ng john
Install Beryl (nVidia) (I also find it convinient to add a sessions entry for beryl-manager so the stone shows up in the tray)

Trick out Firefox

Add desirable extensions
Customize Google toolbar
Edit /etc/hosts to block ads
Install multimedia plugins
Change default colors Edit->Preferences->Content->Colors
Set a master password Edit->Preferences->Security->Use a master password

Trick out the Terminal (and partners in crime)

Either create a new profile or edit the current profile
Disable General->Show menubar by default in new terminals
Set Colors to aesthetically pleasing values
Set Effects->Transparent background and set the bar about 2/3 of the way to Maximum
Set Scrolling->Scrollback to 2000 lines
Edit ~/.bashrc
Edit ~/.vimrc
Put the line set editing-mode vi in ~/.inputrc (the configuration file for readline... this also makes the set -o vi in the .bashrc redundant)

Trick out GNOME

Fix the Right Alt key System->Preferences->Keyboard->Layout Options->Third level choosers->Press Menu Key to Choose 3rd Level
Change the Desktop Background by right clicking on it (good backgrounds here (OSX Tiger) and here (fractals) and at DeviantArt)
Map the Windows keys to do something cool in System->Preferences->Keyboard Shortcuts (I think "Run a terminal" and "Hide all windows and focus desktop" are good candidates)
Enable visual system beep System->Preferences->Sound->System Beep->Flash Window Titlebar
Make the panels transparent Right Click on Panel->Properties->Background->Solid Color and adjust the slider appropriately (about halfway looks good)
Add the system monitor to the panel Right Click on Panel->Add to Panel->System & Hardware->System Monitor->Add to Panel (I enable all but "Load"... wtf is load?)
Disable launching of unnecessary programs at startup (such as Evolution Alarm Notifier) Preferences->Sessions->Startup Programs
Save your session in a desireable state: gnome-session-save (don't use the automatic option in Prefs->Sessions... it causes problems!)
Put the panels in a desirable place by dragging them across the screen
Get 4 workspaces Right Click on Workspaces on Panel->Preferences->Number of Workspaces
Disable Rhythmbox launching upon iPod detection (Amarok rulez!) System->Preferences->Removable Drives and Media->Multimedia->Uncheck "Play Music Files when Connected" and also Uncheck Storage->"Browse removable media when inserted"
Enable the text-based location bar in Nautilus (in favor of the button-based one) by hitting the notepad button on the top left in a Nautilus window

If on a laptop...

Configure Wireless Networks
In /etc/default/acpi-support set ENABLE_LAPTOP_MODE=true

So what's so different about Feisty (as opposed to Edgy)?

All the latest stable versions of software in the repos, obviously
GNOME network manager manages wireless devices by default (and keyring support to boot)
Fool-proof support for installing restricted drivers, media codecs, etc.
Nice command line assistance if you try and run software that's not installed (prints a "use this repository and this command to get the software"-type message)
Easy enabling of "Desktop Effects"
And more

iPod + Amarok

2007-04-19T22:32:00.000-07:00

Using your iPod with Ubuntu is disgustingly easy. As a Linux user, I was almost disappointed... no config files to edit? No dependencies to chase down? No bugs to fix? Nope, Ubuntu and the Amarok music player make everything simple. Here, I will detail how to use Amarok with your iPod on an Ubuntu system. I'm using Ubuntu Feisty Fawn 7.04 (just installed today!), a fourth-generation iPod and Amarok version 1.4.5.

First install Amarok with sudo aptitude install amarok (and with aptitude, not apt-get). If you want Amarok to be useful with most file formats (such as mp3) it would be a good idea to install proprietary codecs as well: sudo aptitude install gstreamer0.10-ffmpeg gstreamer0.10-gl gstreamer0.10-plugins-base gstreamer0.10-plugins-good gstreamer0.10-plugins-bad gstreamer0.10-plugins-bad-multiverse gstreamer0.10-plugins-ugly gstreamer0.10-plugins-ugly-multiverse libxine-extracodecs w32codecs

Now that those monster installs are finished, you can go ahead and connect your iPod. If it automatically shows up on your desktop as a little iPod icon, great! If not, check out this thread for more detailed info. When I popped in my iPod, Amarok did me one better and recognized the device, popping up a little dialog asking me what kind of player it was. When this happens, you should select Apple iPod Media Device in the drop-down box. There are a few more options you can set regarding the device -- check out Settings->Configure Amarok->Media Devices in the Amarok toolbar menu. (If this explanation was too complicated for you, there's a guide with nice pictures here describing a slightly different way of going about things.)

To add files to the iPod, go to the Collection tab on the left and highlight whatever you want to transfer to the iPod in the left frame. Right click on it and select Transfer to Media Device. Then go to the Devices tab (with the little iPod logo) and hit Transfer at the top to transfer your files. When you're done transferring files, hit Disconnect at the top. If you're lucky, Amarok will unmount the device for you. If not, you'll have to right click on the desktop icon of the iPod and unmount it that way. I asked in the Amarok IRC channel if it was possible to sync the iPod and apparently it's not... for now, just delete everything off of the iPod and put the new stuff back on, as time consuming as that may be.

There are some other things you can do in Amarok to tweak the settings for the iPod. In the Devices tab under the (usually hidden) iPod menu you can set the model of your iPod. This is used for advanced support such as photo and video copying.

It's annoying that Rhythmbox, the default GNOME music application, launches whenever you connect your iPod. To disable this, go to Preferences->Removable Drives and Media->Multimedia->Portable Music Players and disable Play music files when connected. Just doing this will make a window open up in Nautilus every time you insert a removable drive, including an iPod. To turn this off, go to the Storage tab and uncheck Browse removable media when inserted.

Links

My .exrc file (vi/m) and then some

2007-04-06T21:24:00.000-07:00

What follows is my .exrc file for various versions of vi, and some more related stuff. I have yet to look through the vim documentation exhasutively to see if there are any more useful options -- this is just stuff I have gleaned from random places on the Internet and from friends.

Plain vanilla vi (as on Solaris)
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns set showmatch " show matches of ()'s, {}'s, and []'s
set showmode " show what mode (e.g. INSERT) you're in
set autoindent " indent to the extent of the previous line

Vim:
syntax enable " enables syntax highlighting
set nu " show line numbers
set ts=4 " tabs take up four columns
set sw=4 " set shiftwidth (>> or <<) to 4 columns
set title " sets title of terminal in menubar to that of file currently being edited
set expandtab " tabs aren't really tabs, but many spaces
set softtabstop=4 " delete four spaces when you delete a "tab"
set ignorecase " perform case-insensitive searches...
set smartcase " ... unless there are capitalized characters in the search pattern
set mousehide " hide the mouse when editing
set exrc " eliminates the possibility of another .exrc getting confused with mine on multi-user systems
set backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
set bg=light "this almost always looks better -- take this out or set it to dark if things look funny
set nohlsearch "no annoying search highlighting

" for common typos
command! Q quit
command! W write
command! Wq wq

Vim Python
set autoindent " indent to the extent of the previous line

Vim C/C++/Java
set cindent " indenting for C-style languages
set showmatch " show matches of ()'s, {}'s, and []'s

For Text (Vim 7.0 only)
setlocal spell spelllang=en_us " English(US) spellcheck on (Vim 7.x only)
set tw=0 wrap linebreak " don't split words between lines, only wrap at end of word
"lots of abbreviations (varies depending on the subject material)
ab STIA Science, Technology and International Affairs

How to produce nicely formatted C code in Vim (From p.221 of Learning the vi editor):
set nocp incsearch
set cinoptions=:0,p0,t0
set cinwords=if,else,while,do,for,switch,case
set formatoptions=tcqr
set cindent
syntax on
source ~/.exrc

Options set when vim 6.4/7 starts on my system (Ubuntu Linux):
helplang=en " language to display help in
ruler " show row and column in bottom right
ttyfast " indicates a fast terminal connection
viminfo='20,"50 " configuration info
history=50 " number of : commands and previous search patters to be remembered
scroll=32 " number of lines to scroll with CRTL-U and CTRL-D commands
ttymouse=xterm2 " mouse codes
backspace=indent,eol,start "explicitly makes the backspace work when at the beginning of a line
fileencodings=ucs-bom,utf-8,latin1 " list of character encodings considered when starting to edit a file
printoptions=paper:letter " controls format and output of printing
runtimepath=~/.vim,/usr/share/vim/addons,/usr/share/vim/vimfiles,/usr/share/vim/vimcurrent,/usr/share/vim/vimfiles/after,/usr/share/vim/addons/after,~/.vim/after " list of directories which will be searched for runtime files
suffixes=.bak,~,.swp,.o,.info,.aux,.log,.dvi,.bbl,.blg,.brf,.cb,.ind,.idx,.ilg,.inx,.out,.toc " Files with these suffixes get a lower priority when multiple files match a wildcard.

Cool stuff that I have seen various places but I didn't find particularly useful (maybe someone else will):
set list "replaces all EOLs and TABs with $ and ^I
"auto-complete of brackets
inoremap [ [] <left>
inoremap ( () <left>
inoremap { {} <left>
inoremap " "" <left>
inoremap ' '' <left>

To see the options vi(m) has enabled, enter :set
To get help with options in Vim, enter :help set or :set all for regular vi
Or just :help for the help contents... Vim has excellent documentation

Links

My .bashrc file

2007-04-06T20:42:00.000-07:00

export TERM="xterm-color" # xterm terminal emulation (with color -- be sure to put at top of file on Ubuntu)
export PYTHONSTARTUP="$HOME/.pythonrc.py" # python startup file
set -o vi # sets vi command-line editing mode
bind -m vi-command -r 'v' # so that every time you hit v in command mode, an editor doesn't launch
ulimit -c unlimited # dump core files, no matter how big they are
export EDITOR="vi" #use vi as the default editor for some commands, like "fc"

alias screen='TERM=screen screen' # workaround for screen backspace bug

alias d="date '+%r -- %A, %D'" # human-readable date output

alias ll="ls -lh" # long listing (with human-readable file sizes)
alias lr="ls -ltrh" # list by reverse modification time
alias la="ls -Ah" # list hidden files (except implied . and ..)

alias <name>="cd <important>" # get to important directories fast
alias <name>="ssh <login>@<important>" # log into oft-used remote machines fast


# prevent myself from doing stupid things with vi
function vi {
if [ ! -e "$1" ]; then
if [ "$1" == "" ]; then
vim
else
vim "$1"
fi
elif [ -d "$1" ]; then
cd "$1"
elif [[ -c "$1" || -b "$1" || -p "$1" || -S "$1" || ! -r "$1" ]]; then
file "$1"
else
vim "$1"
fi
}

The following I only enable on Solaris systems:
export PAGER="less" #uses less to view manpages instead of more

Links:

HOWTO: Customizing bash prompts

Common Bash Tasks

2007-04-04T14:28:00.000-07:00

I don't use Bash for much scripting. Usually, if I have to open up a file to write a script, it will be done in Python. However, the less I have to do that for mundane tasks that can be accomplished on the command line with Bash, the better. This entry may not have much in it now, but every time I do something cool/useful/time-saving in Bash from now on, it'll go on here.

Common Tasks

Rename *.foo files to *.bar files: for i in *.foo; do mv $i ${i%.foo}.bar; done (BashFAQ)
Rename foo.* files to bar.* files: for i in foo.*; do mv $i bar.${i#foo.}; done (also check out the perl regex-based rename Linux utility)
Do something to multiple arguments: for i in arg1 arg2 arg3; do echo $i; done
Print 0 through 9 on separate lines: for i in {0..9}; do echo $i; done
Flatten output onto one line: <Multi-line output> | xargs (10 habits)

Links

GNU Bash Reference Manual
Bash Manpage
BashFaq Bash FAQs and answers
Bash Pitfalls common mistakes
Advanced Bash Scripting Guide Everything you would want to know about Bash scripting
Bash Guide for Beginners A beginner's guide to Bash, some shell examples
10 Good UNIX Habits Advanced tips on the command line

Finding and Killing a Process in GNOME

2007-04-04T13:23:00.000-07:00

There are a few different ways to find and kill a process in Linux. Using the GNOME interface, you can call up the System Monitor (similar to the one on Windows XP) under System->Administration->System Monitor. Under the Processes tab, right clicking on a process will give you a variety of options: stop/end/kill/continue the process, change the priority (nice), display a memory map (pmap) and display open files (lsof -p <procID>).

Embedding the system monitor in a desktop panel is useful for constantly monitoring the system and quickly determining if and where there is a problem. Do this with Right Click on Panel->Add to Panel...->System & Hardware->System Monitor->Add. The display provides graphs of Processor, Memory, Network, Swap Space, Hard disk and Load activity (not quite sure what load is, perhaps some metric computed from all the others? I don't use it, personally).

On the command line, list processes with the ps command or ps -e for all users' processes. Pipe that through grep (i.e., ps -e | grep firefox) to find a process ID of the target process (if you know the exact name of the program, you can also use pidof firefox-bin). To kill a process, use the kill <procID> command (with the -9 flag if you want to kill it unconditionally). Alternatively, if the process uses a gui, you can use xkill to merely click on the gui to kill the corresponding process. There is also killall if you want to kill by process table name (i.e. killall firefox-bin).

pgrep and pkill are two very useful utilities, especially for dealing with processes with long command line invocations. pgrep -fl -u danny "ruby.*asdf" will print the process id and name of every process owned by danny whose command line invocation matches the passed in regular expression. The -f argument tells it to match against the full command line invocation instead of just the process name. pkill -fu dannyc "ruby.*asdf" will kill those same processes (for the life of me, I can't seem to coax similar behavior out of killall).

The top command provides an integrated way of finding and killing a process on the command line. If the process-to-be-killed is using a lot of system resources, it will appear towards the top of the display. Hit k and then the process ID to kill the process. Top comes with a lot of other functionality -- hit h for help.

UPDATE: I recommend using htop instead of top, which is top with less suck. It has lots of good features like color, mouse support, a more responsive interface, a tree-based process view, etc.

Google Toolbar 3 has made my life 1.7% better

2007-03-14T17:05:00.000-07:00

Google Toolbar Version 3 rules. It is a tiny little piece of technology that has made my life significantly better since I started using it. Linux users will have to go the extra mile (or in this case, inch) to make sure it works (nothing new...). See this thread and Google's toolbar for Linux page for instructions for older builds; apparently it appears to be playing nice now.

The coolest feature of the Toolbar is the ability to make custom search buttons. Going to any page with a search bar, right clicking, and selecting "Generate Custom Search" will create a nice little button on the Toolbar (corresponding to a site's favicon). Using this, you can search with the Toolbar as if you were on the page you generated a search for, the effect being that you never need to visit the page to use their search engine -- you can search from any arbitrary page! It's also useful for doing the same search (i.e. "Donnie Darko") over several sites you have a custom search (i.e. RottenTomatoes, IMDB, Wikipedia). If you don't know what buttons you want on your Toolbar, Google has provided a nice grab bag to choose from. My custom searches (as of right now) are:

Google Image Search
Google Groups (usenet)
Google News
Google Maps
Google Docs & Spreadsheets (online word processing + spreadsheets) (
Google Linux Search
Google Calendar
Gmail
Google Reader (RSS reader)
IsoHunt (torrents)
Newegg (hardware)
RottenTomatoes (movies, especially reviews)
IMDB (movies)
Gamespot (video games)
Amazon (everything)
YouTube (videos)
MusicBrainz (music metadata and info)
Wikipedia (online encyclopedia)
Yelp (local search and reviews)
Allmusic (music reviews and metadata)

Some more very useful features include enabling GMail to launch every time a mailto: link is clicked (Finally! No more surprise Evolution/Outlook/etc. "setup page" launches!), an icon indicating if you're signed in to your Google account, and some keyboard shortcuts (more would be nice... especially for accessing custom buttons) like Shift+Enter for an "I'm Feeling Lucky" search (keeping in the spirit of eliminating useless page visits). There are more features that I don't use (but many do) detailed on the features page. Some of these are pretty cutting edge and have generated a considerable amount of controversy, especially AutoLink. Others, such as the RSS subscribe and spellcheck, have already been implemented in Firefox 2. In any event, you should get it. Now.

A couple of things that could use improvement:

More keyboard shortcuts (as mentioned above)
Web spam reporting buttons, analogous to the "Report Spam" button that Gmail has... perhaps Google can build a database of splog, etc. sites to blacklist, or maybe work with other search engines to do so?
Sometimes the "Generate Custom Search..." option doesn't work... amazingly, Blogger is one of those examples... when trying to do a custom search on a particular blog, I get this (wtf?): "Information Unavailable The custom button cannot be installed. Custom button values out of range."

Which Ubuntu version am I using, again?

2007-03-07T22:10:00.000-08:00

On the command line: lsb_release -a

Alternatively, look in one of these two files for the answer (links to sources where I found info): /etc/lsb-release or /etc/issue

Also, find out if your install is 64-bit via uname -m (thread1, thread2).

Yahoo! Pipes and its Shortcomings

2007-03-07T16:31:00.000-08:00

For the past couple of days, I have been fooling around with Yahoo! Pipes. It's a tool and an interface for aggregating and/or manipulating RSS feeds. It's an interesting experiment and has been receiving some hype, most notably from Tim O'Reilly who called it a "milestone in the history of the internet." So how useful/powerful is it? My assessment is: right now, not very. There are several problems that are keeping it from being a useful (or sometimes even usable) tool at present.

The first problem is it is SLOW. When manipulating the first one or two modules, things run fine, even to the point of web 2.0 AJAXy nirvana. The scrolling is snappy, the debugger (which is a well-thought out touch) and DHTML animations look nice, and the interface is responsive. However, after that things start crawling. And after 10 or so modules, it slows to a why-am-I-even-bothering speed (and I'm on a pretty damn fast MIT connection right now with a capable PC).

The second problem are the all-too-frequent site outages. A recent blog post by the development team humorously noted: "Our apologies for the unscheduled downtime this evening. As a reminder, todays (sic) post was sponsored by the letters “b” “e” “t” and “a”."

The third problem is the support of only RSS inputs. But judging from the Pipes developers' posts, however, this problem looks like it will be overcome in the future.

So what can one do right now with Pipes? The canonical example has been putting the RSS feed of the New York Times through a Flickr filter, the result being that you can see the NYT stories in photo form. Which brings us to problem number four: your results are only as good as your data. While a lot of these photos are applicable to current events, others are tangential and some are just irrelevant. Kind of makes one think that a well-implemented semantic web isn't such a bad idea, after all.

Another thing one can do with Pipes is what I did with my first pipe, Ray's Deals. It simply takes several RSS feeds from bargain electronics sites and aggregates them into one feed. I then made the feed (or, more precisely, the titles of the items of the feed) searchable. The utility of this Pipe is that Ray now only needs to subscribe to the RSS feed from this pipe instead of several RSS feeds. (Sidenote: I actually added way more modules than necessary here: instead putting each URL in a separate Fetch module, I could have just had multiple entries in a single Fetch module... live and learn, I guess.)

That's about all the useful things one can do with Pipes at present -- my subsequent, more adventurous, Pipe experiments were fraught with frustration and, ultimately, failure. This was in part due to Pipes problem #5: the limited amount of operators that Pipes provides. There are currently no functions for performing the most basic of operations, such as generating random numbers (see the failure of my Pipe Random Jokes (Incomplete)) or extracting specific information from a feed. There are some workarounds for things like selecting a particular item from a feed, as my second hackish pipe, Selection Experiment 1, demonstrates. To select a certain oldest item, I sorted the feed by Date and then cut off the feed after a certain number of items (6). I then reversed the feed and cut off all but the first item, thus producing the selected item. Wow, that's a lot of work. Not to mention the limitations of some of the other operators, such as Sort not taking in a key value to sort by, etc. To be fair, it's still a beta product and the selection and power of modules will no doubt grow with time.

There are other concerns too, some of which are voiced in the responses to Tim O'Reilly's post. Among these are IP concerns -- is your mashup your property, or the feed providers' partial (or full) property? Also, there is the question of accessibility: will Pipes be a place that democratizes mashups, or is it just for geeky developers (who might be better off coding their own specialized apps anyways)?

All in all, Yahoo! Pipes looks like a tool that has some potential. It has a nice interface (when it's not super slow), the concept is fresh, and it's relatively easy to use (again, when it works). Its current shortcomings, however, make it little more than a novelty item. Once these get addressed, I'll be willing to give it a second try.

Connecting to servers in Ubuntu with "Connect to Server..."

2007-03-06T08:36:00.000-08:00

I don't know how this has eluded me for so long, but there is actually a very nice interface built into Ubuntu that allows you to connect to servers using any of the following protocols: ftp, ssh, windows share, WebDAV (http), Secure WebDAV (https), and more. This is a nice alternative to using third party software like FireFTP (even though that program rocks) or the command line.

Simply pulling down the Places->Connect to Server... option in GNOME opens a dialog box which allows you to enter the appropriate information about the server you want to connect to. After the connection is successful, a folder representing the remote file system will appear on both the Desktop and in the Places menu. Another nice feature is that the folder even has some little letters on it telling you what protocol it is connecting over. You can now drag and drop files via the Nautilus interface, as if the directory was local. How convenient!

This tip found via the free online book Ubuntu Linux Bible. (Reviews on Amazon) By the way, this is an incredibly comprehensive reference on all things that an end user might want to do on Ubuntu. It also has sections on setting up subversion, using LaTeX, setting up other servers such as Apache, multimedia usage, and just about everything else. UPDATE: Whoops! This actually got posted by mistake by Lifehacker... the book has been taken down... it's still a good book, though!

UPDATE: For some more (unrelated) GNOME fun, right click on an icon and select "Stretch Icon" to blow up the icon to a cartoonish size. Click "Restore Icon's Original Size" in the same menu if you get tired of it (which you will, after 2 seconds).

whois

2007-02-14T08:48:00.000-08:00

The WHOIS database is a nice way to determine contact information for a web site. Query the site with whois <site> It shows the name and address of the registrant, through who he/she registered the site, a technical and administrative contact, and the domain servers corresponding to the site. Use it along with dig, host, ping, etc. (and if you're feeling naughty, nmap) to get information about a site.

Links

UNIX status commands on Solaris

2007-01-20T12:16:00.000-08:00

Proc tools:

pflags Print the /proc tracing flags, the pending and held signals, and other /proc status information for each lwp in each process.
pcred Print or set the credentials (effective, real, saved UIDs and GIDs) of each process.
pldd List the dynamic libraries linked into each process, including shared objects explicitly attached using dlopen(3C). See also ldd(1).
psig List the signal actions and handlers of each process. See signal.h(3HEAD).
pstack Print a hex+symbolic stack trace for each lwp in each process.
pfiles Report fstat(2) and fcntl(2) information for all open files in each process. In addition, a path to the file is reported if the information is available from /proc/pid/path. This is not necessarily the same name used to open the file. See proc(4) for more information.
pwdx Print the current working directory of each process.
pstop Stop each process (PR_REQUESTED stop).
prun Set each process running (inverse of pstop).
pwait Wait for all of the specified processes to terminate.
ptime Time the command, like time(1), but using microstate accounting for reproducible precision. Unlike time(1), children of the command are not timed.

Other commands:

arch display system architecture (i86pc)
uname print name of current system (SunOS)
pagesize display the size of pages of memory
iostat report I/O stats for terminal, disk, tape, etc
vmstat report virtual memory statistics
mpstat report CPU statistics
busstat report bus-related support statistics
lsof list open files (sockets (lsof -i), etc.) Introductory lsof examples
nohup run a command immune to hangups
kstat display kernel statistics

While navigating manpages, note that Solaris uses more to view them, which is supremely annoying. Use less with export PAGER="less" (tip found here).

HOWTO Starcraft/Brood War on Ubuntu Linux Edgy Eft with Wine

2007-01-04T17:02:00.000-08:00

One of the (very very very) few things for which I use my Windows partition anymore is playing Starcraft /Brood War. Of course, as anyone that dual-boots knows, restarting your machine just for one Windows application and then back to Linux is highly annoying and time-consuming. One solution to running Windows apps on Linux is Wine, which I will describe how to configure to play SC/BW in this post. I'm using Ubuntu 6.10 (Edgy Eft). Thanks to JTerry for his help with this.

First, you're going to need a few things. Open a terminal and run sudo apt-get install wine to get wine. After that finishes, insert the Starcraft CD. cd to whatever cdrom drive you put the cd in (under the /media/ directory). Use Wine to setup SC with wine setup.exe. You'll get the install screen for SC -- enter your CD key and install the software. The SC executable will be installed under the path of ~/.wine/drive_c/Program\ Files/Starcraft/starcraft.exe. If you try and play SC now, you might notice that the sound does not work. Run winecfg and go to the Audio tab. Set Hardware Acceleration to Emulation. The sound should now work fine. If you want to play Brood War, read on; if not, skip to the next paragraph. Eject the SC CD and insert the BW CD. Again, cd to the appropriate /media/ directory and run wine setup.exe. To play SC/BW, run wine ~/.wine/drive_c/Program\ Files/Starcraft/starcraft.exe.

Playing online over Battle.net requires a little bit more work. First, you'll need the proper fonts that Battle.net uses... sudo apt-get install msttcorefonts for this. You'll also need the newest patch for Battle.net which updates BW to version 1.14. Get this patch here (other patches, for instance for regular SC, can be found here). Use wine to run the downloaded file, BW-114.exe. An updater should run, after which you'll be able to play online using Battle.net. It works, but all is not perfect -- the display of the Battle.net interface is kind of screwed up (it never refreshes when you change views). However, if you're familiar with the Battle.net interface you should be able to get into a game after which SC/BW runs without a hitch.

Some tips: tweak your mouse sensitivity and acceleration to play SC/BW. It runs at a low resolution and your cursor will be uncontrollably flying all over the place if you don't. Change this in System->Preferences->Mouse->Motion. Also, sometimes (rarely) the screen resolution doesn't change back to what you had it at before playing SC/BW. Change this in the System->Preferences->Screen Resolution menu. If you hear a lot of static from your speakers, try switching from OSS to ALSA in winecfg->Audio.

More information on this topic can be found at the Wine Application Database page for SC/BW.

Update: here's another good page on Ubuntu forums

Another update: yet another good page from the Ubuntu community detailing how to enable the 'no CD' patch

Why default settings on your wireless router is a BAD thing

2007-01-01T20:20:00.000-08:00

Lots of people recognize that 802.11 wireless networks are really cool and want to have them. They go to Best Buy, grab a Linksys router, run home and, without going through the standard setup, plug the router into their network connection and turn it on. Hopefully you are not one of these people because these people leave their router completely vulnerable to whoever wants to take control of it.

Here, I will show how to discover one of these networks, how to get in to it, and what you can do once you're in command. I will be using the Linksys WRT54GL as an example, but the lessons here are very applicable to other makes and models as well.

First, the discovery. You're going to need a tool like kismet for this. If you're on Ubuntu Edgy Eft (like me), a simple sudo apt-get install kismet will install it on your machine. If you don't have apt-get or a comprable tool (such as yum on Red Hat flavors), you'll have to download and compile it from source yourself. Once you have kismet, edit the /etc/kismet/kismet.conf file. Edit the line that begins with "source=" to whatever is appropriate. The first value should be what comes up after the Nickname section of this command: iwconfig | grep Nickname. The second value should be the desired wireless interface (something like eth1... also on that same line of the previous command) and the third value is whatever you want to name this interface. Mine is source= ipw2100,eth1,wireless. More info is in the README under "Capture Sources." Fire up kismet with sudo kismet. Any detected networks that have the "F" under Flags (and, with color enabled, the line should come up as red) are networks with factory-default settings. These routers are often named "linksys" or "NETGEAR" or the like. If you want to change around the interface in kismet, edit the /etc/kismet/kismet_ui.conf file.

Now that you have your target, log onto the network and (hoping that this isn't a honeypot) point your browser to 192.168.1.1 (this IP address may vary depending on the make and model of the network that you're hacking... NETGEAR routers, for example, are 192.168.0.1). When prompted for a username and password, enter admin/admin. These are factory-set values that can be found for other models by searching on the Internet. For example, here and here (and here for D-link and here for Belkin). Congratulations, you're in.

What to do now? Well, as far as the router is concerned, a lot of things. Change the broadcasted ESSID of the network to "lol u R h4cked!" under Wireless->Basic Wireless Settings. Enable a password in Wireless->Wireless Security so that the legitimate users of the network will not be able to log on. In Access Restrictions, set the router so that it ceases to operate for five hours on Tuesdays. Etc. Of course, there are more things that you could do by exploiting the actual network itself, but that's beyond the scope of this entry.

HOWTO scrub iTunes' DRM off of your purchased music with QTFairUse

2006-12-30T21:26:00.000-08:00

DRM sucks. A lot. Case in point: I just received a $25 dollar gift certificate from a loving relative who wanted me to buy some music. I summarily went to the iTunes Music Store and downloaded about twenty-five bucks worth of Cannibal Corpse songs. All was good, until I tried to play the songs on another music player. Surprise! The .m4p files are a iTunes-only, DRMed format. That is when I resolved to immediately blog about how to get around it (a mild form of protest, I know).

(Disclaimer: doing this may be violating the DMCA. But if you want to stick it to the man and protect your fair use rights, read on. Besides, the DCMA sucks, too.)

First, you're going to need Windows (sorry... I guess you could always use Wine or Virtualization inside Linux, but I don't have that set up yet -- I'm just dual-booting Ubuntu Edgy and Windows XP). Next, download QTFairUse which is a utility that will scrub the DRM from your music files. (There is also another utility for doing this called myFairTunes and an older one that doesn't work anymore called JHymn). Unzip the QTFairUse archive and launch the .exe file to start the program. Use the drag-and-drop interface to select which songs you want scrubbed and then click the button at the bottom to begin the conversion. You now have .m4a-format, DRM-free audio files!

UPDATE:
Neither of these programs work under Wine... neither are in the Wine Application DB... looks like I'll have to stick with Windows (ugh) for now...

QTFairUse throws a "ImportError: MemoryLoadLibrary failed loading win32api.pyd" when using the console version and an "ImportError: MemoryLoadLibrary failed loading win32gui.pyd" when using the GUI version

myFairTunes sets up fine but throws this error when I run the application under Wine: "install the Windows version of Mono to run .NET executables" ... searched Google and it seems like this is a long way from being fixed

UPDATE 2:

new links:

Be sure your computer is 'authorized' to play 'protected' files on iTunes or this won't work.

Static IPs in Debian-based Linux

2006-12-21T14:05:00.000-08:00

Here's a brief overview of how to set up an internet connection using a static IP address on Debian-based Linux systems. Most large networks nowadays use DHCP to get a user an IP address, but knowledge of static IP configuration can still be helpful in a lot of situations. First, edit the /etc/network/interfaces file as a superuser so that the entry for the desired interface looks something like this (replace x's with numeric values):

iface eth0 inet static # static, as opposed to dhcp... replace eth0 with desired interface
address xxx.xxx.xxx.xxx # ip address you want assigned to this computer
netmask xxx.xxx.xxx.xxx # subnet mask
network xxx.xxx.xxx.xxx # optional
broadcast xxx.xxx.xxx.xxx # optional
gateway xxx.xxx.xxx.xxx # gateway (usually router/firewall) ip address

Once you have done this, restart your network interfaces with sudo /etc/init.d/networking restart

The gateway can also be specified on the command line if it's not in the interfaces file: sudo route add default gw xxx.xxx.xxx.xxx The command netstat -nr can show the routes, including gateways, that the kernel is currently using.

To explicitly release a dhcp connection, do something like dhclient -r eth0

If your computer and network is set up correctly, you should now be able to ping your gateway and access sites on the internet by IP address. However, in order to access sites by name you need the help of DNS. Add entries to the /etc/resolv.conf file as a superuser in this format (you can have as many as you want):

nameserver xxx.xxx.xxx.xxx

Now, if you point your browser to a domain name like www.yahoo.com, you should be able to get there. Welcome to t3h 1nt3rw3b. You can also do all the above by accessing the System->Administration->Network menu.

Links:

HOWTO Basic Network Troubleshooting An Ubuntu Forums post about basic networking... check out the great script towards the bottom of the first page
HOWTO Ubuntu Linux convert DHCP network to static IP Explains how to do this, and much more, on the command line
Ubuntu Networking for Basic and Advanced Users A visual guide to using the Ubuntu Networking GUI
SearchNetworking.com A good, general-purpose networking resource

Update (in light of ubuntu-users post and responses):
For switching from one interface configuration to another (assuming eth0 and correct /etc/network/interfaces):

sudo ifdown eth0
edit /etc/network/interfaces
sudo ifup eth0

SIOCDELRT: No such process means something is screwy. You probably have the wrong number (i.e. gateway) somewhere. Haven't determined exactly what this means.