Friday, December 14, 2007

Learning Ruby from Python

I just started to pick up Ruby today (to install: sudo apt-get install ruby irb ri rubygems). Here are some things I learned along the way, coming from a Python mindset. The (very good and highly recommendable) references I looked at were (in order):
  1. Ruby in Twenty Minutes
  2. To Ruby From Python
  3. Ruby Essentials (free, on-line book)
  4. Important Language Features and Some Gotchas
Similarities
  • Brackets for arrays, braces for dictionaries (called hashes in Ruby)
  • Strong dynamic typing
  • Everything is an object, variables are just references
  • Exceptions are similar
  • No special line termination characters
  • # for single line comments
  • def to define a method, class to define a class
Differences
  • Interactive prompt: python => irb
  • Interactive prompt help: help(str.count) => help "String#count"
  • Interactive prompt reload: reload(foo) => load "foo.rb"
  • Command line docs: pydoc => ri
  • File extension: .py => .rb
  • Shebang line: #!/usr/bin/python => #!/usr/bin/env ruby
  • Indentation and blocks: : and tabs => either { } or end
  • Strings: immutable => mutable (can use freeze method for immutability)
  • Naming conventions: unenforced => enforced (ex. class names start with a capital letter, variables start with a lowercase letter)
  • Raw strings: r"blah" => 'blah'
  • Parantheses: mandatory => sometimes optional
  • Booleans: True, False => true, false
  • Null value: None => nil
  • Else-if statement: elif => elsif
  • Module import: import foo => require "foo"
  • Boolean conversion: 0, False, None and anything empty => Only nil and false
  • Doc generation: docstrings below things => regular comments above things
  • Output: print => puts (or print if you don't want a trailing newline)
  • Command line execution: python -c => ruby -e
  • Global Variables
  • Ranges: range(x,y) => x...y exclusive, x..y inclusive (.to_a to return array)
  • Slicing: arr[1:3] => arr[1...3] or arr[1..2] (inclusive)
  • Object initialization: __init__ => initialize
  • Object creation: a() => a.new
  • Ternary operator: if_true if statement else if_false => statement ? if_true : if_false
  • Substring matching: 'foo' in str => str['foo']
  • "Main": if __name__ == '__main__' => if __FILE__ == $0 ... end
Philosophical differences
  • Attribute access: direct access => method calls
    • Need getter and setter methods to access attribute outside class
  • Access permissions: convention by underscore => public, protected, private
  • Multiple inheritance: allowed => mixins
Stuff Ruby has that Python doesn't
  • Double quoted strings allow expression substitution #{} and escape sequences \t\n
  • You can re-open a class at any time and add more methods
  • Block comments: ==begin and ==end
  • Special variable characters (begins with...):
    • $ global
    • @ instance
    • [a-z_] local
    • [A-Z] constant (triggers warnings when reassigned)
    • @@ class
  • Arrays support some set operations
    • difference -
    • intersection &
    • union |
  • Method conventions (ends with...)
    • ! changes internal state of object
    • ? returns boolean
  • do keyword -- optional loop keyword (unless on single line)
  • Crazy for loop alternatives
  • General delimited strings
  • Case statement
  • Lots of options for string element access and string substitution and insertion
  • Regular expressions as first-order objects
  • Perl-like control flow constructs that can be tacked onto the end of an expression

Tuesday, December 11, 2007

"Why don't you write a Facebook Application?"

There is a lot of hype surrounding the newly released Facebook application platform. Developers are falling all over themselves to get a product rushed out the door, funding for Facebook apps is flowing in from VCs and you can even take a class in Facebook application development at Stanford. Amid all of this irrational exuberence, I find myself frequently getting asked why I don't write to it. Well, there are a lot of good reasons to stay as far away from the platform as possible. It's proprietary, unpredictable, not a level playing field in the least and crowded with unworthy offerings. My time and effort is much better spent somewhere else.

First of all, even if we assume for the sake of argument that I am interested in developing a Facebook app, the deck is stacked against me. Tim O'Reilly points out that there is a steep drop off in users after the first four or five most popular apps (the graph on O'Reilly's page is instructive). There are a few reasons for this. The first is that the door has largely closed on the willingness of users to add new applications or suggest them to their friends; the fad has come and gone. In other words, "[W]e must acknowledge that ... users have grown accustomed to a small subset of applications available to all users, and the exploratory period has come to an end... Facebook is all about communicating an image of yourself to your peers, and unless someone creates a truly captivating, innovative, or better F8 application, no one is going to adopt it on a whim." The quickest functional and moderately compelling apps to get out the door were the ones that users pounced on. The second reason for the disparity in users parallels the disparity in the finances and professionalism of the makers of the apps. Five of the top six apps right now are made by just two companies: Slide and RockYou!. [A] Making little flash widgets is essentially these companies' entire business and both of them are well financed. Amateur developers do not stand a chance against a small army of VC-backed Flash coders who do this for a living. [B] Any entry into the Facebook app market now is essentially futile.

The survivability of the Facebook platform is another issue. It bears some eerie similarities to "walled garden" initiatives past. Perhaps most illustrative is the case of AOL, whose proprietary platform, of course, nobody uses anymore. Should I really bother developing against a technology predicting that it won't survive 10 years? Perhaps, but I would rather write to a platform that is built on open standards. Jason Kottke explains: [E]
As it happens, we already have a platform on which anyone can communicate and collaborate with anyone else, individuals and companies can develop applications which can interoperate with one another through open and freely available tools, protocols, and interfaces. It's called the internet and it's more compelling than AOL was in 1994 and Facebook in 2007.
The fact that Facebook could change its platform or go out of business at any time is a disincentive to write a Facebook application. I don't like my creations functioning at the whims of Facebook or its investors; it's a better bet to go with an established, open platform.

I ran across a similar post on Facebook app suckiness that raised an additional reason not to develop a Facebook app. If, against all odds, your application is truly innovative and distinguishes itself in the crowd of petty profile pollution, you still are on Facebook's turf; that carries some important consequences. Specifically, Facebook could copy your idea with a feature that they subsequently build in to Facebook itself. It will be accessible to all users, not just the ones that choose to add it as an application, and there will be no use for your app anymore. Facebook could take your idea and reimplement it, thus sucking away all of your users, and there would be nothing you could do about it.

Then there is the embarrassing stigma of being in the company of Facebook application developers. The (many) applications are written by developers who want to jump on the Facebook hype bandwagon. Most of them are amateurs, and it shows. Take the Quizzes application, for instance (ranked 14th in most active users, so a reasonably popular app). It lacks basic functionality that one would expect in such an application: namely, the ability to see the correct answers after you take the quiz. It also creates a quiz that other people can take before you even indicate you are finished adding questions! Furthermore, I couldn't access the application at all today because, presumably, it is choking under the load that Facebook is sending to it (another all-to-frequent problem of these applications). Quizzes, admittedly, might be an application that one might use occasionally. I can not say the same about other applications, most of which can be broken down into four categories:
  • Horoscope
    • What flower are you? -- "Are you rose, lilly, foxglove, daffodil, nigella, sunflower, violet? Find out."
    • Birthstones -- "From the time gemstones were discovered, they were believed to have mystical powers and attributes that could be passed to the wearer. The red of ruby was fiery and passionate; cool blue sapphire was calm and composed. What's yours?"
    • Psychic Breeze - Psychic Readings and Fortune Telling -- "Accurate psychic, psychic medium and clairvoyant readings are available today. Our extraordinary psychics, mediums and clairvoyants use their psychic ability to provide you with guidance in relationships, love, finances, and other life hurdles."
  • Decorative trinkets
    • Snow Globe Gifts -- "Snow Globe Gifts! Send your friends realistic, shakeable, snow globes..."
    • Christmas Music Tree -- "Create your own special Christmas Tree. Decorate your tree with your friends and make them play many famous Christmas songs. It's your Musical Christmas Tree."
    • My Fab Bag -- "Buy and give your friends designer handbags. Choose your favorite to carry and display on your profile."
  • Finding a boyfriend
    • Likeness UNRATED -- "Find out who you're like on mature topics like purity, vices, and the seven deadly sins. Can you find a perfect match?"
    • Will you KISS me? -- "Send kisses to loved ones, lovers, romantics, mistresses and everyone! Ever wanted a friend or crush to kiss youand find that they want to kiss you too What if it happens on a Full moon night? Give and take kisses"
  • Just good ol' pestering people
    • Zombies -- "Bite your friends and make them zombies! Mmmmm... Brains! Also - you can fight Zombies, Vampires and Werewolves now!"
    • Vampires -- "From the makers of Zombies comes Vampires! Kind of like Zombies. Except that the vampires are hot. VERY hot. Also - you can fight Zombies, Vampires and Werewolves now!" [C]
    • X Me -- "Tired of just poking? X me opens up a whole new world of action-based messaging, for example 'Hug Her, Slap Him, Tickle Them!'"
Is this trash what I really want to spend my time developing? Do people actually waste their time using this filth? It's depressing that humans' frontal lobes have not yet developed sufficiently to resist soliciting the advice of fortune-teller applications. [D] Given this kind of crowd, I don't think I could expect anyone to take me seriously if I told him I was developing a Facebook application. I would anticipate him laughing in my face.

That's why I don't develop for the Facebook platform: it's pure hype. Nothing of significance has been produced in the form of a Facebook application, and the only applications that get any views are ones that are made by companies who specialize in them. This environment exists at the whim of Facebook and its investors, and could come crashing to a halt if the business goes belly-up or Mr. Zuckerberg has a change of heart. I'll take my code somewhere else, thank you very much.

===========================

[A] Prepare for some awful, awful music and graphics if you go to RockYou!'s page. I had the displeasure of being aurally assaulted by Lil' Mama blowing kisses at me. You have been warned.

[B] To hammer this point into the ground, RockYou!'s CEO discusses here how, in order to prepare for an expected spike in traffic, they ran around filling a 24-ft truck full of servers and hooking them up over a weekend. Does this sound like the kind of thing you could pull off with the change you found under the sofa? I didn't think so.

[C] "From the makers of Zombies..." As if Zombies was a fantastic, useful product to stake their reputation on. They have WereWolves, too. Maybe there's more, I didn't care to look.

[D] And there's a hell of a lot more dumb applications, if you want to take a peek.

[E] He expands on the "Facebook as AOL" theme in this post. Scott Rosenberg also has his thoughts on the subject.

Why I am Still on Facebook

Facebook sucks. And yet, I can't bring myself to terminate my account. It has a few essential features that I just could not do without because of its dominant position as today's preeminent social platform. Like the AOL of yesteryear, everyone else uses it (a bit too much) and expects you to use it to communicate with them. These few communication and sharing features are the site's strengths. Facebook counts on users sticking around in order to use these features regardless of how many ads they plaster on their pages or how many privacy violations they can rack up. Facebook's flaws are outweighed by its utility, which is why I haven't quit Facebook... yet.

Facebook's best feature is what it started out as: a directory of contact information. The average person does not have the time, skill, interest or money to register a domain name, set up a personal web page and push it to the top of Google's index for searches against his name. A social networking page is a much more convenient way to disseminate contact information. Facebook is the best solution for this because it shares contact information only with people of which the person approves. This arrangement makes many more comfortable than if they had shared their contact information on a public site such as MySpace. As Facebook also started out as a college network about the time that I was in college, it is the platform of choice of virtually all the people I would desire to contact. Most of the time, the aggregate contact information for someone just starting their professional life is not available anywhere else. I can not count the number of times when I quickly needed to contact somebody, only to realize that I did not have her AIM screen name, email address, phone number, etc and fetched the information off Facebook.

The other killer feature of Facebook is photo sharing, which is why it is now the largest photo sharing site in the United States with over a billion pictures. If I were to withdraw from Facebook, I would no longer have access to my friends' photos and they would no longer have access to mine. Of course, I could always migrate all of my pictures to Flickr or some other site and then point all my friends there. But is it really worth the time and effort to do that, knowing that Facebook already has a great framework for doing this that my friends and I currently use? Again, Facebook's privacy controls are also an important feature: if I transitioned to another platform, I would need to re-implement the privacy controls there (assuming they were even supported!). Additionally, all of the tag data and comments that Facebook supports would be lost in translation.

And let us not forget the networking part of social networking. Facebook does provide opportunities to network, in the business sense. [A] It's easier to approach a person if he looks familiar, you have some background on him and you can draw from a number of topics of conversation. Also, even if the only contact two people have is seeing each others' name go by on their respective News Feeds and having the other's face pop up once in a while on sample friend lists, that connection is sufficient to leave a lasting impression on the subconscious mind. [B] And requesting someone to be a friend on Facebook can be sometimes interpreted as a digital 'good meeting you' compliment. In spite of all this, I believe the networking aspect of Facebook to be a relatively minor incentive for staying on the site.

So photos and contact information are really the only two things that keep me from ending my adventures with Facebook. For the time being, it seems that I'm just going to have to swallow my pride, eat my words and keep using the site.

=============================

[A] "Expanding one's social network or sphere of influence by initiating mutually advantageous new relationships with people." (link)

[B] I didn't come up with this theory, my friend Monique did.

Monday, December 10, 2007

Facebook Sucks

Every time I sign on to Facebook, a little part of me dies. Not only am I usually wasting my time, but I am allowing Facebook to violate my privacy, potentially offending hundreds of "friends" and being bombarded with ads and spam. Furthermore, I am forced to use Facebook's clumsy tools to communicate with others on Facebook who seem to never have heard of email, all while wading through the ostentatious posturing of Facebook's users. In short, Facebook sucks.

Facebook is a great opportunity to offend people. As if I didn't have enough trouble minding my etiquette in the real world, the choice to friend or not to friend (or grant restricted access, or defriend) provides daily chances for someone accidentally or intentionally insult someone else. The heart of the problem is that some people have different conceptions of what a Facebook friend actually entails. Does it mean you are friends in real life? Is it meaningless? Some are willing to Facebook friend total strangers and others keep a very small circle Facebook friends that might actually be closer to the number of good friends they have in real life. There is a point at which this managing of digital networks becomes tiresome, evoking a social network fatigue. The value of a particular user's experience on Facebook (or on any social network) rises and then falls as the number of users increases. [B] At first, the user is excited to connect with all of his friends and perhaps reconnect with some that he had lost touch with. But over time, as more people join the site, more time is spent on fending off unwanted friend requests and friend network management. This eventually drives the user to become much less active on the website, if not to opt out of it completely.

Facebook is a black hole that sucks up time. There is certainly something compelling about browsing your "social network" through a hyperlinked photo yearbook. In fact, it's too compelling -- some have complained of "Facebook addiction." Facebook exacerbates this problem (well, certainly not a problem for them) by sending you incessant reminders of activity on your account by default ("Someone has done x to you on Facebook") which pull you back to the site again. [A] Apps have worsened this addiction because now every app requires its own micro-management and sends its own messages to your inbox. Here is a picture of the overwhelming number of annoyances a typical Facebook user might face upon login. All of the time spent on Facebook wouldn't be wasted if there was substantive communication taking place on the site but, for the most part, there's not. It's all just about how many people you've converted into zombies or whether you identify more with pirates or ninjas.

Facebook reinvents the wheel in a variety of ways, moving online communication a step backwards. Since Facebook wants you to stay within the site's walls, Facebook provides tools for you to accomplish certain goals, no matter how mediocre those tools may be. For example, Facebook provides a "Marketplace" for users to buy and sell items on their site. Of course, there are many superior auction/barter/market sites already on the Internet: Amazon.com, eBay and Craigslist, to name a few. Facebook provides "Posted Items" and "Notes," whose features are poor substitutes for nearly any blogging platform. And Yahoo and Google groups are many times more advanced than Facebook's groups. The most irritating example of Facebook's compulsive re-engineering is Facebook messages -- it reminds me of a dark age when GMail didn't exist, and also gives me another inbox to manage (much more clumsily, mind you).

Twice Facebook has disregarded its responsibilities to its users and precipitated privacy invasions, both for which Zuckerberg promptly issued apologies. First there was the News Feed, which broadcasted users' actions to all of their friends. Facebook followed that with Beacon, a system that tracked a users' actions on affiliate sites, such as the New York Times, and then fed information back to Facebook (and that users' friends through the News Feed) about a users' behavior. Twice Facebook has recklessly played fast and loose with its users' data, and twice it has pushed its audacity to the limit until it faced a revolt by its users. The most shocking part of this whole story is that these systems never went away! In each instance Zuckerberg waved his hands to make an apology, as if users' concerns had been assuaged, and only partially disabled the systems that caused the uproar. The News Feed, although it did get some controls, still doesn't give the user a choice if some types of stories are broadcasted. Beacon is also wholly intact, but was changed from an opt-out to an opt-in system. [C] There is no reason to think this is the last time this pattern will happen, as Scott Rosenberg points out. To justify its massive valuation, Facebook is under a lot of pressure to find additional ways to monetize its service, and there is good money to be made selling out users. What privacy-infringing "feature" will Facebook be pressured to invent next? [D]

And then there are ads -- lots of ads. In addition to the easily blockable banner ads on the bottom and sides of the page (an Internet staple since as far as I can remember), Facebook has devised ways to deliver ads to users that are not so trivially thwarted. Facebook actually embeds ads inside the News Feed that come from the same server as the rest of the News Feed, unlike other embedded ads (like Google's) which come from a third-party server and are thus easy to identify and block. Fortunately, there are some ways to rid your eyeballs of these menaces. It is also much harder to tell that you are looking at an ad in the news feed: Facebook blends them in so well to almost make them indistinguishable from bona fide News Feed stories. This approach is in stark contrast to what Google and other sites do, clearly identifying which content is sponsored and which content is not. This practice is irritating at best and deceptive at worst.

And speaking of deceptive ads, how about using my image in an ad for a sponsor, as if I were sending a personal recommendation to a friend? Taking a "social action" (as Facebook puts it) is not a license to use me as a viral marketing stooge for Blockbuster, et al. [I] To add insult to injury, Facebook is now allowing advertisers to send targeted emails directly to your Facebook inbox (the first line of the most recent one I received from CbsSports.com: "Hey College Hoops Fan!"). Hm, unwanted emails in my inbox trying to sell stuff; I think that's better known by its more conventional name -- spam. You spam your "friends" with application requests, corporations spam you with messages in your inbox, your "friends" spam you with pokes and news feed items. This is essentially what Facebook has become: a very efficient platform for spamming people.

You can put a lot of data in to Facebook, but getting that data out is an entirely different story. It is quite easy, for example, to import your contacts from another platform into Facebook. Facebook, however, provides no convenient method for exporting those contacts into Outlook, Gmail, or the other social network flavor of the week. The same goes for photos, videos and all other multimedia. Looking for a "backup my photos" link? Sorry, it doesn't exist. There are ways to get data out of Facebook, but they are inconvenient and few. One is to use the API either by writing an app yourself (clearly out of the reach of most users) or using an application like FriendCSV [K]. The API, however, doesn't allow extraction of some types of information, like email. Another is to scrape the site, which is against the terms of use (like most companies') and can lead to the termination of your account if they catch you doing so. And it is also impossible to get Facebook to delete your information from their servers, even if you quit using the site! Facebook is not only a black hole for your time, but also for your personal data.

Facebook has become the victim of its own success: phishers are starting to use the site as a launchpad for attacks. Phishers embed links on a user's wall that point to a malicious domain that harvests their names and passwords for Facebook. This, in turn, can lead to more phishing attempts as well as stealing other credentials (such as banking login information) and/or spamming for pharmesuticals, etc. Of course, no site is immune from the scrutiny of attackers, so this is hardly Facebook's fault. As a commenter on the Wired blog puts it, "Anywhere there is popularity and potential profit, there will be hackers and scammers." However, it is notable that criminals now see Facebook as a lucrative target. Facebook needs to crack down on these activities if it expects users to continue to feel comfortable using it. [J]

Perhaps this isn't the fault of Facebook per se, but a lot of the people on Facebook are really annoying. You know the ones I'm talking about. The coward who thinks that the epitome of activism is clicking a button that says "Join Group." [E] The gullible student that believes the world will be changed by joining groups with titles like "For every [number] people that join this group, I will donate [amount] to [cause]." [F] The narcissist that ceaselessly uploads pictures of themselves and her friends partying and broadcasts her status message to the world at least ten times a day. The clueless folk carrying on what, prior to Facebook, would have been a private conversation on each others' walls. [G] And people that have way too much free time giving each other gifts [H] and engaging in poke wars (or now, thanks to SuperPoke, throwing cow wars or the like). Facebook is often a cesspool of narcissism and ignorance that I could do without.

It may come as a surprise that, despite all of these grievances, I haven't terminated my Facebook account. It is true that I still grudgingly sign on to the service at least once a day because it provides some tangible benefits that no other service offers. Regardless of its flaws, I haven't quit Facebook... yet. I intend to write two follow up articles to this one, the first discussing what Facebook gets right and the second as an answer to "Why don't you write a Facebook application?" Stay tuned.

Update: I'm finally getting off of Facebook. The straw the eventually broke the camel's back for me was the sheer unusability of the site. Nearly every page load on Facebook maxes out my processor (on a decent machine). It's not just the sheer load of crap that Facebook is bringing into each page; even the most basic user actions cause my browser to lock up. For example, entering characters into a text box (for commenting on a photo or sending an email) has a delay of several seconds between when I hit the keys on the keyboard and when the letters show up onscreen. These inexcusable bugs plague the site. Congratulations, Facebook, you've finally driven me away.

=========================

[A] Yes, I know you can change this in your settings. Yes, I know that they now send the contents of messages in the email as well. Everything else, however, still gets you the same information-void kind of notification that begs you to come to Facebook if you want to find out what was actually said.

[B] I'm certainly not the first person to identify this phenomenon, by the way. I'm not sure who, if anyone, is the right person to attribute this to. Thoughts?

[C] And who knows what Facebook thinks "opt-in" means? The devil is in the details: does not clicking on an "I don't want this" indicate the user wants to participate? Zuckerberg, upon Beacon's release, already had some interesting ideas about what "opt-in" meant.

[D] Ed Felten provides an excellent Beacon post-mortem here.

[E] Some think that the best way to protest Facebook's practices and policies is to join a group whose cause is to recognize the fact that all its members dislike a new feature. It's not. The best way to protest is to delete your Facebook profile.

[F] A frequent question I ponder when I see groups like this is, why do people waste their time supporting these groups if they have zero assurance that the donation/action/whatever will actually happen?

[G] I really, really don't need to know the day-to-day private details of your life. And I REALLY don't need them broadcasted to me in my News Feed. If you're negotiating a play date with your friend, take it off Facebook! If you're dumping your boyfriend, take it off Facebook!

[H] Perhaps these do serve a cause since Facebook donates $1 for most of them to charity. But it annoys me when it is implied that there is some kind of scarcity to information, playing to misconceptions about the Internet. Okay, pet peevey rant over.

[I] In the legal sense, as well: could this practice be illegal?

[J] It is also a testament to the cleverness of the phishers (and the nature of Facebook's users) that they are using such well-targeted bait in the text for their links: "lol i can't believe these pics got posted.... it's going to be BADDDD when her boyfriend sees these,"

[K] Careful, FriendCSV's creators try to sign you up for their own social network when you use their product. How hypocritical, offering a way out of someone else's frying pan and into their fire.

===================================
UPDATES:

Facebook is sharing too much data with application developers. (Link)

Facebook, in violation of their privacy policy, is now sharing your personal data with Microsoft. Hm, does that have anything to do with taking $240 million of their money?

Facebook is arbitrarily removing applications that don't seem to be in violation of their privacy policy, a la Apple and the iPhone store. The victim this time? Burger King.

Facebook may be eliminating local networks, exposing more personal data to more people.

In a rare moment of good news, Facebook has agreed to abandon Beacon.

Another reason to stay off Facebook: STDs.

Tuesday, December 04, 2007

Code formatting

Here are some tools to make your code pretty (in Ubuntu repos):
For indent, I find that the following arguments seem to work well for code I write: -bap -bbb -bl -blf -bli0 -bls -cli3 -di1 -fca -hnl -i3 -ip0 -l80 -lc80 -nbbo -nut -nsaf -nsai -nsaw -psl

Monday, November 12, 2007

Blocking Facebook News Feed Ads with Greasemonkey

I hate ads, so I block them. If I want something, I'll search it out. I don't need marketers wasting my time trying to convince me that I need something I don't. Most of the time I have ads blocked by aliasing most known ad servers to 127.0.01 in /etc/hosts/. This only works when the ads are coming through a server that is listed in the file (say, ad.doubleclick.net) and not when the ads embedded in the page are served from the same server from which the original page was requested.

This is exactly the problem with the ads from Facebook, which just deployed a new ads system that has ads targeted on what you've listed in your profile. For example, if you listed Jane Austen novels as some of your favorite books, you might see text ads in the sidebar or in your news feed advertising "Jane Austen: The Ultimate Sappy Romance Collection" ... or something. So how to block these ads?

Greasemonkey is a Firefox (and IE, although I wouldn't know much about that :-) ) extension to inject arbitrary Javascript into web pages, meaning that you can manipulate a page, or parts of a page, automatically as you see fit. Its behavior is defined by scripts, many of which can be found at userscripts.org. If you're familiar with web design, you can write your own scripts quite easily. Mozdev has a good quickstart guide, and Mark Pilgrim has an online guide and a book if you want to research this further.

There are a few ways to do block Facebook's new ads. The first is using document.getElementsByTagName('div') to fetch all the divs in the page and then loop through them, testing them for known ad-serving attributes (for example, a className like 'social_ad_advert' (sidebar) or 'feed_item clearfix social_ad' (news feed) or an id of 'ssponsor', 'sponsor', or 'announce' (all banner ads)) [1] and then either hiding them with element.style.display = 'none' or removing the element entirely using element.parentNode.removeChild(element). A second is using XPath, which makes for much more compact code. [2] userscripts.org is awash in (mostly mediocre [3]) scripts to block Facebook ads, but the most comprehensive one I found was "Remove All Facebook Ads" which I'm using in my browser right now. The only suggestion I have for this script would be to use the window.addEventListener to trigger execution of the code rather than embedding it in an anonymous function, as explained in the quickstart guide (under "Tips").

Greasemonkey scripts are executed after the DOM is loaded, which means that all the ads will be fetched from their sources and displayed on the page before they can be stomped out, unfortunately. Therefore, to make the page load faster by not fetching and displaying ads in the first place, it would be wise to use an /etc/hosts blocking scheme instead of a Greasemonkey solution where possible. Greasemonkey should be used only as a last resort in combination with /etc/hosts where /etc/hosts can't block ads that aren't fetched from a server different than the one used for the requested page, as in the case of Facebook news feed ads.

[1] It would probably be faster to not test for the id matches and just using getElementById for those few cases outside of the loop instead (within try-catch blocks if the elements aren't guaranteed to be on to page and could potentially throw an error).
[2] I'm not sure about the efficiency implications of using one vs the other. Anyone care to comment?
[3] For instance, "Hide Facebook Ads" tries to do some browser detection... I'm not sure how large the market for IE4 Greasemonkey users is...

UPDATE:

Now using New Facebook Layout Ad Killer for the new Facebook layout

Argh, it seems Facebook has added another lame ad sidebar. Just extend the NFLAK script by looking for a document ID called 'fadbar' and hide it if it's there.

Wednesday, October 24, 2007

Gutsy (Ubuntu 7.10) Tweaks

Just upgraded to Gutsy from within Feisty, which took about an hour, and things look great, for the most part. But here's a few things just to make sure everything was in tip-top shape:
  • To enable Compiz Fusion, System->Preferences->Appearance->Visual Effects->Extra
  • Gutsy doesn't ship with the Compiz Fusion settings manager, and I wanted my old (Beryl) settings back... run sudo apt-get install compizconfig-settings-manager in a terminal to install it and ccsm in a terminal to launch it
    • Don't forget to create a new profile in Preferences or else you'll be screwing with the default settings
    • Personally, from the defaults I enable Rotate Cube, Paint fire, Water effect, Wobbly windows, Annotate, Splash, Cube Caps, Shift Switcher; I disable Expo; I tweak Desktop Cube (General->Multi Output Mode->One Big Cube) and Animations
  • Fonts on my system were oddly stretched out vertically. Here's a solution from this thread: edit the line in /etc/gdm/gdm.conf that says -command=/usr/bin/X -br -audit 0 to say -command=/usr/bin/X -br -audit 0 -dpi 96
  • The fonts still look a little screwy. I'm just going to bite the bullet and install the MS core fonts ... a HOWTO is here.
  • Download updates automatically in the background: System->Administration->Software Sources->Updates->Download all updates in the background
  • Synchronize your clock to NTP servers sudo apt-get install ntp
  • Autohide pannels: Right click on a panel->Properties->Autohide
    • Set how much of a panel can be seen when it is hidden: link

Tuesday, October 23, 2007

An Indictment of Apple, Part 1

Unless you have been in a cave for the past several years, you've probably noticed the increasing brightness of an odd, white glow that radiates wherever students, geeks or professionals congregate. That strange luminosity is the combined brilliance of the hypnotizing and stylish logos on Apple notebooks that everyone seems to be carrying around these days. It's no secret that Apple's hardware (and accompanying software) has been selling well recently, with its popular laptops, iPods, iPhones, etc. Everyone I know seems to be purchasing one of Apple's shiny toys: my family, my friends, my UNIX zealot colleagues and technoignorant acquaintances alike. The media constantly flatters Apple with positive reviews of its products, editorials praising CEO Steve Jobs' visionary genius and provides an extraordinary amount of marketing hype for its new products. Could this company possibly do any wrong?

The answer to that question is a resounding "Yes." It could, it can, and it does.

Apple's products have become a liability to own. They are often ladden with restrictions, break easily, frequently can not be extended by third-party developers and are, in some cases, just a prime example of bad design. Furthermore, it is perplexing that Apple's products continue to be so popular when superior alternatives exist. In this article, I will focus primarily on how a top-down culture of control inherent in Apple products decreases their value. I will address other issues in subsequent articles.

Some of Apple's problems stem from its desire for the user to have an experience using its product over which it has total control. Another way to phrase that would be that Apple wants you to lock you in to its product line, so that it can compel you to fork over your cash for the next edition of OSX or the newest personal electronic device. It is a culture of top-down control that, on the whole, does not benefit the user. For example, about a month ago new iPods shipped with a technical restriction that prevented them from working with other operating systems (Fortunately, the gtkpod developers broke the lock within days). Basically, Apple is saying to the consumer, "In order to use our products, you need to buy even more of our products. Otherwise, tough." Apple is similarly inconveniencing customers with iPod's ability to pipe video output to a TV: in order to use this feature, you need to either use a device with a built-in Apple authentication chip or purchase an Apple video cable for $49.

The recent tug-of-war between iPhone owners who want their pricey gadget to be as useful as possible and Apple's not-unless-we-approve attitude is an illustrative case in point. In case you haven't been following the news, Apple made a deal with AT&T so that it would be the exclusive carrier of iPhone voice traffic for five years and built technical mechanisms into the phone to ensure this arrangement. Naturally, customers of other carriers wanted to use the phone on the network they were already paying for, which meant they would have to hack the phone to get this functionality. Thus, the iPhone was hacked, again and again. Apple estimates as many as 250,000 people, roughly one sixth of iPhone buyers, have hacked their phone. In response, Apple released a firmware update to the phone which made the unlocked phones inoperable. Apple's consistent attempts to prevent its customers doing what they want with Apple products are an insult to its users' intelligence and detrimental to their ultimate experience with the iPhone. As Jon Lech Johansen puts it, "When Steve Jobs claimed the iPhone was 5 years ahead of every other phone, was he talking about the iPhone’s revolutionary handcuffs? In a world where open technologies are increasingly becoming the norm, Apple’s way of Thinking Different means marching in the opposite direction."

Apple is hostile to not only users' interests, but developers' interests as well. Will Shipley, a developer for the Apple platform, writes an excellent post about third parties' inability to put their applications on Apple's platforms. He explains that Apple's arrogant attitude is damaging its relationship with users and developers alike:
And the iPhone is a closed system, like the iPods before it, so third parties can only develop software for it if they are EXTREMELY close to Apple. This is an incredibly frightening trend. As Apple gets more and more of its revenue from non-Mac devices, they are also getting more and more of their revenue from devices that simply exclude third parties...

But with the iPod Touch, what's Apple's excuse for locking up the platform? Why can't I write programs for this device? Who might it hurt? Why is Steve announcing that he's playing cat-and-mouse with developers who intend to do so? Is Apple so far removed from its customers that even when the latter overwhelming votes for extending a device (by downloading iPhone programs in the hundreds of thousands), Apple's response is, "No, you can't do that. We know what you want, you don't. You want AJAX apps, you just don't know it yet."

That sure reminds me of the old, crappy Apple. The one that almost went bankrupt because of its hubris.
When Apple disallows third parties from writing to its platforms, then everybody suffers. The users suffer because they don't get to use the added functionality of their devices, the developers suffer since there are less opportunities to extend Apple's software and Apple suffers because less people buy their gizmos and consumer relations are soured. Innovation often doesn't happen at the center -- it happens at the edge, and Apple does not have a monopoly on good developers or good ideas.

Apple's poisonous politics don't stop at simply excluding developers from its platforms -- they permeate the very coding tools engineers use to create. I submit to you the bizarre incidents of Apple crippling both the dtrace and gdb versions on OSX. Both programs' source contain uncommented portions of code that disallow low-level examination of certain Apple-built programs, including iTunes. These suspicious lines are not in any other version of these utilities on any other platform. Why Apple did this is anyone's guess -- it seems rather stupid to try and outwit many extremely talented programmers who will inevitably discover these unpleasant surprises (as they did). What is certain is that Apple's handicapping of OSX versions of open-source tools make Macs a less attractive purchase for developers.

Now we turn to iTunes and its accompanying Digital Rights Management (DRM), which is quite the antithesis of Apple's "It just works" slogan that it uses to push its products. iTunes has an elaborate system of "Authorization" that limits the number of computers on which one can play their music. To play a song from another computer, you must get that computer to "authorize" yours. which requires it dialing Apple HQ to ensure that this is all right with Mr. Jobs. If you don't happen to have a network connection at that point, you can't make this connection and thus can't play your friend's music -- tough luck. Of course, only a total of 5 computers can be authorized per source computer, so if your friend has already authorized the limit, he'll have to de-authorize someone else (perhaps even himself, which is allowed -- revoking the right to play songs he paid for via the iTunes store on his own computer... the gall of Apple amazes me!), at which point that person who was just de -authorized will start wondering why she can't play certain songs on her computer. What a mess. I find it repulsive, not to mention inconvenient, that I need to ask a corporation's permission to listen to certain songs in certain ways.

iTunes also has a bad habit of rolling back its already limited functionality with its frequent 'upgrades.' iTunes 4.5 decreased the amount of CDs one can create with a single playlist from 10 to 7 and also detected and blocked similar playlists. iTunes 4.7.1 took away the previous ability to stream music to anyone on a local network. One wonders when the march towards never being allowed to play any music anywhere will stop. And let's not forget the insidious 'features' that Apple included in iTunes from the very beginning: the draconian measure of embedding user data into songs [1] they purchased (presumably as a deterrent against file-sharing) and restricting iTunes-purchased songs to be played the iPod only to the exclusion of all other digital music devices. [2] [3]

I can already hear the Apple faithful clamoring that all of these malfunctions are not the fault of Apple, but the fault of the record companies whose music Apple licenses. Even Steve Jobs wrote a much-cited article which put the blame for DRM at the feet of the RIAA constituents. This "They made me do it" excuse, as John Lech Johansen points out, is quite lame because Apple both refuses to open up FairPlay (its DRM scheme) to other companies and applies DRM to songs whose labels do not require their songs to be DRMed. Not to mention that other online music services have managed to operate without DRM just fine. Amazon can do it, Magnatune can do it, why not Apple? For a very good reason: Apple profits from the lock-in that DRM provides them (see Johansen's article for some telling quotes from Apple employees). Apple is today's main proponent and user of DRM at the same time that it makes overtures declaring its wish to end DRM once and for all. This is pure hypocrisy. If you hate DRM so much, Mr. Jobs, then put your money where your mouth is and take all the DRM (in its many shapes and forms) out of your products. Apparently Apple's CEO prefers cash in his pocket to shooting straight with the public.

Apple loves controlling how people use its products. Unfortunately for Apple, users often don't like being told exactly how they are supposed to use their computer, digital music player, et al. And people extremely dislike having unnecessary restrictions placed on products for which they paid a substantial amount. Limiting and sometimes rolling back the functionality of a company's products (and therefore angering customers and third-party developers) is not a winning strategy in the long term. Apple would do well to abandon it.

------------------
Footnotes
[1] I also responded to many posts on the linked Lifehacker discussion board. It's amazing to see how willing people are to let Apple dictate right and wrong to them. I also wrote about this on one of my other blogs.
[2] Makes me all the more embarrassed that I wrote a gushing article about iTunes for a school magazine when it came out as a freshman in college.
[3] I'm not going to even go into more reasons why you shouldn't install iTunes (especially on Windows). See iTunes spying on you and bloat and ugliness and what exactly does ituneshelper.exe do?. I'm sure there's more.
-------------------
Updates
  • Apple has continued its controlling ways with the iPhone App Store: they are actively restricting the applications that people can download, even after they have been released live to the public. An application that allows one to tether a PC to an iPhone connection is an example. BoxOffice is another. I'm sure there will be dozens more.
  • iPhone development is the biggest Apple money-maker I have ever seen. To simply write programs for an iPhone and get them uploaded to the iPhone store, you have to buy three things from Apple: 1) Some kind of Apple Computer (one to two grand) 2) an iPhone (two hundred to six hundred dollars, depending on when you bought it, to say nothing of the AT&T connection fees which account for the majority of cost) and 3) an enterprise or standard development license ($299 and $99, respectively). You can not develop on anything else besides an Mac. You can not develop without a license for the SDK. With all of these restrictions, you've just put around two to three thousand dollars into Apple's pockets.
  • With regards to the iPhone App Store, Apple is no longer banning only apps that they consider unacceptable in some way (see above) but also those that contain duplicate functionality. Not only that, but Apple has been notifying developers of their rejection with a letter that has a non-discolsure agreement on it -- according to Apple's lawyers, one can't even talk about the details of why he was rejected.
  • Here's an interesting reason to kill an iPhone app: it's using up too much bandwidth. But that's just what Apple did to streaming audio app CastCatcher.
  • Apple traffics in deceptive advertising. On the company's online store, supposed 'photos' of products are actually hand-generated drawings that misrepresent certain features of the products. Furthermore, Apple has been forced to pull an ad that grossly exaggerated the browsing speed of the iPhone.
  • Taking user oppression to the next level, Apple is now arguing that jail-breaking your phone is illegal under the DMCA.
  • The last place you would expect DRM to be is in headphones, but alas, Apple has put DRM in its headphones as well (false alarm)
  • Apple has outlawed Project Gutenberg from the iPhone because some of the books contain pornographic material. Could this get any more ridiculous? Yes, they are now censoring a dictionary.
  • Apple is working on technology to detect when a customer violates the warranty. Essentially, your device tells Apple what you have done with it when you bring it in for repair. Apple already has deployed liquid submersion detectors in some of its hardware.
  • Apple, not AT&T, blocked Google Voice on the iPhone
  • The iPhone secretly tracks your location
  • Apple has joined Facebook in shutting down Palestinian-related apps

Friday, October 05, 2007

Making Windows XP less Painful

That is, making Windows XP more like Linux. Occasionally, I need to boot into Windows XP to use some application that won't run with WINE on Linux. Examples include QTFairUse and the drivers and software included with my Nokia E70 phone. Whenever I need to do this, however, I feel like I'm being suffocated by the constricting, feeble Windows environment and feel like doing the same to an unlucky bystander who just happens to bear the brunt of my Windows agitation. Fortunately, I finally sat down and figured out a few things to makeXP a bit more usable... a bit.

A (Somewhat) Usable Shell
The first thing I needed was a command line that wasn't dropped as a baby. Cygwin, which provides a UNIX-like environment on top of Windows, is perfect for this (another alternative is MinGW). The key to setting up Cygwin is configuring what applications you want installed in one of the setup menus. Of course, if you miss something on the initial setup, you can runCygwin's setup.exe again and reconfigure the included applications. Cygwin mounts the Windows C drive at /cygdrive/c. I would recommend browsing the Cygwin Properties menu and tweaking it (especially the fonts) to your preferences. Do this by clicking on the Cygwin icon in the upper left hand corner of the window and selecting Properties.

Cutting and pasting from Cygwin is a pain in the ass. To copy text, click the Cygwin icon and go to Edit->Mark. You now have a "Visual Block" - type selection box. I'm not sure if it's possible to select in a line-wrapping mode, the lack of which is also annoying. Hit Enter to copy the selected text. To paste text into theCygwin shell, click on the icon and hit Edit->Paste.

Install Software
Block Ads on the Web
Ads suck. Block them. Download a sample hosts file here and save it (in Notepad or something: Start->All Programs->Accessories->Notepad) to your file system. Now copy that file to c:\WINDOWS\system32\drivers\etc and name it HOSTS.MVP. If that's too much work for you, this page provides an automated tool to do it.

Add a Button to Show the Desktop
It's really aggravating if, in order to use the point-and-click Windows interface, you need to close all of the currently open windows to access the Desktop where a lot of clicking occurs. To do this, open Notepad and paste the following lines into it:
[Shell]
Command=2
IconFile=explorer.exe,3
[Taskbar]
Command=ToggleDesktop
Save the file as Show Desktop.scf. Right click on the task bar (that blue bar at the bottom of the screen) and select Toolbars->Quick Launch. Now drag the icon of the file you just created into the Quick Launch area. If all goes well, hitting that icon in the Quick Launch area should hide all the windows you have open and focus the desktop (or show them if you have them hidden).

Remove Unwanted Icons from the System Tray
msconfig is your friend. Good resources are here and here.

Remap keys
There is a pretty easy tool that allows you to remap your keys called SharpKeys. I'm a fan of Caps Lock to Escape remapping, myself.

Personalization
I recommend some BSOD wallpaper just to remind you from time to time that you're using an inferior operating system.

Monday, September 24, 2007

Nokia E70 for phone n00bs

I just bought the Symbian S60 v3-based Nokia E70 mobile phone based mostly on the glowing recommendation of Maddox. Since this is my first somewhat advanced mobile phone (taking pictures on a phone is still a novelty for me), I've chronicled all of the hoops I needed to jump through here in a E70-for-dummies type guide. In addition to this guide, of course, I would highly suggest browsing the excellent user manual that ships with the phone to navigate using its myriad features.

Phone calls
I got the phone off of Amazon.com for $385. Since it is an unlocked phone, inserting the SIM card from my previous phone (a pretty old-school Nokia ... unsure about the model number) enabled the phone calling features.

Internet Service
I upgraded to the unlimited Internet service from my provider (AT&T) for an additional $20 a month by going into their (pitifully understaffed and crowded) store and asking for it. Selecting the globe icon on the main screen will launch the web browser. The web browser lacks a couple of things that you would expect to find on a regular browser, namely support for animated gifs and flash movies.

Firmware update
Updating the firmware is probably the first thing you should do when you get your hands on the phone, as it will erase all stored data and potentially make previously installed applications unusable. The updates for the US edition are available from Nokia's site here. From reading some message boards, it appears that the main benefits of this update are that applications are faster and less memory-intensive. There have been a lot of reports of the phones bricking (those posts, however, are mostly about a year old), so don't say you haven't been warned! My phone, since I bought it recently, shipped with the newest firmware installed and I did not have to go through this step.

Installing applications
The E70 allows you to install applications on your phone, both from Nokia and from third parties. Once they install, you can find it under Menu->Installations (Menu is the key both in portrait and landscape mode with the blue icon and nothing else on it).

Google Maps
Google provides a Maps application for Symbian-based phones. In the E70's web browser, navigate to maps.google.com and accept the web site's offer to install the application. One of the confusing things about the application is that the labels that tell you how to control Google Maps via the buttons on your phone do not adjust to fit landscape mode. Press the left/bottom selection key for the menu, the middle joystick to zoom in and the top/right selection key to zoom out.

SSH
There is a SSH client called Putty for Symbian phones. To install it, you'll need to download the application and then install it via the Nokia PC suite, which makes installation pretty easy (on a Windows PC, that is -- Ubuntu 7.04 didn't recognize the phone as anything: camera, storage, phone, or otherwise). You must enable self-signing applications to install themselves via the directions here. Installing fonts for Putty might also make your experience with it more pleasant -- get them here. Unfortunately, there's often a lot of lag between when you type something and when you see the output. Check out the options menu to send special characters (Esc comes in handy for using vi!).

YouTube
YouTube has a website for mobile users that your E70 is automatically redirected to when trying to access YouTube. To view videos, you must first set the default access point in Menu->Media->RealPlayer->Options->Settings->Connection->Network to whatever connection you want the video delivered over. Then click away on your favorite video. Or at least your favorite video among those offered -- YouTube only provides a limited subset of all of its videos for viewing on its mobile site ("Only a portion of YouTube videos are available on mobile at this time. We are working hard to bring you more!"). The video quality is pretty good.

Transferring Phone Numbers
AT&T claims that they can transfer your phone numbers if you have all of your phone numbers saved to the phone's SIM card. Apparently they have some device at the AT&T store that does this and they will do it for free. I have a hard timebeliving AT&T (a.k.a. 'Nickel and Dimeing you for Every Minor Service to Protect our Failing Business Model') would offer anything for free, but that's what they claim.

Text Messages
My phone is able to receive text messages but not send them. This is because the phone ships without any message center specified, which means the phone has no idea where to send the messages to get them to their final destination. To change this, go to Menu->Messages->Options->Settings->Text message->Message centers->Message center->Msg. center number and change that number to whatever your message center's number is (reference). Mine was +13123149810. Alternatively, there are services like text140 that allow you to send text messages to a phone from a computer, but it lacks a lot of the convenience of sending a text message from your phone. Using this message will also not charge you for sending text messages (you're just accessing the 'unlimited' Internet, right?).

AT&T Phone Support
This is an arduous process I suspect I'll be going through a few more times, so here's what you need to do to talk to an actual person on AT&T phone support (thePalo Alto store can be reached at (650) 617-8931):
  1. 1 to continue in english
  2. 3 for an existing customer
  3. 2 for all other information
  4. Enter phone number in question
  5. 0 to speak to customer service
  6. 0 (again) to speak to customer service
  7. 2 to not participate in a survey
Incidentally, once you start talking to a person and not a machine, they have been quite helpful. Be prepared to give the last four digits of the social security number on the account if you want to make any changes.

Using WLAN Access Points for Web Access
I've found that using WLAN Access Points is one of the weaker features of the E70. Connections fail for every reason, whether it be the WPA authentication screwing up, the phone dropping connections or just failing to connect to an access point that is literally a foot away. If you can get them to work, however, they're a lot faster than the cellular network. Use Menu->Connect->Conn. Mgr.->Availab. WLAN->Options->Define Access Point to put an access point that your phone can currently see into a permanent list of access points. Use Menu->Tools->Settings->Connection->Access points to access this list and configure the details on each point (encryption, hidden network, SSID, etc.). To select the connection to use upon each launch of Web, select Web->Options->Settings->Access point->Always ask.

Bluetooth
The Bluetooth menu can be accessed via Menu->Connect->Bluetooth. To view paired devices, move the joystick to the right.

Camera
The resolution on the camera is pretty bad, and the phone doesn't come with a flash, which drastically limits the utility of the camera. Getting photos on and off the phone, however, is pretty easy since you can access the phone file system via Windows Explorer just like any other series of directories.

Gmail and Google Calendar
Both of these Google services provide very convenient mobile web interfaces. Perfect for someone who already has all of their data in El Goog's dirty paws.

Python
Snakes on a phone! Most of the documentation for S60 Python for Nokia phones is on the wiki. To install, follow the instructions here and download the appropriate things from the Sourceforge page. Now you've got a Python interpreter (and more) on your phone... what to do with it? Well, you could install a variety of applications, for one. For help, the Nokia-supported Python forum is the definitive place for answers. There's also an IRC channel, #pys60 on freenode. There's some additional documentation for developers.

Other useful things to know

  • Cut and paste works just like on any other PC: Ctrl+X, Ctrl+V, Ctrl+C
  • Press and hold Menu to get a list of running applications
  • If you're in an app and choose the "Exit" softkey (or use the task-manager shortcut above) that will actually shut down that application, and take you to whatever you were doing before. But if you press the red end (hang-up) key to jump to the home screen, (or switch to another application,) the app stays running in the background.
Links

Sunday, August 26, 2007

VIM split windows and macros quick reference

ex commands for split windows
  • :sp filename create a new split with filename
  • :vsp filename create a new vertical split with filename
  • :new create a new split with an empty buffer
  • :vne create a new vertical split with an empty buffer
  • :wa save all open files
  • :wqa save and quit all open files
Ctrl-W related commands for split windows
  • C-w h moves split to the left of current split
  • C-w j moves split below the current split
  • C-w k moves split above the current split
  • C-w l moves split to the right of the current split
  • C-w + increase the current split by one line (or, prefix this command with a number to do that many line increases)
  • C-w - decrease the current split by one line (same addendum as above)
  • C-w < decrease the current split width
  • C-w > increase the current split width
  • C-w _ Maximize the current split
Split windows in the .vimrc (taken from this page ... also see the same author's excellent Efficient Editing with VIM )
  • map <C-J> <C-W>j<C-W>_ " Hit C-j to move the current split down and maximize it
  • map <C-K> <C-W>k<C-W>_ " Hit C-k to move the current split up and maximize it
  • set wmh = 0 " allow splits with 0 lines open
Macros
  • q<register> start recording and save the macro in the specified register (use a lowercase letter, for example)
  • q (again) end recording
  • @<register> playback the macro

Screen

Screen is an awesome UNIX utility that allows you to continue running commands on a remote system even after you have logged out! Magic! Here is a nice "screen for dummies" tutorial with some links to related sites at the bottom.

On the command line
  • screen (will spew some text and then give a prompt) start a screen session
  • screen -r reattach to screen session
  • screen -d detach an attached screen session (good for logging in after network cuts out)
Links
Tips
  • Launch screen with an alias such as alias screen='TERM=screen screen' in .bashrc to avoid your backspaces getting mangled
  • There is a ~/.screenrc file with which you can customize screen
My .screenrc


# don't mangle backspaces
termcapinfo xtermc kD=\E[3~

# no startup message
startup_message off

# set shell that launches
shell /bin/bash

# lots of scrollback
defscrollback 10000

# ignore case when searching in copy mode
ignorecase on

# make it obvious we're within screen
hardstatus string "screen%n - %h"

# vim-style bindings
bind k focus up # cycle to the above split
bind j focus down # cycle to the below split
bind l next # cycle to the next split
bind h prev # cycle to the previous split
bind n split # create a new split
bind ^N split # create a new split
bind q remove # remove the split
bind + resize +3 # increase split three lines
bind - resize -3 # decrease split three lines
bind _ resize max # maximize current split
bind = resize = # make all splits equal size
bind g info # display info on the split
bind ^G info # display info on the split

# other (some defaults, just to remind myself they exist)
bind Q only # get rid of all other splits but the current one
bind M monitor # toggle monitoring of the current window
bind t title # enter a title for this shell
bind a other # go to most recently viewed other split
bind \' windowlist -b # go to window list
bind x kill # close the current shell
bind ^X kill # close the current shell
bind \\ quit # close all shells

Monday, August 06, 2007

Things I learned at DEFCON 15

I just went to DEFCON in Las Vegas for the first time this year and it was a blast. Tons of smart people, cool hacks, great speakers, fun activities and stuff to learn. Here's a brief summary of some of the things I picked up this past weekend (in no particular order):
  • Virtually any lock that you have bought can be picked, easily. Even the (supposedly) secure Medeco locks that they have on the white house and other high-security areas can be broken by a 12-year-old girl. For more, see the in.security.org blog entry (by the guys that made this presentation, Marc Weber Tobias and Matt Fiddler) here. Gun locks, also, are completely trivial for a kid to break as this amazing video proves (more on this subject: blog, report). Also from the talk: most hotel safes are terribly insecure. To break those that use your credit card to open/lock them, the code of the master card that unlocks all of them in stored in memory inside the safe. All you have to do is open the safe (legitimately) and then dump that code and write it to a card to have a "master card" for all the safes in the hotel... yikes!
  • SCADA systems, the systems that run critical infrastructure such as water treatment plants, electrical grids and nuclear power plants have an overwhelming number of vulnerabilities. Scary. Nationwide emergency alert systems also have relatively easy attack vectors (from the talk by Ganesh Devarajan).
  • There is always a motive for Internet crimes, just like ones off the net. The motive for the attack on the Dolphin Stadium website prior to the 2007 Super Bowl was particularly interesting because the attack, which was made on several other lower-profile websites, was linked to a Chinese syndicate that wanted users' World of Warcraft online credentials to acquire additional WoW gold! This is the first attack of its kind. (From the Internet Wars 2007 panel)
  • One interesting way of getting malware on a user's computer is to set up a website and register malware to be downloaded on that website as a codec with Microsoft. That way, when a victim visits the attacker's site, he is prompted with a "Additional codecs are required to display content on this page... would you like to download them?" message. If he does, he downloads the malware onto his computer. (From the Internet Wars 2007 panel)
  • If you're a NBC Dateline reporter, don't refuse a press pass and then try to brew up some sensationalist report about hackers at DEFCON. You will get owned.
  • Even the badges at the conference were a hack unto themselves (video). How cool is that? And there is a rap about the conference, too!
  • If you want to start up an advocacy organization, the most important thing you can do is have a paid person sitting by the phone who knows what expert to contact (and how to contact them) when called by reporters. PledgeBank is a good site to help organize the effort to raise this money. (Funny quote from the same Danny O'Brien talk: "[Imitating a typical clueless reporter that calls EFF] People will call and say, 'I heard that such-and-such technology can make someone's penis fall off; can this be done with Ruby on Rails?' ... In this way, EFF operates as a clearinghouse for idiots")
  • Sam Bowne offers a class at San Francisco City College called "Ethical Hacking and Network Defense" ... sweet! (He recommends Hack this Site and Irongeek.com as good references)
  • Brendan O'Connor gave a really good talk about the extra layer of "security" that banks are now layering onto their authentication services and how this layer improves neither the user's privacy nor his security -- in fact, it may be degrading both. Bruce Potter similarly lambasted "Defense in Depth" for being a lame attempt at covering up bad code with extra layers of "security."
  • Bruce Potter of the Shmoo Group gave a talk about how the dynamics of vulnerability disclosure are changing: instead of informing the vendor of the weakness in their product, hackers are often now selling information about those vulnerabilities to third parties (who may or may not have good intentions). Potter called for a discussion on whether this was ethical and/or good for the security community.
  • Thomas J. Holt presented on the economics and dynamics of the malware marketplace. This market is mainly organized around forms based in Russia and Eastern Europe where sellers start forum threads advertising their product. Moderators of the forum then test the software to see if it does what the seller says it does and then give their opinion of it on the same thread. If the opinion is good, and the seller has a good reputation, then buyers start asking questions about the software and perhaps buy it. The final step is the reviews of the software by the buyers which attest to the malware's quality (or lack therof). Some sell not only hacking tools like Pinch, Nuclear Grabber and PG Universal Grabber but the data obtained by using these hacking tools. Some forums maintain lists of sellers who are "rippers" -- rip off artists -- and blacklist them.
  • AgentX gave a whirlwind talk about "22 Things that Keep me up at Night." Among these: "Shrinking the Gap" (based off of ideas of Thomas Barnett), low cost pervasive bandwidth, open source warfare (the terrorists are all sharing techniques -- why aren't we?), the security industrial complex (complacency = bad), homogeneity of the hacker (all white males, but starting to change -- he notes good progress at DEFCON 15), why aren't you encrypting all your communications? (because the NSA is listening)
  • Mike Murray had some interesting things to say about social engineering/NLP/hypnotism. Such as: you are four times more likely to follow a command following a non-grammatical sentence, you wouldn't do something hypnotized that you wouldn't do in the first place (such as kill a commanding officer -- study), tag questions ("won't you?" "right?") are convincing, confusion is the key!, the right tone can be categorized as "artfully vague," stories are more powerful than reasoning, questions are more powerful than statements, Milton Erikson = best hypnotist ever
  • I missed Bruce Schneier's talk, but someone posted a video of it here. The whole presentation is interesting, but probably the most stimulating is his discussion of how you DON'T need an ID to fly on an airplane; you just get a little extra screening.
Other notable moments/events include the Guitar Hero contest, Brew Wars, Phreaking Challenge, Capture the Flag, Hacker Jeopardy, TCP/IP Drinking Game, Lockpicking Village, automated airgun target contest, the Wall of Sheep and the Wireless Village. Definitely going next year!

UPDATE:
Videos of all of the presentations have been posted here.