Archive for the ‘OS X’ Category

Good on you, Wolf Rentzch

Wednesday, May 12th, 2010

rentzsch.tumblr.com: [C4 release]:

“With resistance to Section 3.3.1 so scattershot and meek, it’s become clear that I haven’t made the impact I wanted with C4. It’s also clear my interests and the Apple programming community’s interests are farther apart than I had hoped.”

I signed up for an iPhone developer account this spring, just before the change to the developer agreement which included the revised 3.3.1. The change was galling. Never mind Flash; Apple was showing what it thinks of its developer community. “We can further limit you any time we want. Not a big deal. Just rewrite your code.”

I bought back into Apple in 2002, drawn by OS X. OS X developers are still free from the strictures imposed on iP* developers. Let’s hope it stays that way. If not, then it’s back to Linux for me.

Meanwhile, here’s hoping for good things from HP/Palm, Notion Ink, and the Google Android ecosystem.

Upgrading a VMware VM to Ubuntu 10.04

Saturday, May 1st, 2010

Yesterday I tried to update an Ubuntu 9.10 VM, running VMware Fusion 3.1 beta, to Ubuntu 10.04. Two significant problems arose:

  1. At the login screen, the keyboard didn’t work
  2. the boot process paused because hgfs could not be mounted

Here’s a log of the problems and worries encountered, together with their solutions.

Grub Configuration

Late in the Ubuntu upgrade process a warning appeared, saying Grub could not be configured (or words to that effect — I wasn’t paying close attention) and asking for confirmation that the installation should continue without any changes to grub’s configuration. I confirmed. After all, this was a VM instance with only a single operating system installed.

Can’t Mount hgfs

On reboot the console reported that a problem had occurred while mounting /mnt/hgfs. If offered to continue without mounting, and I accepted.

Unresponsive Keyboard

When the Ubuntu login screen appeared, the keyboard was unresponsive. This made it difficult to enter my password.

Thanks to this thread in the VMware Communities forums, I was able to login after enabling the onscreen keyboard. (Click on the accessibility icon — the “little man in the circle” — check the onscreen keyboard toggle, and restart the VM.) Once logged in, the keyboard worked just fine.

VMware Tools Out of Date

VMware detected that VMware Tools were out of date for this VM. (It may have reported that they weren’t installed. I can’t recall.)

No xorg.conf?

I started the VMware Tools installation process and encountered a warning I hadn’t seen before:

Detected X.org version 7.6.6.

The configuration file /etc/X11/xorg.conf can not be found.  Do you want to create a new one? (yes/no) [yes]

Not sure what to do, I accepted the default response.

Keyboard Fixed

After the tools finished installing I rebooted the VM. This time, at the login screen the keyboard worked. A wild guess is that the VMware Tools re-installation fixed this problem by reconfiguring the console.

Fixing hgfs Ownership

/mnt/hgfs had been auto-mounted, but as usual the ownership and permissions were wrong. I patched this as documented in a previous post.

Fixing hgfs automount Failure

Ubuntu continued to complain at startup that it had failed to mount hgfs. It continued to request user permission to skip mounting hgfs. But once the boot completed /mnt/hgfs was mounted with correct permissions.

To prevent Ubuntu complaining of failure to mount hgfs, I added a “nobootwait” option to its entry to tell mountall(8) not to hold up the boot for this filesystem. The entry now reads:

.host:/ /mnt/hgfs vmhgfs rw,ttl=1,uid=my_uid,gid=my_gid,nobootwait 0 0

Now Ubuntu boots without complaint, the keyboard works, and /mnt/hgfs is mounted with correct ownership and permissions.

VMware Fusion 3.0.1, Ubuntu 9.10 and shared folders

Wednesday, March 3rd, 2010

In VMware Fusion I share a folder with many of my VMs. In a new Ubuntu 9.10 VM, that shared folder is always mounted with the wrong owner and group IDs. Here’s a fix.

After booting the Ubuntu VM for the first time, I downloaded and installed the latest vmware-tools distribution, using the VMware “Virtual Machine -> Install VMware Tools” menu item. Then I added an entry to /etc/fstab, specifying the correct owner and group ID for the /mnt/hgfs mount point.

Unfortunately, the owner and group ID specifications were ignored. As it turns out, /etc/init.d/vmware-tools does not examine /etc/fstab when deciding whether or how to mount the shared filesystem.

Here’s how I changed /etc/init.d/vmware-tools to address this problem.

Old

# Mount all hgfs filesystems
vmware_mount_vmhgfs() {
  if [ "`is_vmhgfs_mounted`" = "no" ]; then
    vmware_exec_selinux "mount -t vmhgfs .host:/ $vmhgfs_mnt"
  fi
}

New

is_vmhgfs_in_fstab() {
    if grep -q " $vmhgfs_mnt " /etc/fstab; then
        echo "yes"
    else
        echo "no"
    fi
}
# Mount all hgfs filesystems
vmware_mount_vmhgfs() {
  if [ "`is_vmhgfs_mounted`" = "no" ]; then
    if [ "`is_vmhgfs_in_fstab`" = "yes" ]; then
        vmware_exec_selinux "mount $vmhgfs_mnt"
    else
        vmware_exec_selinux "mount -t vmhgfs .host:/ $vmhgfs_mnt"
    fi
  fi
}

Mercurial, Migration Assistant, and dotfiles

Monday, February 22nd, 2010

I recently upgraded my iMac. Migration Assistant moved all of my files to the new machine without issue — or so it seemed.

I had created Mercurial repositories in a couple of virtualenv environments, to track changes locally.[1] I didn’t notice that Mercurial had put each virtual environment’s .Python file under revision control.

Shortly after completing the migration I made changes in one of these virtual environments. A quick ‘hg status’ before committing, and…

$ hg status
abort: data/.Python.i@13b27e856c38: no match found!

WTF?

After much investigation it appears that the following has happened.

  1. Mercurial represented the .Python link in its .hg/store/data directory as ._Python.i
    1. The leading underscore appears to be Mercurial’s way of noting that the ‘P’ should be capitalized.
  2. I think Migration Assistant uses ditto to copy files.
    1. ditto saw the leading ‘._’ and concluded this was an orphaned resource file.[2] So it didn’t copy the file.
  3. Mercurial knew that it was supposed to have a .hg/store/data/._Python.i file; when it couldn’t find it, it decided the repository was corrupted.

Luckily the problem cropped up before I traded in the old machine, so I was able to copy across the missing files manually.

In my experiments, the problem manifested only when the capitalized dotfile was a symbolic link, not when it was a regular data file.

Well… the above write-up contains several unproven assertions, e.g. about the conditions under which Mercurial will create a ‘._’ filename. I’m not really sure whether this is a bug or merely a caveat regarding an obscure corner condition. For now, the easiest workaround is:

Don’t Track virtualenv .Python Files With Mercurial.

[Update 2010/02/22: Someone has already filed this as a Mercurial bug.]


[1] (Mercurial makes a great filesystem “undo” facility, useful even for directory trees which you never intend to share with anyone else.)

[2] OS X still supports something ike resource forks. In tarballs and other non-OS Extended filesystems, resource forks are represented as dot-files with a leading underscore. (See Norman Walsh’s blog for more info.) For example, the resource fork for a file named ‘foo.txt’ might be ‘._foo.txt’.

Python, webbrowser, OS X, and “execution error”

Thursday, January 28th, 2010

Once in awhile I write Python scripts which produce static HTML pages and then use the webbrowser module to open the results in a browser. I do this just rarely enough that, when the scripts fail on OS X with the following error, I have to waste time diagnosing the problem:

execution error: An error of type -2110 has occurred. (-2110)

Despite the obscure phrasing, the cause of the error is simple. I’m passing a filesystem pathname to webbrowser.open, when I should be passing a file: URL.

This produces the error:
webbrowser.open("/path/to/file.html")

This works:
webbrowser.open("file:///path/to/file.html")

TextMate, Emacs and META indent-region

Thursday, November 12th, 2009

I haven’t used GNU Emacs very much since switching to TextMate in 2005. One Emacs feature which I really miss in TextMate is indent-region. It lets you take an entire region of code, whatever its language, whatever its mix of tabs and spaces and indentation widths, and re-format it using your preferred indentation style.

But wait! Emacs has a batch mode, and you can drive it from TextMate. Many thanks to Gragusa’s Things for showing the way.

The post on Gragusa’s Things is specific to R code, but I’m more interested in re-formatting C and C++ code. Here’s my first cut at a general TextMate Bundle to re-format code regardless of the source language:

#!/usr/local/bin/python2.6
"""
Use Emacs to re-indent regions of the current buffer.
Inspired by 

http://gragusa.wordpress.com/2007/11/11/textmate-emacs-like-indentation-for-r-files/

"""
import tempfile
import os
import sys
import subprocess
# Use the same filename extension so Emacs will know which
# mode to use.
ext = os.path.splitext(os.environ["TM_FILEPATH"])[-1]
outf = tempfile.NamedTemporaryFile(suffix=ext, delete=False)
pathname = outf.name
outf.write(os.environ["TM_SELECTED_TEXT"])
outf.close()
args = [
    "emacs", "-batch", pathname,
    # Assume no emacs-startup.el
    "--eval", "(setq indent-tabs-mode nil)",
    "--eval", '(c-set-style "java")',
    "--eval", "(setq c-basic-offset 4)",
    "--eval", "(indent-region (point-min) (point-max) nil)",
    "-f", "save-buffer"]
p = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if p.returncode:
    print(err)
inf = open(pathname, "r")
sys.stdout.write(inf.read())
inf.close()
os.remove(pathname)

NB:

  1. Due to the use of the delete=False keyword argument to tempfile.NamedTemporaryFile, this command bundle requires Python 2.6+.
  2. TextMate on OS X 10.5 won’t, by default, have /usr/local/bin in its path; hence the pathetic shebang.

Anyway, install this as a new TextMate command bundle, assign a Key Equivalent such as ⌘-Shift-R, and enjoy.

VoodooPad and TextMate

Monday, February 16th, 2009

VoodooPad’s new HTML-savvy application-specific URLs are a great way to retrace your steps in TextMate.

I keep my worklog in VoodooPad. I do most of my code editing in TextMate. When I’m trying to understand a new chunk of code, I often need to jump around through the code base.

It’s hard to keep track of where I’ve been, so I can back out of a code path once I understand what it does. Until now I’ve just jotted down pathnames and line numbers in my worklog, so I could manually retrace my steps.

Puzzle Pieces

  • VoodooPad URLs. They let you inject HTML content — including hyperlinks — into an open VoodooPad document.
  • TextMate URLs of the form txmt://open?url=file://pathname&line=lineNum;column=colNum. Open one of these URLs and TextMate will open a window showing the specified line (and column, if specified) of the specified file.
  • Textmate commands and key equivalents.
    TextMate lets you define custom commands and trigger them with keyboard shortcuts that you specify.

Put these all together and what do you get?

Scenario

  • Select some code of interest in TextMate
  • Type your keyboard shortcut
  • A hyperlink pointing to the selected code is inserted into your VoodooPad document

Later, when you want to get back to that chunk of code, just click on the hyperlink in VoodooPad. TextMate will open the document and jump to the line of code referenced by the hyperlink.

Granted, you’re probably going to edit that code someday; and then your bookmarks will break. But this is a handy way to leave a trail of breadcrumbs while you’re trying to decipher a new body of code.

Code

Here’s a Python script which implements this “Bookmark in VoodooPad” capability in TextMate. You can use it by opening TextMate’s Bundle Editor (Bundles -> Bundle Editor -> Show Bundle Editor), creating a New Command using the tool menu at the bottom left of the Bundle Editor window, and pasting it into the resulting Command(s) text area:

#!/usr/bin/env python2.6
import os, datetime, urllib, subprocess

escape = urllib.quote

# What Textmate location are we bookmarking?
path = os.environ["TM_FILEPATH"]
lineNum = os.environ["TM_LINE_NUMBER"]
_tmURLTemplate = "txmt://open?url=file://{path}&line={lineNum}"
tmURL = _tmURLTemplate.format(path=path, lineNum=lineNum)

# Which VoodooPad worklog page should we add it to?
vpPageName = escape(datetime.date.today().strftime("%Y-%m-%d"))

# What text should Voodoopad show for the link?
currLine = os.environ.get("TM_SELECTED_TEXT", os.environ["TM_CURRENT_LINE"])

# How should the HTML be formatted?
_vpHTMLTemplate = '<a style="font:12px helvetica" href="{tmURL}">{currLine}</a>'
vpMarkup = _vpHTMLTemplate.format(tmURL=tmURL, currLine=currLine)

# What URL do we open to inject the HTML into VoodooPad?
_vpURLTemplate = "voodoopad:html={vpMarkup}&page={vpPageName}"
vpURL = _vpURLTemplate.format(vpMarkup=escape(vpMarkup), vpPageName=escape(vpPageName))

subprocess.check_call(["open", vpURL])

Specify a key equivalent such as ⌘-T (Command-T) and you’re off to the races.

[Like many posts to this blog, this entry is also posted at http://bottledtext.blogspot.com.]

VoodooPad bookmarklets now support HTML

Monday, February 16th, 2009

VoodooPad bookmarklets are very useful. But they let you paste only plaintext from your web browser into your document — until now.

VoodooPad Application URLs

VoodooPad understands application-specific URLs of the form voodoopad:description=encoded_text&page=name_of_VoodooPad_document_page. When you open one of these URLs, VoodooPad inserts the encoded text into the indicated page of the current VP document.

You can open voodoopad: URLs from other applications, such as a web browser. That’s what’s so cool about VP bookmarklets — they let you quickly select some text in your web browser and paste it into a VP document, along with other information such as the URL where the text originated.

The only problem is, the description parameter can contain only plain text. So if you’ve selected part of a web page that contains hyperlinks, those hyperlinks won’t come along for the ride.

New Feature

Gus Mueller, VoodooPad’s creator, recently released a new build. It adds a new html parameter to VoodooPad URLs. Now you can select part of a web page and transfer it as HTML into your VoodooPad document.

Gus summarized the new feature thus:

The format of the bookmarklet is like so:
voodoopad:page=junk&html=<b>hello</b>%20world!

Here’s expanded code for a JavaScript bookmarklet which takes advantage of the new parameter. It works in both Safari and Firefox 3.0.x:

javascript: function pad(v) {
    var res=String(v);while(res.length<2) {
        res='0'+res;
    }
    return res;
};
var d=new Date();
var page=(String(d.getFullYear())+'-'+
          pad(d.getMonth()+1)+'-'+ pad(d.getDate()));
function selHTML() {
    var wr=document.createElement('div');
    var c=window.getSelection().getRangeAt(0).cloneContents();
    wr.appendChild(c);
    return wr.innerHTML;
};
location.href='voodoopad:html='+encodeURIComponent('<div style="font:12px helvetica">'+location.href+'<br/>"""<br/>'+selHTML()+'<br/>"""</div><br/>')+'&page='+page;

Worth noting: in this snippet the html content is wrapped up in a div which has an explicit font specification. The bookmarklet is transferring a chunk of HTML, but that chunk doesn’t include any of the CSS from the original web page. So if you don’t specify a style for the wrapper div, VoodooPad will use its own default style for the current VP page. (I think that’s something like Times Roman 10px.)

Django snippets: DebugFooter middleware with textmate links

Tuesday, January 20th, 2009

Django snippets: DebugFooter middleware with textmate links:

“This version adds TextMate links : if you are working on your local machine and using TextMate you can click on the template paths and they will be opened in TextMate. This speeds up development time considerably!

also, this works with django 1.0″

Very useful.

Removing old man pages from OS X Leopard

Saturday, November 1st, 2008

Back in January I posted about Leopard, ‘ls -l’ and extended attributes. The gist was that my man pages were out of date, so they didn’t explain what the @ character represented, in ‘ls -l’ output.

Some threads on the Apple discussion boards have investigated why the man pages don’t get updated. The conclusion is that Leopard installs a lot of gzip-compressed man pages without deleting the corresponding uncompressed man pages. (It may be that this is true only for upgrade installs.) A recommended workaround was to scan through the man page directories with a shell command, re-locating all uncompressed man pages which had corresponding compressed man pages.

I decided to be cowardly in doing this. Here’s a Python script which moves each uncompressed man page only if it is older than its corresponding gzipped man page.

The script must be run with superuser privileges, e.g.

$ sudo python mv_old_man_pages.py

It processes all of the man pages under /usr/share/man, moving old, uncompressed man page files to ~/tmp/old_man_pages.

I would post this to the Apple discussion forums, but it’s really overkill. The short shell scripts already posted should work fine, if you ignore the harmless error messages and process each man/man* directory manually.

#!/usr/bin/env python
# encoding: utf-8
"Remove obsolete uncompressed man pages on an OS X Leopard system."

import sys, os, logging

def candidateOldManPages():
    os.chdir("/usr/share/man")
    for dirpath, dirnames, filenames in os.walk("."):
        for f in filenames:
            if f.endswith(".gz"):
                fOld = f[:-3]
                if fOld in filenames:
                    yield os.path.join(dirpath, f), os.path.join(dirpath, fOld)

def findOldManPages():
    for p, pOld in candidateOldManPages():
        s = os.stat(p)
        try:
            sOld = os.stat(pOld)
        except os.error, info:
            # Some man pages may be broken symbolic links
            logging.warn("Can't stat (%s, %s): %s" % (p, pOld, info))
            if os.path.islink(pOld):
                yield pOld
        else:
            if s.st_mtime > sOld.st_mtime:
                yield pOld

def mvOldManPages():
    topDestDir = os.path.expanduser("~/tmp/old_man_pages")
    for relpath in findOldManPages():
        dirname = os.path.dirname(relpath)
        filename = os.path.basename(relpath)

        destDir = os.path.join(topDestDir, dirname)
        if not os.path.exists(destDir):
            os.makedirs(destDir)
        logging.debug("mv %s %s" % (relpath, os.path.join(destDir, filename)))
        os.rename(relpath, os.path.join(destDir, filename))

def main():
    """Mainline for standalone execution"""
    logging.basicConfig(level=logging.DEBUG,
                        format="%(levelname)s: %(message)s")
    mvOldManPages()

if __name__ == "__main__":
    main()