Ryan Kavanagh's /dev/brain—Planet Ubuntu feed

Setting up full-disk encryption in OpenBSD 5.3

2013-03-26T13:40:54-04:00

I recently decided to try out a snapshot of OpenBSD 5.3, intrigued by the notice that

softraid(4) RAID1 and crypto volumes are now bootable on i386 and amd64 (full disk encryption).

I’ve always used an encrypted LVM (as setup by the alternate CD) with a fully encrypted root on my netbook and laptop when running Debian/Kubuntu and never noticed much a performance hit. Unfortunately, I’m not the only one who has noticed a significant drop in performance with full-disk encryption on OpenBSD.

Although the steps required to setup full-disk encryption (as opposed to requiring manual intervention at boot or the equivalent of loop-mounted encrypted images via vnconfig(8)) on OpenBSD are not explicitly documented anywhere, searching through the mailing lists, man pages, and trial and error provided the following steps:

Boot from your install media. Select the (S)hell option.
Assuming the disk you’re installing to is ”wd0”, run fdisk -i wd0 if you’re on an i386. If I’m not mistaken, you can skip to the following step otherwise.
Run disklabel -E wd0. Create a label (typically b) of the appropriate size of type swap for swap, and then add a label a using the remainder of the space of type RAID. We aren’t going to put swap on our crypto device since OpenBSD has encrypted the swap partition by default since 2005.
Setup the crypto volume as described in bioctl(8). You will be prompted for a passphrase and presented with a message along the lines of softraid0: SR CRYPTO volume attached as sd0 on successful setup. I used bioctl -c C -l /dev/wd0a softraid0 where - -c C specifies a device with raidlevel “CRYPTO”, - -l /dev/wd0a specifies the label of type RAID created in step 3 to use for the CRYPTO device, and - softraid0 is the softraid device to configure (this value should work unless you already have a softraid0 device).
exit from the shell and (I)nstall as normal, installing to the disk sd0 (or whichever disk the crypto volume got attached as). In the partitioning dialog, you may need to manually adjust things since the auto-partitioning creates an additional swap label on sd0, which we don’t need given we already have wd0b.
Using ed(1), add the line 1a8bab44e9cc178d.b none swap sw to /mnt/etc/fstab, assuming that 1a8bab44e9cc178d is the DUID associated with wd0 (you can find this out by running sysctl hw.disknames, which will output something like hw.disknames=wd0:1a8bab44e9cc178d,cd0:,rd0:6ce80c78714fa32f,sd0:fce7bfa23c8ec20d; just search for the DUID associated with wd0). Alternatively, wait until you boot into your system, and add it to your /etc/fstab using vi(1) or mg(1).
Reboot, and you’ll be prompted for your passhprase. You’re done!

Here’s a screenshot of steps 1 through 4.

Here’s a screenshot of the passphrase prompt, after successfully entering the passphrase:

If I’ve made any errors in the steps above, please point them out in the comments below or by sending me an email.

Search RCS and CVS ",v" files with rcsgrep

2013-02-20T08:52:09-05:00

A few years ago I was doing research comparing how large software distributions handled shared object libraries, and studied Debian, FreeBSD, and Ubuntu. Extracting data about Debian packages was easy thanks to Peter Palfrader’s snapshot.debian.org service, which provides a machine-usable interface to Debian’s package history. FreeBSD’s data is equally accessible, albeit in a less pleasant format: their ports tree was stored in CVS until July 2012. One could easily rsync a copy of the ports tree’s CVS repository to a local machine to analyze the data. This left you with a local tree full of ,v files, each corresponding to the history of a given file with at that location. I needed to extract all kinds of data from a tree full of these files, such as what revisions contained lines matching a regex, when these revisions were checked in, any tags associated with it, etc. To make things easier, it also helped to know the line numbers of the matching lines. Hence the birth of rcsgrep.

rcsgrep is a Python script that makes use of Paul McGuire’s fabulous pyparsing library. It allows you to search a RCS file (the ,v file format used by RCS and CVS to store revision history) using a Python regex, and the output format is customizable to allow printing only certain kinds of information, such as the revision number, the line number, the matching line, the line’s author, the date it appeared, any tags associated with the line, and (useful when running over a large number of files) the file name. To make machine parsing (using AWK of course) easier, you can also specify the column separator.

For example, I entered the lines “The quick brown”, “fox jumped over”, “the lazy dog. Woof!” into the file abc, checking in the changes after each line. The invocation ./rcsgrep -s ' ' -f rlLda '.*' abc,v, with spaces for column separation, and format options r is for revision, l for line number, L for line contents, d for date, and a for author, outputs:

1.3    1    The quick brown    2013.02.20.14.24.09    ryan
1.3    2    jumped over the    2013.02.20.14.24.09    ryan
1.3    3    lazy dog. Woof!    2013.02.20.14.24.09    ryan
1.2    1    The quick brown    2013.02.20.14.23.48    ryan
1.2    2    jumped over the    2013.02.20.14.23.48    ryan
1.1    1    The quick brown    2013.02.20.14.23.25    ryan

I’m particularly proud about my grep() function in rcsfile.py, which goes through each revision, tracking additions and deletions, but only keeping the lines matching the regex in memory. In any case, rcsgrep is licensed under the ISC license and can be found on github.

Addendum: I learned after the fact that O’Reilly’s “UNIX Power Tools” offers something similar by the same name, except that it is runs several processes, such as co, grep and sed, as opposed to a single Python script.

Fixing mutt header cache

2012-07-03T10:47:37-04:00

Every now and then I have to kill mutt, which if done at the wrong time, can leave the header cache in an open state. Then attempting to restart mutt will cause it to hang at the "Selecting INBOX" stage. If mutt is compiled with support for the Tokyo Cabinet header cache backend (as it appears to be in Debian), a quick fix for this issue is to repair the header cache database using the Tokyo Cabinet utility tcbmgr from the tokyocabinet-bin package. Simply running tcbmgr optimize -nl ~/.mutt/cache/headers does the trick, and is much nicer of a fix than deleting the header cache.

mutt-fetchbug: fetch BTS bug reports from mutt

2012-01-05T15:58:00-05:00

I've been a longtime mutt user, but have gotten somewhat annoyed of having to open a new terminal when I want to read or reply to a Debian bug with mutt (using 'bts show 123456'). How nice it would be to be able to fetch a bug report from within mutt! And so, ladies and gentlemen, I present to you mutt-fetchbug. It's extensively based on Zack's mutt-notmuch script (a nice interface between mutt and notmuch for searching mail, I highly recommend it).

By adding a line like the following to your ~/.muttrc, you can simply press "F7" and enter a bug number from your mutt index in order to fetch and display a bug report's mbox file:

    macro index  \
        "unset wait_key~/bin/mutt-fetchbug --prompt search~/.cache/mutt_btsresultsset wait_key" \
        "fetch bug(s) (using bts show)"

The above presupposes you've downloaded mutt-fetchbug and placed it with executable permissions in ~/bin.

Those interested in making improvements or extending mutt-fetchbug are more than welcome to do so, it's licensed under the GPLv3+. It can be fetched from my git branch.

Update: Fixed the broken link to mutt-fetchbug, thanks Christian for the heads up.

New Kubuntu Wiki Theme

2011-10-13T08:20:18-04:00

The Kubuntu Wiki has a new wiki theme based off of the Ubuntu Light wiki theme found on the Ubuntu Wiki. If you want to make this theme your default theme, head to your preferences page and set your preferred theme to kubuntu-light. Thanks to all involved in making this theme a reality!

Holiday MOTD

2010-12-06T20:23:00-05:00

Since it's the holiday season, I thought I'd greet my server's users with a colourful MOTD. Feel free to use it by appending it to your current MOTD.

You can do so with the command sudo sh -c "curl http://ryanak.ca/files/holiday-motd >> /etc/motd" on normal UNIX stations, sudo sh -c "curl http://ryanak.ca/files/holiday-motd >> /etc/motd.tail" on Debian/Ubuntu systems.

It'll end up looking a little like this (to get the full effect, including blinking, run curl http://ryanak.ca/files/holiday-motd):

Happy holidays!

Numbering paragraphs in LaTeX

2010-12-05T13:45:00-05:00

Recently, I was asked to write an essay and to number its paragraphs. I couldn't find anything premade, so I worked something up using the ledmac package. It isn't perfect (you need to restart it after using the quote environment, etc.) but it did the trick for a simple essay. In case anybody else is looking to do something similar, here's how I did it:


                    \documentclass{article}
    \usepackage{lipsum}
    \usepackage{ledmac}
    % \reversemarginpar % Uncomment for in the margin
    % \setlength\marginparwidth{2em} % Uncomment for in the margin
    \newcounter{para}\setcounter{para}{0}
    \newcounter{thispara}\setcounter{thispara}{0}
    \newcommand*{\newpara}{ %
      \refstepcounter{para}
      \setcounter{thispara}{\value{para}}
      % \marginpar} % Uncomment for in the margin
      \textbf{\thepara. } % Comment for in the margin
    }
    \begin{document}
    \title{Lorem Ipsum}
    \author{Ryan Kavanagh}
    \maketitle
    \everypar{\noindent\newpara}
    \lipsum[1]
    \lipsum[2]
    \lipsum[3]
    \end{document}

Paragraph numbering in LaTeX

See the instructions in the comments for having numbers in the margin.

Turnin-NG 1.1 released!

2010-09-01T22:13:00-04:00

Turnin-NG version 1.1 has just been released. Turnin-NG is an assignment submission suite written in Python and composed of turnin and turnincfg. Students can use the turnin command to submit an assignment to a course. Professors and TAs (teaching assistants) can use turnincfg to manage submitted assignments, making them easier to grade.

Version 1.1 fixes a vulnerability found in version 1.0.1. Courses can now be managed by groups of users instead of by a single account. The documentation has been expanded and numerous bugs have been fixed.

Here's the relevant changelog entry:

1.1 2010-09-01
 Global:
  * Ignore build files in Git (generated documentation, build/*, etc.)
  * Always initialize all of the course's options in the config files
  * Upgrade configuration files from old releases / installations. Preserves
    previous settings, just adds the options to the config files.
  * Accept that the legacy symlink may already exist from a previous install

 turnincfg:
  * Add support for group managed courses
  * Don't die if we receive an invalid username or group when creating courses
  * Don't strip a submission if it causes an overwrite
  * Added forgotten import (sys.exit() needs sys to be imported)
  * Don't crash when raising CalledProcessError when gpg --verify fails
  * Warn on dangling signatures
  * Warn on signatures submitted by other students
  * Set the group on the compressed tarball and extracted project directory
  * Also delete tarball on delete_project if project is compressed

 turnin:
  * Don't needlessly import chown
  * Fixed vulnerability. Set student's $HOME/.turnin-ng/list to 0600 so that, on
    off chance their home directory is world readable, other students can't find
    out what their submitted file suffixes are (and thus be able to play with
    them).
  * Print a pretty error message when we try to submit to a closed project
  * Don't crash when GPG fails to sign and we try to copy a non-existent .sig file
  * Remove stale signatures on unsigned resubmits

 Documentation:
  * Fix broken infodir entry
  * Properly install info manual with install-info
  * Expand the administrative section of the info manual

You can download the latest version of Turnin-NG here.

Easily download CTAN packages

2010-05-16T12:13:00-04:00

I use LaTeX for nearly all of my document preparation. Letters, reports, you name it. Because I'm always finding new uses for it, I find myself frequently downloading packages from CTAN (the Comprehensive TeX Archive Network).

I find it a hassle to copy a package's URL from my desktop (where I searched for it) to my netbook (where I want to download it). To solve this problem, I've written a little shell function that downloads a package for me. If you want to use it, just stick it in $HOME/.zshrc if you use zsh, $HOME/.bashrc if you use bash, etc.

ctand() {
    wget http://mirror.ctan.org/macros/latex/contrib/$1.zip && unzip $1.zip
}

To download, say, the program package, you'll want to run ctand program

Note: You'll need to run source $HOME/.zshrc or source $HOME/.bashrc after editing either to be able to use it. Or you can open a new terminal and use it there.

Kobby 1.0 beta 5 now available in Kubuntu Lucid

2010-04-14T14:26:00-04:00

Collaboratively editing a document using Kobby

Kobby 1.0 beta 5 is now available in Kubuntu Lucid. Some of you may be more familiar with a gtk alternative, Gobby.

Kobby is editor for collaboratively editing text documents and source files over a network. All users can work on a file simultaneously without the need to lock it. Kobby supports syntax highlighting for various programming and markup languages.

I hope that the UDS organizers will use the new protocol for collaborative editing so that we can participate remotely using Kobby this time around.

If you want to try it out, and don't have an infinote server to connect to, istall the `infinoted' package and start a server with the command `infinoted --security-policy=no-tls'.

Let's celebrate Human Achievement

2010-03-27T17:30:00-04:00

Human Achievement Hour

Man has progressed dramatically since the paleolithic era. Thanks to human achievements in the realms of science and technology, we live longer, healthier and more easily. In the last century alone, our life expectancy has nearly tripled, from 30 years in the early 20th century ^[1] to 81 years for Canadians today ^[2]. Medicine has eradicated small pox and has drastically reduced the frequency measles, polio andcountless other diseases. Automobiles and planes have made travelling faster and easier… we've even been to the Moon. Electric lighting means we can work and play—even when the Sun has set. The computer and the Internet ease collaboration and commerce and make the transmission of information nearly instantaneous.

Every year, hundreds of millions of people sacrifice these achievements for a single hour. How long could they survive by sacrificing them for a week? A month? A year? To those who support the anthropogenic global warming theory and believe that man has an impact on the environment, I point out that rejecting technology is the opposite of what should be done. Rather, technology should be your solution. Why participate in an event that preaches its sacrifice?

This year, instead of celebrating Earth Hour, I'll be celebrating Human Achievement Hour by enjoying the fruit's of man's mind. I'll be keeping my lights, computers and heating on. We've come a long way and I think its something worth celebrating.

To celebrate Human Achievement Hour, Saturday March 27th 2010 at 20:30, you can:

leave your lights on, watch TV, eat dinner or otherwise enjoy the fruits of man's mind;
spread the word;
join HAH on Facebook.

1. Life expectancy, Wikipedia

2. Life expectancy, Google Public Data

ASCIImations from 1968 and present day

2010-03-22T15:59:00-04:00

Yesterday, Slashdot reported on an ASCII art animated cat from 1968, made by printing hundreds of computer generated images of an ASCII cat and photographing them onto cinefilm. You can check it out on Youtube.

For those interested in ASCIImation, you can also watch a Starwars ASCIImation over telnet by connecting to towel.blinkenlights.nl (IPv4 or IPv6) on port 23. More about it here.

Anybody know of other cool ASCIImations?