And now I know…

August 15th, 2010

… why it is a good idea, nay a must, to keep a live cd around.

After a system upgrade earlier tonight I got a message about an new sudoers file (placed in /etc/sudoers.pacnew) that had been downloaded and wanted me to take a look at it.

So I made a couple of mistakes here:

  1. I didn’t stop to think it through enough, which lead to…
  2. I didn’t edit the new sudoers file first before moving it in place, but also…
  3. I MOVED the original file to a backup spot, instead of COPYING it
  4. Of course, none of this would have mattered, had I logged into the root account before doing stuff, instead of relying on sudo… for tampering with sudoers… I’m so smart! S-M-R-T.

So, instead of doing things the right way, i.e.:

$ sudo su
# cp /etc/sudoers /etc/sudoers.bak
# vim -p /etc/sudoers.pacnew /etc/sudoers.bak
# mv /etc/sudoers.pacnew /etc/sudoers

what I did was (NEVER EVER DO THIS!):

$ sudo mv /etc/sudoers /etc/sudoers.bak

The instant I hit enter, I got a sinking feeling, knowing I’d frakked up. Attempting to undo the damage

$ sudo mv /etc/sudoers.bak /etc/sudoers

only resulted in an error message:

sudo: can’t stat /etc/sudoers: No such file or directory
sudo: no valid sudoers sources found, quitting

There might have been a root password set, but if so I couldn’t for the life of me remember it. Great! What now?

I knew of course I could access the filesystem from a live-cd but I hadn’t actually needed to do this before, so I was unsure about how to proceed once in the live-session (I had a vague notion about having to fiddle with chroot, something I have not done much of) but as luck would have it, hesa was online and helped me out (no need for chroot, just mount the partition, move some files, unmount, restart.)

Upon booting into Knoppix it was a small matter of

  1. remembering which of the disk, and partitions which housed the root-partition, and
  2. mounting it:

$ sudo mount /dev/sdb3 /media/sdb3
$ cd /media/sdb3/etc/
$ sudo mv sudoers.bak sudoers
$ cd /
$ sudo umount /media/sdb3
$ sudo reboot

Now, this would have been the sane thing to do, and start over from inside Arch… but instead I moved sudoers.pacnew into sudoers, forgot to edit it, rebooted, realized my new mistake, rebooted again (a hassle, since sudo reboot wasn’t available), remounted sdb3, edited sudoers adding myself again, and finally rebooted. And all was well.

So now I know, first hand… Live-CDs FTW!

:wq

Blinging out your terminal, adding more colors

August 5th, 2010

I recently installed Arch on my desktop, forsaking Ubuntu.

I still like Ubuntu and all, but it was an old version (Jaunty), too many packages were getting too badly outdated, and the one flaw with Ubuntu is that through bitter experiences I no longer trust dist-upgrades at all.

Arch promised a solution to this, in rolling upgrades. It hasn’t convinced me yet, but having given Ubuntu a chance for little over three years, Arch should get the benefit of a doubt until it does something pants-on-head-crazy at least ;)

One thing (among a whole host) I missed in Arch, which Ubuntu had done for me, was setting up dircolors so that when running the ls command some types of files (determined by their extension) would be colorized differently.

For instance, I quite liked that in Ubuntu, whenever I executed an ls in a directory containing tarballs, they’d all show up with a bright red color instead of the usual bland green.

It made it easier to locate them, which, in theory, would save me time.

So I fired up Firefox and asked Google for

archlinux colorize images and archives in ls

What Google returned was (among others) these two forum-posts:
https://bbs.archlinux.org/viewtopic.php?pid=597273
https://bbs.archlinux.org/viewtopic.php?id=17155

The first didn’t seem all that useful to me, which was why I went to the other post and found almost all I was looking for. The ~/.dir_colors in the second post didn’t have any TERM line for rxvt, but the first post did, so with a little copying and pasting voilà! Colors! Colors everwhere!

So what you want to be doing is:
a. edit your ~/.bashrc (or whatever shell you’re using) adding:

if [ -f ~/.dir_colors ]; then
    eval `dircolors ~/.dir_colors`
fi

b. read the dir_color manpage to understand how the ~/.dir_colors file should be formatted and what the available color codes are. I.e.:

Attribute codes:
00=none 01=bold 04=underscore 05=blink 07=reverse 08=concealed
Text color codes:
30=black 31=red 32=green 33=yellow 34=blue 35=magenta 36=cyan 37=white
Background color codes:
40=black 41=red 42=green 43=yellow 44=blue 45=magenta 46=cyan 47=white

For instance, would you want files of a certain extension to show up as blinking red text on a black background use: 05;31;40
(you probably don’t want do use that specific combination for anything ;D)

c. go crazy with colors in your ~/.dir_colors, e.g.:

# archives
.tar 01;31
.tgz 01;31
.tar.gz 01;31

# images
.jpg 01;35
.jpeg 01;35
.gif 01;35
.png 01;35

etc.

That’s all there is to it really. Changes won’t appear in any existing terminal until you source the .bashrc file but all new terminals will behave “correctly” from the start.

And in closing, a completely unrelated little Arch tip:

If you are trying to get k3b or k9copy to work and it just frakking won’t, you are most likely not part of the group “optical”.

# usermod -a -G optical YourUsernameHere

fixes that (the user added to the optical group would need to log out and in again for changes to take effect).

:wq

Försvarsmakten…

June 14th, 2010

Credit: Mikael Hedberg
Licens: CC BY-NC-SA

Prepending text to a bunch of files

June 10th, 2010

Say you have a project, say it is LaTeX, and that you intent to publish the final product.

Say that you have an upcoming deadline, and you wish the publication to be printed and available at a rather fine conference.

Say that you enter that project late in the game, and (stupidly) don’t spend a thought on the source code license, because there are not much time left until the deadline.

And then, after the deadline, say that there are some people interested in said source code. Since the final product was published under a nice license, the intent was of course always to have the source code that way as well, it just… kindof, slipped between the chairs.

So there we are, source code without any license notice of any kind. What do?

(Obviously the answer is to get a license header into the files)

Say you are lazy. Manually adding those two lines of license data, even if only to a meager count of 15 files, is a chore you’d rather avoid.

You might start experimenting with cat for instance something along the lines of

cat license somefile > somefile

You realize that that approach is full of fail, but, if you’re in luck, you work in a pretty cool place, and get to have pretty cool work buddies. Work buddies which are pretty good at wielding bash, and concoct stuff like:

for f in *.tex; do (cat license; cat $f) > ${f}.new; mv ${f}.new $f; done

The result, finally, speaks for itself.

Easy permission sanitizing using chmod

June 3rd, 2010

Let’s say you have a web app, such as WordPress, and you have installed it on your own server. You are of course security conscious, so you wish to have the permissions set up correctly, no exceptions. this usually means 755 (rwxr-xr-x) for directories and 644 (rw-r–r–) for files.

The way I used to solve this, on every server I worked, I set up a small shell script (sanitize-perms.sh) along the lines of:

#!/bin/sh
TARGET=$1
find $TARGET -type f | xargs chmod 0644
find $TARGET -type d | xargs chmod 0755

This worked well, with one huge caveat: What if you, somewhere in that directory structure had a file which needed to be executable?

I don’t know if such a case exists in WordPress, I’ve used that script on a couple of WP installations without any noticeable side-effects, but it’s obviously a flawed approach.

I’ll side-track this post a bit, since it is relevant to the overall post, that I, through identi.ca, stumbled upon this blog post (which is awesome by the way, go read it!) about why LaTeX is so cool, and why it can be useful writing your résumé using it.

Just by chance I continued into Dan’s code section, and long story short, I found some cool stuff in his .bashrc file. Most notably this little beauty:

# sanitize - set file/directory owner and permissions to normal values (644/755)
# Usage: sanitize <file>
sanitize() {
	chmod -R u=rwX,go=rX "$@"
	chown -R ${USER}.users "$@"
}

I personally, for some reason, have always tended more to the octal representation than the [ugo][+-=][rwx] syntax, but that single chmod line is so outstandingly brilliant that I am almost forced to switch.

In one fell swoop Dan’s command does what I need two commands (really, with the xargs and I suppose one new process per found file/directory to execute chmod, my script needs a lot of processes) to accomplish.

The magic happens in that capital X, which is defined in the chmod man-file as: “execute/search only  if  the file is a directory or already has execute permission for some user”.

Directories automatically receives the executable flag, and any file which already has it, maintains it. Bloody brilliant!

Many thanks to Dan for sharing his configuration files, one of these days I’ll have to follow his good example.

Awk hackery

May 30th, 2010

I’ve always leaned more towards sed than awk, since I’ve always gotten the impression that they have more or less the same capabilities, just with different syntaxes.

But the more command line parsing I do, the more I’ve begun to realize that there are certain things I find easier to do in awk, while some things are easier with sed.

One of these things, that I find awk a better tool for, is getting specific columns of data from structured files (most often, but not limited to, logs).

I have for some time known about

cat somefile | awk '{ print $1 }'

Which will output the first column of every line from somefile. A couple of weeks ago, I needed to fetch two columns from a file (I can’t remember now what the file or task was, I’ll substitute with a poor example instead)

ls -l | awk '{ print $1, $8 }'

This will give you the permissions and names of all directories and files in pwd. One could of course switch places of $1 and $8 (i.e. print $8, $1) to get names first and then the permissions.

Recently I found myself needing to find all the commands executed from a crontab (part of a script, to create another script which was to verify that a migration had gone right, by allowing me to execute those commands whenever I wanted, and not just whenever the crontab specified)

Luckily for me, and this blogpost ;) , none of those commands were executed with parameters, and since I am too lazy to actually count how many fields there are in a crontab file, I got to use:

crontab -l | grep '^*0-9' | awk '{ print $(NF) }'

Which lists the content of the present users crontab, finds all lines which either begin with a number or an asterisk, and then prints the last column of that line. Magic!

Scripting NetworkManager

May 2nd, 2010

At home I have a laptop acting as a central mercurial server (from here on referenced as myserver). This is great, being able to synchronize my desktop and netbook through an intermediary gives me three hard drives on which my data is stored. The setup was, however, not especially useful since my netbook goes with me almost everywhere (read: outside the home network).

This presents a problem as when I clone a repository and my .hg/hgrc file now contain a link to the server using at best a name defined in my /etc/hosts file, and at worst a local IP address (192.168.0.xyz).

I had long since done away with the IP address links, replacing them with hostname links (i.e. ssh://hg@myserver/repo instead of ssh://hg@192.168.0.xyz/repo) which was the first step. Now all I needed to do was make myserver accessible through the firewall (forwarding the appropriate port (since we are talking ssh here that would be 22, unless you are paranoid and have switched the port which sshd listen on) in the router, make the server listen on that port, and finally come up with a way to dynamically switch what IP address myserver should point to.

There probably is a better solution for this, configuring the router to make all this would be one way, but I’m not sure that my router has that capability, and even if it did, I’m not sure I would have the skills to configure it. Software and shell scripting on the other hand, those are things I know how to work.

My solution, was simply to create two new hosts-files in /etc :

# touch /etc/hosts-{home,not-home}

and then insert appropriate information into them such as

# echo "myserver    192.168.0.xyz" > hosts-home
# echo "myserver    aaa.bbb.ccc.ddd" > hosts-not-home

(where aaa.bbb.ccc.ddd is my external IP as seen on http://whatismyip.org/).

I also made a backup of my current hosts file just to be on the safe side

# cp /etc/hosts /etc/hosts.bak

So, the overall plan is that once I’ve identified if I am at home, or somewhere else (i.e. not home) I will overwrite (copy) /etc/hosts with the “correct” hosts-file (-home or -not-home).

And this is where NetworkManager comes into play. Every time there is a change in the connection status of any identified network interface, NetworkManager reacts by executing every script found in /etc/NetworkManager/dispatcher.d/.

Every script is called with two parameters, the first being which interface changed (e.g. “eth0″, “wlan0″, etc.) and the second being how it changed (e.g. “up”, “down”). What I wanted to do was, every time a new interface went up, check if it had connected to a device in my home network (wlan or ethernet).

So, how do you distinguish different devices in a simple manner? For me, distinguishing different MAC addresses (two known addresses at home (wlan and ethernet), and every other address was essentially not home) would do nicely.

Enter the “arp” command. From it, using flag “-i” (for interface) I can discover the IP and MAC address  of the device I’m connected to. When I first discovered the example usage used flag “-a” as well which, according to the man-page, outputs results in an alternate BSD style format, using no fixed columns. This had some benefits, but as I read it again now, the “no fixed columns” remark begin to worry me. The fact that it has worked so far isn’t good enough proof that it will always work. Needless to say, ymmv.

Putting it all together:

#!/bin/sh
# filename: /etc/NetworkManager/dispatcher.d/99switchHosts.sh

# Unless the status is "up" I'm not interested
if [ "$2" != "up" ];
then
    exit 0;
fi

# I am interested in two interfaces, eth0 and wlan0
# each connect through different IP addresses and
# these devices have separate MAC addresses
if [ "$1" == "eth0" ];
then
    TARGETIP="192.168.0.1"
    TARGETMAC="aa:bb:cc:dd:ee:ff"
else
    TARGETIP="192.168.1.1"
    TARGETMAC="12:34:56:78:90:ab"
fi

# Sometimes arp hasn't gotten hold of any IP/MAC addresses
# pinging and waiting does wonders for this
ping -c 1 "$TARGETIP" 1>/dev/null 2>&1
sleep 5
CURMAC=$(arp -ai "$1" | grep "$TARGETIP" | awk '{ print $4 }')

while [ -z "$CURMAC" ];
do
    # Again, arp might not have gotten hold of IP/MAC
    sleep 3
    CURMAC=$(arp -ai "$1" | grep "$TARGETIP" | awk '{ print $4 }')
done

# If MACs match, it is a known address, i.e. home
if [ "$TARGETMAC" == "$CURMAC" ];
then
    cp /etc/hosts-home /etc/hosts
else
    cp /etc/hosts-not-home /etc/hosts
fi

exit 0

This should be put in /etc/NetworkManager/dispatcher.d/99switchHosts.sh (and the script should be made execute:able)

# chmod +x /etc/NetworkManager/dispatcher.d/99switchHosts.sh

All done, not too shabby eh?

NetworkManager and password changes

March 20th, 2010

Last week I changed passphrases (long overdue, I should really know better) which included the user passphrase on my netbook.

Today I attempted to connect to my encrypted WLAN at home, which prompted me with a cryptic password prompt telling me that “Network Manager Applet (nm-applet) wants access to the default keyring but it is locked”.

To my knowledge, the only two keyrings I’ve ever set any passphrase to is to GPG and SSH, and none of those to passphrases (old or new) worked. I was stumped. What other password/passphrase could they mean? What “default keyring”?

I was fairly certain that this was not a case of me setting a passphrase only to never use it and thus forget it, so I started getting a chilling feeling that I might have corrupted files on the disk.

Luckily my google-fu was with me and I found this post, which made everything clear.

What happened was simply that when I changed the passphrase for my user, this change didn’t propagate to the “default keyring” storing network passphrases, so the password for the default keyring was still set to the old user passphrase (which I didn’t try, since I didn’t even consider them being the same)

Honestly, although it is of course a VERY GOOD idea to encrypt such data, I can’t say that it was smart of the developers to “magically” set the default keyring password to the same as the system users (at least without notifying the user about this), or alternatively, to not have it updated along with the system user password…

If you have magic in one place you need to make sure the magic persists all the way, otherwise you just end up with a confused user.

The reason I did not immediately connect the dots between user passphrase change and default keyring password prompt was simply that almost an entire week had passed before I ran into problems (I guess the only secured network I ever connect to is my own, and I obviously haven’t used it in that time…)

Anyway, what finally worked for me (as outlined in the link above) was:

  1. Start “seahorse”
  2. Goto “Passwords” tab
  3. Right-click on “Passwords: login”, chose “Change password”
  4. Enter old system user password, and (preferably) the new system user password (unless you want to be pestered with the default keyring locked password prompt every time you connect to a secured network)

User-friendly magic is cool, undocumented magic… not so much :(

I get all the fun questions

March 14th, 2010

A friend just pinged me wondering if I knew of a way to convert “windows file names” to “Linux file names” (i.e. the character encoding of the file names).

My first thought (stupidly) was “surely this must be solvable using ls > dirlist and some hackery involving tr or sed“.

A quick test on a string (actual file name) sent over from my friend luckily shattered that idea.

So my next thought was simply “surely other people have had this problem before me (us).”

My google-fu was with me on this day, as I soon found convmv, a nifty little tool for changing the character encoding of filenames, pretty much what my friend wanted to do :D

He could of course just have searched for it himself, but I am quite glad that he didn’t as then I wouldn’t have gotten to know about it, this might save me valuable time some other day when my google-fu isn’t with me ;D

Vim + snipMate

March 14th, 2010

As I sat down in front of the computer today and started reading the identi.ca backlog I came across this gem, which lead here, about how to turn Vim into a Python IDE. The blogpost itself contained many useful tips/tricks (the code folding script being on the top of that list), but as is often the case, the really cool stuff gets unearthed in the comments, like this one about snipMate. Also, excellent video presentation is excellent!

That post also lead me to another post on that blog which inspired me to finally start looking into ctags, which is a two step installation, one being installing the package exuberant-ctags from the repository, and the other fetching this script.

All I need to do now is try to learn the key mappings M^e, f, M^f, and TT/F4 ;D oh, and to tab a lot ;D