Posts Tagged ‘string manipulation’


Sunday, December 25th, 2011

Bash variable string operators

I had a file filled with URLs to files I needed to download. Some of the files on the list, however, had already been downloaded, so no need to do it all again.

Should be fairly easy, right? cat the file to a while loop, reading the lines one by one, extracting the filename from the URL, check that it isn’t existing already, and if it isn’t, download it with wget.

So… how do you go about extracting the filename? You could certainly use sed and store the extracted filename in a separate variable, but that seems kindof wasteful, especially in a one-liner while loop. This article provided me with another option.

${line##*/} which deletes the longest possible match from the left (which in this case means up to (including) the last “/”) i.e. everything up to the name of the file.

No can haz censorship plz

If you’d like to make it clear that you too oppose SOPA (which, fittingly, means “garbage” in Swedish) then head over to Github, pick up your very own copy of stopcensorship.js, embed it on your site, and you’re set :)

I am also noting, with some glee, that GoDaddy is catching a whole lot of flak for their support of SOPA.

The only thing companies truly understand is when you hit them where it hurts, and that is their wallets (or as some brilliant person jokingly expressed it: “stop hitting us in our quarterly reports!”), and the only way to do that, is by voting with your own wallet.

I’m so happy about the fact that more and more people are catching on to this realization that I could… shit rainbows :)

Japanese Whaling + Tsunami disaster relief funds = disgusting

Just when I didn’t believe it possible for the Japanese whaling industry to appear as bigger scumbags than they already appear (yes, it is a quite one-sided story we’re getting from “Whale Wars” but according to National Geographic, the whalers have gotten the chance to tell their side of the story, and it would seem likely that they decline because they know full well just what type of scumbags they are… but hey, that’s just my opinion…) they go and do even more disgusting stuff, like using money from the tsunami relief donations to hire security ships to keep the Sea Shepherd Conservation Society away from their dirty business…



Sunday, May 29th, 2011


I have come up with a way to achieve the changes I want, but without introducing sqlite3 as a dependency, and a big part of the solution is to use bash arrays.

Furthermore, I have been thinking about how to, if possible, get timetrack to automagically start a new session when a file in the project is opened.

This won’t help anyone to start the timetracker when thinking about the project, but at least when physically transferring code from brain to hard drive, and the lead I am working off of is inotify.


During this weeks FSCONS meeting jonaso jokingly suggested that I’d try to write an issue tracker in bash. (damn you! ;))

Of course my mind started wandering and although there is no code to back it up, I have a couple of rather interesting ideas about how to pull it off.

For this project, sqlite is the way to go, but I was somewhat worried about concurrent access which I probably shouldn’t be.

My tests indicate (oh yeah, so there exist code, just not any actual issue tracking code) that the sqlite3 library is intelligent enough to lock the file, and thus doesn’t allow concurrent access.

I’ll still need to devise a way of detecting these locks, and have the second script stand in line and try again later, but that should be trivial.


Turning Vim into a modern Python IDE

Learning styles
From what I can gather, I am an assimilator. resistance is futile!

Cheat sheets!

The 9 secret burdens of being a Linux user

Big businesses acting out like this might very well get me to start boycotting them again…

String manipulation in bash

Seemingly nice way of doing HTTP requests in Python

Adventures in Javascript land

Tuesday, May 5th, 2009

Last night an old classmate of mine approached me with a programming problem, this time in Javascript. This is not a language I have spent much time with for various reasons and my previous attempts at grokking it have all been unsuccessful.

But times seem to have changed. Either I am a better programmer now, or Javascript has matured (I haven’t been keeping tabs on what is going on in the Javascript camp so I am still a bit fuzzy on the whole DOM thingy and if you can, nowadays, write a Javascript which both Internet Explorer and all the other browsers will understand without jumping through at least a couple of hoops), but in any case I was able to help him.

The problem he was having oriented around string manipulation. Given a pipe-separated string of “key=value” pairs, extract the values and store them in a variable of their own.

I am guessing he was going to extract all the values, but the example code he sent to me just included the extraction of a single value. His attempt was not bad, combining indexOf() and substring(), and I would like to think he had a pretty smart idea going, but Javascript was not intelligent enough for his idea.

The “mistake” he made was thinking that indexOf() would continue and find the next occurrence once it had been called once thus saving its “state” from the previous search, much like how a file/array iterator would behave.

He also had some rather “funny” constraints, or rather the reasoning behind them. There could be no magic numbers in the code (which is of course a good thing in and of its own) because the given string could be subject to change in the future.

I have no objection to this, except for the fact that he used hard-coded strings as argument to indexOf(). But sure, having to update that one place instead of two is better because otherwise you’ll just miss the other place when you update the first. (The magic numbers constraint surfaced when I suggested adding an offset to the index returned by indexOf() so that he would get the appropriate parts of the string.)

The initial code (obscured for his benefit) looked something like this:

var arg = "key1=value1|key2=value2|key3=value3|key4=value4";
var index = arg.indexOf("key2");
var key2 = arg.substring(index, indexOf("|"));

There were some initial mistakes in the code (second indexOf() called on nothing) as well as the assumption described earlier, that indexOf() kept a state. This code, after modification, would of course have tried to return a negative length substring. Now I haven’t checked, but somehow I doubt that Javascript would actually return a reversed substring of the indicated length and position.

My suggestion was on par with:

var arg = "key1=value1|key2=value2|key3=value3|key4=value4";
var index = arg.indexOf("key2") + 4;
var key2 = arg.substring(index, arg.indexOf("|", index));

This brought up the magic numbers constraint, so I then suggested replacing line 2 with:

var index = arg.indexOf("key2") + "key2".length;

But that was not satisfactory either. I believe he ended up with a solution of his own, calculating a second index finding the first “=” after “key2″ (using index as offset) and then from there in the substring incrementing the second index by one:

var arg = "key1=value1|key2=value2|key3=value3|key4=value4";
var index = arg.indexOf("key2");
var index2 = arg.indexOf("=", index);
var key2 = arg.substring(index2 + 1, arg.indexOf("|", index));

He was happy with that solution, but I was less than enthusiastic with any of that code so I started digging on my own. One solution could be regular expressions (I don’t know if it is a good or bad thing that I almost always nowadays think “regexp” when I have to find and/or break out data from a string…) and I found a pretty neat solution due to the fact that the Javascript string type has a built-in function match() which takes a regexp pattern as argument.

var arg = "key1=value1|key2=value2|key3=value3|key4=value4";
var key2_pattern = /key2=(\d+)/;
var key2 = arg.match(key2_pattern)[1];
// [0] == entire matched string, [1] == matched group (i.e. \d+)

Still I was not entirely satisfied, regular expressions can be great, if kept uncomplicated, but really how often do things stay uncomplicated? I was also wondering whether it would be possible to just store it all, dynamically, in a dictionary type of structure. That’s when I realized that Javascript has associative arrays. :D

var arg = "key1=value1|key2=value2|key3=value3|key4=value4";
var tmp_array = arg.split("|"); // array with "key=value" elements
var dictionary = new Array();
while (e = tmp_array.pop())
    tmp = e.split("=");
    dictionary[tmp[0]] = tmp[1]; // dictionary["key"] = "value";

I haven’t fully tried out this last suggestion (it works, but I haven’t yet tried how Javascript deals with “2” as opposed to 2, and how this might screw things up. I am thinking that it probably will screw things up. On the other hand so should substring (since it per definition also returns a string).

None of this really matters, my friend came to a working solution and I got to play with Javascript. Win-win :)