Archive for the ‘Programming’ Category

« Older Entries
Newer Entries »

2011w47

Sunday, November 27th, 2011

top

So I have been playing around some more with top, and I have to say that I no longer feel any reason to install htop.

Perhaps if I dig into the manpage of htop, I’ll yet again revert to thinking it is better, but for now there’s no need.

I can get coloring (z), I can filter on users (u<username><enter>), I can control how many processes I list (n<int><enter>), and I can have the current sort field highlighted (x), and when I am happy with the configuration, W lets me save it to ${HOME}/.toprc

grep

Pontus showed me a new shiny flag for grep the other day: -s which, to quote the grep manpage, says Suppress error messages about nonexistent or unreadable files.

And this is awesome for when your are doing directory-wide recursive greps in places where you might not have the credentials to look through all the files.

Beware though as there are some differences between GNU grep and UNIX grep.

RabbitMQ

I’ve many times read about RabbitMQ and how that is good to know and if you don’t know what it is you’ve been hiding under a rock (apparantly I have), because it wasn’t until this week I actually found a blogpost that could adequately explain to me what it is and what it’s good for.

And thanks to that blogpost I now have yet one more thing pushed onto the “toLearn” stack…

blockdiag

This next thing I found is a more or less graphviz, wrapped around a python(2) module which helps create block diagrams.

There are actually four modules, blockdiag, seqdiag, actdiag, and finally nwdiag, and I could imagine all four having their use under certain circumstances.

Links

GNU source highlight — For most of your sourcecode highlighting needs

Tags: , ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | Comments Off

2011w43

Sunday, October 30th, 2011

Misc tools and other goodies

Another work week, another set of “discoveries”, like less -S, crontab -r and that when you issue a command which in turn uses $EDITOR to launch an appropriate text editor, and you instead of an editor window is greated with vim: no such command, well then perhaps in one of your profile- or config-files for the shell you have a line looking something like this:

EDITOR=`which vim`

Yes, this happened to me at work on a box which only had vi installed.

Pontus also showed me some SSH escape sequences which could come in handy. The first thing to know about them is how to “activate” them, which is done with the tilde-sign (~).

So on my setup, this would mean “AltGr+¨AltGr+¨” followed by a some sequence (? for help, . to close the connection (very good for when the remote server has rebooted, i.e. the ssh session has died, but the terminal never got wind of it, so it just sits there), or C^z to suspend it.)

cp importantFile{,.bak} is a pretty nice pattern as well.

Finally, I found a new (and totally inappropriate but functional) way of using mscgen: to generate staffing schedules.

In this case, being the “tech responsible” at FSCONS, this means scheduling my eight slave^H^H^H^H^Hcamera persons across the four tracks and two days.

Experiences from last year made me divide each day up into two pieces (AM and PM) which makes for sixteen blocks, divided evenly across the eight volunteers (who I am ever greatful to) for a total of two blocks per person.

For that small amount of data, mscgen worked wonders and gave me a wonderful overview :)

As a sidenote, I really should try to post a “my picks” from the FSCONS schedule soon. Yet another TODO to push onto the stack… ;D

Java

A couple of nights ago Pontus told me about an “array shuffling algorithm” (e.g. good for when you have an array representing a deck of cards and want it shuffled) which basically revolves around iterating through the array once, starting at the back of the array, counting down and for each iteration use the loop-counter as the max value for the random number generator so that it always delivers a number (index) which is within the array itself, and then swap places if the index:th place and the loop-counter:th place of the array. That was a fun excercise :)

Tags: , , , , , , , , , ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | Comments Off

2011w42

Sunday, October 23rd, 2011

Perl

Progress! This week I wrote my first perl script, to parse some data on one of my colleagues nodes. In doing so I also, inadvertedly, made another one of my colleagues express something along the lines of <q>”very nice, now we have another scripting guy on our team.”</q> ;D

grep count occurrences on single line

Say you have a line (or multiple, that you are iterating through one at a time) of data structured in some way representable and matchable by a regular expression, and that you feel an overwhelming need to count the number of occurences in each line.

Did you ever imagine that grep and a couple of pipes were all you’d ever need to realize this wish?

$ echo "foo foo foo" | grep -o 'foo' | grep -c 'foo'

Tags: ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | Comments Off

2011w41

Sunday, October 16th, 2011

There’s not been a whole lot going on this past week, except for work and more work, but at least I am beginning to feel somewhat productive :-)

FSCONS

FSCONS is now less than a month away, YAY!

Perl

And I have started looking into learning Perl. Don’t know how long that will take, don’t really care either, it’s just for fun anyway :-)

Bash webservers (for simple testing)

This if of course nothing for live deployment, but if you don’t want to spend a whole lot of time installing and configuring an entire webserver just for some quick and dirty tests, and then there are some fairly neat-looking scripts. They are untested but doesn’t look like especially sinister constructions.

Random thoughts

I am getting a fair deal of SEO spam in my comments (thank you Akismet for dealing with it) and the only thing that comes to mind is:

IF SEO really works so well, then why do spammers feel the need to… well spam about it?

And especially when they try to link back to their own sites, increasing traffic to… their own sites. Iunno, just seems like a big scam to me.

But then again, my objective for blogging isn’t to get the maximum amount of visitors I can, but to share stuff I find neat, in the hope that it will be useful for someone more than me.

:wq

Tags: , , ,
Posted in English, GNU/Linux, Programming, Testing, Tools | 2 Comments »

2011w40

Sunday, October 9th, 2011

This week I got to do some real work! Felt really good being able to start contributing back. :D

Identi.ca

It saddens me to say this, but identi.ca, having upgraded to version 1.0 of status.net (or is it laconica? iunno *shrug*) more or less sucks.

Now, I know how extremely easy it is to jump on the hate bandwagon, to just do what everyone else is doing, and I admit, several others have brought forth some valid critique, e.g. the new incomprihensible threading crap, renewed incentive for spammers to be on identi.ca, etc, but for my own part I’ve noticed it most prominently due to my third-party client being slowed to a crawl and crashing repeatedly while trying to use identi.ca.

Yes, a crashing client is probably not identi.ca’s fault, but hey, it worked well enough before the so called “upgrade”.

Do we, as a society, really have such a raging hardon for “innovation” and “new shiny stuff” that we’d rather break something which is working, to have something new, than stay stable?

I remember looking at an email tool (could it have been offlineimap? Probably not as it is actively maintained, thank you Luke :) ) and reading comments about it, someone stating that “oh, you can’t use that, development has ended, it is stagnant, abandoned.” I.e. there wouldn’t be any new versions… oh the horror. . .

If a software has met all the sought goals, the project can go down one of two paths:

  1. go into maintenance mode, only releasing updates upon discoveries of bugs or when something (a protocol or whatever) is updated, and the software needs to support that as well, or
  2. bloat up with new crap which it was never intended to handle

Yes, I am a huge supporter of the UNIX philosophy. Separation of responsibility is a good thing.

But anyway, I won’t be abandoning it, I might lower my usage of it, but I won’t increase my presence in some other “social media”.

Vim

For some reason or other, I found myself wanting to put quotes around a word which the cursor was presently centered on.

Now, the simple way (which is much quicker if you are just going to be quoting a few words) is of course just to use navigation commands to get to the front of the word, enter insert mode, input a quote, exit insert mode, move to end of word, enter append mode and insert a quote, rinse and repeat.

But I wanted to do it the the longer, more automated way (not that I had a whole lot of words in need of quoting, I just had one, but it was more about the learning principle than efficiency at that point).

So I thought up:

bve"ay:s/<CTRL>ra/"<CTRL>ra"/<ENTER>

Basically, what you do is go to beginning of word, enter visual mode, go to end of word, copy the selection into a named buffer (a), enter command mode and issue a subsitution command in which the selection is pasted in both the find and replace fields, but in the replace field surrounded by quotes.

It works, you can record a macro to perform these steps so that you only need to place the cursor on the correct word and call the macro.

However, I started wondering if there wasn’t a way to extract the value from a search. I.e. if you put the cursor on a word, and hit * you’ll tell vim to find all occurrences of the word under the cursor. This must mean that vim somewhere stores the sought after word.

I never found a solution for that, but I did however find a shortcut:

:s/<CTRL>r<CTRL>w/"<CTRL>r<CTRL>w"/<ENTER>

This one simply takes the word presently under the cursor, and pastes that. This too, works with macros, i.e.:

qq:s/<CTRL>r<CTRL>w/"<CTRL>r<CTRL>w"/<ENTER>q

This little “quotifier” is now stored in macro buffer q and can be accessed by pressing @q in command mode. :D

Links

Node.js is cancer — whether or not you agree, it is still a thought-provoking read.

mscgen — I’m not sure how well I have been able to communicate that I like source code. I like being able to compile stuff or script stuff so that I can have reproduceable results. mscgen, I believe, uses graphviz to generate Message Sequence Charts, and I played around with it a bit this week when trying to outline how my little “brushing up on Erlang-project” should communicate internally.

All in all I have to say that it performed admirably.

Tags: , , ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | 6 Comments »

2011w39

Sunday, October 2nd, 2011

Stupid shell tricks

A friend of mine asked me this week how one would go about repeating the same command X number of times in a shell.

My first idea was of course a for-loop, along the lines of:

X=15; for i in $(seq 1 ${X?}); do echo "foo bar baz"; done

But his reply to that suggestion was that seems a bit much if all I want is to repeat the command twice…

Ok, so it wasn’t for X equalling any number, it was for X equalling two… sometimes I get the feeling that he is perhaps over-generalizing his questions to me ;D

Anyway, my second answer, given this new input, was: What? You’re too lazy to execute it, push up-arrow and enter?

But of course, his question had already caused me to fork a background process intent on finding a solution.

First I thought about the history command, and I ultimately came up with a solution through reading man history.

echo "foo bar baz"; !#

Note: That semicolon there is frakking important!

“!” when issued as the first char of a new word, should be interpreted as we’re going to do something with the history of this shell.

“#” in turn could roughly be interpreted as On this current line, do again whatever has been done from the start of this line, to where this history command is called

The result is echo “foo bar baz”; echo “foo bar baz”;

I have no idea what he needed that for, it seems pretty limited to me, but either way it’s pretty cool that it worked.

Now guess what echo "foo bar baz"; !# !# does.

How to get into Free Software

A buddy from work and I spoke about open source and free software the other day, and he had a basic grasp about it, but what he felt he lacked were knowledge of useful sites, etc. I.e. perhaps not how to become more involved, he’d already submitted patches to some specific projects, but more along the lines of where likeminded “hang out”?

That’s a poor description as well, and isn’t all that important. It did however get me thinking about it.

There is identi.ca instead of twitter, joindiaspora.com instead of facebook or google+. There is fsf.org, gnu.org and fsfe.org as well as the local (Swedish) ffkp.se for information about the ideology behind free software, but also for information about how to get involved and the types of activism they engage in.

I don’t really know how to categorize ohloh.net, but I guess it could be a fun place to hang out and either get recognition for your own contributions, or recognize the projects you use yourself.

Then I guess there is the part of the FOSS ecosystem which ?doesn’t exist at all? in the proprietary world (I am sure there are some exceptions to this) such as public code repository sites (savannah.{,non}gnu.org, gitorious as well as github and bitbucket).

I have accounts on all four, although I make a conscious effort to prefer the first two services over the latter two.

And then of course there are conferences which one could attend, FSCONS (yes, being biased, I put the one I’m co-organizing first) and FOSDEM springing immediately to mind.

Links

Another Humble Indie Bundle in the making. This game looks like it would be precisely my type of thing. Maybe ;)

This Erlang “hello, world!” tutorial has definitively earned itself some linklove.

:wq

Tags: , , ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | 2 Comments »

2011w35

Sunday, September 4th, 2011

I guess the first big thing to happen this week, which I’d be remiss if I didn’t mention it, is that I got a job :D

passtore

This Tuesday I also awoke with the idea that I really should, somehow make it easy for a user to sign the configuration file of passtore. And of course to check said signature as well.

What I want to achieve with that is to engineer out the flaw in security which would appear if an aggressor got hands on the system and added his/her own key id to the list of recipients in the configuration file.

I’ll have to think about that some more.

chattr +i

I don’t particularly like flash. Sadly a great big chunk of the Internet becomes useless without it, and although I should take a firmer stand against flash I have the flashplugin for Firefox installed. One of the really nasty things about flash is those persistent super-cookies which can be set, and quite frankly, Firefox has become a bit heavy lately, so I have been looking around for ways to cut back on add-ons.

I am currently trying out Privoxy as a standalone ad-blocker, and today I got the idea of trying to replace the “Better Privacy” add-on with some file-system hacking instead. In short “Better Privacy” works by trying to detect when a flash super-cookie has been set, and remove it.

This cookie is stored in a file, locally on the file-system. So I did a little thinking. In my home directory there are two hidden directories: .adobe and .macromedia, both containing a directory named Flash_player. Inside one of those (I guess it depends on the version of flashplayer which directory is the relevant one) flash stores these cookies.

So my first idea was simply to delete those two directories. Which of course is silly. The relevant one will just be recreated, at the latest upon the next Firefox restart. So that wouldn’t work.

I could of course instruct fsniper or incron to watch those two directories and have them pounce any newly created content within with an rm -rf command, and although that would work, I don’t particularly like scripted events which includes an rm command. Also I’d have to divert some system resources to that (yes, there are plenty of resources to go around, that’s beside the point) activity, and that wasn’t to my liking either.

Finally it dawned on me. If I removed the directories, they would just respawn, so I’d need to keep them there as placeholders, and make them non-writeable, and really immutable. That way, any time flash would try to get cute, it would hit a brick wall.

So what I ended up doing was:

  1. Purge any and all contents inside .adobe/ and .macromedia/
  2. chmod 0500 ./{.adobe,.macromedia}
  3. sudo chattr +i ./{.adobe,.macromedia}

So now the owner (me) can only read/list the contents of the directories (not modify them) and with chattr +i they are immutable, requiring root privileges to change those permissions.

Your move flash…

Links

Schneier on Security: The Efficacy of Post-9/11 Counterterrorism

MITnews: Killing a cancer cell from the inside out — although I’d worry about mutation, or weaponisation…

Tags: , , , , , , ,
Posted in Documentation, English, GNU/Linux, Personal, Programming | 2 Comments »

2011w34

Sunday, August 28th, 2011

Imagemagick, again

There has been quite a lot of hacking using imagemagick this past week, all of it for FSCONS use.

My first hack was to create an image for use in the MyConf site, to visually mark up passed timeslots as, well… passed.

The idea I had was to set a background on that html element which would have “Session has ended” written diagonally from the lower left corner to the upper right.

Greg convinced me that there must be better ways to mark this up which would at the same time not interfer with the readability of those sessions, and we ended up going another way, but this is how to create images with diagonal text anyway:

$ convert -size 400x200 xc:none -fill red -pointsize 40 -gravity center -draw "rotate 337.5 text 0,0 'Session has ended'" tmp.png

This will create a 400px wide by 200px high image, with a transparent background and red text, rotated to lay diagonally across the image, beginning in the southwest corner of the image and ending in the northeast.

The next day Rikard had an idea about using an FSCONS crowd image as background for another image, and have parts of the crowd “bleed through” the overlaying image.

This is of course something one can do in GIMP, iff you have learned how to. Rikard struggled with that, got help from Jonas, but ultimately the result wasn’t good, and he’d have to do it all over again, at which point Jonas had left for the day and he didn’t remember what had been done.

Which got me thinking “this must be doable in imagemagick, and repeatable (i.e. a shell script)”. Of course it was.

It is a two-step process, first you’ll need to prepare the overlaying image, by making parts of it transparent, enabling the background image to bleed through. This is done with:

$ convert overlaying-image.png -transparent black new-image.png

In the above example, the color black in “overlaying-image.png” will be made transparent, and the output saved into “new-image.png”.

For my tests, as I only needed a background image, and as anything would suffice, I had imagemagick create one for me:

$ convert -size 525x525 xc:blue bg.png

This will create an 525*525 pixel image with a blue background color.

With this done, all we need do is to merge the two images (“new-image.png” and “background-image.png”) together:

$ composite -gravity center new-image.png background-image.png resulting-image.png

One little gotcha with this command above: I haven’t tried what happens when I use two differently sized images. I am assuming that things will get cropped.

Media Queries

This Thursday I was introduced to Media Queries, a rather cool technique for having CSS determine (well, I suppose it really is the web browser which does all the work, while CSS is just the container for the rules) which styles to apply, depending on certain browser attributes (such as current width of the window, etc.)

Greg has implemented this in MyConf and it is pretty cool when you shrink your browser window down to about 200 pixels or so, and the page transforms before your eyes.

graphvis and neato

On thing which have bothered me about neato for a long time is that I could never find a way to have the nodes not overlap in the generated image.

There is syntax for how to space out nodes inside the graphviz grammar, and it works… sortof, but I actually found a better way to go about it now.

$ neato -Tpng -o resulting-file.png -Goverlap=false graph.dot

WordPress

When I updated the WP Stats plugin this Friday I was “greeted” with the message that I wouldn’t receive any further updates to the stats plugin and that I should get Jetpack instead.

It promised to be great and awesome and connect my blog to the “WP cloud” (whatever that is), but instead of filling me with optimism and making me look forward to that change, all that message managed to do was make me think “ok, I wonder how long before they’re gonna start charging for access to all this Jetpack functionality”.

Automattic is of course free to do so if they feel like it, but I can’t help but feel that it is ass-backwards to have the self-hosted wordpress.org platform, and then try to tie it into that “WP cloud” (whatever that is, again they leave me with more questions than answers)…

The funny thing is that WP Stats was one of the features that I really liked, and which made me hesitant to move somewhere else.

So thanks Automattic… but no thanks. Time to speed up the plans for migrating to fugitive…

Tags: , , , , , , , , , , ,
Posted in Development, Documentation, English, GNU/Linux, Programming, Tools | Comments Off

2011w31

Sunday, August 7th, 2011

(python) mechanize and BeautifulSoup

I’m slowly making preparations to replace WordPress with, as it currently stands, fugitive.

But I have a couple of posts in WordPress, 150—counting this one—to be more precise… which I don’t particularly feel like just abandoning, well… at least not all of them ;)

This means that they need to be preserved, and that’s where python’s two excellent modules mechanize and BeautifulSoup come into play.

Yes, I know relying on Beautiful Soup is discouraged nowadays, but as long as it works, I’ll be using it.

There is this great post about the basics of mechanize which, and since I only wanted to scrape a couple of posts, that was all I needed.

I needed Beautiful Soup to soup.find('link', {'rel': 'next'}) (i.e. get the URL for the “next” post) and then there was the little problem of retrieving the href-attribute from the found link-element.

StackOverflow to the rescue!

rsync

I found myself needing to synchronize a folder between two systems. I.e. new files added in the “source” system needed to be added to the “destination” which is a simple
$ rsync -av /path/to/source/directory/ user@remote:/path/to/destination/directory/
(please note the trailing slashes in BOTH paths, these in the source path, which tell rsync to copy the files INSIDE the directory, not including the directory itself).

However, for the first time I also needed that all removed files in source should be removed at the destination as well. I found this blogpost which gave me the information I needed.

This is… not trickier, but… not something you’d want to frakk up.

So, the flag which will delete on the removed files is simply --delete i.e.:

$ rsync -av --delete /path/to/source/directory/ user@remote:/path/to/destination/directory/

BEFORE you do this you REALLY should check that the operation will perform correctly by also attaching the flag --dry-run (which will simulate the real deal, without doing any changes on the remote end). Very nice.

$ rsync --dry-run -av --delete /path/to/source/directory/ user@remote:/path/to/destination/directory/

Musings

@shiny posted a notice about “open surface” being the perfect term for software that should be avoided.

@webmink, in what feels like true hacker fashion, cleverly played around with the words of the term and came up with “superficially open”.

Links

The Longest joke in the world

:wq

Tags: , , , , ,
Posted in Documentation, English, GNU/Linux, Programming, Project, Tools | 2 Comments »

2011w29

Sunday, July 24th, 2011

vim and ctags

I’ve been meaning to look into ctags for a while, mostly because I’ve wanted some better way of being able to locate the definition or calls of a function, than popping up a new terminal and running grep. I know of :[l]grep and :[l]vimgrep and should really spend a day or two learning those as well, but ctags looks like it could be more useful to know immediately so for now I’ll focus on getting acquainted with these keybindings.

ctags itself is “exuberant ctags” and is found on sourceforge, if not available from your package manager. If you’re using vim you’ll also want to grab the ctags.vim script and just drop it into your ${HOME}/.vim/plugin/ directory.

In your source directory simply run $ ctags -R and you will have yourself a tags file for this directory and all subdirectories. Opening a file and calling :TlistOpen will give you a list of all the found definitions in the project. Unfortunately for me, I want all the newly created windows from a split to appear to the right or below, the original window, EXCEPT for this tag list window ;)

The solution to that came in a StackOverflow thread and is called Ctrl-W R

There are several good posts which helped me come to terms with ctags.

firefox and sorting downloads

For a while now *cough*since around 2006*cough* I have been using the absolutely asskicking add-on Download Sort for Firefox. You set up your rules for where files with a certain extension should go, and then you just start downloading files, and they all end up where they should.

Or so the theory goes. But let’s say that you get tired of having to move all your wallpapers downloaded from deviantart.com or wherever, to your wallpapers folder, from the generic images folder?

Then Download Sort is looking less good. There just isn’t a way to differentiate between an “image” jpg and a “wallpaper” jpg.
Sadly, the only add-on I’ve found which advertises the features I want, Automatic Save Folder, didn’t work for me at all. It might be my overly paranoid security settings screwing things up, but for some reason it just won’t remember any of the rules I set into place…

urxvt-tabbedex

I started playing with tabbed urxvt. I ultimately ended up installing urxvt-tabbedex and using that instead. The differences are small, but tabbedex doesn’t provide a “new” button (a new shell is spawned using Shift-Down, and tabbedex provides (the perhaps less useful) feature of renaming a tab (Shift-Up).

wmii

I have lived under the impression that I was using a really old wmii (even though the package I used was wmii-hg which should pull the latest version from the repository). The reason I was under the impression that this was old was that it never came any updates when I executed a $ yaourt -Sayu.

When I finally looked into it yesterday I finally realized that I was running on bleeding edge. The trick to wmii-hg, apparently, is that you need to manually re-install it to have it fetch the latest updates… What I took as project inactivity (and must thus be an old branch) was in fact not so.

As tagrules are severely broken in wmii-hg it hasn’t been fun using it (and colrules behaved very strange, but strange in a way I have now come to appreciate… go figure) but there were always issues with the wmii package in the repository, most having to do with an insane amount of flickering when creating a new terminal, or moving one about etc.

The flickering parts seems to have been solved, it might have had something to do with my transparent terminals and not a problem with wmii, I don’t know, but now it works.

Thanks goes out to pesa for single-handedly seeking out and vanquishing my final annoyance of wmii 3.9.2: that every now and then it would start spawning more than one terminal when I issued a command to get one.

When I was reloading the configuration, the old process remained, and with it, the listeners for keyboard commands… like starting a new terminal…

mercurial-server

I’ve been running the mercurial-server package on my server to provide me with easy access to (mercurial) repositories. This week, for some reason, it started failing.

I had updated mercurial from 1.3 to 1.9 and my first thought was that this had broken mercurial-server somehow, but after reverting the package, the problem came back.

Not immediately, mind you, which makes this whole thing even stranger.

But this has just provided me with the right amount of kick in the ass to get on my way and convert from mercurial to git, so all in all perhaps not a bad thing?

Links

Why I’m not going near spotify: It sums up pretty well my own feelings about Spotify…

In certain (rather specific) contexts, I even despise Spotify users, and here’s why: Imagine that you are sitting on a train, and you will be doing so for the next seven hours or so. The train has “internet”, via one 3G modem.

Now picture ten other passengers ALSO going online, their objective being to listen to Spotify. Can haz moar bandwifs plz? kthxbai

:wq

Tags: , , , , , ,
Posted in Development, Documentation, English, GNU/Linux, Programming, Tools | Comments Off

« Older Entries
Newer Entries »