Posts Tagged ‘Perl’

My Software Stack 2011 edition

Saturday, December 31st, 2011

I realize that I haven’t written my customary “software stack” post for this year yet. But hey, from where I’m sitting, I still have … 36 minutes to spare ;)

I’ll be using the same categories as last year; system, communications, web, development, office suite, server, organization, and entertainment.

System

The OS of choice is still Archlinux, my window manager is still wmii, my terminal emulator is rxvt-unicode, upgraded by also installing urxvt-tabbedex.

My shell is still bash, my cron daemon is still fcron, and my network manager is wicd.

To this configuration I’ve added the terminal multiplexer tmux, and have lately found out just how useful mc can be. Oh, and qmv from the renameutils package is now a given part of the stack.

Communications

Not much change here, Thunderbird for email, Pidgin for instant messaging, irssi for IRC.

Heybuddy has been replaced by identicurse as my micro-blogging (identi.ca) client. Heybuddy is very nice, but I can use identicurse from the commandline, and it has vim-like bindings.

For Pidgin I use OTR to encrypt conversations. For Thunderbird I use the enigmail addon along with GnuPG.

This means that Thunderbird still hasn’t been replaced by the “mutt-stack” (mutt, msmtp, offlineimap and mairix) and this is mostly due to me not having the energy to learn how to configure mutt.

I also considered trying to replace Pidgin with irssi and bitlbee but Pidgin + OTR works so well, and I have no idea about how well OTR works with bitlbee/irssi (well, actually, I’ve found irssi + OTR to be flaky at best.

Web

Not much changed here either, Firefox dominates, and I haven’t looked further into uzbl although that is still on the TODO list, for some day.

I do some times also use w3m, elinks, wget, curl and perl-libwww.

My Firefox is customized with NoScript, RequestPolicy, some other stuff, and Pentadactyl.

Privoxy is nowadays also part of the loadout, to filter out ads and other undesirable web “resources”.

Development

In this category there has actually been some changes:

  • gvim has been completely dropped
  • eclipse has been dropped, using vim instead
  • mercurial has been replaced by git

Thanks in no small part to my job, I have gotten more intimate knowledge of awk and expect, as well as beginning to learn Perl.

I still do some Python hacking, a whole lot of shell scripting, and for many of these hacks, SQLite is a faithful companion.

Doh! I completely forgot that I’ve been dabbling around with Erlang as well, and that mscgen has been immensely helpful in helping me visualize communication paths between various modules.

“Office suite”

I still use LaTeX for PDF creation (sorry hook, still haven’t gotten around to checking out ConTeXt), I haven’t really used sc at all, it was just too hard to learn the controls, and I had too few spreadsheets in need of creating. I use qalculate almost on a weekly basis, but for shell scripts I’ve started using bc instead.

A potential replacement for sc could be teapot, but again, I usually don’t create spreadsheets…

Server

Since I’ve dropped mercurial, and since the mercurial-server package suddenly stopped working after a system update, I couldn’t be bothered to fix it, and it is now dropped.

screen and irssi is of course always a winning combination.

nginx and uwsgi has not been used to any extent, I haven’t tried setting up a VPN service, but I have a couple of ideas for the coming year (mumble, some VPN service, some nginx + Python/Perl thingies, bitlbee) and maybe replace the Ubuntu installation with Debian.

Organization

I still use both vimwiki and vim outliner, and my Important Dates Notifier script.

Still no TaskJuggler, and I haven’t gotten much use out of abook.

remind has completely replaced when, while I haven’t gotten any use what so ever out of wyrd.

Entertainment

For consuming stuff I use evince (PDF), mplayer (video), while for music, moc has had to step down from the throne, to leave place for mpd and ncmpcpp.

eog along with gthumb (replacing geeqie) handles viewing images.

For manipulation/creation needs I use LaTeX, or possibly Scribus, ffmpeg, audacity, imagemagick, inkscape, and gimp.

Bonus: Security

I thought I’d add another category, security, since I finally have something worthwhile to report here.

I’ve begun encrypting selected parts of my hard drive (mostly my email directory) using EncFS, and I use my passtore script for password management.

And sometimes (this was mostly relevant for when debugging passtore after having begun actively using it) when I have a sensitive file which I for a session need to store on the hard drive, in clear text, I use quixand to create an encrypted directory with a session key only stored in RAM. So once the session has ended, there is little chance of retrieving the key and decrypting the encrypted directory.

Ending notes

That’s about it. Some new stuff, mostly old stuff, only a few things getting kicked off the list. My stack is pretty stable for now. I wonder what cool stuff I will find in 2012 :D

:wq

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Documentation, English, GNU/Linux, Personal, Tools | 4 Comments »

2011w48

Sunday, December 4th, 2011

Where the frakk did this week go?!?!

Work has been progressing, I can’t say that I am good at it yet, but I am better than I was just last week, which is thoroughly encouraging :)

Pontus made me realize that knowing sed is not enough, for some things you really need awk. Another thing to push to the toLearn stack…

I’ve been doing some more Perl hackery, but nothing worth showing, but I did however come across a site which I believe to be rather good and helpful regarding learning basic things about Perl.

Something which passed me by completely was that this Monday saw the release of YaCy 1.0 (in German), but as you can see on Alessandro’s blog I might have been just about the only one who didn’t get that particular news item. Congratulations on making version 1.0 guys!

I was also toying with the idea the other day of making quarterly summaries as well. One blog post a week is great as it forces me to write, thus improving my writing, but it doesn’t really do anything for discerning trends, or changes in the way I work. This could be interesting :)

Finally, I should really start planning for writing my yearly “technology stack” post by diffing what I used back then and what I’m using now.

I am already certain that I’ve disappointed myself in some aspects, and surprised myself in others…

:wq

Tags: , , ,
Posted in Blog, English, GNU/Linux, Programming, Tools | 2 Comments »

2011w44

Sunday, November 6th, 2011

FSCONS

T minus 5 days and counting :D

It’s gonna be great to see all the familiar faces again and hopefully get to know some new people as well.

(h)top

top (and it’s more colorful sibling htop) can be a great tool for keeping track of processes, and to some extent users, in a system. If, for instance, you’re sharing a system with a couple other people, and the systems starts feeling sluggish, top will help you out by displaying the process id (and the username which started the process) of the process consuming the most CPU cycles.

If you’d rather sort on the processes consuming the most system memory that would be On<ESC> (i.e. “<shift>o” followed by “n” followed by <ESC>)
Note that this is done from inside a running top.

You could even single out a specific user for monitoring with

$ top -u <username>

bc

This one took me a while to figure out… I had this problem during the week, which I for the life of me cannot remember now, but it manifested itself in that my attempts to do

$ echo "$part / $total" | bc

didn’t do the trick as it just truncated the value down to 0.

And the solution to this, which I found at Linux By Examples was rather simply to add a scale command to bc:

$ echo "scale=3; $part / $total" | bc

Perl

Named capture groups are great (I already knew of them from Python (Django, urls.py) but they took me quite some time to hunt down in Perl.

Links

The Geek Stuff has some really great posts about various more or less admin-oriented commands (indeed, when I was in a hurry to learn what I needed from top this week, The Geek Stuff was my primary tutor.

Tags: , , ,
Posted in Documentation, English, GNU/Linux, Tools | Comments Off

2011w42

Sunday, October 23rd, 2011

Perl

Progress! This week I wrote my first perl script, to parse some data on one of my colleagues nodes. In doing so I also, inadvertedly, made another one of my colleagues express something along the lines of <q>”very nice, now we have another scripting guy on our team.”</q> ;D

grep count occurrences on single line

Say you have a line (or multiple, that you are iterating through one at a time) of data structured in some way representable and matchable by a regular expression, and that you feel an overwhelming need to count the number of occurences in each line.

Did you ever imagine that grep and a couple of pipes were all you’d ever need to realize this wish?

$ echo "foo foo foo" | grep -o 'foo' | grep -c 'foo'

Tags: ,
Posted in Documentation, English, GNU/Linux, Programming, Tools | Comments Off

2011w41

Sunday, October 16th, 2011

There’s not been a whole lot going on this past week, except for work and more work, but at least I am beginning to feel somewhat productive :-)

FSCONS

FSCONS is now less than a month away, YAY!

Perl

And I have started looking into learning Perl. Don’t know how long that will take, don’t really care either, it’s just for fun anyway :-)

Bash webservers (for simple testing)

This if of course nothing for live deployment, but if you don’t want to spend a whole lot of time installing and configuring an entire webserver just for some quick and dirty tests, and then there are some fairly neat-looking scripts. They are untested but doesn’t look like especially sinister constructions.

Random thoughts

I am getting a fair deal of SEO spam in my comments (thank you Akismet for dealing with it) and the only thing that comes to mind is:

IF SEO really works so well, then why do spammers feel the need to… well spam about it?

And especially when they try to link back to their own sites, increasing traffic to… their own sites. Iunno, just seems like a big scam to me.

But then again, my objective for blogging isn’t to get the maximum amount of visitors I can, but to share stuff I find neat, in the hope that it will be useful for someone more than me.

:wq

Tags: , , ,
Posted in English, GNU/Linux, Programming, Testing, Tools | 2 Comments »

2011w25

Sunday, June 26th, 2011

Last week was rather eventful, the largest thing being the one thing I naturally forgot to write about (go figure…), my appointment as deputy coordinator of FSFE Sweden. This is nice :)

That has however meant that this week hasn’t seemed as eventful, and I don’t know, for some reason I got off to a really slow start of the week, the only worthwhile things to write about started happening this Thursday.

nginx and password-protected directories

My father asked me for help in getting a bunch of files in his possession, over to some friends of his (who are, to the best of my knowledge, as computer illiterate as he is).

This meant that my first idea, to just set up an FTP-account on my server and have them log into that and download the files, wouldn’t work. I would need something simpler, but still with restricted access.

Preferably they’d just surf to some place, enter a password, and download a zip-archive (since all Windows versions since XP handles zip-archives like compressed folders, this should fall into the realm of what a computer user should be able to handle).

Something like Apache’s htpasswd stuff. And I wanted to do it with nginx, because I really want to get better at using and working with it.

The first task, obviously, was to check if nginx had that capability at all (it has), and if so, how it works.

This post showed me that it was possible, and how to do it.

A note here though: I first tried to set a password containing Swedish characters (åäö) and this didn’t work at all.

ticket

I have been wrestling with the question of how I would manage to create a database which individual users can read from and write to, but which they shouldn’t be able to remove from the filesystem (I know, a DROP or DELETE command can be just as devastating, so I must continue thinking about this).

alamar at StackOverflow solved this for me. The solution is to let the file be read and writeable, but have the parent directory not be writable.

This however makes it impossible to add new files to the directory. But since I am working with the idea that there should be a “ticket” user with a corresponding “ticket” group, and that every individual who should have access to the tracker will be in that ticket-group, the directory could disallow writing for group and other, leaving the ticket-user free to create more databases…

Although I now realize that this would make it easy for anyone in the ticket-group to screw around with any ticket database (insert, update, delete).

This clearly needs more design thought put behind it.

ArchLinux and MySQL client binaries

I needed to interact with a MySQL database on another server, but MySQL (the server) wasn’t installed on my desktop, and I didn’t really want to have to install the entire server just to get hold of the mysql client binary so that I could interact with the remote server.

Turns out that in ArchLinux, themysql binaries are split into a clients and a server package, perfect for when you wish to interact with MySQL databases, but not have the entire frakking server installed on your machine.

Accessibility, HTML and myConf

Since FSCONS is striving to be accessible, and the little “myConf” technology demonstrator I wrote the other week was intended for FSCONS, I have been trying to figure out how to make that as accessible as it can be (first of all, I have no idea what so ever, if a screen reader even parses javascript, and as the myConf demonstrator is mostly implemented in jQuery that might present itself a showstopper).

But given the assumption that a screen reader can parse javascript, and will output that big ‘ol table which is created, how do I make an html table accessible? Since a screen reader makes use of the html code, and even a sighted person could get tripped up trying to parse the markup of a table, this looks like a worthwhile venture.

Sadly, like all documents from w3.org, they just leave me more confused and without any questions answered than when I began, but luckily, there seems to be other resources more knowledgeable, and with more understandable wording/examples, although I haven’t had the time to read through them all yet (I’m mostly just dumping them here so that I’ll be able to find the pages again once I again have the time to look into it):

Ed Weissman has taken his most insightful comments from Hacker News and compiled it into a book, which he then graciously made available for free.

Now, I have to admit, until this week, I’d never heard of Ed, and I have rarely read stuff on Hacker News, but from what I’ve read so far of his book, I might have to change this.

Optimizing Vim usage

A fellow… hmmm, a fellow Fellow made me realize just how long it is going to take me to fully grok Vim. I have been using ggVGd to:

  1. Go to the first line (“gg”)
  2. Enter visual mode (spanning entire lines) (“V”)
  3. Go to the last line (“G”)
  4. And finally delete the selection (the entire contents of the file) (“d”)

Or one could just do :%d as the fellow showed me… And I have been using the pattern :%s/foo/bar/ for quite some time, understanding perfectly that “%” in this context means “for every line do…”

I just never made the connection that it could be applied to something simpler than a sed substitution.

Links

Lack of (American) geeks is a national security risk according to DoD. Funny, anyone else who thinkgs that if they just stop prosecuting every kid who is playing around with security systems, or dowload music, or build (more or less dangerous) stuff from schematics they found online, this problem might just go away on its own?

Tags: , , , , , , , , , , , , , ,
Posted in Documentation, English, GNU/Linux, Project, Tools | Comments Off

My software stack

Friday, May 29th, 2009

A week or so ago I stumbled across this blog, which went almost instantly into my RSS feed, due not only to the name of a post which cracks me up (yes, I know my humor is off ;P) but also to the posts I found really interesting.

And then I came along this post which got me thinking about what software I ended up using towards the end of my bachelors. Or the software I have learned of since, but wish I’d known about earlier. I began to write a comment to her post, but realized that it would be too long, so I write here instead. All credit to Hazel though, since without her post I wouldn’t have been inspired to write this one.

My list, as compared to Hazels, will not be as well-rounded, it won’t necessarily fit every student the way her list do. Also, the software I list will only be guaranteed to work in GNU/Linux, as that is what I used in the final semesters, and have continued to use since.

First of all, a text editor. It doesn’t really matter which, just evaluate a bunch until you find one you feel comfortable with. Once you have found “the one” become intimate with it. Become a frakking Jedi-master at wielding it. I’m still a padawan-level user of Vim, but I’m getting there.

I say the same about web browsers, mail clients and instant messaging clients. Find a good one, learn as much as you can about it, and use it effectively. Firefox, Thunderbird and Pidgin are my preferred tools.

A bug-tracker, although often web based creating a need for a web server, can often provide more “good stuff” than just tracking bugs. Stuff like statistics, or, if you think outside the box you’d be able to track things other than bugs, which I guess it was issue-trackers does. Some of these also include a wiki-system, which makes establishing a project-specific knowledge-base kindof easy. In the one university project where we used such a system (and where I realized its potential) we used Trac.

A blogging-system with an RSS-feed capable of being filtered on tags or categories could be used to distribute status updates to other members of a group. That I’m using WordPress should be fairly obvious to all.

Use a version control system wherever and whenever possible. With the next two suggestions on the list, “wherever” will be a lot more commonplace than one might first believe, even for non-programmers. At the university we had access to SVN-servers, and also tried Mercurial, a distributed vcs. Mercurial stuck with me ever since.

From generic suggestions, let’s go specific.

I could encourage you to check out markup languages such as reStructuredText or Markdown, to find one which suits you best and to run with it. And since I’ve now written the terms you’d need to Google, you could do that, but I’ll simply recommend LaTeX. The reason for markup languages in general, and LaTeX specifically is that you can then store your information in one plaintext format (which makes it easy to manage in version control) and can then transform it to a slew of other formats as needed.

Most of the time we needed to hand in PDFs. LaTeX excels in that and manages all the typesetting stuff and (obvious) formatting. Which leaves you with more time to focus on the content. One could also either extend LaTeX with Beamer, to create presentations, or simply generate a PDF and run Impress!ve.

For diagrams, graphs and flowcharts or representations of state-machines, Graphviz would be my recommended way to go. Again using plaintext to control the content, again with the benefits of version control. Inkscape saves files in the SVG format (again, plaintext) which might be usable (especially since it can also save files as both PS and PDF)

If you need graphical representations of statistical data or other plots, matplotlib could be the way to go.

I personally don’t like managing things, or management-related stuff, but lately I have been haunted by the feeling that if I used management tools, even if I would only be managing myself and my pet projects, I could be more organized and efficient. So I have started looking at TaskJuggler. It is similar to Microsoft Project, with the largest difference being that… you guessed it, you code the project plan ;D. Plaintext yet again. And then you compile the plan and TaskJuggler attempts to verify that no resources have been double-booked.

Considering each piece in this list on their own, it might seem like a waste of time to exchange one software with another. I do find each of these softwares impressive in and on their own, but it is when they are put together, when all their strengths are combined, that you tend to get the most out of it.

The all plaintext approach I have tried, both in groupwork at the university, and later on my own, work rather well. That so many of the softwares on the list can be used to communicate and transfer information between parties is also intentional as without communication the chance of a successful project outcome diminish rapidly.

The last (bonus?) item on the list would be to recommend learning, at least superficially, a programming language which you could hack together small scripts with. Something which you could use to “glue” together the other parts. I adore Python, and many of the softwares listed above have python-bindings ready to use. Perl, Ruby and others, which elude me right now, would undoubtedly work equally well or better, but as with the text editor, pick a language you feel comfortable with, and rock on.

Thoughts? Questions?

Update: Fixed broken link

Tags: , , , , , , , , , , , , , , , , , , ,
Posted in Blog, Documentation, GNU/Linux, Programming, Tools | 4 Comments »

Random thoughts of a sleep-deprived mind…

Monday, January 28th, 2008

Couldn’t sleep last night, went to bed around 0100 hours (a little late, but I figured I’d manage with 5 hours worth of sleep, as I always try to trick myself) and just tossed and turned until I finally gave up trying to sleep around 0200, and sat down in front of the computer again, watched a movie… or two… while researching some on Lisp and Perl.

This continued until about 0500 when I finally started to feel so sleepy that I surely couldn’t stay awake even if I wanted to (which I didn’t). Oh how wrong I was. A quarter to six I was beginning to drift out of consciousness and into dreamland, but then I realized the futility and just lay there awaiting the alarm from my cellphone.

Great way to start a new day… really… not. So now I’m in school feeling pretty shitty, tired, queasy and freezing. So why would I want to share with you any thought currently in my mind? Good question, I sure as frack don’t have an answer (btw, may the Lords of Cobol make the writers strike end soon so that we can have season 4), but for what it’s worth (probably not much) here goes:

On a more serious note (not that Python, Perl, Erlang, Firefox, sshfs and rsync isn’t serious business) it gladdens my heart to see that more and more governments around the world are embracing the Open Document Format. Hopefully this more than anything will put an end to the madness that is Microsoft OOXML.

Over and out

Tags: , , , , , , , ,
Posted in GNU/Linux, Tools | Comments Off