Posts Tagged ‘nginx’

My Software Stack 2011 edition

Saturday, December 31st, 2011

I realize that I haven’t written my customary “software stack” post for this year yet. But hey, from where I’m sitting, I still have … 36 minutes to spare ;)

I’ll be using the same categories as last year; system, communications, web, development, office suite, server, organization, and entertainment.

System

The OS of choice is still Archlinux, my window manager is still wmii, my terminal emulator is rxvt-unicode, upgraded by also installing urxvt-tabbedex.

My shell is still bash, my cron daemon is still fcron, and my network manager is wicd.

To this configuration I’ve added the terminal multiplexer tmux, and have lately found out just how useful mc can be. Oh, and qmv from the renameutils package is now a given part of the stack.

Communications

Not much change here, Thunderbird for email, Pidgin for instant messaging, irssi for IRC.

Heybuddy has been replaced by identicurse as my micro-blogging (identi.ca) client. Heybuddy is very nice, but I can use identicurse from the commandline, and it has vim-like bindings.

For Pidgin I use OTR to encrypt conversations. For Thunderbird I use the enigmail addon along with GnuPG.

This means that Thunderbird still hasn’t been replaced by the “mutt-stack” (mutt, msmtp, offlineimap and mairix) and this is mostly due to me not having the energy to learn how to configure mutt.

I also considered trying to replace Pidgin with irssi and bitlbee but Pidgin + OTR works so well, and I have no idea about how well OTR works with bitlbee/irssi (well, actually, I’ve found irssi + OTR to be flaky at best.

Web

Not much changed here either, Firefox dominates, and I haven’t looked further into uzbl although that is still on the TODO list, for some day.

I do some times also use w3m, elinks, wget, curl and perl-libwww.

My Firefox is customized with NoScript, RequestPolicy, some other stuff, and Pentadactyl.

Privoxy is nowadays also part of the loadout, to filter out ads and other undesirable web “resources”.

Development

In this category there has actually been some changes:

  • gvim has been completely dropped
  • eclipse has been dropped, using vim instead
  • mercurial has been replaced by git

Thanks in no small part to my job, I have gotten more intimate knowledge of awk and expect, as well as beginning to learn Perl.

I still do some Python hacking, a whole lot of shell scripting, and for many of these hacks, SQLite is a faithful companion.

Doh! I completely forgot that I’ve been dabbling around with Erlang as well, and that mscgen has been immensely helpful in helping me visualize communication paths between various modules.

“Office suite”

I still use LaTeX for PDF creation (sorry hook, still haven’t gotten around to checking out ConTeXt), I haven’t really used sc at all, it was just too hard to learn the controls, and I had too few spreadsheets in need of creating. I use qalculate almost on a weekly basis, but for shell scripts I’ve started using bc instead.

A potential replacement for sc could be teapot, but again, I usually don’t create spreadsheets…

Server

Since I’ve dropped mercurial, and since the mercurial-server package suddenly stopped working after a system update, I couldn’t be bothered to fix it, and it is now dropped.

screen and irssi is of course always a winning combination.

nginx and uwsgi has not been used to any extent, I haven’t tried setting up a VPN service, but I have a couple of ideas for the coming year (mumble, some VPN service, some nginx + Python/Perl thingies, bitlbee) and maybe replace the Ubuntu installation with Debian.

Organization

I still use both vimwiki and vim outliner, and my Important Dates Notifier script.

Still no TaskJuggler, and I haven’t gotten much use out of abook.

remind has completely replaced when, while I haven’t gotten any use what so ever out of wyrd.

Entertainment

For consuming stuff I use evince (PDF), mplayer (video), while for music, moc has had to step down from the throne, to leave place for mpd and ncmpcpp.

eog along with gthumb (replacing geeqie) handles viewing images.

For manipulation/creation needs I use LaTeX, or possibly Scribus, ffmpeg, audacity, imagemagick, inkscape, and gimp.

Bonus: Security

I thought I’d add another category, security, since I finally have something worthwhile to report here.

I’ve begun encrypting selected parts of my hard drive (mostly my email directory) using EncFS, and I use my passtore script for password management.

And sometimes (this was mostly relevant for when debugging passtore after having begun actively using it) when I have a sensitive file which I for a session need to store on the hard drive, in clear text, I use quixand to create an encrypted directory with a session key only stored in RAM. So once the session has ended, there is little chance of retrieving the key and decrypting the encrypted directory.

Ending notes

That’s about it. Some new stuff, mostly old stuff, only a few things getting kicked off the list. My stack is pretty stable for now. I wonder what cool stuff I will find in 2012 :D

:wq

2011w25

Sunday, June 26th, 2011

Last week was rather eventful, the largest thing being the one thing I naturally forgot to write about (go figure…), my appointment as deputy coordinator of FSFE Sweden. This is nice :)

That has however meant that this week hasn’t seemed as eventful, and I don’t know, for some reason I got off to a really slow start of the week, the only worthwhile things to write about started happening this Thursday.

nginx and password-protected directories

My father asked me for help in getting a bunch of files in his possession, over to some friends of his (who are, to the best of my knowledge, as computer illiterate as he is).

This meant that my first idea, to just set up an FTP-account on my server and have them log into that and download the files, wouldn’t work. I would need something simpler, but still with restricted access.

Preferably they’d just surf to some place, enter a password, and download a zip-archive (since all Windows versions since XP handles zip-archives like compressed folders, this should fall into the realm of what a computer user should be able to handle).

Something like Apache’s htpasswd stuff. And I wanted to do it with nginx, because I really want to get better at using and working with it.

The first task, obviously, was to check if nginx had that capability at all (it has), and if so, how it works.

This post showed me that it was possible, and how to do it.

A note here though: I first tried to set a password containing Swedish characters (åäö) and this didn’t work at all.

ticket

I have been wrestling with the question of how I would manage to create a database which individual users can read from and write to, but which they shouldn’t be able to remove from the filesystem (I know, a DROP or DELETE command can be just as devastating, so I must continue thinking about this).

alamar at StackOverflow solved this for me. The solution is to let the file be read and writeable, but have the parent directory not be writable.

This however makes it impossible to add new files to the directory. But since I am working with the idea that there should be a “ticket” user with a corresponding “ticket” group, and that every individual who should have access to the tracker will be in that ticket-group, the directory could disallow writing for group and other, leaving the ticket-user free to create more databases…

Although I now realize that this would make it easy for anyone in the ticket-group to screw around with any ticket database (insert, update, delete).

This clearly needs more design thought put behind it.

ArchLinux and MySQL client binaries

I needed to interact with a MySQL database on another server, but MySQL (the server) wasn’t installed on my desktop, and I didn’t really want to have to install the entire server just to get hold of the mysql client binary so that I could interact with the remote server.

Turns out that in ArchLinux, themysql binaries are split into a clients and a server package, perfect for when you wish to interact with MySQL databases, but not have the entire frakking server installed on your machine.

Accessibility, HTML and myConf

Since FSCONS is striving to be accessible, and the little “myConf” technology demonstrator I wrote the other week was intended for FSCONS, I have been trying to figure out how to make that as accessible as it can be (first of all, I have no idea what so ever, if a screen reader even parses javascript, and as the myConf demonstrator is mostly implemented in jQuery that might present itself a showstopper).

But given the assumption that a screen reader can parse javascript, and will output that big ‘ol table which is created, how do I make an html table accessible? Since a screen reader makes use of the html code, and even a sighted person could get tripped up trying to parse the markup of a table, this looks like a worthwhile venture.

Sadly, like all documents from w3.org, they just leave me more confused and without any questions answered than when I began, but luckily, there seems to be other resources more knowledgeable, and with more understandable wording/examples, although I haven’t had the time to read through them all yet (I’m mostly just dumping them here so that I’ll be able to find the pages again once I again have the time to look into it):

Ed Weissman has taken his most insightful comments from Hacker News and compiled it into a book, which he then graciously made available for free.

Now, I have to admit, until this week, I’d never heard of Ed, and I have rarely read stuff on Hacker News, but from what I’ve read so far of his book, I might have to change this.

Optimizing Vim usage

A fellow… hmmm, a fellow Fellow made me realize just how long it is going to take me to fully grok Vim. I have been using ggVGd to:

  1. Go to the first line (“gg”)
  2. Enter visual mode (spanning entire lines) (“V”)
  3. Go to the last line (“G”)
  4. And finally delete the selection (the entire contents of the file) (“d”)

Or one could just do :%d as the fellow showed me… And I have been using the pattern :%s/foo/bar/ for quite some time, understanding perfectly that “%” in this context means “for every line do…”

I just never made the connection that it could be applied to something simpler than a sed substitution.

Links

Lack of (American) geeks is a national security risk according to DoD. Funny, anyone else who thinkgs that if they just stop prosecuting every kid who is playing around with security systems, or dowload music, or build (more or less dangerous) stuff from schematics they found online, this problem might just go away on its own?

nginx and flask

Friday, March 4th, 2011

I’ve been wanting to play with nginx for a while, and I have experimented a little with flask. But try as I might, I just could never understand how to get nginx and fastcgi to play with each other. Until bumby invited me to a hack night and he spent an hour or so showing me how to do it. I <3 bumby.

I can’t fault the documentation, in fact, the flask documentation on how to deploy flask with fastcgi under nginx should be quite sufficient (and indeed, now as I read it in retrospect, it is all crystal clear).

The problem I had was that I couldn’t divine how the fastcgi concept works, and I suppose this is mainly due to me having exclusively worked with Apache and PHP, which does stuff differently.

I just couldn’t wrap my head around how things would work out, if nginx didn’t know where the flask (Python) stuff was located.

I suppose that there is some part of the documentation which outlines precisely this, and that I read that part of the documentation poorly.

The “aha-moment” came when bumby made me realize that there won’t be one server process (web server) but two (web server and fastcgi server) and that these two processes won’t know where the other resides, and that this is ok, because they communicate through other means, namely sockets.

So the entire flow goes something like this:

  1. A client makes a request to the (nginx) web server, which determines that this is something to be handled by the fastcgi server, and thus sends it on through the socket
  2. The fastcgi server listens to the socket, handles the request and replies over the same socket
  3. The web server receives the reply and sends it on to the requesting client

Or at least, this is how I have come to understand the work flow.

Either way, the important thing to realize (which I couldn’t on my own) is that the configuration file for nginx won’t contain a path to the web stuff, as Apache would for a PHP-powered site.

This is ok, but quite contrary to what my mental model told me was necessary for the successful operation of a dynamic website. Once I understood the communication between nginx and fastcgi, it all made so much sense.

In case I ever forget it: client -> nginx -(socket)-> fastcgi -(socket)-> nginx -> client

Update: Typos be gone!

:wq

My software stack revisited – Summary and Future

Wednesday, December 29th, 2010

This is the last post in the series. It has taken me a while to write, edit, and then rewrite because I didn’t like the first outline (which I didn’t realize until I got to the end).

The software I have listed in these posts are there for a reason. While they work well in their own right, when put together, the whole forms something greater than the individual pieces.

Lightweight

Most of the software on the list is lightweight, almost a requirement as one of my systems is a netbook, and although the resources on my desktop seem almost infinite in comparison, I want to be able to work with my things on the netbook as well.

CLI

I go for CLI-apps rather than GUIs because that, more often than not, keep my fingers planted on the home row, instead of the right hand scurrying off to the mouse ever so often. Because, more often than not, the CLI-apps are more lightweight than the lightest of GUIs. And because more often than not, a CLI takes up less screen space to convey the same amount of information, as a GUI does.

CLI-based programs will also work when the X-server has broken down, and your stuck in a TTY, or you have your software on a server and are accessing it over SSH from a computer where either you or the system is unable to install the software.

Plaintext

I prefer plaintext over binary storage formats because it leaves me in control. If the worst comes to the worst, I can retrieve the data I need from a simple text-editor.

Not to mention that plaintext is easier to diff, which is good when you check it into version control and run into a conflict.

In the case of LaTeX, one of the best practices I picked up was to let each chapter be its own file, and then have a master-file which includes the chapters.

What this means is that when you want feedback on a specific chapter, you only need to send that, and not the entire project, which is bandwidth friendly.

And please, no comments about fancy gigabit infrastructure all over. You try sending an email or surfing on a long train ride when there are 10+ douche bags hogging all the bandwidth listening to Spotify…

Plaintext is also rather easy to parse. Combine this with a shell script, and your imagination is your only limit to the stuff you could do.

The future

There are definitely more uses for the server, such as VPN and perhaps a mumble server. Time will tell. I do know that I will spend the next couple of days trying to configure nginx and uwsgi so I could start doing some Python web-stuff.

I have been considering dropping Pidgin and instead give bitlbee a try. This would mean that all communication (with the exception of email and microblogging) would instead go through irssi. That would mean one less GUI-app, and also that all the logs would be gathered in one place (on the server) instead of the current situation where Pidgin logs are interspersed between the desktop and the netbook.

I have in a similar fashion thought about replacing Thunderbird with mutt. It would mean that I would replace one software (Thunderbird) with five (mutt, msmtp, offlineimap, abook and mairix).

Edit: Something very similar to this. (Thank you Thomas)

Well, four anyway, abook is already installed and operational.

msmtp is used to send emails (and thusly communicates with an smtp server.

offlineimap is used to sync an IMAP mailbox (fetch new mails, mark mails as read or deleted on the server, etc)

mairix is a full text search engine for Maildirs.

One of these days, I will also try to become better friends with zsh. It has many nice features which I think could increase my efficiency at least somewhat.

That’s all folks, I hope you have enjoyed reading the series as much as I enjoyed writing it. Comments are enabled in all the post, feel free to add questions or suggestions.

:wq