## Posts Tagged ‘friends’

### Solving excel troubles by thinking out of the box

Saturday, July 2nd, 2011

A friend of mine just pinged me, wanting to know if I knew a way to programmatically add a range of number sequences to an excel worksheet.

I know nothing about excel scripting (or VBA-stuff or that type of things). But that didn’t stop me from getting an idea.

So I questioned him relentlessly until I was sure I knew what the expected output should be.

What he wanted was 16560 rows  in a worksheet. Rows 1 through 60 should contain the number 1. Rows 61 to 120 should contain the number 2, etc…

The final 60 rows should contain the number 276.

Why would he want this? I don’t know, some school work I presume. I didn’t particularly care, I assume his teacher wanted that format and was to lazy to provide a template…

I did however care that I found the problem interesting. How could I help, without any knowledge in VBA-scripting, and no desire to learn it within the next two hours?

The solution was rather neat:

$touch file.csv$ for i in seq 1 276;
do
for a in seq 1 60;
do
echo "$i" >> file.csv done done$ unix2dos file.csv

After having created the file, ensured it would work on his windows system, and transferred it to him (and he verified that it worked to import it and copy the results over to where he needed to work with it) I timed the execution of the loop.

2.33 seconds, against several hours if doing it manually, or $DEITY knows how long if it involved learning how to create a VBA script to do it for you… He was pleased, I had fun, all in all 25 well spent minutes. Win-win. And it was definitely easier, for me, to create a csv-file for him to import, than to learn me sum VBA. :wq ### I do believe I have been bitten by Python Saturday, May 30th, 2009 Lately I have found myself writing short texts in Swedish, destined to end up at a friends computer. A Windows-using friend, with all the UTF-8 / ISO-8859-1 hassles this entails. For the first file, I simply copied it onto a memory stick and rebooted into the Windows partition, and search/replaced all the offending characters (å, ä, ö and the odd é). Then rebooted again (since I don’t have my emails set up in Windows) and fired off the mail. I simply figured that this file would be kindof a one-shot deal and nothing more. About two weeks later, I wrote a second file, and re-did the entire reboot-procedure. I found myself writing a third file yesterday… I can’t for the life of me remember the saying, or where I read it, but it was something along the lines of if you do the same thing more than twice, automate the shit out of it. An audience with the great oracle lead me to this blog post and after trying it out manually (which required me to reboot one more time just to verify that the converted file had in fact been converted) I was all set to write a little shell script. I came so far as to write the first lines of error handling in the script (make sure that the script had recieved a filename) before I realized that I really didn’t want to write a shell script. Not when I could piece together a Python script in half that time, which would have better error checking. And yes, that time estimate included researching how to have Python execute a system call. (subprocess.call() is what I settled on, as per advise from StackOverflow. It took me a minute or so of reading the manual to figure out how to redirect the output from that command (the full text, in ISO-8859-1 encoding) to a new file (getting a file pointer to the new file, and redirecting stdout from the subprocess.call() to that file pointer) Something along these lines: fp = open('myfile.iso.txt', 'w') args = ['iconv', '--from-code=UTF-8', '--to-code=ISO-8859-1', 'myfile.txt'] subprocess.call(args, stdout=fp) fp.close() No more silly rebooting to convert plaintext files for me ### My day Wednesday, May 20th, 2009 The two wortwhile things I’ve done today, in chronological order: 1. Cheer up a dear friend who needed some kind words 2. Vote for the Pirate Party in the EU election ### Vim indoctrination Thursday, April 9th, 2009 Having used Vim (mostly gVim) exclusively for the last year or so my muscle-memory has thoroughly set. Which I was reminded of yesterday when a classmate from Uni asked me if I could lighten his load a bit by quickly adding a piece of functionality to some code he was working on. “Sure” I thought, I can do this. So I launched Eclipse to carry out a small scale controlled environment kind of test. The task was to, from an existing for-loop with its own functionality, add in the required statements to have the loop also build a comma-separated string of the values retrieved by the loop. That little adventure made me discover of two things: 1. “:w” won’t save the file in Eclipse… it will however insert those very characters into the code, breaking it. The same goes for “V”, “d” and “Y”. Also, “,cc” won’t comment out a line… that readjustment from Vim to Eclipse took way more time than hacking the actual Java 2. Python has spoiled me But all in all, it worked out pretty good, and I got to use StringBuilder for the first time ever. The resulting code looked something like this: StringBuilder sb = new StringBuilder(); for-loop here { pre-existing code here... if (sb.length() > 0) { sb.append(","); } sb.append(obj.toString()); } String s = sb.toString(); I’m sure there are better ways to accomplish this, like just tacking on a comma after each append and then on the resulting string working some string-manipulation magic to remove the final trailing comma, but for some inexplicable reason this just “felt” like a better solution. It might just be that now most of the actual code is grouped together, so in the event of a refactoring, there is less of a risk that the string manipulation code is forgotten. Anyway, it was almost fun to hack Java again… almost… ### I think the wrong question is being asked Tuesday, March 10th, 2009 Every new year is proclaimed “the year of GNU/Linux”. For me, this has been reality for the last three years. But every time you hear someone say that, someone else will promptly drop the question “is GNU/Linux ready for the desktop?” I now, since a couple of nights ago, believe that to be the wrong question. It is a relevant question to be sure, and for a lot of people, including me, the answer is a resounding “YES!” But a couple of nights ago I got a call from an old classmate. She had computer trouble. Her son had caused a minor accident, with the result that the webpage she was currently browsing now showed up at 50% scale. Her son had accidentally managed to have Firefox zoom out, making the text tiny. Before presenting what I believe to “the right question”, let me just first make it perfectly clear, my classmate is not stupid in any way, she’s simply just an average computer user which, for one reason or another, doesn’t stray out of her usage patterns. I.e. she never explores the features of her system or applications. That is fine, not passing judgement, just making an observation. If anything she has proven to be an attentive “student” at those times she has come to me for help. But immediately after trying the most basic of things (restarting the application) and realizing that the problem persisted, her next thought was “oh well, I guess I’ll have to stop using that website”. I am glad she had a change of heart, and called me instead, since it was an easy fix. But now, presenting my very own question: “Is the user ready for the desktop?” I, obviously, think this was an easy fix. Her knowledge, or lack thereof, made her second reaction be surrender. “If at first you don’t succeed… surrender?!” Obviously, for her this wasn’t an easy fix. Now I don’t have any statistics on the subject, but small things like this seem to happen all over my circle of acquaintances (ok, it doesn’t happen nearly as often in my group of friends from ITU, but sometimes is actually happens there as well). Either I have a nack for befriending computer illiterate people, or … well frankly, users aren’t ready for desktop computers. ### A friend just called me Google… Monday, February 23rd, 2009 I don’t know how to respond to that. I guess it is a great compliment, but aren’t all people walking, breathing knowledge-bases (henceforth kb)? Sure, some are more focused on a narrow field, while some have a broad, but shallow, kb. To be honest I am not especially thrilled with my performance in yesterdays “query”, as it took my friend two queries to get a satisfactory answer. His friend needed to record a music stream, preferably free software. “Streamripper” was my immediate response. Or possibly www.mediaconverter.org, once I learned that it was a stream originating from MySpace. That didn’t pan out. So he reformulated his problem space (something I should have done as well) and next wondered whether or not it wouldn’t be possible to create an application which would “rip” the sound stream as it was sent to the loudspeakers, and redirect it to the microphone port. Then a commonly available recording program could just handle the rest from there on out. “Like Audacity then?” was my immediate answer (with a small delay as I had to Google the link to that page). And that rewarded me with the comment “Damn, you’re Google” (ok, the real comment was in Swedish, but this is close enough) It bugs me that I hadn’t made the connection of saving audio-streams to “ripping” the internal audio-stream before this. But I guess this particular connection has been made in my head now (hopefully). ### LaTeX hacking Tuesday, February 17th, 2009 A friend of mine is currently writing his thesis, and is using LaTeX to do it (I would like to think that I had something to do with him chosing LaTeX over an ordinary word processor, but that remains to be discovered). In any case, he has become increasingly good at groking it (somewhat better at it than me I might add), and since he doesn’t have a blog (I’m working on getting him to “see the light” there as well ;)) and here are two issues which he encountered, and resolved on his own: The first problem was that some of the document classes reset the counter on the footnotes for every chapter, which he didn’t want. The solution: ﻿% no footnote counter reset \usepackage{remreset} \makeatletter \@removefromreset{footnote}{chapter} \makeatother The second problem was how to decrease the counter on an ordered list (for those few occasions when you want your list to start at 0 for instance): ﻿\begin{enumerate} \addtocounter{enumi}{-1} \item{...} \end{enumerate} And there you have it. ### CLI “magic” Wednesday, February 4th, 2009 Another day, another question. A friend of mine is working on his thesis, and wanted to replace all instances of a term, throughout a range of files. The problem could be formulated: For all files of type X, search through them, replacing every instance of foo with bar. In this particular case, the search term needed to be capitalized. So “foo” needed to be “FOO”. Why? Not my place to speculate, and not important to the problem, or the solution. Building upon previous experiences, sed was called back into service. $ sed -i 's/foo/FOO/gi' FILE

does the job. But on one file only. Time to widen the comfort zone a bit. I normally don’t use loops in the shell, mostly due to the fact that I haven’t taken the time to learn the syntax, but also out of a good portion of respect for them. Whatever command executed, is magnified by the use of loops. They should always be handled with a great deal of respect.

Personally I can live with some manual labor (i.e. executing the same command over and over feeding a new parameter every time) as long as I know that I can count on the command. It endows me with a sense of control. But my friend chose to believe in his version control system, and that his disks wouldn’t fail, that his backups wouldn’t be magnetically erased, that the big GNU in the sky (or whatever $DEITY he believes in) would have his back, and that I am competent enough to write a bash-script which would work according to specification. Ballsy, stupid but ballsy So off I went to the Internet, searching for the dark incantation I would need to have the command executed repeatedly over all his designated files. The answer came in the form $ for i in command
> do
> command $i > done After quick experimentation I concluded that “ls *.txt” would indeed only display the files ending with “.txt” in the given directory. Neat! All the pieces are in place, now to put it all together: $ for f in ls *.txt
> do
> sed -i 's/foo/FOO/gi' $f > done which, when collapsed into a single row amounts to: $ ﻿for f in ls *.txt; do sed -i 's/foo/FOO/ig' $f; done Or, you could just manually open up all the files in a text-editor, and for each file hit search and replace… The only thing I feel right now is that there probably exist an option in sed for modifying case built into sed, which would make it a bit more flexible to search for variable terms which share a common root (as an example, what if you wanted to capitalize all occurrences of president, presidents and presidential? There simply must be such a command in sed, so once I find it I will update this post) UPDATE: The solution did indeed exist, and was of course, simple. $ sed -i 's/$$foo$$/\U\1/gi' FILE

In order to do post-processing on the output, it can no longer be a static string (indeed that would not work since the whole point was to be able to match words with a common root, i.e several different but similar words), so it needs to be replaced by a back-reference to whatever was matched. Which means we now have to group the term we are searching for.

So the final incantation would look like this:

$for f in ls *.txt; do sed -i 's/$$foo$$/\U\1/gi'$f; done

### Splitting a PDF

Tuesday, January 20th, 2009

A friend just called me up and asked if I knew of any means to split a multi-page pdf-file into several single-page pdf files. My immediate answer was no, as I did not know of such a tool. A requirement was that it would work under Windows as well.

My instincts told that there would probably exist a web-service which could do what he asked for, however, the problem with online services is trust. How can I know what they do with “my” files after having let them operate on them? More specifically, what if it is sensitive information which they store, without my consent or knowledge?

So web-services was probably also out of the question (although I have no idea about the level of sensitivity surrounding his document). In any case, I hit Google with the the keywords “free” “pdf” “authoring” “software”. The top search result was a Wikipedia page. Scanning the page for the functionality I wanted, I quickly zeroed in on Pdftk (the PDF Toolkit) – can merge, split, en-/decrypt, watermark/stamp and manipulate PDF files.

The Wikipedia page, redirecting to iText didn’t amount to much, but now armed with the knowledge that what seemed like at least a library existed, I could hit Google again. “pdftk”, “split”, “file”.

Opening up a host of tabs from the search results, I stopped dead in my tracks upon finding AngusJ. To quote the site “PDFTK Builder & other PDF Resources for Windows”. I smiled for a bit giving myself a mental pat on the back for my awesome Google-Fu, and then I relayed the search terms to my friend (still on the phone, mind you) and directed him to the angusj.com search result.

For some reason or another, he couldn’t download the software. It cut out half-way through the download. I didn’t try it myself, but instead my mind went into “Plan B” mode. I.e. investigating whether or not I could split the pdf for him, using the command-line pdftk which, for some reason, was already installed on my machine, a fact apt-get promptly informed me of when I tried to install it.

Just then, it seems, he got a call from his boss, and splitting the pdf was no longer an issue. But as I had already started thinking about it, I simply continued my thought process and started experimenting.

Et voila:

pdftk [input_file] [action [arguments_for_action]] output [output_file]

$pdftk a_file.pdf cat 1 output page1.pdf$ pdftk a_file.pdf cat 2 output page2.pdf
\$ pdftk a_file.pdf cat 3-4 output pages3and4.pdf

in short… Awesome!

Of course, there are more uses for this toolkit, as stated on the Wikipedia page, and again on the official page, but since my friend only needed splitting, that is what I cover.

### A piece of Python

Tuesday, January 13th, 2009

The other day, a friend of mine from the university IM:ed me, asking for reading material on Python. My first instinct was to refer her to the Python documentation. Then my curiosity grabbed hold of me. It turns out she had an assignment, which boiled down to reading the first two lines in a sequence of text files.

She had crafted a solution which almost worked, but the inner loop (the one outputting lines 1 and 2 from each file) was giving her a head-ache, so she asked me for input.

c = 0
for line in file:
if c >= 2:
break
# do stuff with "line"
c += 1

It was eerily similar to her attempt, but I felt uncomfortable with the code… it just didn’t “feel” right. It felt as if there had to be a less stupid way to go about this problem (not calling her solution stupid, this was her first attempt at coding Python, I on the other hand, should be ashamed).

Thinking about it, one really didn’t need a loop at all for this assignment. In order to read the first two lines in a file, you’d need a handle to the file, and then call the function readline() on that handle, twice.

file.readline()
file.readline()

This is, of course, not the least bit maintainable, for instance, what would the solution be if the assignment was changed to read the 5, 10 or 15 first lines of text? No, that would just be plain ugly. So we bring the loop back in from the cold, to manage the number of lines to extract:

for i in range(1, NUMBER_OF_LINES):
file.readline()

The assignment didn’t say anything about line numbering, and although we could use “i” for this purpose it seems wasteful to assign a value to it and then ending up never using it, but seeing as we would otherwise have to create a counter anyway, to manually manage when to break out of the loop, I believe this solution to be the cleanest, simplest, most readable, and thus, the most beautiful. At least until someone comes up with something even better.

Writing this post, I am currently pondering about a while-loop approach, something along the lines of:

lines_left = NUMBER_OF_LINES
while lines_left > 0:
lines_left -= 1