Posts Tagged ‘friends’

Thinking outside the box

Monday, October 20th, 2008

Edit: Refurbished the post with a better title, as suggested by mra.

A friend from my old class at the University sent me a message earlier tonight asking me some programming advice. I was in the process of stepping away from the computer but he caught me just in time, and he’s a good friend, and I have to admit that his sporadic inquiries, which also gives me some slight insight into how code in the corporate world can look like, sometimes prove quite challenging, and I do enjoy trying to provide help, guidance and the occasional solution. So I sat back down, made myself comfortable and read his situation:

One thing I have come to notice, whomever it is that asks me, is that they have a pre-conceived notion about how they would like to solve the problem. More often than not, the help I provide is more in line of  offering an alternative way of looking at the problem. And the key my apparent success lies in that I gather as much information beforehand, and then try to either remember a similar situation, or think up a solution based on previous situation or knowledge bordering the problem at hand.

He described that there existed a loop, in which values were being generated, and used to build a query, which was then generated (inside the loop) resulting in a result set. And following the loop, he wanted to make use of the result set derived from the iterations of the loop.

The primary problem, at first, seemed to be that the result set was being overwritten with newer information at every iteration, and at the loops end, there was only data in the result set from the last iterations query.

Problem solving mode kicked in, but soon reported that there was too many unknowns to come up with something resembling a solution, more information was needed.

Well, no, the first thought was how difficult it would be to go into the internal storage-structure of the result set object and “append” the information returned in this query with the result set from the previous query. I rejected that idea within a few seconds time of deliberation. (Don’t know about you, but I would not feel comfortable fracking around with the internal state of a Java object I really don’t know anything about.)

So, more information, did query X+1 depend on the information returned in query X? If not, what was the data being generated in the loop, which was fed to the query?

I am ashamed to admit it, but my next idea was almost as big an ugly hack as the first idea, which essentially was to create a result set array, and store the result set from each query in that array, and instead of manipulating one big result set at the end of the loop, just manipulate each element in the array individually. I honestly cannot say why I thought of that idea, but I think that may be a rather “hard-coded”¬†notion I get when I see Java code… in any case, all the while I was examining this hair-brained idea for flaws (I cannot say that I liked the idea, it would probably work, but it didn’t feel “right”) I started getting answers to my previously stated questions. The generated data was integers, id numbers to be specific, and they did not relate depend on the previous query.

The code, then, was basically something along the lines of:

for (i=0; i<something_predetermined; i++)
    resultSet = SQLquery(query, i);

The little code snippet I got to see looked a little different, it has been obfuscated to preserve his anonymity :)

That little tidbit of information made me think about what he had going into the problem, where the problem could be broken up into smaller pieces etc. Basically, he had a list of id numbers, or could at least generate such a list, independently from the loop and queries. That’s when I remembered a piece of SQL syntax that I have never had a chance to try out for myself. Something I’d read, but never had any use of personally. Something along the lines of:

SELECT * FROM foo WHERE id IN [1,2,3,4];

And I was almost correct. My only hurdles was that I a.) wasn’t sure that it was correct syntax, and b.) the specification use parentheses to represent a list of values, not square brackets. I.e.:

SELECT * FROM foo WHERE id IN (1,2,3,4);

This solution was something I would be able to live with. Not that I really had any stake here, but I couldn’t really recommend the usage of a piece of code I wouldn’t want to be caught dead using myself, but this, outsourcing the work to the database, work it was written to excel at performing, yes, this felt “right”.

Next question, would this syntax work in whatever database his project was powered by? As it turns out, it would seem so. A quick Google revealed that SQL Server should indeed support this syntax. My friend did some limited testing and that also checked out, so I considered my work, at least for now, done.


Friday, September 26th, 2008

I recently wrote a post about a friend of mine, and how he needed some help locating files containing certain phrases. He returned again today, seeking advice on how to remove information at the beginning of each line, from a file.

Depending on the structure of this data (or junk as he referred to it ;)) this may or may not prove a hard nut to crack.

He wanted to remove the junk data so that he could run a diff between another trace file and this trace file, which is really clever, and I was happy that I could be of assistance.

My first instinctual question to him was if the (junk) data was uniform in any way. It contained timestamps in the format HH:MM:SS, so the structure was identical, but each line would look a little different.

New angle of attack, is the junk data separated from the … well non-junk data in any way?

Yes, between the junk and the valuable data one can always find the string “TRACE: “. Bingo!
I asked for a couple of lines of output, so that I could get to work crafting a sed command which would do his bidding.

An example of the lines I received is:

18340:3063011104] 10:30:40 TRACE: Moving absolute 0 from -1

Basically, numbers, colon, numbers, ending square bracket, space, numbers, colon, numbers, colon, numbers, colon, space, TRACE, colon, space.

My first thought about what tool to use was: “sed”. I’ve used it before, not much as of late, so I had to experiment a bit before being able to hand over a working solution. One of the things that clung to my mind and wouldn’t go away is that I recalled needing to use the flag -i if I wanted to work on a file. Several attempts with the resulting error message “sed: no input files” later, I finally gave up on -i and tried without it.

UPDATE: As pesa pointed out, I was almost correct, and had I tried it a bit further, I would have dropped the “<” from the command, and it had worked. As he also suggested, the command I ended up with, cat:ing the file and piping the output to sed, was probably more portable as some installations of sed, on various systems, might not support the notion of an input file. Anyway, my memory wasn’t wrong, I was just in to great a rush to actually figure it out while trying to help out.

The first attempt, which FAILED miserably:

$ sed -i -e 's/^[0-9:\] ]* TRACE: //' < sed.txt > sed.1

I am a big fan of leaving the source untouched, and redirecting any changes into a new file instead, so the idea was simply to execute the substitution on the data taken from sed.txt, and put it in sed.1. Don’t ask me about the filename, I couldn’t tell you. “sed.txt.bak” was already taken by a backup of sed.txt (in case I would screw the original up I wanted to be able to do more tests).

I won’t bore you with all the attempts, but the final, WORKING command was:

$ cat sed.txt | sed -e 's/^[0-9]*:[0-9]*\] [0-9]*:[0-9]*:[0-9]* TRACE: //' > sed.1

There is probably room for a lot of improvement on the regexp, but it works, my friend is happier and more productive than ever (his own words) and all is well.

Mission accomplished.


Monday, August 25th, 2008

Grep is one of those tools that every GNU/Linux user should have at least a rudimentary understanding of. You will get by without it of course, but it can speed up things quite a bit.

Just today a friend and former classmate had a problem: In a large C++ code base, find the one file printing a specific error message. Opening every file and manually checking them: not feasible and surely not cost-effective.

He asked me for any insight in searching, and from the ole’ toolbox I brought grep. Now I will readily admit, I am no superuser, or guru or anything of the sort. My grep skills are not what they probably should be, so my first attempts was rather unsuccessful.

Framing the problem even more, the .cpp files where spread over a number of directories, and in the project root directory there where no files, only directories.

Since I mostly program in Python those where the files I had available to test my grep commands on:

$ grep 'import' *.py

I was greeted with an error, *.py no such file or directory. But the syntax was right, right? Went into a sub directory containing python files, ran the same command again, and was rewarded with a list of files.

Ok, so the problem wasn’t the syntax, it was targeting. What about

$ grep -R 'import' *.py

Again with the error message… ok, quick check in the man-page, yes, -R -r or –recursive all works, great, next try:

$ grep -r 'import' ./*.py

That error message is getting tedious… what about

$ grep -r 'import' ./

Now we are rolling, but it is chewing on things I have no interest in listing… like Vim’s .swp files etc. How do we fix that? Enter the man-page again, aha –include

$ grep -r --include '*.py' 'import' ./

Very nice, recursive search throughout all sub directories for files ending with .py containing the string ‘import’. Now to help him out a little more, let’s add -n also, so that he will see on what line the error message is printed.

$ grep -r -n --include '*.py' 'import' ./

And there you have it. Just one of the various uses of grep.


Friday, April 4th, 2008

Although yesterday was a fairly decent day in its own right, you made it special.
Yesterday was a first for me, and for that, I thank you my friend.

A great success!

Saturday, August 18th, 2007

This summer I have had the opportunity to perform a complementary project in order to get a grade in the AI project, which I flunked pretty hard during the third term. Yesterday, approximately one week after the deadline, the lecturers most closely involved in that project held and examination session in which me and the members of “my” group where to present our work, and explain / defend the choices we made as the project progressed.

I am happy to say that we all passed. In the end it was worth all the struggle; giving away all of the summer… however, one thing that should really be considered is that had we actually gotten started earlier, actually doing things, effectively, from an early stage, we would probably have had a working agent, plus documentation, ready and handed in after about one and a half months, and had plenty of time for summer too.

But I am happy, we all passed, we get to continue in term 5 now after the summer, with our classmates, if it continues this way I will graduate in roughly one years time from now.

So in closing; Gabe, Robbie: Good work! :D