Analyze and Re-Analyze.

Today, like most days, my day was spent solving problems that were not mine to begin with, but became either too complex or the person just needs someone to hash it out with. Either way, I enjoy these opportunities.

The majority of problems start with the lack of analysis, or the complication the analysis turns up. This afternoon, I was meeting with one of our tech system guys going over a support issue we received when a co-worker dropped by to hash out his new issue, which brings me to this blog post.

Analyze, Analyze, and then Analyze again! Analysis is a major part of every project and it something that should be taken seriously and completely is at Central (where I work). My co-worker had done quite a bit but had major questions about the project and the path he found best so far, primarily, would the data set he wanted to achieve be too large to handle efficiently. So, considering one of our tech system guys was already at my desk, and I am a bit of a SQL genius myself, we hashed it out.

We estimated that in the worst case scenario we would be approaching 15 MB of data, which is fairly large. However, during our discussion we also established that the data would only need to be pulled out of the database once per day. Now that is very important! Think about it, 15 MB is nothing if all you need it is one time. If you have to call it 25 times a day, that makes a hell of a difference.

However, the code path my co-worker choose would have made him pull out the data 25 times. Analysis with others, or re-analyzing your project due to the questions you are left with should be taken seriously. If the data your application is going to use will never change for 24 hours, then program it so it take advantage of this tidbit. Persist that data outside of your storage solution to reuse throughout the day so you do not have to hit up the server to retrieve that data over and over again (in a sense, cache it).

If the data might exceed a decent size to cache (ie: 15 MB), then write it to a flat file and open, read, and close immediately thereafter. That way, you still only pull the data from SQL Server once, and for the remainder of the day you work with the flat file.

By all means, keep your co-workers informed and when questions arise, talk out your solution with them to see if they have any better ideas, or agree with your approach.