SURRAnet Development

The King is Dead... Long Live the King!

2010-07-07T10:13:00.000-07:00

Long story short - this will probably be my last blog entry. "The king is dead..." If this blog had thousands of readers and other developers who where participating in the discussion, then I might reconsider keeping this blog alive, but that is not the reality. However...

"Long live the king!" While I've decided that blogging takes too much time from actual coding, I'm not totally leaving the scene. In a way, you can think of this as a regeneration (Doctor Who fans will understand). The blog has died, but it's reborn in a new, different way. If you are interested in SURRAnet or software development in general, and you want to continue following my development of SURRAnet (as well as other software products like my new iPad app called "Word Cross"), then I invite you to follow me at these sites:

The SURRAnet Facebook Fanpage - this page will basically pick up where this blog left off, except that it won't provide such long and detailed writeups. Facebook allows me to do more while requiring me to do less - more frequent, quicker, almost Twitter-like posts, but also screenshots, notes for when I want to write something "long and detailed", discussions, etc.

The SURRAtech Website - SURRAtech is the name of my "company" website, which right now is just me :-) I needed a domain from which to host apps, open source code, etc. This website is a work in progress, but it will be where I host SURRAnet code, instructions on how to download and use SURRAnet, etc. It also will host my other projects, which I expect will grow once SURRAnet 3.0 is finished.

The SURRAtech Twitter Feed - For those who prefer Twitter updates over Facebook, this feed will basically provide a mirror to the Facebook page, but it also includes updates regarding other projects of mine as well.

My GHCA Blog - If your interest is more in me and the type of things I do, then you may be interested in my main blog at Greater Houlton Christian Academy. GHCA is my primary employer, where I work as their system administrator and a part-time Computer Technology teacher.

I originally was going to delete this blog, but I realized that it's a good "diary" of at least some of my thoughts and experiences working on SURRAnet 3.0. In some ways I wish I had started a blog like this way back in 2002 when SURRAnet first began to take form. I'll keep the blog online for these memories, and for the links to the above resources. Thank you all who have been following my work, and I hope the new "king" will continue to serve you well!

Consistent Variables

2010-03-31T06:47:00.000-07:00

Sort of sounds like an oxymoron, huh? For whatever reason, I've gotten used to naming variables one way in Python and another in Javascript. In Javascript, I've been using the "variableNameExample" scheme, whereas in Python I've been using the "variable_name_example" scheme. I've been going back and forth between the two a lot lately, and I'm asking myself "Why?"

Part of me whats to switch everything to the first example, since I think is easier to type and looks "prettier", and it definitely is the more common scheme used in Javascript. Of course, changing everything over on the Python side will take time that I currently don't have. I may just start using the case-switch scheme for here on out, and then later go back and "sed" many of the underscore variables to case-switch variables..

Hindsight is always 20x20!

CS = Computer Sculpting

2010-03-22T10:07:00.000-07:00

We usually think of "CS" as being the abbreviation for Computer Science, but sometimes I wonder how much "science" goes into writing a program. Now I know of all the arguments - CS is more about the algorithms whereas CE is more about actually "engineering" a computer program, but in my experience, there is a whole lot of "art" that goes into programming. In fact, I feel more like a sculptor than a scientist or even an engineer!

I find that I often will write code, look at it, rethink it, "carve" stuff away, add new "stuff", tweak, modify, carve, and add until the blob of code begins to look like something beautiful, much like how an artist shapes a blob of clay. I do have a picture in my head of what I want, but unlike actual clay, software code has some very strict limitations, some of which I don't discover until I'm knee-deep pursuing a specific method of doing something. This is particularly true as I work with the UI, thinking that certain combinations of XML, CSS, and Javascript will give me what I'm looking for. But that isn't always the case, regardless of what the standards promise. Other times I get what I was looking for, but actually seeing it and using it makes me realize that what I had in my head wasn't such a good idea after all! Often it is a spur-of-the-moment idea that ends up giving me a user interface that just "feels right." These ideas rarely come to me on the first go around.

Like the UI, I also strive to make beautiful code. Simplistic and elegant, that's what I want, seasoned with a dash of cleverness. I also like the idea of optimized code, though I believe that simplicity will often result in optimization, and I'm not going to throw out beauty to shave off a few milliseconds on the rendering time of a page.

Maybe I'm just goofy this way. Maybe there are programmers out there who know from start to finish exactly what they want, and that's what they get on the first try. For me, the process of developing a large piece of software is more like how Yoda describes seeing the future - "Difficult to see. Always in motion [it is]..."

ps - usually lengths of time between blog posts mean that I'm distracted by other obligations (SURRAnet being a project that I work on in my spare time), but lately SURRAnet has been receiving a lot of attention. If the mind can focus on code, then the blog takes a back-burner. Today my mind is a bit fuzzy, so here I am! I have a new screenshot of my work on the academic section of SURRAnet. I also uploaded the new SN3.0 code a while back (which currently does not include the academics).

IANAL

2010-02-27T09:04:00.000-08:00

I have invested years working on SURRAnet. Most of the world doesn't know this because it hasn't been something that I've blogged about or posted online until recently. The code I'm working on now is the third major "rewrite" of the SURRAnet database, and I'm finally "proud enough" of the code to release it to the world.

I've become very comfortable with programming over the years, but as the saying goes, "I am not a lawyer." Originally when I published a copy of SURRAnet 2.0, I released it under the GPL. As a general fan and user of open source software, that just seemed the way to go. But as I considered how much work I've put into SURRAnet, I started rethinking this. The reality is that I'm more pragmatic regarding software "freedom" than Richard Stallman and the Free Software Foundation are. I'm more like Linus in this regards.

I may even have a little Bill Gates in me. WHAT?? Well, I find myself thinking thoughts that have some parallels to his famous letter to the hacker community. Why should I give up all my rights to the software I've spent so many hours on? I know, I'm walking a slippery slope with open source advocates, but ultimately the decision is mine as to how I license SURRAnet to the world.

I have two concerns. The first is that I "lose control" of SURRAnet. Now I'm sounding like Steve Jobs! The main reason for this is that I have very clearly defined goals for SURRAnet, and I have a specific way of how to get there. Once SURRAnet is finished, in the sense that I've met my goals, then I think I'll be more open to outside input. In the meantime, I don't want to lose control of the "master copy".

The other concern revolves around compensation. No, I'm not expecting to license every copy of SURRAnet for a fee or royalty. I don't want the headache - I'm a programmer, not a businessman! I don't mind other people using SURRAnet for free, modifying the source code for their own use, making copies for their organization, etc. I don't even mind Linux distributions packaging SURRAnet. In fact, that would be quite the honor! What bothers me is the idea of some corporation coming along and "stealing" SURRAnet, with some fat-cat businessman making millions selling SURRAnet as a package or SAAS (software as a service), while I'm left out in the cold.

HA - sounds like I'm dreaming, huh? Probably, but you never know! I can dare to dream big, and I can dare to think that SURRAnet may offer something that other programs don't, something that a corporation might invest in.

So what am I going to do? Well, I am not a lawyer... After spending a few days reading, researching, and bouncing around ideas, I've picked a license. I'm not 100% sure if it addresses all of my needs and concerns, but it's the closest one I can find that seems to provide a balance between users rights and my rights. The license I picked - Artistic License 2.0.

I like this license for a couple of reasons. First, it's simple. I can read it and understand what I'm reading! Second, it's not "preachy". I go to church to hear a preacher; I don't need my software license to preach at me, nor do I want to use a license that preaches a "dogma" that I don't fully agree with. That rules out the GPL.

But what I like most of all is that the license seems to reserve some of my own rights. From the license, "The intent is that the Copyright Holder maintains some artistic control over the development of that Package while still keeping the Package available as open source and free software." Since IANAL, I am still a little unsure about how the license protects my work from being exploited by a "fat cat businessman". Maybe there's a loophole, but hopefully if some corporation someday takes interest in SURRAnet, they'll see the wisdom of bringing the author of the software onto their team.

I'll have the first pre-alpha release of version 3.0 available for download soon. (I know, I've been saying that for months.) When I do release it, it will be under the Artistic License 2.0, unless somebody (maybe a lawyer) convinces me otherwise.

Note - there are some graphical elements in SURRAnet which I don't own. Those parts retain the licenses of the copyright holders (for example, Creative Commons).

HTML5, Python 3000, and Other Shifting Gangplanks

2010-02-22T08:40:00.000-08:00

I've spent some time at sea, and I'm quite familiar with that "shifting" feeling when boarding a ship via the spanning bridge called the gangplank. This is especially noticeable when sea conditions result in the ship and/or pier bobbing up and down. It takes a special kind of walking, sea legs they call it, to not look like a drunk while boarding a ship.

Programming SURRAnet feels a little like this, too. The "ground" I'm "walking on" is shifting. New features in HTML5 and the release of Python 3000 is causing me to compensate by considering what new features obsolete the current code that I have in place. The key is not to overcompensate. Otherwise SURRAnet will never get done!

What does this mean for SURRAnet? First off, I'm just not ready to make the switch to Python 3000. I thought about it, and even tried running my code in Python 3000 mode (a feature of Python 2.6), but the reality is that all my servers run Python 2.6, and whatever Python 3000 gives me isn't currently worth the pain of forcing it onto my servers. Someday the switch will happen, but not this release.

In fact, that's what it all boils down to - what is and is not worth the pain? HTML 5 also is very interesting, providing lots of nifty new tags and features. I'm hoping it provides better handling of tables (especially the ability to scroll the table body separate frowhat is and is not worth the pain?m headers, footers, and sidebars). However, there are some features that don't really make my life any easier except that it makes the code slightly prettier. Since I'm already using XML encapsulated in Python objects, I'm not ready to rewrite the whole markup language of SURRAnet just because HTML 5 came out. What I AM willing to do is to see if I can, using HTML 5, reduce complexity. If I can use a single HTML 5 tag where I normally would need multiple HTML 4 tags, then I'm all for it!

Figuring out what new the features in Firefox and Python offer me as a developer is a bit like walking on a gangplank in rough seas. My code may "bob up and down" a bit as I try different things. However, in an effort not to overcompensate, don't expect the SURRAnet code to be completely rewritten overnight to use all the new features of HTML 5, Python 3000, and whatever else new comes down the block. I'll fall overboard and drown if I try to do all of that at once!

New Look. New Policies.

2010-01-27T04:45:00.000-08:00

Call me crazy, but if I'm going from version 2.0 to 3.0, and I'm making changes to the GUI for the purpose of user-friendliness, then I feel the need to make the "theme" of SURRAnet look fresh and new as well. Unfortunately, artistic design is not my greatest strength, but I think I've come up with a new look that is pretty "cool". I was actually inspired by the Personas feature of Firefox 3.6. Unlike Personas, the look of SURRAnet is not customizable at this time, but hopefully it's easy enough on the eyes that users will enjoy the experience of spending time in SURRAnet. Who says that efficient must be boring?

The new policies are not SURRAnet related, but Google Groups related. I want to love Google Groups, but the nonstop spam makes it very hard to do so. In order to prevent spam from stacking up in my moderation folder, I changed the policies so that I must explicitly invite members to join. You cannot request membership directly to my SURRAnet group, but you can, through this blog, email me and request membership that way. This should thwart the spammers. So far, the group isn't very active, nor do I expect it to be until SURRAnet "takes off", assuming that ever happens. That said, if you wish to start a deep discussion regarding a particular piece of code in SURRAnet, the group is there for this purpose. Otherwise, you can always just comment on one of my blog posts.

Today I am focusing on SURRAnet for much of the day, except during Steve Job's announcement :-)

Version Numbers

2010-01-25T07:07:00.000-08:00

The more I work with this new release of SURRAnet, the more I realize that this is a ground-up rewrite of much of the code. Part of this is a result of cutting the tether with Apache, and a large part is a result of my better mastery of object oriented design in Python. Oh, and then there is enhanced browser support for UI elements.

My point is this - what I'm coming up with is different enough from the original "SURRAnet 2.0", that I think it warrants being designated "SURRAnet 3.0". Right now, it's SURRAnet 3.0-alpha, but hopefully with my focus now on this project, it can move to the beta stage in a few months. I'll probably keep calling it a beta throughout the new school year until all of the obvious bugs are discovered by my teachers and worked out.

Stay tuned - the first code release of SURRAnet 3.0-alpha is on its way!

Now We Have Hours Instead of Days

2010-01-20T07:52:00.000-08:00

I'm reminded of a scene in Star Trek II, Wrath of Khan, where Kirk and Spock trick Khan by replacing "hours" with "days" in explaining how long it will take to do basic repairs to the Enterprise. I had some projects, as part of my regular job, that should have only taken hours. Instead, they took many, many, MANY hours (days). It's like Scotty's miracle fixes in reverse.

SO... Here I am weeks later and not very far ahead with SURRAnet. Time is whizzing by! I really want to get the next version up and running before summer, because the cold main winters are much better used for programming than our beautiful summers.

Even today, I was supposed to be solely focused on SURRAnet, and yet my other duties and just random "stuff" have gotten in the way. As they say in the movies, Take Two!

Starting a New Year with New Resolve

2010-01-04T06:21:00.000-08:00

I find that a huge project like SURRAnet has its own sort of inertia, and with various other "distractions" in 2009 taking me away from SURRAnet, it has followed the law, "an object at rest tends to stay at rest." Now that 2010 is here, I have new resolve to apply an "outside force" to get SURRAnet production underway again.

Helping me regain my enthusiasm for this undertaking is a recent accolade I received during the accreditation process that my school, Greater Houlton Christian Academy, is undertaking. I'll post the exact wording of the commendation once that report is released for public viewing. Needless to say, it's a lot easier working on a big project without those nagging doubts, "Is this really worth it?" and "Does anybody really care?" SURRAnet may never go beyond the borders of GHCA, but its role in our school is important enough that I am recommitted to finishing this major rewrite.

A new year, even a new decade, is a great time to reconsider other aspects of life as well. Related to SURRAnet, I'm repurposing my twitter account to share my progress with SURRAnet (as well as other technology-related projects) instead of boring people with my personal life. As a relatively private person in "real life", I'm thinking that posting every personal thought and action online for the universe to read really isn't the best idea anyway. By using twitter as a microblog of my progress with SURRAnet, I'll be able to keep those interested "in the loop" without wasting too much time typing long blog entries like this one. I'll save this blog for milestone events, announcements, or the sharing of thoughts that require more than 140 characters.

Well, enough typing of English paragraphs. It's time to start typing Python code. I suspect today will be mostly spend reading my code and remember what on earth I was thinking back last summer when I last worked on SURRAnet. Keep an eye out for a new source code release soon!

Back to the Drawing Board

2009-11-16T09:11:00.001-08:00

It's been months since I last looked at the new SURRAnet code. When I realized that I wouldn't have the revised code ready for the new school year, I refocused on "old" SURRAnet to make sure the data can more easily transition to the new SURRAnet when it's time to make the swap. Of particular interest was the elimination of quarters and grade weights.

When I say, "Back to the drawing board," I'm not talking about redesign, but about rewrapping my mind around my original design plans. It may take me a week just to set the "stage" of my brain up with all the objects and objectives of the new SURRAnet.

Now that winter is almost here, I will be dedicating those cold, chilly days to SURRAnet, when I'm off from school, of course. Winter is a nice, quiet time of year, and I find it a great time to pound out computer code because who wants to be outside in sub-zero temperatures?

I would really, really like to be able to finish the majority of the new code before we print the final report cards this school year (June). I make no guarantees, as we know what happened the last time. I sometimes even wonder if it is worth it, now that open source SIS programs are more readily available, but, I put my hand to this plow, so I'd like to finish what I started. Maybe, just maybe, my SURRAnet has something to offer that the other programs do not.

Time to clear out the cobwebs and remember everything I forgot over the last three months!

When Real Life Interferes With Our Plans

2009-07-14T09:34:00.000-07:00

Today I received a letter that stated the following:

Dear Mr. Surran:

Justice Hunter has reviewed your request to be excused from jury service and has denied your request. Please report for jury service on ______, as instructed in the summons you previously received....... We will do our best to minimize the inconvenience.

In that previous summons, I was told not to plan anything from late July until late September, which very inconveniently overlaps with the start of the school year (as a reminder, I'm the system administrator and part-time teacher at Greater Houlton Christian Academy). I requested a change in schedule, but it appears that the judge doesn't agree that teachers should be available for the first month of classes...

My ability to finish SURRAnet for the new school year was being taxed before receiving this letter due to other major work-related projects. Now, it's practically impossible without making some insane sacrifices on my part.

After discussing the situation with my boss, we have decided that we're pushing back our goal for the launch of the new SURRAnet at GHCA until the following school year, giving me an extra year to finish it.

Part of me is relieved to have an extra year to complete SURRAnet since this is a massive project - a project that has to be tabled for a few months while I perform my civic duties. That's not to say that I won't be working on it, but SURRAnet will be truly a "in my spare time" project for the next few months.

I do want to push to finish the code on the Record system so that I can upload a snapshot of the new code for you to have, especially since this code has some drastic changes from what's currently available to download.

As for missing my first month of teaching classes (Computer Fundamentals and Computer Programming), along with the major time-crunch that my other projects now face, I can only say Grrrr!!!!

Google Chrome and Other Tidbits

2009-07-09T04:46:00.000-07:00

Recently Google announced a new operating system tied to their web browser, Chrome. I find this interesting for a number of reasons, but one reason should be obvious in relation to this blog - SURRAnet was written for Chrome.

Okay, let me rephrase - SURRAnet was written for the browser, and Chrome promises to be designed with browser apps in mind, so you could say that SURRAnet was written for Chrome, or even more appropriately, Chrome was written for SURRAnet!

I'm a bit tickled because SURRAnet was first conceived and designed as a web app before web apps were commonplace. It also is refreshing to know that SURRAnet has a good run ahead of it because it's using technology that's just now becoming mainstream. Cool!

Here are a few other tidbits. I'm hoping to release my latest work on SURRAnet sometime this weekend. I'm trying to finish up the majority of the Records system before doing this. This will be the first release with SURRAnet's own HTTP server code, meaning that it no longer needs Apache to run. The new release also uses XML vs HTML and has some major enhancements in the base classes. If you compare the new code to the old code, you will hopefully appreciate the differences. I'll set it up so that interested parties can try SURRAnet "out of the box" with little setup overhead.

One of the things slowing me down is the amount of time seemingly simple tasks can take to code for. Today's browsers still have shortcomings that making programming applications for them "tricky". I've been spending time working on the AJAX communication between client and server, and having solved those issues, I'm not working on the final polishing of the Records system.

Another thing slowing me down is distraction. It seems that all of my quiet programming spots are, well, NOT quiet lately! My office at the school is subjected to construction noise as summer renovations happen all around me. My home office is shaken by the noise of the constant convoy of dump trucks and farm traffic that plague my road. Our local library is surprisingly noisy and distracting. I have a friend with a cabin in the middle of the woods, and I'm tempted to shack up there for a week to get some work done! Oh, and don't get me started on the poorly-timed jury duty....

Well, there's not much more to say. Hopefully my next post will be accompanied by code! I'll end on this thrilling note - a tarball of SURRAnet, including a few years of student data, currently fits on a single floppy disk!! (No, I don't use floppy disks, but I remember their capacity). In the world of bloat, I find this very cool indeed!

Dancing With The Client

2009-07-01T03:44:00.000-07:00

This title is inspired by the show, "Dancing With The Stars". I was thrilled yesterday when I had an eureka moment, at least IMHO, regarding the "dance" that the server and client will perform to make the user experience of SURRAnet.

SURRAnet was a web 2.0 app before there was a web 2.0. Back when I decided to make SURRAnet's GUI engine the browser, it really wasn't common for applications to be built for the browser (and when they were, they were Java or Flash apps). Since SURRAnet's initial release, web apps like Google Apps, Facebook, and even this blog have pushed the boundaries of what a browser can do, thanks to the ever-expanding capabilities of modern browsers like Firefox. One of the reasons I'm rewriting SURRAnet is to take advantage of these new capabilities and thus remove the clunkyness of the "old web."

One goal for this rewrite is to find the right balance between server-side and client-side processing. There is a place for each, and if the right balance is not found, the product suffers. In fact, I often wonder if sites like Facebook are missing this balance, resulting in the client "stepping on the feet" of the server. In my opinion, using client-side scripting to do something that is easily done on the server-side or even in simple static HTML/XML is a mistake. Facebook has become extremely client-side processor intensive, and often Firefox just freezes up for seconds while the insane amount of Javascript churns away to generate the page you see. Take a look at their source code sometime - it's crazy!!!

I obviously don't want SURRAnet to perform sluggish like Facebook or even Google Docs. I want it to be fast and responsive. I also want it to require the least amount of work to use (right now it requires too much clicking and navigating). I'm definitely going to use both client and server-side processing, but what is the correct balance? How will these two dance together for the best performance?

My eureka moment came while working on the validation phase of my student records manager. In the past, the client Javascript did the validation. This required plenty of data to be copied from the server to the client, and Javascript has limitations that make this phase of writing SURRAnet less than fun. The old SURRAnet represented the student records as a form that needed to be submitted in order to be recorded (old web). The new version sends individual field data immediately to the server via AJAX, thus providing instant updates to the database and eliminating the need for a "Click here to save" button. It dawned on me that since I'm using AJAX in this fashion, I can make the server do the validation!

This provides a number of benefits, including:

Small and simple Javascript - This reduces the file size of the .js file needed to be loaded and processed by the client. On slower clients, like old computers and netbooks, this simple Javascript will run much snappier, as it basically just displays the results sent back from the server. Since the server is the powerhouse, it makes sense that it do more of the work when it can.
Complex code stays in Python - As a personal preference, I'd rather code in Python than Javascript any day! It also makes validation and data processing much easier with direct access to the database objects. Otherwise I'd need to copy many internal server objects to the client Javascript.
Security - Client-side validation places trust in the client, but what if a hacker tries to post data directly to SURRAnet and thus bypasses validation meant to protect the database from errors? By keeping the validation on the server, the data sent it will be checked regardless of its source, and bad data is not accepted.
Maintained Web 2.0 experience - Since I'm using AJAX, and SURRAnet is designed to run on a LAN / WAN, the user still gets instant feedback regarding mistakes. Even though validation is taking place on the server, it feels like it is happening client-side because it is just as fast. This might be different if someone were using dialup to connect to the school intranet, but for the average teacher using SURRAnet on a 100 Mbps LAN, this method is perceivably instantaneous.

I'm not pretending that I'm the first one to think of this fine balance of server and client scripting. I'm sure there are many web apps that do this already. For SURRAnet, however, it definitely is a move in a new and exciting direction. For me as a developer, it should shave off weeks of coding since the resulting code will be much simpler and "Pythonic". For the user, it will result in an interface that is fast, easy to use, and to the point. It's all good!

Location, Location, Location

2009-06-17T04:39:00.001-07:00

My wife and I have treated ourselves to a little present - an MSI Wind (hers is pink). The Wind is a nice little netbook with a 10 inch screen and great keyboard. It isn't as fancy as some of the latest netbooks, but with features like a 160 GB hard drive, card reader, and great battery life, it is perfect for my work, especially at the $300 price tag!

I made this purchase for a few reasons, but the main reason was portability while maintaining usability. I have one of the orginal Asus Eee PCs, and while that netbook is great for surfing the web and checking email, I would not want to code SURRAnet on it, for my eyes and hands would suffer before long! The MSI Wind, on the other hand, is a great little platform to work from. It gives me flexibility to take my work "on the road" in a way that my large, heavy, and power-hungry 17" HP laptop prevents me from doing.

As you can see from the picture, I can find some great locations to program from! I was inspired by Steve Gibson who is pounding out hours of code at his local Starbucks. For me, "location, location, location" is very important regarding how much work I can get done, since SURRAnet requires me to be very focused in order to juggle all the details and big-picture goals without dropping them all on my mental floor.

The one negative is the 10" screen, since SURRAnet is designed for a larger monitor and resolution. However, I can plug the netbook into a full-sized monitor when I have one at my disposal, and I do a lot of testing even with a limited screen size.

I'm still looking for the perfect programming environment. My home office is great when the dump truck army isn't shaking my house. The hillside is great when swarms of hornets aren't buzzing over my head. Maybe I'll try the local library. If only the beach were closer!! With this little netbook, the sky is the limit!

if insomnia or dump_truck_noise: raise RuntimeError

2009-06-09T06:12:00.000-07:00

There's nothing like a 10 day stretch of bad sleep to mess up one's mind and make complex, object oriented programming almost impossible. Add to this a fleet of noisy dump trucks going back and forth all day long on an otherwise peaceful country road, and code production comes to a standstill.

People who do not program really don't understand the mental exercise this discipline is, especially with large programs that have hundreds of pieces that one must mentally keep track of at all times. There are plenty of tasks that I can do with "half a brain", but programming SURRAnet is not one of them! I find that I almost have to enter a higher state of consciousness in order to see the big picture and simultaneously focus on specific lines of code. It's like 3D chess in some ways, with the need to predict the outcome many moves into the future. Needless to say, a tired, fuzzy mind and a distracting environment are poison to the process.

So... I'm a week behind where I'd like to be. I'm slowly getting back on a decent sleep schedule, and soon I'll have a more portable programming platform (the MSI Wind) so that I can take my software development on the road to quiet and peaceful locations. When my mind is clear and alert, my productivity increases 10 fold!

BTW, in tracking down why I haven't been able to sleep, I've come across multiple reports that Splenda can cause insomnia. I recently switched to Splenda as part of new eating habits I'm adopting (with mental clarity being one of my goals). I have since stopped using Splenda and hope to return to a normal sleep pattern very shortly. The brain is a very complex biologic machine, and chemistry has a lot to do with its efficient operation. That chemistry is determined in large part by what we eat. As SURRAnet becomes a focus and my September deadline looms, I'll be eating lots of brainfood! Feel free to share what helps keep your brain fit by commenting on this blog post.

The True Beauty of Object Oriented Programming

2009-05-29T02:50:00.000-07:00

When first introduced to object-oriented programming, I really didn't like it. Looking back, I think I understand why. Besides the obvious fact that OOP takes a while to wrap one's mind around, it often is presented as another way to do procedural programming. In fact, I remember seeing many such examples where I scratched my head and thought, "Why do you add all these layers of complexity just to do that?!?" Think about it, how many times have complex classes been used where a simple function or even a simple script would have done the job with less code and less confusion?

As I was reading the book Dreaming in Code, I came to a part that resonated with me. Actually that happened many times, but the time related to this blog entry was when the Chandler team hired a Python expert to look at their code. He scolded the developers for using Java methodologies in Python, thus creating a huge and overly complex code base. I look at how OOP in Python is presented in many books and websites, and I think that the Chandler developers are not the only ones guilty of this.

So in rewriting SURRAnet, I'm trying to break away from some of the mistakes I've copied from examples I've seen in the past. Often classes are treated as functions on steroids or variables with attributes, but I think the true beauty of OOP is deeper than that. In the SURRAnet rewrite, I'm trying to harness the true power of Python OOP by creating the equivalent of custom types.

I've already done this in some places. For example, the "core" SURRAnet database class, SDBM, is basically a simple Python dictionary (dict) with file operations added to it. It's simple yet beautiful. The SchoolYear class is another example of objects that behave like special variable types, allowing me to use those objects in very intuitive and simple ways.

Yesterday I reworked the StudentData class (now just Student) in light of the added flexibility I have since breaking away from Apache. Instead of being a sort of ugly extension of the StudentDB class, it is truly a "student type", and instead of handling student IDs all the time, I can actually just handle Student objects!

This adds many benefits and simplifies much of my code. Let me leave you with this one example. In the past, sorting students by their name was a big pain, since students were represented by integers (their ID), and sorting those integers would not do. If you look at the current code I have available for download (at the time of this writing), you'll see this strange voodoo sort method in the StudentDB class (sort_stuids_by_fieldname or something like that). Now, by overloading the __cmp__ operator of the Student class, I can have lists of Student objects and just call a regular sort() on that list, and poof - sorted students! No funky voodoo! Here's an example:

SDB = database.StudentDB() students = SDB.students students.sort() for student in students: print student.fullname

Needless to say, the code is simple, beautiful, and logical. I'm quite looking forward to using my new student "type" today as I continue to switch SURRAnet over from Apache-based to a stand-alone server program.

My Own Little Web Server

2009-05-27T04:24:00.000-07:00

As detailed in my last entry, I've been contemplating the need for Apache as part of the SURRAnet stack. Yesterday I whipped up a nice little web server that handles all SURRAnet's static content (html, xml, css, js, images, etc.) quickly and efficiently. In fact, I was amazed at how easy it was to make my "own little Apache" server!

I am thus putting my hand to the plow and not looking back. Apache is bye-bye! Now let me say for the record again, I love Apache - it's a terrific tool. It's just too much tool for my job. I love big jets like the 747, but I'd rather bike to work than fly a big jet to work every day.

In my last blog post I mention some of the great benefits of going Python only, but there is another huge benefit that I'm still wrapping my head around. You see, Apache always ran in the background, but SURRAnet only ran when generating a page for the user. The problem with this is that big, expensive (in cycles) to set up objects like the databases only existed for a fraction of a second and would then disappear into the computer "ether" until the next page was called for, requiring these objects to be regenerated all over again. I tried to figure a way to keep objects alive in mod_python / Apache, but never found a satisfying way. I worked out a way during this current rewrite to at least make object creation as efficient as possible, but now:

Now I can have SURRAnet itself running full time in the background. My "snserver", which is my little web page server (technically it's an http request handler) can build all the core database objects when the program is first started and keep them in RAM for the duration! RAM usage may go up just a tad, though I bet cutting out Apache will more than compensate, but CPU and I/O utilization should decrease drastically. There is a lot of stuff that only needs to happen once, like sorting students into their classes, retrieving records from the hard drive, etc.

This may end being the biggest change to the "behind the scenes" in SURRAnet since switching from PHP to Python. My big goal for this rewrite is to remove more complexity than I add, and I believe going to a 100% Python solution will definitely remove complexity. Now I'm focusing on how exactly I'll handle "sessions" and per-page code. I have some ideas. Hopefully by today's end, I'll have a working model.

Apache Or Not Apache, This Is My Question..

2009-05-25T11:06:00.000-07:00

I'm very seriously considering removing Apache from the SURRAnet stack. If it is possible without too much heartache, then now is the time to do it while I'm undergoing such a major rewrite of the core code.

Apache is there to begin with because SURRAnet was originally written in PHP. Boy did I learn a lot from that mistake! SURRAnet 2.0 is the "2.0" because it went from PHP to Python, and I couldn't be happier with my choice to use this language. However, my original Python code (much of which remains in the latest tarball that I uploaded) is "ugly" due to my trying to marry HTML and Python.

A lot of my work this past Spring was creating a nice abstraction layer to separate the HTML (now XML) and Python via my widgets.py module, which continues to be refined and grow with new features. In fact, I'm not far from being able to generate very nice XHTML in an easy and clean Pythonic way.

This got me to thinking - why do I need Apache? Besides the obvious "to serve content via http", Apache basically is just glue binding my Python code to simple HTML templates. In this regard, it seems like a lot of overkill. It also brings extra baggage in configuration, installing mod_python, etc. If SURRAnet will someday be an easy "download, install, and use" program, it would be nice if the whole thing can be done in Python so that all the end-user needs is Python, nothing else.

However, there is that nagging need to "server content via http." As a reminder, SURRAnet is designed to be used on an intranet by a relatively small group of teachers. The web component is handled separately via PHP on an external web server. Again, Apache seems like overkill for this environment. I think of something like CUPS, which has its own HTTP server, and I'm wondering just how hard it would be to make a very simple HTTP server for SURRAnet using the BaseHTTPServer module.

The changes this would bring would go beyond just how the content is delivered between SURRAnet and the client. If I wrote a HTTP server, it would keep a "living" database object in RAM at all times, which could greatly reduce I/O stress and increase performance. However, that means that the internal changes that would need to be made would be much bigger than I originally planned for. I need to have a working system by September for the start of GHCA's 2009-2010 school year. Finding myself with so much in common with the story told in Dreaming In Code, I wonder if this is wise.

I suppose the first thing for me to do is play around with the BaseHTTPServer module and see how capable it is and how easily I can plug SURRAnet code into it. I would love to ditch Apache, not because Apache is a bad thing, but because I want SURRAnet to be as lightweight, easy to install, and easy to configure as possible. Why use a wrecking ball to pound a nail when a hammer does just fine?

I'll keep you posted as I experiment with this idea.

I Had to Cheat and Lie

2009-05-20T04:30:00.000-07:00

I finally was able to create the user interface that I've been aiming for, but I had to "cheat and lie" in order to do it. I don't like cheating or lying. I prefer honesty, both in my daily life and my computer code. I'm hoping that once the CSS spec adds some sorely missing features and web browsers do a better job of implementing this spec that I'll be able to remove the various hacks that I needed.

Before I give you some examples, let me explain that I have indeed switched to XML for providing SURRAnet data. So what looks like a typical HTML table on the screen is actually custom XML that has been "shaped" using CSS into a table layout.

Now let me give you some examples of my cheats and lies. First example - In order to provide both horizontal and vertical scrollbars to a table body that allows it to be scrolled without losing the top header row, I had treat the body as if it were a header. Using CSS this looks like:

table-layout: table-header-group;

If I use the intuitive css "table-row-group", everything breaks.. Very strange. So by using this cheat I can have both scrollbars available at the same time, and the header stays on the screen just like a frozen spreadsheet row.

Another cheat is needed to keep the header in sync as I scroll the table horizontally. It's a pretty easy cheat which uses Javascript to manually scroll the header the same amount that the table body is scrolled. That said, the UI I'm creating isn't anything new. One would think that CSS would allow for this as easily as it does horizontal scrolling.

I also had to hack my way around some buggy implementations of Javascript in Firefox, etc. It's a tad bit annoying due to the amount of time needed to discover why something doesn't work as it intuitively should, and then to brainstorm "hacks" to overcome deficiencies in the platform.

The good news is, I have finally achieved what I was beginning to think as impossible. Having conquered this obstacle, I hope to get back to creating useful and productivity-enhancing features into SURRAnet.

Boldly Going Where I Have Not Gone Before

2009-05-13T06:12:00.000-07:00

Okay, I must admit, "boldly" may be a the wrong word. Maybe "hesitantly" would be a better adverb. What am I talking about? XML.

XHTML to be precise. I actually considered the role of XML back when I first started SURRAnet version 2, because SURRAnet is indeed data-centric, and XML seems better suited for handling custom data. It was my unfamiliarity with the details of XML / XHTML and its relationship with page layout in a browser that kept me sticking with safe and familiar HTML.

However, recent trouble with tables has me rethinking how I build these tables to get past the limitations of the HTML-specific table structure. The "work horse" of modern website design seems to be the <div> tag, which is then controlled by CSS to build all sorts of interesting layouts. If I were to go this route, I'd need to "micromanage" my <div> tags, which is what I would have to do with my own custom XML tags anyway, so why not have XML tags that are specific to SURRAnet? After all, SURRAnet is more of a intranet application than a website.

Now I just have to wrap my head around XHTML. I'm "cool" with the idea of creating custom tags and using CSS to control their visual properties. What I need to spend time with is the DTD and how to write my own. In some ways this feels like more work on top of an already big project, but one of my big goals is to "future proof" SURRAnet so that the hard work I do now will stand the test of time for years to come. Moving to XHTML seems like a good idea in this regard.

Luckily I chose to do this while I'm in the middle of rewriting the widgets.py module which serves as an abstration layer between Python-only code and the PSP templates. In fact, I'm quite happy in how nice and "object oriented" the widget objects are turning out. As I just said, the hardest part of this will be figuring out the DTD.

I suppose the best way to learn something new is to dive in! Let's see if I can create a custom table based on my own XML scheme... Maybe I'll even get the on-screen layout that I've been working so hard to achieve!

The Trouble With Tables

2009-05-12T04:55:00.000-07:00

My title is inspired by the "The Trouble with Tribbles" episode of Star Trek, the original series. Like Kirk, I'm finding some basic lack of features within my beloved CSS to be a great distraction from the serious work to be done.

Shown to the right is a screen shot of what I'm working on. Notice the difference in the menu system on the left. My goal is to have a navigation system that is intuitive and requires the least amount of clicks to get from home to destination. Now I would like to draw your attention to the table itself. It looks basically the same as before, but there are indeed some major improvements. One such improvement is the locked header that stays put as the list is scrolled vertically. This is easily achieved with CSS and the use of a table header and a table body. The scroll bar attaches to the table body instead of the entire browser window.

But wait, where is that scrollbar? It's off the screen, since the horizontal size of the table is bigger than the window width! This is NOT what I want. I would like to be able to attach the horizontal scroll bar to the table as easily as I attached the vertical scroll bar and allow both to be visible at the same time. However, it seems that CSS stops short of providing what would be a very useful feature.

That's not to say that it is impossible to achieve. Google Spreadsheet does this. However, it appears that Google has a ton of "Javascript magic" working in the background, and I bet the HTML is an ugly hack to make this work. I strongly dislike ugly, processor-intensive hacks. Strongly.

In fact, I'm somewhat old-fashioned that way. That I'm writing SURRAnet in 100% text editor, including the HTML, is proof. I'm much like the venerated Steve Gibson, creator of SpinRite, in that I believe strongly in simple, fast, to-the-point code. That said, I'm not going to switch over to assembly language for SURRAnet, but I'm also not ready to start using some of the massive Javascript libraries out there that do indeed achieve my goals. Not yet. I want to see if there is an easy, fast, SURRAnet-specific way to get this job done. I don't want SURRAnet becoming the next Facebook - bloated and slow as molasses.

So this is my focus right now. The table is a key part of SURRAnet, and I'm making it even more so as I program additional functionality into various table views. The "ultimate" table is the gradebook, which emulates the look and feel of a spreadsheet, with some obvious differences. The more I can do via CSS and the less "hackery" I have to do in creating the table and manipulating it with Javascript, the better. Thus is my focus again today.

Back On Track

2009-05-08T02:46:00.000-07:00

SURRAnet often takes a back seat to other time-sensitive tasks related to my job, but as the new school year rapidly approaches, SURRAnet is becoming one of those time-sensitive tasks! Thankfully I am able to rearrange my schedule to allow more coding time.

Yesterday I worked on the records table view, which is almost complete. I continued to further abstract the HTML from the Python via my widgets module, so that there is zero HTML in the code that generates the pages. On the flip-side, I've reduced the Python code in the HTML layout to a bare minimum, so that everything is as "pure" as possible, making for much better readability, tighter code, uniformity across all the pages, etc.

I hit a glitch with the CSS tag max-height, which does not seem to work properly in Firefox for some odd reason. I need to research this.

Of course a lot of the day was spent debugging and looking for oddball glitches...

I must admit, programming huge (for me) pieces of software with multiple modules each with multiple objects is not my favorite thing. While I like my classes and the flexibility that their objects provide, I struggle keeping all these things in my head at once. It tires me out much more than what I typically need to write as a system administrator. I suppose the problem is that I'm one person working on a team-sized project, but that's most likely the way it will be until Fall. Once I have the core of my "vision" in code, I'll be open to allow others to help me with it.

My goal for the coming week is to finish the records portion of SURRAnet and move on to the curriculum section, which is one of the most complex sections behind-the-scenes. I need to rethink how I treat our curriculum because so many students have taylored courses. Oy.

I'll post a developer's snapshot as soon as I finish with the records portion, in case anyone is interested in what's happening behind-the-scenes.

The Wonderful New World of CSS

2009-05-06T07:17:00.000-07:00

I've been a fan of cascading style sheets for some time, but with recent advancements in the CSS definitions, and more importantly, browser support, CSS is taking on a whole new sense of wonderful!

When I first started SURRAnet, CSS support was very minimal. SURRAnet version 2 was able to take advantage of many more CSS features, but I still found that I had to use a lot of "tricks" in Javascript and the generation of the HTML to do what I want. In my recent efforts to clean up the SURRAnet code and provide the features listed in previous blog entries, I have thrilled to see that CSS takes care of so much more than it used to.

One big example is scrolling tables with locked headers. SURRAnet has a lot of tables and even simulates a spreadsheet in many ways with the gradebook, but the "lost" headers due to scrolling was a pain, and the work-around was messy and complicated. Now I can have some simple CSS take care of the header locks and focus on generating the content in my Python code.

The one problem with CSS, and also Javascript, is that browser support is all across the board. What I have chosen to do is to focus on one browser - Firefox. Most of SURRAnet is designed to be deployed in a single organization where the IT department can provide the same browser to everyone. Firefox is open source and cross-platform, and it seems much more standards-friendly than IE. Understand that I'm not trying to program FOR Firefox (taking advantage of Netscape-only tags), but if CSS and Javascript works great in Firefox but breaks in IE, oh well..

CSS is helping me separate all the various layers of SURRAnet. Separating database processing from content generation from layout results in much easier to maintain and modify code. I'm also taking advantage of extra features that are now easier to implement thanks to CSS. Hopefully you'll find the new role CSS has in SURRAnet as a huge benefit for users and developers!

New Ideas

2009-03-03T07:33:00.000-08:00

While I am doing things like cleaning up the code and restructuring SURRAnet to be more configurable for other schools, I am also thinking of ways of extending SURRAnet's capabilities. As stated previously, I'm not interested in it becoming the next Emacs, but I am interested in making SURRAnet a flexible and powerful student information system. Here are things that I am considering:

Part-time students - I want our school to offer "A-la-carte" classes. This may be via our upcoming online courses or the ability for homeschoolers to take a class or two without becoming full-time students. Currently SURRAnet has no sense of a part-time vs. full-time student. It should be relatively easy to implement through the addition of an extra field and code to consider this in forms, etc.
Adult education - similar to my previous note, I want to be able to add adult students as well.
Customized courses - In a class of 20 students, a large range of academic abilities may be represented. This often results in tweaking the course for individual needs. Currently SURRAnet has no easy mechanism for this except to create an entirely separate course for that student. I want to change this. This may result in changing how assignment data is stored, in that more information will be stored in student assignment records (like title, description, etc.) It also may change how course data itself is stored. I will be giving this more thought, as it will require some major code changes.
Transfer students and course add / drops - We get students joining us part-way throughout the schoolyear, and we also have students dropping and adding courses. This ties in with my previous bullet - I need more flexibility in mapping courses to students.
External data feeds - what I mean by this is that I want an interface that allows me to import and export data easily between SURRAnet and external applications. I'm particularly interested in creating two special apps - one is the ability for computerized exams to directly record grades to SURRAnet. The other is:
Automated attendance system - Attendance is a "tricky" subject, because what makes an absence vs. a tardy vs. a dismissal? If somebody is dismissed two hours after arriving at school, shouldn't that be considered an absence? My goal is to move to a "punch clock" style of attendance tracking. Students "punch in" and "punch out" and SURRAnet will track those times, hours present per day, percentage present, etc. To facilitate this, I would love to have some sort of automated system, like a barcode reader or a keypad to allow students to easily and quickly clock in and out.
Customizable extend records - I want users to be able to add their own "extended records" to what SURRAnet provides as default record information. For example, locker numbers! If users can create their own fields, this will provide a whole new level of flexibility to SURRAnet while not greatly changing the current design.

The big theme here is "flexibility". Obviously I can't make it too flexible at this point without a complete rebuild-from-scratch design (maybe someday there WILL be a SURRAnet 3.0).

Anyway, enough blogging - I have coding to do!

Sickness Setback

2009-02-27T11:00:00.000-08:00

My day job is the system administrator / CIO at Greater Houlton Christian Academy. Did you know that schools are like germ factories? I spent all week at the school while everyone else was on vacation, no problem. But the day everyone comes back, I get a dreadful cold..

Maybe it's age or maybe it's me or maybe it's normal for everyone, but I have a hard time writing complex code like what's found in SURRAnet when my mind is even a little bit foggy. Needless to say, production has been on hold. I'm praying that my immune system kicks up the counter-attack and drives these invaders from my body soon!

Cleanup Begins

2009-02-25T04:15:00.000-08:00

I'm going to start today cleaning up the modules. I have a confession - while I'm a big supporter of open source, I had a hard time initially releasing SURRAnet, because like many others, some of my code embarrasses me. Especially the early stuff.

I'll be starting a new "branch" that will be purely development from now until this Fall when I put it online at my school. This will allow me to gut out the old stuff without breaking anything (actually, I'll be breaking all sorts of stuff, but I won't care because the goal is to fix it again).

Since I am going through this process, I am also taking some time to consider what SURRAnet is and what I want it to do. I do NOT want it to become the next EMACS (in the sense that it has too many features). What it does do I want it to do very well. SURRAnet is a student information system, so I am considering just what information is handy for a school to keep track of for students.

Of course I want to keep it basically the same - reinventing the wheel over and over again is a mistake that too many software designers fall into (read Dreaming in Code). At the same time, I don't want to have to redesign the core modules every year just to add a new feature.

Anyway, I'm going to attempt to do a better job at the "post early, post often" motto of open source. SourceForge is right now overkill for my purposes, so I am going to create a page on the GHCA website that will make it easier to get SURRAnet development code and look at it. This blog will also be a major part of keeping track of what I'm up to as I plug away at this daunting project :-)

Beauty and Efficiency

2009-02-20T02:54:00.000-08:00

These are two goals that I have before me as I go through the SURRAnet code base. Sometimes one leads to the other, and other times one must be sacrificed for the other. I shall explain what I mean by beauty and efficiency.

I'll start with efficiency. I recently did quite a bit of research and experimentation in optimizing Python code. I was amazed to see how certain "tweaks" can drastically speed up code execution, especially when done in a large or multi-layered loop. One example is the overhead of looking up variables in the namespace. For example, a local variable "x" is quicker to access then an instance variable "self.x". By copying instance variables into local variables (x = self.x) before entering a large loop, many seconds can be shaved from code execution time. This and other "tricks" will be especially useful in generating large tables and reports.

Of course, it is often "prettier", based on simplicity, to just deal with "self.x" without the extra code to copy back and forth to a local variable. I am taking the advice of one programmer, "Optimize just the inner loop." In other words, if I am iterating through a loop 10 times, the extra code is not worth just a couple of milliseconds of performance.

There are two other areas of optimization I am working on. One is fixing a mistake that I made in my understanding of how mod_python handles namespaces, global variables, and objects across multiple sessions. As a result, I have new objects being created and destroyed multiple times during heavy looping. This definitely adds overhead, especially if I am creating new database objects. For example, by making StudentData a child of the StudentDB class, every time I looped through a bunch of students, the underlying StudentDB class would be initialized over and over. I have recently rewrote this so that StudentData now takes a single instance of StudentDB that is created beforehand, which is obviously before any looping. This will bring logical and perceivable speedup to SURRAnet.

The other area is caching. The idea is this - it takes a lot of work for SURRAnet to generate reports and large tables (like the grade sheet). Since the output to the screen is simple HTML, why not save a snapshot of this HTML to a cache and just reuse it the next time the user wants that report / table? I already have this implemented in later code in SURRAnet and it works well. The trick is determining when the cache is stale, which is basically when any data is changed that affects the given report.

Regarding beauty - Python is a beautiful language to program in. I personally need beauty in my code, not just for the sake of beauty, but for the sake of readability, debugging, and enhancing. I am going through and applying techniques and new algorithms to areas of older code written when I didn't know the best way to do something (examples including using map(), enumerate(), some new Python 2.5 features, etc.).

However, what makes my code "ugly" is the HTML scattered through it like buckshot. Let's face it, what you browser renders when using SURRAnet is HTML. The Python code must generate this HTML. So how do I get the HTML out of the Python code? Wrappers.

I started with with the widgets module, but that module was a bit too high end and not enough low end. For example, I would provide a list of data and try to generate a table from that. That works for simple tables, but not complex ones. It also adds the overhead of looping twice - once to create the data list and again to render the HTML. My new approach is to wrap the simple HTML entities into objects like TableCell, TableRow, etc. These new objects allow the building of the HTML during the initial loop. In fact, the objects themselves become the containers, instead of using separate lists! The end result? Much cleaner Python code. Below is an example of new code that generates a list of absent, tardy, and dismissed students for the day (note - my blog deletes the Python indents:

# Now on to generating our absentee table
header = TableRow('th')
header.addseq((' ','Absence List','Tardy List','Dismissal List'))
table = BasicTable(header)
# Provide a function to determine how we display the info in our table
def format_data(name, data):
if data['type'] == 'absence':
return "%s (%s)"%(name, data['reason'])
else:
date = database.datetypes.Day(data['timestamp'])
return "%s at %s (%s)"%(name, date.time, data['reason'])
for grade in self.gradelevels:
row = TableRow()
columns = {'absence':List(), 'check-in':List(), 'check-out':List()}
grade_sIDlist = list(sIDlist.intersection(SDB.gradelist[grade]))
if not grade_sIDlist: continue
for sID in SDB.sort_stuids_by_field(grade_sIDlist):
student = StudentData(SDB, sID)
attdict = self.day_attendance[sID]
columns[attdict['type']].append(format_data(student.name, attdict))
row.addcell(grade,'gradelevel')
for col in ['absence','check-in','check-out']: row.addcell(columns[col], col)
table.addrow(row)
self.html = table.html

Moving Forward - The Big Picture

2009-02-19T13:31:00.000-08:00

There is a time when technology is first adopted that it is forced into the "old wineskins" - those old ways of doing things before the technology was available. For example, many of the first websites were nothing more than electronic brochures. Then people start to realize that they can do so much more with the technology once they let go of the old ways and adopt "new wineskins." Facebook is a good example of just what a website can be.

Our school's administrative team is starting to see the possibilities of doing education using the proverbial new wineskins. For example, the question was raised, "Why do we print report cards every quarter?" Since SURRAnet allows parents and students to access their grades at any moment online live, what is the point of wasting this paper and setting aside four "report card days"? This question led to the next logical question - why divide the academic year up into four quarters to begin with when most of our classes are single year-long courses?

They say the best programmers don't add code to make their software better, they remove code. There is a ton of complexity built into SURRAnet around this concept of academic quarters. There is other complexity as well, like weighted points, that is required because SURRAnet was forced into old wineskins. I've been given the green light to remove all this complexity from SURRAnet. Let me tell you, I am thrilled to no end!

When I am done with SURRAnet, it will no longer care about time, except in regards to the academic school year (the start and end date). Grades won't close at artificial markers on the calendar (our current quarters) nor will there be four individual grades and a final average. Each course will only have one grade - the final average. Nor will it matter if one course is a semester long course while another is a year long. Courses will be identified by their credits, and this will allow some students to take a full year to finish a 1/2 course credit if necessary. It will also let fast students finish a year course in less time. Obviously that will require flexibility in the school's schedule, self-pacing, etc., but SURRAnet need not be concerned about this. It only needs to track the grades.

I do realize that there is benefit of generating reports for a specific slice of time, which is the original purpose of the academic quarter, but it is even more useful to have that time slice be fluid. For example, it would be nice to generate averages for our basketball season, or maybe the month of December, or for the entire semester. My goal is to create a fluid report system that will accept a start and end date each time reports are generated, similar to a QuickBooks report. This way a school that needs to could emulate the four quarters method, or three trimesters, or two semesters. The one difference is that the overall average will always be calculated by the overall points earned divided by the total points.

In fact, think about it for a minute - is it fair that four quarters are treated equally? Are they really equal? For our school some quarters are interrupted with snow days, basketball dismissals, and vacation time. Sure, they may be close, but what if teachers give less work during a sports season? If a student does 2/3 of the work during the 2nd quarter compared to the other quarters, is it fair or even logical that all quarters are treated equal?

Because the changes are pretty drastic, and because I am also doing some fundamental reorganizing of the code base itself (I'll discuss that in my next entry), I'm going to develop these changes separate from my current version. Usually I patch our working system "in flight", that is the SURRAnet that is running right now at GHCA. These changes will be big enough that I want to develop on a separate, non-critical system with the goal of swapping out the new SURRAnet for the old by August of 2009. This means that I'll be actively developing SURRAnet over the next number of months.

I'm excited about the simplicity that removing this "old wineskin" code will bring to SURRAnet, and that should allow me to do things with it that so far have only been ideas!

The Story Behind SURRAnet

2009-02-18T04:58:00.000-08:00

SURRAnet started back in 2002 as a very simple discipline database written in Python. The private school where I work, GHCA, had grown tremendously since the 90s, and we adopted a more formalized system of issuing and tracking discipline issues. SURRAnet wasn't called SURRAnet back then, it was just called "Discipline Database."

We used to keep our grades via spreadsheet, but the growth of our school, which included additional staff and classes, made this system unwieldly. We actually owned a student information system, but it was incredibly difficult to use, it didn't have features that we needed (though it had tons of useless features), and the licensing scheme was prohibitive for a private school. In my blissful ignorance I told my boss, "I can build us a grading system over the summer!" Thus SURRAnet was born.

The original SURRAnet was written in PHP because that seemed at the time like the language everyone was using to create web applications. I designed SURRAnet to be an intranet-based web application. Our school had a number of older computers at the time, and I had noticed how they could just barely run our spreadsheet software. I wanted a central server to handle the hard work, so that as long as the client computer had enough horsepower to run a browser, it could run SURRAnet. I also felt more comfortable with HTML than with GUI toolkits like GTK, QT, etc.

I learned very quickly that I had bitten off more than I could chew. SURRAnet was "finished" in time for the next school year, sort of, but I sacrificed almost all of my personal time to make it happen. During the process I realized a few things. First, PHP is, IMHO, a terrible language for doing something large scale. I know there are plenty of sites out there that pull it off, but as SURRAnet grew, my code got uglier and more difficult to manage. Second, when I thought SURRAnet would be easy to write, I based this on the notion that it would do a relatively simple task of recording grades and generating reports. I did not fully understand the depth of human stupidity (I include myself in this category). I didn't plan for people entering the wrong information into the fields, like entering a name into the date field, or a letter grade where it specifically states "points". I also didn't expect that students would be moved from class to class throughout the school year as their grades were assessed, or that we'd have a volume of transfer students arrive part-way through the year, or 100 other unforeseen variables that added an expotential layer of complexity to SURRAnet.

I also made some dumb mistakes when first designing the database engine. I didn't want to use SQL or version thereof (and I still don't), I wanted to have my own system. There are what I consider good reasons for this that I may save for another blog post. The problem with the original database engine was that it broke under certain conditions because of its dependency on delimiters, special symbols, etc. It also was a pain to expand upon.

I came to realize that SURRAnet 1.0 was not going to cut it. I really, really wanted to get away from PHP, and wasn't I delighted when I discovered mod_python for Apache! Python is my language of choice for larger projects (another topic for a future blog post). Using Python opened up a new world of possibilities in both the logic of SURRAnet and the database itself. SURRAnet 2.0 is much "cleaner" from the use of modules, custom objects, a simple to use but powerful database object based on a Python dictionary, etc. Coding SURRAnet 2.n is much more of a joy!

I also had access to new client-side features that made SURRAnet more powerful and easy to code for, especially in the area of CSS. In the past I had to hand-code different GUI-like interfaces. For example, if a name didn't fit in a table cell, I had to hand-code the shortening of the name - "John Sm..." Now CSS takes care of this. CSS also takes care of things like scrolling table bodies, various mouseover effects, and much more.

This brings up another point - SURRAnet isn't just Python, it's also a lot of Javascript. There is much that I try to take care of on the client side, especially in form validation. Javascript has really come a long way since I first started coding SURRAnet. So has HTML, which is the "glue" that I use to hold everything together.

Let me end this post on a final note, since I hope to have many future posts that look into specific details of SURRAnet. SURRAnet was originally created for GHCA. In fact, my first commitment remains to the Academy. It had a lot of our school "hard coded" into the base code itself. To be honest, while I am a big supporter of open source, I never thought anyone would be interested in my little program with all the other SIS options out there. I was wrong. Now that I see there is a demand, and also in light of the ever-changing landscape at our own school, I am working to make SURRAnet more flexible and open for anyone who wants to use it. It is not there yet. Much of this blog will follow my efforts to move SURRAnet to a 2.1 status (btw - it really never hit 2.0 status due to a number of incomplete features). Even if you are not interested in SURRAnet itself, you may find this blog interesting if you like programming, open-source software, the Python language, etc. Feel free to follow this blog and contact me with any feedback!