Lately, I have been noticing a problem with Visual Studio’s local web development server and terminal services.  I use Windows 7 Professional, which works wonderfully.  Often, I will lock the workstation and remotely connect to the workstation via RDP when I am at home.

Occasionally, I will have a problem getting to the desktop.  The symptom is that the terminal services client will negotiate a connection, go past the login screen after credentials are submitted, show the welcome screen for a second or so, and then… the screen just goes blank and does not recover.

As it happened over time, I began to notice that the problem only seemed to occur if I left the desktop in a state where the Visual Studio local web server was running.  Even though the screen goes blank, all of the service ports are running and responding.

Just a head’s up in case you encounter this.

When the righteous prosper, the city rejoices;
when the wicked perish, there are shouts of joy. – Proverbs 11:10

He said to them: “It is not for you to know the
times or dates the Father has set by his own authority. – Acts 1:7

As the news of Osama bin Laden’s death spread across the world, I found myself intrigued as I looked back over the 18+ years this man wrecked havoc with both Western and Middle Eastern powers. I was even comparing his similar history with Saddam Hussein. Saddam Hussein was a US ally during the Iran-Iraq war. The US had an enemy in Ayatollah Ruhollah Khomeini of Iran, who took the country into fundamental Islamic governance after the Shah was ousted. Similarly, Osama bin Laden had been supported by the United States during the Soviet occupation of Afghanistan, in his and the Mujahadeen’s efforts to oust the Soviet army from that country.

My how things change.

So as time progressed, we overthrew and captured Saddam Hussein. It only made sense that, over time, we would also find and catch Osama bin Laden. And May 1st, when we did, was an amazing day indeed.

But there is something in this scenario that I think we all need to stop and realize. OSB was a prolific figure in the Muslim world. Even peaceful Muslims are quietly amazed at the successes he was having against the Western powers, even if they also knew in their hearts it was only for a time. The 10-year manhunt, which followed 6 years on the FBI’s most-wanted list, was long and difficult. It even spanned three US Presidential terms.

And that is when I realized something I had not put together before: the difference between the Bush and Obama personalities, and how each was needed at the right time. George W. Bush has a reputation as a cowboy, and the anti-linguist. Barack Obama is well-spoken, with a very magnetic personality. One President, with a cowboy reputation, stood at Ground Zero and used a bull-horn to motivate the American people into believing that “the people who did this will hear from us.” It worked: it was leadership at its finest, and exactly what needed to be said at the moment.

And on May 1st, a few days after the President gives the order for the raid on bin Laden’s compound, President Obama speaks to the nation to announce that bin Laden had been found and killed. And in that well-dressed, very magnetic personality of Barack Obama comes a speech which follows that everyone should listen to more than once.  Consider this very significant part which was addressed at Muslims around the world:

I’ve made clear, just as President Bush did shortly after 9/11, that our war is not against Islam. Bin Laden was not a Muslim leader. He was a mass murderer of Muslims. Indeed, al Qaeda slaughtered scores of Muslims in many countries including our own. So his demise should be welcomed by all who believe in peace and human dignity.

Isn’t it interesting what has happened in the Middle East since bin Laden’s death has been announced by the President.  There have been a few localized marches in the Middle East, but … no major clashes.  That seems somewhat strange, for a region which went absolutely ballistic over a Florida man’s burning of the Koran a few weeks ago.  Or, maybe it is not… reread Proverbs 11:10 above.  In some way, I think Barack Obama’s connection to Muslims has dampened any truly harsh reaction.. but we can’t really measure that.

To understand God’s timing, I want you to imagine another scenario.  What would have happened on May 1st if George W. Bush were still president, or even if John McCain were president?  They don’t have a connection to Muslims like Barack Obama does.  Ever since he was elected, the Muslim world has seen Barack Obama as a favored son.  So the words I extracted from his speech, when delivered by a man they can relate to, has a very powerful and broad impact.  I am very convinced that George Bush or John McCain would have had a much more difficult time making that kind of connection, the way Barack Obama can do.

So the Lord puts in a strong cowboy as President to start the manhunt, and ends it with a President who spent a period of his youth growing up in the Muslim culture to understand and relate to it.  And both men are Christians.  I can not imagine a better scenario.

That’s why the Lord points out in his word, to submit to, and pray for the powers he appoints over man.. even if it means you didn’t vote for them.  Before May 1st, 2011, who would have known?

Donald Trump did a really fantastic thing with the Obama birth certificate debate this week, and then… what the hell !?!? It seems that the birthers have let anger get the worst of them, and it has now become an anchor sinking their cause. And they don’t even see it.. not even Donald Trump. That’s sad.

I’ve never doubted that Barack Obama was born in the United States, but I was supporting the cause of the people who were demanding that his birth certificate–the official document–be produced when the president and his staff were producing any document except the birth certificate. It’s sad that it took over two years and the efforts of a very famous billionaire, Donald Trump, to finally get that document released to the public.

But this was a fantastic thing that Donald Trump did. It had two effects:

#1: It finally caused the official document to be revealed to the public, which should have been done immediately after the question was raised during the election of 2008.

.. And I am sorry for any die hard birthers who still claim that Barack Obama is not a natural born citizen, but this official certificate added to the report of live birth, newspaper articles about the birth, etc, puts the final nail in the coffin for this question. If you continue to insist still that this solves nothing, then any label of “lunatic fringe which is out of touch with reality” is not a label that the “liberal, right-wing” press is putting on you… that you are doing to yourself.

#2: (very important) .. It pointed out that the document existed, and could have been revealed to the American public long ago. It would have stopped any debate from starting, yet a group of people chose not to reveal it.

It is another piece of evidence in a chain of evidence to the American people of just how beligerent this administration is, and how it thumbs its nose at the American people, to whom they are accountable. Add this to big bailouts which the American people opposed, and add the railroaded passage of the health care bill which the American people opposed, and it is a rock solid case of why this administration needs to go. The act of holding back of the birth certificate alone should be repeated over and over again, until it haunts the people who chose to hold it back. They earned it: they were thumbing their nose at the Constitution, not honoring it.

As of now, the stage is set to get this extremely unaccountable and damaging administration out of office in the 2010 general elections. BUT A STRONG WORD OF WARNING TO BIRTHERS STILL CLINGING TO THE IDEA THAT THIS ISN’T ENOUGH: don’t undo this momentum with stupid rhetoric that our President is not legitimate because the “birth certificate” isn’t enough for you. There’s a whole group of people ready to use it against you. Rationality will determine whether your efforts will yield fruit from this two year plus effort, or not.

The user interface is cleaned up a little, especially when resizing. It also now has a change button to select the starting folder, and an about button. The About form is fairly generic, and available in the source code.

Go to the download page for the new version.

After the involvement of US forces in the coalition attack on Libyan military targets, starting the enforcement of the No-Fly zone in Libya, I had a number of people point me to various opinion articles.  One of the most interesting is this one on the American Thinker web site: Obama Attacks Libya, and Where’s Congress?

http://www.americanthinker.com/2011/03/obama_attacks_libya_and_wheres.html

As I read the article, what I noticed was a pattern that people have seemed to develop lately: let’s compare Barack Obama to his predecessor, George W. Bush.  Whereas Obama ordered the military participation without consulting Congress, George W. Bush consulted Congress about invading Iraq.  And the comparisions go on and on…

But, let’s be fair: compare apples to apples and oranges to oranges.  The Iraq invasion came after a history with Saddam Hussein, starting back in 1979 when he seized power and was exacerbated by the Kuwait invasion.  That in turn started a coalition military operation to enforce a nearly 10-year long no fly-zone enforcement over the Iraqi land mass. There was time to work with Congress and get all to agree on the invasion: a military operation on a far greater scale than what is now happening in Libya.  It was a strategic problem.

But the situation in Libya is different.  It is a tactical problem.  Muammar Gaddafi reacted very violently to a popular uprising, using his military to put down the rebel forces gathering strength.  He also publicly promised (not threatened) to murder those who oppose his regime.  He is following a pattern of behavior he has had since he took power in Libya.  Unlike popular risings in neighboring countries, he has made no offer to hold talks with the opposition.  He has a history of barbarism, both in his own country and overseas.

And unlike Iraq, Libya is a fast moving situation where anything which needs to be done as far as intervention is fighting extreme time pressure.  The opposition forces can only hold out so long, before the powerful military machine of Gaddafi would brutally wipe them out.  Iraq and Libya is an apples to oranges comparison.

So let’s make a fair comparision to Libya: Grenada.  In November, 1983, President Ronald Reagan ordered US Troops to this Carribean island after its leader Maurice Bishop was overthrown in a military coup, and arrested. Bishop later escaped, but was recaptured and murdered.  The situation in Grenada was volatile, and any number of forces (including Cuba) were trying to manipulate the situation. While Cuba was denying any attempt to manipulate the politics, their military strength of presence and fierce opposition to US forces on the ground says otherwise.

While Reagan kept Congress notified once the invasion began, he did not consult with them on the invasion itself.  There were also a number of Congressman and Senators who publicly complained of being “out of the loop”.  In fact, one of them drew up a declaration for impeaching Reagan.  Sound similar to talk-show fodder you are hearing lately about Obama (yes, it’s Dennis Kucinich this time)?

This action in Grenada had broad support in the US.  Of course, as a result of the US action, a UN vote was taken on the matter which resulted in a vote of 108-09 to confirm a resolution condemning the action against a sovereign state.  And what was Reagan’s reaction to this: very simple.  “So?”  One thing that Reagan and his advisors were very good at, was ignoring what the world thought and doing the right thing.  This is part of his legacy as President.  It’s leadership. It’s ultimately what makes the United States give hope to the world: the defender of democracy–like it or not.

So Barack Obama exceeded his Presidential powers and ought to be impeached?  Think again.  He’s taking a stand that is right.  There is a strong US interest in blocking a known sponsor of terrorism from crushing a rebellion of the people he claims support him.  After all, the State of Libya under Muammar Gaddafi planned and executed the bombing of Pan Am 103 over Lockerbie.. killing innocent civilians.  We understand EXACTLY what his own people involved in the uprising are up against.

One ironic point of the new no-fly zone: Russia condemned the coalition’s military action against Libya, and the official Russian reaction to the military intervention has been “regret”.  It’s worth pointing out that Libya along with other countries in the region use a lot of Russian-built and Soviet-built military equipment and material in their armies.  So anytime the United States launches (or even just participates in) an attack on the countries the Russians sold military equipment to, and wipes out most of it on the first pass, that can not be good for pending sales.

So “regret” may not imply what it first looks like.

For the third time in three years, my Gateway MX laptop has a broken DC power jack connector on the motherboard.  I always fix this myself. It takes about 2 hours each time (after waiting for the new part to arrive in the mail from eBay) and, surprisingly, the repair time is quite relaxing for me.

Complaints about this DC jack are found throughout the internet with a simple Google search, and it’s not just Gateway laptops that have the problem. The last time I replaced the power jack, I noticed the problem causing my failures. Because of the way these jacks are designed and positioned so close to the edge of the case, the jack has nothing to protect it from the energy of the power cable being jerked. And while the outer shell of the power jack is anchored by 6 large solder points, the back connector only has a single, thin metal connector that can not flex. So the force of the power cord being pulled ripples through the inner connector, and snaps its connection to the back connector.

This time, I wondered what would happen if I just removed the stress point.  And the logical way to do that was replace the single, thin metal connector in the back with something that would flex. My answer was to replace metal connector with a flexible copper wire.  So on the power jack, I just used wire cutters to clip off the metal connector just behind the plastic, leaving only enough for a solder connection to the wire.  The other end of the wire is simply soldered into the motherboard.  There’s only a very slight slack in the wire, but that’s all it needs.

Figures A and B (below) show the change.  You need to be good with your hands in small spaces to make this work, and you might want to purchase two of the jacks in case you clip one too close, etc. 

So far, the new jack is working fine. Of course, the real test will be if my son’s Golden Lab puppy again goes into “play with me now” mode, and blindly runs into my laptop’s power cord like a Navy fighter aircraft’s tail hook slams into the arresting cable on the carrier’s deck.  But short of a catastrophic stress event like that, I’m confident that this modification will survive the usual stress that makes these unmodified jacks so vulnerable.

I do much less with hardware nowadays, than when I was younger.  I no longer build entire custom systems from scratch. Still, the occasional simple maintenance task, like changing out a hard drive, is something that I enjoy doing.  And in fact, have done for years. But today, I was fearing that the worst thing that can happen had come to pass: killing something on your system by mis-installing a cable, or zapping the box.

After replacing the dying hard drive on my HP Pavilion p6230y, I turned on the power and got … nothing.  No screen display, no keyboard lights resetting, nothing.

Now I have had this happen before, but it is usually a misseated power cable.  So I powered down the computer and checked the power and data cable seating. This time when I powered up, I saw the hard drive light come on about 1 second after the power-on light, I heard the hard drive heads move around a bit, and then the hard drive light went out.  Still, no keyboard reset, no light on the optical mouse, a blank screen.

This worried me a little.  It’s easy to re-seat a cable and get the hard drive to work.  That behavior can happen even when the motherboard is not starting properly: its a normal startup sequence for a hard drive being powered up.

So now, what’s different?  I don’t see anything.  The keyboard lights not resetting and the mouse light not coming on worry me.  This is usually a bad sign.  Still, just one more check before I decide to involve others, who I worry, may confirm my worst fear.

And then, I see something a little different.  I had the DVI cable for the monitor plugged into the video card (not the main onboard video card).  That shouldn’t make a difference.  I should have at least seen the keyboard lights reset and the mouse light come on if the video was not working.

But I give in… I change the cable connection to the onboard video.  I’ve got nothing to lose at this point.  So I power on the computer, and …

The keyboard lights all flash on and off, the mouse’s optical light comes on, and I’ve got a BIOS display on the screen.  A great relief indeed, but … strange. Having the wrong video port connected was actually causing the BIOS to not start the boot sequence.  That is a first.  Every other desktop I have worked with would always reset the keyboard and start up, regardless of whether the display was connected.  All the BIOS cared about was that a video adapter was installed and working, not whether the operator was bothering to look at.

So if you are changing out the hardware in your desktop, remember this story.  The BIOS is more picky about connected hardware than it used to be.  At least, on this particular HP Pavilion model.

I was using Google Maps to derive some lat/longs to show my son some examples of the results for a distance formula, and I noticed something odd. If you enter the address 1600 Pennsylvania Avenue, Washington DC (the White House), it will display the location on the map ( link to results ).

If you use that address as either an origin or a destination to Get Directions, Google’s response is that it doesn’t know how to calculate the directions ( link to results ).

Change 1600 to 1200, and you’ll get an answer (link to results ).

This smells like Homeland Security-ware to me.  If it is, it is one of those clear cases of an action based on fear, rather than rational thought. Whoever intended to protect the White House (and possibly other government locations) really didn’t account for very many user cases in this action. And this is the most well-known building in the world. What are they thinking?

I just hope Google isn’t caving into different kind of pressure from a Government to censor results. To Google’s credit, the previous censorship that was being applied on their site in China is now gone. Two years ago, if I had searched for “Tienanmen Square” on Google’s site in China, I would have seen nothing other than smiling tourists, and bright colored photos that looked like they were retouched with Photoshop for a travel brochure. The results would never have shown any no indication of the 1989 protests and military intervention which followed.  On the US site, the 1989 Tiananmen Square events are ranked at the top.

You can view the current results from the US site here, and the results from their site in China here.  Not much difference between the two.  The Chinese results now come from a domain ending in cn.hk (Hong Kong). Previously it was coming from www.google.cn when censorship was applied.  Whatever is implied by that change, it works… Google got away from the heavy hand of censorship.  Kudos!

I hope the same common sense prevails with Google Maps as well.  It is ironic that the same type of information suppression occurs when you enter Pyong Yang (North Korea).  Yes, this can be a slippery slope.

(Follow-up): Upon further investigation, it became clear that Google’s inability to give driving directions occurs because the actual street address is not accessible by vehicle: it is physically blocked off.  There is similar behavior where the starting point is one continent and ends in another.  The new versions of the software take in to account that there is no physical roadway access to the destination point, and therefore just report that driving directions are impossible to calculate.

Still, the empty map of North Korea is a bit scary.

I am amazed at the amount of SQL and other code I encounter, which employs poor methodologies of establishing start and end points in time for a filter. I want to show some examples, and share some simple guidelines which can help you avoid the pitfalls of miscalculating time, or introducing gaps or overlaps in coverage.

I recently ran into this code example in a stored procedure:

SET @dMonthStartDate = '2010-12-01'
 
SET @dStartDate = DATEADD(MONTH, -1, @dMonthStartDate)
SET @dEndDate = DATEADD(SECOND, -1, DATEADD(MONTH, 1, @dMonthStartDate))

It’s obvious that the goal of the code is to set the memory variables @dMonthStartDate to Nov 1st, and @dMonthEndDate to Nov 30th, to filter the data for only the specified month. Later, when the code moves into production, the intent is to change SET @dMonthStartDate = ‘2010-12-01’ to SET @dMonthStartDate = GETDATE() which will dynamically adapt the memory variables to mean “start/end of the previous month”.

SET @dMonthStartDate = GETDATE()   /* The intended production change */
 
SET @dStartDate = DATEADD(MONTH, -1, @dMonthStartDate)
SET @dEndDate = DATEADD(SECOND, -1, DATEADD(MONTH, 1, @dMonthStartDate))

There are three (yes, three!) problems with the logic that lead to some big problems.

#1: It makes an assumption that the code will always run on the first of the month. The goal is to return a subset of data, within the timeframe of last month. But since the test value for @dMonthStartDate was set to the 1st of the month, which is the expected execution time, the two expressions underneath carried that assumption forward into the code.

The code does not allow for the occurrence of a case where, the code fails to execute or needs to be rerun a second time, which occurs after the 1st of the month. Without that, you end up with a time frame of 11/04/2010 to 12/03/2010 when the code runs on December 4th.

The date can be forced to the first, by using DATEADD() like this:

SET @dNow = GETDATE()
 
SET @dMonthStartDate = DATEADD(DD, -(DAY(@dNow) - 1), @dNow)

Note: I always use @dNow to hold a snapshot of the current date/time, when more than one call to GETDATE() is made, and the same value is assumed for every call. It isolates the code from weird, very maniacal answers where the current time from GETDATE() rolls over to a new minute, hour, day, week, etc, when going from one call to the next.

#2: It makes an assumption that the code will always run exactly at midnight. GETDATE() is poorly named: it returns both the date and the time. The test value of ‘2010-12-01’ is actually returning ‘2010-12-01 00:00:00.000’, or Dec 1st at midnight. Assume this code is run at Dec 1st, 2010 at 3:59:23.231 AM. GETDATE(), without rectifying the time portion to midnight, would return ‘2010-12-01 03:59:23.231’, making the start and end dates ‘2010-11-01 03:59:23.231’ and ‘2010-12-01 03:59:22.231’ .

The time portion can be forced to midnight of the 1st of the month using…

SET @dNow = GETDATE()
 
SET @dMonthStartDate = CONVERT(
         VARCHAR, DATEADD(DD, -(DAY(@dNow) - 1), @dNow), 101
)

.. making it the 1st of the month at midnight.

#3: Not all of the time within the month is covered. The clue is the nested DATEADD() methods (add 1 month, subtract 1 second). By doing this, @dMonthEndDate is actually set to 2010-11-30 23:59:59.000 (using the constant date in the first example). Because of this, any entries made in the last second of time in the month are excluded using the filter (e.g. 2010-11-30 23:59:59.231 would be excluded).

Is it a small amount of data? Probably. Will someone notice? Probably not, but it’s always a bad assumption. Will it become noticeable if the system scales to process about 100 actions a second or more, and introduce doubt about your core method? My experience says yes.. definitely.

These types of errors usually manifest when the process is run over several days, and generates different answers. It may mistakenly introduce doubt into the algorithms for the logic using the filtered data, instead of the filter itself. This is a recipe for heartburn when troubleshooting.

One other flaw I see from time to time is this type of evaluation:

SET @dMonthStartDate = '2010-12-01'
SET @dMonthEndDate = '2011-01-01'
 
SELECT ....
WHERE ActionDate >= @dMonthStartDate
AND ActionDate <= @dMonthEndDate

In this scenario, data captured exactly at midnight is included in both this month’s summary and in the next month’s summary. It’s more maniacal, but nonetheless, will show up in high-volume systems.

Having said that, here are my guidelines for handling date ranges.

Rule #1: Always have the end time actually be the “cutoff” time.

Cutoff sounds non-inclusive, whereas End is a bit nebulous.  Think of the time range in your if clause as “From the start time and before the cutoff time”, and write your code that way.

Rule #2: Set the time portion of both the starting and “cutoff” dates to the same value, and evaluate the time range as greater than or equal to start, and less than “cutoff”

Take the following example:

/* drops time from the system time to force time
to midnight of the current day */
SET @dToday = CONVERT(VARCHAR, GETDATE(), 101)
 
/* Change to XXXX-XX-01, or the first of the month for this date. */
SET @dMonthlyReport<strong>CutoffDate</strong> = DATEADD(DAY, -(DAY(@dToday) -1), @dToday)
 
 /* Using the first of the month you just calculated, find the 1st of the previous month at midnight */
SET @dMonthlyReport<strong>StartDate</strong> = DATEADD(MONTH, -1, @dMonthlyReportCutoffDate)
 
SELECT ....
WHERE ActionDate >= @dMonthReportStartDate
AND ActionDate < @dMonthlyReportCutoffDate

Note: This code can be refactored to calculate start time before it calculates end time. This is just my preference.

Notice I use the term “cutoff” date. It’s a personal preference.

For a month filter (as in this example), the time range would be “XXXX-XX-XX 00:00:00.0000″ to XXXX-(XX+1)-XX 00:00:00.0000”. For a week filter, the time range should be “XXXX-XX-XX 00:00:00.0000″ to XXXX-XX-(XX+7) 00:00:00.0000”. For a day filter, the time range should be “XXXX-XX-XX 00:00:00.0000″ to XXXX-XX-(XX+1) 00:00:00.0000”.

For an hour filter, the time range should be “XXXX-XX-XX 00:00:00.0000″ to XXXX-XX-XX 01:00:00.0000”.

… NOT “XXXX-XX-XX 00:00:00.0000” to “XXXX-XX-XX 23:59:59.999999999”
… or “XXXX-XX-XX 01:00:00.0000” to “XXXX-XX-XX 01:59:59.999999999”
, etc, etc.

There are two reasons for this rule. The first reason is simplicity. If I am able to calculate the 1st of the month at midnight, I can easily calculate the first of the next month at midnight by simply adding one month using DATEADD(month, 1, {date}), or the first of the previous month at midnight by simply subtracting one month using DATEADD(month, -1, {date}). The technique stays the same, regardless of the time frame being a month, a day, a week, an hour, a quarter, a year, etc. I just change the “what” of the offset in the DATEADD() method, e.g. DATEADD(hour, 1, {date})

The second reason is accuracy and readability. If I offset the ending time of the day to “23:59:59.9999”, and evaluate it using “less than or equal to”, I would have accurate code on current Windows or Linux systems. If future systems are released (or others systems exist) with an accuracy of 1/10,000th of a second or more, I have to revisit this code and extend it to 5 digits past the decimal point to be complete and accurate.

By using a filter that says “timestamp occurs before cutoff” (i.e. less than cutoff), a system’s accuracy is not going to affect my evaluation. Also, which statement is more easily understandable to you, if you had to maintain it:

SELECT ....
WHERE ActionDate >= '1/1/2010'
AND ActionDate <= '1/31/2010 23:59:59.9999'
 
SELECT ....
WHERE ActionDate >= '1/1/2010'
AND ActionDate < '2/1/2010'

I identify the second code example quickly as being based on a one-month time frame.  It clearly shows the first day of back to back months, and I don’t have to worry about why the seconds have a decimal point.

Rule #3: Never assume your code is running at its scheduled launch time.

Data is often summarized at a regular interval, but sometimes needs to be launched “out-of-cycle” if system problems occur or some source data straggles into the system after its expected arrival. The out-of-cycle execution can easily be a few days after its normal scheduled start of the 1st of the month.

How do you deal with this in code? Simple. Always ensure that your calculated start and end dates do not vary for any system time until the next “reporting” or filtering time range. A weekly time range generated on any given day of the month for a filtering period should always give the same answer. A monthly time range generated on any day of a given month should always give the same answer.  So if I set my test date to 6/8/2010 @ 5:00PM or 6/22/2010, and I was calculating for the previous month, any date within June of 2010 should resolve to a start date of 6/1/2010 00:00:00 and an end date of 7/1/2010 00:00:00.

What do I hope you get out of this? I urge you to take the time to draw out scenarios of time handling to make sure all cases are covered, and understand that not adequately calculating time filters properly (or making assumptions of what the system date/time will be) can lead to serious user distrust in your code over time. This is especially true in system processes. These three guidelines are intended to help avoid those problems.

In fact, treat date/time handling as you would the foundation of a house. If your foundation is off by even a few inches, or is missing small parts of it here and there, it can spell disaster for the whole house as things are added like flooring, plumbing, electrical, walls, and furniture. The last thing someone wants to find as a homeowner in a finished and occupied house, is that low water pressure is due to improper piping under the concrete slab holding the house up. That requires tearing up a lot of the house to get it fixed, and the whole house is affected. After fixing the weak water-pipe, if the water pressure jumped 20 pps and then caused pipes to leak in a bunch of other places in the house, those leaks now have to be fixed. It gets ugly. Even worse… the integrity of the whole house comes into question.

That’s the effect bad date handling can potentially have as your data trickles downstream to the consumers of your results. So take care when processing time stamped data.

Lately I have been frustrated in general with forum-based sites, regardless of subject matter. And yet, I don’t blame them for my frustration. The problem is systemic in nature.

Forum-based web sites function as an archive of data over long stretches of time, like a traditional library, but are much more dynamically built. Its content as a whole has differing value to various people over the course of time. The forums contain a lot of information which, like all other information, needs to be evaluated against a number of factors to properly determine the value to the consumer. Just like any research, these factors include author experience and motivation, source of original information, logic and empirical evidence, verification, etc, etc.

One of these critical factors is the time of origination (or posting). Forum software puts a time stamp on any entry made in it. That time stamp tells you a lot of things. When researching the Iraq War, an entry made in 2003 is likely going to contain information from a more-speculative perspective, than one made 7 years after the war’s start. As a developer, seeing an entry about C# dated in 2003 tells me I am not looking at a technique which will contain functionality designed into version 4.0 of the language.

And lately, I go to the header or footer line of a forum entry to see when the author posted it, only to discover that it is … BLANK !!

To understand what is happening, you need to remember that most forum sites are supported by revenue from visitors clicking on ads displayed on the site’s pages. And the higher the site ranks in a Google (or another engines’) search, the more visitors the site will get, thus maximizing their ad revenue potential.

The forum sites have a commercial need to get the best rank possible. But Google and other search engines factor the freshness of a site’s content into its rank. Generally, the fresher the content, the more points it gets for its final rank determination.

How does a search engine determine how “fresh” or “new” a web site’s content is? Obviously you need some kind of time stamp derived directly or indirectly. The various bots and crawlers which sweep the net looking for this information have at least two options, which are simple to implement. At the protocol level, a GET or HEAD command to a web site will return the date for a page’s content in the response header. Unfortunately, it is always going to make that time stamp the current time, if the page content is built dynamically by a script or other executable. That date in the header is only reliable for static pages (*.html, *.txt, *.pdf, etc).

Content crawlers have to ignore what the server tells them about dynamic pages, and use option 2: dive into the content itself to determine the time the information was created.  Not so ironically, one of the simplest ways to achieve that is to harvest an existing time stamp being rendered right on the page. Forum sites are figuring this out. Since they archive discussions which are not new, timestamps putting their content into anything other than the very near past (24-48 hours, maybe a week) risks lowering their ranking in the results.

So one of the simplest ways to fight back is by denying the bots time information within the page content. If you have a time stamp associated with the content, display only the content to the user. This is done at the penalty of the consumers of the information, who need that time stamp to effectively evaluate the information as a whole.

This scenario is, to me, one of the best examples of the classic public interest versus commercial interest conflict we are seeing on the NET today. In the United States, we use a library system that was founded on principles designed by Ben Franklin. The libraries are supported by common public funds allocated for their operation. The libraries don’t need to compete with other libraries to get funds and, hence, don’t have to be reactive to how they catalog information based on a third party’s opinion and promotion of their libraries’ content. Web site forums are mostly privately financed, and don’t have this luxury.

Google’s advanced search options do have overrides to specify what data to include in the results, which includes a time span. But the web site operators are listening to optimization experts, who are telling them to eliminate the time stamp which can work against their ranking. And it makes sense, since most people aren’t going to override the default or even know how that default behaves. Google’s advanced search options use a default of “anytime” for data content. Still, what we don’t know is whether the algorithm generating the rank is using that setting to dynamically alter the rank. I also do not know what Bing or other search sites have implemented in their algorithms that may address this issue.

The bottom line… until Google and others make their methodology for handling time vividly clear to web site operators, the operators will continue to believe that time stamps work against their commercial interest, and a valuable piece of information for evaluating data will be suppressed to the users.