Cache.Add absolute expiration – UTC based or not?

The examples for Cache.Add uses DateTime.Now.Add to compute the expiration, i.e. it passes:

 DateTime.Now.AddSeconds(60)

as the value of the absoluteExpiration parameter.

I’d have thought that computing it relative to DateTime.UtcNow would be more correct [as there is no ambiguity if Daylight Savings Time starts in the intervening time between now and the expiration point].

Before the introduction of DateTimeKind, I’d have guessed that there’s some ugly hacks in the cache management to make it do something appropriate if the time was not a UTC time.

In .NET 2.0 and later, I’m guessing that it should handle a DateTime calculated as DateTime.UtcNow.AddSeconds(60) correctly given that it has DateTime.Kind to use as an input in its inferences.

I’ve been confidently using DateTime.UtcNow as the base for years, but wasnt able to come up with a rationale that this is definitely the correct thing to do in the absence of anything pointing out the documentation has been highly misleading for 4+ years.

The questions?

  1. Despite much bingage and googling I wasnt able to find any authoritative discussion on this from MS – can anyone locate something regarding this?
  2. Is there any reason why using UtcNow wouldnt be more correct and/or safe?

(Yes, I could peruse the source and/or the Reflector’d source, but am looking for a full blow-by-blow lowdown!)

Combine Sliding and Absolute Expiration

I want to use System.Runtime.Caching.MemoryCache for caching some of my objects. I want to be sure that the object is refreshed once a day (absolute expiration) but I also want to make it expire if it

update cache item with absolute expiration

Is there a way to update an item in the memory cache and maintain its absolute expiration policy? I am writing a simple throttling class that checks if a request should be throttled. For example, thro

ASP.net Cache Absolute Expiration not working

I am storing a single integer value in HttpContext.Cache with an absolute expiration time of 5 minutes from now. However, after waiting 6 minutes (or longer), the integer value is still in the Cache (

Django key based session expiration

I have a website which contains several rule sets for different kind of users. One of the rules (permission) depends on the session expiration. For instance, an unauthenticated users’ session must be

time-based expiration of cache in Rails?

Rails offers several layers of caching: page, action, and fragment. I was surprised that none of these methods offer a time-based expiration of the cache. I for example am showing a list of top selli

Key based cache expiration with Mongoid embedded documents

How would you set up a Russian doll like key based cache expiration with embedded documents? As described by 37 signals I believe touch was added for belongs_to in Mongoid 3.0 but how would you deal w

Siding cookie expiration

Is it possible for cookies to have a sliding expiration or do you have to reset the cookie with a new absolute expiration date on every request?

UTC clock based on server time?

I have a JavaScript clock that gets the time in UTC, it currently works, but the time is based off the client’s computer. What could I do to base the time off of the server instead? I am using PHP as

Cache expiration

How can I control the cache expiration in asp.net? I use absolute expiration and the cache expires every 3 hours. NOw I want to add a condition, so that only if the condition is true, the cache should

Ruby – Redis based mutex with expiration implementation

I’m trying to implement a memory based, multi process shared mutex, which supports timeout, using Redis. I need the mutex to be non-blocking, meaning that I just need to be able to know if I was able

Answers

Cache.Add converts the expiration date to UTC before storing it.

From Reflector (I’ve omitted most of the parameters to make it easier to read):

public object Add(... DateTime absoluteExpiration ...)
{
    DateTime utcAbsoluteExpiration = DateTimeUtil.ConvertToUniversalTime(absoluteExpiration);
    return this._cacheInternal.DoInsert(... utcAbsoluteExpiration ...);
}

In CacheExpires.FlushExpiredItems, utcAbsoluteExpiration is compared to DateTime.UtcNow. As Joe notes in his answer, this causes unexpected behavior when a cache item’s addition and expiration span the end of daylight saving time.

[Highly derivative of insights from Jeff Sternal’s answer, which I’ve +1’d in incomplete payment :D]

It seems that in 1.1 (didnt look at 1.0 but I assume its similar), in the absence of a DateTime.Kind and having published examples with a DateTime.Now-relative time, they feel comfortable immediately calling ToUniversalTime() immediately.

Hence…

  1. in 1.x, you’ll end up with a mess if you [the API user] use DateTime.UtcNow (and there’s a sensitivity to DST starting during the call to Cache.Add)

  2. in 2.0 [and 3.x], you’re safe to use either as long as the Kind on the time you supply is set [which it normally would be if you got the time from DateTime.Now or UtcNow]. [See Joe’s comments and answer for full rationale] Definitely prefer UtcNow as ambiguity for 1 hour of DST switchover.

  3. The example stays as-is so people working against 1.x dont get misled [and people can go on pretending that DST is a wacky edge case :P]. [Ditto, as pointed out by Joe] But this is a very debatable stance as this can result in stuff staying in the cache for an extra hour.

I’m still very interested to hear more details, including any from any Ponies out there.

EDIT: I realise from Joe’s comments that I didn’t explicitly call out the fact that it is definitely more correct to use UtcNow if using 2.0 or later as one is exposed to risk of the item being cached for an extra hour during the DST ‘groundhog hour’. I also think the MS doc should point this fact out (with the proviso that they need to mention that this does not to apply to 1.1 [regardless of whether the page is marked 2.0+ specific]. Thanks Joe.

EDIT: Noda Time will have a neat wrapper to make this foolproof šŸ˜€

I reported this bug on Microsoft Connect some time ago, but it’s been closed as won’t fix.

You still have a problem in .NET 2.0 if you specify your absolute expiration in local time.

During one hour at the end of daylight savings time, your local time is ambiguous, so you can get unexpected results, i.e. the absolute expiration can be one hour longer than expected.

In Europe, daylight savings time ended at 02:00 on 25 October 2009. The sample below illustrates that if you placed an item in the cache at 01:59 with an expiration of 2 minutes, it would remain in the cache for one hour and two minutes.

DateTime startTime = new DateTime(2009, 10, 25, 1, 59,0);
DateTime endTime = startTime.AddMinutes(2);
// end time is two minutes after start time

DateTime startUtcTime = startTime.ToUniversalTime();
DateTime endUtcTime = endTime.ToUniversalTime();
// end UTC time is one hour and two minutes after start UTC time

Console.WriteLine("Start UTC time = " + startUtcTime.ToString());
Console.WriteLine("End UTC time = " + endUtcTime.ToString());

The workaround for .NET 2.0 or later is to specify the absolute expiration time in UTC as pointed out by Ruben.

Microsoft should perhaps be recommending to use UTC in the examples for absolute expiration, but I guess there is the potential for confusion since this recommendation is only valid for .NET 2.0 and later.

EDIT

From the comments:

But the exposure only occurs if the conversion happens during the overlap. The single conversion actually taking place is when you lodge the item with Cache.Add

The problem will only happen if you insert an item in the cache with an AbsoluteExpiration time in local time during that one ambiguous hour at the end of daylight savings time.

So for example, if your local time zone is Central European (GMT+1 in winter, GMT+2 in summer), and you execute the following code at 01:59:00 on 25 October 2009:

DateTime absoluteExpiration = DateTime.Now.AddMinutes(2);
Cache.Add(... absoluteExpiration ...)

then the item will remain in the cache for one hour and two minutes, rather than the two minutes you would normally expect. This can be a problem for some highly time-critical applications (e.g. stock ticker, airline departures board).

What’s happening here is (assuming European time, but the principle is the same for any time zone):

  • DateTime.Now = 2009-10-25 01:59:00 local. local=GMT+2, so UTC = 2009-10-24 23:59:00

  • .AddMinutes(2) = 2009-10-25 02:01:00 local. local = GMT+1, so UTC = 2009-11-25 01:01:00

  • Cache.Add internally converts the expiration time to UTC (2009-11-25 01:01:00) so the expiration is one hour and two minutes ahead of the current UTC time (23:59:00).

If you use DateTime.UtcNow in place of DateTime.Now, the cache expiration will be two minutes (.NET 2.0 or later):

DateTime absoluteExpiration = DateTime.UtcNow.AddMinutes(2);
Cache.Add(... absoluteExpiration ...)

From the comments:

Or am I missing something?

No you’re not. Your analysis is spot on and if your application is time-critical and runs during that period at the end of DST, you’re right to be using DateTime.UtcNow.

The statement in Ruben’s answer that:

you’re safe to use either as long as the Kind on the time you supply is set

is incorrect.