kivikakk.ee

Illness (1/n)

[Content note: mental health, weight loss, near-death experiences, anxiety, panic attacks, mood disorders, self-harm, boundary violation]

Twelve weeks ago, on the 29th of May, I became extremely ill. I’m still sick now, though I’m much better than I was even one week ago, and one week ago much better than I was a week before that.

I’ve never been this ill in my life, nor have I had an illness that’s lasted so long or been as chronic. The nature of the illness — namely, mental illness — is such that it is heavily stigmatised and often misunderstood. I’m extremely lucky in that I have a job, family and friends who understand much better than the average. With a lot of help, I’ve been able to recover.

I’ve recovered enough that I’m able to write about my experience. I write because this could happen to anyone; could happen to your friend, your family member, your loved one, or you. It struck me suddenly and without warning, and it has easily been the worst experience of my life.

I have spent unbroken lengths of days believing honestly that death was imminent. Despite being a healthy weight before, I lost more than ten kilograms in a week. I missed weeks of work, had countless doctor’s appointments, and have been on three new medications since it began.

I write because maybe helping more people understand the nature of mental illness will help counter the stigma surrounding it; to help convey that that mentally ill people do not want to be ill; that we do not lie about how we feel; that we do not do this for attention; that more than anything we wish for a normal life again.

The first month was living hell. Despite this, whenever I found a few minutes of normality, I’d write a few short lines on what had happened since I last felt lucid, in case I’d ever have a chance to look back on them. Now I do.

Because there’s too much to write at once, this will be a series of posts, covering a little bit at a time.

Saturday, 28th of May (day 0)

I’d had a good couple of days. The previous night I went out and had a nice time. During the day I went to a shopping centre with my kids; my mother and I took them to a play centre and they spent a good hour or two exhausting themselves while we chatted, had hot chocolates, and helped them find each other when they managed to lose themselves in the huge playground.

In the evening, I had a conversation with my partner — at that time overseas — that didn’t go so well. It wasn’t an argument; just, I was struggling with coping with some things and we couldn’t get to a good place in the conversation. Eventually it was time for me to go to bed, so we said goodnight and hung up.

I was feeling frustrated with myself: for being unable to cope with my feelings. I’d only recently realised that the diagnostic criteria for borderline personality disorder described me extremely well, and this realisation had necessitated a lot of self-assessment and viewing my past through a new lens.

This conversation triggered more of the same feelings: inadequacy in being able to cope with emotional triggers, in being able to control my own life, in being able to be a good person.

Sunday, 29th of May (day 1)

Lacking healthy coping mechanisms, after the phone call ended, almost exactly at midnight, I freaked out. Alone in the bedroom, I cried and screamed into my pillow, thrashed about and self-harmed, until I exhausted myself.

It was maybe only five minutes, but this was the trigger. As I lay in bed afterward, my heart wouldn’t stop racing. The minutes passed, but it only got worse. Soon I found it difficult to breathe, and my chest was beginning to ache from my heart beating several times a second.

Figuring this would go away after I somehow fell asleep, I continued to lie there. I occupied myself with my phone, put music on; tried to do any number of things to distract myself from the physical sensations. Eventually it was 5am, and I was feeling worse than ever. Bouts of lightheadedness would wash over me, and breathing felt closer to suffocating.

Googling symptoms indicated I was having a panic attack — albeit a five-hour long one — and that the lightheadedness was due to hyperventilation. Carbon dioxide levels in the blood drop, causing blood pH levels to rise. This produces dizziness, tingling in extremities, can cause fainting and even seizures.

Finding this out was a huge relief. I performed breathing techniques to counteract the hyperventilation and bring my CO2 levels back to normal. It didn’t work. Even when controlling my breathing for ten minutes, my heartrate would not lower, and I continued to experience worsening dizzy spells.

The situation now starting to seem serious, I drove myself to the ER. It took a few hours to be seen, and while in the nearly empty waiting room, I repeatedly had to rebuff the advances of a well-meaning but boundary-oblivious man, whose incessant touching and edging closer were enough to provoke anxiety all in themselves. A lone woman is fair game even in an emergency waiting room.

At 8am or so, I was seen. My lungs were clear, my heart normal. My heartrate was still elevated, but not as much as before. The doctor on call was confident I was physically fine and not at any risk, and that the symptoms were entirely anxiety-based. This calms me down a fair bit: I’m not going to die, even if I felt like that for a while. He suggests breathing techniques and to see my doctor if I’m concerned.

I head home, and start the day somewhat bleary eyed. It’s a Sunday, so I make myself tea and play League of Legends. Eventually lunch time arrives, and I walk down to my favourite little restaurant for a small lunch. While sitting and waiting for food, I notice my heartbeat again; this small feeling of being on edge, of panic. I do my best to ignore it — I know it doesn’t mean anything.

An hour later, I arrive back at home, and the symptoms of a panic attack are back in full swing. Breathing techniques do nothing to moderate it.

Suddenly, I remember that I needed to fill a prescription. I ran out of HRT the day previous, and it completely slipped my mind with the trip to the ER. Missing a day of hormones will just make matters worse, so despite feeling atrocious, I force myself into the car. I’m not even halfway to the pharmacy when things get so bad that I have to force myself to return. I couldn’t concentrate at all.

I don’t know how to explain the feeling of panic properly. Every part of your body is preparing to escape from some unknown predator. After this goes on for more than a few minutes, it becomes completely exhausting. You can do nothing but focus entirely on it, on the sensation that if you do not act immediately, terrible consequences will follow.

But I didn’t know what to do. I simply froze, wishing it would just disappear. With no cause sustaining it, it didn’t take very long to start feeling like death was certain. There was nothing to fear — nothing I was worried or concerned about except the panic — so the sensation of impending doom had nothing to be tied to but being alive itself.

I tried to tell myself that sleep would cause it to disappear, but every part of my body was wired with adrenaline. I still needed HRT. I hoped like hell my mother was available and called her.

Thankfully, she was. She picked up the script for me, and I tried to work out if I needed to go to the ER again. We resolved that instead I’d stay at her house for the night. I ended up staying there for the next three weeks; it was the first time I’d lived with family in eight years, the first time since I moved out.

In the car on the way to my mother’s, the closed environment and sense of impending doom led me further into despair. I couldn’t control my own anguish, crying and writhing in the passenger seat. Eventually, I calm myself down, telling myself that it’ll pass, somehow.

At my mother’s, I nearly immediately settle in the spare bedroom, and with Twitch on my laptop, finally found myself getting sleepy.

Original Estonian lyrics at the bottom. You can listen on YouTube.

to my friends

Come friends, and sit down,
I’ve wine and all the time in the world.
All I want now is to be with you,
laughing together.

When you find the time to show up
there’s no need to speak first.
We can just be there for each other
and at night we can watch the moon.

‘cause now I know that every day
I can give you a hand.
‘cause you’re here, you’re here, and I am too.
And I am too.

I’ve always had this habit;
longing, when I can’t see you.
And if not today, then tomorrow
I’ll be asking after you, determined.

I’m here for you, when you need me;
And when not, I’m here still.
I’ve two hands for embracing you,
And a song to sound ‘til the end.

‘cause now I know that every day
I can give you a hand.
‘cause you’re here, you’re here, and I am too.

And now I know that every day
we belong together.
You have my hands, my hands, and I have yours.
And I have yours.

Sõpradele

Sõbrad tulge ja istume maha,
mul on vein ja ilmatum aeg
sellel hetkel just teiega tahan
olla koos ja teha üks naer

Leidkem aega, et kohale tulla
Pole tarvis teil pruukida suud
võime lihtsalt üksteise jaoks olla
Kui on öö, näiteks vaatame kuud

Sest nüüd ma tean, et iga päev
võin käe teil anda ma
Sest te olete siin, olete siin ja mina olen ka
Ja mina olen ka

Mul on alati olnud üks komme:
igatseda, kui ma teid ei näe
Ja kui mitte täna, siis homme
küsin kindlalt, et kuidas teil läeb

Olen teie jaoks siis, kui on vaja
Ja kui pole, olen ikkagi
Mul on kaks kätt, olen kallistaja
Ja mu laul kõlab lõpuni

Sest nüüd ma tean, et iga päev
võin käe teil anda ma
Sest te olete siin, olete siin ja mina teie ka

Ja nüüd ma tean, et iga päev
võime kokku kuuluda
Teil on mulle käsi, mulle käsi ja minul teile ka
Ja minul teile ka

It’s pretty difficult to find this song anywhere, so I’ve uploaded an MP3 (16MB). Likewise, the original Japanese lyrics don’t appear to be online, so they’re included at the bottom of this post.

blue

Tell me why I was born (I need to know)
Words never fail to hurt me

All I want is to be important to someone
for something only I can do

I’m afraid, so afraid of being hurt; I can’t run from here
But I can’t just stop being me

I want to fly, fly, fly, fly
Soon, now, you’ll see

Is it fine, if I’m here? Someone, tell me.
I’m always searching for myself

I believe I can become something, surely,
but the unease is hard to shake

I’m so afraid of being hurt, and I can’t open my wings
But I just can’t stop being me

I want to fly, fly, fly, fly
Soon, now, I’ll take flight

All I wanted was to spend this time with you, peacefully
Why then does time refuse to wait for us?

Ao-青-

生まれたことの意味を教えてほしいよ(ほしいよ)
言葉がいつでも私を傷つける

私は私にしかできないことの何かで
誰かに私を必要としてほしいよ

傷つくのが怖くてここから動けんないけど
私は私であることを止める訳にはいかないよ

飛びたい 飛びたい 飛びたい 飛びたい
もうすぐさ わかるよ

ここにいてもいいの?誰かを教えてよ
私はいつでも自分を捜してる

私は何にだってなれるはずさきっと
信じてるけど不安で仕方がないよ

傷つくのが怖くてここから飛び出せないけど
私は私であることを止める訳にはいかないよ

飛びたい 飛びたい 飛びたい 飛びたい
もうすぐさ 飛べるよ

私はただゆっくりとした時間を君と過ごしていたいだけなのに
どうして時間は私たちを待ってくれないんだろう

You can listen on ATLANTIS AIRPORT’s official YouTube. They’ve also put the original Japanese lyrics (Wayback) up on their blog

it happened during the movie

yes no, yes no, yes no
a cat’s eye is the signal

yes no, yes no, yes no
I’m not witty enough

mix the whites together
with your finger

turning, smoothly, around

spell it out for me, what they speak
because I want to read it
like a novel

give up / your eyes were diverted
do you hate me? hate this, that, everyone? right;
you think it’s fine if
everyone gets fucked up eventually
don’t you?

blue
stamp it “par avion”

the roar of the waves at night
you and me
running, laughing

not with you, not without you

the cheese of a pessimist

bite me
hurt me
if the memories flow

play the fool; a sad clown
tears all a lie
why?

i miss you

i want to meet soon
but my eyes are getting in the way

good night

cry
somehow
take a breath

i’m sorry

i will grant you anything
i will caress you softly
call my name, and
close your eyes
just close your eyes

like this, let’s go far far away
and paint the sky red
so please, don’t
tell me it was all just a dream
is this farewell?
everything seems just like in a cinema
yet realistic at the same time too

happy like this, i’m by your side
it’s not that kind of ending, y’know?

Note: this is a pretty long article which does a deep dive into breaking some amateur crypto. I go on for quite a bit. Make a cup of tea before reading, and get ready to read some code!

introduction

Everyone knows it. Rolling your own cryptography is a terrible idea. Here’s Bruce Schneier writing about it in 1999. Here’s an excellent answer on the Infosec Stack Exchange about why you shouldn’t do it. Here’s another Scheiner post with an excellent opening sentence.

This, then, is a post about a broken homegrown cryptosystem; namely, that used in CodeIgniter, pre-2.2. This version was current until the release of CodeIgniter 2.2, on the 5th of June, 2014, and you can still find sites on it today.

The attack described in the post depends on a lot of things to go right (or wrong, if you will); it’s not just that they used a bad cipher, but also the fact that they rolled their own session storage, and implemented a fallback, and a dozen other things. This is probably typical for most bugs of this class; a bunch of bad decisions which aren’t thought through find their logical conclusion in complete insecurity.

Let’s get into it!

Read more

I made an extractor for Bemani IFS files. It’s been done a few times before, but this one’s reasonably usable compared to “drag and drop one file on this program, which spawns 50 files more, each of which you drag on this program to decompress, then each decompressed file to another program which spits up a window and has you drag a slider around until it looks right.”

Features:

  • You can open an IFS file and browse the data therein without saving them all out to a file.
  • You can export images as BMP, GIF, JPEG, PNG or TIFF.
  • The index used for guessing the aspect ratio of the image is saved and used automatically when browsing other images with the same pixel count.

It’s a .NET 2.0 application, and is in the public domain.

A screenshot!

a strange guide

This is a semi-comprehensive, non-prosaic (but hardly poetic) reference to vim. Approach with caution.

Modes:

  • normal: move the cursor, delete, enter insert mode, delete then insert, enter visual mode, yank and put (copy and paste)
  • insert: type stuff, move the cursor, scroll, put
  • visual: do things to the selected area, move your cursor about within it

And more:

  • registers: where your deleted and yanked text goes
  • text objects: even more ways to select text
  • what else?: what haven’t I covered?
  • CC0: this text is in the public domain

normal

move the cursor, delete, enter insert mode, delete then insert, enter visual mode, yank and put (copy and paste)

move the cursor

hjkl move the cursor left-down-up-right respectively. It’s good to remember these, but no big deal if you don’t; cursor keys usually work too. On Dvorak, you have jk on your left hand for vertical movement, and hl on your right for horizontal.

w moves a word forward. b moves a word backward. These stop at punctuation, too; if you want to skip a “word” where a word is just anything that’s not whitespace, W and B do the trick.

e moves forward until the end of a word, instead of the start of a word. E has the same rule as above.

These are a good start: using wbWB are pretty good for getting around quickly.

You can prefix any command with a number; 5w moves 5 words, 10B moves backwards 10, uh, ‘big’, words. These work on hjkl as well (and almost anything, honestly).

Go to the start and end of the line with ^$.

Go anywhere by searching with regular expressions; type /, and then complete the regular expression in the buffer beneath. n and N then find the next and previous match. These all function as movement commands in the below contexts, too. ? is the same as /, but backwards. Note that n and N are reversed when you’re in such a reverse search!

You can also go quickly to the following given character on the current line; f followed by the character instantly goes there. fa will put the cursor on the next a. Fa goes backwards. You can also go ‘til the next character, i.e. to the character immediately before, with t; ta puts the cursor just before the following a, and Ta just after the preceding one.

(You’ll note here that uppercase commands tend to vary the lowercase equivalents, by degree, inversion, or other shift in meaning. I do say “tend”, though, because there are plenty that don’t.)

<Enter> is another way of saying j, and <Space> another way of saying l.

) and ( move a sentence forward and backward respectively. } and { move by paragraphs instead.

]) and [( move to the next and previous unmatched ) and ( respectively, as do ]} and [{ for } and {. More understandably put: they move out to the next parenthesis or brace level in the given direction; they go “up” a level.

% moves to (and including) the matching token under the cursor; if your cursor is on a (, it’ll move to the matching ), etc.

G moves to the end of the document. If you prefix a number, it moves to that numbered line; 1G to the very top.

delete

d deletes stuff. You have to tell it what by following with a movement command: it deletes from the cursor position up to where you tell it. dw deletes the word you’re on, up to the start of the next word. dl deletes up to the following character; i.e. the single character your cursor is on.

Note that if you’ve hit d and change your mind, <Escape> will abort that command.

Operations spanning lines are a bit different. dd will delete the entirety of the current line. This repetition of a command to mean ‘this line’ is a pattern. You can use a number: 2dd will delete this line and the one after it. On that, 3dw deletes 3 words. (So does d3w.)

dj (delete down) will delete this and the next line. dk will delete this and the previous line.

D is the same as d$; it deletes to the end of the line.

d/blah<Return> will delete from the cursor until the next blah in the file. d?blah<Return> deletes back to the previous blah, including the entire matching word itself.

dta deletes from the cursor up to and not including the following a. dfa would include the a in its deletion. 2dta deletes up to and not including the second following a, etc.

x deletes a single character, as a handy shorthand. If you have the word programmer and the cursor positioned at the start, 10x will delete the whole word.

d) will delete to the end of the sentence; as will d} the paragraph. d% deletes the entirety of whatever the token under the cursor delimits; if your cursor is on a matched brace, the entire brace will be deleted, and so on.

enter insert mode

i will enter insert mode, leaving the cursor where it is.

a will enter insert mode, but places the cursor after the current character, i.e. appends.

I and A enter insert mode, but first move the cursor to the start or end of the line respectively. (These are handy.) They’re the same as ^i or $a respectively.

Note that numbers work with these too: 3i will enter insert mode, and whatever you type will be inserted thrice in total, once you leave the mode.

o and O enter insert mode, “opening” a line as they do. o opens a new line underneath (same as A<Return>), and O opens a line on the current line (same as I<Return><Escape>ki, or just ko if there’s a line above). You might need a mnemonic to remember which is which; O reaches higher up the screen than o, and so opens the line higher.

R enters replace mode; like insert, but overwriting text instead of inserting before it.

delete then insert

You can delete then insert in the same command; this is called “changing” text. Just use c instead of d, and it functions nearly identically as for deleting, and then enters insert mode on the spot. (cw is the same as dei; cc is the same as ^Di; the differences are that the whole object isn’t removed, that is to say, trailing spaces or newlines are kept. ce and cw do the same thing.)

C functions as Di. Change is often handy as c/, ct, cT, etc.

Note that numbers before change commands affect only the delete portion of the command; 2cw is literally like 2dei, not 2dei2i, in that it removes two words (excluding the trailing space), and then enters insert mode, but what you insert isn’t entered twice. This is almost always what you want.

s is like cl; it’ll change a character on your cursor. 10s erases the ten characters from under your cursor and enters insert. S is the same as cc (since otherwise what would it do different to C?).

r is like s but only for a single character; it waits for you to enter the character, which the one under the cursor will be replaced with. 10ra replaces the current and following 9 characters with the letter a.

enter visual mode

v enters visual mode. The cursor position at the time of entering visual mode becomes one end of the selection, and now you move the other.

Visual mode selects a range of text, traversing newlines as they occur sequentially in the text.

V enters visual line mode; this selects whole lines at a time, inclusive of both ends of the selection.

CTRL-V enters visual block mode; this selects a physical rectangular block of text. It can be a bit weird at first, especially with its behaviour crossing lines and empty lines, but quite useful, especially in combination with r, c, and s.

gv will enter visual mode with the last selection you had already made. (Note that your cursor won’t necessarily be anywhere near where it ends up.)

yank and put

y takes a movement like other commands; yy (copies) yanks the current line, yj the current and next, y2w (or 2yw) the two words from the cursor position onward, etc.

p puts (pastes) what was yanked after the cursor position. If you yanked linewise (like the first two examples above), the put is linewise: they are put as whole lines underneath the current one, as if you’d used o to open.

P yanks at the current cursor position, as if you used i to insert the text. If you yanked linewise, it puts as if you used O to open.

All delete commands, including change commands c, s and so on, actually have been the equivalent of “cut” operations. If you e.g. use cw to change a word, p will then put the deleted word.

See below about different registers in case you want to save some text without it getting clobbered by subsequent yanks or deletes.

insert

type stuff, move the cursor, scroll, put

type stuff

Type stuff! Go on!

move the cursor

If cursor keys are enabled, they probably will let you move the cursor in vim without exiting insert mode.

<Escape> will leave insert mode, by the way.

scroll

There’s a submode of insert mode that seems to be called expand mode; <CTRL-X> enables it. Any keypress not recognised by the mode will just be handled by regular insert mode and quit the expand mode; of note are <CTRL-E> and <CTRL-Y>, which scroll the window down and up respectively. Just start typing to exit the mode.

It’s worth noting these scroll in normal mode, too.

put

You can also put text while in insert mode. <CTRL-R>" will put whatever you last deleted or yanked; see below about registers for more on this; the " is specifying the register to put.

You can get help on insert-mode keys, by they way, by typing something like :help i_CTRL-R.

visual

do things to the selected area, move your cursor about within it

do things to the selected area

Any command, like c and d, will affect the selected mode en masse.

move your cursor about within it

Pressing o will change the end of the selection you’re moving to the other end; useful to adjust the selection.

The three visual modes, accessed with v, V and CTRL-V, are exited by pressing the appropriate key again. You can switch between visual modes by pressing the key for any other visual mode while already in one.

registers

When you delete or yank text, it’s put in a register; a clipboard, essentially. Each register is identified by a character, and they’re referred to by prefixing a ". "a is the a register, "F the F register, etc. There are special registers; the unnamed register, "" (i.e. the register name is double-quote), is what’s used by default by yank, delete and put.

To perform any register-affecting command with a given register, prefix the entire command with the register reference; "ay$ yanks the rest of the line into the "a register. "ap puts that. (Note that dcsxy always affect the "" register, as well as any specified one.)

"_ is the blackhole register; it causes the command to affect no registers. (This is the only exception to the parenthesis prior to this. Ahem.)

Per put in insert, <CTRL-R> waits for a register name; <CTRL-R>a in insert mode will put the contents of the "a register.

Registers "0 thru "9 have special meaning; see |quote_number| (type :help quote_number in normal mode) for more information.

If you specify a letter-named register with lowercase letters, you replace the contents; "ayw sets the contents of the "a register to the word from the cursor on. The same register reference but uppercase causes that register to be appended to.

More special registers exist; see |registers|.

text objects

Text objects are another way to select areas of text. They can’t be used as movement commands in normal mode in and of themselves, as they refer to regions; they may only be movement command arguments to commands that do something to regions, such as d and c. They are, however, very useful.

I lied, though: they can be used directly in visual mode, which selects the region they refer to. Experimenting with this is a good way to see how they work.

Text object commands start with either i or a; the former selects an object not including surrounding whitespace (“inner”), the latter includes surrounding whitespace. (Much like i leaves the cursor put when entering insert mode, but a moves it forward. It sort of makes sense.)

After that, you specify the kind of object; they’re based roughly on movement commands. iw, for instance, refers to the word your cursor is on top of. diw deletes the word your cursor is on. Note how this differs (usefully) from dw; dw deletes forwards from the cursor until the start of the next word, whereas diw deletes the entire word under the cursor, regardless of your cursor’s position.

Some common use cases: change the word under the cursor: ciw. Delete the word under the cursor entirely, closing any gaps: daw. Same, but take any punctuation touching the word with it, too: daW.

Sentences and paragraphs are accessed with s and p, again, following i or a. Give some of these a try in visual mode to see what I mean: enter visual mode, then try iw, then aw. Try ip, then ap. (Note that these will put you in visual line mode.)

Some of the most useful are the matching token operators: i[ (or i] ­ there’s no difference) will select the contents of the nearest matching [...], but not the [] themselves. a[ (or a]) include the tokens themselves.

Same works for {, (, <, ", ', `. Changing a string or HTML attribute quickly? ci". Want to quickly start a selection with a given block of code? va{.

Repetition works: 2di( deletes everything in the second-closest pair of parentheses, but not those parentheses themselves.

Editing HTML, t will work on HTML tags; it the contents of the nearest tag, at the contents and opening and closing tags themselves.

These are really lovely. See |text-objects| for more.

what else?

This is not even close to an exhaustive list of things in vim.

There are recordings (macros), which can be extraordinarily powerful; see |recording|. Note that they record and execute into and of the same registers that you paste in, meaning you can paste out a recording, modify in the text, then yank it back into the register for execution.

There are even more movement commands: * and # search for the word under the cursor, ; and , repeat your last tTfF command forward and backward (again with confusing reversing behaviour), ` and ' operate on marks set by m (see |mark|), and there are a million more commands prefixed by g (see |g|) that either modify the behaviour of the non g-prefixed command, or do something totally different. It’s worth a look.

Further, there’s the entire command-line mode, accessed by pressing : in normal mode. There are an enormous number of commands, many operating on ranges (pressing : in visual will start the command-line with a range matching the visually-selected lines); a useful one is s (see |:s|), for search-and-replace.

There’s ., which repeats your last command. Learn how it behaves in different circumstances and use it.

Keyword completion, triggered directly with CTRL-N and CTRL-P or through expand (^X) mode (see |i_CTRL-N|, |i_CTRL-X_CTRL-N|) just does a lookup based on tokens in open files. There’s omnicomplete (|compl-omni|, |i_CTRL-X_CTRL-O|), which can use plugins to intelligently autocomplete (like IntelliSense).

And a heck of a lot more, too.

Good luck!

thanks

Thanks to Nethanel Elzas for emailing in fixes and suggestions!

CC0

CC0
To the extent possible under law, kivikakk has waived all copyright and related or neighboring rights to “Being sorta useful in vim”. This work is published from: Australia.

It’s impressive how hard it is to use VSTO correctly.

When a COM handle crosses from unmanaged into managed code for the first time, a “runtime callable wrapper” is created for it. This wrapper maintains a reference count, incremented whenever the object is newly requested across the boundary.

The wrapper releases the COM object behind it when its finalizer is run, but this relies on all .NET references to the RCW itself becoming inaccessible to the GC and for the GC to pick it up. This may not reliably happen.

You can force the matter by decrementing the reference count on the RCW yourself: Marshal.ReleaseComObject does this, and should the reference count drop to zero, the COM object will get released immediately. This means taking a lot of care, though, since it’s not always obvious which VSTO calls result in the counter being incremented and which don’t. Something like the following can help work it out:

var refs = new object[3];
for (var i = 0; i < 3; ++i)
    refs[i] = my.VSTO.Call();
for (var i = 0; i < 3; ++i)
    Debug.WriteLine(Marshal.ReleaseComObject(refs[i]).ToString());

If each call to Marshal.ReleaseComObject returns the same number, it implies each call is incrementing the reference count; if the return values are decreasing, the call is not incrementing the count.

VSTO event callbacks are said to not need references released on passed-in COM objects, but an MSDN blog disagrees. There is no consensus here. For anything else at least: remember to release COM objects when you’re done with them, otherwise it can get painful.

Time zones in .NET

I’m a fairly new .NET developer. I worked with the framework a bit in 2005–6, but hadn’t really touched it since then. In the meantime, I’ve been sticking to the Linux ecosystem, and a little OS X, as mentioned in a previous article.

So, time zones. I know they’re a sore point for many environments, but most seem to dig themselves out of the hole and provide something that is, in the end, usable.

Ruby’s builtin Time is actually pretty darn good, and if you use Rails, ActiveSupport makes it even better. pytz seems .. alright. Databases generally have their heads screwed on straight. A lot of the time you can get away with just storing seconds since the epoch and call it a day, because there’s nothing more intrinsic built into the system.

Then I got my new job, and it was time get back into .NET. A lot has changed since 2006; it had only hit 2.0 then, mind.

So I felt confident I was using the latest, modern stuff. We target 4.0 and 4.5 across our projects, and there’s plenty nice about it.

Then I had to work with System.DateTime. Oh. Oh, gosh.

I quote the manual at you.

DateTime.Kind Property

Gets a value that indicates whether the time represented by this instance is based on local time, Coordinated Universal Time (UTC), or neither.

Kind is the lone field on a DateTime which has anything to do with time zones. It can take the value Local, Universal, or Unspecified. What does that even MEAN. Note that Kind is ignored in comparisons, too, which can only mean more fun for application developers.


It would be remiss of me to fail to note the paragraph in the docs which state:

An alternative to the DateTime structure for working with date and time values in particular time zones is the DateTimeOffset structure. The DateTimeOffset structure stores date and time information in a private DateTime field and the number of minutes by which that date and time differs from UTC in a private Int16 field. This makes it possible for a DateTimeOffset value to reflect the time in a particular time zone, whereas a DateTime value can unambiguously reflect only UTC and the local time zone’s time. For a discussion about when to use the DateTime structure or the DateTimeOffset structure when working with date and time values, see Choosing Between DateTime, DateTimeOffset, and TimeZoneInfo.

The linked page states that “although the DateTimeOffset type includes most of the functionality of the DateTime type, it is not intended to replace the DateTime type in application development.”

Intention or not, it should ALWAYS be used. It lists as a suitable application for the DateTimeOffset:

  • Uniquely and unambiguously identify a single point in time.

Because we don’t want that at any other time? When do you want a DateTime which non-specifically and ambiguously identifies several points in time?

On the other hand, listed as suitable for DateTime:

Retrieve date and time information from sources outside the .NET Framework, such as SQL databases. Typically, these sources store date and time information in a simple format that is compatible with the DateTime structure.

No fucking comment.

It continues:

Unless a particular DateTime value represents UTC, that date and time value is often ambiguous or limited in its portability. For example, if a DateTime value represents the local time, it is portable within that local time zone (that is, if the value is deserialized on another system in the same time zone, that value still unambiguously identifies a single point in time). Outside the local time zone, that DateTime value can have multiple interpretations. If the value’s Kind property is DateTimeKind.Unspecified, it is even less portable: it is now ambiguous within the same time zone and possibly even on the same system on which it was first serialized. Only if a DateTime value represents UTC does that value unambiguously identify a single point in time regardless of the system or time zone in which the value is used.

Completely useless. So, we’ll use DateTimeOffset in our application code, right?

Only the ecosystem hasn’t caught up.


Enter Npgsql, a Postgres driver for .NET with a frightening amount of code. It only works with DateTime objects when sending or receiving timestamps to or from Postgres.

Postgres has two column types: timestamp with time zone and timestamp without time zone (or timestamptz and timestamp, respectively). The former is about as good as a DateTime, but without trying to be more than it can: it doesn’t have Kind, which improves its usability by an order of magnitude. You can make a policy decision like “we’ll always store UTC timestamps”, and you’ve solved timezones in your application. They mark a specific point in time unambiguously.

Or you can just use timestamptz and they still unambiguously mark a specific point in time. It’s magic!

So how does Npgsql deal with this?

The genesis of this post was because we were noting strange behaviour: we had read a timestamptz out of the database, and then later SELECTed all rows where that column was strictly less than the value we read out. And yet that same row would be included in the result. It made no sense.

Turns out it really did make no sense.

The rest of this blog is a sequence of test cases which demonstrate just how bad the situation is.

[SetUp]
public void SetUp()
{
    _connection = new NpgsqlConnection(
        "Host=localhost; Port=5432; Database=time_zones_in_dot_net; " +
        "User ID=time_zones_in_dot_net");
    _connection.Open();
}

[TearDown]
public void TearDown()
{
    _connection.Close();
}

[Test]
public void TimeZonesSane()
{
    // introduction
    // ------------

    // This test assumes the *local* machine (running NUnit) has local time of +10.
    // It's agnostic to the time zone setting on the database server.
    // In other words, Postgres +1, .NET -100000000.

    // Render UTC (+0), Postgres (+3) and .NET (+10) distinguishable.
    _connection.Execute("SET TIME ZONE '+3'");

    // In the below tests we assert that the queries yield a .NET DateTime object
    // which, when .ToUniversal() is called on it, produces the given date in
    // "mm/dd/yyyy HH:MM:SS" format.

    // After that is the non-.ToUniversal() date in parenthesis.  This is *always* 10
    // hours ahead for a Local or Unspecified, and the same for Utc.  DateTime
    // objects have no knowledge of offset, only Kind.

    // There's also a character appended to represent what time zone "kind" it came
    // back with; "L" for Local, "?" for Unspecified, "U" for Universal.

    // As noted below, ToUniversal() on a Local or Unspecified returns a new DateTime
    // with Kind set to Universal, and unilaterally subtracts the time zone offset of
    // the machine the code is running on.


    // tests using string literals
    // ---------------------------

    // Not useful in themselves, because we'll never use string literals, but help to
    // demonstrate some initial weirdness.

    // string timestamp time zone unspecified: assumed to be in database local time.
    // Returns with Local Kind.
    QueryEqual("09/11/2013 03:47:03 (09/11/2013 13:47:03) L",
               "SELECT '2013-09-11 06:47:03'::timestamp with time zone");

    // string timestamp with time zone: should come back with the correct universal
    // value, with Local Kind.
    QueryEqual("09/11/2013 05:47:03 (09/11/2013 15:47:03) L",
               "SELECT '2013-09-11 06:47:03+1'::timestamp with time zone");


    // string timestamp without time zone: comes back 'unspecified' with the exact
    // datetime specified.  ToUniversal() assumes unspecified = local, so -10.
    // Returns with Unspecified Kind.
    QueryEqual("09/10/2013 20:47:03 (09/11/2013 06:47:03) ?",
               "SELECT '2013-09-11 06:47:03'::timestamp without time zone");

    // string timestamp with time zone, coerced to not have a time zone: as if the
    // time zone wasn't in the string.  Returns with Unspecified Kind.
    QueryEqual("09/10/2013 20:47:03 (09/11/2013 06:47:03) ?",
               "SELECT '2013-09-11 06:47:03+1'::timestamp without time zone");


    // tests using .NET values as parameters
    // -------------------------------------

    // These represent what we'll usually do.  They're also really messed up.

    // unadorned parameter: regardless of the DateTimeKind, the date is treated as
    // without time zone; the exact date given comes back, but with Unspecified Kind,
    // and so is as good as forced to local time.
    DateTimesEqual("09/10/2013 20:47:03 (09/11/2013 06:47:03) ?",
                   "SELECT @DateTime",
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Local),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Unspecified),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Utc));

    // parameter specified as with time zone: regardless of the DateTimeKind, the
    // date is treated as in the database local time.  It comes back with Local Kind.
    DateTimesEqual("09/11/2013 03:47:03 (09/11/2013 13:47:03) L",
                   "SELECT (@DateTime)::timestamp with time zone",
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Local),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Unspecified),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Utc));

    // parameter specified as without time zone: as for unadorned parameter.
    DateTimesEqual("09/10/2013 20:47:03 (09/11/2013 06:47:03) ?",
                   "SELECT (@DateTime)::timestamp without time zone",
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Local),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Unspecified),
                   new DateTime(2013, 9, 11, 6, 47, 3, DateTimeKind.Utc));


    // discussion
    // -----------

    // DateTime parameters' kinds are ignored completely, as shown above, and are
    // rendered into SQL as a 'timestamp' (== 'timestamp without time zone').  When a
    // comparison between 'timestamp with time zone' and timestamp without time zone'
    // occurs, the one without is treated as being in database local time.

    // Accordingly, you should set your database time zone to UTC, to prevent
    // arbitrary adjustment of incoming DateTime objects.

    // The next thing to ensure is that all your DateTime objects should be in
    // Universal time when going to the database; their Kind will be ignored by
    // npgsql.  If you send a local time, the local time will be treated as the
    // universal one.

    // Note that, per the second group just above, 'timestamp with time zone' comes
    // back as a DateTime with Local Kind.  If you throw that right back into npgsql,
    // as above, the Kind will be summarily ignored and the *local* rendering of that
    // time treated as UTC.  Ouch.


    // conclusions
    // -----------

    // 'timestamp with time zone' is read as DateTime with Local Kind.  Note that the
    // actual value is correct, but it's invariably transposed to local time (i.e.
    // +10) with Local Kind, regardless of the stored time zone.  Calling
    // .ToUniversal() yields the correct DateTime in UTC.

    // DateTime's Kind property is ignored.  To work around, set database or session
    // time zone to UTC and always call ToUniversal() on DateTime parameters.

    // Don't use 'timestamp without time zone' in your schema.
}

private void DateTimesEqual(string expectedUtc, string query,
                            params DateTime[] dateTimes)
{
    foreach (var dateTime in dateTimes) {
        var cursor = _connection.Query<DateTime>(query, new {DateTime = dateTime});
        DatesEqual(expectedUtc, cursor.Single());
    }
}

private void QueryEqual(string expectedUtc, string query)
{
    DatesEqual(expectedUtc, _connection.Query<DateTime>(query).Single());
}

private static void DatesEqual(string expectedUtc, DateTime date)
{
    var code = "_";
    switch (date.Kind) {
        case DateTimeKind.Local:
            code = "L";
            break;
        case DateTimeKind.Unspecified:
            code = "?";
            break;
        case DateTimeKind.Utc:
            code = "U";
            break;
    }

    var uni = date.ToUniversalTime();
    Assert.AreEqual(expectedUtc,
                    string.Format("{0} ({1}) {2}",
                                  uni.ToString(CultureInfo.InvariantCulture),
                                  date.ToString(CultureInfo.InvariantCulture),
                                  code));
}