Lotus Geek :: Sorry about the blog outage...

Category This Blog
Bookmark :

My provider, DominoDeveloper.net (DDN), had a serious problem with their blog server - it died a horrible death. They had a planned outage to fix the problem, but they had a hardware failure before they could have the planned hardware outage. Long story short, the server had a hard crash, and it took awhile to get it back up.

On top of that I have had some internal blog issues - I had a cleanup agent that was not written properly, so my logs had grown huge, and I had not noticed - and my blog had reached 7GB. So, I had fixed the cleanup agent and ran it, and it had reduced the log size down to something manageable, but my blog's whitespace was massive - so I needed to get rid of it. So, to do it the easiest way I simply created a copy of my blog to delete all the view indices and whitespace, and then transeferred that back to the blog server once DDN got the blog server back up.

Well, everything is back to normal, and the blog is (hopefully) running smoothly. Sorry for the outage.

Incidentally, I'll be in Miami next week speaking at the Lotus Advisor Summit in Miami, so if you happen to be going please stop by and say "hi"!

Rock
**Words that soak into your ears are whispered, not yelled.

Comments

1 - And it happens again... this time 24 hours. Apparently due to a firewall going wrong in the US data centre? Hmm.

Posted by Ben Poole At 11:10:29 AM On 10/16/2007 | - Website - |

2 - Ok, I lied. It's not quite that simple. The problem is compounded by the fact that BlogSphere is using a NotesDocumentCollection to render the comments. NotesDocumentCollections are - you guessed it - NOT sorted using the view's sort order (they must be in either NoteID order or the order in which the note was created "in this file"). That normally works fine, but in this scenario it works badly...

Posted by Rob McDonagh At 10:31:12 PM On 09/28/2007 | - Website - |

3 - Would you believe I was in the middle of my presentation, "Social Software - It's all about You!" when I discovered your blog was down. It was the only link in my "examples" that I had a problem. I was a little taken back that it was down(thought I had done something wrong). I quickly recovered by showing another blog.
I'm doing the presentation again tomorrow. Thanks for the update.
No worries. Emoticon

Posted by Curt Stone At 05:13:21 PM On 09/26/2007 | - Website - |

4 - Ok, I couldn't find a Righteous Hack to work around this issue, so I enlisted the aid of Mr NSFTools, Julian Robichaux. Julian resurrected a tip he posted a few years ago about pulling the created date out of the UNID, and I'm going to use that to put a field on the documents containing the original creation date. I'll write it all up on my blog and show the code, but it'll be late tonight before I have the opportunity. Just wanted to update the thread here in case anybody else is trying to solve the problem.

Posted by Rob McDonagh At 10:24:48 AM On 09/28/2007 | - Website - |

5 - Yeah I got hit with this too. In a big plus for Notes, the fact that I had a local replica was a god-send, because I could then ftp my blog and wiki back up to the DDN box, rather than wait for the recovery software to restore the files. Huzzah!

Posted by Ben Poole At 04:41:16 AM On 09/27/2007 | - Website - |

6 - Yep, happened to me as well. And there's still an issue with my blog, and you have the same problem, Rock. Makes sense, we both run Blogsphere.

The problem is that on old (pre-crash) posts, the comments are now completely out of order. If you want to see it in action, look at your post about the Britney VMA - the comment by Carl about how she's not fat is the 4th item, your response to him is the 3rd, and his response back again is the 2nd.

From what I can tell, this is because Blogsphere uses the @Created and doc.Created document properties to do date sorting instead of saving a field for that purpose. Usually that works great. In this case, though, it runs into the issue where the created date IN THIS FILE doesn't match the created date INITIALLY (because both of us used our local replicas to restore). To display the comments, it uses the in this file version, and that's the date it replicated to your local copy. We want the "initial" date, but I haven't come up with a way to get at it yet.

You know way more than I do about this sort of thing - any brilliant ideas?

Posted by Rob McDonagh At 09:35:43 PM On 09/27/2007 | - Website - |