Monday, February 1, 2010

Information Overload

Information abundant: information pools, information pockets, information streams, information databases; Information sources of so many types delivered in so many formats. I have too much information to manage it properly. Well, when I say 'properly' I mean there's too much to remember all of it, digest all of it, to take full potential of all the data being delivered to me.

I am a child of the information age: When I wake up in the morning the first thing I do is get behind my computer to catch up with the world around me. I've got social networks, business networks, email accounts, and am a programmer so have a multitude of other internet services and jobs running on schedules. I need(read want) to know certain things at certain times. If a client has requested my attention via email I need to know as soon as possible. If it's my mothers birthday I need to know 7am in the morning. If my web server goes down, I need to know five minutes ago.

With so many online services and so many of them offering an RSS/Atom stream about nearly everything; An RSS reader does a good job on keeping me up to date with 66.67% of my knowledge source. But even reading my master RSS feed every morning doesnt provide all the information I need, but more importantly it doesnt invoke some simple actions that I find repetitive when I come across certain data.

  • I have a facebook account; I want to be alerted of birthdays in the early morning.
  • I follow some twitterers'; I only want to view tweets that are re-tweeted.
  • I have family members on Flickr and Picasa; If they post photos then I want to see them, better yet funnel them straight to my wireless photo frame in one merged Media RSS stream.
  • I've got an AreMySitesUp account: I need to urgently know when my servers are unavailable.
  • I've got Gtalk, Skype and AIM acocunts...
  • I have setup FTP locations that clients drop work in...
  • I've got several email accounts...
  • I follow several open source projects on forums, bug trackers, and version releases...
  • I run several projects and each of those have several checks and balances in place to make sure all is in order.

I could go on, but the point is made. I've got so many sources of information of varying types and each hold a certain level of importance or urgency to me. Now I want a central way to organise all of this; A way to manage and schedule my information.

I am a programmer; I'm going to make something. An information engine. This engine will:

  • Store data
  • Consume data stream/trigger inputs
  • Publish stream/alert outputs
  • Provide a management console for all of the above
More on this to come...

2 comments:

Patrick said...

Any progress, Mr. Fellow Pat Cullen? I am in the early stages of exploring Apache Lucene for the same reasons.

Unknown said...

Hey, I have had much progress; Unfortunately it's not near any practical use yet. I had seen Lucene, and have just reread through it's Wiki page to get a handle on what it does.. I should probably look into that project as it looks really awesome. The engine I'm trying to build now has a far more narrow scope than Lucene's and I don't want to get distracted from it's core purpose with all the bells and whistles. I'll drop you a line when I've got something to show off. Thanks for connecting. // Pat