Growing Pains - Maintenance Service full re-write

Hi everyone,

You may not be aware but one of the major working components of our system is the "maintenance service". The maintenance service is responsible for most of the background processing of records and sightings to ensure that things show up at the right time and on the right pages after new records are received or when existing records are updated.

Due to our growth, we are now in a situation where the maintenance service cannot keep up with the demand being placed on it. This is pretty much a 3 year old architecture that I originally put in place to alleviate the web site performance issues we were experiencing back then (again due to our growth). The maintenance service did an awesome job and took over processing by moving it offline into background operations so that website users didn't have to wait..

But here we are 2-3 years later, now having maxxed out that architecture and needing to fully rebuild it to handle the next order of magnitude in our growth.

My colleague, Rohan Thomas, has been working extremely flat out day and night to modernise the maintenance service by replacing it with a brand new scalable, queue-based architecture. This is the stuff that powers real enterprise level software systems and the kind of design that we work with day in day out at our normal day jobs. That same design has now become a necessity for NatureMapr.

I know that there have been reports from users noticing some sightings not showing up where they should, or at least, not quickly enough. I want to assure you that this is only a display issue and that the underlying "source of truth" data is fully intact.

While Rohan's important work of redesigning the maintenance service takes place, we expect that you may encounter a few such issues. I also expect this may continue for a couple of weeks. Your data isn't actually missing, it simply may not have been "published" by the maintenance service yet, but it will be soon.

Thanks so much for your patience and continued support. It means a tremendous amount to both Rohan and I.

Please know that we are fully onto it - the result will be a hugely scalable citizen science platform that hums along and is able to support the next wave of NatureMapr's continued growth.