The official weblog of ClickBook.net. ClickBook streamlines the process of appointment scheduling for small business, schools and other organisations. For technical updates please visit the Technical Blog.

    SMS Outage - Why it happened and what we’ve done to prevent it from happening again

    Mark Demicoli  15 March 2010 07:51:24 PM
    The recent SMS outage to North American destinations proved to be a major inconvenience to many customers.

    In this post, I wish to explain what happened and what we've now done to prevent such a long outage from happening again.

    What happened?

    Some two weeks ago, a small minority of Clickatell's customers engaged in sending thousands of inappropriate SMS messages, resulting in Clickatell being banned from many major telecommunications networks.

    The nature of the problem was so severe, that they are still struggling to reconnect to all major networks.

    This has taught them a lesson, and us likewise.  High in our awareness is the importance of the trouble-free availability of ClickBook.net to our customers.  In our online help, we make it clear that there are always risks associated with using any technology, however some basic diligence goes long to outweigh the risks with the many benefits.  A serious problem with your internet connection, for example, would mean that you could not access your schedule in a timely manner.  Being fully aware of this, we have a feature that sends you a daily email with your forthcoming appointments.  This is one measure that helps to mitigate such outages.

    In addition, we always recommend that customers download their appointments on a regular basis, via the Account tab.  However, even these two measures could not have prevented the SMS outage which was completely out of our control.

    What we've done.

    While we never expected such a long outage from our SMS provider, the experience has been an important one, and we have taken steps to increase reliability in future.

    As a result of this event, we have now engaged the services of a second SMS provider, to whom we can switch in a moment's notice, should this sort of thing happen ever again.

    In addition, we are now working on an automatic detection system which will inform customers by email when SMS messages are not delivered.  That way, the system will give you the opportunity to contact customers manually (by phone or SMS) to remind them of their appointments.   Details of this measure will be announced shortly.

    To reimburse the costs of undelivered messages, we have also stopped deducting SMS credits from all accounts for a two week period from March 18.



    Comments

    1Kathy Tynan  13/09/11 10:53:40 AM  Now Email Reminders Are A Problem

    Client email reminders are being received the DAY OF their appointment at the appointed time. I've checked their client file & 24 hours before is indicated. Please advise.

    2Tiffany Pieger  11/09/11 12:17:00 AM  10% vs None

    You can trace and troubleshoot these failures with Clickatel support team (if they are still the gateway).

    With isolated pin pointed reports these types of issues should be less and less.

    I know we can provide you with failures to get you started, surely there are others... This should get you started and lower the failure rate once these type of SMS failures are resolved.

    This sounds more reasonable vs removing the feature altogether. After all, the feature works for so many others, it just fails for certain types of cell phone owners.

    Please let me know if you would like my help. I was an IT Product Development Manager in my past life and am very familiar with software bugs and development. It was my specialty to organize troubleshooting these types of efforts for 20yrs. I would gladly assist to get this issue resolved at no expense. It would be my pleasure.

    Kindest Regards,

    Tiffany Pieger

    Rocky Mountain Massage, Owner

    3Mark Demicoli  10/09/11 1:25:26 PM  SMS issues mainly affecting US

    Just to clarify a bit further, aside from the 10% "definitely did not go through", another 30% are 'unknown' due to lack of feedback from many SMS providers in the States. We are told that 'unknown' means 'probably delivered'.

    4Mark Demicoli  10/09/11 1:16:31 PM  SMS issues mainly affecting US

    From our estimation approximately 10% of SMS messages to US-based handsets do not get delivered. Unfortunately all our efforts to understand why have gone unresolved. The crux of the issue is to do with the variety of SMS providers in the States. We rely upon a brokerage service to negotiate agreements with the various providers in order for our messages to be delivered, and thus the success rate is out of our control. In terms of offering a service, this is the best we can do at the current pricing levels, however we are investigating more established brokerage services at much higher cost. At this stage we're still trying to understand the best balance between the price of ClickBook and the reliability of SMS messages. Due to the reliance that customers develop on SMS reminders in particular, the problem can be magnified. Whether to remove the feature completely until it is 100% reliable is the key question. What do you think?

    5Tiffany Pieger  10/09/11 10:34:10 AM  SMS Status: 50: SMS Failed. Contact customer / client manually.

    Still failing:

    >>>

    TO: ###########

    CLICKBOOK.NET Booking reminder: Friday, 9 September 2011 at 8:30 AM Client: Name Tel: ########### Mob: ########### Duration: 60 Min.

    Sent September 9 2011 05:41 AM

    SMS Status: 50: SMS Failed. Contact customer / client manually.

    <<<

    I have to say, this is the worst customer service I have ever seen!

    Please update this thread as appropriate.

    Thank you.

    6Tiffany Pieger  08/09/11 8:43:26 AM  Still no SMS for clients or employee’s - HELP!

    We are still seeing failures for SMS messages to clients and personnel.

    The following is an example of a failed SMS:

    ===PERSONNEL SMS FAILURE===

    TO: ##########

    CLICKBOOK.NET: New Booking for Mon, 5 Sep at 12:30 PM with NAME Mobile: ########## Dur: 30 Min. "hi ### I am making this app. for ### ...

    Sent September 2 2011 04:59 PM

    SMS Status: 50: SMS Failed. Contact customer / client manually.

    TO: ##########

    CLICKBOOK.NET Booking reminder: Monday, 5 September 2011 at 12:30 PM Client: NAME Tel: ########## Mob: ########## Duration: 30 Min.

    Sent September 5 2011 09:32 AM

    SMS Status: 50: SMS Failed. Contact customer / client manually.

    ===CLIENT SMS FAILURE===

    TO: ##########

    Rocky Mountain Massage Booking reminder for Mon, 5 Sep at 12:30 PM with NAME Tel: ########## Duration: 30 Min. -DoNotReply-.

    Sent September 4 2011 11:53 AM

    SMS Status: 50: SMS Failed. Contact customer / client manually.

    It would be greatly appreciated if you would reply back to let us know that you are aware of the issue and/or are working on it.

    We have opened numerous trouble tickets over the past month without any response from clickbook.

    Thank you in advance.

    7Stacey T  25/08/11 12:57:44 PM  No Reminders

    Now it is affecting us. Sure makes for allot of no shows as the clients count on those reminders. Hope this gets resolved quickly.

    8Kathy Tynan  25/08/11 8:26:12 AM  No Reminders are going out

    Please advise what the status is or if there is going to be a fix soon. I've now lost 3 appointments because of no reminders going out!!!!

    9Liz Majeski  25/08/11 1:23:53 AM  All SMS/Email BUT no reminders

    Everything was working fine on Monday...as far as reminders being sent out. No reminders went out Tuesday for today...I had to send reminders manually at the end of the day and due to that had two cancellations :( ...searched for hours last night for a "fix it" posting but I found nothing. My cancellations, confirmation, and new bookings SMS/emails are going out...It's just the REMINDER SMS/Email that are not being sent to staff or clients. Is there a fix or is this being addressed Mark Demicoli?

    10Tiffany Pieger  22/08/11 4:35:48 AM  Sporadic SMS to clients and staff

    My staff is experiencing SMS issues and we haven't heard anything from Clickbook staff regarding any fixes.

    Essentially, staff members and clients sporadically get SMS text messages for new, canceled and booking reminders. They are not reliable as they may not ever be sent from Clickbook, regardless of settings for staff members and clients.

    Please post an update whether this issue is being looked at.

    Thank you in advance.

    11Steve Shergold  16/03/10 12:18:32 AM  Technology outage

    It would also be really helpful in support of your comments above;

    "A serious problem with your internet .......... could not access your schedule in a timely manner"

    "..........recommend that customers download their appointments on a regular basis, via the Account tab"

    If we could have a printout of the daily appointment sheet to fall back to.