2022-06-18

The parable of the form request form

One of my first jobs after college was at a "startup". Of course, 30 years ago in the UK no one talked about startups, but it was a small technology company building a product to disrupt a market. When I joined the company it was very small and internal procedures were ad hoc.

For example, to request holiday time off you simply worked that out with your manager and the manager let the one person in "personnel" know. As the company grew that didn't work very well and a physical, printed form was introduced.

This form, along with a number of others, was available in pigeon holes around the company. It was a simple A4 sheet of paper: name, manager's name, days requested, and signatures. You filled it in, got your manager to sign it off and left it in a pigeon hole to be routed to personnel.

The supply of forms was handled by a simple distributed procedure. The person taking the last form from a pigeon hole was requested to make five photocopies before filling it in. That way the forms would never run out.

This worked very well. Until one day when a new procedure was introduced.

It was no longer acceptable to photocopy forms. You had to request new blank forms to keep the pigeon holes filled. And to do that there was a form: a form request form.

The form request form had a line for the location where forms were needed and various boxes to tick for different types of form. You filled in the form and placed it in the personnel pigeon hole to be dealt with. At some point personnel would replenish the forms. 

The reason this form was introduced was that the old photocopying method would cause "out of date" forms to be used. And so personnel wanted to ensure that only new revisions of forms were used. The reality was that new revisions of forms were very rarely introduced and so an inefficient procedure was introduced to replace something that worked well to address a problem that rarely happened.

Naturally, the form request form had a box to request additional form request forms. I never did find out what would happen if you requested more forms using an out of date form request form.

2022-06-09

The mysterious behaviour of the Flying Tiger Countdown Clock (and the bug that causes it)

Flying Tiger is a Danish "variety shop". It sells all manner of inexpensive trinkets, utensils and runs through stock quickly so there's always something seasonal. I came across a simple clock that shows the number of days remaining before some event (your birthday, the next solar eclipse, ..., whatever you choose).


It works. Sometimes, but other times it gets the "days remaining" count completely wrong. It can be wildly too large, or wildly too small. I couldn't resist trying to reverse engineer the algorithm used and the bugs that are in the implementation. 

Let's start with some combinations of the current date and the deadline being counted down to that work fine.

  Date        Deadline    Delta in Days  Displayed  Error

  2020-01-01  2021-01-01  366            366        0
  2020-01-01  2022-01-01  731            731        0
  2020-01-01  2023-01-01  1096           1096       0
  2020-01-01  2020-01-02  1              1          0
  2020-01-01  2020-02-01  31             31         0
  2020-01-01  2020-01-01  0              0          0
  2019-01-01  2020-01-01  365            365        0

Now take a look at when things go wrong.

Same Year


But here are some that go wildly wrong. My best guess at understanding why is based on assuming (after a lot of observation) that the programmer has a number of special cases within their code. Firstly, if the month and year are the same in the date and deadline then the number of days is correct (and I assume they just subtract the deadline from the date). This special case actually made tracking down what was going on for other combinations more complex (and later I realized that special case wasn't needed). But here are pairs of dates in the same year.

Date        Deadline    Delta in Days  Displayed  Error

2020-01-01  2020-09-01  244            244        0
2020-01-01  2020-10-01  274            18         -256
2022-09-10  2022-10-27  47             65327      65280
2022-10-10  2022-11-11  32             32         0

For the moment, ignore the third line as it involves more reasoning about what's happening. I believe the algorithm used to calculate the days is as follows (when both dates are in the same year). Here I show the working for the second line above. Notice how the result is correct with dates before and after September. (first and last lines above) We'll see why that matters soon.

  date_as_day_of_year     = day_of_year(2020-01-01)
  deadline_as_day_of_year = day_of_year(2020-10-01)

  delta                   = deadline_as_day_of_year - 
                            date_as_day_of_year

I am assuming that the clock has a day_of_year function that converts a date to a sequential day number (e.g. January 1 is day 0, February 25 is 55, etc.). It appears that the return value of that function is mistakenly turned into an unsigned 8-bit integer:

  date_as_day_of_year     = day_of_year(2020-01-01)     // 0
  deadline_as_day_of_year = day_of_year(2020-10-01)     // 18 (not 274)

  delta                   = deadline_as_day_of_year -
                            date_as_day_of_year         // 18

The reason September is important is that the 255th day of the year is September 12 (September 11 in leap years) and so calculations will go wrong when they cross September. Note that for the fourth line above the calculation works even though the returned values have been turned into unsigned 8-bit integers.


What about the strange number 65327? I think that comes about when the clock tries to display a negative integer. The calculation is:

  date_as_day_of_year     = day_of_year(2020-09-10)     // 253
  deadline_as_day_of_year = day_of_year(2020-10-27)     // 44

  delta                   = deadline_as_day_of_year -
                            date_as_day_of_year         // -209

If delta is an unsigned 16-bit integer then -209 comes out to be 65327. Alternately, it's signed but the output function (sprintf?) assumes it's unsigned.

So, it looks like the clock uses the day of the year for calculations and has type errors that cause weird output. What about spanning more than one year?

Crossing New Year


When crossing the new year the algorithm above won't work but something similar will. First calculate the remaining days in the current year and then the additional days in the subsequent year to reach the deadline. Add them together. Take an actual example from the clock (one that works!)

  Date        Deadline    Delta in Days  Displayed  Error

  2020-09-30  2021-01-02  94             94         0

Here's pseudocode for that:

  days_in_current_year    = days_in_year(2020-09-30)
  date_as_day_of_year     = day_of_year(2020-09-30)
  deadline_as_day_of_year = day_of_year(2021-01-02)

  delta                    = (days_in_current_year -
                              date_as_day_of_year) +
                             deadline_as_day_of_year  

(Note that I've ignored the case where there are multiple intervening years. The clock just adds the number of days in each additional year and does that part correctly.)

That algorithm appears to be what the clock is using and here's a long dump of test dates and deadlines. Look at the lines with errors. They all involve a date or deadline after September. Did the programmer make the same 8-bit unsigned int error twice?

  Date        Deadline    Delta in Days  Displayed  Error

  2020-08-30  2021-01-02  125            125        0
  2020-08-30  2021-02-02  156            156        0
  2020-08-30  2021-03-02  184            184        0
  2020-08-30  2021-04-02  215            215        0
  2020-08-30  2021-05-02  245            245        0
  2020-08-30  2021-06-02  276            276        0
  2020-08-30  2021-07-02  306            306        0
  2020-08-30  2021-08-02  337            337        0
  2020-08-30  2021-09-02  368            368        0
  2020-08-30  2021-10-02  398            142        -256
  2020-08-30  2021-11-02  429            173        -256
  2020-08-30  2021-12-02  459            203        -256
  2020-09-30  2021-01-02  94             94         0
  2020-09-30  2021-02-02  125            125        0
  2020-09-30  2021-03-02  153            153        0
  2020-09-30  2021-04-02  184            184        0
  2020-09-30  2021-05-02  214            214        0
  2020-09-30  2021-06-02  245            245        0
  2020-09-30  2021-07-02  275            275        0
  2020-09-30  2021-08-02  306            306        0
  2020-09-30  2021-09-02  337            337        0
  2020-09-30  2021-10-02  367            111        -256
  2020-09-30  2021-11-02  398            142        -256
  2020-09-30  2021-12-02  428            172        -256
  2020-10-30  2021-01-02  64             320        256
  2020-10-30  2021-02-02  95             351        256
  2020-10-30  2021-03-02  123            379        256
  2020-10-30  2021-04-02  154            410        256
  2020-10-30  2021-05-02  184            440        256
  2020-10-30  2021-06-02  215            471        256
  2020-10-30  2021-07-02  245            501        256
  2020-10-30  2021-08-02  276            532        256
  2020-10-30  2021-09-02  307            563        256
  2020-10-30  2021-10-02  337            337        0
  2020-10-30  2021-11-02  368            368        0
  2020-10-30  2021-12-02  398            398        0

Initially, I thought the programmer must have made the same error, but there was something odd. If it were true then the errors would be non-zero for part of September (after September 11 or 12) as well as October and beyond. But looking at the example I have above of September 30, the answer was correct.

And I suspect that there was more likely to be one bug causing the observed behaviour in all cases than multiple odd places where changing integer size happened.

The Bug


So, what could cause the value of day_of_year to be wrong for October, November and December, and, specifically, be off by 256? Actually, I think it's pretty easy to write that function incorrectly in a subtle way. Here's an implementation of day_of_year that is correct right up until September 30 and then goes wrong in exactly the way the clock fails. (I've ignored leap years for clarity, and because the clock handles leap years just fine).

  #include "stdio.h"
  #include "stdint.h"

  uint8_t days[12] = {31,28,31,30,31,30,31,31,30,31,30,31};

  uint16_t day_of_year(int month, int day) {
    uint8_t count = 0;
  
    for (int i = 1; i < month; i++) {
      count += days[i-1];
    }

    return count + day;
  }

  int main() {
    int m = 10;
    int d = 5;

    printf("%02d-%02d %d\n", m, d, day_of_year(m, d));
    return 0;
  }

Can you see the error? count only overflows after September. Naturally, there could be other ways to implement this function, including pre-computing the days in each month. But I think this is likely the source of the error that causes the clock to fail oddly.

So, Flying Tiger, where's the source code, and can I fix it?