Posted by John Viega on Sun, Sep 07, 2003 (10:21 PM) GMT

Problem

A change in API semantics that doesn't break the call by changing the signature can lead to insecurities when a developer tries to call one version of the API but gets a different version. This can be a problem when APIs are slightly different between platforms.

Solution

Keeping informed of subtle API changes is the hardest part. Generally, one should either detect which version of the API is in use or write code that is generic to either version.

Discussion

For various reasons, APIs can evolve over time or vary from platform to platform.

One example is the (v)snprintf() call for C and C++ programs, which originally returned -1 if the destination buffer wasn't big enough to hold the output string, but now returns the number of characters that would have been written into the buffer had there been enough room (in both cases, the API truncates the string if it's too long).

Let's say one is assuming the original behavior and writes some code as such:

  ...
  int   len;
  char buf[BUFSZ], *ptr = buf;
  
  len = vsnprintf(ptr, sizeof(buf) - 100, "%s", user_input_vargs_list);
  if (len < 0) {
     fprintf(stderr, "Too much input!\n");
     abort();
  }
  ptr += len;
  ...

If this code runs with the original vsnprintf() semantics then ptr will always point within the bounds of buf, leaving at least 100 bytes in which other data can be written. If this code links against a modern vsnprintf() implementation, then the value of ptr can be set to values that are absolutely out of bounds by passing in too big an input. While this doesn't cause the individual call to vsnprintf() to overflow, it will probably be a big problem if the user controls the next write to ptr.

Another gotcha with this particular API is that Microsoft's implementation does not provide NUL-termination in the case where the result is truncated, which can lead to problems when the user later assumes the string is NUL terminated.

What are the options beyond using an alternative that is more portable? If one can query library versions (for example, with configuration querying tools such as GNU autoconf) then one should do so. Another alternative is to simply write the code in such a way as to work no matter which version of the API is in use. In this particular case, there are three general ways to do this:

1. Use only on the behavior that is the same between two versions. For example, one could check to see if the length is in bounds by ones self.
2. Check dynamically to see which version of the call is being used. For example, one might first call vsnprintf() with a response that always overflows and check to see whether the return value is positive or not.
3. Design a test that works despite the API. In this case, the test if (len < 0 || len >= sizeof(buf) - 100) always assures us that we detect conditions where there isn't space in the buffer (i.e., either the buffer was truncated, or there was no room for a terminating NUL in some APIs).