Category Archives: Bad

A short story

A few days ago, I had a lengthy reflection on how I used to "write code" when I was a boy (somewhere between 12 and 15 years old); I cannot remember what initialised the reflection, but it seemed interesting enough so I thought I would share it...

When I was a boy, my programs executed slowly. I don't think this is unexpected from programs developed by a kid, but on many occasions, mine were slow deliberately. I had this insane idea that by making my program do something in a complicated and/or convoluted way, it would be "smarter" or "better". It would be more awesome than the version that completed almost instantly.

I'm not sure what brought on this behaviour. Perhaps it was rooted in a desire to boast? i.e. because nobody I grew up with could program at all, I could boast about how complicated the task was which my program was solving and they wouldn't know any better? Another possible reason was related to computer games: I used to play games on my 486 DX and almost every game which I played had some sort of "loading" screen. The loading screen was where I would be anticipating what was to come - very soon, I was going to be placed into the action of the game. Maybe this instilled some idea that anything worth doing or having requires some period of waiting? Deep.

Thankfully, I abandoned this attitude a long time ago. It is a mind-numbingly stupid attitude.

However, I was lead to ponder the question: are there any adult software developers out there who actually write code like this? Maybe it's not the code that ends up being shipped (because it cannot meet performance requirements), but some internal thing that assists in creating the code which will ship? Maybe they think to themselves: "Ah-ha! I will make this code slow and - because they don't understand it - they will think I am a genius! Nobody will ever know!"? It all goes well for them until somebody proves their solution is so utterly inferior that they could not possibly have created it by accident... even the thought that these people could be employed somewhere makes me cringe.

Why you should not use C99 exact-width integer types

Is there really ever a time where you need an integer type containing exactly N-bits? There are C99 types which guarantee at least N-bits. There are even C90 types which guarantee at least 8, 16 and 32 bits (the standard C integer types). Why not use one of those?

I never use C99 exact-width types in code... ever. Chances are that you shouldn't either because:

Exact width integer types reduce portability

This is because:

1) Exact width integer types do not exist before C99

Sure you could create an abstraction that detects if the standard is less than C99 and introduce the types, but then you would be overriding the POSIX namespace by defining your own integer types suffixed with "_t". POSIX.1-2008 - The System Interfaces: 2.2.2 The Name Space

GCC also will not like you:

The names of all library types, macros, variables and functions that come from the ISO C standard are reserved unconditionally; your program may not redefine these names.
GNU libc manual: 1.3.3 Reserved Names

From my own experience using GCC on OS X, the fixed width types are defined even when using --std=c90, meaning you'll just get errors if you try to redefine them. Bummer.

2) Exact width integer types are not guaranteed to exist at all:

These types are optional. However, if an implementation provides integer types with widths of 8, 16, 32 or 64 bits, it shall define the corresponding typedef names.
ISO/IEC 9899:1999 - 7.18.1.1 Exact-width integer types

Even in C99, the (u)intN_t type does not need to exist unless there is a native integer type of that width. You may argue and say that there are not many platforms which do not have these types - there are: DSPs. If you start using these types, you limit the platforms on which your software can run - and are also probably developing bad habits.

Using exact width integer types could have a negative performance impact

If you need at least N-bits and it does not matter if there are more, why restrict yourself to a type which which could require additional overhead? If you are writing C99 code, use one of the (u)int_fastN_t types. Maybe, you could even use a standard C integer type!

The endianness of exact width integer types is unspecified

I am not not implying that the endianness is specified for other C types. I am just trying to make a point: you cannot even use these types for portable serialisation/de-serialisation without feral-octet-swapping-macro-garbage as the underlying layout of the type is system dependent.

If you are interested in the conditions for when memcpy can be used to copy memory into a particular type, maybe you should check out the abstraction which is part of my digest program. It contains a heap of checks to ensure that memcpy is only used on systems when it is known that it will do the right thing. It tries to deal with potential padding, non 8-bit chars and endianness in a clean way that isn't broken.

This article deliberately did not discuss the signed variants of these types...

Wave File Format Implementation Errors

I've read through and written many wave reader/writer implementations over the years and most of them are wrong. I tend to blame this on a vast number of websites which incorrectly "document" the wave format rather than point to the original IBM/Microsoft specifications (which, to be fair, are also pretty poor).

This post describes some of the main points of contention which exist in the wave format and points out some of the common issues which I've seen in implementations. It is a long post but I hope it will be useful to people writing software which uses wave files.

I will refer to the following three documents in this post:

  • IBM and Microsoft's "Multimedia Programming Interface and Data Specifications" version 1.0 dated August 1991.
  • Microsoft's "New Multimedia Data Types and Data Techniques" version 3.0 dated April 1994.
  • The MSDN article "Multiple Channel Audio Data and WAVE Files" dated 7th March 2007.

All of these documents (including the MSDN page) are available as PDF downloads here.

Wave chunks are two-byte aligned

All chunks (even the main RIFF chunk) must be two-byte aligned ("Multimedia Programming Interface and Data Specifications", pp. 11). A broken wave implementation which fails to do this will most of the time still input or output wave files correctly, unless:

  • Writing an 8 bit, mono wave file with an odd number of samples that has chunks following the data chunk
  • Potentially when reading/writing a compressed wave format or
  • Writing more complicated chunks like "associated data lists" which may contain strings with odd numbers of characters.

The format chunk is not a fixed sized record

The format chunk contains the structure given below followed by an optional set of "format-specific-fields" ("Multimedia Programming Interface and Data Specifications", pp. 56):

struct {
WORD wFormatTag;        // Format category
WORD wChannels;         // Number of channels
DWORD dwSamplesPerSec;  // Sampling rate
DWORD dwAvgBytesPerSec; // For buffer estimation
WORD wBlockAlign;       // Data block size
};

For WAVE_FORMAT_PCM types, the "format-specific-fields" is simply a WORD-type named wBitsPerSample which:

... specifies the number of bits of data used to represent each sample of each channel
("Multimedia Programming Interface and Data Specifications", pp. 58)

Be aware that this definition is vague and could mean either:

  • The number of valid resolution bits or
  • The bits used by the sample container

The WAVE_FORMAT_EXTENSIBLE format tag solves this confusion by defining wBitsPerSample to be the container size and providing an additional wValidBitsPerSample field ("Multiple Channel Audio Data and WAVE Files", pp. 3-4).

The original specification did not specify what "format-specific-fields" was for any other format. However in the 1994 update, the use of WAVEFORMATEX became mandated for any wFormatTag which is not WAVE_FORMAT_PCM ("New Multimedia Data Types and Data Techniques", pp. 19). This structure is defined as follows:

/* general extended waveform format structure */
/* Use this for all NON PCM formats */
/* (information common to all formats) */
typedef struct waveformat_extended_tag {
WORD wFormatTag;       /* format type */
WORD nChannels;        /* number of channels (i.e. mono, stereo...) */
DWORD nSamplesPerSec;  /* sample rate */
DWORD nAvgBytesPerSec; /* for buffer estimation */
WORD nBlockAlign;      /* block size of data */
WORD wBitsPerSample;   /* Number of bits per sample of mono data */
WORD cbSize;           /* The count in bytes of the extra size */
} WAVEFORMATEX;

What this means is that all wave formats contain the members up to and including wBitsPerSample. Only wave files with formats which are not WAVE_FORMAT_PCM are required to have the cbSize member.

When WAVE_FORMAT_EXTENSIBLE should be used

WAVE_FORMAT_EXTENSIBLE should be used when:

  • The channel configuration of the wave file is not mono or left/right. This is because other channel configurations are ambiguous unless WAVE_FORMAT_EXTENSIBLE is used.
  • The valid data bits per sample is not a multiple of 8. This is because the meaning of wBitsPerSample in the wave format is ambiguous unless WAVE_FORMAT_EXTENSIBLE is used.

WAVE_FORMAT_EXTENSIBLE should not be used when:

  • Compatibility with ancient, incorrect and/or broken wave reading implementations is required.

FACT chunks are required for any wave format which is not WAVE_FORMAT_PCM

The following dot points are taken from "Multimedia Programming Interface and Data Specifications" (pp. 61) and "New Multimedia Data Types and Data Techniques" (pp. 12) in relation to the 'fact' chunk:

  • 1991) "The 'fact' chunk is required if the waveform data is contained in a 'wavl' LIST chunk and for all compressed audio formats. The chunk is not required for PCM files using the 'data' chunk format."
  • 1994) "The fact chunk is required for all new WAVE formats. The chunk is not required for the standard WAVE_FORMAT_PCM files."
  • 1991 and 1994) "The 'fact' chunk will be expanded to include any other information required by future WAVE formats. Added fields will appear following the dwSampleLength field. Applications can use the chunk size field to determine which fields are present."

From this information, it is always necessary to include a fact chunk when the wave format chunk has wFormatTag set to anything other than WAVE_FORMAT_PCM (this includes WAVE_FORMAT_IEEE_FLOAT). From recollection, Windows Media Player will not play floating point wave files which do not have a fact chunk - however I am not prepared to install Windows to validate if this is still true.

On the ordering of wave chunks

I saved the worst point for the end...

Almost every website which I have seen which attempts to define the wave format states: wave chunks can be in any order but the format chunk must precede the data chunk. One website in particular even stated (although it did not recommend) that the format chunk could come after the data chunk - this is plain wrong ("Multimedia Programming Interface and Data Specifications", pp. 56). Unfortunately in relation to all other chunks, the specification appears to be inconsistent. The original specifications states while defining the RIFF format:

... Following the form-type code is a series of subchunks. Which subchunks are present depends on the form type. The definition of a particular RIFF form typically includes the following:

  • A unique four-character code identifying the form type
  • A list of mandatory chunks
  • A list of optional chunks
  • Possibly, a required order for the chunks

Multimedia Programming Interface and Data Specifications, page 12

The presence of the fourth point hints that maybe the ordering of the chunks does not matter. However, on reading the "Extended Notation for Representing RIFF Form Definitions" (page 17 onwards), many examples are given which seem to suggest that mandatory ordering is implied through the grammar. See the sections on <name:type>, [elements], element... and [element]....

The form of a RIFF/WAVE file is defined on page 56 of "Multimedia Programming Interface and Data Specifications" and again on page 12 of "New Multimedia Data Types and Data Techniques" as:

RIFF(
'WAVE'
<fmt-ck>            // Format
[<fact-ck>]         // Fact chunk
[<cue-ck>]          // Cue points
[<playlist-ck>]     // Playlist
[<assoc-data-list>] // Associated data list
<wave-data>         // Wave data
)

Given the notation examples, this grammar would suggest that the chunks listed should be supplied in that order. Chunks which are not listed in the grammar, but still defined for the form type (for example, smpl or inst chunks), presumably can be located anywhere.

That being said, if we look at the grammar for the associated data list chunk (pp. 63 of the wave specification):

LIST(
'adtl'
<labl-ck> // Label
<note-ck> // Note
<ltxt-ck> // Text with data length
<file-ck> // Media file
)

It would appear that this list must contain exactly one label, one note, one labelled text and one media file chunk. This is clearly incorrect in the specifications given the purpose of the chunk. The definition given in "New Multimedia Data Types and Data Techniques" (pp. 14) is even worse - it is completely mangled on the page. There is a stray close-brace floating towards the end of the definition (which hints that there may have been an attempt to fix the issue) but it is still incorrect. All this tends to suggest that the grammar used to define RIFF/WAVE chunks is unreliable.

In writing an implementation, I would presume that the most correct solution would be to write the chunks in the order supplied by the grammar with any additional chunks being written anywhere. *sigh* If anyone can provide more information from a reliable source to clarify the chunk ordering issue, I would appreciate it.

That's enough. I'm going to make dinner.