Synthesizing Sound

Newton Developer Technical Information: Newton Programmer's Guide: 2.1 OS Addendum /: Chapter 7 - Sound / Using Sound

Synthesizing Sound
The Newton 2.1 OS contains a built-in synthesizer codec that can be used to generate tones based on sine wave addition. You can specify up to 12 individual sine wave tones, which are mixed together to create a sound. Alternately, you can use one of several variants of FM synthesis to create up to four modulated tones (from the base 12), which are mixed together to create a sound.
Because the system can play up to four sound channels simultaneously, you can generate quite complex synthesized sounds by using multiple sound channels playing at the same time. Note that multiple sound channels use more heap space, so depending on your application, you may be limited if you are low on heap space.
Synthesized sounds are typically much smaller than sampled sounds, so if your application plays custom sounds, you may want to synthesize them rather than using a recorded sample.
To use the synthesizer codec, set up the slots in the sound frame like this:
 {sndFrameType: 'codec, // use a codec

codecName: "TDTMFCodec", // select synthesizer codec
bufferSize: 5000, // size of codec buffers
bufferCount: 4, // # of codec buffers
compressionType: kSampleLinear, // for playback
dataType: k16bit, // for playback

samples: mySynthData, // synthesizer binary data
samplingRate: 21600,
 }
There is a shortcut for using the synthesizer codec. You can simply call the *PlaySound function and pass a binary object of the class 'TDTMFCodec, like this:
 PlaySound(mySynthSound); // class of sound object must be 'TDTMFCodec
When passed a binary object of this class, *PlaySound plays it using the synthesizer codec. If the codec is not found, an exception is thrown. This shortcut works for the whole family of *PlaySoundxxx functions.
The data in the samples slot of the sound frame is a binary object of the class 'samples. You can create binary synthesizer data by using the compile-time function *MakeBinaryFromHex.
The data consists of several unsigned short (16-bit) values, structured as follows:
 type 'DTMF' {

unsigned short; /* Parameter block type, set to 1 */
unsigned short; /* Synthesis type (0-4) */
unsigned short; /* Reserved, set to 0 */
unsigned short; /* loop count */
unsigned short = $$CountOf(DTMFTones); /* Number of tone blocks */
   wide array DTMFTones { /* specify 1 to 12 tone blocks */
   unsigned short; /* frequency integer part */
   unsigned short; /* frequency fractional part */
   unsigned short; /* sustain amplitude */
   unsigned short; /* leading silence in ms */
   unsigned short; /* attack in ms */
   unsigned short; /* decay in ms */
   unsigned short; /* sustain in ms */
   unsigned short; /* release in ms */
   unsigned short; /* peak amplitude */
   unsigned short; /* trailing silence in ms */
   };
 };
You can specify up to 12 DTMFTones blocks describing individual tones that are mixed (added) or modulated together to generate a sound. The different sound synthesis types are explained in Table 7-2.
Table 7-2 Sound synthesis types

Synth type Description
0 This multiple-synthesis type mixes from 1 to 12 pure sine wave tones. Each tone can have its own envelope, amplitude, and frequency. The minimum number of tones required to use this type is 1.
1 This is the most basic FM (frequency modulation) type, and a minimum of 2 tones must be specified to use it. Each tone can have its own amplitude and modulation. The first tone in each pair is the carrier, and the second is the modulation frequency. You can have a maximum of 6 pairs of tones. For example, tones 1 and 2 are the first FM pair, tones 3 and 4 are the second FM pair, and so on. Each pair of tones is modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 6 tones.
2 This is a more complex type of FM, where each resulting tone consists of 3 base tones. Tone 1 is the carrier, tone 2 is the first modulator, and tone 3 is the second modulator. For this type of FM, the two modulators are mixed and then used to modulate the carrier. In this way, each group of 3 tones are mixed and modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 4 resulting tones.
3 This type is a variation on type 3. Again 3 base tones are used to generate one tone. Using this modulation type, the modulator, tone 2, is first modulated by tone 3, then the resulting modulated waveform is used to modulate tone 1, the base carrier. In this way, each group of 3 tones are modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 4 resulting tones.
4 This is the most complex FM mode. This mode requires 4 base tones to generate one tone. The first tone is the carrier, the second tone is modulated by the third tone, and the third tone is modulated by the fourth tone. The result is then used to modulate the carrier. In this way, each group of 4 tones are modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 3 resulting tones.

The loop count plays a sound multiple times. This is useful for creating groups of short tone bursts. This value specifies the number of times to repeat the sound after it's played once. So specify 0 to play the sound just once, specify 1 to repeat it once, and so on.
The frequency is specified in Hz, and is a fixed point real number constructed from two fields: the integer part and the fractional part. For example, if you specify 340 for the integer part and 2 for the fractional part, the codec generates a tone of 340.2 Hz.
The time values used to play a tone are collectively referred to as the tone envelope, illustrated in Figure 7-4.
Figure 7-4 Tone envelope

All time values specified in the synthesizer data are in milliseconds (1000 ms. = 1 second). There are four time values that make up a tone envelope: attack, decay, sustain, and release. In Figure 7-4, two additional values, leading silence and trailing silence, are also shown.
Attack is the time in which the tone ramps in amplitude from 0 to the peak amplitude. Decay is the time in which the tone ramps from the peak amplitude to the sustain amplitude. Sustain is the time which the tone remains at the sustain amplitude. Finally, release is the time in which the tone amplitude drops from the sustain amplitude to 0.
All of the times except sustain can be 0. This means, for example, that a tone would leap from 0 amplitude to the sustain amplitude and then from the sustain amplitude to 0 at the end, with no ramp. All ramps are linear.
The leading silence and trailing silence values simply specify a period of silence before or after the sound plays. This is useful when mixing multiple tones and you want a delay in the start of one of the tones, or a delay in the finish when looping is used.
The peak amplitude and sustain amplitude are used at two different points in the tone envelope, as shown in Figure 7-4. The maximum amplitude is 32768; anything above that is clipped. The complexity is that if you are mixing (adding) two or more sine waves, then you must limit the total amplitude to 32768 to avoid clipping. Note that you can specify higher amplitude values, resulting in clipping distortion, which is sometimes a desirable part of the sound.

Synth type	Description
0	This multiple-synthesis type mixes from 1 to 12 pure sine wave tones. Each tone can have its own envelope, amplitude, and frequency. The minimum number of tones required to use this type is 1.
1	This is the most basic FM (frequency modulation) type, and a minimum of 2 tones must be specified to use it. Each tone can have its own amplitude and modulation. The first tone in each pair is the carrier, and the second is the modulation frequency. You can have a maximum of 6 pairs of tones. For example, tones 1 and 2 are the first FM pair, tones 3 and 4 are the second FM pair, and so on. Each pair of tones is modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 6 tones.
2	This is a more complex type of FM, where each resulting tone consists of 3 base tones. Tone 1 is the carrier, tone 2 is the first modulator, and tone 3 is the second modulator. For this type of FM, the two modulators are mixed and then used to modulate the carrier. In this way, each group of 3 tones are mixed and modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 4 resulting tones.
3	This type is a variation on type 3. Again 3 base tones are used to generate one tone. Using this modulation type, the modulator, tone 2, is first modulated by tone 3, then the resulting modulated waveform is used to modulate tone 1, the base carrier. In this way, each group of 3 tones are modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 4 resulting tones.
4	This is the most complex FM mode. This mode requires 4 base tones to generate one tone. The first tone is the carrier, the second tone is modulated by the third tone, and the third tone is modulated by the fourth tone. The result is then used to modulate the carrier. In this way, each group of 4 tones are modulated together to create one resulting tone, and these resulting tones are then mixed together to generate the sound. So using this type, you can mix up to 3 resulting tones.